未验证 提交 efab3a00 编写于 作者: W whs 提交者: GitHub

[cherry-pick/release1.0.0]Move introduction to index page (#154)

* Move introduction to index page
1. Copy some section to intro.md
2. Include intro.md in index.rst
3. Translate install document

* Add darts_nas_turorial to index
上级 1e4656fe
...@@ -75,7 +75,7 @@ pip install paddleslim -i https://pypi.org/simple ...@@ -75,7 +75,7 @@ pip install paddleslim -i https://pypi.org/simple
| 压缩方法 | mAP(baseline: 76.2%) | 模型大小(baseline: 94MB) | | 压缩方法 | mAP(baseline: 76.2%) | 模型大小(baseline: 94MB) |
| :---------------------: | :------------: | :------------:| | :---------------------: | :------------: | :------------:|
| 知识蒸馏(ResNet34-YOLOv3) | [+2.8%]() | - | | 知识蒸馏(ResNet34-YOLOv3) | [+2.8%](#) | - |
| 剪裁 FLOPs -52.88% | [+1.4%]() | [-67.76%]() | | 剪裁 FLOPs -52.88% | [+1.4%]() | [-67.76%]() |
|知识蒸馏(ResNet34-YOLOv3)+剪裁(FLOPs-69.57%)| [+2.6%]()|[-67.00%]()| |知识蒸馏(ResNet34-YOLOv3)+剪裁(FLOPs-69.57%)| [+2.6%]()|[-67.00%]()|
......
...@@ -45,6 +45,7 @@ extensions = [ ...@@ -45,6 +45,7 @@ extensions = [
'sphinx.ext.napoleon', 'sphinx.ext.napoleon',
'recommonmark', 'recommonmark',
'sphinx_markdown_tables', 'sphinx_markdown_tables',
'm2r',
] ]
# Add any paths that contain templates here, relative to this directory. # Add any paths that contain templates here, relative to this directory.
......
...@@ -3,8 +3,8 @@ ...@@ -3,8 +3,8 @@
You can adapt this file completely to your liking, but it should at least You can adapt this file completely to your liking, but it should at least
contain the root `toctree` directive. contain the root `toctree` directive.
Welcome to use PaddleSlim. Index
======== ==============
.. toctree:: .. toctree::
:maxdepth: 1 :maxdepth: 1
...@@ -16,3 +16,5 @@ Welcome to use PaddleSlim. ...@@ -16,3 +16,5 @@ Welcome to use PaddleSlim.
tutorials/index_en tutorials/index_en
api_en/index_en api_en/index_en
model_zoo_en.md model_zoo_en.md
.. mdinclude:: intro_en.md
# Install # Install
安装PaddleSlim前,请确认已正确安装Paddle1.6版本或更新版本。Paddle安装请参考:[Paddle安装教程](https://www.paddlepaddle.org.cn/install/quick) Please ensure you have installed PaddlePaddle1.7+. [How to install PaddlePaddle](https://www.paddlepaddle.org.cn/install/quick)
- 安装develop版本 - Install by pip
```bash ```bash
git clone https://github.com/PaddlePaddle/PaddleSlim.git pip install paddleslim -i https://pypi.org/simple
cd PaddleSlim
python setup.py install
``` ```
- 安装官方发布的最新版本 - Install from source
```bash ```bash
pip install paddleslim -i https://pypi.org/simple git clone https://github.com/PaddlePaddle/PaddleSlim.git
cd PaddleSlim
python setup.py install
``` ```
- 安装历史版本 - History packages
请点击[pypi.org](https://pypi.org/project/paddleslim/#history)查看可安装历史版本。 History packages is available in [pypi.org](https://pypi.org/project/paddleslim/#history).
# Introduction
PaddleSlim is a toolkit for model compression. It contains a collection of compression strategies, such as pruning, fixed point quantization, knowledge distillation, hyperparameter searching and neural architecture search.
# Introduction PaddleSlim provides solutions of compression on computer vision models, such as image classification, object detection and semantic segmentation. Meanwhile, PaddleSlim Keeps exploring advanced compression strategies for language model. Furthermore, benckmark of compression strategies on some open tasks is available for your reference.
PaddleSlim also provides auxiliary and primitive API for developer and researcher to survey, implement and apply the method in latest papers. PaddleSlim will support developer in ability of framework and technology consulting.
## Features
### Pruning
- Uniform pruning of convolution
- Sensitivity-based prunning
- Automated pruning based evolution search strategy
- Support pruning of various deep architectures such as VGG, ResNet, and MobileNet.
- Support self-defined range of pruning, i.e., layers to be pruned.
### Fixed Point Quantization
- **Training aware**
- Dynamic strategy: During inference, we quantize models with hyperparameters dynamically estimated from small batches of samples.
- Static strategy: During inference, we quantize models with the same hyperparameters estimated from training data.
- Support layer-wise and channel-wise quantization.
- **Post training**
### Knowledge Distillation
- **Naive knowledge distillation:** transfers dark knowledge by merging the teacher and student model into the same Program
- **Paddle large-scale scalable knowledge distillation framework Pantheon:** a universal solution for knowledge distillation, more flexible than the naive knowledge distillation, and easier to scale to the large-scale applications.
- Decouple the teacher and student models --- they run in different processes in the same or different nodes, and transfer knowledge via TCP/IP ports or local files;
- Friendly to assemble multiple teacher models and each of them can work in either online or offline mode independently;
- Merge knowledge from different teachers and make batch data for the student model automatically;
- Support the large-scale knowledge prediction of teacher models on multiple devices.
### Neural Architecture Search
- Neural architecture search based on evolution strategy.
- Support distributed search.
- One-Shot neural architecture search.
- Support FLOPs and latency constrained search.
- Support the latency estimation on different hardware and platforms.
## Performance
### Image Classification
Dataset: ImageNet2012; Model: MobileNetV1;
|Method |Accuracy(baseline: 70.91%) |Model Size(baseline: 17.0M)|
|:---:|:---:|:---:|
| Knowledge Distillation(ResNet50)| **+1.06%** | |
| Knowledge Distillation(ResNet50) + int8 quantization |**+1.10%**| **-71.76%**|
| Pruning(FLOPs-50%) + int8 quantization|**-1.71%**|**-86.47%**|
### Object Detection
#### Dataset: Pascal VOC; Model: MobileNet-V1-YOLOv3
As a submodule of PaddlePaddle framework, PaddleSlim is an open-source library for deep model compression and architecture search. PaddleSlim supports current popular deep compression techniques such as pruning, quantization, and knowledge distillation. Further, it also automates the search of hyperparameters and the design of lightweight deep architectures. In the future, we will develop more practically useful compression techniques for industrial-level applications and transfer these techniques to models in NLP. | Method | mAP(baseline: 76.2%) | Model Size(baseline: 94MB) |
| :---------------------: | :------------: | :------------:|
| Knowledge Distillation(ResNet34-YOLOv3) | **+2.8%** | |
| Pruning(FLOPs -52.88%) | **+1.4%** | **-67.76%** |
|Knowledge DistillationResNet34-YOLOv3)+Pruning(FLOPs-69.57%)| **+2.6%**|**-67.00%**|
## Methods #### Dataset: COCO; Model: MobileNet-V1-YOLOv3
- Pruning | Method | mAP(baseline: 29.3%) | Model Size|
- Uniform pruning | :---------------------: | :------------: | :------:|
- Sensitivity-based pruning | Knowledge Distillation(ResNet34-YOLOv3) | **+2.1%** |-|
- Automated model pruning | Knowledge Distillation(ResNet34-YOLOv3)+Pruning(FLOPs-67.56%) | **-0.3%** | **-66.90%**|
- Quantization ### NAS
- Training-aware quantization: Quantize models with hyperparameters dynamically estimated from small batches of samples.
- Training-aware quantization: Quantize models with the same hyperparameters estimated from training data.
- Support global quantization of weights and Channel-Wise quantization
- Knowledge Distillation Dataset: ImageNet2012; Model: MobileNetV2
- Single-process knowledge distillation
- Multi-process distributed knowledge distillation
- Network Architecture Search(NAS) |Device | Infer time cost | Top1 accuracy(baseline:71.90%) |
- Simulated Annealing (SA)-based lightweight network architecture search method.(Light-NAS) |:---------------:|:---------:|:--------------------:|
- One-Shot network structure automatic search. (One-Shot-NAS) | RK3288 | **-23%** | +0.07% |
- PaddleSlim supports FLOPs and latency constrained search. | Android cellphone | **-20%** | +0.16% |
- PaddleSlim supports the latency estimation on different hardware and platforms. | iPhone 6s | **-17%** | +0.32% |
...@@ -2,3 +2,4 @@ sphinx ...@@ -2,3 +2,4 @@ sphinx
recommonmark recommonmark
sphinx_markdown_tables sphinx_markdown_tables
sphinx_rtd_theme sphinx_rtd_theme
m2r
...@@ -45,6 +45,7 @@ extensions = [ ...@@ -45,6 +45,7 @@ extensions = [
'sphinx.ext.napoleon', 'sphinx.ext.napoleon',
'recommonmark', 'recommonmark',
'sphinx_markdown_tables', 'sphinx_markdown_tables',
'm2r',
] ]
# Add any paths that contain templates here, relative to this directory. # Add any paths that contain templates here, relative to this directory.
......
...@@ -3,7 +3,7 @@ ...@@ -3,7 +3,7 @@
You can adapt this file completely to your liking, but it should at least You can adapt this file completely to your liking, but it should at least
contain the root `toctree` directive. contain the root `toctree` directive.
欢迎使用PaddleSlim 首页
================== ==================
.. toctree:: .. toctree::
...@@ -17,3 +17,5 @@ ...@@ -17,3 +17,5 @@
api_cn/index api_cn/index
model_zoo.md model_zoo.md
algo/algo.md algo/algo.md
.. mdinclude:: ./intro.md
# 介绍
PaddleSlim是一个模型压缩工具库,包含模型剪裁、定点量化、知识蒸馏、超参搜索和模型结构搜索等一系列模型压缩策略。
# PaddleSlim简介 对于业务用户,PaddleSlim提供完整的模型压缩解决方案,可用于图像分类、检测、分割等各种类型的视觉场景。
同时也在持续探索NLP领域模型的压缩方案。另外,PaddleSlim提供且在不断完善各种压缩策略在经典开源任务的benchmark,
以便业务用户参考。
对于模型压缩算法研究者或开发者,PaddleSlim提供各种压缩策略的底层辅助接口,方便用户复现、调研和使用最新论文方法。
PaddleSlim会从底层能力、技术咨询合作和业务场景等角度支持开发者进行模型压缩策略相关的创新工作。
PaddleSlim是PaddlePaddle框架的一个子模块,主要用于压缩图像领域模型。在PaddleSlim中,不仅实现了目前主流的网络剪枝、量化、蒸馏三种压缩策略,还实现了超参数搜索和小模型网络结构搜索功能。在后续版本中,会添加更多的压缩策略,以及完善对NLP领域模型的支持。
## 功能 ## 功能
- 模型剪裁 - 模型剪裁
- 支持通道均匀模型剪裁(uniform pruning) - 卷积通道均匀剪裁
- 基于敏感度的模型剪裁 - 基于敏感度的卷积通道剪裁
- 基于进化算法的自动模型剪裁三种方式 - 基于进化算法的自动剪裁
- 量化训练 - 定点量化
- 在线量化训练(training aware) - 在线量化训练(training aware)
- 离线量化(post training) - 离线量化(post training)
- 支持对权重全局量化和Channel-Wise量化
- 知识蒸馏 - 知识蒸馏
- 支持单进程知识蒸馏 - 支持单进程知识蒸馏
- 支持多进程分布式知识蒸馏 - 支持多进程分布式知识蒸馏
- 神经网络结构自动搜索(NAS) - 神经网络结构自动搜索(NAS)
- 支持One-Shot网络结构自动搜索(Ont-Shot-NAS) - 支持基于进化算法的轻量神经网络结构自动搜索
- 支持基于进化算法的轻量神经网络结构自动搜索(Light-NAS) - 支持One-Shot网络结构自动搜索
- 支持 FLOPS / 硬件延时约束 - 支持 FLOPS / 硬件延时约束
- 支持多平台模型延时评估 - 支持多平台模型延时评估
- 支持用户自定义搜索算法和搜索空间
## 部分压缩策略效果
### 分类模型
数据: ImageNet2012; 模型: MobileNetV1;
|压缩策略 |精度收益(baseline: 70.91%) |模型大小(baseline: 17.0M)|
|:---:|:---:|:---:|
| 知识蒸馏(ResNet50)| **+1.06%** | |
| 知识蒸馏(ResNet50) + int8量化训练 |**+1.10%**| **-71.76%**|
| 剪裁(FLOPs-50%) + int8量化训练|**-1.71%**|**-86.47%**|
### 图像检测模型
#### 数据:Pascal VOC;模型:MobileNet-V1-YOLOv3
| 压缩方法 | mAP(baseline: 76.2%) | 模型大小(baseline: 94MB) |
| :---------------------: | :------------: | :------------:|
| 知识蒸馏(ResNet34-YOLOv3) | **+2.8%** | |
| 剪裁 FLOPs -52.88% | **+1.4%** | **-67.76%** |
|知识蒸馏(ResNet34-YOLOv3)+剪裁(FLOPs-69.57%)| **+2.6%**|**-67.00%**|
#### 数据:COCO;模型:MobileNet-V1-YOLOv3
| 压缩方法 | mAP(baseline: 29.3%) | 模型大小|
| :---------------------: | :------------: | :------:|
| 知识蒸馏(ResNet34-YOLOv3) | **+2.1%** | |
| 知识蒸馏(ResNet34-YOLOv3)+剪裁(FLOPs-67.56%) | **-0.3%** | **-66.90%**|
### 搜索
数据:ImageNet2012; 模型:MobileNetV2
|硬件环境 | 推理耗时 | Top1准确率(baseline:71.90%) |
|:---------------:|:---------:|:--------------------:|
| RK3288 | **-23%** | +0.07% |
| Android cellphone | **-20%** | +0.16% |
| iPhone 6s | **-17%** | +0.32% |
# SANAS进阶版实验教程 # SANAS进阶版实验教程-压缩DARTS产出模型
## 收益情况 ## 收益情况
利用DARTS搜索出来的最终模型结构(以下简称为DARTS_model)构造相应的搜索空间,根据PaddleSlim提供的SANAS搜索方法进行搜索实验,最终得到的模型结构(以下简称为DARTS_SA)相比DARTS_model的精度提升<font color=green>0.141%</font>,模型大小下降<font color=green>11.2%</font> 利用DARTS搜索出来的最终模型结构(以下简称为DARTS_model)构造相应的搜索空间,根据PaddleSlim提供的SANAS搜索方法进行搜索实验,最终得到的模型结构(以下简称为DARTS_SA)相比DARTS_model的精度提升<font color=green>0.141%</font>,模型大小下降<font color=green>11.2%</font>
......
...@@ -3,9 +3,8 @@ ...@@ -3,9 +3,8 @@
======== ========
.. toctree:: .. toctree::
:maxdepth: 2 :maxdepth: 1
:caption: Contents:
image_classification_sensitivity_analysis_tutorial.md image_classification_sensitivity_analysis_tutorial.md
image_classification_nas_quick_start.ipynb darts_nas_turorial.md
Markdown is supported
0% .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑!
想要评论请 注册