update pages.

_sources/algo/algo.md.txt

+# 算法原理
+
+## 目录
+
+- [量化原理介绍](#1-quantization-aware-training量化介绍)
+- [剪裁原理介绍](#2-卷积核剪裁原理)
+- [蒸馏原理介绍](#3-蒸馏)
+- [轻量级模型结构搜索原理介绍](#4-轻量级模型结构搜索)
+
+## 1. Quantization Aware Training量化介绍
+
+### 1.1 背景
+
+近年来，定点量化使用更少的比特数（如8-bit、3-bit、2-bit等）表示神经网络的权重和激活已被验证是有效的。定点量化的优点包括低内存带宽、低功耗、低计算资源占用以及低模型存储需求等。
+
+<p align="center">
+<img src="https://raw.githubusercontent.com/PaddlePaddle/PaddleSlim/develop/docs/docs/images/algo/quan_table_0.png" height=258 width=600 hspace='10'/> <br />
+<strong>表1: 不同类型操作的开销对比</strong>
+</p>
+
+由表1可知，低精度定点数操作的硬件面积大小及能耗比高精度浮点数要少几个数量级。 使用定点量化可带来4倍的模型压缩、4倍的内存带宽提升，以及更高效的cache利用(很多硬件设备，内存访问是主要能耗)。除此之外，计算速度也会更快(通常具有2x-3x的性能提升)。由表2可知，在很多场景下，定点量化操作对精度并不会造成损失。另外，定点量化对神经网络于嵌入式设备上的推断来说是极其重要的。
+
+<p align="center">
+<img src="https://raw.githubusercontent.com/PaddlePaddle/PaddleSlim/develop/docs/docs/images/algo/quan_table_1.png" height=155 width=500 hspace='10'/> <br />
+<strong>表2：模型量化前后精度对比</strong>
+</p>
+
+目前，学术界主要将量化分为两大类：`Post Training Quantization`和`Quantization Aware Training`。`Post Training Quantization`是指使用KL散度、滑动平均等方法确定量化参数且不需要重新训练的定点量化方法。`Quantization Aware Training`是在训练过程中对量化进行建模以确定量化参数，它与`Post Training Quantization`模式相比可以提供更高的预测精度。
+
+### 1.2 量化原理
+
+#### 1.2.1 量化方式
+目前，存在着许多方法可以将浮点数量化成定点数。例如：
+$$ r = min(max(x, a), b)$$ $$ s = \frac{b - a}{n - 1} $$ $$ q = \left \lfloor \frac{r - a}{s} \right \rceil $$
+式中，$x$是待量化的浮点值，$[a, b]$是量化范围，$a$是待量化浮点数中的最小值， $b$ 是待量化浮点数中的最大值。$\left \lfloor \right \rceil$ 表示将结果四舍五入到最近的整数。如果量化级别为$k$，则$n$为$2^k$。例如，若$k$为8，则$n$为256。$q$是量化得到的整数。
+PaddleSlim框架中选择的量化方法为最大绝对值量化(`max-abs`)，具体描述如下：
+$$ M = max(abs(x)) $$ $$ q = \left \lfloor \frac{x}{M} * (n - 1) \right \rceil $$
+式中，$x$是待被量化的浮点值，$M$是待量化浮点数中的绝对值最大值。$\left \lfloor \right \rceil$表示将结果四舍五入到最近的整数。对于8bit量化，PaddleSlim采用`int8_t`，即$n=2^7=128$。$q$是量化得到的整数。
+无论是`min-max量化`还是`max-abs量化`，他们都可以表示为如下形式：
+$q = scale * r + b$
+其中`min-max`和`max-abs`被称为量化参数或者量化比例或者量化范围。
+
+#### 1.2.2 量化训练
+##### 1.2.2.1 前向传播
+前向传播过程采用模拟量化的方式，具体描述如下：
+
+<p align="center">
+<img src="https://raw.githubusercontent.com/PaddlePaddle/PaddleSlim/develop/docs/docs/images/algo/quan_forward.png" height=433 width=335 hspace='10'/> <br />
+<strong>图1：基于模拟量化训练的前向过程</strong>
+</p>
+
+由图1可知，基于模拟量化训练的前向过程可被描述为以下四个部分：
+1) 输入和权重均被量化成8-bit整数。
+2) 在8-bit整数上执行矩阵乘法或者卷积操作。
+3) 反量化矩阵乘法或者卷积操作的输出结果为32-bit浮点型数据。
+4) 在32-bit浮点型数据上执行偏置加法操作。此处，偏置并未被量化。
+对于通用矩阵乘法(`GEMM`)，输入$X$和权重$W$的量化操作可被表述为如下过程：
+$$ X_q = \left \lfloor \frac{X}{X_m} * (n - 1) \right \rceil $$ $$ W_q = \left \lfloor \frac{W}{W_m} * (n - 1) \right \rceil $$
+执行通用矩阵乘法：
+$$ Y_q = X_q * W_q $$
+对量化乘积结果$Yq$进行反量化:
+$$
+\begin{align}
+Y_{dq} = \frac{Y_q}{(n - 1) * (n - 1)} * X_m * W_m \
+=\frac{X_q * W_q}{(n - 1) * (n - 1)} * X_m * W_m \
+=(\frac{X_q}{n - 1} * X_m) * (\frac{W_q}{n - 1} * W_m) \
+\end{align}
+$$
+上述公式表明反量化操作可以被移动到`GEMM`之前，即先对$Xq$和$Wq$执行反量化操作再做`GEMM`操作。因此，前向传播的工作流亦可表示为如下方式：
+
+<p align="center">
+<img src="https://raw.githubusercontent.com/PaddlePaddle/PaddleSlim/develop/docs/docs/images/algo/quan_fwd_1.png" height=435 width=341 hspace='10'/> <br />
+<strong>图2：基于模拟量化训练前向过程的等价工作流</strong>
+</p>
+
+训练过程中，PaddleSlim使用图2中所示的等价工作流。在设计中，量化Pass在IrGraph中插入量化op和反量化op。因为在连续的量化、反量化操作之后输入仍然为32-bit浮点型数据。因此，PaddleSlim量化训练框架所采用的量化方式被称为模拟量化。
+
+##### 1.2.2.2 反向传播
+由图3可知，权重更新所需的梯度值可以由量化后的权重和量化后的激活求得。反向传播过程中的所有输入和输出均为32-bit浮点型数据。注意，梯度更新操作需要在原始权重上进行，即计算出的梯度将被加到原始权重上而非量化后或反量化后的权重上。
+
+<p align="center">
+<img src="https://raw.githubusercontent.com/PaddlePaddle/PaddleSlim/develop/docs/docs/images/algo/quan_bwd.png" height=300 width=650 hspace='10'/> <br />
+<strong>图3：基于模拟量化训练的反向传播和权重更新过程</strong>
+</p>
+
+因此，量化Pass也会改变相应反向算子的某些输入。
+
+##### 1.2.2.3 确定量化比例系数
+存在着两种策略可以计算求取量化比例系数，即动态策略和静态策略。动态策略会在每次迭代过程中计算量化比例系数的值。静态策略则对不同的输入采用相同的量化比例系数。
+对于权重而言，在训练过程中采用动态策略。换句话说，在每次迭代过程中量化比例系数均会被重新计算得到直至训练过程结束。
+对于激活而言，可以选择动态策略也可以选择静态策略。若选择使用静态策略，则量化比例系数会在训练过程中被评估求得，且在推断过程中被使用(不同的输入均保持不变)。静态策略中的量化比例系数可于训练过程中通过如下三种方式进行评估：
+
+1. 在一个窗口中计算激活最大绝对值的平均值。
+
+2. 在一个窗口中计算激活最大绝对值的最大值。
+
+3. 在一个窗口中计算激活最大绝对值的滑动平均值，计算公式如下：
+
+$$ Vt = (1 - k) * V + k * V_{t-1} $$
+
+式中，$V$ 是当前batch的最大绝对值， $Vt$是滑动平均值。$k$是一个因子，例如其值可取为0.9。
+
+#### 1.2.4 训练后量化
+
+训练后量化是基于采样数据，采用KL散度等方法计算量化比例因子的方法。相比量化训练，训练后量化不需要重新训练，可以快速得到量化模型。
+
+训练后量化的目标是求取量化比例因子，主要有两种方法：非饱和量化方法 ( No Saturation) 和饱和量化方法 (Saturation)。非饱和量化方法计算FP32类型Tensor中绝对值的最大值`abs_max`，将其映射为127，则量化比例因子等于`abs_max/127`。饱和量化方法使用KL散度计算一个合适的阈值`T` (`0<T<mab_max`)，将其映射为127，则量化比例因子等于`T/127`。一般而言，对于待量化op的权重Tensor，采用非饱和量化方法，对于待量化op的激活Tensor（包括输入和输出），采用饱和量化方法 。
+
+训练后量化的实现步骤如下：
+* 加载预训练的FP32模型，配置`DataLoader`；
+* 读取样本数据，执行模型的前向推理，保存待量化op激活Tensor的数值；
+* 基于激活Tensor的采样数据，使用饱和量化方法计算它的量化比例因子；
+* 模型权重Tensor数据一直保持不变，使用非饱和方法计算它每个通道的绝对值最大值，作为每个通道的量化比例因子；
+* 将FP32模型转成INT8模型，进行保存。
+
+
+## 2. 卷积核剪裁原理
+
+该策略参考paper: [Pruning Filters for Efficient ConvNets](https://arxiv.org/pdf/1608.08710.pdf)
+
+该策略通过减少卷积层中卷积核的数量，来减小模型大小和降低模型计算复杂度。
+
+### 2.1  剪裁卷积核
+
+**剪裁注意事项1**
+剪裁一个conv layer的filter，需要修改后续conv layer的filter. 如**图4**所示，剪掉Xi的一个filter，会导致$X_{i+1}$少一个channel, $X_{i+1}$对应的filter在input_channel纬度上也要减1.
+
+
+<p align="center">
+<img src="https://raw.githubusercontent.com/PaddlePaddle/PaddleSlim/develop/docs/docs/images/algo/pruning_0.png" height=200 width=600 hspace='10'/> <br />
+<strong>图4</strong>
+</p>
+
+
+**剪裁注意事项2**
+
+如**图5**所示，剪裁完$X_i$之后，根据注意事项1我们从$X_{i+1}$的filter中删除了一行（图中蓝色行），在计算$X_{i+1}$的filters的l1_norm(图中绿色一列)的时候，有两种选择：
+算上被删除的一行：independent pruning
+减去被删除的一行：greedy pruning
+
+<p align="center">
+<img src="https://raw.githubusercontent.com/PaddlePaddle/PaddleSlim/develop/docs/docs/images/algo/pruning_1.png" height=200 width=450 hspace='10'/> <br />
+<strong>图5</strong>
+</p>
+
+**剪裁注意事项3**
+在对ResNet等复杂网络剪裁的时候，还要考虑到后当前卷积层的修改对上一层卷积层的影响。
+如**图6**所示，在对residual block剪裁时，$X_{i+1}$层如何剪裁取决于project shortcut的剪裁结果，因为我们要保证project shortcut的output和$X_{i+1}$的output能被正确的concat.
+
+
+<p align="center">
+<img src="https://raw.githubusercontent.com/PaddlePaddle/PaddleSlim/develop/docs/docs/images/algo/pruning_2.png" height=240 width=600 hspace='10'/> <br />
+<strong>图6</strong>
+</p>
+
+### 2.2 Uniform剪裁卷积网络
+
+每层剪裁一样比例的卷积核。
+在剪裁一个卷积核之前，按l1_norm对filter从高到低排序，越靠后的filter越不重要，优先剪掉靠后的filter.
+
+
+### 2.3 基于敏感度剪裁卷积网络
+
+根据每个卷积层敏感度的不同，剪掉不同比例的卷积核。
+
+#### 两个假设
+
+- 在一个conv layer的parameter内部，按l1_norm对filter从高到低排序，越靠后的filter越不重要。
+- 两个layer剪裁相同的比例的filters，我们称对模型精度影响更大的layer的敏感度相对高。
+
+#### 剪裁filter的指导原则
+
+- layer的剪裁比例与其敏感度成反比
+- 优先剪裁layer内l1_norm相对低的filter
+
+#### 敏感度的理解
+
+<p align="center">
+<img src="https://raw.githubusercontent.com/PaddlePaddle/PaddleSlim/develop/docs/docs/images/algo/pruning_3.png" height=200 width=400 hspace='10'/> <br />
+<strong>图7</strong>
+</p>
+
+如**图7**所示，横坐标是将filter剪裁掉的比例，竖坐标是精度的损失，每条彩色虚线表示的是网络中的一个卷积层。
+以不同的剪裁比例**单独**剪裁一个卷积层，并观察其在验证数据集上的精度损失，并绘出**图7**中的虚线。虚线上升较慢的，对应的卷积层相对不敏感，我们优先剪不敏感的卷积层的filter.
+
+#### 选择最优的剪裁率组合
+
+我们将**图7**中的折线拟合为**图8**中的曲线，每在竖坐标轴上选取一个精度损失值，就在横坐标轴上对应着一组剪裁率，如**图8**中黑色实线所示。
+用户给定一个模型整体的剪裁率，我们通过移动**图5**中的黑色实线来找到一组满足条件的且合法的剪裁率。
+
+<p align="center">
+<img src="https://raw.githubusercontent.com/PaddlePaddle/PaddleSlim/develop/docs/docs/images/algo/pruning_4.png" height=200 width=400 hspace='10'/> <br />
+<strong>图8</strong>
+</p>
+
+#### 迭代剪裁
+考虑到多个卷积层间的相关性，一个卷积层的修改可能会影响其它卷积层的敏感度，我们采取了多次剪裁的策略，步骤如下：
+
+- step1: 统计各卷积层的敏感度信息
+- step2: 根据当前统计的敏感度信息，对每个卷积层剪掉少量filter, 并统计FLOPS，如果FLOPS已满足要求，进入step4，否则进行step3。
+- step3: 对网络进行简单的fine-tune，进入step1
+- step4: fine-tune训练至收敛
+
+## 3. 蒸馏
+
+   一般情况下，模型参数量越多，结构越复杂，其性能越好，但参数也越允余，运算量和资源消耗也越大；模型蒸馏是将复杂网络中的有用信息将复杂网络中的有用信息提取出来提取出来，迁移到一个更小的网络中去，在我们的工具包中，支持两种蒸馏的方法。
+    第一种是传统的蒸馏方法（参考论文：[Distilling the Knowledge in a Neural Network](https://arxiv.org/pdf/1503.02531.pdf)）
+   使用复杂的网络作为teacher模型去监督训练一个参数量和运算量更少的student模型。teacher模型可以是一个或者多个提前训练好的高性能模型。student模型的训练有两个目标：一个是原始的目标函数，为student模型输出的类别概率和label的交叉熵，记为hard-target；另一个是student模型输出的类别概率和teacher模型输出的类别概率的交叉熵，记为soft target，这两个loss加权后得到最终的训练loss，共同监督studuent模型的训练。
+   第二种是基于FSP的蒸馏方法（参考论文：[A Gift from Knowledge Distillation:
+Fast Optimization, Network Minimization and Transfer Learning](http://openaccess.thecvf.com/content_cvpr_2017/papers/Yim_A_Gift_From_CVPR_2017_paper.pdf)）
+   相比传统的蒸馏方法直接用小模型去拟合大模型的输出，该方法用小模型去拟合大模型不同层特征之间的转换关系，其用一个FSP矩阵（特征的内积）来表示不同层特征之间的关系，大模型和小模型不同层之间分别获得多个FSP矩阵，然后使用L2 loss让小模型的对应层FSP矩阵和大模型对应层的FSP矩阵尽量一致，具体如下图所示。这种方法的优势，通俗的解释是，比如将蒸馏类比成teacher（大模型）教student（小模型）解决一个问题，传统的蒸馏是直接告诉小模型问题的答案，让小模型学习，而学习FSP矩阵是让小模型学习解决问题的中间过程和方法，因此其学到的信息更多。
+
+<p align="center">
+<img src="https://raw.githubusercontent.com/PaddlePaddle/PaddleSlim/develop/docs/docs/images/algo/distillation_0.png" height=300 width=600 hspace='10'/> <br />
+<strong>图9</strong>
+</p>
+
+   由于小模型和大模型之间通过L2 loss进行监督，必须保证两个FSP矩阵的维度必须相同，而FSP矩阵的维度为M*N，其中M、N分别为输入和输出特征的channel数，因此大模型和小模型的FSP矩阵需要一一对应。
+
+## 4. 轻量级模型结构搜索
+
+深度学习模型在很多任务上都取得了不错的效果，网络结构的好坏对最终模型的效果有非常重要的影响。手工设计网络需要非常丰富的经验和众多尝试，并且众多的超参数和网络结构参数会产生爆炸性的组合，常规的random search几乎不可行，因此最近几年自动模型搜索技术（Neural Architecture Search）成为研究热点。区别于传统NAS，我们专注在搜索精度高并且速度快的模型结构，我们将该功能统称为Light-NAS.
+
+### 4.1 搜索策略
+
+搜索策略定义了使用怎样的算法可以快速、准确找到最优的网络结构参数配置。常见的搜索方法包括：强化学习、贝叶斯优化、进化算法、基于梯度的算法。我们当前的实现以模拟退火算法为主。
+
+#### 4.1.1 模拟退火
+
+模拟退火算法来源于固体退火原理，将固体加温至充分高，再让其徐徐冷却，加温时，固体内部粒子随温升变为无序状，内能增大，而徐徐冷却时粒子渐趋有序，在每个温度都达到平衡态，最后在常温时达到基态，内能减为最小。
+
+鉴于物理中固体物质的退火过程与一般组合优化问题之间的相似性，我们将其用于网络结构的搜索。
+
+使用模拟退火算法搜索模型的过程如下:
+
+$$
+T_k = T_0*\theta^k
+$$
+
+\begin{equation}
+P(r_k) =
+\begin{cases}
+e^{\frac{(r_k-r)}{T_k}} & r_k < r\\
+1 & r_k>=r
+\end{cases}
+\end{equation}
+
+在第k次迭代，搜到的网络为$N_k$, 对$N_k$训练若干epoch后，在测试集上得到reward为$r_k$, 以概率$P(r_k)$接受$r_k$，即执行$r=r_k$。$r$在搜索过程起始时被初始化为0. $T_0$为初始化温度，$\theta$为温度衰减系数，$T_k$为第k次迭代的温度。
+
+
+在我们的NAS任务中，区别于RL每次重新生成一个完整的网络，我们将网络结构映射成一段编码，第一次随机初始化，然后每次随机修改编码中的一部分（对应于网络结构的一部分）生成一个新的编码，然后将这个编码再映射回网络结构，通过在训练集上训练一定的epochs后的精度以及网络延时融合获得reward，来指导退火算法的收敛。
+
+
+### 4.2 搜索空间
+
+搜索空间定义了优化问题的变量，变量规模决定了搜索算法的难度和搜索时间。因此为了加快搜索速度，定义一个合理的搜索空间至关重要。在Light-NAS中，为了加速搜索速度，我们将一个网络划分为多个block，先手动按链状层级结构堆叠c，再 使用搜索算法自动搜索每个block内部的结构。
+
+因为要搜索出在移动端运行速度快的模型，我们参考了MobileNetV2中的Linear Bottlenecks和Inverted residuals结构，搜索每一个Inverted residuals中的具体参数，包括kernelsize、channel扩张倍数、重复次数、channels number。如图10所示：
+
+<p align="center">
+<img src="https://raw.githubusercontent.com/PaddlePaddle/PaddleSlim/develop/docs/docs/images/algo/light-nas-block.png" height=300 width=600 hspace='10'/> <br />
+<strong>图10</strong>
+</p>
+
+
+### 4.3 模型延时评估
+
+搜索过程支持 FLOPS 约束和模型延时约束。而基于 Android/iOS 移动端、开发板等硬件平台，迭代搜索过程中不断测试模型的延时不仅消耗时间而且非常不方便，因此我们开发了模型延时评估器来评估搜索得到模型的延时。通过延时评估器评估得到的延时与模型实际测试的延时波动偏差小于 10%。
+
+延时评估器分为配置硬件延时评估器和评估模型延时两个阶段，配置硬件延时评估器只需要执行一次，而评估模型延时则在搜索过程中不断评估搜索得到的模型延时。
+
+- 配置硬件延时评估器
+
+    1. 获取搜索空间中所有不重复的 op 及其参数
+    2. 获取每组 op 及其参数的延时
+
+- 评估模型延时
+
+    1. 获取给定模型的所有 op 及其参数
+    2. 根据给定模型的所有 op 及参数，利用延时评估器去估计模型的延时
+
+
+## 5. 参考文献
+
+1. [High-Performance Hardware for Machine Learning](https://media.nips.cc/Conferences/2015/tutorialslides/Dally-NIPS-Tutorial-2015.pdf)
+
+2. [Quantizing deep convolutional networks for efficient inference: A whitepaper](https://arxiv.org/pdf/1806.08342.pdf)
+
+3. [Pruning Filters for Efficient ConvNets](https://arxiv.org/pdf/1608.08710.pdf)
+
+4. [Distilling the Knowledge in a Neural Network](https://arxiv.org/pdf/1503.02531.pdf)
+
+5. [A Gift from Knowledge Distillation: Fast Optimization, Network Minimization and Transfer Learning](http://openaccess.thecvf.com/content_cvpr_2017/papers/Yim_A_Gift_From_CVPR_2017_paper.pdf)

_sources/api_cn/analysis_api.rst.txt

+模型分析
+=======
+
+FLOPs
+-----
+
+.. py:function:: paddleslim.analysis.flops(program, detail=False)
+
+`源代码 <https://github.com/PaddlePaddle/PaddleSlim/blob/develop/paddleslim/analysis/flops.py>`_
+
+获得指定网络的浮点运算次数(FLOPs)。
+
+**参数：**
+
+- **program(paddle.fluid.Program)** - 待分析的目标网络。更多关于Program的介绍请参考：`Program概念介绍 <https://www.paddlepaddle.org.cn/documentation/docs/zh/api_cn/fluid_cn/Program_cn.html#program>`_。
+
+- **detail(bool)** - 是否返回每个卷积层的FLOPs。默认为False。
+
+- **only_conv(bool)** - 如果设置为True，则仅计算卷积层和全连接层的FLOPs，即浮点数的乘加（multiplication-adds）操作次数。如果设置为False，则也会计算卷积和全连接层之外的操作的FLOPs。
+
+**返回值：**
+
+- **flops(float)** - 整个网络的FLOPs。
+
+- **params2flops(dict)** - 每层卷积对应的FLOPs，其中key为卷积层参数名称，value为FLOPs值。
+
+**示例：**
+
+.. code-block:: python
+
+    import paddle.fluid as fluid
+    from paddle.fluid.param_attr import ParamAttr
+    from paddleslim.analysis import flops
+    
+    def conv_bn_layer(input,
+                      num_filters,
+                      filter_size,
+                      name,
+                      stride=1,
+                      groups=1,
+                      act=None):
+        conv = fluid.layers.conv2d(
+            input=input,
+            num_filters=num_filters,
+            filter_size=filter_size,
+            stride=stride,
+            padding=(filter_size - 1) // 2,
+            groups=groups,
+            act=None,
+            param_attr=ParamAttr(name=name + "_weights"),
+            bias_attr=False,
+            name=name + "_out")
+        bn_name = name + "_bn"
+        return fluid.layers.batch_norm(
+            input=conv,
+            act=act,
+            name=bn_name + '_output',
+            param_attr=ParamAttr(name=bn_name + '_scale'),
+            bias_attr=ParamAttr(bn_name + '_offset'),
+            moving_mean_name=bn_name + '_mean',
+            moving_variance_name=bn_name + '_variance', )
+    
+    main_program = fluid.Program()
+    startup_program = fluid.Program()
+    #   X       X              O       X              O
+    # conv1-->conv2-->sum1-->conv3-->conv4-->sum2-->conv5-->conv6
+    #     |            ^ |                    ^
+    #     |____________| |____________________|
+    #
+    # X: prune output channels
+    # O: prune input channels
+    with fluid.program_guard(main_program, startup_program):
+        input = fluid.data(name="image", shape=[None, 3, 16, 16])
+        conv1 = conv_bn_layer(input, 8, 3, "conv1")
+        conv2 = conv_bn_layer(conv1, 8, 3, "conv2")
+        sum1 = conv1 + conv2
+        conv3 = conv_bn_layer(sum1, 8, 3, "conv3")
+        conv4 = conv_bn_layer(conv3, 8, 3, "conv4")
+        sum2 = conv4 + sum1
+        conv5 = conv_bn_layer(sum2, 8, 3, "conv5")
+        conv6 = conv_bn_layer(conv5, 8, 3, "conv6")
+    
+    print("FLOPs: {}".format(flops(main_program)))
+
+model_size
+----------
+
+.. py:function:: paddleslim.analysis.model_size(program)
+
+`源代码 <https://github.com/PaddlePaddle/PaddleSlim/blob/develop/paddleslim/analysis/model_size.py>`_
+
+获得指定网络的参数数量。
+
+**参数：**
+
+- **program(paddle.fluid.Program)** - 待分析的目标网络。更多关于Program的介绍请参考：`Program概念介绍 <https://www.paddlepaddle.org.cn/documentation/docs/zh/api_cn/fluid_cn/Program_cn.html#program>`_。
+
+**返回值：**
+
+- **model_size(int)** - 整个网络的参数数量。
+
+**示例：**
+
+.. code-block:: python
+
+    import paddle.fluid as fluid
+    from paddle.fluid.param_attr import ParamAttr
+    from paddleslim.analysis import model_size
+    
+    def conv_layer(input,
+                      num_filters,
+                      filter_size,
+                      name,
+                      stride=1,
+                      groups=1,
+                      act=None):
+        conv = fluid.layers.conv2d(
+            input=input,
+            num_filters=num_filters,
+            filter_size=filter_size,
+            stride=stride,
+            padding=(filter_size - 1) // 2,
+            groups=groups,
+            act=None,
+            param_attr=ParamAttr(name=name + "_weights"),
+            bias_attr=False,
+            name=name + "_out")
+        return conv
+    
+    main_program = fluid.Program()
+    startup_program = fluid.Program()
+    #   X       X              O       X              O
+    # conv1-->conv2-->sum1-->conv3-->conv4-->sum2-->conv5-->conv6
+    #     |            ^ |                    ^
+    #     |____________| |____________________|
+    #
+    # X: prune output channels
+    # O: prune input channels
+    with fluid.program_guard(main_program, startup_program):
+        input = fluid.data(name="image", shape=[None, 3, 16, 16])
+        conv1 = conv_layer(input, 8, 3, "conv1")
+        conv2 = conv_layer(conv1, 8, 3, "conv2")
+        sum1 = conv1 + conv2
+        conv3 = conv_layer(sum1, 8, 3, "conv3")
+        conv4 = conv_layer(conv3, 8, 3, "conv4")
+        sum2 = conv4 + sum1
+        conv5 = conv_layer(sum2, 8, 3, "conv5")
+        conv6 = conv_layer(conv5, 8, 3, "conv6")
+    
+    print("FLOPs: {}".format(model_size(main_program)))
+
+TableLatencyEvaluator
+---------------------
+
+.. py:class:: paddleslim.analysis.TableLatencyEvaluator(table_file, delimiter=",")
+
+`源代码 <https://github.com/PaddlePaddle/PaddleSlim/blob/develop/paddleslim/analysis/latency.py>`_
+
+基于硬件延时表的模型延时评估器。
+
+**参数：**
+
+- **table_file(str)** - 所使用的延时评估表的绝对路径。关于演示评估表格式请参考：PaddleSlim硬件延时评估表格式
+
+- **delimiter(str)** - 在硬件延时评估表中，操作信息之前所使用的分割符，默认为英文字符逗号。
+
+**返回值：**
+
+- **Evaluator** - 硬件延时评估器的实例。
+
+   .. py:method:: latency(graph)
+
+   获得指定网络的预估延时。
+
+   **参数：**
+   
+   - **graph(Program)** - 待预估的目标网络。
+   
+   **返回值：**
+   
+   - **latency** - 目标网络的预估延时。

_sources/api_cn/index.rst.txt

+.. PaddleSlim documentation master file, created by
+   sphinx-quickstart on Wed Feb  5 14:04:52 2020.
+   You can adapt this file completely to your liking, but it should at least
+   contain the root `toctree` directive.
+
+API文档
+======================================
+
+.. toctree::
+   :maxdepth: 1
+
+   analysis_api.rst
+   nas_api.rst
+   one_shot_api.rst
+   pantheon_api.md
+   prune_api.rst
+   quantization_api.rst
+   single_distiller_api.rst
+   search_space.rst
+   table_latency.md

_sources/api_cn/nas_api.rst.txt

+SA-NAS
+========
+
+搜索空间参数的配置
+----------------------
+
+
+通过参数配置搜索空间。更多搜索空间的使用可以参考: [search_space](../search_space.md)
+
+**参数：**
+
+- **input_size(int|None)**：- ``input_size`` 表示输入 ``feature map`` 的大小。 ``input_size`` 和 ``output_size`` 用来计算整个模型结构中下采样次数。
+
+- **output_size(int|None)**：- ``output_size`` 表示输出feature map的大小。 ``input_size`` 和 ``output_size`` 用来计算整个模型结构中下采样次数。
+
+- **block_num(int|None)**：- ``block_num`` 表示搜索空间中block的数量。
+
+- **block_mask(list|None)**：- ``block_mask`` 是一组由0、1组成的列表，0表示当前block是normal block，1表示当前block是reduction block。reduction block表示经过这个block之后的feature map大小下降为之前的一半，normal block表示经过这个block之后feature map大小不变。如果设置了  ``block_mask`` ，则主要以 ``block_mask`` 为主要配置， ``input_size`` ， ``output_size`` 和 ``block_num`` 三种配置是无效的。
+
+SANAS
+------
+
+.. py:class:: paddleslim.nas.SANAS(configs, server_addr=("", 8881), init_temperature=None, reduce_rate=0.85, init_tokens=None, search_steps=300, save_checkpoint='./nas_checkpoint', load_checkpoint=None, is_server=True)
+
+`源代码 <https://github.com/PaddlePaddle/PaddleSlim/blob/develop/paddleslim/nas/sa_nas.py#L36>`_
+
+SANAS（Simulated Annealing Neural Architecture Search）是基于模拟退火算法进行模型结构搜索的算法，一般用于离散搜索任务。
+
+**参数：**
+
+- **configs(list<tuple>)** - 搜索空间配置列表，格式是 ``[(key, {input_size, output_size, block_num, block_mask})]`` 或者 ``[(key)]`` （MobileNetV2、MobilenetV1和ResNet的搜索空间使用和原本网络结构相同的搜索空间，所以仅需指定 ``key`` 即可）, ``input_size`` 和 ``output_size`` 表示输入和输出的特征图的大小， ``block_num`` 是指搜索网络中的block数量， ``block_mask`` 是一组由0和1组成的列表，0代表不进行下采样的block，1代表下采样的block。 更多paddleslim提供的搜索空间配置可以参考[Search Space](../search_space.md)。
+- **server_addr(tuple)** - SANAS的地址，包括server的ip地址和端口号，如果ip地址为None或者为""的话则默认使用本机ip。默认：（"", 8881）。
+- **init_temperature(float)** - 基于模拟退火进行搜索的初始温度。如果init_template为None而且init_tokens为None，则默认初始温度为10.0，如果init_template为None且init_tokens不为None，则默认初始温度为1.0。详细的温度设置可以参考下面的Note。默认：None。
+- **reduce_rate(float)** - 基于模拟退火进行搜索的衰减率。详细的退火率设置可以参考下面的Note。默认：0.85。
+- **init_tokens(list|None)** - 初始化token，若init_tokens为空，则SA算法随机生成初始化tokens。默认：None。
+- **search_steps(int)** - 搜索过程迭代的次数。默认：300。
+- **save_checkpoint(str|None)** - 保存checkpoint的文件目录，如果设置为None的话则不保存checkpoint。默认： ``./nas_checkpoint`` 。
+- **load_checkpoint(str|None)** - 加载checkpoint的文件目录，如果设置为None的话则不加载checkpoint。默认：None。
+- **is_server(bool)** - 当前实例是否要启动一个server。默认：True。
+
+**返回：**
+一个SANAS类的实例
+
+**示例代码：**
+
+.. code-block:: python
+
+   from paddleslim.nas import SANAS
+   config = [('MobileNetV2Space')]
+   sanas = SANAS(configs=config)
+
+.. note::
+
+  - 初始化温度和退火率的意义:
+
+    - SA算法内部会保存一个基础token（初始化token可以自己传入也可以随机生成）和基础score（初始化score为-1），下一个token会在当前SA算法保存的token的基础上产生。在SA的搜索过程中，如果本轮的token训练得到的score大于SA算法中保存的score，则本轮的token一定会被SA算法接收保存为下一轮token产生的基础token。
+
+    - 初始温度越高表示SA算法当前处的阶段越不稳定，本轮的token训练得到的score小于SA算法中保存的score的话，本轮的token和score被SA算法接收的可能性越大。
+
+    - 初始温度越低表示SA算法当前处的阶段越稳定，本轮的token训练得到的score小于SA算法中保存的score的话，本轮的token和score被SA算法接收的可能性越小。
+
+    - 退火率越大，表示SA算法收敛的越慢，即SA算法越慢到稳定阶段。
+
+    - 退火率越低，表示SA算法收敛的越快，即SA算法越快到稳定阶段。
+
+  - 初始化温度和退火率的设置: 
+
+    - 如果原本就有一个较好的初始化token，想要基于这个较好的token来进行搜索的话，SA算法可以处于一个较为稳定的状态进行搜索r这种情况下初始温度可以设置的低一些，例如设置为1.0，退火率设置的大一些，例如设置为0.85。如果想要基于这个较好的token利用贪心算法进行搜索，即只有当本轮token训练得到的score大于SA算法中保存的score，SA算法才接收本轮token，则退火率可设置为一个极小的数字，例如设置为0.85 ** 10。
+
+    - 初始化token如果是随机生成的话，代表初始化token是一个比较差的token，SA算法可以处于一种不稳定的阶段进行搜索，尽可能的随机探索所有可能得token，从而找到一个较好的token。初始温度可以设置的高一些，例如设置为1000，退火率相对设置的小一些。
+
+.. 
+
+   .. py:method:: next_archs()
+
+   获取下一组模型结构。
+   
+   **返回：**
+   返回模型结构实例的列表，形式为list。
+   
+   **示例代码：**
+
+   .. code-block:: python
+
+      import paddle.fluid as fluid
+      from paddleslim.nas import SANAS
+      config = [('MobileNetV2Space')]
+      sanas = SANAS(configs=config)
+      input = fluid.data(name='input', shape=[None, 3, 32, 32], dtype='float32')
+      archs = sanas.next_archs()
+      for arch in archs:
+          output = arch(input)
+          input = output
+      print(output)
+   
+   .. py:method:: reward(score)
+
+   把当前模型结构的得分情况回传。
+   
+   **参数：**
+   
+   - **score<float>:** - 当前模型的得分，分数越大越好。
+   
+   **返回：**
+   模型结构更新成功或者失败，成功则返回 ``True`` ，失败则返回 ``False`` 。
+   
+   **示例代码：**
+
+   .. code-block:: python
+
+      import paddle.fluid as fluid
+      from paddleslim.nas import SANAS
+      config = [('MobileNetV2Space')]
+      sanas = SANAS(configs=config)
+      archs = sanas.next_archs()
+      
+      ### 假设网络计算出来的score是1，实际代码中使用时需要返回真实score。
+      score=float(1.0)
+      sanas.reward(float(score))
+   
+   
+   .. py:methd:: tokens2arch(tokens)
+
+   通过一组tokens得到实际的模型结构，一般用来把搜索到最优的token转换为模型结构用来做最后的训练。tokens的形式是一个列表，tokens映射到搜索空间转换成相应的网络结构，一组tokens对应唯一的一个网络结构。
+   
+   **参数：**
+   
+   - **tokens(list):** - 一组tokens。tokens的长度和范围取决于搜索空间。
+   
+   **返回：**
+   根据传入的token得到一个模型结构实例。
+   
+   **示例代码：**
+
+   .. code-block:: python
+
+      import paddle.fluid as fluid
+      from paddleslim.nas import SANAS
+      config = [('MobileNetV2Space')]
+      sanas = SANAS(configs=config)
+      input = fluid.data(name='input', shape=[None, 3, 32, 32], dtype='float32')
+      tokens = ([0] * 25)
+      archs = sanas.tokens2arch(tokens)[0]
+      print(archs(input))
+   
+   .. py:method:: current_info()
+
+   返回当前token和搜索过程中最好的token和reward。
+   
+   **返回：**
+   搜索过程中最好的token，reward和当前训练的token，形式为dict。
+   
+   **示例代码：**
+
+   .. code-block:: python
+      import paddle.fluid as fluid
+      from paddleslim.nas import SANAS
+      config = [('MobileNetV2Space')]
+      sanas = SANAS(configs=config)
+      print(sanas.current_info())

_sources/api_cn/one_shot_api.rst.txt

+OneShotNAS
+=============
+
+OneShotSearch
+------------------
+
+.. py:function:: paddleslim.nas.one_shot.OneShotSearch(model, eval_func, strategy='sa', search_steps=100)
+
+从超级网络中搜索出一个最佳的子网络。
+
+**参数：**
+
+- **model(fluid.dygraph.layer):** 通过在 ``OneShotSuperNet`` 前后添加若该模块构建的动态图模块。因为 ``OneShotSuperNet`` 是一个超网络，所以 ``model`` 也是一个超网络。换句话说，在 ``model`` 模块的子模块中，至少有一个是 ``OneShotSuperNet`` 的实例。该方法从 ``model`` 超网络中搜索得到一个最佳的子网络。超网络 ``model`` 需要先被训练，具体细节请参考[OneShotSuperNet]()。
+
+- **eval_func:** 用于评估子网络性能的回调函数。该回调函数需要接受 ``model`` 为参数，并调用 ``model`` 的 ``forward`` 方法进行性能评估。
+
+- **strategy(str):** 搜索策略的名称。默认为 ``sa`` ， 当前仅支持 ``sa`` .
+
+- **search_steps(int):** 搜索轮次数。默认为100。
+
+**返回：**
+
+- **best_tokens:** 表示最佳子网络的编码信息（tokens）。
+
+**示例代码：**
+
+请参考[one-shot NAS示例]()
+
+
+OneShotSuperNet
+-----------------
+
+.. py:class:: paddleslim.nas.one_shot.OneShotSuperNet(name_scope)
+
+用于`OneShot`搜索策略的超级网络的基类，所有超级网络的实现要继承该类。
+
+**参数：**
+
+- **name_scope:(str) **超级网络的命名空间。
+
+**返回：**
+
+- **super_net:** 一个`OneShotSuperNet`实例。
+
+
+   .. py:method:: init_tokens()
+   
+   获得当前超级网络的初始化子网络的编码，主要用于搜索。
+   
+   **返回：**
+   
+   - **tokens(list<int>):** 一个子网络的编码。
+   
+   range_table()
+   
+   : 超级网络中各个子网络由一组整型数字编码表示，该方法返回编码每个位置的取值范围。
+   
+   **返回：**
+   
+   - **range_table(tuple):** 子网络编码每一位的取值范围。 ``range_table`` 格式为 ``(min_values, max_values)`` ，其中， ``min_values`` 为一个整型数组，表示每个编码位置可选取的最小值； ``max_values`` 表示每个编码位置可选取的最大值。
+   
+   .. py:method:: _forward_impl(input, tokens)
+   
+   前向计算函数。 ``OneShotSuperNet`` 的子类需要实现该函数。
+   
+   **参数：**
+   
+   - **input(Variable):** 超级网络的输入。
+   
+   - **tokens(list<int>):** 执行前向计算所用的子网络的编码。默认为 ``None`` ，即随机选取一个子网络执行前向。
+   
+   **返回：**
+   
+   - **output(Variable):** 前向计算的输出
+   
+   .. py:method:: forward(self, input, tokens=None)
+   
+   执行前向计算。
+   
+   **参数：**
+   
+   - **input(Variable):** 超级网络的输入。
+   
+   - **tokens(list<int>):** 执行前向计算所用的子网络的编码。默认为 ``None`` ，即随机选取一个子网络执行前向。
+   
+   **返回：**
+   
+   - **output(Variable):** 前向计算的输出
+   
+   
+   .. py:method:: _random_tokens()
+   
+   随机选取一个子网络，并返回其编码。
+   
+   **返回：**
+   
+   - **tokens(list<int>):** 一个子网络的编码。
+
+SuperMnasnet
+--------------
+
+
+.. py:class:: paddleslim.nas.one_shot.SuperMnasnet(name_scope, input_channels=3, out_channels=1280, repeat_times=[6, 6, 6, 6, 6, 6], stride=[1, 1, 1, 1, 2, 1], channels=[16, 24, 40, 80, 96, 192, 320], use_auxhead=False)
+
+在 `Mnasnet <https://arxiv.org/abs/1807.11626>`_ 基础上修改得到的超级网络, 该类继承自 ``OneShotSuperNet`` .
+
+**参数：**
+
+- **name_scope(str):** 命名空间。
+
+- **input_channels(str):** 当前超级网络的输入的特征图的通道数量。
+
+- **out_channels(str):** 当前超级网络的输出的特征图的通道数量。
+
+- **repeat_times(list):** 每种 ``block`` 重复的次数。
+
+- **stride(list):** 一种 ``block`` 重复堆叠成 ``repeat_block`` ， ``stride`` 表示每个 ``repeat_block`` 的下采样比例。
+
+- **channels(list):** ``channels[i]`` 和 ``channels[i+1]`` 分别表示第i个 ``repeat_block`` 的输入特征图的通道数和输出特征图的通道数。
+
+- **use_auxhead(bool):** 是否使用辅助特征图。如果设置为 ``True`` ，则 ``SuperMnasnet`` 除了返回输出特征图，还还返回辅助特征图。默认为False.
+
+**返回：**
+
+- **instance(SuperMnasnet):** 一个 ``SuperMnasnet`` 实例
+
+**示例：**
+.. code-block:: python
+
+   import paddle
+   import paddle.fluid as fluid
+   class MNIST(fluid.dygraph.Layer):
+       def __init__(self):
+           super(MNIST, self).__init__()
+           self.arch = SuperMnasnet(
+               name_scope="super_net", input_channels=20, out_channels=20)
+           self.pool_2_shape = 50 * 13 * 13
+           SIZE = 10
+           scale = (2.0 / (self.pool_2_shape**2 * SIZE))**0.5
+           self._fc = Linear(
+               self.pool_2_shape,
+               10,
+               param_attr=fluid.param_attr.ParamAttr(
+                   initializer=fluid.initializer.NormalInitializer(
+                       loc=0.0, scale=scale)),
+               act="softmax")
+   
+       def forward(self, inputs, label=None, tokens=None):
+   
+           x = self.arch(inputs, tokens=tokens)
+           x = fluid.layers.reshape(x, shape=[-1, self.pool_2_shape])
+           x = self._fc(x)
+           if label is not None:
+               acc = fluid.layers.accuracy(input=x, label=label)
+               return x, acc
+           else:
+               return x

_sources/api_cn/pantheon_api.md.txt

+# 多进程蒸馏
+
+## Teacher
+
+pantheon.Teacher() [source](https://github.com/PaddlePaddle/PaddleSlim/blob/develop/paddleslim/pantheon/teacher.py#L78)
+
+: The class defined for the teacher model. Generate knowledge data and transfer them to the student model.
+
+**Args:**
+
+- **out\_path (str|None)** - The path to dump knowledge data for offline mode.
+
+- **out\_port (int|None)** - The IP port number to send out knowledge for online mode, should be unique when launching multiple teachers in the same node.
+
+**Return:** An object of class Teacher
+
+
+pantheon.Teacher.start() [source](https://github.com/PaddlePaddle/PaddleSlim/blob/develop/paddleslim/pantheon/teacher.py#L133)
+
+: Start teacher service, sychronize with student and launch the thread
+  to monitor commands from student.
+
+**Args:** None
+
+**Return:** None
+
+
+pantheon.Teacher.send(data) [source](https://github.com/PaddlePaddle/PaddleSlim/blob/develop/paddleslim/pantheon/teacher.py#L181)
+
+: Send one data object to student.
+
+**Args:**
+
+- **data (Python data):** - The data to be sent, can be any type of Python data object.
+
+**Return:** None
+
+
+pantheon.Teacher.recv() [source](https://github.com/PaddlePaddle/PaddleSlim/blob/develop/paddleslim/pantheon/teacher.py#L196)
+
+: Recieve one data object from student.
+
+**Args:** None
+
+**Return:**
+
+- The received data, can be any type of Python data object.
+
+
+pantheon.Teacher.dump(knowledge) [source](https://github.com/PaddlePaddle/PaddleSlim/blob/develop/paddleslim/pantheon/teacher.py#L214)
+
+: Dump one batch knowledge data into the output file, only used in the offline mode.
+
+**Args:**
+
+- **knowledge (dict):** - The knowledge data to be dumped.  
+
+**Return:** None
+
+
+pantheon.Teacher.start\_knowledge\_service(feed\_list, schema, program, reader\_config, exe, buf\_size=10, times=1) [source](https://github.com/PaddlePaddle/PaddleSlim/blob/develop/paddleslim/pantheon/teacher.py#L259)
+
+: Start the knowledge service to generate and transfer knowledge data. In GPU mode, the devices to execute knowledge prediction will be determined by the
+  environment variable **FLAGS\_selected\_gpus**, or by **CUDA\_VISIBLE\_DEVICES** if it is not set, and by **CPU\_NUM** (default 1) in CPU mode. Only supported in static graph.
+
+ **Args:**
+
+ - **feed\_list (list):** - A list of feed Variables or their names for the
+                              input teacher Program.
+ - **schema (dict):** - A dict to specify keys and fetched Variables  
+                        to generate knowledge.
+ - **program (fluid.Program):** - Inference Program of the teacher model.
+ - **reader\_config (dict):** - The config for data reader. Support all the three types of generators used by [fluid.io.PyReader](https://www.paddlepaddle.org.cn/documentation/docs/en/api/io/PyReader.html) and [fluid.io.DataLoader](https://www.paddlepaddle.org.cn/documentation/docs/en/api/io/DataLoader.html#dataloader), and their configs contain the key-value pair of the generator type and a generator object, plus other necessary argument pairs. See the following:
+
+     1) **sample generator:**
+
+     ```
+     reader_config={"sample_generator": some_sample_generator,
+                    "batch_size": batch_size, "drop_last": drop_last}
+     # drop_last set to True by default
+     ```
+
+     2) **sample list generator:**
+
+     ```
+     reader_config={"sample_list_generator": some_sample_list_generator}
+     ```
+
+     3) **batch generator:**
+
+     ```
+     reader_config={"batch_generator": some_batch_genrator}
+     ```
+
+     The trial to parse config will be in the order of 1) -> 3), and any other unrelated keys in these configs will be ignored.
+
+- **exe (fluid.Executor):** The executor to run the input program.
+- **buf\_size (int):** The size of buffers for data reader and knowledge
+                            writer on each device.
+- **times (int):** The maximum repeated serving times, default 1. Whenever
+                         the public method **get\_knowledge\_generator()** in **Student**
+                         object called once, the serving times will be added one,
+                         until reaching the maximum and ending the service.
+
+**Return:** None
+
+**Examples:**
+
+```python
+import paddle
+import paddle.fluid as fluid
+from paddleslim.pantheon import Teacher
+
+startup = fluid.Program()
+program = fluid.Program()
+with fluid.program_guard(program, startup):
+    images = fluid.data(
+            name='pixel', shape=[None, 3 * 32 * 32], dtype='float32')
+    labels = fluid.data(name='label', shape=[None, 1], dtype='int64')
+    logits = fluid.layers.fc(input=images, size=10)
+    loss = fluid.layers.softmax_with_cross_entropy(logits, labels)
+
+place = fluid.CPUPlace()
+exe = fluid.Executor(place)
+exe.run(startup)
+
+train_reader = paddle.batch(
+        paddle.dataset.cifar.train10(), batch_size=32)
+
+teacher = Teacher(out_path="example_knowledge.dat", # offline mode
+                  #out_port=5000                    # online mode
+                  )
+teacher.start()
+
+teacher.start_knowledge_service(
+    feed_list=[images, labels],
+    schema={"logits": logits,
+            "labels": labels},
+    program=program,
+    reader_config={"sample_list_generator": train_reader},
+    exe=exe)
+```
+
+!!! note "Note"
+    This example should be run with the example of class **Student**.
+
+
+## Student
+
+pantheon.Student(merge_strategy=None) [source](https://github.com/PaddlePaddle/PaddleSlim/blob/develop/paddleslim/pantheon/student.py#L34)
+
+: The class defined for the student model. Receive knowledge data from
+    teacher model and carry out knowledge merging.  
+
+ **Args:**
+
+ - **merge\_strategy (dict|None):** - A dict whose keys are the common schemas shared by different teachers, and each corresponding value specifies the merging strategy for different schemas respectively, supporting **sum** and **mean** now.
+
+**Return:** An object of class Student.
+
+
+pantheon.Student.register\_teacher(in\_path=None, in\_address=None) [source](https://github.com/PaddlePaddle/PaddleSlim/blob/develop/paddleslim/pantheon/student.py#L72)
+
+: Register one teacher model and assign the order number to it as  its id, with the file path (offline mode) or IP address (online  mode) that the teacher model writes knowledge data to.
+
+**Args:**
+
+- **in\_path (str|None):** The input file path. Default None.
+- **in\_address (str|None):** The input IP address, in the format "<IP\_address>:<IP\_port>" (e.g. "127.0.0.1:8080"). Default None.
+
+**Return:**  None
+
+
+pantheon.Student.start() [source](https://github.com/PaddlePaddle/PaddleSlim/blob/develop/paddleslim/pantheon/student.py#L213)
+
+: End teachers' registration and synchronize with all of them.
+
+**Args:** None
+
+**Return:**  None
+
+pantheon.Student.send(self, data, teacher_ids=None) [source](https://github.com/PaddlePaddle/PaddleSlim/blob/develop/paddleslim/pantheon/student.py#L240)
+
+: Send data to teachers.
+
+**Args:**
+
+- **data (Python data):** - A Python data object to be sent.
+- **teacher_ids (list|None):** - A list of teacher ids to send data. If set to None, send the data to all teachers. Default None.
+
+**Return:**  None
+
+pantheon.Student.recv(teacher_id) [source](https://github.com/PaddlePaddle/PaddleSlim/blob/develop/paddleslim/pantheon/student.py#L262)
+
+: Receive data from one teacher.
+
+ **Args:**
+
+- **teacher\_id (int):** - The id of teacher that receives data from.
+
+**Return:**  
+
+- The received data object.
+
+pantheon.Student.get\_knowledge\_desc() [source](https://github.com/PaddlePaddle/PaddleSlim/blob/develop/paddleslim/pantheon/student.py#L283)
+
+ : Get description for knowledge, including shape, data type and lod level for each schema.
+
+ **Args:** None
+
+ **Return:**  
+
+ - Knowledge description, which is a dict.
+
+
+pantheon.Student.get\_knowledge\_qsize() [source](https://github.com/PaddlePaddle/PaddleSlim/blob/develop/paddleslim/pantheon/student.py#L318)
+
+ : Get the real-time size of knowledge queue. If this size is denoted as
+   **qsize**, it means that there are **qsize** batch knowledge data
+   already pushed into knowledge queue and waiting for the knowledge
+   generator to pop out. It's dynamic and limited up to 100, the capacity
+   of the knowledge queue.
+
+ **Args:** None
+
+ **Return:**  
+
+ - The real-time size of knowledge queue.
+
+pantheon.Student.get\_knowledge\_generator(batch\_size, drop\_last=False) [source](https://github.com/PaddlePaddle/PaddleSlim/blob/develop/paddleslim/pantheon/student.py#L334)
+
+: Get the generator for knowledge data, return None if last generator doesn't finish yet.
+
+**Args:**
+
+- **batch\_size (int):** - The batch size of returned knowledge data.
+- **drop\_last (bool):** - Whether to drop the last batch if its size is less than batch size.
+
+**Return:**
+
+- The wrapper of knowledge data generator.
+
+**Examples:**
+
+```python
+from paddleslim.pantheon import Student
+
+student = Student()
+
+student.register_teacher(in_path="example_knowledge.dat",  # offline mode
+                         #in_address="127.0.0.1:5000"      # online mode
+                         )
+student.start()
+
+knowledge_desc = student.get_knowledge_desc()
+data_generator = student.get_knowledge_generator(
+    batch_size=128, drop_last=True)
+
+# get knowledge data
+for knowledge in data_generator():
+    print("knowledge queue size: {}".format(student.get_knowledge_qsize()))
+
+    # do something else
+```
+
+!!! note "Note"
+    This example should be run with the example of class **Teacher**.

_sources/api_cn/prune_api.rst.txt

+卷积层通道剪裁
+================
+
+Pruner
+----------
+
+.. py:class:: paddleslim.prune.Pruner(criterion="l1_norm")
+
+`源代码 <https://github.com/PaddlePaddle/PaddleSlim/blob/develop/paddleslim/prune/pruner.py#L28>`_
+
+对卷积网络的通道进行一次剪裁。剪裁一个卷积层的通道，是指剪裁该卷积层输出的通道。卷积层的权重形状为 ``[output_channel, input_channel, kernel_size, kernel_size]`` ，通过剪裁该权重的第一纬度达到剪裁输出通道数的目的。
+
+**参数：**
+
+- **criterion** - 评估一个卷积层内通道重要性所参考的指标。目前仅支持 ``l1_norm`` 。默认为 ``l1_norm`` 。
+
+**返回：** 一个Pruner类的实例
+
+**示例代码：**
+
+.. code-block:: python
+
+   from paddleslim.prune import Pruner
+   pruner = Pruner()
+
+..
+ 
+   .. py:method:: paddleslim.prune.Pruner.prune(program, scope, params, ratios, place=None, lazy=False, only_graph=False, param_backup=False, param_shape_backup=False)
+
+   对目标网络的一组卷积层的权重进行裁剪。
+   
+   **参数：**
+   
+   - **program(paddle.fluid.Program)** - 要裁剪的目标网络。更多关于Program的介绍请参考：`Program概念介绍 <https://www.paddlepaddle.org.cn/documentation/docs/zh/api_cn/fluid_cn/Program_cn.html#program>`_。
+   
+   - **scope(paddle.fluid.Scope)** - 要裁剪的权重所在的 ``scope`` ，Paddle中用 ``scope`` 实例存放模型参数和运行时变量的值。Scope中的参数值会被 ``inplace`` 的裁剪。更多介绍请参考: `Scope概念介绍 <>`_
+   
+   - **params(list<str>)** - 需要被裁剪的卷积层的参数的名称列表。可以通过以下方式查看模型中所有参数的名称:
+   
+   .. code-block:: python
+   
+      for block in program.blocks:
+          for param in block.all_parameters():
+              print("param: {}; shape: {}".format(param.name, param.shape))
+   
+   - **ratios(list<float>)** - 用于裁剪 ``params`` 的剪切率，类型为列表。该列表长度必须与 ``params`` 的长度一致。
+   
+   - **place(paddle.fluid.Place)** - 待裁剪参数所在的设备位置，可以是 ``CUDAPlace`` 或 ``CPUPlace`` 。[Place概念介绍]()
+   
+   - **lazy(bool)** - ``lazy`` 为True时，通过将指定通道的参数置零达到裁剪的目的，参数的 ``shape保持不变`` ； ``lazy`` 为False时，直接将要裁的通道的参数删除，参数的 ``shape`` 会发生变化。
+   
+   - **only_graph(bool)** - 是否只裁剪网络结构。在Paddle中，Program定义了网络结构，Scope存储参数的数值。一个Scope实例可以被多个Program使用，比如定义了训练网络的Program和定义了测试网络的Program是使用同一个Scope实例的。 ``only_graph`` 为True时，只对Program中定义的卷积的通道进行剪裁； ``only_graph`` 为false时，Scope中卷积参数的数值也会被剪裁。默认为False。
+   
+   - **param_backup(bool)** - 是否返回对参数值的备份。默认为False。
+   
+   - **param_shape_backup(bool)** - 是否返回对参数 ``shape`` 的备份。默认为False。
+   
+   **返回：**
+   
+   - **pruned_program(paddle.fluid.Program)** - 被裁剪后的Program。
+   
+   - **param_backup(dict)** - 对参数数值的备份，用于恢复Scope中的参数数值。
+   
+   - **param_shape_backup(dict)** - 对参数形状的备份。
+   
+   **示例：**
+   
+   点击 `AIStudio <https://aistudio.baidu.com/aistudio/projectDetail/200786>`_ 执行以下示例代码。
+
+   .. code-block:: python
+   
+      import paddle.fluid as fluid
+      from paddle.fluid.param_attr import ParamAttr
+      from paddleslim.prune import Pruner
+      
+      def conv_bn_layer(input,
+                        num_filters,
+                        filter_size,
+                        name,
+                        stride=1,
+                        groups=1,
+                        act=None):
+          conv = fluid.layers.conv2d(
+              input=input,
+              num_filters=num_filters,
+              filter_size=filter_size,
+              stride=stride,
+              padding=(filter_size - 1) // 2,
+              groups=groups,
+              act=None,
+              param_attr=ParamAttr(name=name + "_weights"),
+              bias_attr=False,
+              name=name + "_out")
+          bn_name = name + "_bn"
+          return fluid.layers.batch_norm(
+              input=conv,
+              act=act,
+              name=bn_name + '_output',
+              param_attr=ParamAttr(name=bn_name + '_scale'),
+              bias_attr=ParamAttr(bn_name + '_offset'),
+              moving_mean_name=bn_name + '_mean',
+              moving_variance_name=bn_name + '_variance', )
+      
+      main_program = fluid.Program()
+      startup_program = fluid.Program()
+      #   X       X              O       X              O
+      # conv1-->conv2-->sum1-->conv3-->conv4-->sum2-->conv5-->conv6
+      #     |            ^ |                    ^
+      #     |____________| |____________________|
+      #
+      # X: prune output channels
+      # O: prune input channels
+      with fluid.program_guard(main_program, startup_program):
+          input = fluid.data(name="image", shape=[None, 3, 16, 16])
+          conv1 = conv_bn_layer(input, 8, 3, "conv1")
+          conv2 = conv_bn_layer(conv1, 8, 3, "conv2")
+          sum1 = conv1 + conv2
+          conv3 = conv_bn_layer(sum1, 8, 3, "conv3")
+          conv4 = conv_bn_layer(conv3, 8, 3, "conv4")
+          sum2 = conv4 + sum1
+          conv5 = conv_bn_layer(sum2, 8, 3, "conv5")
+          conv6 = conv_bn_layer(conv5, 8, 3, "conv6")
+      
+      place = fluid.CPUPlace()
+      exe = fluid.Executor(place)
+      scope = fluid.Scope()
+      exe.run(startup_program, scope=scope)
+      pruner = Pruner()
+      main_program, _, _ = pruner.prune(
+          main_program,
+          scope,
+          params=["conv4_weights"],
+          ratios=[0.5],
+          place=place,
+          lazy=False,
+          only_graph=False,
+          param_backup=False,
+          param_shape_backup=False)
+      
+      for param in main_program.global_block().all_parameters():
+          if "weights" in param.name:
+              print("param name: {}; param shape: {}".format(param.name, param.shape))
+      
+
+sensitivity
+--------------
+
+.. py:function:: paddleslim.prune.sensitivity(program, place, param_names, eval_func, sensitivities_file=None, pruned_ratios=None)
+
+`源代码 <https://github.com/PaddlePaddle/PaddleSlim/blob/develop/paddleslim/prune/sensitive.py>`_
+
+计算网络中每个卷积层的敏感度。每个卷积层的敏感度信息统计方法为：依次剪掉当前卷积层不同比例的输出通道数，在测试集上计算剪裁后的精度损失。得到敏感度信息后，可以通过观察或其它方式确定每层卷积的剪裁率。
+
+**参数：**
+
+- **program(paddle.fluid.Program)** - 待评估的目标网络。更多关于Program的介绍请参考：`Program概念介绍 <https://www.paddlepaddle.org.cn/documentation/docs/zh/api_cn/fluid_cn/Program_cn.html#program>`_。
+
+- **place(paddle.fluid.Place)** - 待分析的参数所在的设备位置，可以是 ``CUDAPlace`` 或 ``CPUPlace`` 。[Place概念介绍]()
+
+- **param_names(list<str>)** - 待分析的卷积层的参数的名称列表。可以通过以下方式查看模型中所有参数的名称:
+
+.. code-block:: python
+   for block in program.blocks:
+       for param in block.all_parameters():
+           print("param: {}; shape: {}".format(param.name, param.shape))
+
+- **eval_func(function)** - 用于评估裁剪后模型效果的回调函数。该回调函数接受被裁剪后的 ``program`` 为参数，返回一个表示当前program的精度，用以计算当前裁剪带来的精度损失。
+
+- **sensitivities_file(str)** - 保存敏感度信息的本地文件系统的文件。在敏感度计算过程中，会持续将新计算出的敏感度信息追加到该文件中。重启任务后，文件中已有敏感度信息不会被重复计算。该文件可以用 ``pickle`` 加载。
+
+- **pruned_ratios(list<float>)** - 计算卷积层敏感度信息时，依次剪掉的通道数比例。默认为 ``[0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9]`` 。
+
+**返回：**
+
+- **sensitivities(dict)** - 存放敏感度信息的dict，其格式为：
+
+.. code-block:: python
+
+  {"weight_0":
+     {0.1: 0.22,
+      0.2: 0.33
+     },
+   "weight_1":
+     {0.1: 0.21,
+      0.2: 0.4
+     }
+  }
+
+其中， ``weight_0`` 是卷积层参数的名称， ``sensitivities['weight_0']`` 的 ``value`` 为剪裁比例， ``value`` 为精度损失的比例。
+
+**示例：**
+
+点击 `AIStudio <https://aistudio.baidu.com/aistudio/projectdetail/201401>`_ 运行以下示例代码。
+
+.. code-block:: python
+
+   import paddle
+   import numpy as np
+   import paddle.fluid as fluid
+   from paddle.fluid.param_attr import ParamAttr
+   from paddleslim.prune import sensitivity
+   import paddle.dataset.mnist as reader
+   
+   def conv_bn_layer(input,
+                     num_filters,
+                     filter_size,
+                     name,
+                     stride=1,
+                     groups=1,
+                     act=None):
+       conv = fluid.layers.conv2d(
+           input=input,
+           num_filters=num_filters,
+           filter_size=filter_size,
+           stride=stride,
+           padding=(filter_size - 1) // 2,
+           groups=groups,
+           act=None,
+           param_attr=ParamAttr(name=name + "_weights"),
+           bias_attr=False,
+           name=name + "_out")
+       bn_name = name + "_bn"
+       return fluid.layers.batch_norm(
+           input=conv,
+           act=act,
+           name=bn_name + '_output',
+           param_attr=ParamAttr(name=bn_name + '_scale'),
+           bias_attr=ParamAttr(bn_name + '_offset'),
+           moving_mean_name=bn_name + '_mean',
+           moving_variance_name=bn_name + '_variance', )
+   
+   main_program = fluid.Program()
+   startup_program = fluid.Program()
+   #   X       X              O       X              O
+   # conv1-->conv2-->sum1-->conv3-->conv4-->sum2-->conv5-->conv6
+   #     |            ^ |                    ^
+   #     |____________| |____________________|
+   #
+   # X: prune output channels
+   # O: prune input channels
+   image_shape = [1,28,28]
+   with fluid.program_guard(main_program, startup_program):
+       image = fluid.data(name='image', shape=[None]+image_shape, dtype='float32')
+       label = fluid.data(name='label', shape=[None, 1], dtype='int64')  
+       conv1 = conv_bn_layer(image, 8, 3, "conv1")
+       conv2 = conv_bn_layer(conv1, 8, 3, "conv2")
+       sum1 = conv1 + conv2
+       conv3 = conv_bn_layer(sum1, 8, 3, "conv3")
+       conv4 = conv_bn_layer(conv3, 8, 3, "conv4")
+       sum2 = conv4 + sum1
+       conv5 = conv_bn_layer(sum2, 8, 3, "conv5")
+       conv6 = conv_bn_layer(conv5, 8, 3, "conv6")
+       out = fluid.layers.fc(conv6, size=10, act="softmax")
+   #    cost = fluid.layers.cross_entropy(input=out, label=label)
+   #    avg_cost = fluid.layers.mean(x=cost)
+       acc_top1 = fluid.layers.accuracy(input=out, label=label, k=1)
+   #    acc_top5 = fluid.layers.accuracy(input=out, label=label, k=5)
+   
+   
+   place = fluid.CPUPlace()
+   exe = fluid.Executor(place)
+   exe.run(startup_program)
+   
+   val_reader = paddle.batch(reader.test(), batch_size=128)
+   val_feeder = feeder = fluid.DataFeeder(
+           [image, label], place, program=main_program)
+   
+   def eval_func(program):
+   
+       acc_top1_ns = []
+       for data in val_reader():
+           acc_top1_n = exe.run(program,
+                                feed=val_feeder.feed(data),
+                                fetch_list=[acc_top1.name])
+           acc_top1_ns.append(np.mean(acc_top1_n))
+       return np.mean(acc_top1_ns)
+   param_names = []
+   for param in main_program.global_block().all_parameters():
+       if "weights" in param.name:
+           param_names.append(param.name)
+   sensitivities = sensitivity(main_program,
+                               place,
+                               param_names,
+                               eval_func,
+                               sensitivities_file="./sensitive.data",
+                               pruned_ratios=[0.1, 0.2, 0.3])
+   print(sensitivities)
+   
+
+merge_sensitive
+----------------
+
+.. py:function:: paddleslim.prune.merge_sensitive(sensitivities)
+
+`源代码 <https://github.com/PaddlePaddle/PaddleSlim/blob/develop/paddleslim/prune/sensitive.py>`_
+
+合并多个敏感度信息。
+
+参数：
+
+- **sensitivities(list<dict> | list<str>)** - 待合并的敏感度信息，可以是字典的列表，或者是存放敏感度信息的文件的路径列表。
+
+返回：
+
+- **sensitivities(dict)** - 合并后的敏感度信息。其格式为：
+
+.. code-block:: bash
+
+   {"weight_0":
+      {0.1: 0.22,
+       0.2: 0.33
+      },
+    "weight_1":
+      {0.1: 0.21,
+       0.2: 0.4
+      }
+   }
+   
+
+其中， ``weight_0`` 是卷积层参数的名称， ``sensitivities['weight_0']`` 的 ``value`` 为剪裁比例， ``value`` 为精度损失的比例。
+
+示例：
+
+.. code-block:: python
+
+   from paddleslim.prune import merge_sensitive
+   sen0 = {"weight_0":
+      {0.1: 0.22,
+       0.2: 0.33
+      },
+    "weight_1":
+      {0.1: 0.21,
+       0.2: 0.4
+      }
+   }
+   sen1 = {"weight_0":
+      {0.3: 0.41,
+      },
+    "weight_2":
+      {0.1: 0.10,
+       0.2: 0.35
+      }
+   }
+   sensitivities = merge_sensitive([sen0, sen1])
+   print(sensitivities)
+
+
+load_sensitivities
+---------------------
+
+.. py:function:: paddleslim.prune.load_sensitivities(sensitivities_file)
+
+`源代码 <https://github.com/PaddlePaddle/PaddleSlim/blob/develop/paddleslim/prune/sensitive.py#L184>`_
+
+从文件中加载敏感度信息。
+
+参数：
+
+- **sensitivities_file(str)** - 存放敏感度信息的本地文件.
+
+返回：
+
+- **sensitivities(dict)** - 敏感度信息。
+
+示例：
+
+.. code-block:: python
+
+  import pickle
+  from paddleslim.prune import load_sensitivities
+  sen = {"weight_0":
+     {0.1: 0.22,
+      0.2: 0.33
+     },
+   "weight_1":
+     {0.1: 0.21,
+      0.2: 0.4
+     }
+  }
+  sensitivities_file = "sensitive_api_demo.data"
+  with open(sensitivities_file, 'w') as f:
+      pickle.dump(sen, f)
+  sensitivities = load_sensitivities(sensitivities_file)
+  print(sensitivities)
+
+get_ratios_by_loss
+-------------------
+
+.. py:function:: paddleslim.prune.get_ratios_by_loss(sensitivities, loss)
+
+`源代码 <https://github.com/PaddlePaddle/PaddleSlim/blob/develop/paddleslim/prune/sensitive.py>`_
+
+根据敏感度和精度损失阈值计算出一组剪切率。对于参数 ``w`` , 其剪裁率为使精度损失低于 ``loss`` 的最大剪裁率。
+
+**参数：**
+
+- **sensitivities(dict)** - 敏感度信息。
+
+- **loss** - 精度损失阈值。
+
+**返回：**
+
+- **ratios(dict)** - 一组剪切率。 ``key`` 是待剪裁参数的名称。 ``value`` 是对应参数的剪裁率。
+
+**示例：**
+
+.. code-block:: python
+   
+  from paddleslim.prune import get_ratios_by_loss
+  sen = {"weight_0":
+     {0.1: 0.22,
+      0.2: 0.33
+     },
+   "weight_1":
+     {0.1: 0.21,
+      0.2: 0.4
+     }
+  }
+  
+  ratios = get_ratios_by_loss(sen, 0.3)
+  print(ratios)

_sources/api_cn/quantization_api.rst.txt

+量化
+====
+
+量化配置
+---------------
+通过字典配置量化参数
+
+.. code-block:: python
+
+
+   TENSORRT_OP_TYPES = [
+       'mul', 'conv2d', 'pool2d', 'depthwise_conv2d', 'elementwise_add',
+       'leaky_relu'
+   ]
+   TRANSFORM_PASS_OP_TYPES = ['conv2d', 'depthwise_conv2d', 'mul']
+   
+   QUANT_DEQUANT_PASS_OP_TYPES = [
+           "pool2d", "elementwise_add", "concat", "softmax", "argmax", "transpose",
+           "equal", "gather", "greater_equal", "greater_than", "less_equal",
+           "less_than", "mean", "not_equal", "reshape", "reshape2",
+           "bilinear_interp", "nearest_interp", "trilinear_interp", "slice",
+           "squeeze", "elementwise_sub", "relu", "relu6", "leaky_relu", "tanh", "swish"
+       ]
+   
+   _quant_config_default = {
+       # weight quantize type, default is 'channel_wise_abs_max'
+       'weight_quantize_type': 'channel_wise_abs_max',
+       # activation quantize type, default is 'moving_average_abs_max'
+       'activation_quantize_type': 'moving_average_abs_max',
+       # weight quantize bit num, default is 8
+       'weight_bits': 8,
+       # activation quantize bit num, default is 8
+       'activation_bits': 8,
+       # ops of name_scope in not_quant_pattern list, will not be quantized
+       'not_quant_pattern': ['skip_quant'],
+       # ops of type in quantize_op_types, will be quantized
+       'quantize_op_types': ['conv2d', 'depthwise_conv2d', 'mul'],
+       # data type after quantization, such as 'uint8', 'int8', etc. default is 'int8'
+       'dtype': 'int8',
+       # window size for 'range_abs_max' quantization. defaulf is 10000
+       'window_size': 10000,
+       # The decay coefficient of moving average, default is 0.9
+       'moving_rate': 0.9,
+       # if True, 'quantize_op_types' will be TENSORRT_OP_TYPES
+       'for_tensorrt': False,
+       # if True, 'quantoze_op_types' will be TRANSFORM_PASS_OP_TYPES + QUANT_DEQUANT_PASS_OP_TYPES
+       'is_full_quantize': False
+   }
+
+**参数：**
+
+- **weight_quantize_type(str)** - 参数量化方式。可选 ``'abs_max'`` ,  ``'channel_wise_abs_max'`` , ``'range_abs_max'`` , ``'moving_average_abs_max'`` 。如果使用 ``TensorRT`` 加载量化后的模型来预测，请使用 ``'channel_wise_abs_max'`` 。 默认 ``'channel_wise_abs_max'`` 。
+- **activation_quantize_type(str)** - 激活量化方式，可选 ``'abs_max'`` ,  ``'range_abs_max'`` ,  ``'moving_average_abs_max'`` 。如果使用 ``TensorRT`` 加载量化后的模型来预测，请使用 ``'range_abs_max', 'moving_average_abs_max'`` 。，默认 ``'moving_average_abs_max'`` 。
+- **weight_bits(int)** - 参数量化bit数，默认8, 推荐设为8。
+- **activation_bits(int)** -  激活量化bit数，默认8， 推荐设为8。
+- **not_quant_pattern(str | list[str])** - 所有 ``name_scope`` 包含 ``'not_quant_pattern'`` 字符串的 ``op`` ，都不量化, 设置方式请参考 `fluid.name_scope <https://www.paddlepaddle.org.cn/documentation/docs/zh/api_cn/fluid_cn/name_scope_cn.html#name-scope>`_ 。
+- **quantize_op_types(list[str])** -  需要进行量化的 ``op`` 类型，目前支持 ``'conv2d', 'depthwise_conv2d', 'mul'``  。
+- **dtype(int8)** - 量化后的参数类型，默认 ``int8`` , 目前仅支持 ``int8`` 。
+- **window_size(int)** -  ``'range_abs_max'`` 量化方式的 ``window size`` ，默认10000。
+- **moving_rate(int)** - ``'moving_average_abs_max'`` 量化方式的衰减系数，默认 0.9。
+- **for_tensorrt(bool)** - 量化后的模型是否使用 ``TensorRT`` 进行预测。如果是的话，量化op类型为： ``TENSORRT_OP_TYPES`` 。默认值为False.
+- **is_full_quantize(bool)** - 是否量化所有可支持op类型。默认值为False.
+
+.. :note::
+
+   目前 ``Paddle-Lite`` 有int8 kernel来加速的op只有 ``['conv2d', 'depthwise_conv2d', 'mul']``, 其他op的int8 kernel将陆续支持。
+
+
+quant_aware
+------------
+
+.. py:function:: paddleslim.quant.quant_aware(program, place, config, scope=None, for_test=False)[[源代码]](https://github.com/PaddlePaddle/PaddleSlim/blob/develop/paddleslim/quant/quanter.py)
+
+在 ``program`` 中加入量化和反量化 ``op``, 用于量化训练。
+
+
+**参数：**
+
+- **program (fluid.Program)** -  传入训练或测试 ``program`` 。
+- **place(fluid.CPUPlace | fluid.CUDAPlace)** -  该参数表示 ``Executor`` 执行所在的设备。
+- **config(dict)** -  量化配置表。
+- **scope(fluid.Scope, optional)** -  传入用于存储 ``Variable`` 的 ``scope`` ，需要传入 ``program`` 所使用的 ``scope`` ，一般情况下，是 `fluid.global_scope() <https://www.paddlepaddle.org.cn/documentation/docs/zh/develop/api_cn/executor_cn/global_scope_cn.html>`_ 。设置为 ``None`` 时将使用 `fluid.global_scope() <https://www.paddlepaddle.org.cn/documentation/docs/zh/develop/api_cn/executor_cn/global_scope_cn.html>`_ ，默认值为 ``None`` 。
+- **for_test(bool)** -  如果 ``program`` 参数是一个测试 ``program`` ， ``for_test`` 应设为 ``True`` ，否则设为 ``False`` 。
+
+**返回**
+
+含有量化和反量化 ``operator`` 的 ``program`` 。
+
+**返回类型**
+
+- 当 ``for_test=False`` ，返回类型为 ``fluid.CompiledProgram`` ， **注意，此返回值不能用于保存参数** 。
+- 当 ``for_test=True`` ，返回类型为 ``fluid.Program`` 。
+
+.. note::
+
+   - 此接口会改变 ``program`` 结构，并且可能增加一些``persistable``的变量，所以加载模型参数时请注意和相应的``program``对应。
+   - 此接口底层经历了``fluid.Program``-> ``fluid.framework.IrGraph``->``fluid.Program``的转变，在``fluid.framework.IrGraph``中没有``Parameter``的概念，``Variable``只有``persistable``和``not persistable``的区别，所以在保存和加载参数时，请使用``fluid.io.save_persistables``和``fluid.io.load_persistables``接口。
+   - 由于此接口会根据``program``的结构和量化配置来对``program``添加op，所以``Paddle``中一些通过``fuse op``来加速训练的策略不能使用。已知以下策略在使用量化时必须设为``False``： ``fuse_all_reduce_ops, sync_batch_norm``。
+   - 如果传入的 ``program`` 中存在和任何op都没有连接的 ``Variable`` ，则会在量化的过程中被优化掉。
+
+
+
+convert
+---------
+
+.. py:function:: paddleslim.quant.convert(program, place, config, scope=None, save_int8=False)
+
+`源代码 <https://github.com/PaddlePaddle/PaddleSlim/blob/develop/paddleslim/quant/quanter.py>`_
+
+
+把训练好的量化 ``program`` ，转换为可用于保存 ``inference model`` 的 ``program`` 。
+
+**参数：**
+
+- **program (fluid.Program)** -  传入测试``program``。
+- **place(fluid.CPUPlace | fluid.CUDAPlace)** - 该参数表示``Executor``执行所在的设备。
+- **config(dict)** -  量化配置表。
+- **scope(fluid.Scope)** - 传入用于存储``Variable``的``scope``，需要传入``program``所使用的``scope``，一般情况下，是 `fluid.global_scope() <https://www.paddlepaddle.org.cn/documentation/docs/zh/develop/api_cn/executor_cn/global_scope_cn.html>`_ 。设置为 ``None`` 时将使用 `fluid.global_scope() <https://www.paddlepaddle.org.cn/documentation/docs/zh/develop/api_cn/executor_cn/global_scope_cn.html>`_ ，默认值为 ``None`` 。
+- **save_int8（bool)** -  是否需要返回参数为 ``int8`` 的 ``program`` 。该功能目前只能用于确认模型大小。默认值为 ``False`` 。
+
+**返回**
+
+- **program (fluid.Program)** - freezed program，可用于保存inference model，参数为``float32``类型，但其数值范围可用int8表示。
+- **int8_program (fluid.Program)** - freezed program，可用于保存inference model，参数为``int8``类型。当``save_int8``为``False``时，不返回该值。
+
+.. note::
+
+   因为该接口会对``op``和``Variable``做相应的删除和修改，所以此接口只能在训练完成之后调用。如果想转化训练的中间模型，可加载相应的参数之后再使用此接口。
+
+**代码示例**
+
+.. code-block:: python
+
+   #encoding=utf8
+   import paddle.fluid as fluid
+   import paddleslim.quant as quant
+   
+   
+   train_program = fluid.Program()
+   
+   with fluid.program_guard(train_program):
+       image = fluid.data(name='x', shape=[None, 1, 28, 28])
+       label = fluid.data(name='label', shape=[None, 1], dtype='int64')
+       conv = fluid.layers.conv2d(image, 32, 1)
+       feat = fluid.layers.fc(conv, 10, act='softmax')
+       cost = fluid.layers.cross_entropy(input=feat, label=label)
+       avg_cost = fluid.layers.mean(x=cost)
+   
+   use_gpu = True
+   place = fluid.CUDAPlace(0) if use_gpu else fluid.CPUPlace()
+   exe = fluid.Executor(place)
+   exe.run(fluid.default_startup_program())
+   eval_program = train_program.clone(for_test=True)
+   #配置
+   config = {'weight_quantize_type': 'abs_max',
+           'activation_quantize_type': 'moving_average_abs_max'}
+   build_strategy = fluid.BuildStrategy()
+   exec_strategy = fluid.ExecutionStrategy()
+   #调用api
+   quant_train_program = quant.quant_aware(train_program, place, config, for_test=False)
+   quant_eval_program = quant.quant_aware(eval_program, place, config, for_test=True)
+   #关闭策略
+   build_strategy.fuse_all_reduce_ops = False
+   build_strategy.sync_batch_norm = False
+   quant_train_program = quant_train_program.with_data_parallel(
+       loss_name=avg_cost.name,
+       build_strategy=build_strategy,
+       exec_strategy=exec_strategy)
+   
+   inference_prog = quant.convert(quant_eval_program, place, config)
+
+更详细的用法请参考 `量化训练demo <https://github.com/PaddlePaddle/PaddleSlim/tree/develop/demo/quant/quant_aware>`_ 。
+
+quant_post
+---------------
+
+.. py:function:: paddleslim.quant.quant_post(executor, model_dir, quantize_model_path,sample_generator, model_filename=None, params_filename=None, batch_size=16,batch_nums=None, scope=None, algo='KL', quantizable_op_type=["conv2d", "depthwise_conv2d", "mul"], is_full_quantize=False, weight_bits=8, activation_bits=8, is_use_cache_file=False, cache_dir="./temp_post_training")
+
+`源代码 <https://github.com/PaddlePaddle/PaddleSlim/blob/develop/paddleslim/quant/quanter.py>`_
+
+对保存在 ``${model_dir}`` 下的模型进行量化，使用 ``sample_generator`` 的数据进行参数校正。
+
+**参数:**
+
+- **executor (fluid.Executor)** - 执行模型的executor，可以在cpu或者gpu上执行。
+- **model_dir（str)** - 需要量化的模型所在的文件夹。
+- **quantize_model_path(str)** - 保存量化后的模型的路径
+- **sample_generator(python generator)** - 读取数据样本，每次返回一个样本。
+- **model_filename(str, optional)** - 模型文件名，如果需要量化的模型的参数存在一个文件中，则需要设置``model_filename``为模型文件的名称，否则设置为``None``即可。默认值是``None``。
+- **params_filename(str)** - 参数文件名，如果需要量化的模型的参数存在一个文件中，则需要设置``params_filename``为参数文件的名称，否则设置为``None``即可。默认值是``None``。
+- **batch_size(int)** - 每个batch的图片数量。默认值为16 。
+- **batch_nums(int, optional)** - 迭代次数。如果设置为``None``，则会一直运行到``sample_generator`` 迭代结束， 否则，迭代次数为``batch_nums``, 也就是说参与对 ``Scale`` 进行校正的样本个数为 ``'batch_nums' * 'batch_size'`` .
+- **scope(fluid.Scope, optional)** - 用来获取和写入 ``Variable`` , 如果设置为 ``None`` ,则使用 `fluid.global_scope() <https://www.paddlepaddle.org.cn/documentation/docs/zh/develop/api_cn/executor_cn/global_scope_cn.html>`_ . 默认值是``None``.
+- **algo(str)** - 量化时使用的算法名称，可为 ``'KL'`` 或者 ``'direct'`` 。该参数仅针对激活值的量化，因为参数值的量化使用的方式为 ``'channel_wise_abs_max'`` . 当 ``algo`` 设置为 ``'direct'`` 时，使用校正数据的激活值的绝对值的最大值当作 ``Scale`` 值，当设置为 ``'KL'`` 时，则使用KL散度的方法来计算 ``Scale`` 值。默认值为 ``'KL'`` 。
+- **quantizable_op_type(list[str])** -  需要量化的 ``op`` 类型列表。默认值为 ``["conv2d", "depthwise_conv2d", "mul"]`` 。
+- **is_full_quantize(bool)** - 是否量化所有可支持的op类型。如果设置为False, 则按照 ``'quantizable_op_type'`` 的设置进行量化。
+- **weight_bits(int)** - weight的量化比特位数。
+- **activation_bits(int)** - 激活值的量化比特位数。
+- **is_use_cache_file(bool)** - 是否使用硬盘对中间结果进行存储。如果为False, 则将中间结果存储在内存中。
+- **cache_dir(str)** - 如果 ``'is_use_cache_file'`` 为True, 则将中间结果存储在此参数设置的路径下。
+
+**返回**
+
+无。
+
+.. note::
+
+   - 因为该接口会收集校正数据的所有的激活值，当校正图片比较多时，请设置``'is_use_cache_file'``为True, 将中间结果存储在硬盘中。另外，``'KL'``散度的计算比较耗时。
+   - 目前``Paddle-Lite``有int8 kernel来加速的op只有 ``['conv2d', 'depthwise_conv2d', 'mul']``, 其他op的int8 kernel将陆续支持。
+
+**代码示例**
+
+.. warning::
+
+   此示例不能直接运行，因为需要加载``${model_dir}``下的模型，所以不能直接运行。
+
+.. code-block:: python
+
+   import paddle.fluid as fluid
+   import paddle.dataset.mnist as reader
+   from paddleslim.quant import quant_post
+   val_reader = reader.train()
+   use_gpu = True
+   place = fluid.CUDAPlace(0) if use_gpu else fluid.CPUPlace()
+   
+   exe = fluid.Executor(place)
+   quant_post(
+           executor=exe,
+           model_dir='./model_path',
+           quantize_model_path='./save_path',
+           sample_generator=val_reader,
+           model_filename='__model__',
+           params_filename='__params__',
+           batch_size=16,
+           batch_nums=10)
+
+
+更详细的用法请参考 `离线量化demo <https://github.com/PaddlePaddle/PaddleSlim/tree/develop/demo/quant/quant_post>`_ 。
+
+quant_embedding
+-------------------
+
+.. py:function:: paddleslim.quant.quant_embedding(program, place, config, scope=None)
+
+`源代码 <https://github.com/PaddlePaddle/PaddleSlim/blob/develop/paddleslim/quant/quant_embedding.py>`_
+
+对 ``Embedding`` 参数进行量化。
+
+**参数:**
+
+- **program(fluid.Program)** - 需要量化的program
+- **scope(fluid.Scope, optional)** - 用来获取和写入``Variable``, 如果设置为``None``,则使用 `fluid.global_scope() <https://www.paddlepaddle.org.cn/documentation/docs/zh/develop/api_cn/executor_cn/global_scope_cn.html>`_ .
+- **place(fluid.CPUPlace | fluid.CUDAPlace)** - 运行program的设备
+- **config(dict)** - 定义量化的配置。可以配置的参数有：
+    - ``'params_name'`` (str, required): 需要进行量化的参数名称，此参数必须设置。
+    - ``'quantize_type'`` (str, optional): 量化的类型，目前支持的类型是 ``'abs_max'``, 待支持的类型有 ``'log', 'product_quantization'`` 。 默认值是``'abs_max'`` .
+    - ``'quantize_bits'`` （int, optional): 量化的 ``bit`` 数，目前支持的 ``bit`` 数为8。默认值是8.
+    - ``'dtype'`` (str, optional): 量化之后的数据类型， 目前支持的是 ``'int8'``. 默认值是 ``int8`` 。
+    - ``'threshold'`` (float, optional): 量化之前将根据此阈值对需要量化的参数值进行 ``clip``. 如果不设置，则跳过 ``clip`` 过程直接量化。
+
+**返回**
+
+量化之后的program
+
+**返回类型**
+
+fluid.Program
+
+**代码示例**
+
+.. code-block:: python
+
+   import paddle.fluid as fluid
+   import paddleslim.quant as quant
+   
+   train_program = fluid.Program()
+   with fluid.program_guard(train_program):
+       input_word = fluid.data(name="input_word", shape=[None, 1], dtype='int64')
+       input_emb = fluid.embedding(
+           input=input_word,
+           is_sparse=False,
+           size=[100, 128],
+           param_attr=fluid.ParamAttr(name='emb',
+           initializer=fluid.initializer.Uniform(-0.005, 0.005)))
+   
+   infer_program = train_program.clone(for_test=True)
+   
+   use_gpu = True
+   place = fluid.CUDAPlace(0) if use_gpu else fluid.CPUPlace()
+   exe = fluid.Executor(place)
+   exe.run(fluid.default_startup_program())
+   
+   config = {'params_name': 'emb', 'quantize_type': 'abs_max'}
+   quant_program = quant.quant_embedding(infer_program, place, config)
+
+更详细的用法请参考 `Embedding量化demo <https://github.com/PaddlePaddle/PaddleSlim/tree/develop/demo/quant/quant_embedding'>`_ 

_sources/api_cn/search_space.rst.txt

+搜索空间
+=========
+搜索空间是神经网络搜索中的一个概念。搜索空间是一系列模型结构的汇集, SANAS主要是利用模拟退火的思想在搜索空间中搜索到一个比较小的模型结构或者一个精度比较高的模型结构。
+
+paddleslim.nas 提供的搜索空间
+--------
+
+根据初始模型结构构造搜索空间:
+
+1. MobileNetV2Space<br>
+&emsp; MobileNetV2的网络结构可以参考：[代码](https://github.com/PaddlePaddle/models/blob/develop/PaddleCV/image_classification/models/mobilenet_v2.py#L29)，[论文](https://arxiv.org/abs/1801.04381)
+
+2. MobileNetV1Space<br>
+&emsp; MobilNetV1的网络结构可以参考：[代码](https://github.com/PaddlePaddle/models/blob/develop/PaddleCV/image_classification/models/mobilenet_v1.py#L29)，[论文](https://arxiv.org/abs/1704.04861)
+
+3. ResNetSpace<br>
+&emsp; ResNetSpace的网络结构可以参考：[代码](https://github.com/PaddlePaddle/models/blob/develop/PaddleCV/image_classification/models/resnet.py#L30)，[论文](https://arxiv.org/pdf/1512.03385.pdf)
+
+
+根据相应模型的block构造搜索空间:
+1. MobileNetV1BlockSpace<br>
+&emsp; MobileNetV1Block的结构可以参考：[代码](https://github.com/PaddlePaddle/models/blob/develop/PaddleCV/image_classification/models/mobilenet_v1.py#L173)
+
+2. MobileNetV2BlockSpace<br>
+&emsp; MobileNetV2Block的结构可以参考：[代码](https://github.com/PaddlePaddle/models/blob/develop/PaddleCV/image_classification/models/mobilenet_v2.py#L174)
+
+3. ResNetBlockSpace<br>
+&emsp; ResNetBlock的结构可以参考：[代码](https://github.com/PaddlePaddle/models/blob/develop/PaddleCV/image_classification/models/resnet.py#L148)
+
+4. InceptionABlockSpace<br>
+&emsp; InceptionABlock的结构可以参考：[代码](https://github.com/PaddlePaddle/models/blob/develop/PaddleCV/image_classification/models/inception_v4.py#L140)
+
+5. InceptionCBlockSpace<br>
+&emsp; InceptionCBlock结构可以参考：[代码](https://github.com/PaddlePaddle/models/blob/develop/PaddleCV/image_classification/models/inception_v4.py#L291)
+
+
+搜索空间使用示例
+--------
+
+1. 使用paddleslim中提供用初始的模型结构来构造搜索空间的话，仅需要指定搜索空间名字即可。例如：如果使用原本的MobileNetV2的搜索空间进行搜索的话，传入SANAS中的configs直接指定为[('MobileNetV2Space')]。
+2. 使用paddleslim中提供的block搜索空间构造搜索空间：<br>
+  2.1 使用`input_size`, `output_size`和`block_num`来构造搜索空间。例如：传入SANAS的configs可以指定为[('MobileNetV2BlockSpace', {'input_size': 224, 'output_size': 32, 'block_num': 10})]。<br>
+  2.2 使用`block_mask`构造搜索空间。例如：传入SANAS的configs可以指定为[('MobileNetV2BlockSpace', {'block_mask': [0, 1, 1, 1, 1, 0, 1, 0]})]。
+
+
+自定义搜索空间(search space)
+--------
+
+自定义搜索空间类需要继承搜索空间基类并重写以下几部分：<br>
+&emsp; 1. 初始化的tokens(`init_tokens`函数)，可以设置为自己想要的tokens列表, tokens列表中的每个数字指的是当前数字在相应的搜索列表中的索引。例如本示例中若tokens=[0, 3, 5]，则代表当前模型结构搜索到的通道数为[8, 40, 128]。<br>
+&emsp; 2. tokens中每个数字的搜索列表长度(`range_table`函数)，tokens中每个token的索引范围。<br>
+&emsp; 3. 根据tokens产生模型结构(`token2arch`函数)，根据搜索到的tokens列表产生模型结构。 <br>
+
+以新增reset block为例说明如何构造自己的search space。自定义的search space不能和已有的search space同名。
+
+```python
+### 引入搜索空间基类函数和search space的注册类函数
+from .search_space_base import SearchSpaceBase
+from .search_space_registry import SEARCHSPACE
+import numpy as np
+
+### 需要调用注册函数把自定义搜索空间注册到space space中
+@SEARCHSPACE.register
+### 定义一个继承SearchSpaceBase基类的搜索空间的类函数
+class ResNetBlockSpace2(SearchSpaceBase):
+    def __init__(self, input_size, output_size, block_num, block_mask):
+        ### 定义一些实际想要搜索的内容，例如：通道数、每个卷积的重复次数、卷积核大小等等
+        ### self.filter_num 代表通道数的搜索列表
+        self.filter_num = np.array([8, 16, 32, 40, 64, 128, 256, 512])
+
+    ### 定义初始化token，初始化token的长度根据传入的block_num或者block_mask的长度来得到的
+    def init_tokens(self):
+        return [0] * 3 * len(self.block_mask)
+
+    ### 定义token的index的取值范围
+    def range_table(self):
+        return [len(self.filter_num)] * 3 * len(self.block_mask)
+
+    ### 把token转换成模型结构
+    def token2arch(self, tokens=None):
+        if tokens == None:
+            tokens = self.init_tokens()
+
+        self.bottleneck_params_list = []
+        for i in range(len(self.block_mask)):
+            self.bottleneck_params_list.append(self.filter_num[tokens[i * 3 + 0]],
+                                               self.filter_num[tokens[i * 3 + 1]],
+                                               self.filter_num[tokens[i * 3 + 2]],
+                                               2 if self.block_mask[i] == 1 else 1)
+
+        def net_arch(input):
+            for i, layer_setting in enumerate(self.bottleneck_params_list):
+                channel_num, stride = layer_setting[:-1], layer_setting[-1]
+                input = self._resnet_block(input, channel_num, stride, name='resnet_layer{}'.format(i+1))
+
+            return input
+
+        return net_arch
+
+    ### 构造具体block的操作
+    def _resnet_block(self, input, channel_num, stride, name=None):
+        shortcut_conv = self._shortcut(input, channel_num[2], stride, name=name)
+        input = self._conv_bn_layer(input=input, num_filters=channel_num[0], filter_size=1, act='relu', name=name + '_conv0')
+        input = self._conv_bn_layer(input=input, num_filters=channel_num[1], filter_size=3, stride=stride, act='relu', name=name + '_conv1')
+        input = self._conv_bn_layer(input=input, num_filters=channel_num[2], filter_size=1, name=name + '_conv2')
+        return fluid.layers.elementwise_add(x=shortcut_conv, y=input, axis=0, name=name+'_elementwise_add')
+
+    def _shortcut(self, input, channel_num, stride, name=None):
+        channel_in = input.shape[1]
+        if channel_in != channel_num or stride != 1:
+            return self.conv_bn_layer(input, num_filters=channel_num, filter_size=1, stride=stride, name=name+'_shortcut')
+        else:
+            return input
+
+    def _conv_bn_layer(self, input, num_filters, filter_size, stride=1, padding='SAME', act=None, name=None):
+        conv = fluid.layers.conv2d(input, num_filters, filter_size, stride, name=name+'_conv')
+        bn = fluid.layers.batch_norm(conv, act=act, name=name+'_bn')
+        return bn
+```

_sources/api_cn/single_distiller_api.rst.txt

+简单蒸馏
+=========
+
+merge
+---------
+
+.. py:function:: paddleslim.dist.merge(teacher_program, student_program, data_name_map, place, scope=fluid.global_scope(), name_prefix='teacher_')
+
+`源代码 <https://github.com/PaddlePaddle/PaddleSlim/blob/develop/paddleslim/dist/single_distiller.py#L19>`_
+
+merge将teacher_program融合到student_program中。在融合的program中，可以为其中合适的teacher特征图和student特征图添加蒸馏损失函数，从而达到用teacher模型的暗知识（Dark Knowledge）指导student模型学习的目的。
+
+**参数：**
+
+- **teacher_program** (Program)-定义了teacher模型的 `paddle program <https://www.paddlepaddle.org.cn/documentation/docs/zh/api_cn/fluid_cn/Program_cn.html#program>`_
+- **student_program** (Program)-定义了student模型的 `paddle program <https://www.paddlepaddle.org.cn/documentation/docs/zh/api_cn/fluid_cn/Program_cn.html#program>`_
+- **data_name_map** (dict)-teacher输入接口名与student输入接口名的映射，其中dict的 *key* 为teacher的输入名，*value* 为student的输入名
+- **place** (fluid.CPUPlace()|fluid.CUDAPlace(N))-该参数表示程序运行在何种设备上，这里的N为GPU对应的ID
+- **scope** (Scope)-该参数表示程序使用的变量作用域，如果不指定将使用默认的全局作用域。默认值： `fluid.global_scope() <https://www.paddlepaddle.org.cn/documentation/docs/zh/api_cn/fluid_cn/global_scope_cn.html#global-scope>`_
+- **name_prefix** (str)-merge操作将统一为teacher的 `Variables <https://www.paddlepaddle.org.cn/documentation/docs/zh/1.3/api_guides/low_level/program.html#variable>`_ 添加的名称前缀name_prefix。默认值：'teacher_'
+
+**返回：** 无
+
+.. note::
+
+    *data_name_map* 是 **teacher_var name到student_var name的映射** ，如果写反可能无法正确进行merge
+
+
+**使用示例：**
+
+.. code-block:: python
+
+   import paddle.fluid as fluid
+   import paddleslim.dist as dist
+   student_program = fluid.Program()
+   with fluid.program_guard(student_program):
+       x = fluid.layers.data(name='x', shape=[1, 28, 28])
+       conv = fluid.layers.conv2d(x, 32, 1)
+       out = fluid.layers.conv2d(conv, 64, 3, padding=1)
+   teacher_program = fluid.Program()
+   with fluid.program_guard(teacher_program):
+       y = fluid.layers.data(name='y', shape=[1, 28, 28])
+       conv = fluid.layers.conv2d(y, 32, 1)
+       conv = fluid.layers.conv2d(conv, 32, 3, padding=1)
+       out = fluid.layers.conv2d(conv, 64, 3, padding=1)
+   data_name_map = {'y':'x'}
+   USE_GPU = False
+   place = fluid.CUDAPlace(0) if USE_GPU else fluid.CPUPlace()
+   dist.merge(teacher_program, student_program,
+                             data_name_map, place)
+
+
+fsp_loss
+---------
+
+.. py:function:: paddleslim.dist.fsp_loss(teacher_var1_name, teacher_var2_name, student_var1_name, student_var2_name, program=fluid.default_main_program())
+
+`源代码 <https://github.com/PaddlePaddle/PaddleSlim/blob/develop/paddleslim/dist/single_distiller.py#L90>`_
+
+fsp_loss为program内的teacher var和student var添加fsp loss，出自论文 `A Gift from Knowledge Distillation: Fast Optimization, Network Minimization and Transfer Learning <http://openaccess.thecvf.com/content_cvpr_2017/papers/Yim_A_Gift_From_CVPR_2017_paper.pdf>`_
+
+**参数：**
+
+- **teacher_var1_name** (str): teacher_var1的名称. 对应的variable是一个形为`[batch_size, x_channel, height, width]`的4-D特征图Tensor，数据类型为float32或float64
+- **teacher_var2_name** (str): teacher_var2的名称. 对应的variable是一个形为`[batch_size, y_channel, height, width]`的4-D特征图Tensor，数据类型为float32或float64。只有y_channel可以与teacher_var1的x_channel不同，其他维度必须与teacher_var1相同
+- **student_var1_name** (str): student_var1的名称. 对应的variable需与teacher_var1尺寸保持一致，是一个形为`[batch_size, x_channel, height, width]`的4-D特征图Tensor，数据类型为float32或float64
+- **student_var2_name** (str): student_var2的名称. 对应的variable需与teacher_var2尺寸保持一致，是一个形为`[batch_size, y_channel, height, width]`的4-D特征图Tensor，数据类型为float32或float64。只有y_channel可以与student_var1的x_channel不同，其他维度必须与student_var1相同
+- **program** (Program): 用于蒸馏训练的fluid program。默认值： `fluid.default_main_program() <https://www.paddlepaddle.org.cn/documentation/docs/zh/1.3/api_cn/fluid_cn.html#default-main-program>`_
+
+**返回：** 由teacher_var1, teacher_var2, student_var1, student_var2组合得到的fsp_loss
+
+**使用示例：**
+
+.. code-block:: python
+   import paddle.fluid as fluid
+   import paddleslim.dist as dist
+   student_program = fluid.Program()
+   with fluid.program_guard(student_program):
+       x = fluid.layers.data(name='x', shape=[1, 28, 28])
+       conv = fluid.layers.conv2d(x, 32, 1, name='s1')
+       out = fluid.layers.conv2d(conv, 64, 3, padding=1, name='s2')
+   teacher_program = fluid.Program()
+   with fluid.program_guard(teacher_program):
+       y = fluid.layers.data(name='y', shape=[1, 28, 28])
+       conv = fluid.layers.conv2d(y, 32, 1, name='t1')
+       conv = fluid.layers.conv2d(conv, 32, 3, padding=1)
+       out = fluid.layers.conv2d(conv, 64, 3, padding=1, name='t2')
+   data_name_map = {'y':'x'}
+   USE_GPU = False
+   place = fluid.CUDAPlace(0) if USE_GPU else fluid.CPUPlace()
+   dist.merge(teacher_program, student_program, data_name_map, place)
+   with fluid.program_guard(student_program):
+       distillation_loss = dist.fsp_loss('teacher_t1.tmp_1', 'teacher_t2.tmp_1',
+                                         's1.tmp_1', 's2.tmp_1', student_program)
+   
+
+
+l2_loss
+------------
+
+.. py:function:: paddleslim.dist.l2_loss(teacher_var_name, student_var_name, program=fluid.default_main_program())[[源代码]](https://github.com/PaddlePaddle/PaddleSlim/blob/develop/paddleslim/dist/single_distiller.py#L118)
+
+: l2_loss为program内的teacher var和student var添加l2 loss
+
+**参数：**
+
+- **teacher_var_name** (str): teacher_var的名称.
+- **student_var_name** (str): student_var的名称.
+- **program** (Program): 用于蒸馏训练的fluid program。默认值： `fluid.default_main_program() <https://www.paddlepaddle.org.cn/documentation/docs/zh/1.3/api_cn/fluid_cn.html#default-main-program>`_
+
+**返回：** 由teacher_var, student_var组合得到的l2_loss
+
+**使用示例：**
+
+.. code-block:: python
+   import paddle.fluid as fluid
+   import paddleslim.dist as dist
+   student_program = fluid.Program()
+   with fluid.program_guard(student_program):
+       x = fluid.layers.data(name='x', shape=[1, 28, 28])
+       conv = fluid.layers.conv2d(x, 32, 1, name='s1')
+       out = fluid.layers.conv2d(conv, 64, 3, padding=1, name='s2')
+   teacher_program = fluid.Program()
+   with fluid.program_guard(teacher_program):
+       y = fluid.layers.data(name='y', shape=[1, 28, 28])
+       conv = fluid.layers.conv2d(y, 32, 1, name='t1')
+       conv = fluid.layers.conv2d(conv, 32, 3, padding=1)
+       out = fluid.layers.conv2d(conv, 64, 3, padding=1, name='t2')
+   data_name_map = {'y':'x'}
+   USE_GPU = False
+   place = fluid.CUDAPlace(0) if USE_GPU else fluid.CPUPlace()
+   dist.merge(teacher_program, student_program, data_name_map, place)
+   with fluid.program_guard(student_program):
+       distillation_loss = dist.l2_loss('teacher_t2.tmp_1', 's2.tmp_1',
+                                        student_program)
+
+
+
+soft_label_loss
+-------------------
+
+.. py:function:: paddleslim.dist.soft_label_loss(teacher_var_name, student_var_name, program=fluid.default_main_program(), teacher_temperature=1., student_temperature=1.)[[源代码]](https://github.com/PaddlePaddle/PaddleSlim/blob/develop/paddleslim/dist/single_distiller.py#L136)
+
+soft_label_loss为program内的teacher var和student var添加soft label loss，出自论文 `Distilling the Knowledge in a Neural Network <https://arxiv.org/pdf/1503.02531.pdf>`_
+
+**参数：**
+
+- **teacher_var_name** (str): teacher_var的名称.
+- **student_var_name** (str): student_var的名称.
+- **program** (Program): 用于蒸馏训练的fluid program。默认值： `fluid.default_main_program() <https://www.paddlepaddle.org.cn/documentation/docs/zh/1.3/api_cn/fluid_cn.html#default-main-program>`_
+- **teacher_temperature** (float): 对teacher_var进行soft操作的温度值，温度值越大得到的特征图越平滑
+- **student_temperature** (float): 对student_var进行soft操作的温度值，温度值越大得到的特征图越平滑
+
+**返回：** 由teacher_var, student_var组合得到的soft_label_loss
+
+**使用示例：**
+
+.. code-block:: python
+   import paddle.fluid as fluid
+   import paddleslim.dist as dist
+   student_program = fluid.Program()
+   with fluid.program_guard(student_program):
+       x = fluid.layers.data(name='x', shape=[1, 28, 28])
+       conv = fluid.layers.conv2d(x, 32, 1, name='s1')
+       out = fluid.layers.conv2d(conv, 64, 3, padding=1, name='s2')
+   teacher_program = fluid.Program()
+   with fluid.program_guard(teacher_program):
+       y = fluid.layers.data(name='y', shape=[1, 28, 28])
+       conv = fluid.layers.conv2d(y, 32, 1, name='t1')
+       conv = fluid.layers.conv2d(conv, 32, 3, padding=1)
+       out = fluid.layers.conv2d(conv, 64, 3, padding=1, name='t2')
+   data_name_map = {'y':'x'}
+   USE_GPU = False
+   place = fluid.CUDAPlace(0) if USE_GPU else fluid.CPUPlace()
+   dist.merge(teacher_program, student_program, data_name_map, place)
+   with fluid.program_guard(student_program):
+       distillation_loss = dist.soft_label_loss('teacher_t2.tmp_1',
+                                                's2.tmp_1', student_program, 1., 1.)
+
+
+
+loss
+--------
+
+.. py:function:: paddleslim.dist.loss(loss_func, program=fluid.default_main_program(), **kwargs) [[源代码]](https://github.com/PaddlePaddle/PaddleSlim/blob/develop/paddleslim/dist/single_distiller.py#L165)
+
+: loss函数支持对任意多对teacher_var和student_var使用自定义损失函数
+
+**参数：**
+
+- **loss_func**( python function): 自定义的损失函数，输入为teacher var和student var，输出为自定义的loss
+- **program** (Program): 用于蒸馏训练的fluid program。默认值： `fluid.default_main_program() <https://www.paddlepaddle.org.cn/documentation/docs/zh/1.3/api_cn/fluid_cn.html#default-main-program>`_
+- **\**kwargs** : loss_func输入名与对应variable名称
+
+**返回** ：自定义的损失函数loss
+
+**使用示例：**
+
+.. code-block:: python
+   import paddle.fluid as fluid
+   import paddleslim.dist as dist
+   student_program = fluid.Program()
+   with fluid.program_guard(student_program):
+       x = fluid.layers.data(name='x', shape=[1, 28, 28])
+       conv = fluid.layers.conv2d(x, 32, 1, name='s1')
+       out = fluid.layers.conv2d(conv, 64, 3, padding=1, name='s2')
+   teacher_program = fluid.Program()
+   with fluid.program_guard(teacher_program):
+       y = fluid.layers.data(name='y', shape=[1, 28, 28])
+       conv = fluid.layers.conv2d(y, 32, 1, name='t1')
+       conv = fluid.layers.conv2d(conv, 32, 3, padding=1)
+       out = fluid.layers.conv2d(conv, 64, 3, padding=1, name='t2')
+   data_name_map = {'y':'x'}
+   USE_GPU = False
+   place = fluid.CUDAPlace(0) if USE_GPU else fluid.CPUPlace()
+   dist.merge(teacher_program, student_program, data_name_map, place)
+   def adaptation_loss(t_var, s_var):
+       teacher_channel = t_var.shape[1]
+       s_hint = fluid.layers.conv2d(s_var, teacher_channel, 1)
+       hint_loss = fluid.layers.reduce_mean(fluid.layers.square(s_hint - t_var))
+       return hint_loss
+   with fluid.program_guard(student_program):
+       distillation_loss = dist.loss(adaptation_loss, student_program,
+               t_var='teacher_t2.tmp_1', s_var='s2.tmp_1')
+
+.. note::
+
+    在添加蒸馏loss时会引入新的variable，需要注意新引入的variable不要与student variables命名冲突。这里建议两种用法（两种方法任选其一即可）：
+
+    1. 建议与student_program使用同一个命名空间，以避免一些未指定名称的variables(例如tmp_0, tmp_1...)多次定义为同一名称出现命名冲突
+
+    2. 建议在添加蒸馏loss时指定一个命名空间前缀，具体用法请参考Paddle官方文档 `fluid.name_scope <https://www.paddlepaddle.org.cn/documentation/docs/zh/api_cn/fluid_cn/name_scope_cn.html#name-scope>`_

_sources/api_cn/table_latency.md.txt

+# 硬件延时评估表
+
+硬件延时评估表用于快速评估一个模型在特定硬件环境和推理引擎上的推理速度。
+该文档主要用于定义PaddleSlim支持的硬件延时评估表的格式。
+
+## 概述
+
+硬件延时评估表中存放着所有可能的操作对应的延时信息，该表中的一个操作包括操作类型和操作参数，比如：操作类型可以是`conv2d`，对应的操作参数有输入特征图的大小、卷积核个数、卷积核大小等。
+给定操作的延时依赖于硬件环境和推理引擎。
+
+## 整体格式
+
+硬件延时评估表以文件或多行字符串的形式保存。
+
+硬件延时评估表第一行保存版本信息，后续每行为一个操作和对应的延时信息。
+
+## 版本信息
+
+版本信息以英文字符逗号分割，内容依次为硬件环境名称、推理引擎名称和时间戳。
+
+- **硬件环境名称：** 用于标识硬件环境，可以包含计算架构类型、版本号等信息。
+
+- **推理引擎名称：** 用于标识推理引擎，可以包含推理引擎名称、版本号、优化选项等信息。
+
+- **时间戳：** 该评估表的创建时间。
+
+## 操作信息
+
+操作信息字段之间以逗号分割。操作信息与延迟信息之间以制表符分割。
+
+### conv2d
+
+**格式**
+
+```text
+op_type,flag_bias,flag_relu,n_in,c_in,h_in,w_in,c_out,groups,kernel,padding,stride,dilation\tlatency
+```
+
+**字段解释**
+
+- **op_type(str)** - 当前op类型。
+- **flag_bias (int)** - 是否有 bias（0：无，1：有）。
+- **flag_relu (int)** - 是否有 relu（0：无，1：有）。
+- **n_in (int)** - 输入 Tensor 的批尺寸 (batch size)。
+- **c_in (int)** - 输入 Tensor 的通道 (channel) 数。
+- **h_in (int)** - 输入 Tensor 的特征高度。
+- **w_in (int)** - 输入 Tensor 的特征宽度。
+- **c_out (int)** - 输出 Tensor 的通道 (channel) 数。
+- **groups (int)** - 卷积二维层（Conv2D Layer）的组数。
+- **kernel (int)** - 卷积核大小。
+- **padding (int)** - 填充 (padding) 大小。
+- **stride (int)** - 步长 (stride) 大小。
+- **dilation (int)** - 膨胀 (dilation) 大小。
+- **latency (float)** - 当前op的延时时间
+
+### activation
+
+**格式**
+
+```text
+op_type,n_in,c_in,h_in,w_in\tlatency
+```
+
+**字段解释**
+
+- **op_type(str)** - 当前op类型。
+- **n_in (int)** - 输入 Tensor 的批尺寸 (batch size)。
+- **c_in (int)** - 输入 Tensor 的通道 (channel) 数。
+- **h_in (int)** - 输入 Tensor 的特征高度。
+- **w_in (int)** - 输入 Tensor 的特征宽度。
+- **latency (float)** - 当前op的延时时间
+
+### batch_norm
+
+**格式**
+
+```text
+op_type,active_type,n_in,c_in,h_in,w_in\tlatency
+```
+
+**字段解释**
+
+- **op_type(str)** - 当前op类型。
+- **active_type (string|None)** - 激活函数类型，包含：relu, prelu, sigmoid, relu6, tanh。
+- **n_in (int)** - 输入 Tensor 的批尺寸 (batch size)。
+- **c_in (int)** - 输入 Tensor 的通道 (channel) 数。
+- **h_in (int)** - 输入 Tensor 的特征高度。
+- **w_in (int)** - 输入 Tensor 的特征宽度。
+- **latency (float)** - 当前op的延时时间
+
+### eltwise
+
+**格式**
+
+```text
+op_type,n_in,c_in,h_in,w_in\tlatency
+```
+
+**字段解释**
+
+- **op_type(str)** - 当前op类型。
+- **n_in (int)** - 输入 Tensor 的批尺寸 (batch size)。
+- **c_in (int)** - 输入 Tensor 的通道 (channel) 数。
+- **h_in (int)** - 输入 Tensor 的特征高度。
+- **w_in (int)** - 输入 Tensor 的特征宽度。
+- **latency (float)** - 当前op的延时时间
+
+### pooling
+
+**格式**
+
+```text
+op_type,flag_global_pooling,n_in,c_in,h_in,w_in,kernel,padding,stride,ceil_mode,pool_type\tlatency
+```
+
+**字段解释**
+
+- **op_type(str)** - 当前op类型。
+- **flag_global_pooling (int)** - 是否为全局池化（0：不是，1：是）。
+- **n_in (int)** - 输入 Tensor 的批尺寸 (batch size)。
+- **c_in (int)** - 输入 Tensor 的通道 (channel) 数。
+- **h_in (int)** - 输入 Tensor 的特征高度。
+- **w_in (int)** - 输入 Tensor 的特征宽度。
+- **kernel (int)** - 卷积核大小。
+- **padding (int)** - 填充 (padding) 大小。
+- **stride (int)** - 步长 (stride) 大小。
+- **ceil_mode (int)** - 是否用 ceil 函数计算输出高度和宽度。0 表示使用 floor 函数，1 表示使用 ceil 函数。
+- **pool_type (int)** - 池化类型，其中 1 表示 pooling_max，2 表示 pooling_average_include_padding，3 表示 pooling_average_exclude_padding。
+- **latency (float)** - 当前op的延时时间
+
+### softmax
+
+**格式**
+
+```text
+op_type,axis,n_in,c_in,h_in,w_in\tlatency
+```
+
+**字段解释**
+
+- **op_type(str)** - 当前op类型。
+- **axis (int)** - 执行 softmax 计算的维度索引，应该在 [−1，rank − 1] 范围内，其中 rank 是输入变量的秩。
+- **n_in (int)** - 输入 Tensor 的批尺寸 (batch size)。
+- **c_in (int)** - 输入 Tensor 的通道 (channel) 数。
+- **h_in (int)** - 输入 Tensor 的特征高度。
+- **w_in (int)** - 输入 Tensor 的特征宽度。
+- **latency (float)** - 当前op的延时时间

_sources/api_en/index_en.rst.txt

+.. PaddleSlim documentation master file, created by
+   sphinx-quickstart on Wed Feb  5 14:04:52 2020.
+   You can adapt this file completely to your liking, but it should at least
+   contain the root `toctree` directive.
+
+API Documents
+==============
+
+.. toctree::
+   :maxdepth: 1
+
+   paddleslim.analysis.rst
+   paddleslim.prune.rst
+   paddleslim.dist.rst
+   paddleslim.quant.rst
+   paddleslim.nas.rst
+   paddleslim.nas.one_shot.rst
+   paddleslim.pantheon.rst
+   search_space_en.rst

_sources/api_en/modules.rst.txt

+paddleslim
+==========
+
+.. toctree::
+   :maxdepth: 4
+
+   paddleslim

_sources/api_en/paddleslim.analysis.rst.txt

+paddleslim\.analysis package
+============================
+
+.. automodule:: paddleslim.analysis
+    :members:
+    :undoc-members:
+    :show-inheritance:
+
+Submodules
+----------
+
+paddleslim\.analysis\.flops module
+----------------------------------
+
+.. automodule:: paddleslim.analysis.flops
+    :members:
+    :undoc-members:
+    :show-inheritance:
+
+paddleslim\.analysis\.latency module
+------------------------------------
+
+.. automodule:: paddleslim.analysis.latency
+    :members:
+    :undoc-members:
+    :show-inheritance:
+
+paddleslim\.analysis\.model\_size module
+----------------------------------------
+
+.. automodule:: paddleslim.analysis.model_size
+    :members:
+    :undoc-members:
+    :show-inheritance:
+
+

_sources/api_en/paddleslim.common.rst.txt

+paddleslim\.common package
+==========================
+
+.. automodule:: paddleslim.common
+    :members:
+    :undoc-members:
+    :show-inheritance:
+
+Submodules
+----------
+
+paddleslim\.common\.cached\_reader module
+-----------------------------------------
+
+.. automodule:: paddleslim.common.cached_reader
+    :members:
+    :undoc-members:
+    :show-inheritance:
+
+paddleslim\.common\.controller module
+-------------------------------------
+
+.. automodule:: paddleslim.common.controller
+    :members:
+    :undoc-members:
+    :show-inheritance:
+
+paddleslim\.common\.controller\_client module
+---------------------------------------------
+
+.. automodule:: paddleslim.common.controller_client
+    :members:
+    :undoc-members:
+    :show-inheritance:
+
+paddleslim\.common\.controller\_server module
+---------------------------------------------
+
+.. automodule:: paddleslim.common.controller_server
+    :members:
+    :undoc-members:
+    :show-inheritance:
+
+paddleslim\.common\.lock module
+-------------------------------
+
+.. automodule:: paddleslim.common.lock
+    :members:
+    :undoc-members:
+    :show-inheritance:
+
+paddleslim\.common\.log\_helper module
+--------------------------------------
+
+.. automodule:: paddleslim.common.log_helper
+    :members:
+    :undoc-members:
+    :show-inheritance:
+
+paddleslim\.common\.sa\_controller module
+-----------------------------------------
+
+.. automodule:: paddleslim.common.sa_controller
+    :members:
+    :undoc-members:
+    :show-inheritance:
+
+

_sources/api_en/paddleslim.core.rst.txt

+paddleslim\.core package
+========================
+
+.. automodule:: paddleslim.core
+    :members:
+    :undoc-members:
+    :show-inheritance:
+
+Submodules
+----------
+
+paddleslim\.core\.graph\_wrapper module
+---------------------------------------
+
+.. automodule:: paddleslim.core.graph_wrapper
+    :members:
+    :undoc-members:
+    :show-inheritance:
+
+paddleslim\.core\.registry module
+---------------------------------
+
+.. automodule:: paddleslim.core.registry
+    :members:
+    :undoc-members:
+    :show-inheritance:
+
+

_sources/api_en/paddleslim.dist.rst.txt

+paddleslim\.dist package
+========================
+
+.. automodule:: paddleslim.dist
+    :members:
+    :undoc-members:
+    :show-inheritance:
+
+Submodules
+----------
+
+paddleslim\.dist\.single\_distiller module
+------------------------------------------
+
+.. automodule:: paddleslim.dist.single_distiller
+    :members:
+    :undoc-members:
+    :show-inheritance:
+
+

_sources/api_en/paddleslim.models.rst.txt

+paddleslim\.models package
+==========================
+
+.. automodule:: paddleslim.models
+    :members:
+    :undoc-members:
+    :show-inheritance:
+
+Submodules
+----------
+
+paddleslim\.models\.classification\_models module
+-------------------------------------------------
+
+.. automodule:: paddleslim.models.classification_models
+    :members:
+    :undoc-members:
+    :show-inheritance:
+
+paddleslim\.models\.mobilenet module
+------------------------------------
+
+.. automodule:: paddleslim.models.mobilenet
+    :members:
+    :undoc-members:
+    :show-inheritance:
+
+paddleslim\.models\.mobilenet\_v2 module
+----------------------------------------
+
+.. automodule:: paddleslim.models.mobilenet_v2
+    :members:
+    :undoc-members:
+    :show-inheritance:
+
+paddleslim\.models\.resnet module
+---------------------------------
+
+.. automodule:: paddleslim.models.resnet
+    :members:
+    :undoc-members:
+    :show-inheritance:
+
+paddleslim\.models\.util module
+-------------------------------
+
+.. automodule:: paddleslim.models.util
+    :members:
+    :undoc-members:
+    :show-inheritance:
+
+

_sources/api_en/paddleslim.nas.one_shot.rst.txt

+paddleslim\.nas\.one\_shot package
+==================================
+
+.. automodule:: paddleslim.nas.one_shot
+    :members:
+    :undoc-members:
+    :show-inheritance:
+
+Submodules
+----------
+
+paddleslim\.nas\.one\_shot\.one\_shot\_nas module
+-------------------------------------------------
+
+.. automodule:: paddleslim.nas.one_shot.one_shot_nas
+    :members:
+    :undoc-members:
+    :show-inheritance:
+
+paddleslim\.nas\.one\_shot\.super\_mnasnet module
+-------------------------------------------------
+
+.. automodule:: paddleslim.nas.one_shot.super_mnasnet
+    :members:
+    :undoc-members:
+    :show-inheritance:
+
+

_sources/api_en/paddleslim.nas.rst.txt

+paddleslim\.nas package
+=======================
+
+Subpackages
+-----------
+
+.. toctree::
+
+    paddleslim.nas.one_shot
+
+Submodules
+----------
+
+paddleslim\.nas\.sa\_nas module
+-------------------------------
+
+.. automodule:: paddleslim.nas.sa_nas
+    :members:
+    :undoc-members:
+    :show-inheritance:
+
+

_sources/api_en/paddleslim.pantheon.rst.txt

+paddleslim\.pantheon package
+============================
+
+.. automodule:: paddleslim.pantheon
+    :members:
+    :undoc-members:
+    :show-inheritance:
+
+Submodules
+----------
+
+paddleslim\.pantheon\.student module
+------------------------------------
+
+.. automodule:: paddleslim.pantheon.student
+    :members:
+    :undoc-members:
+    :show-inheritance:
+
+paddleslim\.pantheon\.teacher module
+------------------------------------
+
+.. automodule:: paddleslim.pantheon.teacher
+    :members:
+    :undoc-members:
+    :show-inheritance:
+
+paddleslim\.pantheon\.utils module
+----------------------------------
+
+.. automodule:: paddleslim.pantheon.utils
+    :members:
+    :undoc-members:
+    :show-inheritance:
+
+

_sources/api_en/paddleslim.prune.rst.txt

+paddleslim\.prune package
+=========================
+
+.. automodule:: paddleslim.prune
+    :members:
+    :undoc-members:
+    :show-inheritance:
+
+Submodules
+----------
+
+paddleslim\.prune\.auto\_pruner module
+--------------------------------------
+
+.. automodule:: paddleslim.prune.auto_pruner
+    :members:
+    :undoc-members:
+    :show-inheritance:
+
+paddleslim\.prune\.prune\_io module
+-----------------------------------
+
+.. automodule:: paddleslim.prune.prune_io
+    :members:
+    :undoc-members:
+    :show-inheritance:
+
+paddleslim\.prune\.prune\_walker module
+---------------------------------------
+
+.. automodule:: paddleslim.prune.prune_walker
+    :members:
+    :undoc-members:
+    :show-inheritance:
+
+paddleslim\.prune\.pruner module
+--------------------------------
+
+.. automodule:: paddleslim.prune.pruner
+    :members:
+    :undoc-members:
+    :show-inheritance:
+
+paddleslim\.prune\.sensitive module
+-----------------------------------
+
+.. automodule:: paddleslim.prune.sensitive
+    :members:
+    :undoc-members:
+    :show-inheritance:
+
+paddleslim\.prune\.sensitive\_pruner module
+-------------------------------------------
+
+.. automodule:: paddleslim.prune.sensitive_pruner
+    :members:
+    :undoc-members:
+    :show-inheritance:
+
+

_sources/api_en/paddleslim.quant.rst.txt

+paddleslim\.quant package
+=========================
+
+.. automodule:: paddleslim.quant
+    :members:
+    :undoc-members:
+    :show-inheritance:
+
+Submodules
+----------
+
+paddleslim\.quant\.quant\_embedding module
+------------------------------------------
+
+.. automodule:: paddleslim.quant.quant_embedding
+    :members:
+    :undoc-members:
+    :show-inheritance:
+
+paddleslim\.quant\.quanter module
+---------------------------------
+
+.. automodule:: paddleslim.quant.quanter
+    :members:
+    :undoc-members:
+    :show-inheritance:
+
+

_sources/api_en/paddleslim.rst.txt

+paddleslim package
+==================
+
+.. automodule:: paddleslim
+    :members:
+    :undoc-members:
+    :show-inheritance:
+
+Subpackages
+-----------
+
+.. toctree::
+
+    paddleslim.analysis
+    paddleslim.common
+    paddleslim.core
+    paddleslim.dist
+    paddleslim.models
+    paddleslim.nas
+    paddleslim.pantheon
+    paddleslim.prune
+    paddleslim.quant
+
+Submodules
+----------
+
+paddleslim\.version module
+--------------------------
+
+.. automodule:: paddleslim.version
+    :members:
+    :undoc-members:
+    :show-inheritance:
+
+

_sources/api_en/search_space_en.rst.txt

+search space
+========
+Search Space used in neural architecture search. Search Space is a collection of model architecture, the purpose of SANAS is to get a model which FLOPs or latency is smaller or percision is higher.
+
+search space which paddleslim.nas provided
+-------
+
+Based on origin model architecture:
+1. MobileNetV2Space<br>
+&emsp; MobileNetV2's architecture can reference: [code](https://github.com/PaddlePaddle/models/blob/develop/PaddleCV/image_classification/models/mobilenet_v2.py#L29), [paper](https://arxiv.org/abs/1801.04381)
+
+2. MobileNetV1Space<br>
+&emsp; MobilNetV1's architecture can reference: [code](https://github.com/PaddlePaddle/models/blob/develop/PaddleCV/image_classification/models/mobilenet_v1.py#L29), [paper](https://arxiv.org/abs/1704.04861)
+
+3. ResNetSpace<br>
+&emsp; ResNetSpace's architecture can reference: [code](https://github.com/PaddlePaddle/models/blob/develop/PaddleCV/image_classification/models/resnet.py#L30), [paper](https://arxiv.org/pdf/1512.03385.pdf)
+
+
+Based on block from different model:
+1. MobileNetV1BlockSpace<br>
+&emsp; MobileNetV1Block's architecture can reference: [code](https://github.com/PaddlePaddle/models/blob/develop/PaddleCV/image_classification/models/mobilenet_v1.py#L173)
+
+2. MobileNetV2BlockSpace<br>
+&emsp; MobileNetV2Block's architecture can reference: [code](https://github.com/PaddlePaddle/models/blob/develop/PaddleCV/image_classification/models/mobilenet_v2.py#L174)
+
+3. ResNetBlockSpace<br>
+&emsp; ResNetBlock's architecture can reference: [code](https://github.com/PaddlePaddle/models/blob/develop/PaddleCV/image_classification/models/resnet.py#L148)
+
+4. InceptionABlockSpace<br>
+&emsp; InceptionABlock's architecture can reference: [code](https://github.com/PaddlePaddle/models/blob/develop/PaddleCV/image_classification/models/inception_v4.py#L140)
+
+5. InceptionCBlockSpace<br>
+&emsp; InceptionCBlock's architecture can reference: [code](https://github.com/PaddlePaddle/models/blob/develop/PaddleCV/image_classification/models/inception_v4.py#L291)
+
+
+How to use search space
+--------
+1. Only need to specify the name of search space if use the space based on origin model architecture, such as configs for class SANAS is [('MobileNetV2Space')] if you want to use origin MobileNetV2 as search space.
+2. Use search space paddleslim.nas provided based on block:<br>
+  2.1 Use `input_size`, `output_size` and `block_num` to construct search space, such as configs for class SANAS is ('MobileNetV2BlockSpace', {'input_size': 224, 'output_size': 32, 'block_num': 10})].<br>
+  2.2 Use `block_mask` to construct search space, such as configs for class SANAS is [('MobileNetV2BlockSpace', {'block_mask': [0, 1, 1, 1, 1, 0, 1, 0]})].
+
+How to write yourself search space
+--------
+If you want to write yourself search space, you need to inherit base class named SearchSpaceBase and overwrite following functions:<br>
+&emsp; 1. Function to get initial tokens(function `init_tokens`), set the initial tokens which you want, every token in tokens means index of search list, such as if tokens=[0, 3, 5], it means the list of channel of current model architecture is [8, 40, 128].
+&emsp; 2. Function about the length of every token in tokens(function `range_table`), range of every token in tokens.
+&emsp; 3. Function to get model architecture according to tokens(function `token2arch`), get model architecture according to tokens in the search process.
+
+For example, how to add a search space with resnet block. New search space can NOT has the same name with existing search space.
+
+```python
+### import necessary head file
+from .search_space_base import SearchSpaceBase
+from .search_space_registry import SEARCHSPACE
+import numpy as np
+
+### use decorator SEARCHSPACE.register to register yourself search space to search space NameSpace
+@SEARCHSPACE.register
+### define a search space class inherit the base class SearchSpaceBase
+class ResNetBlockSpace2(SearchSpaceBase):
+    def __init__(self, input_size, output_size, block_num, block_mask):
+        ### define the iterm you want to search, such as the numeber of channel, the number of convolution repeat, the size of kernel.
+        ### self.filter_num represents the search list about the numeber of channel.
+        self.filter_num = np.array([8, 16, 32, 40, 64, 128, 256, 512])
+
+    ### define initial tokens, the length of initial tokens according to block_num or block_mask.
+    def init_tokens(self):
+        return [0] * 3 * len(self.block_mask)
+
+    ### define the range of index in tokens.
+    def range_table(self):
+        return [len(self.filter_num)] * 3 * len(self.block_mask)
+
+    ### transform tokens to model architecture.
+    def token2arch(self, tokens=None):
+        if tokens == None:
+            tokens = self.init_tokens()
+
+        self.bottleneck_params_list = []
+        for i in range(len(self.block_mask)):
+            self.bottleneck_params_list.append(self.filter_num[tokens[i * 3 + 0]],
+                                               self.filter_num[tokens[i * 3 + 1]],
+                                               self.filter_num[tokens[i * 3 + 2]],
+                                               2 if self.block_mask[i] == 1 else 1)
+
+        def net_arch(input):
+            for i, layer_setting in enumerate(self.bottleneck_params_list):
+                channel_num, stride = layer_setting[:-1], layer_setting[-1]
+                input = self._resnet_block(input, channel_num, stride, name='resnet_layer{}'.format(i+1))
+
+            return input
+
+        return net_arch
+
+    ### code to get block.
+    def _resnet_block(self, input, channel_num, stride, name=None):
+        shortcut_conv = self._shortcut(input, channel_num[2], stride, name=name)
+        input = self._conv_bn_layer(input=input, num_filters=channel_num[0], filter_size=1, act='relu', name=name + '_conv0')
+        input = self._conv_bn_layer(input=input, num_filters=channel_num[1], filter_size=3, stride=stride, act='relu', name=name + '_conv1')
+        input = self._conv_bn_layer(input=input, num_filters=channel_num[2], filter_size=1, name=name + '_conv2')
+        return fluid.layers.elementwise_add(x=shortcut_conv, y=input, axis=0, name=name+'_elementwise_add')
+
+    def _shortcut(self, input, channel_num, stride, name=None):
+        channel_in = input.shape[1]
+        if channel_in != channel_num or stride != 1:
+            return self.conv_bn_layer(input, num_filters=channel_num, filter_size=1, stride=stride, name=name+'_shortcut')
+        else:
+            return input
+
+    def _conv_bn_layer(self, input, num_filters, filter_size, stride=1, padding='SAME', act=None, name=None):
+        conv = fluid.layers.conv2d(input, num_filters, filter_size, stride, name=name+'_conv')
+        bn = fluid.layers.batch_norm(conv, act=act, name=name+'_bn')
+        return bn
+```

_sources/index.rst.txt

+.. PaddleSlim documentation master file, created by
+   sphinx-quickstart on Wed Feb  5 14:04:52 2020.
+   You can adapt this file completely to your liking, but it should at least
+   contain the root `toctree` directive.
+
+中文文档
+========
+
+.. toctree::
+   :maxdepth: 1

_sources/index_en.rst.txt

+.. PaddleSlim documentation master file, created by
+   sphinx-quickstart on Wed Feb  5 14:04:52 2020.
+   You can adapt this file completely to your liking, but it should at least
+   contain the root `toctree` directive.
+
+Welcome to use PaddleSlim.
+========
+
+.. toctree::
+   :maxdepth: 1
+
+   index
+   intro_en.md
+   install_en.md
+   quick_start/index_en
+   tutorials/index_en
+   api_en/index_en
+   model_zoo_en.md

_sources/install.md.txt

+# 安装
+
+安装PaddleSlim前，请确认已正确安装Paddle1.6版本或更新版本。Paddle安装请参考：[Paddle安装教程](https://www.paddlepaddle.org.cn/install/quick)。
+
+
+- 安装develop版本
+
+
+```bash
+git clone https://github.com/PaddlePaddle/PaddleSlim.git
+cd PaddleSlim
+python setup.py install
+```
+
+- 安装官方发布的最新版本
+
+```bash
+pip install paddleslim -i https://pypi.org/simple
+```
+
+- 安装历史版本
+
+请点击[pypi.org](https://pypi.org/project/paddleslim/#history)查看可安装历史版本。

_sources/install_en.md.txt

+# Install
+
+安装PaddleSlim前，请确认已正确安装Paddle1.6版本或更新版本。Paddle安装请参考：[Paddle安装教程](https://www.paddlepaddle.org.cn/install/quick)。
+
+
+- 安装develop版本
+
+
+```bash
+git clone https://github.com/PaddlePaddle/PaddleSlim.git
+cd PaddleSlim
+python setup.py install
+```
+
+- 安装官方发布的最新版本
+
+```bash
+pip install paddleslim -i https://pypi.org/simple
+```
+
+- 安装历史版本
+
+请点击[pypi.org](https://pypi.org/project/paddleslim/#history)查看可安装历史版本。

_sources/intro.md.txt

+
+
+# PaddleSlim简介
+
+PaddleSlim是PaddlePaddle框架的一个子模块，主要用于压缩图像领域模型。在PaddleSlim中，不仅实现了目前主流的网络剪枝、量化、蒸馏三种压缩策略，还实现了超参数搜索和小模型网络结构搜索功能。在后续版本中，会添加更多的压缩策略，以及完善对NLP领域模型的支持。
+
+## 功能
+
+- 模型剪裁
+  - 支持通道均匀模型剪裁（uniform pruning)
+  - 基于敏感度的模型剪裁
+  - 基于进化算法的自动模型剪裁三种方式
+
+- 量化训练
+  - 在线量化训练（training aware）
+  - 离线量化（post training）
+  - 支持对权重全局量化和Channel-Wise量化
+
+- 知识蒸馏
+  - 支持单进程知识蒸馏
+  - 支持多进程分布式知识蒸馏
+
+- 神经网络结构自动搜索（NAS）
+  - 支持One-Shot网络结构自动搜索（Ont-Shot-NAS）
+  - 支持基于进化算法的轻量神经网络结构自动搜索（Light-NAS）
+  - 支持 FLOPS / 硬件延时约束
+  - 支持多平台模型延时评估

_sources/intro_en.md.txt

+
+
+# Introduction
+
+As a submodule of PaddlePaddle framework, PaddleSlim is an open-source library for deep model compression and architecture search. PaddleSlim supports current popular deep compression techniques such as pruning, quantization, and knowledge distillation. Further, it also automates the search of hyperparameters and the design of lightweight deep architectures. In the future, we will develop more practically useful compression techniques for industrial-level applications and transfer these techniques to models in NLP.
+
+
+## Methods
+
+- Pruning
+  - Uniform pruning
+  - Sensitivity-based pruning
+  - Automated model pruning
+
+- Quantization
+  - Training-aware quantization: Quantize models with hyperparameters dynamically estimated from small batches of samples.
+  - Training-aware quantization: Quantize models with the same hyperparameters estimated from training data.
+  - Support global quantization of weights and Channel-Wise quantization
+
+- Knowledge Distillation
+  - Single-process knowledge distillation
+  - Multi-process distributed knowledge distillation
+
+- Network Architecture Search（NAS）
+  - Simulated Annealing (SA)-based lightweight network architecture search method.（Light-NAS）
+  - One-Shot network structure automatic search. (One-Shot-NAS)
+  - PaddleSlim supports FLOPs and latency constrained search.
+  - PaddleSlim supports the latency estimation on different hardware and platforms.

_sources/model_zoo_en.md.txt

+# Model Zoo
+
+## 1. 图象分类
+
+数据集：ImageNet1000类
+
+### 1.1 量化
+
+| 模型 | 压缩方法 | Top-1/Top-5 Acc | 模型体积（MB） | 下载 |
+|:--:|:---:|:--:|:--:|:--:|
+|MobileNetV1|-|70.99%/89.68%| xx | [下载链接]() |
+|MobileNetV1|quant_post|xx%/xx%| xx | [下载链接]() |
+|MobileNetV1|quant_aware|xx%/xx%| xx | [下载链接]() |
+| MobileNetV2 | - |72.15%/90.65%| xx | [下载链接]() |
+| MobileNetV2 | quant_post |xx%/xx%| xx | [下载链接]() |
+| MobileNetV2 | quant_aware |xx%/xx%| xx | [下载链接]() |
+|ResNet50|-|76.50%/93.00%| xx | [下载链接]() |
+|ResNet50|quant_post|xx%/xx%| xx | [下载链接]() |
+|ResNet50|quant_aware|xx%/xx%| xx | [下载链接]() |
+
+
+
+### 1.2 剪裁
+
+
+| 模型 | 压缩方法 | Top-1/Top-5 Acc | 模型体积（MB） | GFLOPs | 下载 |
+|:--:|:---:|:--:|:--:|:--:|:--:|
+| MobileNetV1 |    Baseline    |         70.99%/89.68%         |       17       |  1.11  | [下载链接](http://paddle-imagenet-models-name.bj.bcebos.com/MobileNetV1_pretrained.tar) |
+| MobileNetV1 |  uniform -50%  | 69.4%/88.66% (-1.59%/-1.02%)  |       9        |  0.56  | [下载链接](https://paddlemodels.bj.bcebos.com/PaddleSlim/MobileNetV1_uniform-50.tar) |
+| MobileNetV1 | sensitive -30% |  70.4%/89.3% (-0.59%/-0.38%)  |       12       |  0.74  | [下载链接](https://paddlemodels.bj.bcebos.com/PaddleSlim/MobileNetV1_sensitive-30.tar) |
+| MobileNetV1 | sensitive -50% | 69.8% / 88.9% (-1.19%/-0.78%) |       9        |  0.56  | [下载链接](https://paddlemodels.bj.bcebos.com/PaddleSlim/MobileNetV1_sensitive-50.tar) |
+| MobileNetV2 |       -        |         72.15%/90.65%         |       15       |  0.59  | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/MobileNetV2_pretrained.tar) |
+| MobileNetV2 |  uniform -50%  | 65.79%/86.11% (-6.35%/-4.47%) |       11       | 0.296  | [下载链接](https://paddlemodels.bj.bcebos.com/PaddleSlim/MobileNetV2_uniform-50.tar) |
+|  ResNet34   |       -        |         72.15%/90.65%         |       84       |  7.36  | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/ResNet34_pretrained.tar) |
+|  ResNet34   |  uniform -50%  | 70.99%/89.95% (-1.36%/-0.87%) |       41       |  3.67  | [下载链接](https://paddlemodels.bj.bcebos.com/PaddleSlim/ResNet34_uniform-50.tar) |
+|  ResNet34   |  auto -55.05%  | 70.24%/89.63% (-2.04%/-1.06%) |       33       |  3.31  | [下载链接](https://paddlemodels.bj.bcebos.com/PaddleSlim/ResNet34_auto-55.tar) |
+
+
+
+
+### 1.3 蒸馏
+
+| 模型 | 压缩方法 | Top-1/Top-5 Acc | 模型体积（MB） | 下载 |
+|:--:|:---:|:--:|:--:|:--:|
+| MobileNetV1 |                     student                     |  70.99%/89.68%  |       17       | [下载链接](http://paddle-imagenet-models-name.bj.bcebos.com/MobileNetV1_pretrained.tar) |
+|ResNet50_vd|teacher|79.12%/94.44%| 99 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/ResNet50_vd_pretrained.tar) |
+|MobileNetV1|ResNet50_vd<sup>[1](#trans1)</sup> distill|72.77%/90.68% (+1.78%/+1.00%)| 17 | [下载链接](https://paddlemodels.bj.bcebos.com/PaddleSlim/MobileNetV1_distilled.tar) |
+| MobileNetV2 |                     student                     |  72.15%/90.65%  |       15       | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/MobileNetV2_pretrained.tar) |
+| MobileNetV2 |            ResNet50_vd distill             |  74.28%/91.53% (+2.13%/+0.88%)  |       15       | [下载链接](https://paddlemodels.bj.bcebos.com/PaddleSlim/MobileNetV2_distilled.tar) |
+|  ResNet50   |                     student                     |  76.50%/93.00%  |       99       | [下载链接](http://paddle-imagenet-models-name.bj.bcebos.com/ResNet50_pretrained.tar) |
+|ResNet101|teacher|77.56%/93.64%| 173 | [下载链接](http://paddle-imagenet-models-name.bj.bcebos.com/ResNet101_pretrained.tar) |
+|  ResNet50   |             ResNet101 distill              |  77.29%/93.65% (+0.79%/+0.65%)  |       99       | [下载链接](https://paddlemodels.bj.bcebos.com/PaddleSlim/ResNet50_distilled.tar) |
+
+!!! note "Note"
+
+    <a name="trans1">[1]</a>：带_vd后缀代表该预训练模型使用了Mixup，Mixup相关介绍参考[mixup: Beyond Empirical Risk Minimization](https://arxiv.org/abs/1710.09412)
+
+
+## 2. 目标检测
+
+### 2.1 量化
+
+数据集： COCO 2017
+
+|              模型              |  压缩方法   | 数据集 | Image/GPU | 输入608 Box AP | 输入416 Box AP | 输入320 Box AP | 模型体积（MB） |     下载     |
+| :----------------------------: | :---------: | :----: | :-------: | :------------: | :------------: | :------------: | :------------: | :----------: |
+|      MobileNet-V1-YOLOv3       |      -      |  COCO  |     8     |      29.3      |      29.3      |      27.1      |       xx       | [下载链接]() |
+|      MobileNet-V1-YOLOv3       | quant_post  |  COCO  |     8     |       xx       |       xx       |       xx       |       xx       | [下载链接]() |
+|      MobileNet-V1-YOLOv3       | quant_aware |  COCO  |     8     |       xx       |       xx       |       xx       |       xx       | [下载链接]() |
+| R50-dcn-YOLOv3 obj365_pretrain |      -      |  COCO  |     8     |      41.4      |       xx       |       xx       |       xx       | [下载链接]() |
+| R50-dcn-YOLOv3 obj365_pretrain | quant_post  |  COCO  |     8     |       xx       |       xx       |       xx       |       xx       | [下载链接]() |
+| R50-dcn-YOLOv3 obj365_pretrain | quant_aware |  COCO  |     8     |       xx       |       xx       |       xx       |       xx       | [下载链接]() |
+
+
+
+数据集：WIDER-FACE
+
+
+
+|      模型      |  压缩方法   | Image/GPU | 输入尺寸 | Easy/Medium/Hard  | 模型体积（MB） |     下载     |
+| :------------: | :---------: | :-------: | :------: | :---------------: | :------------: | :----------: |
+|   BlazeFace    |      -      |     8     |   640    | 0.915/0.892/0.797 |       xx       | [下载链接]() |
+|   BlazeFace    | quant_post  |     8     |   640    |     xx/xx/xx      |       xx       | [下载链接]() |
+|   BlazeFace    | quant_aware |     8     |   640    |     xx/xx/xx      |       xx       | [下载链接]() |
+| BlazeFace-Lite |      -      |     8     |   640    | 0.909/0.885/0.781 |       xx       | [下载链接]() |
+| BlazeFace-Lite | quant_post  |     8     |   640    |     xx/xx/xx      |       xx       | [下载链接]() |
+| BlazeFace-Lite | quant_aware |     8     |   640    |     xx/xx/xx      |       xx       | [下载链接]() |
+| BlazeFace-NAS  |      -      |     8     |   640    | 0.837/0.807/0.658 |       xx       | [下载链接]() |
+| BlazeFace-NAS  | quant_post  |     8     |   640    |     xx/xx/xx      |       xx       | [下载链接]() |
+| BlazeFace-NAS  | quant_aware |     8     |   640    |     xx/xx/xx      |       xx       | [下载链接]() |
+
+### 2.2 剪裁
+
+数据集：Pasacl VOC & COCO 2017
+
+|              模型              |     压缩方法      |   数据集   | Image/GPU | 输入608 Box AP | 输入416 Box AP | 输入320 Box AP | 模型体积(MB) | GFLOPs (608*608) |                             下载                             |
+| :----------------------------: | :---------------: | :--------: | :-------: | :------------: | :------------: | :------------: | :----------: | :--------------: | :----------------------------------------------------------: |
+|      MobileNet-V1-YOLOv3       |     Baseline      | Pascal VOC |     8     |      76.2      |      76.7      |      75.3      |      94      |      40.49       | [下载链接](https://paddlemodels.bj.bcebos.com/object_detection/yolov3_mobilenet_v1_voc.tar) |
+|      MobileNet-V1-YOLOv3       | sensitive -52.88% | Pascal VOC |     8     |  77.6 (+1.4)   |   77.7 (1.0)   |  75.5 (+0.2)   |      31      |      19.08       | [下载链接](https://paddlemodels.bj.bcebos.com/PaddleSlim/yolov3_mobilenet_v1_voc_prune.tar) |
+|      MobileNet-V1-YOLOv3       |         -         |    COCO    |     8     |      29.3      |      29.3      |      27.0      |      95      |      41.35       | [下载链接](https://paddlemodels.bj.bcebos.com/object_detection/yolov3_mobilenet_v1.tar) |
+|      MobileNet-V1-YOLOv3       | sensitive -51.77% |    COCO    |     8     |  26.0 (-3.3)   |  25.1 (-4.2)   |  22.6 (-4.4)   |      32      |      19.94       | [下载链接](https://paddlemodels.bj.bcebos.com/PaddleSlim/yolov3_mobilenet_v1_prune.tar) |
+|         R50-dcn-YOLOv3         |         -         |    COCO    |     8     |      39.1      |       -        |       -        |     177      |      89.60       | [下载链接](https://paddlemodels.bj.bcebos.com/object_detection/yolov3_r50vd_dcn.tar) |
+|         R50-dcn-YOLOv3         | sensitive -9.37%  |    COCO    |     8     |  39.3 (+0.2)   |       -        |       -        |     150      |      81.20       | [下载链接](https://paddlemodels.bj.bcebos.com/PaddleSlim/yolov3_r50vd_dcn_prune.tar) |
+|         R50-dcn-YOLOv3         | sensitive -24.68% |    COCO    |     8     |  37.3 (-1.8)   |       -        |       -        |     113      |      67.48       | [下载链接](https://paddlemodels.bj.bcebos.com/PaddleSlim/yolov3_r50vd_dcn_prune578.tar) |
+| R50-dcn-YOLOv3 obj365_pretrain |         -         |    COCO    |     8     |      41.4      |       -        |       -        |     177      |      89.60       | [下载链接](https://paddlemodels.bj.bcebos.com/object_detection/yolov3_r50vd_dcn_obj365_pretrained_coco.tar) |
+| R50-dcn-YOLOv3 obj365_pretrain | sensitive -9.37%  |    COCO    |     8     |  40.5 (-0.9)   |       -        |       -        |     150      |      81.20       | [下载链接](https://paddlemodels.bj.bcebos.com/PaddleSlim/yolov3_r50vd_dcn_obj365_pretrained_coco_prune.tar) |
+| R50-dcn-YOLOv3 obj365_pretrain | sensitive -24.68% |    COCO    |     8     |  37.8 (-3.3)   |       -        |       -        |     113      |      67.48       | [下载链接](https://paddlemodels.bj.bcebos.com/PaddleSlim/yolov3_r50vd_dcn_obj365_pretrained_coco_prune578.tar) |
+
+### 2.3 蒸馏
+
+数据集：Pasacl VOC & COCO 2017
+
+
+|        模型         |        压缩方法         |   数据集   | Image/GPU | 输入608 Box AP | 输入416 Box AP | 输入320 Box AP | 模型体积（MB） |                             下载                             |
+| :-----------------: | :---------------------: | :--------: | :-------: | :------------: | :------------: | :------------: | :------------: | :----------------------------------------------------------: |
+| MobileNet-V1-YOLOv3 |            -            | Pascal VOC |     8     |      76.2      |      76.7      |      75.3      |       94       | [下载链接](https://paddlemodels.bj.bcebos.com/object_detection/yolov3_mobilenet_v1_voc.tar) |
+|   ResNet34-YOLOv3   |            -            | Pascal VOC |     8     |      82.6      |      81.9      |      80.1      |      162       | [下载链接](https://paddlemodels.bj.bcebos.com/object_detection/yolov3_r34_voc.tar) |
+| MobileNet-V1-YOLOv3 | ResNet34-YOLOv3 distill | Pascal VOC |     8     |  79.0 (+2.8)   |  78.2 (+1.5)   |  75.5 (+0.2)   |       94       | [下载链接](https://paddlemodels.bj.bcebos.com/PaddleSlim/yolov3_mobilenetv1_voc_distilled.tar) |
+| MobileNet-V1-YOLOv3 |            -            |    COCO    |     8     |      29.3      |      29.3      |      27.0      |       95       | [下载链接](https://paddlemodels.bj.bcebos.com/object_detection/yolov3_mobilenet_v1.tar) |
+|   ResNet34-YOLOv3   |            -            |    COCO    |     8     |      36.2      |      34.3      |      31.4      |      163       | [下载链接](https://paddlemodels.bj.bcebos.com/object_detection/yolov3_r34.tar) |
+| MobileNet-V1-YOLOv3 | ResNet34-YOLOv3 distill |    COCO    |     8     |  31.4 (+2.1)   |  30.0 (+0.7)   |  27.1 (+0.1)   |       95       | [下载链接](https://paddlemodels.bj.bcebos.com/PaddleSlim/yolov3_mobilenetv1_coco_distilled.tar) |
+
+
+## 3. 图像分割
+
+数据集：Cityscapes
+
+### 3.1 量化
+
+|          模型          |  压缩方法   | mIoU  | 模型体积（MB） |     下载     |
+| :--------------------: | :---------: | :---: | :------------: | :----------: |
+| DeepLabv3+/MobileNetv1 |      -      | 63.26 |       xx       | [下载链接]() |
+| DeepLabv3+/MobileNetv1 | quant_post  |  xx   |       xx       | [下载链接]() |
+| DeepLabv3+/MobileNetv1 | quant_aware |  xx   |       xx       | [下载链接]() |
+| DeepLabv3+/MobileNetv2 |      -      | 69.81 |       xx       | [下载链接]() |
+| DeepLabv3+/MobileNetv2 | quant_post  |  xx   |       xx       | [下载链接]() |
+| DeepLabv3+/MobileNetv2 | quant_aware |  xx   |       xx       | [下载链接]() |
+
+### 3.2 剪裁
+
+|   模型    |     压缩方法      |     mIoU      | 模型体积（MB） | GFLOPs |                             下载                             |
+| :-------: | :---------------: | :-----------: | :------------: | :----: | :----------------------------------------------------------: |
+| fast-scnn |     baseline      |     69.64     |       11       | 14.41  | [下载链接](https://paddlemodels.bj.bcebos.com/PaddleSlim/fast_scnn_cityscape.tar) |
+| fast-scnn | uniform  -17.07%  | 69.58 (-0.06) |      8.5       | 11.95  | [下载链接](https://paddlemodels.bj.bcebos.com/PaddleSlim/fast_scnn_cityscape_uniform-17.tar) |
+| fast-scnn | sensitive -47.60% | 66.68 (-2.96) |      5.7       |  7.55  | [下载链接](https://paddlemodels.bj.bcebos.com/PaddleSlim/fast_scnn_cityscape_sensitive-47.tar) |

_sources/quick_start/distillation_tutorial.md.txt

+#  图像分类模型知识蒸馏-快速开始
+
+该教程以图像分类模型MobileNetV1为例，说明如何快速使用[PaddleSlim的知识蒸馏接口](https://paddlepaddle.github.io/PaddleSlim/api/single_distiller_api/)。
+该示例包含以下步骤：
+
+1. 导入依赖
+2. 定义student_program和teacher_program
+3. 选择特征图
+4. 合并program(merge)并添加蒸馏loss
+5. 模型训练
+
+以下章节依次介绍每个步骤的内容。
+
+## 1. 导入依赖
+
+PaddleSlim依赖Paddle1.7版本，请确认已正确安装Paddle，然后按以下方式导入Paddle和PaddleSlim:
+
+```
+import paddle
+import paddle.fluid as fluid
+import paddleslim as slim
+```
+
+## 2. 定义student_program和teacher_program
+
+本教程在MNIST数据集上进行知识蒸馏的训练和验证，输入图片尺寸为`[1, 28, 28]`，输出类别数为10。
+选择`ResNet50`作为teacher对`MobileNet`结构的student进行蒸馏训练。
+
+```python
+model = models.__dict__['MobileNet']()
+student_program = fluid.Program()
+student_startup = fluid.Program()
+with fluid.program_guard(student_program, student_startup):
+    image = fluid.data(
+        name='image', shape=[None] + [1, 28, 28], dtype='float32')
+    label = fluid.data(name='label', shape=[None, 1], dtype='int64')
+    out = model.net(input=image, class_dim=10)
+    cost = fluid.layers.cross_entropy(input=out, label=label)
+    avg_cost = fluid.layers.mean(x=cost)
+    acc_top1 = fluid.layers.accuracy(input=out, label=label, k=1)
+    acc_top5 = fluid.layers.accuracy(input=out, label=label, k=5)
+```
+
+
+
+```python
+teacher_model = models.__dict__['ResNet50']()
+teacher_program = fluid.Program()
+teacher_startup = fluid.Program()
+with fluid.program_guard(teacher_program, teacher_startup):
+    with fluid.unique_name.guard():
+        image = fluid.data(
+            name='image', shape=[None] + [1, 28, 28], dtype='float32')
+        predict = teacher_model.net(image, class_dim=10)
+exe = fluid.Executor(fluid.CPUPlace())
+exe.run(teacher_startup)
+```
+
+## 3. 选择特征图
+
+我们可以用student_的list_vars方法来观察其中全部的Variables，从中选出一个或多个变量（Variable）来拟合teacher相应的变量。
+
+```python
+# get all student variables
+student_vars = []
+for v in student_program.list_vars():
+    student_vars.append((v.name, v.shape))
+#uncomment the following lines to observe student's variables for distillation
+#print("="*50+"student_model_vars"+"="*50)
+#print(student_vars)
+
+# get all teacher variables
+teacher_vars = []
+for v in teacher_program.list_vars():
+    teacher_vars.append((v.name, v.shape))
+#uncomment the following lines to observe teacher's variables for distillation
+#print("="*50+"teacher_model_vars"+"="*50)
+#print(teacher_vars)
+```
+
+经过筛选我们可以看到，teacher_program中的'bn5c_branch2b.output.1.tmp_3'和student_program的'depthwise_conv2d_11.tmp_0'尺寸一致，可以组成蒸馏损失函数。
+
+## 4. 合并program (merge)并添加蒸馏loss
+merge操作将student_program和teacher_program中的所有Variables和Op都将被添加到同一个Program中，同时为了避免两个program中有同名变量会引起命名冲突，merge也会为teacher_program中的Variables添加一个同一的命名前缀name_prefix，其默认值是'teacher_'
+
+为了确保teacher网络和student网络输入的数据是一样的，merge操作也会对两个program的输入数据层进行合并操作，所以需要指定一个数据层名称的映射关系data_name_map，key是teacher的输入数据名称，value是student的
+
+```python
+data_name_map = {'image': 'image'}
+main = slim.dist.merge(teacher_program, student_program, data_name_map, fluid.CPUPlace())
+with fluid.program_guard(student_program, student_startup):
+    l2_loss = slim.dist.l2_loss('teacher_bn5c_branch2b.output.1.tmp_3', 'depthwise_conv2d_11.tmp_0', student_program)
+    loss = l2_loss + avg_cost
+    opt = fluid.optimizer.Momentum(0.01, 0.9)
+    opt.minimize(loss)
+exe.run(student_startup)
+```
+
+## 5. 模型训练
+
+为了快速执行该示例，我们选取简单的MNIST数据，Paddle框架的`paddle.dataset.mnist`包定义了MNIST数据的下载和读取。 代码如下：
+
+```python
+train_reader = paddle.batch(
+    paddle.dataset.mnist.train(), batch_size=128, drop_last=True)
+train_feeder = fluid.DataFeeder(['image', 'label'], fluid.CPUPlace(), student_program)
+```
+
+```python
+for data in train_reader():
+    acc1, acc5, loss_np = exe.run(student_program, feed=train_feeder.feed(data), fetch_list=[acc_top1.name, acc_top5.name, loss.name])
+    print("Acc1: {:.6f}, Acc5: {:.6f}, Loss: {:.6f}".format(acc1.mean(), acc5.mean(), loss_np.mean()))
+```

_sources/quick_start/distillation_tutorial_en.md.txt

+#  Knowledge Distillation for Image Classification
+
+In this tutorial, you will learn how to use knowledge distillation API of PaddleSlim
+by a demo of MobileNetV1 model on MNIST dataset. This tutorial following workflow:
+
+1. Import dependency
+2. Define student_program and teacher_program
+3. Select feature maps
+4. Merge program and add distillation loss
+5. Train distillation model
+
+## 1. Import dependency
+
+PaddleSlim dependents on Paddle1.7. Please ensure that you have installed paddle correctly. Import Paddle and PaddleSlim as below:
+
+```
+import paddle
+import paddle.fluid as fluid
+import paddleslim as slim
+```
+
+## 2. Define student_program and teacher_program
+
+This tutorial trains and verifies distillation model on the MNIST dataset. The input image shape is `[1, 28, 28] `and the number of output categories is 10.
+Select `ResNet50` as the teacher to perform distillation training on the students of the` MobileNet` architecture.
+
+```python
+model = models.__dict__['MobileNet']()
+student_program = fluid.Program()
+student_startup = fluid.Program()
+with fluid.program_guard(student_program, student_startup):
+    image = fluid.data(
+        name='image', shape=[None] + [1, 28, 28], dtype='float32')
+    label = fluid.data(name='label', shape=[None, 1], dtype='int64')
+    out = model.net(input=image, class_dim=10)
+    cost = fluid.layers.cross_entropy(input=out, label=label)
+    avg_cost = fluid.layers.mean(x=cost)
+    acc_top1 = fluid.layers.accuracy(input=out, label=label, k=1)
+    acc_top5 = fluid.layers.accuracy(input=out, label=label, k=5)
+```
+
+
+
+```python
+teacher_model = models.__dict__['ResNet50']()
+teacher_program = fluid.Program()
+teacher_startup = fluid.Program()
+with fluid.program_guard(teacher_program, teacher_startup):
+    with fluid.unique_name.guard():
+        image = fluid.data(
+            name='image', shape=[None] + [1, 28, 28], dtype='float32')
+        predict = teacher_model.net(image, class_dim=10)
+exe = fluid.Executor(fluid.CPUPlace())
+exe.run(teacher_startup)
+```
+
+## 3. Select feature maps
+
+We can use the student_program's list_vars method to observe all the Variables, and select one or more variables from it to fit the corresponding variables of the teacher.
+
+```python
+# get all student variables
+student_vars = []
+for v in student_program.list_vars():
+    student_vars.append((v.name, v.shape))
+#uncomment the following lines to observe student's variables for distillation
+#print("="*50+"student_model_vars"+"="*50)
+#print(student_vars)
+
+# get all teacher variables
+teacher_vars = []
+for v in teacher_program.list_vars():
+    teacher_vars.append((v.name, v.shape))
+#uncomment the following lines to observe teacher's variables for distillation
+#print("="*50+"teacher_model_vars"+"="*50)
+#print(teacher_vars)
+```
+
+we can see that the shape of 'bn5c_branch2b.output.1.tmp_3' in the teacher_program and the 'depthwise_conv2d_11.tmp_0' of the student are the same and can form the distillation loss function.
+
+## 4. Merge program and add distillation loss
+The merge operation adds all Variables and Ops in teacher_program to student_Program. At the same time, in order to avoid naming conflicts caused by variables with the same name in two programs, merge will also add a unified naming prefix **name_prefix** for Variables in teacher_program, which The default value is 'teacher_'_.
+
+In order to ensure that the data of the teacher network and the student network are the same, the merge operation also merges the input data layers of the two programs, so you need to specify a data layer name mapping ***data_name_map***, where key is the input data name of the teacher, and value Is student's.
+
+```python
+data_name_map = {'image': 'image'}
+main = slim.dist.merge(teacher_program, student_program, data_name_map, fluid.CPUPlace())
+with fluid.program_guard(student_program, student_startup):
+    l2_loss = slim.dist.l2_loss('teacher_bn5c_branch2b.output.1.tmp_3', 'depthwise_conv2d_11.tmp_0', student_program)
+    loss = l2_loss + avg_cost
+    opt = fluid.optimizer.Momentum(0.01, 0.9)
+    opt.minimize(loss)
+exe.run(student_startup)
+```
+
+## 5. Train distillation model
+
+The package `paddle.dataset.mnist` of Paddle define the downloading and reading of MNIST dataset.
+Define training data reader and test data reader as below：
+
+```python
+train_reader = paddle.batch(
+    paddle.dataset.mnist.train(), batch_size=128, drop_last=True)
+train_feeder = fluid.DataFeeder(['image', 'label'], fluid.CPUPlace(), student_program)
+```
+
+Excute following code to run an `epoch` training:
+
+
+```python
+for data in train_reader():
+    acc1, acc5, loss_np = exe.run(student_program, feed=train_feeder.feed(data), fetch_list=[acc_top1.name, acc_top5.name, loss.name])
+    print("Acc1: {:.6f}, Acc5: {:.6f}, Loss: {:.6f}".format(acc1.mean(), acc5.mean(), loss_np.mean()))
+```

_sources/quick_start/index.rst.txt

+
+快速开始
+========
+
+.. toctree::
+   :maxdepth: 1
+
+   pruning_tutorial.md
+   distillation_tutorial.md
+   quant_aware_tutorial.md
+   quant_post_tutorial.md
+   nas_tutorial.md
+    

_sources/quick_start/index_en.rst.txt

+
+Quick Start
+========
+
+.. toctree::
+   :maxdepth: 1
+
+   pruning_tutorial_en.md
+   nas_tutorial_en.md
+   quant_aware_tutorial_en.md
+   quant_post_tutorial_en.md
+    

_sources/quick_start/nas_tutorial.md.txt

+# 图像分类网络结构搜索-快速开始
+
+该教程以图像分类模型MobileNetV2为例，说明如何在cifar10数据集上快速使用[网络结构搜索接口](../api/nas_api.md)。
+该示例包含以下步骤：
+
+1. 导入依赖
+2. 初始化SANAS搜索实例
+3. 构建网络
+4. 定义输入数据函数
+5. 定义训练函数
+6. 定义评估函数
+7. 启动搜索实验
+  7.1 获取模型结构
+  7.2 构造program
+  7.3 定义输入数据
+  7.4 训练模型
+  7.5 评估模型
+  7.6 回传当前模型的得分
+8. 完整示例
+
+
+以下章节依次介绍每个步骤的内容。
+
+## 1. 导入依赖
+请确认已正确安装Paddle，导入需要的依赖包。
+```python
+import paddle
+import paddle.fluid as fluid
+import paddleslim as slim
+import numpy as np
+```
+
+## 2. 初始化SANAS搜索实例
+```python
+sanas = slim.nas.SANAS(configs=[('MobileNetV2Space')], server_addr=("", 8337), save_checkpoint=None)
+```
+
+## 3. 构建网络
+根据传入的网络结构构造训练program和测试program。
+```python
+def build_program(archs):
+    train_program = fluid.Program()
+    startup_program = fluid.Program()
+    with fluid.program_guard(train_program, startup_program):
+        data = fluid.data(name='data', shape=[None, 3, 32, 32], dtype='float32')
+        label = fluid.data(name='label', shape=[None, 1], dtype='int64')
+        output = archs(data)
+        output = fluid.layers.fc(input=output, size=10)
+
+        softmax_out = fluid.layers.softmax(input=output, use_cudnn=False)
+        cost = fluid.layers.cross_entropy(input=softmax_out, label=label)
+        avg_cost = fluid.layers.mean(cost)
+        acc_top1 = fluid.layers.accuracy(input=softmax_out, label=label, k=1)
+        acc_top5 = fluid.layers.accuracy(input=softmax_out, label=label, k=5)
+        test_program = fluid.default_main_program().clone(for_test=True)
+            
+        optimizer = fluid.optimizer.Adam(learning_rate=0.1)
+        optimizer.minimize(avg_cost)
+
+        place = fluid.CPUPlace()
+        exe = fluid.Executor(place)
+        exe.run(startup_program)
+    return exe, train_program, test_program, (data, label), avg_cost, acc_top1, acc_top5
+```
+
+## 4. 定义输入数据函数
+使用的数据集为cifar10，paddle框架中`paddle.dataset.cifar`包括了cifar数据集的下载和读取，代码如下：
+```python
+def input_data(inputs):
+    train_reader = paddle.batch(paddle.reader.shuffle(paddle.dataset.cifar.train10(cycle=False), buf_size=1024),batch_size=256)
+    train_feeder = fluid.DataFeeder(inputs, fluid.CPUPlace())
+    eval_reader = paddle.batch(paddle.dataset.cifar.test10(cycle=False), batch_size=256)
+    eval_feeder = fluid.DataFeeder(inputs, fluid.CPUPlace())
+    return train_reader, train_feeder, eval_reader, eval_feeder
+```
+
+## 5. 定义训练函数
+根据训练program和训练数据进行训练。
+```python
+def start_train(program, data_reader, data_feeder): 
+    outputs = [avg_cost.name, acc_top1.name, acc_top5.name]
+    for data in data_reader():
+        batch_reward = exe.run(program, feed=data_feeder.feed(data), fetch_list = outputs)
+        print("TRAIN: loss: {}, acc1: {}, acc5:{}".format(batch_reward[0], batch_reward[1], batch_reward[2]))
+```
+
+## 6. 定义评估函数
+根据评估program和评估数据进行评估。
+```python
+def start_eval(program, data_reader, data_feeder):
+    reward = []
+    outputs = [avg_cost.name, acc_top1.name, acc_top5.name]
+    for data in data_reader():
+        batch_reward = exe.run(program, feed=data_feeder.feed(data), fetch_list = outputs)
+        reward_avg = np.mean(np.array(batch_reward), axis=1)
+        reward.append(reward_avg)
+        print("TEST: loss: {}, acc1: {}, acc5:{}".format(batch_reward[0], batch_reward[1], batch_reward[2]))
+    finally_reward = np.mean(np.array(reward), axis=0)
+    print("FINAL TEST: avg_cost: {}, acc1: {}, acc5: {}".format(finally_reward[0], finally_reward[1], finally_reward[2]))
+    return finally_reward
+```
+
+## 7. 启动搜索实验
+以下步骤拆解说明了如何获得当前模型结构以及获得当前模型结构之后应该有的步骤，如果想要看如何启动搜索实验的完整示例可以看步骤9。
+
+### 7.1 获取模型结构
+调用`next_archs()`函数获取到下一个模型结构。
+```python
+archs = sanas.next_archs()[0]
+```
+
+### 7.2 构造program
+调用步骤3中的函数，根据4.1中的模型结构构造相应的program。
+```python
+exe, train_program, eval_program, inputs, avg_cost, acc_top1, acc_top5 = build_program(archs)
+```
+
+### 7.3 定义输入数据
+```python
+train_reader, train_feeder, eval_reader, eval_feeder = input_data(inputs)
+```
+
+### 7.4 训练模型
+根据上面得到的训练program和评估数据启动训练。
+```python
+start_train(train_program, train_reader, train_feeder)
+```
+### 7.5 评估模型
+根据上面得到的评估program和评估数据启动评估。
+```python
+finally_reward = start_eval(eval_program, eval_reader, eval_feeder)
+```
+### 7.6 回传当前模型的得分
+```
+sanas.reward(float(finally_reward[1]))
+```
+
+## 8. 完整示例
+以下是一个完整的搜索实验示例，示例中使用FLOPs作为约束条件，搜索实验一共搜索3个step，表示搜索到3个满足条件的模型结构进行训练，每搜索到一个网络结构训练7个epoch。
+```python
+for step in range(3):
+    archs = sanas.next_archs()[0]
+    exe, train_program, eval_progarm, inputs, avg_cost, acc_top1, acc_top5 = build_program(archs)
+    train_reader, train_feeder, eval_reader, eval_feeder = input_data(inputs)
+
+    current_flops = slim.analysis.flops(train_program)
+    if current_flops > 321208544:
+        continue
+    
+    for epoch in range(7):
+        start_train(train_program, train_reader, train_feeder)
+
+    finally_reward = start_eval(eval_program, eval_reader, eval_feeder)
+
+    sanas.reward(float(finally_reward[1]))
+```

_sources/quick_start/nas_tutorial_en.md.txt

+# Nerual Architecture Search for Image Classification 
+
+This tutorial shows how to use [API](../api/nas_api.md) about SANAS in PaddleSlim. We start experiment based on MobileNetV2 as example. The tutorial contains follow section.
+
+1. necessary imports
+2. initial SANAS instance
+3. define function about building program
+4. define function about input data 
+5. define function about training
+6. define funciton about evaluation
+7. start search
+  7.1 fetch model architecture
+  7.2 build program
+  7.3 define input data
+  7.4 train model
+  7.5 evaluate model
+  7.6 reture score
+8. full example
+
+
+The following chapter describes each steps in order.
+
+## 1. import dependency
+Please make sure that you haved installed Paddle correctly, then do the necessary imports.
+```python
+import paddle
+import paddle.fluid as fluid
+import paddleslim as slim
+import numpy as np
+```
+
+## 2. initial SANAS instance
+```python
+sanas = slim.nas.SANAS(configs=[('MobileNetV2Space')], server_addr=("", 8337), save_checkpoint=None)
+```
+
+## 3. define function about building program
+Build program about training and evaluation according to the model architecture.
+```python
+def build_program(archs):
+    train_program = fluid.Program()
+    startup_program = fluid.Program()
+    with fluid.program_guard(train_program, startup_program):
+        data = fluid.data(name='data', shape=[None, 3, 32, 32], dtype='float32')
+        label = fluid.data(name='label', shape=[None, 1], dtype='int64')
+        output = archs(data)
+        output = fluid.layers.fc(input=output, size=10)
+
+        softmax_out = fluid.layers.softmax(input=output, use_cudnn=False)
+        cost = fluid.layers.cross_entropy(input=softmax_out, label=label)
+        avg_cost = fluid.layers.mean(cost)
+        acc_top1 = fluid.layers.accuracy(input=softmax_out, label=label, k=1)
+        acc_top5 = fluid.layers.accuracy(input=softmax_out, label=label, k=5)
+        test_program = fluid.default_main_program().clone(for_test=True)
+            
+        optimizer = fluid.optimizer.Adam(learning_rate=0.1)
+        optimizer.minimize(avg_cost)
+
+        place = fluid.CPUPlace()
+        exe = fluid.Executor(place)
+        exe.run(startup_program)
+    return exe, train_program, test_program, (data, label), avg_cost, acc_top1, acc_top5
+```
+
+## 4. define function about input data 
+The dataset we used is cifar10, and `paddle.dataset.cifar` in Paddle including the download and pre-read about cifar. 
+```python
+def input_data(inputs):
+    train_reader = paddle.batch(paddle.reader.shuffle(paddle.dataset.cifar.train10(cycle=False), buf_size=1024),batch_size=256)
+    train_feeder = fluid.DataFeeder(inputs, fluid.CPUPlace())
+    eval_reader = paddle.batch(paddle.dataset.cifar.test10(cycle=False), batch_size=256)
+    eval_feeder = fluid.DataFeeder(inputs, fluid.CPUPlace())
+    return train_reader, train_feeder, eval_reader, eval_feeder
+```
+
+## 5. define function about training
+Start training.
+```python
+def start_train(program, data_reader, data_feeder): 
+    outputs = [avg_cost.name, acc_top1.name, acc_top5.name]
+    for data in data_reader():
+        batch_reward = exe.run(program, feed=data_feeder.feed(data), fetch_list = outputs)
+        print("TRAIN: loss: {}, acc1: {}, acc5:{}".format(batch_reward[0], batch_reward[1], batch_reward[2]))
+```
+
+## 6. define funciton about evaluation
+Start evaluating.
+```python
+def start_eval(program, data_reader, data_feeder):
+    reward = []
+    outputs = [avg_cost.name, acc_top1.name, acc_top5.name]
+    for data in data_reader():
+        batch_reward = exe.run(program, feed=data_feeder.feed(data), fetch_list = outputs)
+        reward_avg = np.mean(np.array(batch_reward), axis=1)
+        reward.append(reward_avg)
+        print("TEST: loss: {}, acc1: {}, acc5:{}".format(batch_reward[0], batch_reward[1], batch_reward[2]))
+    finally_reward = np.mean(np.array(reward), axis=0)
+    print("FINAL TEST: avg_cost: {}, acc1: {}, acc5: {}".format(finally_reward[0], finally_reward[1], finally_reward[2]))
+    return finally_reward
+```
+
+## 7. start search
+The following steps describes how to get current model architecture and what need to do after get the model architecture. If you want to start a full example directly, please jump to Step 9.
+
+### 7.1 fetch model architecture
+Get Next model architecture by `next_archs()`.
+```python
+archs = sanas.next_archs()[0]
+```
+
+### 7.2 build program
+Get program according to the function in Step3 and model architecture from Step 7.1.
+```python
+exe, train_program, eval_program, inputs, avg_cost, acc_top1, acc_top5 = build_program(archs)
+```
+
+### 7.3 define input data
+```python
+train_reader, train_feeder, eval_reader, eval_feeder = input_data(inputs)
+```
+
+### 7.4 train model
+Start training according to train program and data.
+```python
+start_train(train_program, train_reader, train_feeder)
+```
+### 7.5 evaluate model
+Start evaluation according to evaluation program and data.
+```python
+finally_reward = start_eval(eval_program, eval_reader, eval_feeder)
+```
+### 7.6 reture score
+```
+sanas.reward(float(finally_reward[1]))
+```
+
+## 8. full example
+The following is a full example about neural architecture search, it uses FLOPs as constraint and includes 3 steps, it means train 3 model architectures which is satisfied constraint, and train 7 epoch for each model architecture.
+```python
+for step in range(3):
+    archs = sanas.next_archs()[0]
+    exe, train_program, eval_progarm, inputs, avg_cost, acc_top1, acc_top5 = build_program(archs)
+    train_reader, train_feeder, eval_reader, eval_feeder = input_data(inputs)
+
+    current_flops = slim.analysis.flops(train_program)
+    if current_flops > 321208544:
+        continue
+    
+    for epoch in range(7):
+        start_train(train_program, train_reader, train_feeder)
+
+    finally_reward = start_eval(eval_program, eval_reader, eval_feeder)
+
+    sanas.reward(float(finally_reward[1]))
+```

_sources/quick_start/pruning_tutorial.md.txt

+#  图像分类模型通道剪裁-快速开始
+
+该教程以图像分类模型MobileNetV1为例，说明如何快速使用[PaddleSlim的卷积通道剪裁接口]()。
+该示例包含以下步骤：
+
+1. 导入依赖
+2. 构建模型
+3. 剪裁
+4. 训练剪裁后的模型
+
+以下章节依次次介绍每个步骤的内容。
+
+## 1. 导入依赖
+
+PaddleSlim依赖Paddle1.7版本，请确认已正确安装Paddle，然后按以下方式导入Paddle和PaddleSlim:
+
+```
+import paddle
+import paddle.fluid as fluid
+import paddleslim as slim
+```
+
+## 2. 构建网络
+
+该章节构造一个用于对MNIST数据进行分类的分类模型，选用`MobileNetV1`，并将输入大小设置为`[1, 28, 28]`，输出类别数为10。
+为了方便展示示例，我们在`paddleslim.models`下预定义了用于构建分类模型的方法，执行以下代码构建分类模型：
+
+```
+exe, train_program, val_program, inputs, outputs =
+    slim.models.image_classification("MobileNet", [1, 28, 28], 10, use_gpu=False)
+```
+
+>注意：paddleslim.models下的API并非PaddleSlim常规API，是为了简化示例而封装预定义的一系列方法，比如：模型结构的定义、Program的构建等。
+
+## 3. 剪裁卷积层通道
+
+### 3.1 计算剪裁之前的FLOPs
+
+```
+FLOPs = slim.analysis.flops(train_program)
+print("FLOPs: {}".format(FLOPs))
+```
+
+### 3.2 剪裁
+
+我们这里对参数名为`conv2_1_sep_weights`和`conv2_2_sep_weights`的卷积层进行剪裁，分别剪掉20%和30%的通道数。
+代码如下所示：
+
+```
+pruner = slim.prune.Pruner()
+pruned_program, _, _ = pruner.prune(
+        train_program,
+        fluid.global_scope(),
+        params=["conv2_1_sep_weights", "conv2_2_sep_weights"],
+        ratios=[0.33] * 2,
+        place=fluid.CPUPlace())
+```
+
+以上操作会修改`train_program`中对应卷积层参数的定义，同时对`fluid.global_scope()`中存储的参数数组进行裁剪。
+
+### 3.3 计算剪裁之后的FLOPs
+
+```
+FLOPs = paddleslim.analysis.flops(train_program)
+print("FLOPs: {}".format(FLOPs))
+```
+
+## 4. 训练剪裁后的模型
+
+### 4.1 定义输入数据
+
+为了快速执行该示例，我们选取简单的MNIST数据，Paddle框架的`paddle.dataset.mnist`包定义了MNIST数据的下载和读取。
+代码如下：
+
+```
+import paddle.dataset.mnist as reader
+train_reader = paddle.batch(
+        reader.train(), batch_size=128, drop_last=True)
+train_feeder = fluid.DataFeeder(inputs, fluid.CPUPlace())
+```
+
+### 4.2 执行训练
+以下代码执行了一个`epoch`的训练：
+
+```
+for data in train_reader():
+    acc1, acc5, loss = exe.run(pruned_program, feed=train_feeder.feed(data), fetch_list=outputs)
+    print(acc1, acc5, loss)
+```

_sources/quick_start/pruning_tutorial_en.md.txt

+# Channel Pruning for Image Classification
+
+In this tutorial, you will learn how to use channel pruning API of PaddleSlim
+by a demo of MobileNetV1 model on MNIST dataset. This tutorial following workflow:
+
+1. Import dependency
+2. Build model
+3. Prune model
+4. Train pruned model
+
+## 1. Import dependency
+
+PaddleSlim dependents on Paddle1.7. Please ensure that you have installed paddle correctly. Import Paddle and PaddleSlim as below:
+
+```
+import paddle
+import paddle.fluid as fluid
+import paddleslim as slim
+```
+
+## 2. Build Model
+
+This section will build a classsification model based `MobileNetV1` for MNIST task. The shape of the input is `[1, 28, 28]` and the output number is 10.
+
+To make the code simple, we define a function in package `paddleslim.models` to build classification model.
+Excute following code to build a model,
+
+```
+exe, train_program, val_program, inputs, outputs =
+    slim.models.image_classification("MobileNet", [1, 28, 28], 10, use_gpu=False)
+```
+
+>Note：The functions in paddleslim.models is just used in tutorials or demos.
+
+## 3. Prune model
+
+### 3.1 Compute FLOPs bofore pruning
+
+```
+FLOPs = slim.analysis.flops(train_program)
+print("FLOPs: {}".format(FLOPs))
+```
+
+### 3.2 Pruning
+
+The section will prune the parameters named `conv2_1_sep_weights` and `conv2_2_sep_weights` by 20% and 30%.
+
+```
+pruner = slim.prune.Pruner()
+pruned_program, _, _ = pruner.prune(
+        train_program,
+        fluid.global_scope(),
+        params=["conv2_1_sep_weights", "conv2_2_sep_weights"],
+        ratios=[0.33] * 2,
+        place=fluid.CPUPlace())
+```
+
+It will change the shapes of parameters defined in `train_program`. And the parameters` values stored in `fluid.global_scope()` will be pruned.
+
+
+### 3.3 Compute FLOPs after pruning
+
+```
+FLOPs = paddleslim.analysis.flops(train_program)
+print("FLOPs: {}".format(FLOPs))
+```
+
+## 4. Train pruned model
+
+### 4.1 Define dataset
+
+To make you easily run this demo, it will training on MNIST dataset. The package `paddle.dataset.mnist` of Paddle defines the downloading and reading of MNIST dataset.
+Define training data reader and test data reader as below：
+
+```
+import paddle.dataset.mnist as reader
+train_reader = paddle.batch(
+        reader.train(), batch_size=128, drop_last=True)
+train_feeder = fluid.DataFeeder(inputs, fluid.CPUPlace())
+```
+
+### 4.2 Training
+
+Excute following code to run an `epoch` training:
+
+```
+for data in train_reader():
+    acc1, acc5, loss = exe.run(pruned_program, feed=train_feeder.feed(data), fetch_list=outputs)
+    print(acc1, acc5, loss)
+```

_sources/quick_start/quant_aware_tutorial.md.txt

+# 图像分类模型量化训练-快速开始
+
+该教程以图像分类模型MobileNetV1为例，说明如何快速使用PaddleSlim的[量化训练接口](https://github.com/PaddlePaddle/PaddleSlim/blob/develop/docs/docs/api/quantization_api.md)。 该示例包含以下步骤：
+
+1. 导入依赖
+2. 构建模型
+3. 训练模型
+4. 量化
+5. 训练和测试量化后的模型
+6. 保存量化后的模型
+
+## 1. 导入依赖
+PaddleSlim依赖Paddle1.7版本，请确认已正确安装Paddle，然后按以下方式导入Paddle和PaddleSlim:
+
+
+```python
+import paddle
+import paddle.fluid as fluid
+import paddleslim as slim
+import numpy as np
+```
+
+## 2. 构建网络
+该章节构造一个用于对MNIST数据进行分类的分类模型，选用`MobileNetV1`，并将输入大小设置为`[1, 28, 28]`，输出类别数为10。               为了方便展示示例，我们在`paddleslim.models`下预定义了用于构建分类模型的方法，执行以下代码构建分类模型：
+
+>注意：paddleslim.models下的API并非PaddleSlim常规API，是为了简化示例而封装预定义的一系列方法，比如：模型结构的定义、Program的构建等。
+
+
+```python
+exe, train_program, val_program, inputs, outputs = \
+    slim.models.image_classification("MobileNet", [1, 28, 28], 10, use_gpu=True)
+```
+
+## 3. 训练模型
+该章节介绍了如何定义输入数据和如何训练和测试分类模型。先训练分类模型的原因是量化训练过程是在训练好的模型上进行的，也就是说是在训练好的模型的基础上加入量化反量化op之后，用小学习率进行参数微调。
+
+### 3.1 定义输入数据
+
+为了快速执行该示例，我们选取简单的MNIST数据，Paddle框架的`paddle.dataset.mnist`包定义了MNIST数据的下载和读取。
+代码如下：
+
+
+```python
+import paddle.dataset.mnist as reader
+train_reader = paddle.batch(
+        reader.train(), batch_size=128, drop_last=True)
+test_reader = paddle.batch(
+        reader.train(), batch_size=128, drop_last=True)
+train_feeder = fluid.DataFeeder(inputs, fluid.CPUPlace())
+```
+
+### 3.2 训练和测试
+先定义训练和测试函数，正常训练和量化训练时只需要调用函数即可。在训练函数中执行了一个epoch的训练，因为MNIST数据集数据较少，一个epoch就可将top1精度训练到95%以上。
+
+
+```python
+def train(prog):
+    iter = 0
+    for data in train_reader():
+        acc1, acc5, loss = exe.run(prog, feed=train_feeder.feed(data), fetch_list=outputs)
+        if iter % 100 == 0:
+            print('train iter={}, top1={}, top5={}, loss={}'.format(iter, acc1.mean(), acc5.mean(), loss.mean()))
+        iter += 1
+        
+def test(prog):
+    iter = 0
+    res = [[], []]
+    for data in train_reader():
+        acc1, acc5, loss = exe.run(prog, feed=train_feeder.feed(data), fetch_list=outputs)
+        if iter % 100 == 0:
+            print('test iter={}, top1={}, top5={}, loss={}'.format(iter, acc1.mean(), acc5.mean(), loss.mean()))
+        res[0].append(acc1.mean())
+        res[1].append(acc5.mean())
+        iter += 1
+    print('final test result top1={}, top5={}'.format(np.array(res[0]).mean(), np.array(res[1]).mean()))
+```
+
+调用``train``函数训练分类网络，``train_program``是在第2步：构建网络中定义的。
+
+
+```python
+train(train_program)
+```
+
+
+调用``test``函数测试分类网络，``val_program``是在第2步：构建网络中定义的。
+
+
+```python
+test(val_program)
+```
+
+
+## 4. 量化
+
+按照[默认配置](https://paddlepaddle.github.io/PaddleSlim/api/quantization_api/#_1)在``train_program``和``val_program``中加入量化和反量化op.
+
+
+```python
+quant_program = slim.quant.quant_aware(train_program, exe.place, for_test=False)
+val_quant_program = slim.quant.quant_aware(val_program, exe.place, for_test=True)
+```
+
+
+## 5. 训练和测试量化后的模型
+微调量化后的模型，训练一个epoch后测试。
+
+
+```python
+train(quant_program)
+```
+
+
+测试量化后的模型，和``3.2 训练和测试``中得到的测试结果相比，精度相近，达到了无损量化。
+
+
+```python
+test(val_quant_program)
+```
+
+
+## 6. 保存量化后的模型
+
+在``4. 量化``中使用接口``slim.quant.quant_aware``接口得到的模型只适合训练时使用，为了得到最终使用时的模型，需要使用[slim.quant.convert](https://paddlepaddle.github.io/PaddleSlim/api/quantization_api/#convert)接口，然后使用[fluid.io.save_inference_model](https://www.paddlepaddle.org.cn/documentation/docs/zh/develop/api_cn/io_cn/save_inference_model_cn.html#save-inference-model)保存模型。``float_prog``的参数数据类型是float32，但是数据范围是int8, 保存之后可使用fluid或者paddle-lite加载使用，paddle-lite在使用时，会先将类型转换为int8。``int8_prog``的参数数据类型是int8, 保存后可看到量化后模型大小，不可加载使用。
+
+
+```python
+float_prog, int8_prog = slim.quant.convert(val_quant_program, exe.place, save_int8=True)
+target_vars = [float_prog.global_block().var(name) for name in outputs]
+fluid.io.save_inference_model(dirname='./inference_model/float',
+        feeded_var_names=[var.name for var in inputs],
+        target_vars=target_vars,
+        executor=exe,
+        main_program=float_prog)
+fluid.io.save_inference_model(dirname='./inference_model/int8',
+        feeded_var_names=[var.name for var in inputs],
+        target_vars=target_vars,
+        executor=exe,
+        main_program=int8_prog)
+```

_sources/quick_start/quant_aware_tutorial_en.md.txt

+# Training-aware Quantization of image classification model - quick start
+
+This tutorial shows how to do training-aware quantization using [API](https://github.com/PaddlePaddle/PaddleSlim/blob/develop/docs/docs/api/quantization_api.md) in PaddleSlim. We use MobileNetV1 to train image classification model as example. The tutorial contains follow sections:
+
+1. Necessary imports
+2. Model architecture
+3. Train normal model
+4. Quantization
+5. Train model after quantization
+6. Save model after quantization
+
+## 1. Necessary imports
+PaddleSlim depends on Paddle1.7. Please make true that you have installed Paddle correctly. Then do the necessary imports:
+
+```python
+import paddle
+import paddle.fluid as fluid
+import paddleslim as slim
+import numpy as np
+```
+
+## 2. Model architecture
+
+The section constructs a classification model, which use ``MobileNetV1`` and MNIST dataset. The model's input size is `[1, 28, 28]` and output size is 10. In order to show tutorial conveniently, we pre-defined a method to get image classification model in `paddleslim.models`.
+
+>note: The APIs in `paddleslim.models` are not formal inferface in PaddleSlim. They are defined to simplify the tutorial such as the definition of model structure and the construction of Program.
+
+
+```python
+exe, train_program, val_program, inputs, outputs = \
+    slim.models.image_classification("MobileNet", [1, 28, 28], 10, use_gpu=True)
+```
+
+## 3. Train normal model
+
+The section shows how to define model inputs, train and test model. The reason for training the normal image classification model first is that the quantization model's training process is performed on the well-trained model. We add quantization and dequantization operators in well-trained model and finetune using smaller learning rate.
+
+### 3.1 input data definition
+
+To speed up training process, we select MNIST dataset to train image classification model. The API `paddle.dataset.mnist` in Paddle framework contains downloading and reading the images in dataset.
+
+```python
+import paddle.dataset.mnist as reader
+train_reader = paddle.batch(
+        reader.train(), batch_size=128, drop_last=True)
+test_reader = paddle.batch(
+        reader.train(), batch_size=128, drop_last=True)
+train_feeder = fluid.DataFeeder(inputs, fluid.CPUPlace())
+```
+
+### 3.2 training model and testing
+
+Define functions to train and test model. We only need call the functions when formal model training and quantization model training. The function does one epoch training because that MNIST dataset is small and top1 accuracy will reach 95% after one epoch.
+
+```python
+def train(prog):
+    iter = 0
+    for data in train_reader():
+        acc1, acc5, loss = exe.run(prog, feed=train_feeder.feed(data), fetch_list=outputs)
+        if iter % 100 == 0:
+            print('train iter={}, top1={}, top5={}, loss={}'.format(iter, acc1.mean(), acc5.mean(), loss.mean()))
+        iter += 1
+
+def test(prog):
+    iter = 0
+    res = [[], []]
+    for data in train_reader():
+        acc1, acc5, loss = exe.run(prog, feed=train_feeder.feed(data), fetch_list=outputs)
+        if iter % 100 == 0:
+            print('test iter={}, top1={}, top5={}, loss={}'.format(iter, acc1.mean(), acc5.mean(), loss.mean()))
+        res[0].append(acc1.mean())
+        res[1].append(acc5.mean())
+        iter += 1
+    print('final test result top1={}, top5={}'.format(np.array(res[0]).mean(), np.array(res[1]).mean()))
+```
+
+Call ``train`` function to train normal classification model. ``train_program`` is defined in 2. Model architecture.
+
+```python
+train(train_program)
+```
+
+Call ``test`` function to test normal classification model. ``val_program`` is defined in 2. Model architecture.
+
+```python
+test(val_program)
+```
+
+
+## 4. Quantization
+
+We call ``quant_aware`` API to add quantization and dequantization operators in ``train_program`` and ``val_program`` according to [default configuration](https://paddlepaddle.github.io/PaddleSlim/api/quantization_api/#_1).
+
+```python
+quant_program = slim.quant.quant_aware(train_program, exe.place, for_test=False)
+val_quant_program = slim.quant.quant_aware(val_program, exe.place, for_test=True)
+```
+
+
+## 5. Train model after quantization
+
+Finetune the model after quantization. Test model after one epoch training.
+
+```python
+train(quant_program)
+```
+
+Test model after quantization. The top1 and top5 accuracy are close to result in ``3.2 training model and testing``. We preform the training-aware quantization without loss on this image classification model.
+
+```python
+test(val_quant_program)
+```
+
+
+## 6. Save model after quantization
+
+The model in ``4. Quantization`` after calling ``slim.quant.quant_aware`` API is only suitable to train. To get the inference model, we should use [slim.quant.convert](https://paddlepaddle.github.io/PaddleSlim/api/quantization_api/#convert) API to change model architecture and use [fluid.io.save_inference_model](https://www.paddlepaddle.org.cn/documentation/docs/zh/develop/api_cn/io_cn/save_inference_model_cn.html#save-inference-model) to save model. ``float_prog``'s parameters are float32 dtype but in int8's range which can be used in ``fluid`` or ``paddle-lite``. ``paddle-lite`` will change the parameters' dtype from float32 to int8 first when loading the inference model. ``int8_prog``'s parameters are int8 dtype and we can get model size after quantization by saving it. ``int8_prog`` cannot be used in ``fluid`` or ``paddle-lite``.
+
+
+```python
+float_prog, int8_prog = slim.quant.convert(val_quant_program, exe.place, save_int8=True)
+target_vars = [float_prog.global_block().var(name) for name in outputs]
+fluid.io.save_inference_model(dirname='./inference_model/float',
+        feeded_var_names=[var.name for var in inputs],
+        target_vars=target_vars,
+        executor=exe,
+        main_program=float_prog)
+fluid.io.save_inference_model(dirname='./inference_model/int8',
+        feeded_var_names=[var.name for var in inputs],
+        target_vars=target_vars,
+        executor=exe,
+        main_program=int8_prog)
+```

_sources/quick_start/quant_post_tutorial.md.txt

+ # 图像分类模型离线量化-快速开始
+
+该教程以图像分类模型MobileNetV1为例，说明如何快速使用PaddleSlim的[离线量化接口](https://github.com/PaddlePaddle/PaddleSlim/blob/develop/docs/docs/api/quantization_api.md)。 该示例包含以下步骤：
+
+1. 导入依赖
+2. 构建模型
+3. 训练模型
+4. 离线量化
+
+## 1. 导入依赖
+PaddleSlim依赖Paddle1.7版本，请确认已正确安装Paddle，然后按以下方式导入Paddle和PaddleSlim:
+
+
+```python
+import paddle
+import paddle.fluid as fluid
+import paddleslim as slim
+import numpy as np
+```
+
+## 2. 构建网络
+该章节构造一个用于对MNIST数据进行分类的分类模型，选用`MobileNetV1`，并将输入大小设置为`[1, 28, 28]`，输出类别数为10。               为了方便展示示例，我们在`paddleslim.models`下预定义了用于构建分类模型的方法，执行以下代码构建分类模型：
+
+>注意：paddleslim.models下的API并非PaddleSlim常规API，是为了简化示例而封装预定义的一系列方法，比如：模型结构的定义、Program的构建等。
+
+
+```python
+exe, train_program, val_program, inputs, outputs = \
+    slim.models.image_classification("MobileNet", [1, 28, 28], 10, use_gpu=True)
+```
+
+## 3. 训练模型
+该章节介绍了如何定义输入数据和如何训练和测试分类模型。先训练分类模型的原因是离线量化需要一个训练好的模型。
+
+### 3.1 定义输入数据
+
+为了快速执行该示例，我们选取简单的MNIST数据，Paddle框架的`paddle.dataset.mnist`包定义了MNIST数据的下载和读取。
+代码如下：
+
+
+```python
+import paddle.dataset.mnist as reader
+train_reader = paddle.batch(
+        reader.train(), batch_size=128, drop_last=True)
+test_reader = paddle.batch(
+        reader.train(), batch_size=128, drop_last=True)
+train_feeder = fluid.DataFeeder(inputs, fluid.CPUPlace())
+```
+
+### 3.2 训练和测试
+先定义训练和测试函数。在训练函数中执行了一个epoch的训练，因为MNIST数据集数据较少，一个epoch就可将top1精度训练到95%以上。
+
+
+
+```python
+def train(prog):
+    iter = 0
+    for data in train_reader():
+        acc1, acc5, loss = exe.run(prog, feed=train_feeder.feed(data), fetch_list=outputs)
+        if iter % 100 == 0:
+            print('train', acc1.mean(), acc5.mean(), loss.mean())
+        iter += 1
+        
+def test(prog, outputs=outputs):
+    iter = 0
+    res = [[], []]
+    for data in train_reader():
+        acc1, acc5, loss = exe.run(prog, feed=train_feeder.feed(data), fetch_list=outputs)
+        if iter % 100 == 0:
+            print('test', acc1.mean(), acc5.mean(), loss.mean())
+        res[0].append(acc1.mean())
+        res[1].append(acc5.mean())
+        iter += 1
+    print('final test result', np.array(res[0]).mean(), np.array(res[1]).mean())
+```
+
+调用``train``函数训练分类网络，``train_program``是在第2步：构建网络中定义的。
+
+
+```python
+train(train_program)
+```
+
+
+调用``test``函数测试分类网络，``val_program``是在第2步：构建网络中定义的。
+
+
+```python
+test(val_program)
+```
+
+
+保存inference model，将训练好的分类模型保存在``'./inference_model'``下，后续进行离线量化时将加载保存在此处的模型。
+
+
+```python
+target_vars = [val_program.global_block().var(name) for name in outputs]
+fluid.io.save_inference_model(dirname='./inference_model',
+        feeded_var_names=[var.name for var in inputs],
+        target_vars=target_vars,
+        executor=exe,
+        main_program=val_program)
+```
+
+## 4. 离线量化
+
+调用离线量化接口，加载文件夹``'./inference_model'``训练好的分类模型，并使用10个batch的数据进行参数校正。此过程无需训练，只需跑前向过程来计算量化所需参数。离线量化后的模型保存在文件夹``'./quant_post_model'``下。
+
+
+```python
+slim.quant.quant_post(
+        executor=exe,
+        model_dir='./inference_model',
+        quantize_model_path='./quant_post_model',
+        sample_generator=reader.test(),
+        batch_nums=10)
+```
+
+
+加载保存在文件夹``'./quant_post_model'``下的量化后的模型进行测试，可看到精度和``3.2 训练和测试``中得到的测试精度相近，因此离线量化过程对于此分类模型几乎无损。
+
+
+```python
+quant_post_prog, feed_target_names, fetch_targets = fluid.io.load_inference_model(
+        dirname='./quant_post_model',
+        executor=exe)
+test(quant_post_prog, fetch_targets)
+```

_sources/quick_start/quant_post_tutorial_en.md.txt

+# Post-training Quantization of image classification model - quick start
+
+This tutorial shows how to do post training quantization using [API](https://github.com/PaddlePaddle/PaddleSlim/blob/develop/docs/docs/api/quantization_api.md) in PaddleSlim. We use MobileNetV1 to train image classification model as example. The tutorial contains follow sections:
+
+1. Necessary imports
+2. Model architecture
+3. Train normal model
+4. Post training quantization
+
+## 1. Necessary imports
+PaddleSlim depends on Paddle1.7. Please make true that you have installed Paddle correctly. Then do the necessary imports:
+
+
+```python
+import paddle
+import paddle.fluid as fluid
+import paddleslim as slim
+import numpy as np
+```
+## 2. Model architecture
+
+The section constructs a classification model, which use ``MobileNetV1`` and MNIST dataset. The model's input size is `[1, 28, 28]` and output size is 10. In order to show tutorial conveniently, we pre-defined a method to get image classification model in `paddleslim.models`.
+
+>note: The APIs in `paddleslim.models` are not formal inferface in PaddleSlim. They are defined to simplify the tutorial such as the definition of model structure and the construction of Program.
+
+
+```python
+exe, train_program, val_program, inputs, outputs = \
+    slim.models.image_classification("MobileNet", [1, 28, 28], 10, use_gpu=True)
+```
+
+## 3. Train normal model
+
+The section shows how to define model inputs, train and test model. The reason for training the normal image classification model first is that the post training quantization is performed on the well-trained model.
+
+### 3.1 input data definition
+
+To speed up training process, we select MNIST dataset to train image classification model. The API `paddle.dataset.mnist` in Paddle framework contains downloading and reading the images in dataset.
+
+```python
+import paddle.dataset.mnist as reader
+train_reader = paddle.batch(
+        reader.train(), batch_size=128, drop_last=True)
+test_reader = paddle.batch(
+        reader.train(), batch_size=128, drop_last=True)
+train_feeder = fluid.DataFeeder(inputs, fluid.CPUPlace())
+```
+
+### 3.2 training model and testing
+
+Define functions to train and test model. We only need call the functions when formal model training and quantization model training. The function does one epoch training because that MNIST dataset is small and top1 accuracy will reach 95% after one epoch.
+
+```python
+def train(prog):
+    iter = 0
+    for data in train_reader():
+        acc1, acc5, loss = exe.run(prog, feed=train_feeder.feed(data), fetch_list=outputs)
+        if iter % 100 == 0:
+            print('train', acc1.mean(), acc5.mean(), loss.mean())
+        iter += 1
+
+def test(prog, outputs=outputs):
+    iter = 0
+    res = [[], []]
+    for data in train_reader():
+        acc1, acc5, loss = exe.run(prog, feed=train_feeder.feed(data), fetch_list=outputs)
+        if iter % 100 == 0:
+            print('test', acc1.mean(), acc5.mean(), loss.mean())
+        res[0].append(acc1.mean())
+        res[1].append(acc5.mean())
+        iter += 1
+    print('final test result', np.array(res[0]).mean(), np.array(res[1]).mean())
+```
+
+Call ``train`` function to train normal classification model. ``train_program`` is defined in 2. Model architecture.
+
+
+```python
+train(train_program)
+```
+
+Call ``test`` function to test normal classification model. ``val_program`` is defined in 2. Model architecture.
+
+```python
+test(val_program)
+```
+
+
+Save inference model. Save well-trained model in ``'./inference_model'``. We will load the model when doing post training quantization.
+
+
+```python
+target_vars = [val_program.global_block().var(name) for name in outputs]
+fluid.io.save_inference_model(dirname='./inference_model',
+        feeded_var_names=[var.name for var in inputs],
+        target_vars=target_vars,
+        executor=exe,
+        main_program=val_program)
+```
+
+## 4. Post training quantization
+
+Call ``slim.quant.quant_post`` API to do post training quantization. The API will load the inference model in ``'./inference_model'`` first and calibrate the quantization parameters using data in sample_generator. In this tutorial, we use 10 mini-batch data to calibrate the quantization parameters. There is no need to train model but run forward to get activations for quantization scales calculation. The model after post training quantization are saved in ``'./quant_post_model'``.
+
+```python
+slim.quant.quant_post(
+        executor=exe,
+        model_dir='./inference_model',
+        quantize_model_path='./quant_post_model',
+        sample_generator=reader.test(),
+        batch_nums=10)
+```
+
+Load the model after post training quantization in ``'./quant_post_model'`` and run ``test`` function. The top1 and top5 accuracy are close to result in ``3.2 training model and testing``. We preform the post training quantization without loss on this image classification model.
+
+```python
+quant_post_prog, feed_target_names, fetch_targets = fluid.io.load_inference_model(
+        dirname='./quant_post_model',
+        executor=exe)
+test(quant_post_prog, fetch_targets)
+```

_sources/tutorials/image_classification_sensitivity_analysis_tutorial.md.txt

+#  图像分类模型通道剪裁-敏感度分析
+
+该教程以图像分类模型MobileNetV1为例，说明如何快速使用[PaddleSlim的敏感度分析接口](https://paddlepaddle.github.io/PaddleSlim/api/prune_api/#sensitivity)。
+该示例包含以下步骤：
+
+1. 导入依赖
+2. 构建模型
+3. 定义输入数据
+4. 定义模型评估方法
+5. 训练模型
+6. 获取待分析卷积参数名称
+7. 分析敏感度
+8. 剪裁模型
+
+以下章节依次介绍每个步骤的内容。
+
+## 1. 导入依赖
+
+PaddleSlim依赖Paddle1.7版本，请确认已正确安装Paddle，然后按以下方式导入Paddle和PaddleSlim:
+
+
+```python
+import paddle
+import paddle.fluid as fluid
+import paddleslim as slim
+```
+
+## 2. 构建网络
+
+该章节构造一个用于对MNIST数据进行分类的分类模型，选用`MobileNetV1`，并将输入大小设置为`[1, 28, 28]`，输出类别数为10。
+为了方便展示示例，我们在`paddleslim.models`下预定义了用于构建分类模型的方法，执行以下代码构建分类模型：
+
+
+```python
+exe, train_program, val_program, inputs, outputs = slim.models.image_classification("MobileNet", [1, 28, 28], 10, use_gpu=True)
+place = fluid.CUDAPlace(0)
+```
+
+## 3 定义输入数据
+
+为了快速执行该示例，我们选取简单的MNIST数据，Paddle框架的`paddle.dataset.mnist`包定义了MNIST数据的下载和读取。
+代码如下：
+
+
+```python
+import paddle.dataset.mnist as reader
+train_reader = paddle.batch(
+        reader.train(), batch_size=128, drop_last=True)
+test_reader = paddle.batch(
+        reader.test(), batch_size=128, drop_last=True)
+data_feeder = fluid.DataFeeder(inputs, place)
+```
+
+## 4. 定义模型评估方法
+
+在计算敏感度时，需要裁剪单个卷积层后的模型在测试数据上的效果，我们定义以下方法实现该功能：
+
+
+```python
+import numpy as np
+def test(program):
+    acc_top1_ns = []
+    acc_top5_ns = []
+    for data in test_reader():
+        acc_top1_n, acc_top5_n, _ = exe.run(
+            program,
+            feed=data_feeder.feed(data),
+            fetch_list=outputs)
+        acc_top1_ns.append(np.mean(acc_top1_n))
+        acc_top5_ns.append(np.mean(acc_top5_n))
+    print("Final eva - acc_top1: {}; acc_top5: {}".format(
+        np.mean(np.array(acc_top1_ns)), np.mean(np.array(acc_top5_ns))))
+    return np.mean(np.array(acc_top1_ns))
+```
+
+## 5. 训练模型
+
+只有训练好的模型才能做敏感度分析，因为该示例任务相对简单，我这里用训练一个`epoch`产出的模型做敏感度分析。对于其它训练比较耗时的模型，您可以加载训练好的模型权重。
+
+以下为模型训练代码：
+
+
+```python
+for data in train_reader():
+    acc1, acc5, loss = exe.run(train_program, feed=data_feeder.feed(data), fetch_list=outputs)
+print(np.mean(acc1), np.mean(acc5), np.mean(loss))
+```
+
+用上节定义的模型评估方法，评估当前模型在测试集上的精度：
+
+
+```python
+test(val_program)
+```
+
+## 6. 获取待分析卷积参数
+
+```python
+params = []
+for param in train_program.global_block().all_parameters():
+    if "_sep_weights" in param.name:
+        params.append(param.name)
+print(params)
+params = params[:5]
+```
+
+## 7. 分析敏感度
+
+### 7.1 简单计算敏感度
+
+调用[sensitivity接口](https://paddlepaddle.github.io/PaddleSlim/api/prune_api/#sensitivity)对训练好的模型进行敏感度分析。
+
+在计算过程中，敏感度信息会不断追加保存到选项`sensitivities_file`指定的文件中，该文件中已有的敏感度信息不会被重复计算。
+
+先用以下命令删除当前路径下可能已有的`sensitivities_0.data`文件:
+
+
+```python
+!rm -rf sensitivities_0.data
+```
+
+除了指定待分析的卷积层参数，我们还可以指定敏感度分析的粒度和范围，即单个卷积层参数分别被剪裁掉的比例。
+
+如果待分析的模型比较敏感，剪掉单个卷积层的40%的通道，模型在测试集上的精度损失就达90%，那么`pruned_ratios`最大设置到0.4即可，比如：
+`[0.1, 0.2, 0.3, 0.4]`
+
+为了得到更精确的敏感度信息，我可以适当调小`pruned_ratios`的粒度，比如：`[0.1, 0.15, 0.2, 0.25, 0.3, 0.35, 0.4]`
+
+`pruned_ratios`的粒度越小，计算敏感度的速度越慢。
+
+
+```python
+sens_0 = slim.prune.sensitivity(
+        val_program,
+        place,
+        params,
+        test,
+        sensitivities_file="sensitivities_0.data",
+        pruned_ratios=[0.1, 0.2])
+print(sens_0)
+```
+
+### 7.2 扩展敏感度信息
+
+第7.1节计算敏感度用的是`pruned_ratios=[0.1, 0.2]`, 我们可以在此基础上将其扩展到`[0.1, 0.2, 0.3]`
+
+
+```python
+sens_0 = slim.prune.sensitivity(
+        val_program,
+        place,
+        params,
+        test,
+        sensitivities_file="sensitivities_0.data",
+        pruned_ratios=[0.3])
+print(sens_0)
+```
+
+### 7.3 多进程加速计算敏感度信息
+
+敏感度分析所用时间取决于待分析的卷积层数量和模型评估的速度，我们可以通过多进程的方式加速敏感度计算。
+
+在不同的进程设置不同`pruned_ratios`, 然后将结果合并。
+
+#### 7.3.1 多进程计算敏感度
+
+在以上章节，我们计算了`pruned_ratios=[0.1, 0.2, 0.3]`的敏感度，并将其保存到了文件`sensitivities_0.data`中。
+
+在另一个进程中，我们可以设置`pruned_ratios=[0.4]`，并将结果保存在文件`sensitivities_1.data`中。代码如下：
+
+
+```python
+sens_1 = slim.prune.sensitivity(
+        val_program,
+        place,
+        params,
+        test,
+        sensitivities_file="sensitivities_1.data",
+        pruned_ratios=[0.4])
+print(sens_1)
+```
+
+#### 7.3.2 加载多个进程产出的敏感度文件
+
+```python
+s_0 = slim.prune.load_sensitivities("sensitivities_0.data")
+s_1 = slim.prune.load_sensitivities("sensitivities_1.data")
+print(s_0)
+print(s_1)
+```
+
+#### 7.3.3 合并敏感度信息
+
+
+```python
+s = slim.prune.merge_sensitive([s_0, s_1])
+print(s)
+```
+
+## 8. 剪裁模型
+
+根据以上章节产出的敏感度信息，对模型进行剪裁。
+
+### 8.1 计算剪裁率
+
+首先，调用PaddleSlim提供的[get_ratios_by_loss](https://paddlepaddle.github.io/PaddleSlim/api/prune_api/#get_ratios_by_loss)方法根据敏感度计算剪裁率，通过调整参数`loss`大小获得合适的一组剪裁率：
+
+
+```python
+loss = 0.01
+ratios = slim.prune.get_ratios_by_loss(s_0, loss)
+print(ratios)
+```
+
+### 8.2 剪裁训练网络
+
+
+```python
+pruner = slim.prune.Pruner()
+print("FLOPs before pruning: {}".format(slim.analysis.flops(train_program)))
+pruned_program, _, _ = pruner.prune(
+        train_program,
+        fluid.global_scope(),
+        params=ratios.keys(),
+        ratios=ratios.values(),
+        place=place)
+print("FLOPs after pruning: {}".format(slim.analysis.flops(pruned_program)))
+```
+
+### 8.3 剪裁测试网络
+
+>注意：对测试网络进行剪裁时，需要将`only_graph`设置为True，具体原因请参考[Pruner API文档](https://paddlepaddle.github.io/PaddleSlim/api/prune_api/#pruner)
+
+
+```python
+pruner = slim.prune.Pruner()
+print("FLOPs before pruning: {}".format(slim.analysis.flops(val_program)))
+pruned_val_program, _, _ = pruner.prune(
+        val_program,
+        fluid.global_scope(),
+        params=ratios.keys(),
+        ratios=ratios.values(),
+        place=place,
+        only_graph=True)
+print("FLOPs after pruning: {}".format(slim.analysis.flops(pruned_val_program)))
+```
+
+测试一下剪裁后的模型在测试集上的精度：
+
+```python
+test(pruned_val_program)
+```
+
+### 8.4 训练剪裁后的模型
+
+对剪裁后的模型在训练集上训练一个`epoch`:
+
+
+```python
+for data in train_reader():
+    acc1, acc5, loss = exe.run(pruned_program, feed=data_feeder.feed(data), fetch_list=outputs)
+print(np.mean(acc1), np.mean(acc5), np.mean(loss))
+```
+
+测试训练后模型的精度：
+
+```python
+test(pruned_val_program)
+```

_sources/tutorials/index.rst.txt

+
+进阶教程
+========
+
+.. toctree::
+   :maxdepth: 2
+   :caption: Contents:
+
+   image_classification_sensitivity_analysis_tutorial.md
+   image_classification_nas_quick_start.ipynb
+    

_sources/tutorials/index_en.rst.txt

+Aadvanced Tutorials
+========
+
+.. toctree::
+   :maxdepth: 1
+
+   sensitivity_tutorial_en.md
+    

_sources/tutorials/sensitivity_tutorial.md.txt

+#  图像分类模型通道剪裁-敏感度分析
+
+该教程以图像分类模型MobileNetV1为例，说明如何快速使用[PaddleSlim的敏感度分析接口](https://paddlepaddle.github.io/PaddleSlim/api/prune_api/#sensitivity)。
+该示例包含以下步骤：
+
+1. 导入依赖
+2. 构建模型
+3. 定义输入数据
+4. 定义模型评估方法
+5. 训练模型
+6. 获取待分析卷积参数名称
+7. 分析敏感度
+8. 剪裁模型
+
+以下章节依次次介绍每个步骤的内容。
+
+## 1. 导入依赖
+
+PaddleSlim依赖Paddle1.7版本，请确认已正确安装Paddle，然后按以下方式导入Paddle和PaddleSlim:
+
+```
+import paddle
+import paddle.fluid as fluid
+import paddleslim as slim
+```

_sources/tutorials/sensitivity_tutorial_en.md.txt

+# Pruning of image classification model - sensitivity
+
+该教程以图像分类模型MobileNetV1为例，说明如何快速使用[PaddleSlim的敏感度分析接口](https://paddlepaddle.github.io/PaddleSlim/api/prune_api/#sensitivity)。
+该示例包含以下步骤：
+
+1. 导入依赖
+2. 构建模型
+3. 定义输入数据
+4. 定义模型评估方法
+5. 训练模型
+6. 获取待分析卷积参数名称
+7. 分析敏感度
+8. 剪裁模型
+
+以下章节依次次介绍每个步骤的内容。
+
+## 1. 导入依赖
+
+PaddleSlim依赖Paddle1.7版本，请确认已正确安装Paddle，然后按以下方式导入Paddle和PaddleSlim:
+
+```
+import paddle
+import paddle.fluid as fluid
+import paddleslim as slim
+```

_static/css/badge_only.css

+.fa:before{-webkit-font-smoothing:antialiased}.clearfix{*zoom:1}.clearfix:before,.clearfix:after{display:table;content:""}.clearfix:after{clear:both}@font-face{font-family:FontAwesome;font-weight:normal;font-style:normal;src:url("../font/fontawesome_webfont.eot");src:url("../font/fontawesome_webfont.eot?#iefix") format("embedded-opentype"),url("../font/fontawesome_webfont.woff") format("woff"),url("../font/fontawesome_webfont.ttf") format("truetype"),url("../font/fontawesome_webfont.svg#FontAwesome") format("svg")}.fa:before{display:inline-block;font-family:FontAwesome;font-style:normal;font-weight:normal;line-height:1;text-decoration:inherit}a .fa{display:inline-block;text-decoration:inherit}li .fa{display:inline-block}li .fa-large:before,li .fa-large:before{width:1.875em}ul.fas{list-style-type:none;margin-left:2em;text-indent:-0.8em}ul.fas li .fa{width:0.8em}ul.fas li .fa-large:before,ul.fas li .fa-large:before{vertical-align:baseline}.fa-book:before{content:""}.icon-book:before{content:""}.fa-caret-down:before{content:""}.icon-caret-down:before{content:""}.fa-caret-up:before{content:""}.icon-caret-up:before{content:""}.fa-caret-left:before{content:""}.icon-caret-left:before{content:""}.fa-caret-right:before{content:""}.icon-caret-right:before{content:""}.rst-versions{position:fixed;bottom:0;left:0;width:300px;color:#fcfcfc;background:#1f1d1d;border-top:solid 10px #343131;font-family:"Lato","proxima-nova","Helvetica Neue",Arial,sans-serif;z-index:400}.rst-versions a{color:#2980B9;text-decoration:none}.rst-versions .rst-badge-small{display:none}.rst-versions .rst-current-version{padding:12px;background-color:#272525;display:block;text-align:right;font-size:90%;cursor:pointer;color:#27AE60;*zoom:1}.rst-versions .rst-current-version:before,.rst-versions .rst-current-version:after{display:table;content:""}.rst-versions .rst-current-version:after{clear:both}.rst-versions .rst-current-version .fa{color:#fcfcfc}.rst-versions .rst-current-version .fa-book{float:left}.rst-versions .rst-current-version .icon-book{float:left}.rst-versions .rst-current-version.rst-out-of-date{background-color:#E74C3C;color:#fff}.rst-versions .rst-current-version.rst-active-old-version{background-color:#F1C40F;color:#000}.rst-versions.shift-up .rst-other-versions{display:block}.rst-versions .rst-other-versions{font-size:90%;padding:12px;color:gray;display:none}.rst-versions .rst-other-versions hr{display:block;height:1px;border:0;margin:20px 0;padding:0;border-top:solid 1px #413d3d}.rst-versions .rst-other-versions dd{display:inline-block;margin:0}.rst-versions .rst-other-versions dd a{display:inline-block;padding:6px;color:#fcfcfc}.rst-versions.rst-badge{width:auto;bottom:20px;right:20px;left:auto;border:none;max-width:300px}.rst-versions.rst-badge .icon-book{float:none}.rst-versions.rst-badge .fa-book{float:none}.rst-versions.rst-badge.shift-up .rst-current-version{text-align:right}.rst-versions.rst-badge.shift-up .rst-current-version .fa-book{float:left}.rst-versions.rst-badge.shift-up .rst-current-version .icon-book{float:left}.rst-versions.rst-badge .rst-current-version{width:auto;height:30px;line-height:30px;padding:0 6px;display:block;text-align:center}@media screen and (max-width: 768px){.rst-versions{width:85%;display:none}.rst-versions.shift{display:block}img{width:100%;height:auto}}
+/*# sourceMappingURL=badge_only.css.map */

_static/js/modernizr.min.js

+/* Modernizr 2.6.2 (Custom Build) | MIT & BSD
+ * Build: http://modernizr.com/download/#-fontface-backgroundsize-borderimage-borderradius-boxshadow-flexbox-hsla-multiplebgs-opacity-rgba-textshadow-cssanimations-csscolumns-generatedcontent-cssgradients-cssreflections-csstransforms-csstransforms3d-csstransitions-applicationcache-canvas-canvastext-draganddrop-hashchange-history-audio-video-indexeddb-input-inputtypes-localstorage-postmessage-sessionstorage-websockets-websqldatabase-webworkers-geolocation-inlinesvg-smil-svg-svgclippaths-touch-webgl-shiv-mq-cssclasses-addtest-prefixed-teststyles-testprop-testallprops-hasevent-prefixes-domprefixes-load
+ */
+;window.Modernizr=function(a,b,c){function D(a){j.cssText=a}function E(a,b){return D(n.join(a+";")+(b||""))}function F(a,b){return typeof a===b}function G(a,b){return!!~(""+a).indexOf(b)}function H(a,b){for(var d in a){var e=a[d];if(!G(e,"-")&&j[e]!==c)return b=="pfx"?e:!0}return!1}function I(a,b,d){for(var e in a){var f=b[a[e]];if(f!==c)return d===!1?a[e]:F(f,"function")?f.bind(d||b):f}return!1}function J(a,b,c){var d=a.charAt(0).toUpperCase()+a.slice(1),e=(a+" "+p.join(d+" ")+d).split(" ");return F(b,"string")||F(b,"undefined")?H(e,b):(e=(a+" "+q.join(d+" ")+d).split(" "),I(e,b,c))}function K(){e.input=function(c){for(var d=0,e=c.length;d<e;d++)u[c[d]]=c[d]in k;return u.list&&(u.list=!!b.createElement("datalist")&&!!a.HTMLDataListElement),u}("autocomplete autofocus list placeholder max min multiple pattern required step".split(" ")),e.inputtypes=function(a){for(var d=0,e,f,h,i=a.length;d<i;d++)k.setAttribute("type",f=a[d]),e=k.type!=="text",e&&(k.value=l,k.style.cssText="position:absolute;visibility:hidden;",/^range$/.test(f)&&k.style.WebkitAppearance!==c?(g.appendChild(k),h=b.defaultView,e=h.getComputedStyle&&h.getComputedStyle(k,null).WebkitAppearance!=="textfield"&&k.offsetHeight!==0,g.removeChild(k)):/^(search|tel)$/.test(f)||(/^(url|email)$/.test(f)?e=k.checkValidity&&k.checkValidity()===!1:e=k.value!=l)),t[a[d]]=!!e;return t}("search tel url email datetime date month week time datetime-local number range color".split(" "))}var d="2.6.2",e={},f=!0,g=b.documentElement,h="modernizr",i=b.createElement(h),j=i.style,k=b.createElement("input"),l=":)",m={}.toString,n=" -webkit- -moz- -o- -ms- ".split(" "),o="Webkit Moz O ms",p=o.split(" "),q=o.toLowerCase().split(" "),r={svg:"http://www.w3.org/2000/svg"},s={},t={},u={},v=[],w=v.slice,x,y=function(a,c,d,e){var f,i,j,k,l=b.createElement("div"),m=b.body,n=m||b.createElement("body");if(parseInt(d,10))while(d--)j=b.createElement("div"),j.id=e?e[d]:h+(d+1),l.appendChild(j);return f=["&#173;",'<style id="s',h,'">',a,"</style>"].join(""),l.id=h,(m?l:n).innerHTML+=f,n.appendChild(l),m||(n.style.background="",n.style.overflow="hidden",k=g.style.overflow,g.style.overflow="hidden",g.appendChild(n)),i=c(l,a),m?l.parentNode.removeChild(l):(n.parentNode.removeChild(n),g.style.overflow=k),!!i},z=function(b){var c=a.matchMedia||a.msMatchMedia;if(c)return c(b).matches;var d;return y("@media "+b+" { #"+h+" { position: absolute; } }",function(b){d=(a.getComputedStyle?getComputedStyle(b,null):b.currentStyle)["position"]=="absolute"}),d},A=function(){function d(d,e){e=e||b.createElement(a[d]||"div"),d="on"+d;var f=d in e;return f||(e.setAttribute||(e=b.createElement("div")),e.setAttribute&&e.removeAttribute&&(e.setAttribute(d,""),f=F(e[d],"function"),F(e[d],"undefined")||(e[d]=c),e.removeAttribute(d))),e=null,f}var a={select:"input",change:"input",submit:"form",reset:"form",error:"img",load:"img",abort:"img"};return d}(),B={}.hasOwnProperty,C;!F(B,"undefined")&&!F(B.call,"undefined")?C=function(a,b){return B.call(a,b)}:C=function(a,b){return b in a&&F(a.constructor.prototype[b],"undefined")},Function.prototype.bind||(Function.prototype.bind=function(b){var c=this;if(typeof c!="function")throw new TypeError;var d=w.call(arguments,1),e=function(){if(this instanceof e){var a=function(){};a.prototype=c.prototype;var f=new a,g=c.apply(f,d.concat(w.call(arguments)));return Object(g)===g?g:f}return c.apply(b,d.concat(w.call(arguments)))};return e}),s.flexbox=function(){return J("flexWrap")},s.canvas=function(){var a=b.createElement("canvas");return!!a.getContext&&!!a.getContext("2d")},s.canvastext=function(){return!!e.canvas&&!!F(b.createElement("canvas").getContext("2d").fillText,"function")},s.webgl=function(){return!!a.WebGLRenderingContext},s.touch=function(){var c;return"ontouchstart"in a||a.DocumentTouch&&b instanceof DocumentTouch?c=!0:y(["@media (",n.join("touch-enabled),("),h,")","{#modernizr{top:9px;position:absolute}}"].join(""),function(a){c=a.offsetTop===9}),c},s.geolocation=function(){return"geolocation"in navigator},s.postmessage=function(){return!!a.postMessage},s.websqldatabase=function(){return!!a.openDatabase},s.indexedDB=function(){return!!J("indexedDB",a)},s.hashchange=function(){return A("hashchange",a)&&(b.documentMode===c||b.documentMode>7)},s.history=function(){return!!a.history&&!!history.pushState},s.draganddrop=function(){var a=b.createElement("div");return"draggable"in a||"ondragstart"in a&&"ondrop"in a},s.websockets=function(){return"WebSocket"in a||"MozWebSocket"in a},s.rgba=function(){return D("background-color:rgba(150,255,150,.5)"),G(j.backgroundColor,"rgba")},s.hsla=function(){return D("background-color:hsla(120,40%,100%,.5)"),G(j.backgroundColor,"rgba")||G(j.backgroundColor,"hsla")},s.multiplebgs=function(){return D("background:url(https://),url(https://),red url(https://)"),/(url\s*\(.*?){3}/.test(j.background)},s.backgroundsize=function(){return J("backgroundSize")},s.borderimage=function(){return J("borderImage")},s.borderradius=function(){return J("borderRadius")},s.boxshadow=function(){return J("boxShadow")},s.textshadow=function(){return b.createElement("div").style.textShadow===""},s.opacity=function(){return E("opacity:.55"),/^0.55$/.test(j.opacity)},s.cssanimations=function(){return J("animationName")},s.csscolumns=function(){return J("columnCount")},s.cssgradients=function(){var a="background-image:",b="gradient(linear,left top,right bottom,from(#9f9),to(white));",c="linear-gradient(left top,#9f9, white);";return D((a+"-webkit- ".split(" ").join(b+a)+n.join(c+a)).slice(0,-a.length)),G(j.backgroundImage,"gradient")},s.cssreflections=function(){return J("boxReflect")},s.csstransforms=function(){return!!J("transform")},s.csstransforms3d=function(){var a=!!J("perspective");return a&&"webkitPerspective"in g.style&&y("@media (transform-3d),(-webkit-transform-3d){#modernizr{left:9px;position:absolute;height:3px;}}",function(b,c){a=b.offsetLeft===9&&b.offsetHeight===3}),a},s.csstransitions=function(){return J("transition")},s.fontface=function(){var a;return y('@font-face {font-family:"font";src:url("https://")}',function(c,d){var e=b.getElementById("smodernizr"),f=e.sheet||e.styleSheet,g=f?f.cssRules&&f.cssRules[0]?f.cssRules[0].cssText:f.cssText||"":"";a=/src/i.test(g)&&g.indexOf(d.split(" ")[0])===0}),a},s.generatedcontent=function(){var a;return y(["#",h,"{font:0/0 a}#",h,':after{content:"',l,'";visibility:hidden;font:3px/1 a}'].join(""),function(b){a=b.offsetHeight>=3}),a},s.video=function(){var a=b.createElement("video"),c=!1;try{if(c=!!a.canPlayType)c=new Boolean(c),c.ogg=a.canPlayType('video/ogg; codecs="theora"').replace(/^no$/,""),c.h264=a.canPlayType('video/mp4; codecs="avc1.42E01E"').replace(/^no$/,""),c.webm=a.canPlayType('video/webm; codecs="vp8, vorbis"').replace(/^no$/,"")}catch(d){}return c},s.audio=function(){var a=b.createElement("audio"),c=!1;try{if(c=!!a.canPlayType)c=new Boolean(c),c.ogg=a.canPlayType('audio/ogg; codecs="vorbis"').replace(/^no$/,""),c.mp3=a.canPlayType("audio/mpeg;").replace(/^no$/,""),c.wav=a.canPlayType('audio/wav; codecs="1"').replace(/^no$/,""),c.m4a=(a.canPlayType("audio/x-m4a;")||a.canPlayType("audio/aac;")).replace(/^no$/,"")}catch(d){}return c},s.localstorage=function(){try{return localStorage.setItem(h,h),localStorage.removeItem(h),!0}catch(a){return!1}},s.sessionstorage=function(){try{return sessionStorage.setItem(h,h),sessionStorage.removeItem(h),!0}catch(a){return!1}},s.webworkers=function(){return!!a.Worker},s.applicationcache=function(){return!!a.applicationCache},s.svg=function(){return!!b.createElementNS&&!!b.createElementNS(r.svg,"svg").createSVGRect},s.inlinesvg=function(){var a=b.createElement("div");return a.innerHTML="<svg/>",(a.firstChild&&a.firstChild.namespaceURI)==r.svg},s.smil=function(){return!!b.createElementNS&&/SVGAnimate/.test(m.call(b.createElementNS(r.svg,"animate")))},s.svgclippaths=function(){return!!b.createElementNS&&/SVGClipPath/.test(m.call(b.createElementNS(r.svg,"clipPath")))};for(var L in s)C(s,L)&&(x=L.toLowerCase(),e[x]=s[L](),v.push((e[x]?"":"no-")+x));return e.input||K(),e.addTest=function(a,b){if(typeof a=="object")for(var d in a)C(a,d)&&e.addTest(d,a[d]);else{a=a.toLowerCase();if(e[a]!==c)return e;b=typeof b=="function"?b():b,typeof f!="undefined"&&f&&(g.className+=" "+(b?"":"no-")+a),e[a]=b}return e},D(""),i=k=null,function(a,b){function k(a,b){var c=a.createElement("p"),d=a.getElementsByTagName("head")[0]||a.documentElement;return c.innerHTML="x<style>"+b+"</style>",d.insertBefore(c.lastChild,d.firstChild)}function l(){var a=r.elements;return typeof a=="string"?a.split(" "):a}function m(a){var b=i[a[g]];return b||(b={},h++,a[g]=h,i[h]=b),b}function n(a,c,f){c||(c=b);if(j)return c.createElement(a);f||(f=m(c));var g;return f.cache[a]?g=f.cache[a].cloneNode():e.test(a)?g=(f.cache[a]=f.createElem(a)).cloneNode():g=f.createElem(a),g.canHaveChildren&&!d.test(a)?f.frag.appendChild(g):g}function o(a,c){a||(a=b);if(j)return a.createDocumentFragment();c=c||m(a);var d=c.frag.cloneNode(),e=0,f=l(),g=f.length;for(;e<g;e++)d.createElement(f[e]);return d}function p(a,b){b.cache||(b.cache={},b.createElem=a.createElement,b.createFrag=a.createDocumentFragment,b.frag=b.createFrag()),a.createElement=function(c){return r.shivMethods?n(c,a,b):b.createElem(c)},a.createDocumentFragment=Function("h,f","return function(){var n=f.cloneNode(),c=n.createElement;h.shivMethods&&("+l().join().replace(/\w+/g,function(a){return b.createElem(a),b.frag.createElement(a),'c("'+a+'")'})+");return n}")(r,b.frag)}function q(a){a||(a=b);var c=m(a);return r.shivCSS&&!f&&!c.hasCSS&&(c.hasCSS=!!k(a,"article,aside,figcaption,figure,footer,header,hgroup,nav,section{display:block}mark{background:#FF0;color:#000}")),j||p(a,c),a}var c=a.html5||{},d=/^<|^(?:button|map|select|textarea|object|iframe|option|optgroup)$/i,e=/^(?:a|b|code|div|fieldset|h1|h2|h3|h4|h5|h6|i|label|li|ol|p|q|span|strong|style|table|tbody|td|th|tr|ul)$/i,f,g="_html5shiv",h=0,i={},j;(function(){try{var a=b.createElement("a");a.innerHTML="<xyz></xyz>",f="hidden"in a,j=a.childNodes.length==1||function(){b.createElement("a");var a=b.createDocumentFragment();return typeof a.cloneNode=="undefined"||typeof a.createDocumentFragment=="undefined"||typeof a.createElement=="undefined"}()}catch(c){f=!0,j=!0}})();var r={elements:c.elements||"abbr article aside audio bdi canvas data datalist details figcaption figure footer header hgroup mark meter nav output progress section summary time video",shivCSS:c.shivCSS!==!1,supportsUnknownElements:j,shivMethods:c.shivMethods!==!1,type:"default",shivDocument:q,createElement:n,createDocumentFragment:o};a.html5=r,q(b)}(this,b),e._version=d,e._prefixes=n,e._domPrefixes=q,e._cssomPrefixes=p,e.mq=z,e.hasEvent=A,e.testProp=function(a){return H([a])},e.testAllProps=J,e.testStyles=y,e.prefixed=function(a,b,c){return b?J(a,b,c):J(a,"pfx")},g.className=g.className.replace(/(^|\s)no-js(\s|$)/,"$1$2")+(f?" js "+v.join(" "):""),e}(this,this.document),function(a,b,c){function d(a){return"[object Function]"==o.call(a)}function e(a){return"string"==typeof a}function f(){}function g(a){return!a||"loaded"==a||"complete"==a||"uninitialized"==a}function h(){var a=p.shift();q=1,a?a.t?m(function(){("c"==a.t?B.injectCss:B.injectJs)(a.s,0,a.a,a.x,a.e,1)},0):(a(),h()):q=0}function i(a,c,d,e,f,i,j){function k(b){if(!o&&g(l.readyState)&&(u.r=o=1,!q&&h(),l.onload=l.onreadystatechange=null,b)){"img"!=a&&m(function(){t.removeChild(l)},50);for(var d in y[c])y[c].hasOwnProperty(d)&&y[c][d].onload()}}var j=j||B.errorTimeout,l=b.createElement(a),o=0,r=0,u={t:d,s:c,e:f,a:i,x:j};1===y[c]&&(r=1,y[c]=[]),"object"==a?l.data=c:(l.src=c,l.type=a),l.width=l.height="0",l.onerror=l.onload=l.onreadystatechange=function(){k.call(this,r)},p.splice(e,0,u),"img"!=a&&(r||2===y[c]?(t.insertBefore(l,s?null:n),m(k,j)):y[c].push(l))}function j(a,b,c,d,f){return q=0,b=b||"j",e(a)?i("c"==b?v:u,a,b,this.i++,c,d,f):(p.splice(this.i++,0,a),1==p.length&&h()),this}function k(){var a=B;return a.loader={load:j,i:0},a}var l=b.documentElement,m=a.setTimeout,n=b.getElementsByTagName("script")[0],o={}.toString,p=[],q=0,r="MozAppearance"in l.style,s=r&&!!b.createRange().compareNode,t=s?l:n.parentNode,l=a.opera&&"[object Opera]"==o.call(a.opera),l=!!b.attachEvent&&!l,u=r?"object":l?"script":"img",v=l?"script":u,w=Array.isArray||function(a){return"[object Array]"==o.call(a)},x=[],y={},z={timeout:function(a,b){return b.length&&(a.timeout=b[0]),a}},A,B;B=function(a){function b(a){var a=a.split("!"),b=x.length,c=a.pop(),d=a.length,c={url:c,origUrl:c,prefixes:a},e,f,g;for(f=0;f<d;f++)g=a[f].split("="),(e=z[g.shift()])&&(c=e(c,g));for(f=0;f<b;f++)c=x[f](c);return c}function g(a,e,f,g,h){var i=b(a),j=i.autoCallback;i.url.split(".").pop().split("?").shift(),i.bypass||(e&&(e=d(e)?e:e[a]||e[g]||e[a.split("/").pop().split("?")[0]]),i.instead?i.instead(a,e,f,g,h):(y[i.url]?i.noexec=!0:y[i.url]=1,f.load(i.url,i.forceCSS||!i.forceJS&&"css"==i.url.split(".").pop().split("?").shift()?"c":c,i.noexec,i.attrs,i.timeout),(d(e)||d(j))&&f.load(function(){k(),e&&e(i.origUrl,h,g),j&&j(i.origUrl,h,g),y[i.url]=2})))}function h(a,b){function c(a,c){if(a){if(e(a))c||(j=function(){var a=[].slice.call(arguments);k.apply(this,a),l()}),g(a,j,b,0,h);else if(Object(a)===a)for(n in m=function(){var b=0,c;for(c in a)a.hasOwnProperty(c)&&b++;return b}(),a)a.hasOwnProperty(n)&&(!c&&!--m&&(d(j)?j=function(){var a=[].slice.call(arguments);k.apply(this,a),l()}:j[n]=function(a){return function(){var b=[].slice.call(arguments);a&&a.apply(this,b),l()}}(k[n])),g(a[n],j,b,n,h))}else!c&&l()}var h=!!a.test,i=a.load||a.both,j=a.callback||f,k=j,l=a.complete||f,m,n;c(h?a.yep:a.nope,!!i),i&&c(i)}var i,j,l=this.yepnope.loader;if(e(a))g(a,0,l,0);else if(w(a))for(i=0;i<a.length;i++)j=a[i],e(j)?g(j,0,l,0):w(j)?B(j):Object(j)===j&&h(j,l);else Object(a)===a&&h(a,l)},B.addPrefix=function(a,b){z[a]=b},B.addFilter=function(a){x.push(a)},B.errorTimeout=1e4,null==b.readyState&&b.addEventListener&&(b.readyState="loading",b.addEventListener("DOMContentLoaded",A=function(){b.removeEventListener("DOMContentLoaded",A,0),b.readyState="complete"},0)),a.yepnope=k(),a.yepnope.executeStack=h,a.yepnope.injectJs=function(a,c,d,e,i,j){var k=b.createElement("script"),l,o,e=e||B.errorTimeout;k.src=a;for(o in d)k.setAttribute(o,d[o]);c=j?h:c||f,k.onreadystatechange=k.onload=function(){!l&&g(k.readyState)&&(l=1,c(),k.onload=k.onreadystatechange=null)},m(function(){l||(l=1,c(1))},e),i?k.onload():n.parentNode.insertBefore(k,n)},a.yepnope.injectCss=function(a,c,d,e,g,i){var e=b.createElement("link"),j,c=i?h:c||f;e.href=a,e.rel="stylesheet",e.type="text/css";for(j in d)e.setAttribute(j,d[j]);g||(n.parentNode.insertBefore(e,n),m(c,0))}}(this,document),Modernizr.load=function(){yepnope.apply(window,[].slice.call(arguments,0))};

_static/pygments.css

+.highlight .hll { background-color: #ffffcc }
+.highlight  { background: #ffffff; }
\ No newline at end of file

_static/translations.js

+Documentation.addTranslations({"locale": "zh_Hans_CN", "messages": {"%(filename)s &#8212; %(docstitle)s": "", "&#169; <a href=\"%(path)s\">Copyright</a> %(copyright)s.": "", "&#169; Copyright %(copyright)s.": "", ", in ": "\uff0c\u5728", "About these documents": "\u5173\u4e8e\u8fd9\u4e9b\u6587\u6863", "Automatically generated list of changes in version %(version)s": "\u81ea\u52a8\u751f\u6210\u7684 %(version)s \u7248\u672c\u4e2d\u7684\u66f4\u6539\u5217\u8868", "C API changes": "C API \u66f4\u6539", "Changes in Version %(version)s &#8212; %(docstitle)s": "", "Collapse sidebar": "\u6298\u53e0\u8fb9\u680f", "Complete Table of Contents": "\u5b8c\u6574\u7684\u5185\u5bb9\u8868", "Contents": "\u76ee\u5f55", "Copyright": "\u7248\u6743\u6240\u6709", "Created using <a href=\"http://sphinx-doc.org/\">Sphinx</a> %(sphinx_version)s.": "\u7531 <a href=\"http://sphinx-doc.org/\">Sphinx</a> %(sphinx_version)s \u521b\u5efa\u3002", "Expand sidebar": "\u5c55\u5f00\u8fb9\u680f", "From here you can search these documents. Enter your search\n    words into the box below and click \"search\". Note that the search\n    function will automatically search for all of the words. Pages\n    containing fewer words won't appear in the result list.": "\u5728\u8fd9\u513f\uff0c\u4f60\u53ef\u4ee5\u5bf9\u8fd9\u4e9b\u6587\u6863\u8fdb\u884c\u641c\u7d22\u3002\u5411\u641c\u7d22\u6846\u4e2d\u8f93\u5165\u4f60\u6240\u8981\u641c\u7d22\u7684\u5173\u952e\u5b57\u5e76\u70b9\u51fb\u201c\u641c\u7d22\u201d\u3002\u6ce8\u610f\uff1a\u641c\u7d22\u5f15\u64ce\u4f1a\u81ea\u52a8\u641c\u7d22\u6240\u6709\u7684\u5173\u952e\u5b57\u3002\u5c06\u4e0d\u4f1a\u641c\u7d22\u5230\u90e8\u5206\u5173\u952e\u5b57\u7684\u9875\u9762.", "Full index on one page": "\u4e00\u9875\u7684\u5168\u90e8\u7d22\u5f15", "General Index": "\u603b\u76ee\u5f55", "Global Module Index": "\u5168\u5c40\u6a21\u5757\u7d22\u5f15", "Go": "\u8f6c\u5411", "Hide Search Matches": "\u9690\u85cf\u641c\u7d22\u7ed3\u679c", "Index": "\u7d22\u5f15", "Index &ndash; %(key)s": "\u7d22\u5f15 &ndash; %(key)s", "Index pages by letter": "\u6309\u7167\u5b57\u6bcd\u7684\u7d22\u5f15\u9875", "Indices and tables:": "\u7d22\u5f15\u548c\u8868\u683c\uff1a", "Last updated on %(last_updated)s.": "\u6700\u540e\u66f4\u65b0\u4e8e %(last_updated)s.", "Library changes": "\u5e93\u66f4\u6539", "Navigation": "\u5bfc\u822a", "Next topic": "\u4e0b\u4e00\u4e2a\u4e3b\u9898", "Other changes": "\u5176\u4ed6\u66f4\u6539", "Overview": "\u6982\u8ff0", "Permalink to this definition": "\u6c38\u4e45\u94fe\u63a5\u81f3\u76ee\u6807", "Permalink to this headline": "\u6c38\u4e45\u94fe\u63a5\u81f3\u6807\u9898", "Please activate JavaScript to enable the search\n    functionality.": "\u8bf7\u6fc0\u6d3b JavaScript \u4ee5\u5f00\u542f\u641c\u7d22\u529f\u80fd", "Preparing search...": "\u51c6\u5907\u641c\u7d22\u2026\u2026", "Previous topic": "\u4e0a\u4e00\u4e2a\u4e3b\u9898", "Quick search": "\u5feb\u901f\u641c\u7d22", "Search": "\u641c\u7d22", "Search Page": "\u641c\u7d22\u9875\u9762", "Search Results": "\u641c\u7d22\u7ed3\u679c", "Search finished, found %s page(s) matching the search query.": "\u641c\u7d22\u5b8c\u6210\uff0c\u6709 %s \u4e2a\u9875\u9762\u5339\u914d\u3002", "Search within %(docstitle)s": "\u5728 %(docstitle)s \u4e2d\u641c\u7d22", "Searching": "\u641c\u7d22\u4e2d", "Show Source": "\u663e\u793a\u6e90\u4ee3\u7801", "Table Of Contents": "\u5167\u5bb9\u76ee\u5f55", "This Page": "\u672c\u9875", "Welcome! This is": "\u6b22\u8fce\uff01\u8fd9\u662f", "Your search did not match any documents. Please make sure that all words are spelled correctly and that you've selected enough categories.": "\u6ca1\u6709\u4efb\u4f55\u6587\u6863\u5339\u914d\u60a8\u7684\u641c\u7d22\u3002\u8bf7\u786e\u4fdd\u4f60\u8f93\u5165\u7684\u8bcd\u62fc\u5199\u6b63\u786e\u5e76\u9009\u62e9\u4e86\u5408\u9002\u7684\u5206\u7c7b\u3002", "all functions, classes, terms": "\u6240\u7684\u51fd\u6570\uff0c\u7c7b\uff0c\u672f\u8bed", "can be huge": "\u53ef\u80fd\u4f1a\u5f88\u591a", "last updated": "\u6700\u540e\u66f4\u65b0\u4e8e", "lists all sections and subsections": "\u5217\u51fa\u6240\u6709\u7684\u7ae0\u8282\u548c\u90e8\u5206", "next chapter": "\u4e0b\u4e00\u7ae0", "previous chapter": "\u4e0a\u4e00\u7ae0", "quick access to all modules": "\u5feb\u901f\u67e5\u770b\u6240\u6709\u7684\u6a21\u5757", "search": "\u641c\u7d22", "search this documentation": "\u641c\u7d22\u6587\u6863", "the documentation for": "\u8fd9\u4efd\u6587\u6863\u662f"}, "plural_expr": "0"});
\ No newline at end of file