Merge branch 'dygraph' into sdmgr

doc/doc_ch/enhanced_ctc_loss.md

+# Enhanced CTC Loss
+
+在OCR识别中， CRNN是一种在工业界广泛使用的文字识别算法。 在训练阶段，其采用CTCLoss来计算网络损失； 在推理阶段，其采用CTCDecode来获得解码结果。虽然CRNN算法在实际业务中被证明能够获得很好的识别效果， 然而用户对识别准确率的要求却是无止境的，如何进一步提升文字识别的准确率呢？ 本文以CTCLoss为切人点，分别从难例挖掘、 多任务学习、 Metric Learning 3个不同的角度探索了CTCLoss的改进融合方案，提出了EnhancedCTCLoss，其包括如下3个组成部分： Focal-CTC Loss，A-CTC Loss， C-CTC Loss。
+
+## 1. Focal-CTC Loss
+Focal Loss 出自论文《Focal Loss for Dense Object Detection》, 该loss最先提出的时候主要是为了解决one-stage目标检测中正负样本比例严重失衡的问题。该损失函数降低了大量简单负样本在训练中所占的权重，也可理解为一种困难样本挖掘。
+其损失函数形式如下：
+<div align="center"> 
+<img src="./focal_loss_formula.png" width = "600" /> 
+</div>
+ 
+其中,  y' 是经过激活函数的输出，取值在0-1之间。其在原始的交叉熵损失的基础上加了一个调制系数（1 – y’)^ &gamma;和平衡因子&alpha;。 当&alpha; = 1，y=1时，其损失函数与交叉熵损失的对比如下图所示:   
+<div align="center"> 
+<img src="./focal_loss_image.png" width = "600" /> 
+</div>
+
+从上图可以看到, 当&gamma;> 0时，调整系数（1-y’）^&gamma; 赋予易分类样本损失一个更小的权重，使得网络更关注于困难的、错分的样本。 调整因子&gamma;用于调节简单样本权重降低的速率，当&gamma;为0时即为交叉熵损失函数，当&gamma;增加时，调整因子的影响也会随之增大。实验发现&gamma;为2是最优。平衡因子&alpha;用来平衡正负样本本身的比例不均，文中&alpha;取0.25。
+
+对于经典的CTC算法，假设某个特征序列（f<sub>1</sub>, f<sub>2</sub>, ......f<sub>t</sub>), 经过CTC解码之后结果等于label的概率为y’, 则CTC解码结果不为label的概率即为（1-y’)；不难发现 CTCLoss值和y’有如下关系：
+<div align="center"> 
+<img src="./equation_ctcloss.png" width = "250" /> 
+</div>
+
+结合Focal Loss的思想，赋予困难样本较大的权重，简单样本较小的权重，可以使网络更加聚焦于对困难样本的挖掘，进一步提升识别的准确率，由此我们提出了Focal-CTC Loss； 其定义如下所示：
+<div align="center"> 
+<img src="./equation_focal_ctc.png" width = "500" /> 
+</div>
+
+实验中，&gamma;取值为2, &alpha;= 1, 具体实现见:  [rec_ctc_loss.py](../../ppocr/losses/rec_ctc_loss.py)
+
+## 2. A-CTC Loss
+A-CTC Loss是CTC Loss + ACE Loss的简称。 其中ACE Loss出自论文< Aggregation Cross-Entropy for Sequence Recognition>.  ACE Loss相比于CTCLoss，主要有如下两点优势: 
++ ACE Loss能够解决2-D文本的识别问题;  CTCLoss只能够处理1-D文本
++ ACE Loss 在时间复杂度和空间复杂度上优于CTC loss
+
+前人总结的OCR识别算法的优劣如下图所示：
+<div align="center">
+<img src="./rec_algo_compare.png" width = "1000" /> 
+</div>
+ 
+虽然ACELoss确实如上图所说，可以处理2D预测，在内存占用及推理速度方面具备优势，但在实践过程中，我们发现单独使用ACE Loss,  识别效果并不如CTCLoss.  因此，我们尝试将CTCLoss和ACELoss进行组合，同时以CTCLoss为主，将ACELoss 定位为一个辅助监督loss。 这一尝试收到了效果，在我们内部的实验数据集上，相比单独使用CTCLoss，识别准确率可以提升1%左右。
+A_CTC Loss定义如下:  
+<div align="center">
+<img src="./equation_a_ctc.png" width = "300" /> 
+</div>
+
+实验中，λ = 0.1.  ACE loss实现代码见:  [ace_loss.py](../../ppocr/losses/ace_loss.py)
+
+## 3. C-CTC Loss
+C-CTC Loss是CTC Loss + Center Loss的简称。 其中Center Loss出自论文 < A Discriminative Feature Learning Approach for Deep Face Recognition>.  最早用于人脸识别任务，用于增大累间距离，减小类内距离,  是Metric Learning领域一种较早的、也比较常用的一种算法。 
+在中文OCR识别任务中，通过对badcase分析， 我们发现中文识别的一大难点是相似字符多，容易误识。 由此我们想到是否可以借鉴Metric Learing的想法， 增大相似字符的类间距，从而提高识别准确率。然而，MetricLearning主要用于图像识别领域，训练数据的标签为一个固定的值；而对于OCR识别来说，其本质上是一个序列识别任务，特征和label之间并不具有显式的对齐关系，因此两者如何结合依然是一个值得探索的方向。
+通过尝试Arcmargin, Cosmargin等方法， 我们最终发现Centerloss 有助于进一步提升识别的准确率。C_CTC Loss定义如下：
+<div align="center">
+<img src="./equation_c_ctc.png" width = "300" /> 
+</div>
+
+实验中，我们设置λ=0.25. center_loss实现代码见:  [center_loss.py](../../ppocr/losses/center_loss.py)
+
+值得一提的是， 在C-CTC Loss中，选择随机初始化Center并不能够带来明显的提升. 我们的Center初始化方法如下：
++ 基于原始的CTCLoss， 训练得到一个网络N
++ 挑选出训练集中，识别完全正确的部分, 组成集合G
++ 将G中的每个样本送入网络，进行前向计算， 提取最后一个FC层的输入（即feature）及其经过argmax计算的结果（即index）之间的对应关系
++ 将相同index的feature进行聚合，计算平均值，得到各自字符的初始center. 
+
+以配置文件`configs/rec/ch_PP-OCRv2/ch_PP-OCRv2_rec.yml`为例， center提取命令如下所示:
+```
+python tools/export_center.py -c configs/rec/ch_PP-OCRv2/ch_PP-OCRv2_rec.yml -o  Global.pretrained_model: "./output/rec_mobile_pp-OCRv2/best_accuracy"
+```
+运行完后，会在PaddleOCR主目录下生成`train_center.pkl`.
+
+## 4. 实验
+对于上述的三种方案，我们基于百度内部数据集进行了训练、评测，实验情况如下表所示：
+|algorithm| Focal_CTC | A_CTC | C-CTC |
+|:------| :------| ------: | :------: |
+|gain| +0.3% | +0.7% | +1.7% | 
+
+基于上述实验结论，我们在PP-OCRv2中，采用了C-CTC的策略。 值得一提的是，由于PP-OCRv2 处理的是6625个中文字符的识别任务，字符集比较大，形似字较多，所以在该任务上C-CTC 方案带来的提升较大。 但如果换做其他OCR识别任务，结论可能会有所不同。大家可以尝试Focal-CTC，A-CTC, C-CTC以及组合方案EnhancedCTC，相信会带来不同程度的提升效果。
+统一的融合方案见如下文件：  [rec_enhanced_ctc_loss.py](../../ppocr/losses/rec_enhanced_ctc_loss.py)

ppocr/losses/rec_ctc_loss.py

@@ -38,7 +38,7 @@ class CTCLoss(nn.Layer):
         if self.use_focal_loss:
             weight = paddle.exp(-loss)
             weight = paddle.subtract(paddle.to_tensor([1.0]), weight)
-            weight = paddle.square(weight) * self.focal_loss_alpha
+            weight = paddle.square(weight)
             loss = paddle.multiply(loss, weight)
-        loss = loss.mean()  # sum
+        loss = loss.mean()
         return {'loss': loss}

ppocr/losses/rec_enhanced_ctc_loss.py

+# copyright (c) 2021 PaddlePaddle Authors. All Rights Reserve.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from __future__ import absolute_import
+from __future__ import division
+from __future__ import print_function
+
+import paddle
+from paddle import nn
+from .ace_loss import ACELoss
+from .center_loss import CenterLoss
+from .rec_ctc_loss import CTCLoss
+
+
+class EnhancedCTCLoss(nn.Layer):
+    def __init__(self,
+                 use_focal_loss=False,
+                 use_ace_loss=False,
+                 ace_loss_weight=0.1,
+                 use_center_loss=False,
+                 center_loss_weight=0.05,
+                 num_classes=6625,
+                 feat_dim=96,
+                 init_center=False,
+                 center_file_path=None,
+                 **kwargs):
+        super(EnhancedCTCLoss, self).__init__()
+        self.ctc_loss_func = CTCLoss(use_focal_loss=use_focal_loss)
+
+        self.use_ace_loss = False
+        if use_ace_loss:
+            self.use_ace_loss = use_ace_loss
+            self.ace_loss_func = ACELoss()
+            self.ace_loss_weight = ace_loss_weight
+
+        self.use_center_loss = False
+        if use_center_loss:
+            self.use_center_loss = use_center_loss
+            self.center_loss_func = CenterLoss(
+                num_classes=num_classes,
+                feat_dim=feat_dim,
+                init_center=init_center,
+                center_file_path=center_file_path)
+            self.center_loss_weight = center_loss_weight
+
+    def __call__(self, predicts, batch):
+        loss = self.ctc_loss_func(predicts, batch)["loss"]
+
+        if self.use_center_loss:
+            center_loss = self.center_loss_func(
+                predicts, batch)["loss_center"] * self.center_loss_weight
+            loss = loss + center_loss
+
+        if self.use_ace_loss:
+            ace_loss = self.ace_loss_func(
+                predicts, batch)["loss_ace"] * self.ace_loss_weight
+            loss = loss + ace_loss
+
+        return {'enhanced_ctc_loss': loss}

tests/ocr_det_params.txt → tests/configs/ppocr_det_mobile_params.txt

@@ -40,13 +40,13 @@ infer_quant:False
 inference:tools/infer/predict_det.py
 --use_gpu:True|False
 --enable_mkldnn:True|False
---cpu_threads:6
+--cpu_threads:1|6
 --rec_batch_num:1
 --use_tensorrt:False|True
 --precision:fp32|fp16|int8
 --det_model_dir:
 --image_dir:./inference/ch_det_data_50/all-sum-510/
---save_log_path:null
+null:null
 --benchmark:True
 null:null
 ===========================cpp_infer_params===========================
@@ -79,4 +79,20 @@ op.det.local_service_conf.thread_num:1|6
 op.det.local_service_conf.use_trt:False|True
 op.det.local_service_conf.precision:fp32|fp16|int8
 pipline:pipeline_http_client.py --image_dir=../../doc/imgs
-
+===========================kl_quant_params===========================
+infer_model:./inference/ch_ppocr_mobile_v2.0_det_infer/
+infer_export:tools/export_model.py -c configs/det/ch_ppocr_v2.0/ch_det_mv3_db_v2.0.yml -o
+infer_quant:False
+inference:tools/infer/predict_det.py
+--use_gpu:True|False
+--enable_mkldnn:True|False
+--cpu_threads:1|6
+--rec_batch_num:1
+--use_tensorrt:False|True
+--precision:fp32|fp16|int8
+--det_model_dir:
+--image_dir:./inference/ch_det_data_50/all-sum-510/
+null:null
+--benchmark:True
+null:null
+null:null
\ No newline at end of file

tests/ocr_det_server_params.txt → tests/configs/ppocr_det_server_params.txt

@@ -12,10 +12,10 @@ train_model_name:latest
 train_infer_img_dir:./train_data/icdar2015/text_localization/ch4_test_images/
 null:null
 ##
-trainer:norm_train|pact_train
-norm_train:tools/train.py -c tests/configs/det_r50_vd_db.yml -o Global.pretrained_model=""
-pact_train:null
-fpgm_train:null
+trainer:norm_train|pact_train|fpgm_export
+norm_train:tools/train.py -c tests/configs/det_r50_vd_db.yml -o 
+quant_export:deploy/slim/quantization/export_model.py -c tests/configs/det_r50_vd_db.yml -o 
+fpgm_export:deploy/slim/prune/export_prune_model.py -c tests/configs/det_r50_vd_db.yml -o 
 distill_train:null
 null:null
 null:null
@@ -34,8 +34,8 @@ distill_export:null
 export1:null
 export2:null
 ##
-infer_model:./inference/ch_ppocr_server_v2.0_det_infer/
-infer_export:null
+train_model:./inference/ch_ppocr_server_v2.0_det_train/best_accuracy
+infer_export:tools/export_model.py -c configs/det/ch_ppocr_v2.0/ch_det_res18_db_v2.0.yml -o
 infer_quant:False
 inference:tools/infer/predict_det.py
 --use_gpu:True|False

tests/ocr_kl_quant_params.txt

-===========================train_params===========================
-model_name:ocr_system
-python:python3.7
-gpu_list:null
-Global.use_gpu:null
-Global.auto_cast:null
-Global.epoch_num:null
-Global.save_model_dir:./output/
-Train.loader.batch_size_per_card:null
-Global.pretrained_model:null
-train_model_name:null
-train_infer_img_dir:null
-null:null
-##
-trainer:
-norm_train:null
-pact_train:null
-fpgm_train:null
-distill_train:null
-null:null
-null:null
-##
-===========================eval_params=========================== 
-eval:null
-null:null
-##
-===========================infer_params===========================
-Global.save_inference_dir:./output/
-Global.pretrained_model:
-norm_export:null 
-quant_export:null 
-fpgm_export:null 
-distill_export:null
-export1:null
-export2:null
-##
-infer_model:./inference/ch_ppocr_mobile_v2.0_det_infer/
-kl_quant:deploy/slim/quantization/quant_kl.py -c configs/det/ch_ppocr_v2.0/ch_det_mv3_db_v2.0.yml  -o  
-infer_quant:True
-inference:tools/infer/predict_det.py
---use_gpu:TrueFalse
---enable_mkldnn:True|False
---cpu_threads:1|6
---rec_batch_num:1
---use_tensorrt:False|True
---precision:fp32|fp16|int8
---det_model_dir:
---image_dir:./inference/ch_det_data_50/all-sum-510/
---save_log_path:null
---benchmark:True
-null:null

tests/prepare.sh

 #!/bin/bash
 FILENAME=$1
 
-# MODE be one of ['lite_train_infer' 'whole_infer' 'whole_train_infer', 'infer', 'cpp_infer', 'serving_infer']
+# MODE be one of ['lite_train_infer' 'whole_infer' 'whole_train_infer',  'infer', 
+#                 'cpp_infer', 'serving_infer', 'klquant_infer']
+
 MODE=$2
 
 dataline=$(cat ${FILENAME})
@@ -72,9 +74,9 @@ elif [ ${MODE} = "infer" ];then
         wget -nc  -P ./inference https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_mobile_v2.0_det_train.tar
         cd ./inference && tar xf ${eval_model_name}.tar && tar xf ch_det_data_50.tar && cd ../
     elif [ ${model_name} = "ocr_server_det" ]; then
-        wget -nc -P ./inference https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_server_v2.0_det_infer.tar
+        wget -nc -P ./inference https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_server_v2.0_det_train.tar
         wget -nc -P ./inference https://paddleocr.bj.bcebos.com/dygraph_v2.0/test/ch_det_data_50.tar
-        cd ./inference && tar xf ch_ppocr_server_v2.0_det_infer.tar && tar xf ch_det_data_50.tar && cd ../
+        cd ./inference && tar xf ch_ppocr_server_v2.0_det_train.tar && tar xf ch_det_data_50.tar && cd ../
     elif  [ ${model_name} = "ocr_system_mobile" ]; then
         wget -nc  -P ./inference https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_mobile_v2.0_det_infer.tar
         wget -nc -P ./inference https://paddleocr.bj.bcebos.com/dygraph_v2.0/test/ch_det_data_50.tar
@@ -98,6 +100,12 @@ elif [ ${MODE} = "infer" ];then
         wget -nc  -P ./inference https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_server_v2.0_rec_infer.tar
         cd ./inference && tar xf ${eval_model_name}.tar && tar xf rec_inference.tar && cd ../
     fi 
+elif [ ${MODE} = "klquant_infer" ];then
+    if [ ${model_name} = "ocr_det" ]; then
+        wget -nc  -P ./inference https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_mobile_v2.0_det_infer.tar
+        wget -nc -P ./inference https://paddleocr.bj.bcebos.com/dygraph_v2.0/test/ch_det_data_50.tar
+        cd ./inference && tar xf ch_ppocr_mobile_v2.0_det_infer.tar && tar xf ch_det_data_50.tar && cd ../
+    fi
 elif [ ${MODE} = "cpp_infer" ];then
     if [ ${model_name} = "ocr_det" ]; then
         wget -nc -P ./inference https://paddleocr.bj.bcebos.com/dygraph_v2.0/test/ch_det_data_50.tar

tests/test.sh

 #!/bin/bash
 FILENAME=$1
-# MODE be one of ['lite_train_infer' 'whole_infer' 'whole_train_infer', 'infer', 'cpp_infer']
+# MODE be one of ['lite_train_infer' 'whole_infer' 'whole_train_infer', 'infer', 'cpp_infer', 'serving_infer', 'klquant_infer']
 MODE=$2
-
-dataline=$(cat ${FILENAME})
+if [ ${MODE} = "cpp_infer" ]; then
+    dataline=$(awk 'NR==67, NR==81{print}'  $FILENAME)
+elif [ ${MODE} = "serving_infer" ]; then
+    dataline=$(awk 'NR==52, NR==66{print}'  $FILENAME)
+elif [ ${MODE} = "klquant_infer" ]; then
+    dataline=$(awk 'NR==82, NR==98{print}'  $FILENAME)
+else
+    dataline=$(awk 'NR==1, NR==51{print}'  $FILENAME)
+fi
 
 # parser params
 IFS=$'\n'
@@ -144,61 +151,93 @@ benchmark_key=$(func_parser_key "${lines[49]}")
 benchmark_value=$(func_parser_value "${lines[49]}")
 infer_key1=$(func_parser_key "${lines[50]}")
 infer_value1=$(func_parser_value "${lines[50]}")
-# parser serving
-trans_model_py=$(func_parser_value "${lines[67]}")
-infer_model_dir_key=$(func_parser_key "${lines[68]}")
-infer_model_dir_value=$(func_parser_value "${lines[68]}")
-model_filename_key=$(func_parser_key "${lines[69]}")
-model_filename_value=$(func_parser_value "${lines[69]}")
-params_filename_key=$(func_parser_key "${lines[70]}")
-params_filename_value=$(func_parser_value "${lines[70]}")
-serving_server_key=$(func_parser_key "${lines[71]}")
-serving_server_value=$(func_parser_value "${lines[71]}")
-serving_client_key=$(func_parser_key "${lines[72]}")
-serving_client_value=$(func_parser_value "${lines[72]}")
-serving_dir_value=$(func_parser_value "${lines[73]}")
-web_service_py=$(func_parser_value "${lines[74]}")
-web_use_gpu_key=$(func_parser_key "${lines[75]}")
-web_use_gpu_list=$(func_parser_value "${lines[75]}")
-web_use_mkldnn_key=$(func_parser_key "${lines[76]}")
-web_use_mkldnn_list=$(func_parser_value "${lines[76]}")
-web_cpu_threads_key=$(func_parser_key "${lines[77]}")
-web_cpu_threads_list=$(func_parser_value "${lines[77]}")
-web_use_trt_key=$(func_parser_key "${lines[78]}")
-web_use_trt_list=$(func_parser_value "${lines[78]}")
-web_precision_key=$(func_parser_key "${lines[79]}")
-web_precision_list=$(func_parser_value "${lines[79]}")
-pipeline_py=$(func_parser_value "${lines[80]}")
 
+# parser serving
+if [ ${MODE} = "klquant_infer" ]; then
+    # parser inference model 
+    infer_model_dir_list=$(func_parser_value "${lines[1]}")
+    infer_export_list=$(func_parser_value "${lines[2]}")
+    infer_is_quant=$(func_parser_value "${lines[3]}")
+    # parser inference 
+    inference_py=$(func_parser_value "${lines[4]}")
+    use_gpu_key=$(func_parser_key "${lines[5]}")
+    use_gpu_list=$(func_parser_value "${lines[5]}")
+    use_mkldnn_key=$(func_parser_key "${lines[6]}")
+    use_mkldnn_list=$(func_parser_value "${lines[6]}")
+    cpu_threads_key=$(func_parser_key "${lines[7]}")
+    cpu_threads_list=$(func_parser_value "${lines[7]}")
+    batch_size_key=$(func_parser_key "${lines[8]}")
+    batch_size_list=$(func_parser_value "${lines[8]}")
+    use_trt_key=$(func_parser_key "${lines[9]}")
+    use_trt_list=$(func_parser_value "${lines[9]}")
+    precision_key=$(func_parser_key "${lines[10]}")
+    precision_list=$(func_parser_value "${lines[10]}")
+    infer_model_key=$(func_parser_key "${lines[11]}")
+    image_dir_key=$(func_parser_key "${lines[12]}")
+    infer_img_dir=$(func_parser_value "${lines[12]}")
+    save_log_key=$(func_parser_key "${lines[13]}")
+    benchmark_key=$(func_parser_key "${lines[14]}")
+    benchmark_value=$(func_parser_value "${lines[14]}")
+    infer_key1=$(func_parser_key "${lines[15]}")
+    infer_value1=$(func_parser_value "${lines[15]}")
+fi
+# parser serving
+if [ ${MODE} = "server_infer" ]; then
+    trans_model_py=$(func_parser_value "${lines[1]}")
+    infer_model_dir_key=$(func_parser_key "${lines[2]}")
+    infer_model_dir_value=$(func_parser_value "${lines[2]}")
+    model_filename_key=$(func_parser_key "${lines[3]}")
+    model_filename_value=$(func_parser_value "${lines[3]}")
+    params_filename_key=$(func_parser_key "${lines[4]}")
+    params_filename_value=$(func_parser_value "${lines[4]}")
+    serving_server_key=$(func_parser_key "${lines[5]}")
+    serving_server_value=$(func_parser_value "${lines[5]}")
+    serving_client_key=$(func_parser_key "${lines[6]}")
+    serving_client_value=$(func_parser_value "${lines[6]}")
+    serving_dir_value=$(func_parser_value "${lines[7]}")
+    web_service_py=$(func_parser_value "${lines[8]}")
+    web_use_gpu_key=$(func_parser_key "${lines[9]}")
+    web_use_gpu_list=$(func_parser_value "${lines[9]}")
+    web_use_mkldnn_key=$(func_parser_key "${lines[10]}")
+    web_use_mkldnn_list=$(func_parser_value "${lines[10]}")
+    web_cpu_threads_key=$(func_parser_key "${lines[11]}")
+    web_cpu_threads_list=$(func_parser_value "${lines[11]}")
+    web_use_trt_key=$(func_parser_key "${lines[12]}")
+    web_use_trt_list=$(func_parser_value "${lines[12]}")
+    web_precision_key=$(func_parser_key "${lines[13]}")
+    web_precision_list=$(func_parser_value "${lines[13]}")
+    pipeline_py=$(func_parser_value "${lines[14]}")
+fi
 
 if [ ${MODE} = "cpp_infer" ]; then
     # parser cpp inference model 
-    cpp_infer_model_dir_list=$(func_parser_value "${lines[53]}")
-    cpp_infer_is_quant=$(func_parser_value "${lines[54]}")
+    cpp_infer_model_dir_list=$(func_parser_value "${lines[1]}")
+    cpp_infer_is_quant=$(func_parser_value "${lines[2]}")
     # parser cpp inference 
-    inference_cmd=$(func_parser_value "${lines[55]}")
-    cpp_use_gpu_key=$(func_parser_key "${lines[56]}")
-    cpp_use_gpu_list=$(func_parser_value "${lines[56]}")
-    cpp_use_mkldnn_key=$(func_parser_key "${lines[57]}")
-    cpp_use_mkldnn_list=$(func_parser_value "${lines[57]}")
-    cpp_cpu_threads_key=$(func_parser_key "${lines[58]}")
-    cpp_cpu_threads_list=$(func_parser_value "${lines[58]}")
-    cpp_batch_size_key=$(func_parser_key "${lines[59]}")
-    cpp_batch_size_list=$(func_parser_value "${lines[59]}")
-    cpp_use_trt_key=$(func_parser_key "${lines[60]}")
-    cpp_use_trt_list=$(func_parser_value "${lines[60]}")
-    cpp_precision_key=$(func_parser_key "${lines[61]}")
-    cpp_precision_list=$(func_parser_value "${lines[61]}")
-    cpp_infer_model_key=$(func_parser_key "${lines[62]}")
-    cpp_image_dir_key=$(func_parser_key "${lines[63]}")
-    cpp_infer_img_dir=$(func_parser_value "${lines[63]}")
-    cpp_infer_key1=$(func_parser_key "${lines[64]}")
-    cpp_infer_value1=$(func_parser_value "${lines[64]}")
-    cpp_benchmark_key=$(func_parser_key "${lines[65]}")
-    cpp_benchmark_value=$(func_parser_value "${lines[65]}")
+    inference_cmd=$(func_parser_value "${lines[3]}")
+    cpp_use_gpu_key=$(func_parser_key "${lines[4]}")
+    cpp_use_gpu_list=$(func_parser_value "${lines[4]}")
+    cpp_use_mkldnn_key=$(func_parser_key "${lines[5]}")
+    cpp_use_mkldnn_list=$(func_parser_value "${lines[5]}")
+    cpp_cpu_threads_key=$(func_parser_key "${lines[6]}")
+    cpp_cpu_threads_list=$(func_parser_value "${lines[6]}")
+    cpp_batch_size_key=$(func_parser_key "${lines[7]}")
+    cpp_batch_size_list=$(func_parser_value "${lines[7]}")
+    cpp_use_trt_key=$(func_parser_key "${lines[8]}")
+    cpp_use_trt_list=$(func_parser_value "${lines[8]}")
+    cpp_precision_key=$(func_parser_key "${lines[9]}")
+    cpp_precision_list=$(func_parser_value "${lines[9]}")
+    cpp_infer_model_key=$(func_parser_key "${lines[10]}")
+    cpp_image_dir_key=$(func_parser_key "${lines[11]}")
+    cpp_infer_img_dir=$(func_parser_value "${lines[12]}")
+    cpp_infer_key1=$(func_parser_key "${lines[13]}")
+    cpp_infer_value1=$(func_parser_value "${lines[13]}")
+    cpp_benchmark_key=$(func_parser_key "${lines[14]}")
+    cpp_benchmark_value=$(func_parser_value "${lines[14]}")
 fi
 
 
+
 LOG_PATH="./tests/output"
 mkdir -p ${LOG_PATH}
 status_log="${LOG_PATH}/results.log"
@@ -414,7 +453,7 @@ function func_cpp_inference(){
     done
 }
 
-if [ ${MODE} = "infer" ]; then
+if [ ${MODE} = "infer" ] || [ ${MODE} = "klquant_infer" ]; then
     GPUID=$3
     if [ ${#GPUID} -le 0 ];then
         env=" "
@@ -447,7 +486,6 @@ if [ ${MODE} = "infer" ]; then
         func_inference "${python}" "${inference_py}" "${save_infer_dir}" "${LOG_PATH}" "${infer_img_dir}" ${is_quant}
         Count=$(($Count + 1))
     done
-
 elif [ ${MODE} = "cpp_infer" ]; then
     GPUID=$3
     if [ ${#GPUID} -le 0 ];then
@@ -481,6 +519,8 @@ elif [ ${MODE} = "serving_infer" ]; then
     #run serving
     func_serving "${web_service_cmd}"
 
+
+
 else
     IFS="|"
     export Count=0

tools/export_center.py

+# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from __future__ import absolute_import
+from __future__ import division
+from __future__ import print_function
+
+import os
+import sys
+import pickle
+
+__dir__ = os.path.dirname(os.path.abspath(__file__))
+sys.path.append(__dir__)
+sys.path.append(os.path.abspath(os.path.join(__dir__, '..')))
+
+from ppocr.data import build_dataloader
+from ppocr.modeling.architectures import build_model
+from ppocr.postprocess import build_post_process
+from ppocr.utils.save_load import init_model, load_dygraph_params
+from ppocr.utils.utility import print_dict
+import tools.program as program
+
+
+def main():
+    global_config = config['Global']
+    # build dataloader
+    config['Eval']['dataset']['name'] = config['Train']['dataset']['name']
+    config['Eval']['dataset']['data_dir'] = config['Train']['dataset'][
+        'data_dir']
+    config['Eval']['dataset']['label_file_list'] = config['Train']['dataset'][
+        'label_file_list']
+    eval_dataloader = build_dataloader(config, 'Eval', device, logger)
+
+    # build post process
+    post_process_class = build_post_process(config['PostProcess'],
+                                            global_config)
+
+    # build model
+    # for rec algorithm
+    if hasattr(post_process_class, 'character'):
+        char_num = len(getattr(post_process_class, 'character'))
+        config['Architecture']["Head"]['out_channels'] = char_num
+
+    #set return_features = True
+    config['Architecture']["Head"]["return_feats"] = True
+
+    model = build_model(config['Architecture'])
+
+    best_model_dict = load_dygraph_params(config, model, logger, None)
+    if len(best_model_dict):
+        logger.info('metric in ckpt ***************')
+        for k, v in best_model_dict.items():
+            logger.info('{}:{}'.format(k, v))
+
+    # get features from train data
+    char_center = program.get_center(model, eval_dataloader, post_process_class)
+
+    #serialize to disk
+    with open("train_center.pkl", 'wb') as f:
+        pickle.dump(char_center, f)
+    return
+
+
+if __name__ == '__main__':
+    config, device, logger, vdl_writer = program.preprocess()
+    main()

tools/program.py

@@ -404,6 +404,57 @@ def eval(model,
     return metric
 
 
+def update_center(char_center, post_result, preds):
+    result, label = post_result
+    feats, logits = preds
+    logits = paddle.argmax(logits, axis=-1)
+    feats = feats.numpy()
+    logits = logits.numpy()
+
+    for idx_sample in range(len(label)):
+        if result[idx_sample][0] == label[idx_sample][0]:
+            feat = feats[idx_sample]
+            logit = logits[idx_sample]
+            for idx_time in range(len(logit)):
+                index = logit[idx_time]
+                if index in char_center.keys():
+                    char_center[index][0] = (
+                        char_center[index][0] * char_center[index][1] +
+                        feat[idx_time]) / (char_center[index][1] + 1)
+                    char_center[index][1] += 1
+                else:
+                    char_center[index] = [feat[idx_time], 1]
+    return char_center
+
+
+def get_center(model, eval_dataloader, post_process_class):
+    pbar = tqdm(total=len(eval_dataloader), desc='get center:')
+    max_iter = len(eval_dataloader) - 1 if platform.system(
+    ) == "Windows" else len(eval_dataloader)
+    char_center = dict()
+    for idx, batch in enumerate(eval_dataloader):
+        if idx >= max_iter:
+            break
+        images = batch[0]
+        start = time.time()
+        preds = model(images)
+
+        batch = [item.numpy() for item in batch]
+        # Obtain usable results from post-processing methods
+        total_time += time.time() - start
+        # Evaluate the results of the current batch
+        post_result = post_process_class(preds, batch[1])
+
+        #update char_center
+        char_center = update_center(char_center, post_result, preds)
+        pbar.update(1)
+
+    pbar.close()
+    for key in char_center.keys():
+        char_center[key] = char_center[key][0]
+    return char_center
+
+
 def preprocess(is_train=False):
     FLAGS = ArgsParser().parse_args()
     profiler_options = FLAGS.profiler_options