离线量化（无需训练） | + + +- 预测相关：基于训练是否使用量化，可以将训练产出的模型可以分为`正常模型`和`量化模型`，这两类模型对应的预测功能汇总如下， + +| 模型类型 |device | batchsize | tensorrt | mkldnn | cpu多线程 | +| ---- | ---- | ---- | :----: | :----: | :----: | +| 正常模型 | CPU | 1/6 | - | fp32 | 支持 | +| 量化模型 | CPU | 1/6 | - | int8 | 支持 | + + +## 2. 测试流程 + +Mac端无GPU，环境准备只需要Python环境即可，安装PaddlePaddle等依赖参考下述文档。 + +### 2.1 安装依赖 +- 安装PaddlePaddle >= 2.0 +- 安装PaddleOCR依赖 + ``` + pip install -r ../requirements.txt + ``` +- 安装autolog（规范化日志输出工具） + ``` + git clone https://github.com/LDOUBLEV/AutoLog + cd AutoLog + pip install -r requirements.txt + python setup.py bdist_wheel + pip install ./dist/auto_log-1.0.0-py3-none-any.whl + cd ../ + ``` +- 安装PaddleSlim (可选) + ``` + # 如果要测试量化、裁剪等功能，需要安装PaddleSlim + pip install paddleslim + ``` + + +### 2.2 功能测试 + +先运行`prepare.sh`准备数据和模型，然后运行`test_train_inference_python.sh`进行测试，最终在```test_tipc/output```目录下生成`python_infer_*.log`格式的日志文件。 + +`test_train_inference_python.sh`包含5种运行模式，每种模式的运行数据不同，分别用于测试速度和精度，分别是： + +- 模式1：lite_train_lite_infer，使用少量数据训练，用于快速验证训练到预测的走通流程，不验证精度和速度； +```shell +# 同linux端运行不同的是，Mac端测试使用新的配置文件mac_ppocr_det_mobile_params.txt， +# 配置文件中默认去掉了GPU和mkldnn相关的测试链条 +bash test_tipc/prepare.sh ./test_tipc/configs/mac_ppocr_det_mobile_params.txt 'lite_train_lite_infer' +bash test_tipc/test_train_inference_python.sh ./test_tipc/configs/mac_ppocr_det_mobile_params.txt 'lite_train_lite_infer' +``` + +- 模式2：lite_train_whole_infer，使用少量数据训练，一定量数据预测，用于验证训练后的模型执行预测，预测速度是否合理； +```shell +bash test_tipc/prepare.sh ./test_tipc/configs/mac_ppocr_det_mobile_params.txt 'lite_train_whole_infer' +bash test_tipc/test_train_inference_python.sh ./test_tipc/configs/mac_ppocr_det_mobile_params.txt 'lite_train_whole_infer' +``` + +- 模式3：whole_infer，不训练，全量数据预测，走通开源模型评估、动转静，检查inference model预测时间和精度; +```shell +bash test_tipc/prepare.sh ./test_tipc/configs/mac_ppocr_det_mobile_params.txt 'whole_infer' +# 用法1: +bash test_tipc/test_train_inference_python.sh ./test_tipc/configs/mac_ppocr_det_mobile_params.txt 'whole_infer' +# 用法2: 指定GPU卡预测，第三个传入参数为GPU卡号 +bash test_tipc/test_train_inference_python.sh ./test_tipc/configs/mac_ppocr_det_mobile_params.txt 'whole_infer' '1' +``` + +- 模式4：whole_train_whole_infer，CE： 全量数据训练，全量数据预测，验证模型训练精度，预测精度，预测速度；（Mac端不建议运行此模式） +```shell +bash test_tipc/prepare.sh ./test_tipc/configs/mac_ppocr_det_mobile_params.txt 'whole_train_whole_infer' +bash test_tipc/test_train_inference_python.sh ./test_tipc/configs/mac_ppocr_det_mobile_params.txt 'whole_train_whole_infer' +``` + +- 模式5：klquant_whole_infer，测试离线量化； +```shell +bash test_tipc/prepare.sh ./test_tipc/configs/mac_ppocr_det_mobile_params.txt 'klquant_whole_infer' +bash test_tipc/test_train_inference_python.sh test_tipc/configs/mac_ppocr_det_mobile_params.txt 'klquant_whole_infer' +``` + +运行相应指令后，在`test_tipc/output`文件夹下自动会保存运行日志。如`lite_train_lite_infer`模式下，会运行训练+inference的链条，因此，在`test_tipc/output`文件夹有以下文件： +``` +test_tipc/output/ +|- results_python.log # 运行指令状态的日志 +|- norm_train_gpus_-1_autocast_null/ # CPU上正常训练的训练日志和模型保存文件夹 +|- pact_train_gpus_-1_autocast_null/ # CPU上量化训练的训练日志和模型保存文件夹 +...... +|- python_infer_cpu_usemkldnn_False_threads_1_batchsize_1.log # CPU上关闭Mkldnn线程数设置为1，测试batch_size=1条件下的预测运行日志 +...... +``` + +其中`results_python.log`中包含了每条指令的运行状态，如果运行成功会输出： +``` +Run successfully with command - python3.7 tools/train.py -c tests/configs/det_mv3_db.yml -o Global.pretrained_model=./pretrain_models/MobileNetV3_large_x0_5_pretrained Global.use_gpu=False Global.save_model_dir=./tests/output/norm_train_gpus_-1_autocast_null Global.epoch_num=1 Train.loader.batch_size_per_card=2 ! +Run successfully with command - python3.7 tools/export_model.py -c tests/configs/det_mv3_db.yml -o Global.pretrained_model=./tests/output/norm_train_gpus_-1_autocast_null/latest Global.save_inference_dir=./tests/output/norm_train_gpus_-1_autocast_null! +...... +``` +如果运行失败，会输出： +``` +Run failed with command - python3.7 tools/train.py -c tests/configs/det_mv3_db.yml -o Global.pretrained_model=./pretrain_models/MobileNetV3_large_x0_5_pretrained Global.use_gpu=Faslse Global.save_model_dir=./tests/output/norm_train_gpus_-1_autocast_null Global.epoch_num=1 Train.loader.batch_size_per_card=2 ! +Run failed with command - python3.7 tools/export_model.py -c tests/configs/det_mv3_db.yml -o Global.pretrained_model=./tests/output/norm_train_gpus_0_autocast_null/latest Global.save_inference_dir=./tests/output/norm_train_gpus_-1_autocast_null! +...... +``` +可以很方便的根据`results_python.log`中的内容判定哪一个指令运行错误。 + +### 2.3 精度测试 + +使用compare_results.py脚本比较模型预测的结果是否符合预期，主要步骤包括： +- 提取日志中的预测坐标； +- 从本地文件中提取保存好的坐标结果； +- 比较上述两个结果是否符合精度预期，误差大于设置阈值时会报错。 + +#### 使用方式 +运行命令： +```shell +python test_tipc/compare_results.py --gt_file=./test_tipc/results/python_*.txt --log_file=./test_tipc/output/python_*.log --atol=1e-3 --rtol=1e-3 +``` + +参数介绍： +- gt_file： 指向事先保存好的预测结果路径，支持*.txt 结尾，会自动索引*.txt格式的文件，文件默认保存在test_tipc/result/ 文件夹下 +- log_file: 指向运行test_tipc/test_train_inference_python.sh 脚本的infer模式保存的预测日志，预测日志中打印的有预测结果，比如：文本框，预测文本，类别等等，同样支持python_infer_*.log格式传入 +- atol: 设置的绝对误差 +- rtol: 设置的相对误差 + +#### 运行结果 + +正常运行效果如下图： +
+
+出现不一致结果时的运行输出：
+
+
+
+## 3. 更多教程
+本文档为功能测试用，更丰富的训练预测使用教程请参考：
+[模型训练](https://github.com/PaddlePaddle/PaddleOCR/blob/dygraph/doc/doc_ch/training.md)
+[基于Python预测引擎推理](https://github.com/PaddlePaddle/PaddleOCR/blob/dygraph/doc/doc_ch/inference.md)
diff --git a/test_tipc/docs/ssh_termux_ls.png b/test_tipc/docs/ssh_termux_ls.png
new file mode 100644
index 0000000000000000000000000000000000000000..2df78026b23b2bb71ac98092d7820e5d02ad611c
Binary files /dev/null and b/test_tipc/docs/ssh_termux_ls.png differ
diff --git a/test_tipc/docs/termux.jpg b/test_tipc/docs/termux.jpg
new file mode 100644
index 0000000000000000000000000000000000000000..cb87c4ccc21bab6411b87f61e76f03b5b5f6f557
Binary files /dev/null and b/test_tipc/docs/termux.jpg differ
diff --git a/test_tipc/docs/termux_for_android.md b/test_tipc/docs/termux_for_android.md
new file mode 100644
index 0000000000000000000000000000000000000000..73ecbb2e93be5a8fdef593fb492d3a27d33c6b52
--- /dev/null
+++ b/test_tipc/docs/termux_for_android.md
@@ -0,0 +1,128 @@
+# 安卓手机通过Termux连接电脑
+
+由于通过adb方式连接手机后，很多linux命令无法运行，自动化测试受阻，所以此处特此介绍另外一种通过Termux的连接方式，不仅可以运行大部分linux命令，方便开发者在手机上在线调试，甚至还可以多实现台机器同时连接手机。Termux不是真实的Linux环境，但是Termux可以安装真实的Linux，而且不会损失性能，与此同时，Termux不需要root。在配置Termux之前，请确保电脑已经安装adb工具，安装方式请参考[Lite端部署](https://github.com/PaddlePaddle/PaddleOCR/blob/develop/deploy/lite/readme.md) 。在运行以下命令后确保可以显示安卓设备信息。
+
+```
+adb devices
+```
+连接成功信息提示：
+```
+List of devices attached
+744be294 device
+```
+
+## 1.安卓手机安装termux app
+
+### 1.1 下载termux apk文件
+
+由于目前该app目前各大商城暂无，所以可以直接下载如下apk文件。
+
+打开电脑终端，执行以下命令：
+
+```
+wget http://10.12.121.133:8911/cuicheng01/fullchain/termux-v1.0.3.apk
+```
+
+### 1.2 安装termux到手机上
+
+在手机端的开发者模式下，允许USB调试，允许USB安装。在电脑终端，执行如下命令，将termux app安装到手机上：
+
+```
+adb install termux-v1.0.3.apk
+```
+
+此处需要手机端确认安装，点击确认。
+
+### 1.3 验证是否安装成功
+
+打开手机，检验termux是否安装成功，如果没有，重新执行1.2，如果有相应的app，点击进入，会有如下显示。
+
+
+
+接下来的配置环境需要在手机上此终端运行相关命令。
+
+## 2.手机端配置termux
+
+首先将手机联网，最好可以连接外网，部分的配置需要外网。打开Termux终端，执行以下命令安装基础件`proot`，并使用`termux-chroot`命令可以模拟 root 环境与标准的 Linux 目录结构。
+
+```
+pkg i -y proot
+termux-chroot
+```
+
+Termux 默认只能访问自身内部的数据，如果要访问手机中其它的数据，输入下面的命令后，手机弹出对请求权限的窗口，允许即可（方便对部分运行出的结果在手机端可视化）。
+
+```
+termux-setup-storage
+```
+
+### 2.1 配置SSH
+
+作为 Linux 终端或者服务器，必须有SSH。不管你是 SSH 连接到 Termux还是使用Termux去连其它主机，都需要先安装openssh。如果安装失败，请重复执行命令。
+
+```
+pkg i -y openssh
+```
+
+启动 SSH 服务端，默认端口号为8022
+
+```
+sshd
+```
+
+
+### 2.2 电脑通过SSH方式连接手机
+
+1.保证手机和电脑处于同一局域网下
+手机端分别输入以下命令获得ip地址和当前用户：
+
+```
+# 获取ip地址
+ifconfig
+
+# 获取当前用户
+whoami
+```
+
+如获取到的ip地址和当前用户分别是`172.24.162.117`和`u0_a374`。
+
+2.电脑端通过SSH连接手机
+
+```
+#默认端口号为8022
+ssh u0_a374@172.24.162.117 -p 8022
+```
+
+3.运行ls命令后，会有如下显示：
+
+```
+ls
+```
+
+
+
+
+### 2.3 通过scp传输数据
+
+1.在当前目录上新建test目录
+
+```
+mkdir test
+```
+
+2.测试scp功能
+
+将电脑中的某个文件拷贝到手机上：
+```
+scp -P 8022 test.txt u0_a374@172.24.162.117:/home/storage/test
+```
+
+3.手机端查看
+
+打开手机终端，在`/home/storage/test`下查看是否存在`test.txt`
+
+
+## 3. 更多教程
+
+本教程可以完成Termux基本配置，更多关于Termux的用法，请参考：[Termux高级终端安装使用配置教程](https://www.sqlsec.com/2018/05/termux.html)。
+
diff --git a/PTDN/docs/test.png b/test_tipc/docs/test.png
similarity index 100%
rename from PTDN/docs/test.png
rename to test_tipc/docs/test.png
diff --git a/test_tipc/docs/test_inference_cpp.md b/test_tipc/docs/test_inference_cpp.md
new file mode 100644
index 0000000000000000000000000000000000000000..cd757a895bb957e498fda61cf52d2132d660ca8f
--- /dev/null
+++ b/test_tipc/docs/test_inference_cpp.md
@@ -0,0 +1,88 @@
+# C++预测功能测试
+
+C++预测功能测试的主程序为`test_inference_cpp.sh`，可以测试基于C++预测库的模型推理功能。
+
+## 1. 测试结论汇总
+
+基于训练是否使用量化，进行本测试的模型可以分为`正常模型`和`量化模型`，这两类模型对应的C++预测功能汇总如下：
+
+| 模型类型 |device | batchsize | tensorrt | mkldnn | cpu多线程 |
+| ---- | ---- | ---- | :----: | :----: | :----: |
+| 正常模型 | GPU | 1/6 | fp32/fp16 | - | - |
+| 正常模型 | CPU | 1/6 | - | fp32 | 支持 |
+| 量化模型 | GPU | 1/6 | int8 | - | - |
+| 量化模型 | CPU | 1/6 | - | int8 | 支持 |
+
+## 2. 测试流程
+运行环境配置请参考[文档](./install.md)的内容配置TIPC的运行环境。
+
+### 2.1 功能测试
+先运行`prepare.sh`准备数据和模型，然后运行`test_inference_cpp.sh`进行测试，最终在```test_tipc/output```目录下生成`cpp_infer_*.log`后缀的日志文件。
+
+```shell
+bash test_tipc/prepare.sh ./test_tipc/configs/ppocr_det_mobile_params.txt "cpp_infer"
+
+# 用法1:
+bash test_tipc/test_inference_cpp.sh ./test_tipc/configs/ppocr_det_mobile_params.txt
+# 用法2: 指定GPU卡预测，第三个传入参数为GPU卡号
+bash test_tipc/test_inference_cpp.sh ./test_tipc/configs/ppocr_det_mobile_params.txt '1'
+```
+
+运行预测指令后，在`test_tipc/output`文件夹下自动会保存运行日志，包括以下文件：
+
+```shell
+test_tipc/output/
+|- results_cpp.log # 运行指令状态的日志
+|- cpp_infer_cpu_usemkldnn_False_threads_1_precision_fp32_batchsize_1.log # CPU上不开启Mkldnn，线程数设置为1，测试batch_size=1条件下的预测运行日志
+|- cpp_infer_cpu_usemkldnn_False_threads_6_precision_fp32_batchsize_1.log # CPU上不开启Mkldnn，线程数设置为6，测试batch_size=1条件下的预测运行日志
+|- cpp_infer_gpu_usetrt_False_precision_fp32_batchsize_1.log # GPU上不开启TensorRT，测试batch_size=1的fp32精度预测日志
+|- cpp_infer_gpu_usetrt_True_precision_fp16_batchsize_1.log # GPU上开启TensorRT，测试batch_size=1的fp16精度预测日志
+......
+```
+其中results_cpp.log中包含了每条指令的运行状态，如果运行成功会输出：
+
+```
+Run successfully with command - ./deploy/cpp_infer/build/ppocr det --use_gpu=False --enable_mkldnn=False --cpu_threads=6 --det_model_dir=./inference/ch_ppocr_mobile_v2.0_det_infer/ --rec_batch_num=1 --image_dir=./inference/ch_det_data_50/all-sum-510/ --benchmar k=True > ./test_tipc/output/cpp_infer_cpu_usemkldnn_False_threads_6_precision_fp32_batchsize_1.log 2>&1 !
+Run successfully with command - ./deploy/cpp_infer/build/ppocr det --use_gpu=True --use_tensorrt=False --precision=fp32 --det_model_dir=./inference/ch_ppocr_mobile_v2.0_det_infer/ --rec_batch_num=1 --image_dir=./inference/ch_det_data_50/all-sum-510/ --benchmark =True > ./test_tipc/output/cpp_infer_gpu_usetrt_False_precision_fp32_batchsize_1.log 2>&1 !
+......
+```
+如果运行失败，会输出：
+```
+Run failed with command - ./deploy/cpp_infer/build/ppocr det --use_gpu=True --use_tensorrt=True --precision=fp32 --det_model_dir=./inference/ch_ppocr_mobile_v2.0_det_infer/ --rec_batch_num=1 --image_dir=./inference/ch_det_data_50/all-sum-510/ --benchmark=True > ./test_tipc/output/cpp_infer_gpu_usetrt_True_precision_fp32_batchsize_1.log 2>&1 !
+Run failed with command - ./deploy/cpp_infer/build/ppocr det --use_gpu=True --use_tensorrt=True --precision=fp16 --det_model_dir=./inference/ch_ppocr_mobile_v2.0_det_infer/ --rec_batch_num=1 --image_dir=./inference/ch_det_data_50/all-sum-510/ --benchmark=True > ./test_tipc/output/cpp_infer_gpu_usetrt_True_precision_fp16_batchsize_1.log 2>&1 !
+......
+```
+可以很方便的根据results_cpp.log中的内容判定哪一个指令运行错误。
+
+
+### 2.2 精度测试
+
+使用compare_results.py脚本比较模型预测的结果是否符合预期，主要步骤包括：
+- 提取日志中的预测坐标；
+- 从本地文件中提取保存好的坐标结果；
+- 比较上述两个结果是否符合精度预期，误差大于设置阈值时会报错。
+
+#### 使用方式
+运行命令：
+```shell
+python3.7 test_tipc/compare_results.py --gt_file=./test_tipc/results/cpp_*.txt --log_file=./test_tipc/output/cpp_*.log --atol=1e-3 --rtol=1e-3
+```
+
+参数介绍：
+- gt_file： 指向事先保存好的预测结果路径，支持*.txt 结尾，会自动索引*.txt格式的文件，文件默认保存在test_tipc/result/ 文件夹下
+- log_file: 指向运行test_tipc/test_inference_cpp.sh 脚本的infer模式保存的预测日志，预测日志中打印的有预测结果，比如：文本框，预测文本，类别等等，同样支持cpp_infer_*.log格式传入
+- atol: 设置的绝对误差
+- rtol: 设置的相对误差
+
+#### 运行结果
+
+正常运行效果如下图：
+
+
+出现不一致结果时的运行输出：
+
+
+
+## 3. 更多教程
+
+本文档为功能测试用，更详细的c++预测使用教程请参考：[服务器端C++预测](https://github.com/PaddlePaddle/PaddleOCR/tree/dygraph/deploy/cpp_infer)
diff --git a/test_tipc/docs/test_lite.md b/test_tipc/docs/test_lite.md
new file mode 100644
index 0000000000000000000000000000000000000000..01ae0cb4b471f1219f88ffa9e2c11d50765233d3
--- /dev/null
+++ b/test_tipc/docs/test_lite.md
@@ -0,0 +1,72 @@
+# Lite预测功能测试
+
+Lite预测功能测试的主程序为`test_lite.sh`，可以测试基于Lite预测库的模型推理功能。
+
+## 1. 测试结论汇总
+
+目前Lite端的样本间支持以方式的组合：
+
+**字段说明：**
+- 输入设置：包括C++预测、python预测、java预测
+- 模型类型：包括正常模型（FP32）和量化模型（FP16）
+- batch-size：包括1和4
+- predictor数量：包括多predictor预测和单predictor预测
+- 功耗模式：包括高性能模式（LITE_POWER_HIGH）和省电模式（LITE_POWER_LOW）
+- 预测库来源：包括下载方式和编译方式，其中编译方式分为以下目标硬件：(1)ARM CPU;(2)Linux XPU;(3)OpenCL GPU;(4)Metal GPU
+
+| 模型类型 | batch-size | predictor数量 | 功耗模式 | 预测库来源 | 支持语言 |
+| :----: | :----: | :----: | :----: | :----: | :----: |
+| 正常模型/量化模型 | 1 | 1 | 高性能模式/省电模式 | 下载方式 | C++预测 |
+
+
+## 2. 测试流程
+运行环境配置请参考[文档](./install.md)的内容配置TIPC的运行环境。
+
+### 2.1 功能测试
+
+先运行`prepare.sh`准备数据和模型，模型和数据会打包到test_lite.tar中，将test_lite.tar上传到手机上，解压后进`入test_lite`目录中，然后运行`test_lite.sh`进行测试，最终在`test_lite/output`目录下生成`lite_*.log`后缀的日志文件。
+
+```shell
+
+# 数据和模型准备
+bash test_tipc/prepare.sh ./test_tipc/configs/ppocr_det_mobile_params.txt "lite_infer"
+
+# 手机端测试:
+bash test_lite.sh ppocr_det_mobile_params.txt
+
+```
+
+**注意**：由于运行该项目需要bash等命令，传统的adb方式不能很好的安装。所以此处推荐通在手机上开启虚拟终端的方式连接电脑，连接方式可以参考[安卓手机termux连接电脑](./termux_for_android.md)。
+
+#### 运行结果
+
+各测试的运行情况会打印在 `./output/` 中：
+运行成功时会输出：
+
+```
+Run successfully with command - ./ocr_db_crnn det ./models/ch_ppocr_mobile_v2.0_det_slim_opt.nb INT8 4 1 LITE_POWER_LOW ./test_data/icdar2015_lite/text_localization/ch4_test_images/img_233.jpg ./config.txt True > ./output/lite_ch_ppocr_mobile_v2.0_det_slim_opt.nb_precision_INT8_batchsize_1_threads_4_powermode_LITE_POWER_LOW_singleimg_True.log 2>&1!
+Run successfully with command xxx
+...
+```
+
+运行失败时会输出：
+
+```
+Run failed with command - ./ocr_db_crnn det ./models/ch_ppocr_mobile_v2.0_det_slim_opt.nb INT8 4 1 LITE_POWER_LOW ./test_data/icdar2015_lite/text_localization/ch4_test_images/img_233.jpg ./config.txt True > ./output/lite_ch_ppocr_mobile_v2.0_det_slim_opt.nb_precision_INT8_batchsize_1_threads_4_powermode_LITE_POWER_LOW_singleimg_True.log 2>&1!
+Run failed with command xxx
+...
+```
+
+在./output/文件夹下，会存在如下日志，每一个日志都是不同配置下的log结果：
+
+
+
+在每一个log中，都会调用autolog打印如下信息：
+
+
+
+
+
+## 3. 更多教程
+
+本文档为功能测试用，更详细的Lite端预测使用教程请参考：[Lite端部署](https://github.com/PaddlePaddle/PaddleOCR/blob/develop/deploy/lite/readme.md)。
diff --git a/test_tipc/docs/test_paddle2onnx.md b/test_tipc/docs/test_paddle2onnx.md
new file mode 100644
index 0000000000000000000000000000000000000000..5d784c5e93c3a93d00c256004de582dcbf357c45
--- /dev/null
+++ b/test_tipc/docs/test_paddle2onnx.md
@@ -0,0 +1,47 @@
+# Paddle2onnx预测功能测试
+
+PaddleServing预测功能测试的主程序为`test_paddle2onnx.sh`，可以测试Paddle2ONNX的模型转化功能，并验证正确性。
+
+## 1. 测试结论汇总
+
+基于训练是否使用量化，进行本测试的模型可以分为`正常模型`和`量化模型`，这两类模型对应的Paddle2ONNX预测功能汇总如下：
+
+| 模型类型 |device |
+| ---- | ---- |
+| 正常模型 | GPU |
+| 正常模型 | CPU |
+| 量化模型 | GPU |
+| 量化模型 | CPU |
+
+## 2. 测试流程
+### 2.1 功能测试
+先运行`prepare.sh`准备数据和模型，然后运行`test_paddle2onnx.sh`进行测试，最终在```test_tipc/output```目录下生成`paddle2onnx_infer_*.log`后缀的日志文件。
+
+```shell
+bash test_tipc/prepare.sh ./test_tipc/configs/ppocr_det_mobile_params.txt "paddle2onnx_infer"
+
+# 用法:
+bash test_tipc/test_paddle2onnx.sh ./test_tipc/configs/ppocr_det_mobile_params.txt
+```
+
+#### 运行结果
+
+各测试的运行情况会打印在 `test_tipc/output/results_paddle2onnx.log` 中：
+运行成功时会输出：
+
+```
+Run successfully with command - paddle2onnx --model_dir=./inference/ch_ppocr_mobile_v2.0_det_infer/ --model_filename=inference.pdmodel --params_filename=inference.pdiparams --save_file=./inference/det_mobile_onnx/model.onnx --opset_version=10 --enable_onnx_checker=True!
+Run successfully with command - python test_tipc/onnx_inference/predict_det.py --use_gpu=False --image_dir=./inference/ch_det_data_50/all-sum-510/ --det_model_dir=./inference/det_mobile_onnx/model.onnx 2>&1 !
+```
+
+运行失败时会输出：
+
+```
+Run failed with command - paddle2onnx --model_dir=./inference/ch_ppocr_mobile_v2.0_det_infer/ --model_filename=inference.pdmodel --params_filename=inference.pdiparams --save_file=./inference/det_mobile_onnx/model.onnx --opset_version=10 --enable_onnx_checker=True!
+...
+```
+
+
+## 3. 更多教程
+
+本文档为功能测试用，更详细的Paddle2onnx预测使用教程请参考：[Paddle2ONNX](https://github.com/PaddlePaddle/Paddle2ONNX)
diff --git a/test_tipc/docs/test_serving.md b/test_tipc/docs/test_serving.md
new file mode 100644
index 0000000000000000000000000000000000000000..f63d6c7107ce92807c53d81a22a582b09178a712
--- /dev/null
+++ b/test_tipc/docs/test_serving.md
@@ -0,0 +1,80 @@
+# PaddleServing预测功能测试
+
+PaddleServing预测功能测试的主程序为`test_serving.sh`，可以测试基于PaddleServing的部署功能。
+
+## 1. 测试结论汇总
+
+基于训练是否使用量化，进行本测试的模型可以分为`正常模型`和`量化模型`，这两类模型对应的Serving预测功能汇总如下：
+
+| 模型类型 |device | batchsize | tensorrt | mkldnn | cpu多线程 |
+| ---- | ---- | ---- | :----: | :----: | :----: |
+| 正常模型 | GPU | 1/6 | fp32/fp16 | - | - |
+| 正常模型 | CPU | 1/6 | - | fp32 | 支持 |
+| 量化模型 | GPU | 1/6 | int8 | - | - |
+| 量化模型 | CPU | 1/6 | - | int8 | 支持 |
+
+## 2. 测试流程
+运行环境配置请参考[文档](./install.md)的内容配置TIPC的运行环境。
+
+### 2.1 功能测试
+先运行`prepare.sh`准备数据和模型，然后运行`test_serving.sh`进行测试，最终在```test_tipc/output```目录下生成`serving_infer_*.log`后缀的日志文件。
+
+```shell
+bash test_tipc/prepare.sh ./test_tipc/configs/ppocr_det_mobile_params.txt "serving_infer"
+
+# 用法:
+bash test_tipc/test_serving.sh ./test_tipc/configs/ppocr_det_mobile_params.txt
+```
+
+#### 运行结果
+
+各测试的运行情况会打印在 `test_tipc/output/results_serving.log` 中：
+运行成功时会输出：
+
+```
+Run successfully with command - python3.7 pipeline_http_client.py --image_dir=../../doc/imgs > ../../tests/output/server_infer_cpu_usemkldnn_True_threads_1_batchsize_1.log 2>&1 !
+Run successfully with command - xxxxx
+...
+```
+
+运行失败时会输出：
+
+```
+Run failed with command - python3.7 pipeline_http_client.py --image_dir=../../doc/imgs > ../../tests/output/server_infer_cpu_usemkldnn_True_threads_1_batchsize_1.log 2>&1 !
+Run failed with command - python3.7 pipeline_http_client.py --image_dir=../../doc/imgs > ../../tests/output/server_infer_cpu_usemkldnn_True_threads_6_batchsize_1.log 2>&1 !
+Run failed with command - xxxxx
+...
+```
+
+详细的预测结果会存在 test_tipc/output/ 文件夹下，例如`server_infer_gpu_usetrt_True_precision_fp16_batchsize_1.log`中会返回检测框的坐标:
+
+```
+{'err_no': 0, 'err_msg': '', 'key': ['dt_boxes'], 'value': ['[[[ 78. 642.]\n [409. 640.]\n [409. 657.]\n
+[ 78. 659.]]\n\n [[ 75. 614.]\n [211. 614.]\n [211. 635.]\n [ 75. 635.]]\n\n
+[[103. 554.]\n [135. 554.]\n [135. 575.]\n [103. 575.]]\n\n [[ 75. 531.]\n
+[347. 531.]\n [347. 549.]\n [ 75. 549.] ]\n\n [[ 76. 503.]\n [309. 498.]\n
+[309. 521.]\n [ 76. 526.]]\n\n [[163. 462.]\n [317. 462.]\n [317. 493.]\n
+[163. 493.]]\n\n [[324. 431.]\n [414. 431.]\n [414. 452.]\n [324. 452.]]\n\n
+[[ 76. 412.]\n [208. 408.]\n [209. 424.]\n [ 76. 428.]]\n\n [[307. 409.]\n
+[428. 409.]\n [428. 426.]\n [307 . 426.]]\n\n [[ 74. 385.]\n [217. 382.]\n
+[217. 400.]\n [ 74. 403.]]\n\n [[308. 381.]\n [427. 380.]\n [427. 400.]\n
+[308. 401.]]\n\n [[ 74. 363.]\n [195. 362.]\n [195. 378.]\n [ 74. 379.]]\n\n
+[[303. 359.]\n [423. 357.]\n [423. 375.]\n [303. 377.]]\n\n [[ 70. 336.]\n
+[239. 334.]\n [239. 354.]\ n [ 70. 356.]]\n\n [[ 70. 312.]\n [204. 310.]\n
+[204. 327.]\n [ 70. 330.]]\n\n [[303. 308.]\n [419. 306.]\n [419. 326.]\n
+[303. 328.]]\n\n [[113. 2 72.]\n [246. 270.]\n [247. 299.]\n [113. 301.]]\n\n
+ [[361. 269.]\n [384. 269.]\n [384. 296.]\n [361. 296.]]\n\n [[ 70. 250.]\n
+ [243. 246.]\n [243. 265.]\n [ 70. 269.]]\n\n [[ 65. 221.]\n [187. 220.]\n
+[187. 240.]\n [ 65. 241.]]\n\n [[337. 216.]\n [382. 216.]\n [382. 240.]\n
+[337. 240.]]\n\n [ [ 65. 196.]\n [247. 193.]\n [247. 213.]\n [ 65. 216.]]\n\n
+[[296. 197.]\n [423. 191.]\n [424. 209.]\n [296. 215.]]\n\n [[ 65. 167.]\n [244. 167.]\n
+[244. 186.]\n [ 65. 186.]]\n\n [[ 67. 139.]\n [290. 139.]\n [290. 159.]\n [ 67. 159.]]\n\n
+[[ 68. 113.]\n [410. 113.]\n [410. 128.]\n [ 68. 129.] ]\n\n [[277. 87.]\n [416. 87.]\n
+[416. 108.]\n [277. 108.]]\n\n [[ 79. 28.]\n [132. 28.]\n [132. 62.]\n [ 79. 62.]]\n\n
+[[163. 17.]\n [410. 14.]\n [410. 50.]\n [163. 53.]]]']}
+```
+
+
+## 3. 更多教程
+
+本文档为功能测试用，更详细的Serving预测使用教程请参考：[PPOCR 服务化部署](https://github.com/PaddlePaddle/PaddleOCR/blob/dygraph/deploy/pdserving/README_CN.md)
diff --git a/test_tipc/docs/test_train_inference_python.md b/test_tipc/docs/test_train_inference_python.md
new file mode 100644
index 0000000000000000000000000000000000000000..9028e67d093112d23cc7c5d9da10d185f1db9b5b
--- /dev/null
+++ b/test_tipc/docs/test_train_inference_python.md
@@ -0,0 +1,156 @@
+# Linux端基础训练预测功能测试
+
+Linux端基础训练预测功能测试的主程序为`test_train_inference_python.sh`，可以测试基于Python的模型训练、评估、推理等基本功能，包括裁剪、量化、蒸馏。
+
+- Mac端基础训练预测功能测试参考[链接](./mac_test_train_inference_python.md)
+- Windows端基础训练预测功能测试参考[链接](./win_test_train_inference_python.md)
+
+## 1. 测试结论汇总
+
+- 训练相关：
+
+| 算法名称 | 模型名称 | 单机单卡 | 单机多卡 | 多机多卡 | 模型压缩（单机多卡） |
+| :---- | :---- | :---- | :---- | :---- | :---- |
+| DB | ch_ppocr_mobile_v2.0_det| 正常训练 混合精度 | 正常训练
混合精度 | 正常训练
混合精度 | 正常训练：FPGM裁剪、PACT量化
离线量化（无需训练） | +| DB | ch_ppocr_server_v2.0_det| 正常训练
混合精度 | 正常训练
混合精度 | 正常训练
混合精度 | 正常训练：FPGM裁剪、PACT量化
离线量化（无需训练） | +| CRNN | ch_ppocr_mobile_v2.0_rec| 正常训练
混合精度 | 正常训练
混合精度 | 正常训练
混合精度 | 正常训练：PACT量化
离线量化（无需训练） | +| CRNN | ch_ppocr_server_v2.0_rec| 正常训练
混合精度 | 正常训练
混合精度 | 正常训练
混合精度 | 正常训练：PACT量化
离线量化（无需训练） | +|PP-OCR| ch_ppocr_mobile_v2.0| 正常训练
混合精度 | 正常训练
混合精度 | 正常训练
混合精度 | - | +|PP-OCR| ch_ppocr_server_v2.0| 正常训练
混合精度 | 正常训练
混合精度 | 正常训练
混合精度 | - | +|PP-OCRv2| ch_PP-OCRv2 | 正常训练
混合精度 | 正常训练
混合精度 | 正常训练
混合精度 | - | + + +- 预测相关：基于训练是否使用量化，可以将训练产出的模型可以分为`正常模型`和`量化模型`，这两类模型对应的预测功能汇总如下， + +| 模型类型 |device | batchsize | tensorrt | mkldnn | cpu多线程 | +| ---- | ---- | ---- | :----: | :----: | :----: | +| 正常模型 | GPU | 1/6 | fp32/fp16 | - | - | +| 正常模型 | CPU | 1/6 | - | fp32/fp16 | 支持 | +| 量化模型 | GPU | 1/6 | int8 | - | - | +| 量化模型 | CPU | 1/6 | - | int8 | 支持 | + + +## 2. 测试流程 + +运行环境配置请参考[文档](./install.md)的内容配置TIPC的运行环境。 + +### 2.1 安装依赖 +- 安装PaddlePaddle >= 2.0 +- 安装PaddleOCR依赖 + ``` + pip3 install -r ../requirements.txt + ``` +- 安装autolog（规范化日志输出工具） + ``` + git clone https://github.com/LDOUBLEV/AutoLog + cd AutoLog + pip3 install -r requirements.txt + python3 setup.py bdist_wheel + pip3 install ./dist/auto_log-1.0.0-py3-none-any.whl + cd ../ + ``` +- 安装PaddleSlim (可选) + ``` + # 如果要测试量化、裁剪等功能，需要安装PaddleSlim + pip3 install paddleslim + ``` + + +### 2.2 功能测试 +先运行`prepare.sh`准备数据和模型，然后运行`test_train_inference_python.sh`进行测试，最终在```test_tipc/output```目录下生成`python_infer_*.log`格式的日志文件。 + + +`test_train_inference_python.sh`包含5种运行模式，每种模式的运行数据不同，分别用于测试速度和精度，分别是： + +- 模式1：lite_train_lite_infer，使用少量数据训练，用于快速验证训练到预测的走通流程，不验证精度和速度； +```shell +bash test_tipc/prepare.sh ./test_tipc/configs/ppocr_det_mobile_params.txt 'lite_train_lite_infer' +bash test_tipc/test_train_inference_python.sh ./test_tipc/configs/ppocr_det_mobile_params.txt 'lite_train_lite_infer' +``` + +- 模式2：lite_train_whole_infer，使用少量数据训练，一定量数据预测，用于验证训练后的模型执行预测，预测速度是否合理； +```shell +bash test_tipc/prepare.sh ./test_tipc/configs/ppocr_det_mobile_params.txt 'lite_train_whole_infer' +bash test_tipc/test_train_inference_python.sh ./test_tipc/configs/ppocr_det_mobile_params.txt 'lite_train_whole_infer' +``` + +- 模式3：whole_infer，不训练，全量数据预测，走通开源模型评估、动转静，检查inference model预测时间和精度; +```shell +bash test_tipc/prepare.sh ./test_tipc/configs/ppocr_det_mobile_params.txt 'whole_infer' +# 用法1: +bash test_tipc/test_train_inference_python.sh ./test_tipc/configs/ppocr_det_mobile_params.txt 'whole_infer' +# 用法2: 指定GPU卡预测，第三个传入参数为GPU卡号 +bash test_tipc/test_train_inference_python.sh ./test_tipc/configs/ppocr_det_mobile_params.txt 'whole_infer' '1' +``` + +- 模式4：whole_train_whole_infer，CE： 全量数据训练，全量数据预测，验证模型训练精度，预测精度，预测速度； +```shell +bash test_tipc/prepare.sh ./test_tipc/configs/ppocr_det_mobile_params.txt 'whole_train_whole_infer' +bash test_tipc/test_train_inference_python.sh ./test_tipc/configs/ppocr_det_mobile_params.txt 'whole_train_whole_infer' +``` + +- 模式5：klquant_whole_infer，测试离线量化； +```shell +bash test_tipc/prepare.sh ./test_tipc/configs/ppocr_det_mobile_params.txt 'klquant_whole_infer' +bash test_tipc/test_train_inference_python.sh test_tipc/configs/ppocr_det_mobile_params.txt 'klquant_whole_infer' +``` + +运行相应指令后，在`test_tipc/output`文件夹下自动会保存运行日志。如'lite_train_lite_infer'模式下，会运行训练+inference的链条，因此，在`test_tipc/output`文件夹有以下文件： +``` +test_tipc/output/ +|- results_python.log # 运行指令状态的日志 +|- norm_train_gpus_0_autocast_null/ # GPU 0号卡上正常训练的训练日志和模型保存文件夹 +|- pact_train_gpus_0_autocast_null/ # GPU 0号卡上量化训练的训练日志和模型保存文件夹 +...... +|- python_infer_cpu_usemkldnn_True_threads_1_batchsize_1.log # CPU上开启Mkldnn线程数设置为1，测试batch_size=1条件下的预测运行日志 +|- python_infer_gpu_usetrt_True_precision_fp16_batchsize_1.log # GPU上开启TensorRT，测试batch_size=1的半精度预测日志 +...... +``` + +其中`results_python.log`中包含了每条指令的运行状态，如果运行成功会输出： +``` +Run successfully with command - python3.7 tools/train.py -c tests/configs/det_mv3_db.yml -o Global.pretrained_model=./pretrain_models/MobileNetV3_large_x0_5_pretrained Global.use_gpu=True Global.save_model_dir=./tests/output/norm_train_gpus_0_autocast_null Global.epoch_num=1 Train.loader.batch_size_per_card=2 ! +Run successfully with command - python3.7 tools/export_model.py -c tests/configs/det_mv3_db.yml -o Global.pretrained_model=./tests/output/norm_train_gpus_0_autocast_null/latest Global.save_inference_dir=./tests/output/norm_train_gpus_0_autocast_null! +...... +``` +如果运行失败，会输出： +``` +Run failed with command - python3.7 tools/train.py -c tests/configs/det_mv3_db.yml -o Global.pretrained_model=./pretrain_models/MobileNetV3_large_x0_5_pretrained Global.use_gpu=True Global.save_model_dir=./tests/output/norm_train_gpus_0_autocast_null Global.epoch_num=1 Train.loader.batch_size_per_card=2 ! +Run failed with command - python3.7 tools/export_model.py -c tests/configs/det_mv3_db.yml -o Global.pretrained_model=./tests/output/norm_train_gpus_0_autocast_null/latest Global.save_inference_dir=./tests/output/norm_train_gpus_0_autocast_null! +...... +``` +可以很方便的根据`results_python.log`中的内容判定哪一个指令运行错误。 + + +### 2.3 精度测试 + +使用compare_results.py脚本比较模型预测的结果是否符合预期，主要步骤包括： +- 提取日志中的预测坐标； +- 从本地文件中提取保存好的坐标结果； +- 比较上述两个结果是否符合精度预期，误差大于设置阈值时会报错。 + +#### 使用方式 +运行命令： +```shell +python3.7 test_tipc/compare_results.py --gt_file=./test_tipc/results/python_*.txt --log_file=./test_tipc/output/python_*.log --atol=1e-3 --rtol=1e-3 +``` + +参数介绍： +- gt_file： 指向事先保存好的预测结果路径，支持*.txt 结尾，会自动索引*.txt格式的文件，文件默认保存在test_tipc/result/ 文件夹下 +- log_file: 指向运行test_tipc/test_train_inference_python.sh 脚本的infer模式保存的预测日志，预测日志中打印的有预测结果，比如：文本框，预测文本，类别等等，同样支持python_infer_*.log格式传入 +- atol: 设置的绝对误差 +- rtol: 设置的相对误差 + +#### 运行结果 + +正常运行效果如下图： +
+
+出现不一致结果时的运行输出：
+
+
+
+## 3. 更多教程
+本文档为功能测试用，更丰富的训练预测使用教程请参考：
+[模型训练](https://github.com/PaddlePaddle/PaddleOCR/blob/dygraph/doc/doc_ch/training.md)
+[基于Python预测引擎推理](https://github.com/PaddlePaddle/PaddleOCR/blob/dygraph/doc/doc_ch/inference.md)
diff --git a/test_tipc/docs/win_test_train_inference_python.md b/test_tipc/docs/win_test_train_inference_python.md
new file mode 100644
index 0000000000000000000000000000000000000000..b1bb5470de36d14071ed86e29bebf29b104289dd
--- /dev/null
+++ b/test_tipc/docs/win_test_train_inference_python.md
@@ -0,0 +1,151 @@
+# Windows端基础训练预测功能测试
+
+Windows端基础训练预测功能测试的主程序为`test_train_inference_python.sh`，可以测试基于Python的模型训练、评估、推理等基本功能，包括裁剪、量化、蒸馏。
+
+## 1. 测试结论汇总
+
+- 训练相关：
+
+| 算法名称 | 模型名称 | 单机单卡 | 单机多卡 | 多机多卡 | 模型压缩（单机多卡） |
+| :---- | :---- | :---- | :---- | :---- | :---- |
+| DB | ch_ppocr_mobile_v2.0_det| 正常训练 混合精度 | - | - | 正常训练：FPGM裁剪、PACT量化
离线量化（无需训练） | + + +- 预测相关：基于训练是否使用量化，可以将训练产出的模型可以分为`正常模型`和`量化模型`，这两类模型对应的预测功能汇总如下： + +| 模型类型 |device | batchsize | tensorrt | mkldnn | cpu多线程 | +| ---- | ---- | ---- | :----: | :----: | :----: | +| 正常模型 | GPU | 1/6 | fp32/fp16 | - | - | +| 正常模型 | CPU | 1/6 | - | fp32/fp16 | 支持 | +| 量化模型 | GPU | 1/6 | int8 | - | - | +| 量化模型 | CPU | 1/6 | - | int8 | 支持 | + + +## 2. 测试流程 + +运行环境配置请参考[文档](./install.md)的内容配置TIPC的运行环境。 + +另外，由于Windows上和linux的路径管理方式不同，可以在win上安装gitbash终端，在gitbash中执行指令的方式和在linux端执行指令方式相同，更方便tipc测试。gitbash[下载链接](https://git-scm.com/download/win)。 + + +### 2.1 安装依赖 +- 安装PaddlePaddle >= 2.0 +- 安装PaddleOCR依赖 + ``` + pip install -r ../requirements.txt + ``` +- 安装autolog（规范化日志输出工具） + ``` + git clone https://github.com/LDOUBLEV/AutoLog + cd AutoLog + pip install -r requirements.txt + python setup.py bdist_wheel + pip install ./dist/auto_log-1.0.0-py3-none-any.whl + cd ../ + ``` +- 安装PaddleSlim (可选) + ``` + # 如果要测试量化、裁剪等功能，需要安装PaddleSlim + pip install paddleslim + ``` + + +### 2.2 功能测试 +先运行`prepare.sh`准备数据和模型，然后运行`test_train_inference_python.sh`进行测试，最终在```test_tipc/output```目录下生成`python_infer_*.log`格式的日志文件。 + + +`test_train_inference_python.sh`包含5种运行模式，每种模式的运行数据不同，分别用于测试速度和精度，分别是： + +- 模式1：lite_train_lite_infer，使用少量数据训练，用于快速验证训练到预测的走通流程，不验证精度和速度； +```shell +bash test_tipc/prepare.sh ./test_tipc/configs/win_ppocr_det_mobile_params.txt 'lite_train_lite_infer' +bash test_tipc/test_train_inference_python.sh ./test_tipc/configs/win_ppocr_det_mobile_params.txt 'lite_train_lite_infer' +``` + +- 模式2：lite_train_whole_infer，使用少量数据训练，一定量数据预测，用于验证训练后的模型执行预测，预测速度是否合理； +```shell +bash test_tipc/prepare.sh ./test_tipc/configs/win_ppocr_det_mobile_params.txt 'lite_train_whole_infer' +bash test_tipc/test_train_inference_python.sh ./test_tipc/configs/win_ppocr_det_mobile_params.txt 'lite_train_whole_infer' +``` + +- 模式3：whole_infer，不训练，全量数据预测，走通开源模型评估、动转静，检查inference model预测时间和精度; +```shell +bash test_tipc/prepare.sh ./test_tipc/configs/win_ppocr_det_mobile_params.txt 'whole_infer' +# 用法1: +bash test_tipc/test_train_inference_python.sh ./test_tipc/configs/win_ppocr_det_mobile_params.txt 'whole_infer' +# 用法2: 指定GPU卡预测，第三个传入参数为GPU卡号 +bash test_tipc/test_train_inference_python.sh ./test_tipc/configs/win_ppocr_det_mobile_params.txt 'whole_infer' '1' +``` + +- 模式4：whole_train_whole_infer，CE： 全量数据训练，全量数据预测，验证模型训练精度，预测精度，预测速度； +```shell +bash test_tipc/prepare.sh ./test_tipc/configs/win_ppocr_det_mobile_params.txt 'whole_train_whole_infer' +bash test_tipc/test_train_inference_python.sh ./test_tipc/configs/win_ppocr_det_mobile_params.txt 'whole_train_whole_infer' +``` + +- 模式5：klquant_whole_infer，测试离线量化； +```shell +bash test_tipc/prepare.sh ./test_tipc/configs/win_ppocr_det_mobile_params.txt 'klquant_whole_infer' +bash test_tipc/test_train_inference_python.sh test_tipc/configs/win_ppocr_det_mobile_params.txt 'klquant_whole_infer' +``` + + +运行相应指令后，在`test_tipc/output`文件夹下自动会保存运行日志。如'lite_train_lite_infer'模式下，会运行训练+inference的链条，因此，在`test_tipc/output`文件夹有以下文件： +``` +test_tipc/output/ +|- results_python.log # 运行指令状态的日志 +|- norm_train_gpus_0_autocast_null/ # GPU 0号卡上正常训练的训练日志和模型保存文件夹 +|- pact_train_gpus_0_autocast_null/ # GPU 0号卡上量化训练的训练日志和模型保存文件夹 +...... +|- python_infer_cpu_usemkldnn_True_threads_1_batchsize_1.log # CPU上开启Mkldnn线程数设置为1，测试batch_size=1条件下的预测运行日志 +|- python_infer_gpu_usetrt_True_precision_fp16_batchsize_1.log # GPU上开启TensorRT，测试batch_size=1的半精度预测日志 +...... +``` + +其中`results_python.log`中包含了每条指令的运行状态，如果运行成功会输出： +``` +Run successfully with command - python3.7 tools/train.py -c tests/configs/det_mv3_db.yml -o Global.pretrained_model=./pretrain_models/MobileNetV3_large_x0_5_pretrained Global.use_gpu=True Global.save_model_dir=./tests/output/norm_train_gpus_0_autocast_null Global.epoch_num=1 Train.loader.batch_size_per_card=2 ! +Run successfully with command - python3.7 tools/export_model.py -c tests/configs/det_mv3_db.yml -o Global.pretrained_model=./tests/output/norm_train_gpus_0_autocast_null/latest Global.save_inference_dir=./tests/output/norm_train_gpus_0_autocast_null! +...... +``` +如果运行失败，会输出： +``` +Run failed with command - python3.7 tools/train.py -c tests/configs/det_mv3_db.yml -o Global.pretrained_model=./pretrain_models/MobileNetV3_large_x0_5_pretrained Global.use_gpu=True Global.save_model_dir=./tests/output/norm_train_gpus_0_autocast_null Global.epoch_num=1 Train.loader.batch_size_per_card=2 ! +Run failed with command - python3.7 tools/export_model.py -c tests/configs/det_mv3_db.yml -o Global.pretrained_model=./tests/output/norm_train_gpus_0_autocast_null/latest Global.save_inference_dir=./tests/output/norm_train_gpus_0_autocast_null! +...... +``` +可以很方便的根据`results_python.log`中的内容判定哪一个指令运行错误。 + + +### 2.3 精度测试 + +使用compare_results.py脚本比较模型预测的结果是否符合预期，主要步骤包括： +- 提取日志中的预测坐标； +- 从本地文件中提取保存好的坐标结果； +- 比较上述两个结果是否符合精度预期，误差大于设置阈值时会报错。 + +#### 使用方式 +运行命令： +```shell +python test_tipc/compare_results.py --gt_file=./test_tipc/results/python_*.txt --log_file=./test_tipc/output/python_*.log --atol=1e-3 --rtol=1e-3 +``` + +参数介绍： +- gt_file： 指向事先保存好的预测结果路径，支持*.txt 结尾，会自动索引*.txt格式的文件，文件默认保存在test_tipc/result/ 文件夹下 +- log_file: 指向运行test_tipc/test_train_inference_python.sh 脚本的infer模式保存的预测日志，预测日志中打印的有预测结果，比如：文本框，预测文本，类别等等，同样支持python_infer_*.log格式传入 +- atol: 设置的绝对误差 +- rtol: 设置的相对误差 + +#### 运行结果 + +正常运行效果如下图： +
+
+出现不一致结果时的运行输出：
+
+
+
+## 3. 更多教程
+本文档为功能测试用，更丰富的训练预测使用教程请参考：
+[模型训练](https://github.com/PaddlePaddle/PaddleOCR/blob/dygraph/doc/doc_ch/training.md)
+[基于Python预测引擎推理](https://github.com/PaddlePaddle/PaddleOCR/blob/dygraph/doc/doc_ch/inference.md)
diff --git a/test_tipc/prepare.sh b/test_tipc/prepare.sh
new file mode 100644
index 0000000000000000000000000000000000000000..f3ad242538a9471af237c804eae343da06e2b9dd
--- /dev/null
+++ b/test_tipc/prepare.sh
@@ -0,0 +1,220 @@
+#!/bin/bash
+FILENAME=$1
+
+# MODE be one of ['lite_train_lite_infer' 'lite_train_whole_infer' 'whole_train_whole_infer',
+# 'whole_infer', 'klquant_whole_infer',
+# 'cpp_infer', 'serving_infer', 'lite_infer']
+
+MODE=$2
+
+dataline=$(cat ${FILENAME})
+
+# parser params
+IFS=$'\n'
+lines=(${dataline})
+function func_parser_key(){
+ strs=$1
+ IFS=":"
+ array=(${strs})
+ tmp=${array[0]}
+ echo ${tmp}
+}
+function func_parser_value(){
+ strs=$1
+ IFS=":"
+ array=(${strs})
+ tmp=${array[1]}
+ echo ${tmp}
+}
+IFS=$'\n'
+# The training params
+model_name=$(func_parser_value "${lines[1]}")
+
+trainer_list=$(func_parser_value "${lines[14]}")
+
+# MODE be one of ['lite_train_lite_infer' 'lite_train_whole_infer' 'whole_train_whole_infer',
+# 'whole_infer', 'klquant_whole_infer',
+# 'cpp_infer', 'serving_infer', 'lite_infer']
+MODE=$2
+
+if [ ${MODE} = "lite_train_lite_infer" ];then
+ # pretrain lite train data
+ wget -nc -P ./pretrain_models/ https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/MobileNetV3_large_x0_5_pretrained.pdparams --no-check-certificate
+ wget -nc -P ./pretrain_models/ https://paddleocr.bj.bcebos.com/dygraph_v2.0/en/det_mv3_db_v2.0_train.tar --no-check-certificate
+ if [ ${model_name} == "PPOCRv2_ocr_det" ]; then
+ wget -nc -P ./pretrain_models/ https://paddleocr.bj.bcebos.com/PP-OCRv2/chinese/ch_PP-OCRv2_det_distill_train.tar --no-check-certificate
+ cd ./pretrain_models/ && tar xf ch_PP-OCRv2_det_distill_train.tar && cd ../
+ fi
+ cd ./pretrain_models/ && tar xf det_mv3_db_v2.0_train.tar && cd ../
+ rm -rf ./train_data/icdar2015
+ rm -rf ./train_data/ic15_data
+ wget -nc -P ./train_data/ https://paddleocr.bj.bcebos.com/dygraph_v2.0/test/icdar2015_lite.tar --no-check-certificate
+ wget -nc -P ./train_data/ https://paddleocr.bj.bcebos.com/dygraph_v2.0/test/ic15_data.tar --no-check-certificate
+ wget -nc -P ./inference https://paddleocr.bj.bcebos.com/dygraph_v2.0/test/rec_inference.tar --no-check-certificate
+ wget -nc -P ./deploy/slim/prune https://paddleocr.bj.bcebos.com/dygraph_v2.0/test/sen.pickle --no-check-certificate
+
+ cd ./train_data/ && tar xf icdar2015_lite.tar && tar xf ic15_data.tar
+ ln -s ./icdar2015_lite ./icdar2015
+ cd ../
+ cd ./inference && tar xf rec_inference.tar && cd ../
+elif [ ${MODE} = "whole_train_whole_infer" ];then
+ wget -nc -P ./pretrain_models/ https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/MobileNetV3_large_x0_5_pretrained.pdparams --no-check-certificate
+ rm -rf ./train_data/icdar2015
+ rm -rf ./train_data/ic15_data
+ wget -nc -P ./train_data/ https://paddleocr.bj.bcebos.com/dygraph_v2.0/test/icdar2015.tar --no-check-certificate
+ wget -nc -P ./train_data/ https://paddleocr.bj.bcebos.com/dygraph_v2.0/test/ic15_data.tar --no-check-certificate
+ cd ./train_data/ && tar xf icdar2015.tar && tar xf ic15_data.tar && cd ../
+ if [ ${model_name} == "PPOCRv2_ocr_det" ]; then
+ wget -nc -P ./pretrain_models/ https://paddleocr.bj.bcebos.com/PP-OCRv2/chinese/ch_PP-OCRv2_det_distill_train.tar --no-check-certificate
+ cd ./pretrain_models/ && tar xf ch_PP-OCRv2_det_distill_train.tar && cd ../
+ fi
+elif [ ${MODE} = "lite_train_whole_infer" ];then
+ wget -nc -P ./pretrain_models/ https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/MobileNetV3_large_x0_5_pretrained.pdparams --no-check-certificate
+ rm -rf ./train_data/icdar2015
+ rm -rf ./train_data/ic15_data
+ wget -nc -P ./train_data/ https://paddleocr.bj.bcebos.com/dygraph_v2.0/test/icdar2015_infer.tar --no-check-certificate
+ wget -nc -P ./train_data/ https://paddleocr.bj.bcebos.com/dygraph_v2.0/test/ic15_data.tar --no-check-certificate
+ cd ./train_data/ && tar xf icdar2015_infer.tar && tar xf ic15_data.tar
+ ln -s ./icdar2015_infer ./icdar2015
+ cd ../
+ if [ ${model_name} == "PPOCRv2_ocr_det" ]; then
+ wget -nc -P ./pretrain_models/ https://paddleocr.bj.bcebos.com/PP-OCRv2/chinese/ch_PP-OCRv2_det_distill_train.tar --no-check-certificate
+ cd ./pretrain_models/ && tar xf ch_PP-OCRv2_det_distill_train.tar && cd ../
+ fi
+elif [ ${MODE} = "whole_infer" ];then
+ if [ ${model_name} = "ocr_det" ]; then
+ eval_model_name="ch_ppocr_mobile_v2.0_det_train"
+ rm -rf ./train_data/icdar2015
+ wget -nc -P ./inference https://paddleocr.bj.bcebos.com/dygraph_v2.0/test/ch_det_data_50.tar --no-check-certificate
+ wget -nc -P ./inference https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_mobile_v2.0_det_train.tar --no-check-certificate
+ cd ./inference && tar xf ${eval_model_name}.tar && tar xf ch_det_data_50.tar && cd ../
+ elif [ ${model_name} = "ocr_server_det" ]; then
+ wget -nc -P ./inference https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_server_v2.0_det_train.tar --no-check-certificate
+ wget -nc -P ./inference https://paddleocr.bj.bcebos.com/dygraph_v2.0/test/ch_det_data_50.tar --no-check-certificate
+ cd ./inference && tar xf ch_ppocr_server_v2.0_det_train.tar && tar xf ch_det_data_50.tar && cd ../
+ elif [ ${model_name} = "ocr_system_mobile" ]; then
+ wget -nc -P ./inference https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_mobile_v2.0_det_infer.tar --no-check-certificate
+ wget -nc -P ./inference https://paddleocr.bj.bcebos.com/dygraph_v2.0/test/ch_det_data_50.tar --no-check-certificate
+ wget -nc -P ./inference https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_mobile_v2.0_rec_infer.tar --no-check-certificate
+ cd ./inference && tar xf ch_ppocr_mobile_v2.0_det_infer.tar && tar xf ch_ppocr_mobile_v2.0_rec_infer.tar && tar xf ch_det_data_50.tar && cd ../
+ elif [ ${model_name} = "ocr_system_server" ]; then
+ wget -nc -P ./inference https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_server_v2.0_det_infer.tar --no-check-certificate
+ wget -nc -P ./inference https://paddleocr.bj.bcebos.com/dygraph_v2.0/test/ch_det_data_50.tar --no-check-certificate
+ wget -nc -P ./inference https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_server_v2.0_rec_infer.tar --no-check-certificate
+ cd ./inference && tar xf ch_ppocr_server_v2.0_det_infer.tar && tar xf ch_ppocr_server_v2.0_rec_infer.tar && tar xf ch_det_data_50.tar && cd ../
+ elif [ ${model_name} = "ocr_rec" ]; then
+ rm -rf ./train_data/ic15_data
+ eval_model_name="ch_ppocr_mobile_v2.0_rec_infer"
+ wget -nc -P ./inference/ https://paddleocr.bj.bcebos.com/dygraph_v2.0/test/rec_inference.tar --no-check-certificate
+ wget -nc -P ./inference https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_mobile_v2.0_rec_infer.tar --no-check-certificate
+ cd ./inference && tar xf ${eval_model_name}.tar && tar xf rec_inference.tar && cd ../
+ elif [ ${model_name} = "ocr_server_rec" ]; then
+ rm -rf ./train_data/ic15_data
+ eval_model_name="ch_ppocr_server_v2.0_rec_infer"
+ wget -nc -P ./inference/ https://paddleocr.bj.bcebos.com/dygraph_v2.0/test/rec_inference.tar --no-check-certificate
+ wget -nc -P ./inference https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_server_v2.0_rec_infer.tar --no-check-certificate
+ cd ./inference && tar xf ${eval_model_name}.tar && tar xf rec_inference.tar && cd ../
+ fi
+
+ elif [ ${model_name} = "PPOCRv2_ocr_det" ]; then
+ eval_model_name="ch_PP-OCRv2_det_infer"
+ wget -nc -P ./inference https://paddleocr.bj.bcebos.com/dygraph_v2.0/test/ch_det_data_50.tar --no-check-certificate
+ wget -nc -P ./inference/ https://paddleocr.bj.bcebos.com/PP-OCRv2/chinese/ch_PP-OCRv2_det_infer.tar --no-check-certificate
+ cd ./inference && tar xf ${eval_model_name}.tar && tar xf ch_det_data_50.tar && cd ../
+ fi
+
+if [ ${MODE} = "klquant_whole_infer" ]; then
+ if [ ${model_name} = "ocr_det" ]; then
+ wget -nc -P ./inference https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_mobile_v2.0_det_infer.tar --no-check-certificate
+ wget -nc -P ./inference https://paddleocr.bj.bcebos.com/dygraph_v2.0/test/ch_det_data_50.tar --no-check-certificate
+ cd ./inference && tar xf ch_ppocr_mobile_v2.0_det_infer.tar && tar xf ch_det_data_50.tar && cd ../
+ fi
+ if [ ${model_name} = "PPOCRv2_ocr_det" ]; then
+ eval_model_name="ch_PP-OCRv2_det_infer"
+ wget -nc -P ./inference https://paddleocr.bj.bcebos.com/dygraph_v2.0/test/ch_det_data_50.tar --no-check-certificate
+ wget -nc -P ./inference/ https://paddleocr.bj.bcebos.com/PP-OCRv2/chinese/ch_PP-OCRv2_det_infer.tar --no-check-certificate
+ cd ./inference && tar xf ${eval_model_name}.tar && tar xf ch_det_data_50.tar && cd ../
+ fi
+fi
+
+if [ ${MODE} = "cpp_infer" ];then
+ if [ ${model_name} = "ocr_det" ]; then
+ wget -nc -P ./inference https://paddleocr.bj.bcebos.com/dygraph_v2.0/test/ch_det_data_50.tar --no-check-certificate
+ wget -nc -P ./inference https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_mobile_v2.0_det_infer.tar --no-check-certificate
+ cd ./inference && tar xf ch_ppocr_mobile_v2.0_det_infer.tar && tar xf ch_det_data_50.tar && cd ../
+ elif [ ${model_name} = "ocr_rec" ]; then
+ wget -nc -P ./inference/ https://paddleocr.bj.bcebos.com/dygraph_v2.0/test/rec_inference.tar --no-check-certificate
+ wget -nc -P ./inference https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_mobile_v2.0_rec_infer.tar --no-check-certificate
+ cd ./inference && tar xf ch_ppocr_mobile_v2.0_rec_infer.tar && tar xf rec_inference.tar && cd ../
+ elif [ ${model_name} = "ocr_system" ]; then
+ wget -nc -P ./inference https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_mobile_v2.0_det_infer.tar --no-check-certificate
+ wget -nc -P ./inference https://paddleocr.bj.bcebos.com/dygraph_v2.0/test/ch_det_data_50.tar --no-check-certificate
+ wget -nc -P ./inference https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_mobile_v2.0_rec_infer.tar --no-check-certificate
+ cd ./inference && tar xf ch_ppocr_mobile_v2.0_det_infer.tar && tar xf ch_ppocr_mobile_v2.0_rec_infer.tar && tar xf ch_det_data_50.tar && cd ../
+ fi
+fi
+
+if [ ${MODE} = "serving_infer" ];then
+ # prepare serving env
+ python_name=$(func_parser_value "${lines[2]}")
+ wget https://paddle-serving.bj.bcebos.com/chain/paddle_serving_server_gpu-0.0.0.post101-py3-none-any.whl
+ ${python_name} -m pip install install paddle_serving_server_gpu-0.0.0.post101-py3-none-any.whl
+ ${python_name} -m pip install paddle_serving_client==0.6.1
+ ${python_name} -m pip install paddle-serving-app==0.6.3
+ wget -nc -P ./inference https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_mobile_v2.0_det_infer.tar
+ wget -nc -P ./inference https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_mobile_v2.0_rec_infer.tar
+ wget -nc -P ./inference https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_server_v2.0_det_infer.tar
+ wget -nc -P ./inference https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_server_v2.0_rec_infer.tar
+ cd ./inference && tar xf ch_ppocr_mobile_v2.0_det_infer.tar && tar xf ch_ppocr_mobile_v2.0_rec_infer.tar && tar xf ch_ppocr_server_v2.0_rec_infer.tar && tar xf ch_ppocr_server_v2.0_det_infer.tar && cd ../
+fi
+
+
+if [ ${MODE} = "lite_infer" ];then
+ # prepare lite nb model and test data
+ current_dir=${PWD}
+ wget -nc -P ./models https://paddleocr.bj.bcebos.com/dygraph_v2.0/lite/ch_ppocr_mobile_v2.0_det_opt.nb
+ wget -nc -P ./models https://paddleocr.bj.bcebos.com/dygraph_v2.0/lite/ch_ppocr_mobile_v2.0_det_slim_opt.nb
+ wget -nc -P ./test_data https://paddleocr.bj.bcebos.com/dygraph_v2.0/test/icdar2015_lite.tar
+ cd ./test_data && tar -xf icdar2015_lite.tar && rm icdar2015_lite.tar && cd ../
+ # prepare lite env
+ export http_proxy=http://172.19.57.45:3128
+ export https_proxy=http://172.19.57.45:3128
+ paddlelite_url=https://github.com/PaddlePaddle/Paddle-Lite/releases/download/v2.9/inference_lite_lib.android.armv8.gcc.c++_shared.with_extra.with_cv.tar.gz
+ paddlelite_zipfile=$(echo $paddlelite_url | awk -F "/" '{print $NF}')
+ paddlelite_file=${paddlelite_zipfile:0:66}
+ wget ${paddlelite_url}
+ tar -xf ${paddlelite_zipfile}
+ mkdir -p ${paddlelite_file}/demo/cxx/ocr/test_lite
+ mv models test_data ${paddlelite_file}/demo/cxx/ocr/test_lite
+ cp ppocr/utils/ppocr_keys_v1.txt deploy/lite/config.txt ${paddlelite_file}/demo/cxx/ocr/test_lite
+ cp ./deploy/lite/* ${paddlelite_file}/demo/cxx/ocr/
+ cp ${paddlelite_file}/cxx/lib/libpaddle_light_api_shared.so ${paddlelite_file}/demo/cxx/ocr/test_lite
+ cp test_tipc/configs/ppocr_det_mobile_params.txt test_tipc/test_lite.sh test_tipc/common_func.sh ${paddlelite_file}/demo/cxx/ocr/test_lite
+ cd ${paddlelite_file}/demo/cxx/ocr/
+ git clone https://github.com/LDOUBLEV/AutoLog.git
+ unset http_proxy
+ unset https_proxy
+ make -j
+ sleep 1
+ make -j
+ cp ocr_db_crnn test_lite && cp test_lite/libpaddle_light_api_shared.so test_lite/libc++_shared.so
+ tar -cf test_lite.tar ./test_lite && cp test_lite.tar ${current_dir} && cd ${current_dir}
+fi
+
+
+if [ ${MODE} = "paddle2onnx_infer" ];then
+ # prepare serving env
+ python_name=$(func_parser_value "${lines[2]}")
+ ${python_name} -m pip install install paddle2onnx
+ ${python_name} -m pip install onnxruntime==1.4.0
+ # wget model
+ wget -nc -P ./inference https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_mobile_v2.0_det_infer.tar
+ wget -nc -P ./inference https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_mobile_v2.0_rec_infer.tar
+ wget -nc -P ./inference https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_server_v2.0_det_infer.tar
+ wget -nc -P ./inference https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_server_v2.0_rec_infer.tar
+ # wget data
+ wget -nc -P ./inference https://paddleocr.bj.bcebos.com/dygraph_v2.0/test/ch_det_data_50.tar
+ wget -nc -P ./inference/ https://paddleocr.bj.bcebos.com/dygraph_v2.0/test/rec_inference.tar
+ cd ./inference && tar xf ch_ppocr_mobile_v2.0_det_infer.tar && tar xf ch_ppocr_mobile_v2.0_rec_infer.tar && tar xf ch_ppocr_server_v2.0_rec_infer.tar && tar xf ch_ppocr_server_v2.0_det_infer.tar && tar xf ch_det_data_50.tar && tar xf rec_inference.tar && cd ../
+
+fi
diff --git a/PTDN/readme.md b/test_tipc/readme.md
similarity index 96%
rename from PTDN/readme.md
rename to test_tipc/readme.md
index 71e888a2fe05a0a6d700b40250dd80d5f6d041e0..1d8df7da6cf6d1319cedd329e4202fa674e8538b 100644
--- a/PTDN/readme.md
+++ b/test_tipc/readme.md
@@ -1,9 +1,9 @@
-# 推理部署导航
+# 飞桨训推一体认证
## 1. 简介
-飞桨除了基本的模型训练和预测，还提供了支持多端多平台的高性能推理部署工具。本文档提供了PaddleOCR中所有模型的推理部署导航PTDN（Paddle Train Deploy Navigation），方便用户查阅每种模型的推理部署打通情况，并可以进行一键测试。
+飞桨除了基本的模型训练和预测，还提供了支持多端多平台的高性能推理部署工具。本文档提供了PaddleOCR中所有模型的飞桨训推一体认证 (Training and Inference Pipeline Certification(TIPC)) 信息和测试工具，方便用户查阅每种模型的训练推理部署打通情况，并可以进行一键测试。
@@ -58,7 +58,7 @@
### 目录介绍
```shell
-PTDN/
+test_tipc/
├── configs/ # 配置文件目录
├── det_mv3_db.yml # 测试mobile版ppocr检测模型训练的yml文件
├── det_r50_vd_db.yml # 测试server版ppocr检测模型训练的yml文件
@@ -69,7 +69,7 @@ PTDN/
├── ppocr_sys_server_params.txt # 测试server版ppocr检测+识别模型串联的参数配置文件
├── ppocr_det_server_params.txt # 测试server版ppocr检测模型的参数配置文件
├── ppocr_rec_server_params.txt # 测试server版ppocr识别模型的参数配置文件
- ├── ...
+ ├── ...
├── results/ # 预先保存的预测结果，用于和实际预测结果进行精读比对
├── python_ppocr_det_mobile_results_fp32.txt # 预存的mobile版ppocr检测模型python预测fp32精度的结果
├── python_ppocr_det_mobile_results_fp16.txt # 预存的mobile版ppocr检测模型python预测fp16精度的结果
diff --git a/PTDN/results/cpp_ppocr_det_mobile_results_fp16.txt b/test_tipc/results/cpp_ppocr_det_mobile_results_fp16.txt
similarity index 100%
rename from PTDN/results/cpp_ppocr_det_mobile_results_fp16.txt
rename to test_tipc/results/cpp_ppocr_det_mobile_results_fp16.txt
diff --git a/PTDN/results/cpp_ppocr_det_mobile_results_fp32.txt b/test_tipc/results/cpp_ppocr_det_mobile_results_fp32.txt
similarity index 100%
rename from PTDN/results/cpp_ppocr_det_mobile_results_fp32.txt
rename to test_tipc/results/cpp_ppocr_det_mobile_results_fp32.txt
diff --git a/PTDN/results/python_ppocr_det_mobile_results_fp16.txt b/test_tipc/results/python_ppocr_det_mobile_results_fp16.txt
similarity index 100%
rename from PTDN/results/python_ppocr_det_mobile_results_fp16.txt
rename to test_tipc/results/python_ppocr_det_mobile_results_fp16.txt
diff --git a/PTDN/results/python_ppocr_det_mobile_results_fp32.txt b/test_tipc/results/python_ppocr_det_mobile_results_fp32.txt
similarity index 100%
rename from PTDN/results/python_ppocr_det_mobile_results_fp32.txt
rename to test_tipc/results/python_ppocr_det_mobile_results_fp32.txt
diff --git a/PTDN/test_inference_cpp.sh b/test_tipc/test_inference_cpp.sh
similarity index 99%
rename from PTDN/test_inference_cpp.sh
rename to test_tipc/test_inference_cpp.sh
index 124bdacb7dad04bdea07a62ba9c86b248be5a06d..3f8b54b189349aa9c011a56f6f12752b771ce43e 100644
--- a/PTDN/test_inference_cpp.sh
+++ b/test_tipc/test_inference_cpp.sh
@@ -1,5 +1,5 @@
#!/bin/bash
-source tests/common_func.sh
+source test_tipc/common_func.sh
FILENAME=$1
dataline=$(awk 'NR==52, NR==66{print}' $FILENAME)
@@ -35,7 +35,7 @@ cpp_benchmark_key=$(func_parser_key "${lines[14]}")
cpp_benchmark_value=$(func_parser_value "${lines[14]}")
-LOG_PATH="./tests/output"
+LOG_PATH="./test_tipc/output"
mkdir -p ${LOG_PATH}
status_log="${LOG_PATH}/results_cpp.log"
diff --git a/test_tipc/test_lite.sh b/test_tipc/test_lite.sh
new file mode 100644
index 0000000000000000000000000000000000000000..832003ba302fe86995e20029cdb019e72d9ce162
--- /dev/null
+++ b/test_tipc/test_lite.sh
@@ -0,0 +1,69 @@
+#!/bin/bash
+source ./common_func.sh
+export LD_LIBRARY_PATH=${PWD}:$LD_LIBRARY_PATH
+
+FILENAME=$1
+dataline=$(awk 'NR==101, NR==110{print}' $FILENAME)
+echo $dataline
+# parser params
+IFS=$'\n'
+lines=(${dataline})
+
+# parser lite inference
+lite_inference_cmd=$(func_parser_value "${lines[1]}")
+lite_model_dir_list=$(func_parser_value "${lines[2]}")
+lite_cpu_threads_list=$(func_parser_value "${lines[3]}")
+lite_batch_size_list=$(func_parser_value "${lines[4]}")
+lite_power_mode_list=$(func_parser_value "${lines[5]}")
+lite_infer_img_dir_list=$(func_parser_value "${lines[6]}")
+lite_config_dir=$(func_parser_value "${lines[7]}")
+lite_rec_dict_dir=$(func_parser_value "${lines[8]}")
+lite_benchmark_value=$(func_parser_value "${lines[9]}")
+
+LOG_PATH="./output"
+mkdir -p ${LOG_PATH}
+status_log="${LOG_PATH}/results.log"
+
+
+function func_lite(){
+ IFS='|'
+ _script=$1
+ _lite_model=$2
+ _log_path=$3
+ _img_dir=$4
+ _config=$5
+ if [[ $lite_model =~ "slim" ]]; then
+ precision="INT8"
+ else
+ precision="FP32"
+ fi
+ is_single_img=$(echo $_img_dir | grep -E ".jpg|.jpeg|.png|.JPEG|.JPG")
+ if [[ "$is_single_img" != "" ]]; then
+ single_img="True"
+ else
+ single_img="False"
+ fi
+
+ # lite inference
+ for num_threads in ${lite_cpu_threads_list[*]}; do
+ for power_mode in ${lite_power_mode_list[*]}; do
+ for batchsize in ${lite_batch_size_list[*]}; do
+ model_name=$(echo $lite_model | awk -F "/" '{print $NF}')
+ _save_log_path="${_log_path}/lite_${model_name}_precision_${precision}_batchsize_${batchsize}_threads_${num_threads}_powermode_${power_mode}_singleimg_${single_img}.log"
+ command="${_script} ${lite_model} ${precision} ${num_threads} ${batchsize} ${power_mode} ${_img_dir} ${_config} ${lite_benchmark_value} > ${_save_log_path} 2>&1"
+ eval ${command}
+ status_check $? "${command}" "${status_log}"
+ done
+ done
+ done
+}
+
+
+echo "################### run test ###################"
+IFS="|"
+for lite_model in ${lite_model_dir_list[*]}; do
+ #run lite inference
+ for img_dir in ${lite_infer_img_dir_list[*]}; do
+ func_lite "${lite_inference_cmd}" "${lite_model}" "${LOG_PATH}" "${img_dir}" "${lite_config_dir}"
+ done
+done
diff --git a/test_tipc/test_paddle2onnx.sh b/test_tipc/test_paddle2onnx.sh
new file mode 100644
index 0000000000000000000000000000000000000000..5dc6e65ec81e6b8674877fc686c8b3650ce93a59
--- /dev/null
+++ b/test_tipc/test_paddle2onnx.sh
@@ -0,0 +1,76 @@
+#!/bin/bash
+source test_tipc/common_func.sh
+
+FILENAME=$1
+
+dataline=$(cat ${FILENAME})
+lines=(${dataline})
+# common params
+model_name=$(func_parser_value "${lines[1]}")
+python=$(func_parser_value "${lines[2]}")
+
+
+# parser params
+dataline=$(awk 'NR==111, NR==123{print}' $FILENAME)
+IFS=$'\n'
+lines=(${dataline})
+
+# parser paddle2onnx
+padlle2onnx_cmd=$(func_parser_value "${lines[1]}")
+infer_model_dir_key=$(func_parser_key "${lines[2]}")
+infer_model_dir_value=$(func_parser_value "${lines[2]}")
+model_filename_key=$(func_parser_key "${lines[3]}")
+model_filename_value=$(func_parser_value "${lines[3]}")
+params_filename_key=$(func_parser_key "${lines[4]}")
+params_filename_value=$(func_parser_value "${lines[4]}")
+save_file_key=$(func_parser_key "${lines[5]}")
+save_file_value=$(func_parser_value "${lines[5]}")
+opset_version_key=$(func_parser_key "${lines[6]}")
+opset_version_value=$(func_parser_value "${lines[6]}")
+enable_onnx_checker_key=$(func_parser_key "${lines[7]}")
+enable_onnx_checker_value=$(func_parser_value "${lines[7]}")
+# parser onnx inference
+inference_py=$(func_parser_value "${lines[8]}")
+use_gpu_key=$(func_parser_key "${lines[9]}")
+use_gpu_value=$(func_parser_value "${lines[9]}")
+det_model_key=$(func_parser_key "${lines[10]}")
+image_dir_key=$(func_parser_key "${lines[11]}")
+image_dir_value=$(func_parser_value "${lines[11]}")
+
+
+LOG_PATH="./test_tipc/output"
+mkdir -p ./test_tipc/output
+status_log="${LOG_PATH}/results_paddle2onnx.log"
+
+
+function func_paddle2onnx(){
+ IFS='|'
+ _script=$1
+
+ # paddle2onnx
+ _save_log_path="${LOG_PATH}/paddle2onnx_infer_cpu.log"
+ set_dirname=$(func_set_params "${infer_model_dir_key}" "${infer_model_dir_value}")
+ set_model_filename=$(func_set_params "${model_filename_key}" "${model_filename_value}")
+ set_params_filename=$(func_set_params "${params_filename_key}" "${params_filename_value}")
+ set_save_model=$(func_set_params "${save_file_key}" "${save_file_value}")
+ set_opset_version=$(func_set_params "${opset_version_key}" "${opset_version_value}")
+ set_enable_onnx_checker=$(func_set_params "${enable_onnx_checker_key}" "${enable_onnx_checker_value}")
+ trans_model_cmd="${padlle2onnx_cmd} ${set_dirname} ${set_model_filename} ${set_params_filename} ${set_save_model} ${set_opset_version} ${set_enable_onnx_checker}"
+ eval $trans_model_cmd
+ last_status=${PIPESTATUS[0]}
+ status_check $last_status "${trans_model_cmd}" "${status_log}"
+ # python inference
+ set_gpu=$(func_set_params "${use_gpu_key}" "${use_gpu_value}")
+ set_model_dir=$(func_set_params "${det_model_key}" "${save_file_value}")
+ set_img_dir=$(func_set_params "${image_dir_key}" "${image_dir_value}")
+ infer_model_cmd="${python} ${inference_py} ${set_gpu} ${set_img_dir} ${set_model_dir} --use_onnx=True > ${_save_log_path} 2>&1 "
+ eval $infer_model_cmd
+ status_check $last_status "${infer_model_cmd}" "${status_log}"
+}
+
+
+echo "################### run test ###################"
+
+export Count=0
+IFS="|"
+func_paddle2onnx
\ No newline at end of file
diff --git a/PTDN/test_serving.sh b/test_tipc/test_serving.sh
similarity index 98%
rename from PTDN/test_serving.sh
rename to test_tipc/test_serving.sh
index ec79a46c9bf4b51c16b1c0ddfff41b772b13b0ae..be7b594c3848c423937c59336ce3bf686f8f228d 100644
--- a/PTDN/test_serving.sh
+++ b/test_tipc/test_serving.sh
@@ -1,5 +1,5 @@
#!/bin/bash
-source tests/common_func.sh
+source test_tipc/common_func.sh
FILENAME=$1
dataline=$(awk 'NR==67, NR==83{print}' $FILENAME)
@@ -36,8 +36,8 @@ web_precision_key=$(func_parser_key "${lines[15]}")
web_precision_list=$(func_parser_value "${lines[15]}")
pipeline_py=$(func_parser_value "${lines[16]}")
-LOG_PATH="../../tests/output"
-mkdir -p ./tests/output
+LOG_PATH="../../test_tipc/output"
+mkdir -p ./test_tipc/output
status_log="${LOG_PATH}/results_serving.log"
function func_serving(){
diff --git a/PTDN/test_train_inference_python.sh b/test_tipc/test_train_inference_python.sh
similarity index 91%
rename from PTDN/test_train_inference_python.sh
rename to test_tipc/test_train_inference_python.sh
index 28cc037801bb4c1f1bcc10a74855b8c146197f4d..eaeaf9684b1fed6738149d61d3697232e105a72f 100644
--- a/PTDN/test_train_inference_python.sh
+++ b/test_tipc/test_train_inference_python.sh
@@ -1,8 +1,8 @@
#!/bin/bash
-source tests/common_func.sh
+source test_tipc/common_func.sh
FILENAME=$1
-# MODE be one of ['lite_train_infer' 'whole_infer' 'whole_train_infer', 'infer', 'klquant_infer']
+# MODE be one of ['lite_train_lite_infer' 'lite_train_whole_infer' 'whole_train_whole_infer', 'whole_infer', 'klquant_whole_infer']
MODE=$2
dataline=$(awk 'NR==1, NR==51{print}' $FILENAME)
@@ -59,6 +59,7 @@ export_key1=$(func_parser_key "${lines[33]}")
export_value1=$(func_parser_value "${lines[33]}")
export_key2=$(func_parser_key "${lines[34]}")
export_value2=$(func_parser_value "${lines[34]}")
+inference_dir=$(func_parser_value "${lines[35]}")
# parser inference model
infer_model_dir_list=$(func_parser_value "${lines[36]}")
@@ -88,7 +89,7 @@ infer_key1=$(func_parser_key "${lines[50]}")
infer_value1=$(func_parser_value "${lines[50]}")
# parser klquant_infer
-if [ ${MODE} = "klquant_infer" ]; then
+if [ ${MODE} = "klquant_whole_infer" ]; then
dataline=$(awk 'NR==82, NR==98{print}' $FILENAME)
lines=(${dataline})
# parser inference model
@@ -119,7 +120,7 @@ if [ ${MODE} = "klquant_infer" ]; then
infer_value1=$(func_parser_value "${lines[15]}")
fi
-LOG_PATH="./tests/output"
+LOG_PATH="./test_tipc/output"
mkdir -p ${LOG_PATH}
status_log="${LOG_PATH}/results_python.log"
@@ -202,7 +203,7 @@ function func_inference(){
done
}
-if [ ${MODE} = "infer" ] || [ ${MODE} = "klquant_infer" ]; then
+if [ ${MODE} = "whole_infer" ] || [ ${MODE} = "klquant_whole_infer" ]; then
GPUID=$3
if [ ${#GPUID} -le 0 ];then
env=" "
@@ -245,6 +246,7 @@ else
for gpu in ${gpu_list[*]}; do
use_gpu=${USE_GPU_KEY[Count]}
Count=$(($Count + 1))
+ ips=""
if [ ${gpu} = "-1" ];then
env=""
elif [ ${#gpu} -le 1 ];then
@@ -264,6 +266,11 @@ else
env=" "
fi
for autocast in ${autocast_list[*]}; do
+ if [ ${autocast} = "amp" ]; then
+ set_amp_config="Global.use_amp=True Global.scale_loss=1024.0 Global.use_dynamic_loss_scaling=True"
+ else
+ set_amp_config=" "
+ fi
for trainer in ${trainer_list[*]}; do
flag_quant=False
if [ ${trainer} = ${pact_key} ]; then
@@ -290,7 +297,6 @@ else
if [ ${run_train} = "null" ]; then
continue
fi
-
set_autocast=$(func_set_params "${autocast_key}" "${autocast}")
set_epoch=$(func_set_params "${epoch_key}" "${epoch_num}")
set_pretrain=$(func_set_params "${pretrain_model_key}" "${pretrain_model_value}")
@@ -306,11 +312,11 @@ else
set_save_model=$(func_set_params "${save_model_key}" "${save_log}")
if [ ${#gpu} -le 2 ];then # train with cpu or single gpu
- cmd="${python} ${run_train} ${set_use_gpu} ${set_save_model} ${set_epoch} ${set_pretrain} ${set_autocast} ${set_batchsize} ${set_train_params1} "
- elif [ ${#gpu} -le 15 ];then # train with multi-gpu
- cmd="${python} -m paddle.distributed.launch --gpus=${gpu} ${run_train} ${set_save_model} ${set_epoch} ${set_pretrain} ${set_autocast} ${set_batchsize} ${set_train_params1}"
+ cmd="${python} ${run_train} ${set_use_gpu} ${set_save_model} ${set_epoch} ${set_pretrain} ${set_autocast} ${set_batchsize} ${set_train_params1} ${set_amp_config} "
+ elif [ ${#ips} -le 26 ];then # train with multi-gpu
+ cmd="${python} -m paddle.distributed.launch --gpus=${gpu} ${run_train} ${set_use_gpu} ${set_save_model} ${set_epoch} ${set_pretrain} ${set_autocast} ${set_batchsize} ${set_train_params1} ${set_amp_config}"
else # train with multi-machine
- cmd="${python} -m paddle.distributed.launch --ips=${ips} --gpus=${gpu} ${run_train} ${set_save_model} ${set_pretrain} ${set_epoch} ${set_autocast} ${set_batchsize} ${set_train_params1}"
+ cmd="${python} -m paddle.distributed.launch --ips=${ips} --gpus=${gpu} ${run_train} ${set_use_gpu} ${set_save_model} ${set_pretrain} ${set_epoch} ${set_autocast} ${set_batchsize} ${set_train_params1} ${set_amp_config}"
fi
# run train
eval "unset CUDA_VISIBLE_DEVICES"
@@ -342,7 +348,13 @@ else
#run inference
eval $env
save_infer_path="${save_log}"
- func_inference "${python}" "${inference_py}" "${save_infer_path}" "${LOG_PATH}" "${train_infer_img_dir}" "${flag_quant}"
+ if [ ${inference_dir} != "null" ] && [ ${inference_dir} != '##' ]; then
+ infer_model_dir="${save_infer_path}/${inference_dir}"
+ else
+ infer_model_dir=${save_infer_path}
+ fi
+ func_inference "${python}" "${inference_py}" "${infer_model_dir}" "${LOG_PATH}" "${train_infer_img_dir}" "${flag_quant}"
+
eval "unset CUDA_VISIBLE_DEVICES"
fi
done # done with: for trainer in ${trainer_list[*]}; do
diff --git a/tools/infer/predict_cls.py b/tools/infer/predict_cls.py
index 1c68494861e60b4aaef541a4e247071944cf420c..a25cac2600e67667badc76c648c1fcda12981a0f 100755
--- a/tools/infer/predict_cls.py
+++ b/tools/infer/predict_cls.py
@@ -47,6 +47,7 @@ class TextClassifier(object):
self.postprocess_op = build_post_process(postprocess_params)
self.predictor, self.input_tensor, self.output_tensors, _ = \
utility.create_predictor(args, 'cls', logger)
+ self.use_onnx = args.use_onnx
def resize_norm_img(self, img):
imgC, imgH, imgW = self.cls_image_shape
@@ -100,10 +101,16 @@ class TextClassifier(object):
norm_img_batch = np.concatenate(norm_img_batch)
norm_img_batch = norm_img_batch.copy()
- self.input_tensor.copy_from_cpu(norm_img_batch)
- self.predictor.run()
- prob_out = self.output_tensors[0].copy_to_cpu()
- self.predictor.try_shrink_memory()
+ if self.use_onnx:
+ input_dict = {}
+ input_dict[self.input_tensor.name] = norm_img_batch
+ outputs = self.predictor.run(self.output_tensors, input_dict)
+ prob_out = outputs[0]
+ else:
+ self.input_tensor.copy_from_cpu(norm_img_batch)
+ self.predictor.run()
+ prob_out = self.output_tensors[0].copy_to_cpu()
+ self.predictor.try_shrink_memory()
cls_result = self.postprocess_op(prob_out)
elapse += time.time() - starttime
for rno in range(len(cls_result)):
diff --git a/tools/infer/predict_det.py b/tools/infer/predict_det.py
index b24ad2bbb504caf1f262b4e47625348ce32d6fce..5dfe8d648f06f6382e8e101a6002f7f1b7441323 100755
--- a/tools/infer/predict_det.py
+++ b/tools/infer/predict_det.py
@@ -38,6 +38,7 @@ class TextDetector(object):
def __init__(self, args):
self.args = args
self.det_algorithm = args.det_algorithm
+ self.use_onnx = args.use_onnx
pre_process_list = [{
'DetResizeForTest': {
'limit_side_len': args.det_limit_side_len,
@@ -100,7 +101,12 @@ class TextDetector(object):
else:
logger.info("unknown det_algorithm:{}".format(self.det_algorithm))
sys.exit(0)
-
+ if self.use_onnx:
+ pre_process_list[0] = {
+ 'DetResizeForTest': {
+ 'image_shape': [640, 640]
+ }
+ }
self.preprocess_op = create_operators(pre_process_list)
self.postprocess_op = build_post_process(postprocess_params)
self.predictor, self.input_tensor, self.output_tensors, self.config = utility.create_predictor(
@@ -198,15 +204,19 @@ class TextDetector(object):
if self.args.benchmark:
self.autolog.times.stamp()
-
- self.input_tensor.copy_from_cpu(img)
- self.predictor.run()
- outputs = []
- for output_tensor in self.output_tensors:
- output = output_tensor.copy_to_cpu()
- outputs.append(output)
- if self.args.benchmark:
- self.autolog.times.stamp()
+ if self.use_onnx:
+ input_dict = {}
+ input_dict[self.input_tensor.name] = img
+ outputs = self.predictor.run(self.output_tensors, input_dict)
+ else:
+ self.input_tensor.copy_from_cpu(img)
+ self.predictor.run()
+ outputs = []
+ for output_tensor in self.output_tensors:
+ output = output_tensor.copy_to_cpu()
+ outputs.append(output)
+ if self.args.benchmark:
+ self.autolog.times.stamp()
preds = {}
if self.det_algorithm == "EAST":
diff --git a/tools/infer/predict_rec.py b/tools/infer/predict_rec.py
index 936994a215d10d543537b29cb41bfa42b42590c7..41982e3403b11dd4a1893f89af11a9201e0e15d7 100755
--- a/tools/infer/predict_rec.py
+++ b/tools/infer/predict_rec.py
@@ -73,6 +73,7 @@ class TextRecognizer(object):
self.predictor, self.input_tensor, self.output_tensors, self.config = \
utility.create_predictor(args, 'rec', logger)
self.benchmark = args.benchmark
+ self.use_onnx = args.use_onnx
if args.benchmark:
import auto_log
pid = os.getpid()
@@ -106,8 +107,9 @@ class TextRecognizer(object):
return norm_img.astype(np.float32) / 128. - 1.
assert imgC == img.shape[2]
- max_wh_ratio = max(max_wh_ratio, imgW / imgH)
imgW = int((32 * max_wh_ratio))
+ if self.use_onnx:
+ imgW = 100
h, w = img.shape[:2]
ratio = w / float(h)
if math.ceil(imgH * ratio) > imgW:
@@ -297,51 +299,72 @@ class TextRecognizer(object):
gsrm_slf_attn_bias1_list,
gsrm_slf_attn_bias2_list,
]
- input_names = self.predictor.get_input_names()
- for i in range(len(input_names)):
- input_tensor = self.predictor.get_input_handle(input_names[
- i])
- input_tensor.copy_from_cpu(inputs[i])
- self.predictor.run()
- outputs = []
- for output_tensor in self.output_tensors:
- output = output_tensor.copy_to_cpu()
- outputs.append(output)
- if self.benchmark:
- self.autolog.times.stamp()
- preds = {"predict": outputs[2]}
+ if self.use_onnx:
+ input_dict = {}
+ input_dict[self.input_tensor.name] = norm_img_batch
+ outputs = self.predictor.run(self.output_tensors,
+ input_dict)
+ preds = {"predict": outputs[2]}
+ else:
+ input_names = self.predictor.get_input_names()
+ for i in range(len(input_names)):
+ input_tensor = self.predictor.get_input_handle(
+ input_names[i])
+ input_tensor.copy_from_cpu(inputs[i])
+ self.predictor.run()
+ outputs = []
+ for output_tensor in self.output_tensors:
+ output = output_tensor.copy_to_cpu()
+ outputs.append(output)
+ if self.benchmark:
+ self.autolog.times.stamp()
+ preds = {"predict": outputs[2]}
elif self.rec_algorithm == "SAR":
valid_ratios = np.concatenate(valid_ratios)
inputs = [
norm_img_batch,
valid_ratios,
]
- input_names = self.predictor.get_input_names()
- for i in range(len(input_names)):
- input_tensor = self.predictor.get_input_handle(input_names[
- i])
- input_tensor.copy_from_cpu(inputs[i])
- self.predictor.run()
- outputs = []
- for output_tensor in self.output_tensors:
- output = output_tensor.copy_to_cpu()
- outputs.append(output)
- if self.benchmark:
- self.autolog.times.stamp()
- preds = outputs[0]
- else:
- self.input_tensor.copy_from_cpu(norm_img_batch)
- self.predictor.run()
- outputs = []
- for output_tensor in self.output_tensors:
- output = output_tensor.copy_to_cpu()
- outputs.append(output)
- if self.benchmark:
- self.autolog.times.stamp()
- if len(outputs) != 1:
- preds = outputs
+ if self.use_onnx:
+ input_dict = {}
+ input_dict[self.input_tensor.name] = norm_img_batch
+ outputs = self.predictor.run(self.output_tensors,
+ input_dict)
+ preds = outputs[0]
else:
+ input_names = self.predictor.get_input_names()
+ for i in range(len(input_names)):
+ input_tensor = self.predictor.get_input_handle(
+ input_names[i])
+ input_tensor.copy_from_cpu(inputs[i])
+ self.predictor.run()
+ outputs = []
+ for output_tensor in self.output_tensors:
+ output = output_tensor.copy_to_cpu()
+ outputs.append(output)
+ if self.benchmark:
+ self.autolog.times.stamp()
preds = outputs[0]
+ else:
+ if self.use_onnx:
+ input_dict = {}
+ input_dict[self.input_tensor.name] = norm_img_batch
+ outputs = self.predictor.run(self.output_tensors,
+ input_dict)
+ preds = outputs[0]
+ else:
+ self.input_tensor.copy_from_cpu(norm_img_batch)
+ self.predictor.run()
+ outputs = []
+ for output_tensor in self.output_tensors:
+ output = output_tensor.copy_to_cpu()
+ outputs.append(output)
+ if self.benchmark:
+ self.autolog.times.stamp()
+ if len(outputs) != 1:
+ preds = outputs
+ else:
+ preds = outputs[0]
rec_result = self.postprocess_op(preds)
for rno in range(len(rec_result)):
rec_res[indices[beg_img_no + rno]] = rec_result[rno]
diff --git a/tools/infer/utility.py b/tools/infer/utility.py
index 41a3c0f14b6378751a367a3709ad7943ee981a4e..a3cac647982b77f5ee54d8681b07e677987d9ccb 100755
--- a/tools/infer/utility.py
+++ b/tools/infer/utility.py
@@ -121,6 +121,7 @@ def init_args():
parser.add_argument("--save_log_path", type=str, default="./log_output/")
parser.add_argument("--show_log", type=str2bool, default=True)
+ parser.add_argument("--use_onnx", type=str2bool, default=False)
return parser
@@ -144,152 +145,163 @@ def create_predictor(args, mode, logger):
if model_dir is None:
logger.info("not find {} model file path {}".format(mode, model_dir))
sys.exit(0)
- model_file_path = model_dir + "/inference.pdmodel"
- params_file_path = model_dir + "/inference.pdiparams"
- if not os.path.exists(model_file_path):
- raise ValueError("not find model file path {}".format(model_file_path))
- if not os.path.exists(params_file_path):
- raise ValueError("not find params file path {}".format(
- params_file_path))
-
- config = inference.Config(model_file_path, params_file_path)
-
- if hasattr(args, 'precision'):
- if args.precision == "fp16" and args.use_tensorrt:
- precision = inference.PrecisionType.Half
- elif args.precision == "int8":
- precision = inference.PrecisionType.Int8
- else:
- precision = inference.PrecisionType.Float32
+ if args.use_onnx:
+ import onnxruntime as ort
+ model_file_path = model_dir
+ if not os.path.exists(model_file_path):
+ raise ValueError("not find model file path {}".format(
+ model_file_path))
+ sess = ort.InferenceSession(model_file_path)
+ return sess, sess.get_inputs()[0], None, None
+
else:
- precision = inference.PrecisionType.Float32
-
- if args.use_gpu:
- gpu_id = get_infer_gpuid()
- if gpu_id is None:
- raise ValueError(
- "Not found GPU in current device. Please check your device or set args.use_gpu as False"
- )
- config.enable_use_gpu(args.gpu_mem, 0)
- if args.use_tensorrt:
- config.enable_tensorrt_engine(
- precision_mode=precision,
- max_batch_size=args.max_batch_size,
- min_subgraph_size=args.min_subgraph_size)
- # skip the minmum trt subgraph
- if mode == "det":
- min_input_shape = {
- "x": [1, 3, 50, 50],
- "conv2d_92.tmp_0": [1, 120, 20, 20],
- "conv2d_91.tmp_0": [1, 24, 10, 10],
- "conv2d_59.tmp_0": [1, 96, 20, 20],
- "nearest_interp_v2_1.tmp_0": [1, 256, 10, 10],
- "nearest_interp_v2_2.tmp_0": [1, 256, 20, 20],
- "conv2d_124.tmp_0": [1, 256, 20, 20],
- "nearest_interp_v2_3.tmp_0": [1, 64, 20, 20],
- "nearest_interp_v2_4.tmp_0": [1, 64, 20, 20],
- "nearest_interp_v2_5.tmp_0": [1, 64, 20, 20],
- "elementwise_add_7": [1, 56, 2, 2],
- "nearest_interp_v2_0.tmp_0": [1, 256, 2, 2]
- }
- max_input_shape = {
- "x": [1, 3, 2000, 2000],
- "conv2d_92.tmp_0": [1, 120, 400, 400],
- "conv2d_91.tmp_0": [1, 24, 200, 200],
- "conv2d_59.tmp_0": [1, 96, 400, 400],
- "nearest_interp_v2_1.tmp_0": [1, 256, 200, 200],
- "conv2d_124.tmp_0": [1, 256, 400, 400],
- "nearest_interp_v2_2.tmp_0": [1, 256, 400, 400],
- "nearest_interp_v2_3.tmp_0": [1, 64, 400, 400],
- "nearest_interp_v2_4.tmp_0": [1, 64, 400, 400],
- "nearest_interp_v2_5.tmp_0": [1, 64, 400, 400],
- "elementwise_add_7": [1, 56, 400, 400],
- "nearest_interp_v2_0.tmp_0": [1, 256, 400, 400]
- }
- opt_input_shape = {
- "x": [1, 3, 640, 640],
- "conv2d_92.tmp_0": [1, 120, 160, 160],
- "conv2d_91.tmp_0": [1, 24, 80, 80],
- "conv2d_59.tmp_0": [1, 96, 160, 160],
- "nearest_interp_v2_1.tmp_0": [1, 256, 80, 80],
- "nearest_interp_v2_2.tmp_0": [1, 256, 160, 160],
- "conv2d_124.tmp_0": [1, 256, 160, 160],
- "nearest_interp_v2_3.tmp_0": [1, 64, 160, 160],
- "nearest_interp_v2_4.tmp_0": [1, 64, 160, 160],
- "nearest_interp_v2_5.tmp_0": [1, 64, 160, 160],
- "elementwise_add_7": [1, 56, 40, 40],
- "nearest_interp_v2_0.tmp_0": [1, 256, 40, 40]
- }
- min_pact_shape = {
- "nearest_interp_v2_26.tmp_0": [1, 256, 20, 20],
- "nearest_interp_v2_27.tmp_0": [1, 64, 20, 20],
- "nearest_interp_v2_28.tmp_0": [1, 64, 20, 20],
- "nearest_interp_v2_29.tmp_0": [1, 64, 20, 20]
- }
- max_pact_shape = {
- "nearest_interp_v2_26.tmp_0": [1, 256, 400, 400],
- "nearest_interp_v2_27.tmp_0": [1, 64, 400, 400],
- "nearest_interp_v2_28.tmp_0": [1, 64, 400, 400],
- "nearest_interp_v2_29.tmp_0": [1, 64, 400, 400]
- }
- opt_pact_shape = {
- "nearest_interp_v2_26.tmp_0": [1, 256, 160, 160],
- "nearest_interp_v2_27.tmp_0": [1, 64, 160, 160],
- "nearest_interp_v2_28.tmp_0": [1, 64, 160, 160],
- "nearest_interp_v2_29.tmp_0": [1, 64, 160, 160]
- }
- min_input_shape.update(min_pact_shape)
- max_input_shape.update(max_pact_shape)
- opt_input_shape.update(opt_pact_shape)
- elif mode == "rec":
- min_input_shape = {"x": [1, 3, 32, 10]}
- max_input_shape = {"x": [args.rec_batch_num, 3, 32, 2000]}
- opt_input_shape = {"x": [args.rec_batch_num, 3, 32, 320]}
- elif mode == "cls":
- min_input_shape = {"x": [1, 3, 48, 10]}
- max_input_shape = {"x": [args.rec_batch_num, 3, 48, 2000]}
- opt_input_shape = {"x": [args.rec_batch_num, 3, 48, 320]}
+ model_file_path = model_dir + "/inference.pdmodel"
+ params_file_path = model_dir + "/inference.pdiparams"
+ if not os.path.exists(model_file_path):
+ raise ValueError("not find model file path {}".format(
+ model_file_path))
+ if not os.path.exists(params_file_path):
+ raise ValueError("not find params file path {}".format(
+ params_file_path))
+
+ config = inference.Config(model_file_path, params_file_path)
+
+ if hasattr(args, 'precision'):
+ if args.precision == "fp16" and args.use_tensorrt:
+ precision = inference.PrecisionType.Half
+ elif args.precision == "int8":
+ precision = inference.PrecisionType.Int8
+ else:
+ precision = inference.PrecisionType.Float32
else:
- min_input_shape = {"x": [1, 3, 10, 10]}
- max_input_shape = {"x": [1, 3, 1000, 1000]}
- opt_input_shape = {"x": [1, 3, 500, 500]}
- config.set_trt_dynamic_shape_info(min_input_shape, max_input_shape,
- opt_input_shape)
+ precision = inference.PrecisionType.Float32
+
+ if args.use_gpu:
+ gpu_id = get_infer_gpuid()
+ if gpu_id is None:
+ raise ValueError(
+ "Not found GPU in current device. Please check your device or set args.use_gpu as False"
+ )
+ config.enable_use_gpu(args.gpu_mem, 0)
+ if args.use_tensorrt:
+ config.enable_tensorrt_engine(
+ precision_mode=precision,
+ max_batch_size=args.max_batch_size,
+ min_subgraph_size=args.min_subgraph_size)
+ # skip the minmum trt subgraph
+ if mode == "det":
+ min_input_shape = {
+ "x": [1, 3, 50, 50],
+ "conv2d_92.tmp_0": [1, 120, 20, 20],
+ "conv2d_91.tmp_0": [1, 24, 10, 10],
+ "conv2d_59.tmp_0": [1, 96, 20, 20],
+ "nearest_interp_v2_1.tmp_0": [1, 256, 10, 10],
+ "nearest_interp_v2_2.tmp_0": [1, 256, 20, 20],
+ "conv2d_124.tmp_0": [1, 256, 20, 20],
+ "nearest_interp_v2_3.tmp_0": [1, 64, 20, 20],
+ "nearest_interp_v2_4.tmp_0": [1, 64, 20, 20],
+ "nearest_interp_v2_5.tmp_0": [1, 64, 20, 20],
+ "elementwise_add_7": [1, 56, 2, 2],
+ "nearest_interp_v2_0.tmp_0": [1, 256, 2, 2]
+ }
+ max_input_shape = {
+ "x": [1, 3, 2000, 2000],
+ "conv2d_92.tmp_0": [1, 120, 400, 400],
+ "conv2d_91.tmp_0": [1, 24, 200, 200],
+ "conv2d_59.tmp_0": [1, 96, 400, 400],
+ "nearest_interp_v2_1.tmp_0": [1, 256, 200, 200],
+ "conv2d_124.tmp_0": [1, 256, 400, 400],
+ "nearest_interp_v2_2.tmp_0": [1, 256, 400, 400],
+ "nearest_interp_v2_3.tmp_0": [1, 64, 400, 400],
+ "nearest_interp_v2_4.tmp_0": [1, 64, 400, 400],
+ "nearest_interp_v2_5.tmp_0": [1, 64, 400, 400],
+ "elementwise_add_7": [1, 56, 400, 400],
+ "nearest_interp_v2_0.tmp_0": [1, 256, 400, 400]
+ }
+ opt_input_shape = {
+ "x": [1, 3, 640, 640],
+ "conv2d_92.tmp_0": [1, 120, 160, 160],
+ "conv2d_91.tmp_0": [1, 24, 80, 80],
+ "conv2d_59.tmp_0": [1, 96, 160, 160],
+ "nearest_interp_v2_1.tmp_0": [1, 256, 80, 80],
+ "nearest_interp_v2_2.tmp_0": [1, 256, 160, 160],
+ "conv2d_124.tmp_0": [1, 256, 160, 160],
+ "nearest_interp_v2_3.tmp_0": [1, 64, 160, 160],
+ "nearest_interp_v2_4.tmp_0": [1, 64, 160, 160],
+ "nearest_interp_v2_5.tmp_0": [1, 64, 160, 160],
+ "elementwise_add_7": [1, 56, 40, 40],
+ "nearest_interp_v2_0.tmp_0": [1, 256, 40, 40]
+ }
+ min_pact_shape = {
+ "nearest_interp_v2_26.tmp_0": [1, 256, 20, 20],
+ "nearest_interp_v2_27.tmp_0": [1, 64, 20, 20],
+ "nearest_interp_v2_28.tmp_0": [1, 64, 20, 20],
+ "nearest_interp_v2_29.tmp_0": [1, 64, 20, 20]
+ }
+ max_pact_shape = {
+ "nearest_interp_v2_26.tmp_0": [1, 256, 400, 400],
+ "nearest_interp_v2_27.tmp_0": [1, 64, 400, 400],
+ "nearest_interp_v2_28.tmp_0": [1, 64, 400, 400],
+ "nearest_interp_v2_29.tmp_0": [1, 64, 400, 400]
+ }
+ opt_pact_shape = {
+ "nearest_interp_v2_26.tmp_0": [1, 256, 160, 160],
+ "nearest_interp_v2_27.tmp_0": [1, 64, 160, 160],
+ "nearest_interp_v2_28.tmp_0": [1, 64, 160, 160],
+ "nearest_interp_v2_29.tmp_0": [1, 64, 160, 160]
+ }
+ min_input_shape.update(min_pact_shape)
+ max_input_shape.update(max_pact_shape)
+ opt_input_shape.update(opt_pact_shape)
+ elif mode == "rec":
+ min_input_shape = {"x": [1, 3, 32, 10]}
+ max_input_shape = {"x": [args.rec_batch_num, 3, 32, 2000]}
+ opt_input_shape = {"x": [args.rec_batch_num, 3, 32, 320]}
+ elif mode == "cls":
+ min_input_shape = {"x": [1, 3, 48, 10]}
+ max_input_shape = {"x": [args.rec_batch_num, 3, 48, 2000]}
+ opt_input_shape = {"x": [args.rec_batch_num, 3, 48, 320]}
+ else:
+ min_input_shape = {"x": [1, 3, 10, 10]}
+ max_input_shape = {"x": [1, 3, 1000, 1000]}
+ opt_input_shape = {"x": [1, 3, 500, 500]}
+ config.set_trt_dynamic_shape_info(min_input_shape, max_input_shape,
+ opt_input_shape)
- else:
- config.disable_gpu()
- if hasattr(args, "cpu_threads"):
- config.set_cpu_math_library_num_threads(args.cpu_threads)
else:
- # default cpu threads as 10
- config.set_cpu_math_library_num_threads(10)
- if args.enable_mkldnn:
- # cache 10 different shapes for mkldnn to avoid memory leak
- config.set_mkldnn_cache_capacity(10)
- config.enable_mkldnn()
- if args.precision == "fp16":
- config.enable_mkldnn_bfloat16()
- # enable memory optim
- config.enable_memory_optim()
- config.disable_glog_info()
-
- config.delete_pass("conv_transpose_eltwiseadd_bn_fuse_pass")
- if mode == 'table':
- config.delete_pass("fc_fuse_pass") # not supported for table
- config.switch_use_feed_fetch_ops(False)
- config.switch_ir_optim(True)
-
- # create predictor
- predictor = inference.create_predictor(config)
- input_names = predictor.get_input_names()
- for name in input_names:
- input_tensor = predictor.get_input_handle(name)
- output_names = predictor.get_output_names()
- output_tensors = []
- for output_name in output_names:
- output_tensor = predictor.get_output_handle(output_name)
- output_tensors.append(output_tensor)
- return predictor, input_tensor, output_tensors, config
+ config.disable_gpu()
+ if hasattr(args, "cpu_threads"):
+ config.set_cpu_math_library_num_threads(args.cpu_threads)
+ else:
+ # default cpu threads as 10
+ config.set_cpu_math_library_num_threads(10)
+ if args.enable_mkldnn:
+ # cache 10 different shapes for mkldnn to avoid memory leak
+ config.set_mkldnn_cache_capacity(10)
+ config.enable_mkldnn()
+ if args.precision == "fp16":
+ config.enable_mkldnn_bfloat16()
+ # enable memory optim
+ config.enable_memory_optim()
+ config.disable_glog_info()
+
+ config.delete_pass("conv_transpose_eltwiseadd_bn_fuse_pass")
+ if mode == 'table':
+ config.delete_pass("fc_fuse_pass") # not supported for table
+ config.switch_use_feed_fetch_ops(False)
+ config.switch_ir_optim(True)
+
+ # create predictor
+ predictor = inference.create_predictor(config)
+ input_names = predictor.get_input_names()
+ for name in input_names:
+ input_tensor = predictor.get_input_handle(name)
+ output_names = predictor.get_output_names()
+ output_tensors = []
+ for output_name in output_names:
+ output_tensor = predictor.get_output_handle(output_name)
+ output_tensors.append(output_tensor)
+ return predictor, input_tensor, output_tensors, config
def get_infer_gpuid():
diff --git a/tools/program.py b/tools/program.py
index 798e6dff297ad1149942488cca1d5540f1924867..d110f70704028948dff2bc889e07d128e0bc94ea 100755
--- a/tools/program.py
+++ b/tools/program.py
@@ -159,7 +159,8 @@ def train(config,
eval_class,
pre_best_model_dict,
logger,
- vdl_writer=None):
+ vdl_writer=None,
+ scaler=None):
cal_metric_during_train = config['Global'].get('cal_metric_during_train',
False)
log_smooth_window = config['Global']['log_smooth_window']
@@ -211,33 +212,49 @@ def train(config,
for epoch in range(start_epoch, epoch_num + 1):
train_dataloader = build_dataloader(
config, 'Train', device, logger, seed=epoch)
- train_batch_cost = 0.0
train_reader_cost = 0.0
- batch_sum = 0
- batch_start = time.time()
+ train_run_cost = 0.0
+ total_samples = 0
+ reader_start = time.time()
max_iter = len(train_dataloader) - 1 if platform.system(
) == "Windows" else len(train_dataloader)
for idx, batch in enumerate(train_dataloader):
profiler.add_profiler_step(profiler_options)
- train_reader_cost += time.time() - batch_start
+ train_reader_cost += time.time() - reader_start
if idx >= max_iter:
break
lr = optimizer.get_lr()
images = batch[0]
if use_srn:
model_average = True
- if model_type == 'table' or extra_input:
- preds = model(images, data=batch[1:])
+
+ train_start = time.time()
+ # use amp
+ if scaler:
+ with paddle.amp.auto_cast():
+ if model_type == 'table' or extra_input:
+ preds = model(images, data=batch[1:])
+ else:
+ preds = model(images)
else:
- preds = model(images)
+ if model_type == 'table' or extra_input:
+ preds = model(images, data=batch[1:])
+ else:
+ preds = model(images)
loss = loss_class(preds, batch)
avg_loss = loss['loss']
- avg_loss.backward()
- optimizer.step()
+
+ if scaler:
+ scaled_avg_loss = scaler.scale(avg_loss)
+ scaled_avg_loss.backward()
+ scaler.minimize(optimizer, scaled_avg_loss)
+ else:
+ avg_loss.backward()
+ optimizer.step()
optimizer.clear_grad()
- train_batch_cost += time.time() - batch_start
- batch_sum += len(images)
+ train_run_cost += time.time() - train_start
+ total_samples += len(images)
if not isinstance(lr_scheduler, float):
lr_scheduler.step()
@@ -268,12 +285,13 @@ def train(config,
logs = train_stats.log()
strs = 'epoch: [{}/{}], iter: {}, {}, reader_cost: {:.5f} s, batch_cost: {:.5f} s, samples: {}, ips: {:.5f}'.format(
epoch, epoch_num, global_step, logs, train_reader_cost /
- print_batch_step, train_batch_cost / print_batch_step,
- batch_sum, batch_sum / train_batch_cost)
+ print_batch_step, (train_reader_cost + train_run_cost) /
+ print_batch_step, total_samples,
+ total_samples / (train_reader_cost + train_run_cost))
logger.info(strs)
- train_batch_cost = 0.0
train_reader_cost = 0.0
- batch_sum = 0
+ train_run_cost = 0.0
+ total_samples = 0
# eval
if global_step > start_eval_step and \
(global_step - start_eval_step) % eval_batch_step == 0 and dist.get_rank() == 0:
@@ -326,7 +344,7 @@ def train(config,
global_step)
global_step += 1
optimizer.clear_grad()
- batch_start = time.time()
+ reader_start = time.time()
if dist.get_rank() == 0:
save_model(
model,
@@ -367,7 +385,11 @@ def eval(model,
with paddle.no_grad():
total_frame = 0.0
total_time = 0.0
- pbar = tqdm(total=len(valid_dataloader), desc='eval model:')
+ pbar = tqdm(
+ total=len(valid_dataloader),
+ desc='eval model:',
+ position=0,
+ leave=True)
max_iter = len(valid_dataloader) - 1 if platform.system(
) == "Windows" else len(valid_dataloader)
for idx, batch in enumerate(valid_dataloader):
@@ -436,8 +458,6 @@ def get_center(model, eval_dataloader, post_process_class):
batch = [item.numpy() for item in batch]
# Obtain usable results from post-processing methods
- total_time += time.time() - start
- # Evaluate the results of the current batch
post_result = post_process_class(preds, batch[1])
#update char_center
@@ -480,11 +500,6 @@ def preprocess(is_train=False):
'CLS', 'PGNet', 'Distillation', 'NRTR', 'TableAttn', 'SAR', 'PSE',
'SEED'
]
- windows_not_support_list = ['PSE']
- if platform.system() == "Windows" and alg in windows_not_support_list:
- logger.warning('{} is not support in Windows now'.format(
- windows_not_support_list))
- sys.exit()
device = 'gpu:{}'.format(dist.ParallelEnv().dev_id) if use_gpu else 'cpu'
device = paddle.set_device(device)
diff --git a/tools/train.py b/tools/train.py
index 05d295aa99718c25b94a123c23d08c2904fe8c6a..d182af2988cb29511be40a079d2b3e06605ebe28 100755
--- a/tools/train.py
+++ b/tools/train.py
@@ -102,10 +102,27 @@ def main(config, device, logger, vdl_writer):
if valid_dataloader is not None:
logger.info('valid dataloader has {} iters'.format(
len(valid_dataloader)))
+
+ use_amp = config["Global"].get("use_amp", False)
+ if use_amp:
+ AMP_RELATED_FLAGS_SETTING = {
+ 'FLAGS_cudnn_batchnorm_spatial_persistent': 1,
+ 'FLAGS_max_inplace_grad_add': 8,
+ }
+ paddle.fluid.set_flags(AMP_RELATED_FLAGS_SETTING)
+ scale_loss = config["Global"].get("scale_loss", 1.0)
+ use_dynamic_loss_scaling = config["Global"].get(
+ "use_dynamic_loss_scaling", False)
+ scaler = paddle.amp.GradScaler(
+ init_loss_scaling=scale_loss,
+ use_dynamic_loss_scaling=use_dynamic_loss_scaling)
+ else:
+ scaler = None
+
# start train
program.train(config, train_dataloader, valid_dataloader, device, model,
loss_class, optimizer, lr_scheduler, post_process_class,
- eval_class, pre_best_model_dict, logger, vdl_writer)
+ eval_class, pre_best_model_dict, logger, vdl_writer, scaler)
def test_reader(config, device, logger):