Skip to content

P
PaddleOCR

weixin_41840029 / PaddleOCR
与 Fork 源项目一致

Fork自 PaddlePaddle / PaddleOCR

通知 1
Star 1
Fork 0
P
PaddleOCR
“3907c72a0852fa33e6b4258bc2c4310b9b639107”上不存在“ppocr/data/git@gitcode.net:weixin_41840029/PaddleOCR.git”
未验证 提交 abff58d4 编写于 作者: B Bin Lu 提交者： GitHub
浏览文件
操作

Merge branch 'PaddlePaddle:dygraph' into dygraph

上级 2400bacf 3e10cfd4
隐藏空白更改
内联 并排
Showing with 472 addition and 744 deletion
doc/doc_ch/environment.md
浏览文件 @ abff58d4
# 运行环境准备 # 运行环境准备
Windows和Mac用户推荐使用Anaconda搭建Python环境，Linux用户建议使用docker搭建PyThon环境。 Windows和Mac用户推荐使用Anaconda搭建Python环境，Linux用户建议使用docker搭建PyThon环境。
推荐环境：
- PaddlePaddle >= 2.0.0 (2.1.2)
- python3.7
- CUDA10.1 / CUDA10.2
- CUDNN 7.6
如果对于Python环境熟悉的用户可以直接跳到第2步安装PaddlePaddle。 如果对于Python环境熟悉的用户可以直接跳到第2步安装PaddlePaddle。
* [1. Python环境搭建](#1) * [1. Python环境搭建](#1)
...@@ -123,13 +130,13 @@ Windows和Mac用户推荐使用Anaconda搭建Python环境，Linux用户建议使 ...@@ -123,13 +130,13 @@ Windows和Mac用户推荐使用Anaconda搭建Python环境，Linux用户建议使
# !! Contents within this block are managed by 'conda init' !! # !! Contents within this block are managed by 'conda init' !!
__conda_setup="$('/Users/xxx/opt/anaconda3/bin/conda' 'shell.bash' 'hook' 2> /dev/null)" __conda_setup="$('/Users/xxx/opt/anaconda3/bin/conda' 'shell.bash' 'hook' 2> /dev/null)"
if [ $? -eq 0 ]; then if [ $? -eq 0 ]; then
eval "$__conda_setup" eval "$__conda_setup"
else else
if [ -f "/Users/xxx/opt/anaconda3/etc/profile.d/conda.sh" ]; then if [ -f "/Users/xxx/opt/anaconda3/etc/profile.d/conda.sh" ]; then
. "/Users/xxx/opt/anaconda3/etc/profile.d/conda.sh" . "/Users/xxx/opt/anaconda3/etc/profile.d/conda.sh"
else else
export PATH="/Users/xxx/opt/anaconda3/bin:$PATH" export PATH="/Users/xxx/opt/anaconda3/bin:$PATH"
fi fi
fi fi
unset __conda_setup unset __conda_setup
# <<< conda initialize <<< # <<< conda initialize <<<
...@@ -294,11 +301,12 @@ cd /home/Projects ...@@ -294,11 +301,12 @@ cd /home/Projects
# 首次运行需创建一个docker容器，再次运行时不需要运行当前命令 # 首次运行需创建一个docker容器，再次运行时不需要运行当前命令
# 创建一个名字为ppocr的docker容器，并将当前目录映射到容器的/paddle目录下 # 创建一个名字为ppocr的docker容器，并将当前目录映射到容器的/paddle目录下
如果您希望在CPU环境下使用docker，使用docker而不是nvidia-docker创建docker #如果您希望在CPU环境下使用docker，使用docker而不是nvidia-docker创建docker
sudo docker run --name ppocr -v $PWD:/paddle --network=host -it paddlepaddle/paddle:latest-dev-cuda10.1-cudnn7-gcc82 /bin/bash sudo docker run --name ppocr -v $PWD:/paddle --network=host -it registry.baidubce.com/paddlepaddle/paddle:2.1.3-gpu-cuda10.2-cudnn7 /bin/bash
如果使用CUDA10，请运行以下命令创建容器，设置docker容器共享内存shm-size为64G，建议设置32G以上 #如果使用CUDA10，请运行以下命令创建容器，设置docker容器共享内存shm-size为64G，建议设置32G以上
sudo nvidia-docker run --name ppocr -v $PWD:/paddle --shm-size=64G --network=host -it paddlepaddle/paddle:latest-dev-cuda10.1-cudnn7-gcc82 /bin/bash # 如果是CUDA11+CUDNN8，推荐使用镜像registry.baidubce.com/paddlepaddle/paddle:2.1.3-gpu-cuda11.2-cudnn8
sudo nvidia-docker run --name ppocr -v $PWD:/paddle --shm-size=64G --network=host -it registry.baidubce.com/paddlepaddle/paddle:2.1.3-gpu-cuda10.2-cudnn7 /bin/bash
# ctrl+P+Q可退出docker 容器，重新进入docker 容器使用如下命令 # ctrl+P+Q可退出docker 容器，重新进入docker 容器使用如下命令
sudo docker container exec -it ppocr /bin/bash sudo docker container exec -it ppocr /bin/bash
...@@ -321,8 +329,3 @@ python3 -m pip install paddlepaddle -i https://mirror.baidu.com/pypi/simple ...@@ -321,8 +329,3 @@ python3 -m pip install paddlepaddle -i https://mirror.baidu.com/pypi/simple
``` ```
更多的版本需求，请参照[飞桨官网安装文档](https://www.paddlepaddle.org.cn/install/quick)中的说明进行操作。 更多的版本需求，请参照[飞桨官网安装文档](https://www.paddlepaddle.org.cn/install/quick)中的说明进行操作。
doc/doc_en/environment_en.md
浏览文件 @ abff58d4
# Environment Preparation # Environment Preparation
Recommended working environment:
- PaddlePaddle >= 2.0.0 (2.1.2)
- python3.7
- CUDA10.1 / CUDA10.2
- CUDNN 7.6
* [1. Python Environment Setup](#1) * [1. Python Environment Setup](#1)
+ [1.1 Windows](#1.1) + [1.1 Windows](#1.1)
+ [1.2 Mac](#1.2) + [1.2 Mac](#1.2)
+ [1.3 Linux](#1.3) + [1.3 Linux](#1.3)
* [2. Install PaddlePaddle 2.0](#2) * [2. Install PaddlePaddle 2.0](#2)
<a name="1"></a> <a name="1"></a>
## 1. Python Environment Setup ## 1. Python Environment Setup
...@@ -38,7 +45,7 @@ ...@@ -38,7 +45,7 @@
- Check conda to add environment variables and ignore the warning that - Check conda to add environment variables and ignore the warning that
<img src="../install/windows/anaconda_install_env.png" alt="add conda to path" width="500" align="center"/> <img src="../install/windows/anaconda_install_env.png" alt="add conda to path" width="500" align="center"/>
#### 1.1.2 Opening the terminal and creating the conda environment #### 1.1.2 Opening the terminal and creating the conda environment
...@@ -69,7 +76,7 @@ ...@@ -69,7 +76,7 @@
# View the current location of python # View the current location of python
where python where python
``` ```
<img src="../install/windows/conda_list_env.png" alt="create environment" width="600" align="center"/> <img src="../install/windows/conda_list_env.png" alt="create environment" width="600" align="center"/>
The above anaconda environment and python environment are installed The above anaconda environment and python environment are installed
...@@ -133,13 +140,13 @@ The above anaconda environment and python environment are installed ...@@ -133,13 +140,13 @@ The above anaconda environment and python environment are installed
# !!! Contents within this block are managed by 'conda init' !!! # !!! Contents within this block are managed by 'conda init' !!!
__conda_setup="$('/Users/xxx/opt/anaconda3/bin/conda' 'shell.bash' 'hook' 2> /dev/null)" __conda_setup="$('/Users/xxx/opt/anaconda3/bin/conda' 'shell.bash' 'hook' 2> /dev/null)"
if [ $? -eq 0 ]; then if [ $? -eq 0 ]; then
eval "$__conda_setup" eval "$__conda_setup"
else else
if [ -f "/Users/xxx/opt/anaconda3/etc/profile.d/conda.sh" ]; then if [ -f "/Users/xxx/opt/anaconda3/etc/profile.d/conda.sh" ]; then
. "/Users/xxx/opt/anaconda3/etc/profile.d/conda.sh" . "/Users/xxx/opt/anaconda3/etc/profile.d/conda.sh"
else else
export PATH="/Users/xxx/opt/anaconda3/bin:$PATH" export PATH="/Users/xxx/opt/anaconda3/bin:$PATH"
fi fi
fi fi
unset __conda_setup unset __conda_setup
# <<< conda initialize <<< # <<< conda initialize <<<
...@@ -197,11 +204,10 @@ Linux users can choose to run either Anaconda or Docker. If you are familiar wit ...@@ -197,11 +204,10 @@ Linux users can choose to run either Anaconda or Docker. If you are familiar wit
- **Download Anaconda**. - **Download Anaconda**.
- Download at: https://mirrors.tuna.tsinghua.edu.cn/anaconda/archive/?C=M&O=D - Download at: https://mirrors.tuna.tsinghua.edu.cn/anaconda/archive/?C=M&O=D
<img src="../install/linux/anaconda_download.png" akt="anaconda download" width="800" align="center"/> <img src="../install/linux/anaconda_download.png" akt="anaconda download" width="800" align="center"/>
- Select the appropriate version for your operating system - Select the appropriate version for your operating system
- Type `uname -m` in the terminal to check the command set used by your system - Type `uname -m` in the terminal to check the command set used by your system
...@@ -216,12 +222,12 @@ Linux users can choose to run either Anaconda or Docker. If you are familiar wit ...@@ -216,12 +222,12 @@ Linux users can choose to run either Anaconda or Docker. If you are familiar wit
sudo yum install wget # CentOS sudo yum install wget # CentOS
``` ```
```bash ```bash
# Then use wget to download from Tsinghua source # Then use wget to download from Tsinghua source
# If you want to download Anaconda3-2021.05-Linux-x86_64.sh, the download command is as follows # If you want to download Anaconda3-2021.05-Linux-x86_64.sh, the download command is as follows
wget https://mirrors.tuna.tsinghua.edu.cn/anaconda/archive/Anaconda3-2021.05-Linux-x86_64.sh wget https://mirrors.tuna.tsinghua.edu.cn/anaconda/archive/Anaconda3-2021.05-Linux-x86_64.sh
# If you want to download another version, you need to change the file name after the last 1 / to the version you want to download # If you want to download another version, you need to change the file name after the last 1 / to the version you want to download
``` ```
- To install Anaconda. - To install Anaconda.
- Type `sh Anaconda3-2021.05-Linux-x86_64.sh` at the command line - Type `sh Anaconda3-2021.05-Linux-x86_64.sh` at the command line
...@@ -309,7 +315,18 @@ cd /home/Projects ...@@ -309,7 +315,18 @@ cd /home/Projects
# Create a docker container named ppocr and map the current directory to the /paddle directory of the container # Create a docker container named ppocr and map the current directory to the /paddle directory of the container
# If using CPU, use docker instead of nvidia-docker to create docker # If using CPU, use docker instead of nvidia-docker to create docker
sudo docker run --name ppocr -v $PWD:/paddle --network=host -it paddlepaddle/paddle:latest-dev-cuda10.1-cudnn7-gcc82 /bin/bash sudo docker run --name ppocr -v $PWD:/paddle --network=host -it registry.baidubce.com/paddlepaddle/paddle:2.1.3-gpu-cuda10.2-cudnn7 /bin/bash
# If using GPU, use nvidia-docker to create docker
# docker image registry.baidubce.com/paddlepaddle/paddle:2.1.3-gpu-cuda11.2-cudnn8 is recommended for CUDA11.2 + CUDNN8.
sudo nvidia-docker run --name ppocr -v $PWD:/paddle --shm-size=64G --network=host -it registry.baidubce.com/paddlepaddle/paddle:2.1.3-gpu-cuda10.2-cudnn7 /bin/bash
```
You can also visit [DockerHub](https://hub.docker.com/r/paddlepaddle/paddle/tags/) to get the image that fits your machine.
```
# ctrl+P+Q to exit docker, to re-enter docker using the following command:
sudo docker container exec -it ppocr /bin/bash
``` ```
<a name="2"></a> <a name="2"></a>
...@@ -329,4 +346,3 @@ python3 -m pip install paddlepaddle -i https://mirror.baidu.com/pypi/simple ...@@ -329,4 +346,3 @@ python3 -m pip install paddlepaddle -i https://mirror.baidu.com/pypi/simple
``` ```
For more software version requirements, please refer to the instructions in [Installation Document](https://www.paddlepaddle.org.cn/install/quick) for operation. For more software version requirements, please refer to the instructions in [Installation Document](https://www.paddlepaddle.org.cn/install/quick) for operation.
doc/joinus.PNG
查看替换文件 @ 2400bacf 浏览文件 @ abff58d4
doc/joinus.PNG

193.9 KB | W: | H:

doc/joinus.PNG

188.2 KB | W: | H:

doc/joinus.PNG
doc/joinus.PNG
doc/joinus.PNG
doc/joinus.PNG
  • 2-up
  • Swipe
  • Onion skin
ppocr/modeling/backbones/rec_mv1_enhance.py
浏览文件 @ abff58d4
# copyright (c) 2020 PaddlePaddle Authors. All Rights Reserve. # copyright (c) 2021 PaddlePaddle Authors. All Rights Reserve.
# #
# Licensed under the Apache License, Version 2.0 (the "License"); # Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License. # you may not use this file except in compliance with the License.
...@@ -16,26 +16,17 @@ from __future__ import absolute_import ...@@ -16,26 +16,17 @@ from __future__ import absolute_import
from __future__ import division from __future__ import division
from __future__ import print_function from __future__ import print_function
import numpy as np
import paddle
from paddle import ParamAttr
import paddle.nn as nn
import paddle.nn.functional as F
from paddle.nn import Conv2D, BatchNorm, Linear, Dropout
from paddle.nn import AdaptiveAvgPool2D, MaxPool2D, AvgPool2D
from paddle.nn.initializer import KaimingNormal
import math import math
import numpy as np import numpy as np
import paddle import paddle
from paddle import ParamAttr, reshape, transpose, concat, split from paddle import ParamAttr, reshape, transpose
import paddle.nn as nn import paddle.nn as nn
import paddle.nn.functional as F import paddle.nn.functional as F
from paddle.nn import Conv2D, BatchNorm, Linear, Dropout from paddle.nn import Conv2D, BatchNorm, Linear, Dropout
from paddle.nn import AdaptiveAvgPool2D, MaxPool2D, AvgPool2D from paddle.nn import AdaptiveAvgPool2D, MaxPool2D, AvgPool2D
from paddle.nn.initializer import KaimingNormal from paddle.nn.initializer import KaimingNormal
import math
from paddle.nn.functional import hardswish, hardsigmoid
from paddle.regularizer import L2Decay from paddle.regularizer import L2Decay
from paddle.nn.functional import hardswish, hardsigmoid
class ConvBNLayer(nn.Layer): class ConvBNLayer(nn.Layer):
......
tests/docs/test_cpp.md 0 → 100644
浏览文件 @ abff58d4
# C++预测功能测试
C++预测功能测试的主程序为`test_cpp.sh`，可以测试基于C++预测库的模型推理功能。
## 测试结论汇总
| 算法名称 | 模型名称 |device | batchsize | mkldnn | cpu多线程 | tensorrt | 离线量化 |
| ---- | ---- | ---- | ---- | ---- | ---- | ----| --- |
| DB |ch_ppocr_mobile_v2.0_det| CPU/GPU | 1/6 | 支持 | 支持 | fp32/fp16/int8 | 支持 |
| DB |ch_ppocr_server_v2.0_det| CPU/GPU | 1/6 | 支持 | 支持 | fp32/fp16/int8 | 支持 |
| CRNN |ch_ppocr_mobile_v2.0_rec| CPU/GPU | 1/6 | 支持 | 支持 | fp32/fp16/int8 | 支持 |
| CRNN |ch_ppocr_server_v2.0_rec| CPU/GPU | 1/6 | 支持 | 支持 | fp32/fp16/int8 | 支持 |
|PP-OCR|ch_ppocr_server_v2.0 | CPU/GPU | 1/6 | 支持 | 支持 | fp32/fp16/int8 | 支持 |
|PP-OCR|ch_ppocr_server_v2.0 | CPU/GPU | 1/6 | 支持 | 支持 | fp32/fp16/int8 | 支持 |
## 1. 功能测试
先运行`prepare.sh`准备数据和模型，然后运行`test_cpp.sh`进行测试，最终在```tests/output```目录下生成`cpp_infer_*.log`后缀的日志文件。
```shell
bash tests/prepare.sh ./tests/configs/ppocr_det_mobile_params.txt
# 用法1:
bash tests/test_cpp.sh ./tests/configs/ppocr_det_mobile_params.txt
# 用法2: 指定GPU卡预测，第三个传入参数为GPU卡号
bash tests/test_cpp.sh ./tests/configs/ppocr_det_mobile_params.txt '1'
```
## 2. 精度测试
使用compare_results.py脚本比较模型预测的结果是否符合预期，主要步骤包括：
- 提取日志中的预测坐标；
- 从本地文件中提取保存好的坐标结果；
- 比较上述两个结果是否符合精度预期，误差大于设置阈值时会报错。
### 使用方式
运行命令：
```shell
python3.7 tests/compare_results.py --gt_file=./tests/results/*.txt --log_file=./tests/output/infer_*.log --atol=1e-3 --rtol=1e-3
```
参数介绍：
- gt_file： 指向事先保存好的预测结果路径，支持*.txt 结尾，会自动索引*.txt格式的文件，文件默认保存在tests/result/ 文件夹下
- log_file: 指向运行tests/test.sh 脚本的infer模式保存的预测日志，预测日志中打印的有预测结果，比如：文本框，预测文本，类别等等，同样支持infer_*.log格式传入
- atol: 设置的绝对误差
- rtol: 设置的相对误差
### 运行结果
正常运行效果如下图：
<img src="compare_right.png" width="1000">
出现不一致结果时的运行输出：
<img src="compare_wrong.png" width="1000">
tests/docs/test_python.md 0 → 100644
浏览文件 @ abff58d4
# Python功能测试
Python功能测试的主程序为`test_python.sh`，可以测试基于Python的模型训练、评估、推理等基本功能，包括裁剪、量化、蒸馏。
## 测试结论汇总
- 训练相关：
| 算法名称 | 模型名称 | 单机单卡 | 单机多卡 | 多机多卡 | 模型压缩（单机多卡） |
| :---- | :---- | :---- | :---- | :---- | :---- |
| DB | ch_ppocr_mobile_v2.0_det| 正常训练 <br> 混合精度 | 正常训练 <br> 混合精度 | 正常训练 <br> 混合精度 | 正常训练：FPGM裁剪、PACT量化 |
| DB | ch_ppocr_server_v2.0_det| 正常训练 <br> 混合精度 | 正常训练 <br> 混合精度 | 正常训练 <br> 混合精度 | 正常训练：FPGM裁剪、PACT量化 |
| CRNN | ch_ppocr_mobile_v2.0_rec| 正常训练 <br> 混合精度 | 正常训练 <br> 混合精度 | 正常训练 <br> 混合精度 | 正常训练：FPGM裁剪、PACT量化 |
| CRNN | ch_ppocr_server_v2.0_rec| 正常训练 <br> 混合精度 | 正常训练 <br> 混合精度 | 正常训练 <br> 混合精度 | 正常训练：FPGM裁剪、PACT量化 |
|PP-OCR| ch_ppocr_mobile_v2.0| 正常训练 <br> 混合精度 | 正常训练 <br> 混合精度 | 正常训练 <br> 混合精度 | 正常训练：FPGM裁剪、PACT量化 |
|PP-OCR| ch_ppocr_server_v2.0| 正常训练 <br> 混合精度 | 正常训练 <br> 混合精度 | 正常训练 <br> 混合精度 | 正常训练：FPGM裁剪、PACT量化 |
- 预测相关：
| 算法名称 | 模型名称 |device | batchsize | mkldnn | cpu多线程 | tensorrt | 离线量化 |
| ---- | ---- | ---- | ---- | ---- | ---- | ----| --- |
| DB |ch_ppocr_mobile_v2.0_det| CPU/GPU | 1/6 | 支持 | 支持 | fp32/fp16/int8 | 支持 |
| DB |ch_ppocr_server_v2.0_det| CPU/GPU | 1/6 | 支持 | 支持 | fp32/fp16/int8 | 支持 |
| CRNN |ch_ppocr_mobile_v2.0_rec| CPU/GPU | 1/6 | 支持 | 支持 | fp32/fp16/int8 | 支持 |
| CRNN |ch_ppocr_server_v2.0_rec| CPU/GPU | 1/6 | 支持 | 支持 | fp32/fp16/int8 | 支持 |
|PP-OCR|ch_ppocr_server_v2.0 | CPU/GPU | 1/6 | 支持 | 支持 | fp32/fp16/int8 | 支持 |
|PP-OCR|ch_ppocr_server_v2.0 | CPU/GPU | 1/6 | 支持 | 支持 | fp32/fp16/int8 | 支持 |
## 1. 安装依赖
- 安装PaddlePaddle >= 2.0
- 安装PaddleOCR依赖
```
pip3 install -r ../requirements.txt
```
- 安装autolog（规范化日志输出工具）
```
git clone https://github.com/LDOUBLEV/AutoLog
cd AutoLog
pip3 install -r requirements.txt
python3 setup.py bdist_wheel
pip3 install ./dist/auto_log-1.0.0-py3-none-any.whl
cd ../
```
## 2. 功能测试
先运行`prepare.sh`准备数据和模型，然后运行`test_python.sh`进行测试，最终在```tests/output```目录下生成`infer_*.log`格式的日志文件。
test_python.sh包含四种运行模式，每种模式的运行数据不同，分别用于测试速度和精度，分别是：
- 模式1：lite_train_infer，使用少量数据训练，用于快速验证训练到预测的走通流程，不验证精度和速度；
```shell
bash tests/prepare.sh ./tests/configs/ppocr_det_mobile_params.txt 'lite_train_infer'
bash tests/test_python.sh ./tests/configs/ppocr_det_mobile_params.txt 'lite_train_infer'
```
- 模式2：whole_infer，使用少量数据训练，一定量数据预测，用于验证训练后的模型执行预测，预测速度是否合理；
```shell
bash tests/prepare.sh ./tests/configs/ppocr_det_mobile_params.txt 'whole_infer'
bash tests/test_python.sh ./tests/configs/ppocr_det_mobile_params.txt 'whole_infer'
```
- 模式3：infer 不训练，全量数据预测，走通开源模型评估、动转静，检查inference model预测时间和精度;
```shell
bash tests/prepare.sh ./tests/configs/ppocr_det_mobile_params.txt 'infer'
# 用法1:
bash tests/test_python.sh ./tests/configs/ppocr_det_mobile_params.txt 'infer'
# 用法2: 指定GPU卡预测，第三个传入参数为GPU卡号
bash tests/test_python.sh ./tests/configs/ppocr_det_mobile_params.txt 'infer' '1'
```
- 模式4：whole_train_infer , CE： 全量数据训练，全量数据预测，验证模型训练精度，预测精度，预测速度；
```shell
bash tests/prepare.sh ./tests/configs/ppocr_det_mobile_params.txt 'whole_train_infer'
bash tests/test.sh ./tests/configs/ppocr_det_mobile_params.txt 'whole_train_infer'
```
## 3. 精度测试
使用compare_results.py脚本比较模型预测的结果是否符合预期，主要步骤包括：
- 提取日志中的预测坐标；
- 从本地文件中提取保存好的坐标结果；
- 比较上述两个结果是否符合精度预期，误差大于设置阈值时会报错。
### 使用方式
运行命令：
```shell
python3.7 tests/compare_results.py --gt_file=./tests/results/*.txt --log_file=./tests/output/infer_*.log --atol=1e-3 --rtol=1e-3
```
参数介绍：
- gt_file： 指向事先保存好的预测结果路径，支持*.txt 结尾，会自动索引*.txt格式的文件，文件默认保存在tests/result/ 文件夹下
- log_file: 指向运行tests/test.sh 脚本的infer模式保存的预测日志，预测日志中打印的有预测结果，比如：文本框，预测文本，类别等等，同样支持infer_*.log格式传入
- atol: 设置的绝对误差
- rtol: 设置的相对误差
### 运行结果
正常运行效果如下图：
<img src="compare_right.png" width="1000">
出现不一致结果时的运行输出：
<img src="compare_wrong.png" width="1000">
tests/readme.md
浏览文件 @ abff58d4
# 从训练到推理部署工具链测试方法介绍 # 推理部署导航
test.sh和params.txt文件配合使用，完成OCR轻量检测和识别模型从训练到预测的流程测试。 飞桨除了基本的模型训练和预测，还提供了支持多端多平台的高性能推理部署工具。本文档提供了PaddleOCR中所有模型的推理部署导航，方便用户查阅每种模型的推理部署打通情况，并可以进行一键测试。
# 安装依赖 <div align="center">
- 安装PaddlePaddle >= 2.0 <img src="docs/guide.png" width="1000">
- 安装PaddleOCR依赖 </div>
```
pip3 install -r ../requirements.txt
```
- 安装autolog
```
git clone https://github.com/LDOUBLEV/AutoLog
cd AutoLog
pip3 install -r requirements.txt
python3 setup.py bdist_wheel
pip3 install ./dist/auto_log-1.0.0-py3-none-any.whl
cd ../
```
# 目录介绍 打通情况汇总如下，已填写的部分表示可以使用本工具进行一键测试，未填写的表示正在支持中。
```bash | 算法论文 | 模型名称 | 模型类型 | python训练预测 | 其他 |
tests/ | :--- | :--- | :---- | :-------- | :---- |
├── ocr_det_params.txt # 测试OCR检测模型的参数配置文件 | DB |ch_ppocr_mobile_v2.0_det | 检测 | 支持 | Paddle Inference: C++预测 <br> Paddle Serving: Python, C++ <br> Paddle-Lite: Python, C++ / ARM CPU |
├── ocr_rec_params.txt # 测试OCR识别模型的参数配置文件 | DB |ch_ppocr_server_v2.0_det | 检测 | 支持 | Paddle Inference: C++预测 <br> Paddle Serving: Python, C++ <br> Paddle-Lite: Python, C++ / ARM CPU |
├── ocr_ppocr_mobile_params.txt # 测试OCR检测+识别模型串联的参数配置文件 | DB |ch_PP-OCRv2_det | 检测 |
└── prepare.sh # 完成test.sh运行所需要的数据和模型下载 | CRNN |ch_ppocr_mobile_v2.0_rec | 识别 | 支持 | Paddle Inference: C++预测 <br> Paddle Serving: Python, C++ <br> Paddle-Lite: Python, C++ / ARM CPU |
└── test.sh # 测试主程序 | CRNN |ch_ppocr_server_v2.0_rec | 识别 | 支持 | Paddle Inference: C++预测 <br> Paddle Serving: Python, C++ <br> Paddle-Lite: Python, C++ / ARM CPU |
``` | CRNN |ch_PP-OCRv2_rec | 识别 |
| DB |det_mv3_db_v2.0 | 检测 |
| DB |det_r50_vd_db_v2.0 | 检测 |
| EAST |det_mv3_east_v2.0 | 检测 |
| EAST |det_r50_vd_east_v2.0 | 检测 |
| PSENet |det_mv3_pse_v2.0 | 检测 |
| PSENet |det_r50_vd_pse_v2.0 | 检测 |
| SAST |det_r50_vd_sast_totaltext_v2.0 | 检测 |
| Rosetta|rec_mv3_none_none_ctc_v2.0 | 识别 |
| Rosetta|rec_r34_vd_none_none_ctc_v2.0 | 识别 |
| CRNN |rec_mv3_none_bilstm_ctc_v2.0 | 识别 |
| CRNN |rec_r34_vd_none_bilstm_ctc_v2.0| 识别 |
| StarNet|rec_mv3_tps_bilstm_ctc_v2.0 | 识别 |
| StarNet|rec_r34_vd_tps_bilstm_ctc_v2.0 | 识别 |
| RARE |rec_mv3_tps_bilstm_att_v2.0 | 识别 |
| RARE |rec_r34_vd_tps_bilstm_att_v2.0 | 识别 |
| SRN |rec_r50fpn_vd_none_srn | 识别 |
| NRTR |rec_mtb_nrtr | 识别 |
| SAR |rec_r31_sar | 识别 |
| PGNet |rec_r34_vd_none_none_ctc_v2.0 | 端到端|
# 使用方法
test.sh包含四种运行模式，每种模式的运行数据不同，分别用于测试速度和精度，分别是：
- 模式1：lite_train_infer，使用少量数据训练，用于快速验证训练到预测的走通流程，不验证精度和速度； ## 一键测试工具使用
```shell ### 目录介绍
bash tests/prepare.sh ./tests/ocr_det_params.txt 'lite_train_infer'
bash tests/test.sh ./tests/ocr_det_params.txt 'lite_train_infer'
```
- 模式2：whole_infer，使用少量数据训练，一定量数据预测，用于验证训练后的模型执行预测，预测速度是否合理；
```shell
bash tests/prepare.sh ./tests/ocr_det_params.txt 'whole_infer'
bash tests/test.sh ./tests/ocr_det_params.txt 'whole_infer'
```
- 模式3：infer 不训练，全量数据预测，走通开源模型评估、动转静，检查inference model预测时间和精度;
```shell ```shell
bash tests/prepare.sh ./tests/ocr_det_params.txt 'infer' tests/
# 用法1: ├── configs/ # 配置文件目录
bash tests/test.sh ./tests/ocr_det_params.txt 'infer' ├── det_mv3_db.yml # 测试mobile版ppocr检测模型训练的yml文件
# 用法2: 指定GPU卡预测，第三个传入参数为GPU卡号 ├── det_r50_vd_db.yml # 测试server版ppocr检测模型训练的yml文件
bash tests/test.sh ./tests/ocr_det_params.txt 'infer' '1' ├── rec_icdar15_r34_train.yml # 测试server版ppocr识别模型训练的yml文件
``` ├── ppocr_sys_mobile_params.txt # 测试mobile版ppocr检测+识别模型串联的参数配置文件
├── ppocr_det_mobile_params.txt # 测试mobile版ppocr检测模型的参数配置文件
├── ppocr_rec_mobile_params.txt # 测试mobile版ppocr识别模型的参数配置文件
├── ppocr_sys_server_params.txt # 测试server版ppocr检测+识别模型串联的参数配置文件
├── ppocr_det_server_params.txt # 测试server版ppocr检测模型的参数配置文件
├── ppocr_rec_server_params.txt # 测试server版ppocr识别模型的参数配置文件
├── ...
├── results/ # 预先保存的预测结果，用于和实际预测结果进行精读比对
├── ppocr_det_mobile_results_fp32.txt # 预存的mobile版ppocr检测模型fp32精度的结果
├── ppocr_det_mobile_results_fp16.txt # 预存的mobile版ppocr检测模型fp16精度的结果
├── ppocr_det_mobile_results_fp32_cpp.txt # 预存的mobile版ppocr检测模型c++预测的fp32精度的结果
├── ppocr_det_mobile_results_fp16_cpp.txt # 预存的mobile版ppocr检测模型c++预测的fp16精度的结果
├── ...
├── prepare.sh # 完成test_*.sh运行所需要的数据和模型下载
├── test_python.sh # 测试python训练预测的主程序
├── test_cpp.sh # 测试c++预测的主程序
├── test_serving.sh # 测试serving部署预测的主程序
├── test_lite.sh # 测试lite部署预测的主程序
├── compare_results.py # 用于对比log中的预测结果与results中的预存结果精度误差是否在限定范围内
└── readme.md # 使用文档
```
- 模式4：whole_train_infer , CE： 全量数据训练，全量数据预测，验证模型训练精度，预测精度，预测速度； ### 测试流程
```shell 使用本工具，可以测试不同功能的支持情况，以及预测结果是否对齐，测试流程如下：
bash tests/prepare.sh ./tests/ocr_det_params.txt 'whole_train_infer' <div align="center">
bash tests/test.sh ./tests/ocr_det_params.txt 'whole_train_infer' <img src="docs/test.png" width="800">
``` </div>
- 模式5：cpp_infer , CE： 验证inference model的c++预测是否走通； 1. 运行prepare.sh准备测试所需数据和模型；
```shell 2. 运行要测试的功能对应的测试脚本`test_*.sh`，产出log，由log可以看到不同配置是否运行成功；
bash tests/prepare.sh ./tests/ocr_det_params.txt 'cpp_infer' 3. 用`compare_results.py`对比log中的预测结果和预存在results目录下的结果，判断预测精度是否符合预期（在误差范围内）。
bash tests/test.sh ./tests/ocr_det_params.txt 'cpp_infer'
```
# 日志输出 其中，有4个测试主程序，功能如下：
最终在```tests/output```目录下生成.log后缀的日志文件 - `test_python.sh`：测试基于Python的模型训练、评估、推理等基本功能，包括裁剪、量化、蒸馏。
- `test_cpp.sh`：测试基于C++的模型推理。
- `test_serving.sh`：测试基于Paddle Serving的服务化部署功能。
- `test_lite.sh`：测试基于Paddle-Lite的端侧预测部署功能。
各功能测试中涉及GPU/CPU、mkldnn、Tensorrt等多种参数配置，点击相应链接了解更多细节和使用教程：
[test_python使用](docs/test_python.md)
[test_cpp使用](docs/test_cpp.md)
[test_serving使用](docs/test_serving.md)
[test_lite使用](docs/test_lite.md)
tests/test.sh 已删除 100644 → 0
浏览文件 @ 2400bacf
#!/bin/bash
FILENAME=$1
# MODE be one of ['lite_train_infer' 'whole_infer' 'whole_train_infer', 'infer', 'cpp_infer', 'serving_infer', 'klquant_infer']
MODE=$2
if [ ${MODE} = "cpp_infer" ]; then
dataline=$(awk 'NR==67, NR==81{print}' $FILENAME)
elif [ ${MODE} = "serving_infer" ]; then
dataline=$(awk 'NR==52, NR==66{print}' $FILENAME)
elif [ ${MODE} = "klquant_infer" ]; then
dataline=$(awk 'NR==82, NR==98{print}' $FILENAME)
else
dataline=$(awk 'NR==1, NR==51{print}' $FILENAME)
fi
# parser params
IFS=$'\n'
lines=(${dataline})
function func_parser_key(){
strs=$1
IFS=":"
array=(${strs})
tmp=${array[0]}
echo ${tmp}
}
function func_parser_value(){
strs=$1
IFS=":"
array=(${strs})
tmp=${array[1]}
echo ${tmp}
}
function func_set_params(){
key=$1
value=$2
if [ ${key} = "null" ];then
echo " "
elif [[ ${value} = "null" ]] || [[ ${value} = " " ]] || [ ${#value} -le 0 ];then
echo " "
else
echo "${key}=${value}"
fi
}
function func_parser_params(){
strs=$1
IFS=":"
array=(${strs})
key=${array[0]}
tmp=${array[1]}
IFS="|"
res=""
for _params in ${tmp[*]}; do
IFS="="
array=(${_params})
mode=${array[0]}
value=${array[1]}
if [[ ${mode} = ${MODE} ]]; then
IFS="|"
#echo $(func_set_params "${mode}" "${value}")
echo $value
break
fi
IFS="|"
done
echo ${res}
}
function status_check(){
last_status=$1 # the exit code
run_command=$2
run_log=$3
if [ $last_status -eq 0 ]; then
echo -e "\033[33m Run successfully with command - ${run_command}! \033[0m" | tee -a ${run_log}
else
echo -e "\033[33m Run failed with command - ${run_command}! \033[0m" | tee -a ${run_log}
fi
}
IFS=$'\n'
# The training params
model_name=$(func_parser_value "${lines[1]}")
python=$(func_parser_value "${lines[2]}")
gpu_list=$(func_parser_value "${lines[3]}")
train_use_gpu_key=$(func_parser_key "${lines[4]}")
train_use_gpu_value=$(func_parser_value "${lines[4]}")
autocast_list=$(func_parser_value "${lines[5]}")
autocast_key=$(func_parser_key "${lines[5]}")
epoch_key=$(func_parser_key "${lines[6]}")
epoch_num=$(func_parser_params "${lines[6]}")
save_model_key=$(func_parser_key "${lines[7]}")
train_batch_key=$(func_parser_key "${lines[8]}")
train_batch_value=$(func_parser_params "${lines[8]}")
pretrain_model_key=$(func_parser_key "${lines[9]}")
pretrain_model_value=$(func_parser_value "${lines[9]}")
train_model_name=$(func_parser_value "${lines[10]}")
train_infer_img_dir=$(func_parser_value "${lines[11]}")
train_param_key1=$(func_parser_key "${lines[12]}")
train_param_value1=$(func_parser_value "${lines[12]}")
trainer_list=$(func_parser_value "${lines[14]}")
trainer_norm=$(func_parser_key "${lines[15]}")
norm_trainer=$(func_parser_value "${lines[15]}")
pact_key=$(func_parser_key "${lines[16]}")
pact_trainer=$(func_parser_value "${lines[16]}")
fpgm_key=$(func_parser_key "${lines[17]}")
fpgm_trainer=$(func_parser_value "${lines[17]}")
distill_key=$(func_parser_key "${lines[18]}")
distill_trainer=$(func_parser_value "${lines[18]}")
trainer_key1=$(func_parser_key "${lines[19]}")
trainer_value1=$(func_parser_value "${lines[19]}")
trainer_key2=$(func_parser_key "${lines[20]}")
trainer_value2=$(func_parser_value "${lines[20]}")
eval_py=$(func_parser_value "${lines[23]}")
eval_key1=$(func_parser_key "${lines[24]}")
eval_value1=$(func_parser_value "${lines[24]}")
save_infer_key=$(func_parser_key "${lines[27]}")
export_weight=$(func_parser_key "${lines[28]}")
norm_export=$(func_parser_value "${lines[29]}")
pact_export=$(func_parser_value "${lines[30]}")
fpgm_export=$(func_parser_value "${lines[31]}")
distill_export=$(func_parser_value "${lines[32]}")
export_key1=$(func_parser_key "${lines[33]}")
export_value1=$(func_parser_value "${lines[33]}")
export_key2=$(func_parser_key "${lines[34]}")
export_value2=$(func_parser_value "${lines[34]}")
# parser inference model
infer_model_dir_list=$(func_parser_value "${lines[36]}")
infer_export_list=$(func_parser_value "${lines[37]}")
infer_is_quant=$(func_parser_value "${lines[38]}")
# parser inference
inference_py=$(func_parser_value "${lines[39]}")
use_gpu_key=$(func_parser_key "${lines[40]}")
use_gpu_list=$(func_parser_value "${lines[40]}")
use_mkldnn_key=$(func_parser_key "${lines[41]}")
use_mkldnn_list=$(func_parser_value "${lines[41]}")
cpu_threads_key=$(func_parser_key "${lines[42]}")
cpu_threads_list=$(func_parser_value "${lines[42]}")
batch_size_key=$(func_parser_key "${lines[43]}")
batch_size_list=$(func_parser_value "${lines[43]}")
use_trt_key=$(func_parser_key "${lines[44]}")
use_trt_list=$(func_parser_value "${lines[44]}")
precision_key=$(func_parser_key "${lines[45]}")
precision_list=$(func_parser_value "${lines[45]}")
infer_model_key=$(func_parser_key "${lines[46]}")
image_dir_key=$(func_parser_key "${lines[47]}")
infer_img_dir=$(func_parser_value "${lines[47]}")
save_log_key=$(func_parser_key "${lines[48]}")
benchmark_key=$(func_parser_key "${lines[49]}")
benchmark_value=$(func_parser_value "${lines[49]}")
infer_key1=$(func_parser_key "${lines[50]}")
infer_value1=$(func_parser_value "${lines[50]}")
# parser serving
if [ ${MODE} = "klquant_infer" ]; then
# parser inference model
infer_model_dir_list=$(func_parser_value "${lines[1]}")
infer_export_list=$(func_parser_value "${lines[2]}")
infer_is_quant=$(func_parser_value "${lines[3]}")
# parser inference
inference_py=$(func_parser_value "${lines[4]}")
use_gpu_key=$(func_parser_key "${lines[5]}")
use_gpu_list=$(func_parser_value "${lines[5]}")
use_mkldnn_key=$(func_parser_key "${lines[6]}")
use_mkldnn_list=$(func_parser_value "${lines[6]}")
cpu_threads_key=$(func_parser_key "${lines[7]}")
cpu_threads_list=$(func_parser_value "${lines[7]}")
batch_size_key=$(func_parser_key "${lines[8]}")
batch_size_list=$(func_parser_value "${lines[8]}")
use_trt_key=$(func_parser_key "${lines[9]}")
use_trt_list=$(func_parser_value "${lines[9]}")
precision_key=$(func_parser_key "${lines[10]}")
precision_list=$(func_parser_value "${lines[10]}")
infer_model_key=$(func_parser_key "${lines[11]}")
image_dir_key=$(func_parser_key "${lines[12]}")
infer_img_dir=$(func_parser_value "${lines[12]}")
save_log_key=$(func_parser_key "${lines[13]}")
benchmark_key=$(func_parser_key "${lines[14]}")
benchmark_value=$(func_parser_value "${lines[14]}")
infer_key1=$(func_parser_key "${lines[15]}")
infer_value1=$(func_parser_value "${lines[15]}")
fi
# parser serving
if [ ${MODE} = "server_infer" ]; then
trans_model_py=$(func_parser_value "${lines[1]}")
infer_model_dir_key=$(func_parser_key "${lines[2]}")
infer_model_dir_value=$(func_parser_value "${lines[2]}")
model_filename_key=$(func_parser_key "${lines[3]}")
model_filename_value=$(func_parser_value "${lines[3]}")
params_filename_key=$(func_parser_key "${lines[4]}")
params_filename_value=$(func_parser_value "${lines[4]}")
serving_server_key=$(func_parser_key "${lines[5]}")
serving_server_value=$(func_parser_value "${lines[5]}")
serving_client_key=$(func_parser_key "${lines[6]}")
serving_client_value=$(func_parser_value "${lines[6]}")
serving_dir_value=$(func_parser_value "${lines[7]}")
web_service_py=$(func_parser_value "${lines[8]}")
web_use_gpu_key=$(func_parser_key "${lines[9]}")
web_use_gpu_list=$(func_parser_value "${lines[9]}")
web_use_mkldnn_key=$(func_parser_key "${lines[10]}")
web_use_mkldnn_list=$(func_parser_value "${lines[10]}")
web_cpu_threads_key=$(func_parser_key "${lines[11]}")
web_cpu_threads_list=$(func_parser_value "${lines[11]}")
web_use_trt_key=$(func_parser_key "${lines[12]}")
web_use_trt_list=$(func_parser_value "${lines[12]}")
web_precision_key=$(func_parser_key "${lines[13]}")
web_precision_list=$(func_parser_value "${lines[13]}")
pipeline_py=$(func_parser_value "${lines[14]}")
fi
if [ ${MODE} = "cpp_infer" ]; then
# parser cpp inference model
cpp_infer_model_dir_list=$(func_parser_value "${lines[1]}")
cpp_infer_is_quant=$(func_parser_value "${lines[2]}")
# parser cpp inference
inference_cmd=$(func_parser_value "${lines[3]}")
cpp_use_gpu_key=$(func_parser_key "${lines[4]}")
cpp_use_gpu_list=$(func_parser_value "${lines[4]}")
cpp_use_mkldnn_key=$(func_parser_key "${lines[5]}")
cpp_use_mkldnn_list=$(func_parser_value "${lines[5]}")
cpp_cpu_threads_key=$(func_parser_key "${lines[6]}")
cpp_cpu_threads_list=$(func_parser_value "${lines[6]}")
cpp_batch_size_key=$(func_parser_key "${lines[7]}")
cpp_batch_size_list=$(func_parser_value "${lines[7]}")
cpp_use_trt_key=$(func_parser_key "${lines[8]}")
cpp_use_trt_list=$(func_parser_value "${lines[8]}")
cpp_precision_key=$(func_parser_key "${lines[9]}")
cpp_precision_list=$(func_parser_value "${lines[9]}")
cpp_infer_model_key=$(func_parser_key "${lines[10]}")
cpp_image_dir_key=$(func_parser_key "${lines[11]}")
cpp_infer_img_dir=$(func_parser_value "${lines[12]}")
cpp_infer_key1=$(func_parser_key "${lines[13]}")
cpp_infer_value1=$(func_parser_value "${lines[13]}")
cpp_benchmark_key=$(func_parser_key "${lines[14]}")
cpp_benchmark_value=$(func_parser_value "${lines[14]}")
fi
LOG_PATH="./tests/output"
mkdir -p ${LOG_PATH}
status_log="${LOG_PATH}/results.log"
function func_inference(){
IFS='|'
_python=$1
_script=$2
_model_dir=$3
_log_path=$4
_img_dir=$5
_flag_quant=$6
# inference
for use_gpu in ${use_gpu_list[*]}; do
if [ ${use_gpu} = "False" ] || [ ${use_gpu} = "cpu" ]; then
for use_mkldnn in ${use_mkldnn_list[*]}; do
if [ ${use_mkldnn} = "False" ] && [ ${_flag_quant} = "True" ]; then
continue
fi
for threads in ${cpu_threads_list[*]}; do
for batch_size in ${batch_size_list[*]}; do
_save_log_path="${_log_path}/infer_cpu_usemkldnn_${use_mkldnn}_threads_${threads}_batchsize_${batch_size}.log"
set_infer_data=$(func_set_params "${image_dir_key}" "${_img_dir}")
set_benchmark=$(func_set_params "${benchmark_key}" "${benchmark_value}")
set_batchsize=$(func_set_params "${batch_size_key}" "${batch_size}")
set_cpu_threads=$(func_set_params "${cpu_threads_key}" "${threads}")
set_model_dir=$(func_set_params "${infer_model_key}" "${_model_dir}")
set_infer_params1=$(func_set_params "${infer_key1}" "${infer_value1}")
command="${_python} ${_script} ${use_gpu_key}=${use_gpu} ${use_mkldnn_key}=${use_mkldnn} ${set_cpu_threads} ${set_model_dir} ${set_batchsize} ${set_infer_data} ${set_benchmark} ${set_infer_params1} > ${_save_log_path} 2>&1 "
eval $command
last_status=${PIPESTATUS[0]}
eval "cat ${_save_log_path}"
status_check $last_status "${command}" "${status_log}"
done
done
done
elif [ ${use_gpu} = "True" ] || [ ${use_gpu} = "gpu" ]; then
for use_trt in ${use_trt_list[*]}; do
for precision in ${precision_list[*]}; do
if [[ ${_flag_quant} = "False" ]] && [[ ${precision} =~ "int8" ]]; then
continue
fi
if [[ ${precision} =~ "fp16" || ${precision} =~ "int8" ]] && [ ${use_trt} = "False" ]; then
continue
fi
if [[ ${use_trt} = "False" || ${precision} =~ "int8" ]] && [ ${_flag_quant} = "True" ]; then
continue
fi
for batch_size in ${batch_size_list[*]}; do
_save_log_path="${_log_path}/infer_gpu_usetrt_${use_trt}_precision_${precision}_batchsize_${batch_size}.log"
set_infer_data=$(func_set_params "${image_dir_key}" "${_img_dir}")
set_benchmark=$(func_set_params "${benchmark_key}" "${benchmark_value}")
set_batchsize=$(func_set_params "${batch_size_key}" "${batch_size}")
set_tensorrt=$(func_set_params "${use_trt_key}" "${use_trt}")
set_precision=$(func_set_params "${precision_key}" "${precision}")
set_model_dir=$(func_set_params "${infer_model_key}" "${_model_dir}")
set_infer_params1=$(func_set_params "${infer_key1}" "${infer_value1}")
command="${_python} ${_script} ${use_gpu_key}=${use_gpu} ${set_tensorrt} ${set_precision} ${set_model_dir} ${set_batchsize} ${set_infer_data} ${set_benchmark} ${set_infer_params1} > ${_save_log_path} 2>&1 "
eval $command
last_status=${PIPESTATUS[0]}
eval "cat ${_save_log_path}"
status_check $last_status "${command}" "${status_log}"
done
done
done
else
echo "Does not support hardware other than CPU and GPU Currently!"
fi
done
}
function func_serving(){
IFS='|'
_python=$1
_script=$2
_model_dir=$3
# pdserving
set_dirname=$(func_set_params "${infer_model_dir_key}" "${infer_model_dir_value}")
set_model_filename=$(func_set_params "${model_filename_key}" "${model_filename_value}")
set_params_filename=$(func_set_params "${params_filename_key}" "${params_filename_value}")
set_serving_server=$(func_set_params "${serving_server_key}" "${serving_server_value}")
set_serving_client=$(func_set_params "${serving_client_key}" "${serving_client_value}")
trans_model_cmd="${python} ${trans_model_py} ${set_dirname} ${set_model_filename} ${set_params_filename} ${set_serving_server} ${set_serving_client}"
eval $trans_model_cmd
cd ${serving_dir_value}
echo $PWD
unset https_proxy
unset http_proxy
for use_gpu in ${web_use_gpu_list[*]}; do
echo ${ues_gpu}
if [ ${use_gpu} = "null" ]; then
for use_mkldnn in ${web_use_mkldnn_list[*]}; do
if [ ${use_mkldnn} = "False" ]; then
continue
fi
for threads in ${web_cpu_threads_list[*]}; do
_save_log_path="${_log_path}/server_cpu_usemkldnn_${use_mkldnn}_threads_${threads}_batchsize_1.log"
set_cpu_threads=$(func_set_params "${web_cpu_threads_key}" "${threads}")
web_service_cmd="${python} ${web_service_py} ${web_use_gpu_key}=${use_gpu} ${web_use_mkldnn_key}=${use_mkldnn} ${set_cpu_threads} &>${_save_log_path} &"
eval $web_service_cmd
sleep 2s
pipeline_cmd="${python} ${pipeline_py}"
eval $pipeline_cmd
last_status=${PIPESTATUS[0]}
eval "cat ${_save_log_path}"
status_check $last_status "${pipeline_cmd}" "${status_log}"
PID=$!
kill $PID
sleep 2s
ps ux | grep -E 'web_service|pipeline' | awk '{print $2}' | xargs kill -s 9
done
done
elif [ ${use_gpu} = "0" ]; then
for use_trt in ${web_use_trt_list[*]}; do
for precision in ${web_precision_list[*]}; do
if [[ ${_flag_quant} = "False" ]] && [[ ${precision} =~ "int8" ]]; then
continue
fi
if [[ ${precision} =~ "fp16" || ${precision} =~ "int8" ]] && [ ${use_trt} = "False" ]; then
continue
fi
if [[ ${use_trt} = "False" || ${precision} =~ "int8" ]] && [[ ${_flag_quant} = "True" ]]; then
continue
fi
_save_log_path="${_log_path}/infer_gpu_usetrt_${use_trt}_precision_${precision}_batchsize_1.log"
set_tensorrt=$(func_set_params "${web_use_trt_key}" "${use_trt}")
set_precision=$(func_set_params "${web_precision_key}" "${precision}")
web_service_cmd="${python} ${web_service_py} ${web_use_gpu_key}=${use_gpu} ${set_tensorrt} ${set_precision} &>${_save_log_path} & "
eval $web_service_cmd
sleep 2s
pipeline_cmd="${python} ${pipeline_py}"
eval $pipeline_cmd
last_status=${PIPESTATUS[0]}
eval "cat ${_save_log_path}"
status_check $last_status "${pipeline_cmd}" "${status_log}"
PID=$!
kill $PID
sleep 2s
ps ux | grep -E 'web_service|pipeline' | awk '{print $2}' | xargs kill -s 9
done
done
else
echo "Does not support hardware other than CPU and GPU Currently!"
fi
done
}
function func_cpp_inference(){
IFS='|'
_script=$1
_model_dir=$2
_log_path=$3
_img_dir=$4
_flag_quant=$5
# inference
for use_gpu in ${cpp_use_gpu_list[*]}; do
if [ ${use_gpu} = "False" ] || [ ${use_gpu} = "cpu" ]; then
for use_mkldnn in ${cpp_use_mkldnn_list[*]}; do
if [ ${use_mkldnn} = "False" ] && [ ${_flag_quant} = "True" ]; then
continue
fi
for threads in ${cpp_cpu_threads_list[*]}; do
for batch_size in ${cpp_batch_size_list[*]}; do
_save_log_path="${_log_path}/cpp_infer_cpu_usemkldnn_${use_mkldnn}_threads_${threads}_batchsize_${batch_size}.log"
set_infer_data=$(func_set_params "${cpp_image_dir_key}" "${_img_dir}")
set_benchmark=$(func_set_params "${cpp_benchmark_key}" "${cpp_benchmark_value}")
set_batchsize=$(func_set_params "${cpp_batch_size_key}" "${batch_size}")
set_cpu_threads=$(func_set_params "${cpp_cpu_threads_key}" "${threads}")
set_model_dir=$(func_set_params "${cpp_infer_model_key}" "${_model_dir}")
set_infer_params1=$(func_set_params "${cpp_infer_key1}" "${cpp_infer_value1}")
command="${_script} ${cpp_use_gpu_key}=${use_gpu} ${cpp_use_mkldnn_key}=${use_mkldnn} ${set_cpu_threads} ${set_model_dir} ${set_batchsize} ${set_infer_data} ${set_benchmark} ${set_infer_params1} > ${_save_log_path} 2>&1 "
eval $command
last_status=${PIPESTATUS[0]}
eval "cat ${_save_log_path}"
status_check $last_status "${command}" "${status_log}"
done
done
done
elif [ ${use_gpu} = "True" ] || [ ${use_gpu} = "gpu" ]; then
for use_trt in ${cpp_use_trt_list[*]}; do
for precision in ${cpp_precision_list[*]}; do
if [[ ${_flag_quant} = "False" ]] && [[ ${precision} =~ "int8" ]]; then
continue
fi
if [[ ${precision} =~ "fp16" || ${precision} =~ "int8" ]] && [ ${use_trt} = "False" ]; then
continue
fi
if [[ ${use_trt} = "False" || ${precision} =~ "int8" ]] && [ ${_flag_quant} = "True" ]; then
continue
fi
for batch_size in ${cpp_batch_size_list[*]}; do
_save_log_path="${_log_path}/cpp_infer_gpu_usetrt_${use_trt}_precision_${precision}_batchsize_${batch_size}.log"
set_infer_data=$(func_set_params "${cpp_image_dir_key}" "${_img_dir}")
set_benchmark=$(func_set_params "${cpp_benchmark_key}" "${cpp_benchmark_value}")
set_batchsize=$(func_set_params "${cpp_batch_size_key}" "${batch_size}")
set_tensorrt=$(func_set_params "${cpp_use_trt_key}" "${use_trt}")
set_precision=$(func_set_params "${cpp_precision_key}" "${precision}")
set_model_dir=$(func_set_params "${cpp_infer_model_key}" "${_model_dir}")
set_infer_params1=$(func_set_params "${cpp_infer_key1}" "${cpp_infer_value1}")
command="${_script} ${cpp_use_gpu_key}=${use_gpu} ${set_tensorrt} ${set_precision} ${set_model_dir} ${set_batchsize} ${set_infer_data} ${set_benchmark} ${set_infer_params1} > ${_save_log_path} 2>&1 "
eval $command
last_status=${PIPESTATUS[0]}
eval "cat ${_save_log_path}"
status_check $last_status "${command}" "${status_log}"
done
done
done
else
echo "Does not support hardware other than CPU and GPU Currently!"
fi
done
}
if [ ${MODE} = "infer" ] || [ ${MODE} = "klquant_infer" ]; then
GPUID=$3
if [ ${#GPUID} -le 0 ];then
env=" "
else
env="export CUDA_VISIBLE_DEVICES=${GPUID}"
fi
# set CUDA_VISIBLE_DEVICES
eval $env
export Count=0
IFS="|"
infer_run_exports=(${infer_export_list})
infer_quant_flag=(${infer_is_quant})
for infer_model in ${infer_model_dir_list[*]}; do
# run export
if [ ${infer_run_exports[Count]} != "null" ];then
save_infer_dir=$(dirname $infer_model)
set_export_weight=$(func_set_params "${export_weight}" "${infer_model}")
set_save_infer_key=$(func_set_params "${save_infer_key}" "${save_infer_dir}")
export_cmd="${python} ${infer_run_exports[Count]} ${set_export_weight} ${set_save_infer_key}"
echo ${infer_run_exports[Count]}
echo $export_cmd
eval $export_cmd
status_export=$?
status_check $status_export "${export_cmd}" "${status_log}"
else
save_infer_dir=${infer_model}
fi
#run inference
is_quant=${infer_quant_flag[Count]}
func_inference "${python}" "${inference_py}" "${save_infer_dir}" "${LOG_PATH}" "${infer_img_dir}" ${is_quant}
Count=$(($Count + 1))
done
elif [ ${MODE} = "cpp_infer" ]; then
GPUID=$3
if [ ${#GPUID} -le 0 ];then
env=" "
else
env="export CUDA_VISIBLE_DEVICES=${GPUID}"
fi
# set CUDA_VISIBLE_DEVICES
eval $env
export Count=0
IFS="|"
infer_quant_flag=(${cpp_infer_is_quant})
for infer_model in ${cpp_infer_model_dir_list[*]}; do
#run inference
is_quant=${infer_quant_flag[Count]}
func_cpp_inference "${inference_cmd}" "${infer_model}" "${LOG_PATH}" "${cpp_infer_img_dir}" ${is_quant}
Count=$(($Count + 1))
done
elif [ ${MODE} = "serving_infer" ]; then
GPUID=$3
if [ ${#GPUID} -le 0 ];then
env=" "
else
env="export CUDA_VISIBLE_DEVICES=${GPUID}"
fi
# set CUDA_VISIBLE_DEVICES
eval $env
export Count=0
IFS="|"
#run serving
func_serving "${web_service_cmd}"
else
IFS="|"
export Count=0
USE_GPU_KEY=(${train_use_gpu_value})
for gpu in ${gpu_list[*]}; do
use_gpu=${USE_GPU_KEY[Count]}
Count=$(($Count + 1))
if [ ${gpu} = "-1" ];then
env=""
elif [ ${#gpu} -le 1 ];then
env="export CUDA_VISIBLE_DEVICES=${gpu}"
eval ${env}
elif [ ${#gpu} -le 15 ];then
IFS=","
array=(${gpu})
env="export CUDA_VISIBLE_DEVICES=${array[0]}"
IFS="|"
else
IFS=";"
array=(${gpu})
ips=${array[0]}
gpu=${array[1]}
IFS="|"
env=" "
fi
for autocast in ${autocast_list[*]}; do
for trainer in ${trainer_list[*]}; do
flag_quant=False
if [ ${trainer} = ${pact_key} ]; then
run_train=${pact_trainer}
run_export=${pact_export}
flag_quant=True
elif [ ${trainer} = "${fpgm_key}" ]; then
run_train=${fpgm_trainer}
run_export=${fpgm_export}
elif [ ${trainer} = "${distill_key}" ]; then
run_train=${distill_trainer}
run_export=${distill_export}
elif [ ${trainer} = ${trainer_key1} ]; then
run_train=${trainer_value1}
run_export=${export_value1}
elif [[ ${trainer} = ${trainer_key2} ]]; then
run_train=${trainer_value2}
run_export=${export_value2}
else
run_train=${norm_trainer}
run_export=${norm_export}
fi
if [ ${run_train} = "null" ]; then
continue
fi
set_autocast=$(func_set_params "${autocast_key}" "${autocast}")
set_epoch=$(func_set_params "${epoch_key}" "${epoch_num}")
set_pretrain=$(func_set_params "${pretrain_model_key}" "${pretrain_model_value}")
set_batchsize=$(func_set_params "${train_batch_key}" "${train_batch_value}")
set_train_params1=$(func_set_params "${train_param_key1}" "${train_param_value1}")
set_use_gpu=$(func_set_params "${train_use_gpu_key}" "${use_gpu}")
save_log="${LOG_PATH}/${trainer}_gpus_${gpu}_autocast_${autocast}"
# load pretrain from norm training if current trainer is pact or fpgm trainer
if [ ${trainer} = ${pact_key} ] || [ ${trainer} = ${fpgm_key} ]; then
set_pretrain="${load_norm_train_model}"
fi
set_save_model=$(func_set_params "${save_model_key}" "${save_log}")
if [ ${#gpu} -le 2 ];then # train with cpu or single gpu
cmd="${python} ${run_train} ${set_use_gpu} ${set_save_model} ${set_epoch} ${set_pretrain} ${set_autocast} ${set_batchsize} ${set_train_params1} "
elif [ ${#gpu} -le 15 ];then # train with multi-gpu
cmd="${python} -m paddle.distributed.launch --gpus=${gpu} ${run_train} ${set_save_model} ${set_epoch} ${set_pretrain} ${set_autocast} ${set_batchsize} ${set_train_params1}"
else # train with multi-machine
cmd="${python} -m paddle.distributed.launch --ips=${ips} --gpus=${gpu} ${run_train} ${set_save_model} ${set_pretrain} ${set_epoch} ${set_autocast} ${set_batchsize} ${set_train_params1}"
fi
# run train
eval "unset CUDA_VISIBLE_DEVICES"
eval $cmd
status_check $? "${cmd}" "${status_log}"
set_eval_pretrain=$(func_set_params "${pretrain_model_key}" "${save_log}/${train_model_name}")
# save norm trained models to set pretrain for pact training and fpgm training
if [ ${trainer} = ${trainer_norm} ]; then
load_norm_train_model=${set_eval_pretrain}
fi
# run eval
if [ ${eval_py} != "null" ]; then
set_eval_params1=$(func_set_params "${eval_key1}" "${eval_value1}")
eval_cmd="${python} ${eval_py} ${set_eval_pretrain} ${set_use_gpu} ${set_eval_params1}"
eval $eval_cmd
status_check $? "${eval_cmd}" "${status_log}"
fi
# run export model
if [ ${run_export} != "null" ]; then
# run export model
save_infer_path="${save_log}"
set_export_weight=$(func_set_params "${export_weight}" "${save_log}/${train_model_name}")
set_save_infer_key=$(func_set_params "${save_infer_key}" "${save_infer_path}")
export_cmd="${python} ${run_export} ${set_export_weight} ${set_save_infer_key}"
eval $export_cmd
status_check $? "${export_cmd}" "${status_log}"
#run inference
eval $env
save_infer_path="${save_log}"
func_inference "${python}" "${inference_py}" "${save_infer_path}" "${LOG_PATH}" "${train_infer_img_dir}" "${flag_quant}"
eval "unset CUDA_VISIBLE_DEVICES"
fi
done # done with: for trainer in ${trainer_list[*]}; do
done # done with: for autocast in ${autocast_list[*]}; do
done # done with: for gpu in ${gpu_list[*]}; do
fi # end if [ ${MODE} = "infer" ]; then
tests/test_python.sh
浏览文件 @ abff58d4
...@@ -2,7 +2,14 @@ ...@@ -2,7 +2,14 @@
source tests/common_func.sh source tests/common_func.sh
FILENAME=$1 FILENAME=$1
dataline=$(awk 'NR==1, NR==51{print}' $FILENAME) # MODE be one of ['lite_train_infer' 'whole_infer' 'whole_train_infer', 'infer', 'klquant_infer']
MODE=$2
if [ ${MODE} = "klquant_infer" ]; then
dataline=$(awk 'NR==82, NR==98{print}' $FILENAME)
else
dataline=$(awk 'NR==1, NR==51{print}' $FILENAME)
fi
# parser params # parser params
IFS=$'\n' IFS=$'\n'
...@@ -84,6 +91,35 @@ benchmark_value=$(func_parser_value "${lines[49]}") ...@@ -84,6 +91,35 @@ benchmark_value=$(func_parser_value "${lines[49]}")
infer_key1=$(func_parser_key "${lines[50]}") infer_key1=$(func_parser_key "${lines[50]}")
infer_value1=$(func_parser_value "${lines[50]}") infer_value1=$(func_parser_value "${lines[50]}")
# parser klquant_infer
if [ ${MODE} = "klquant_infer" ]; then
# parser inference model
infer_model_dir_list=$(func_parser_value "${lines[1]}")
infer_export_list=$(func_parser_value "${lines[2]}")
infer_is_quant=$(func_parser_value "${lines[3]}")
# parser inference
inference_py=$(func_parser_value "${lines[4]}")
use_gpu_key=$(func_parser_key "${lines[5]}")
use_gpu_list=$(func_parser_value "${lines[5]}")
use_mkldnn_key=$(func_parser_key "${lines[6]}")
use_mkldnn_list=$(func_parser_value "${lines[6]}")
cpu_threads_key=$(func_parser_key "${lines[7]}")
cpu_threads_list=$(func_parser_value "${lines[7]}")
batch_size_key=$(func_parser_key "${lines[8]}")
batch_size_list=$(func_parser_value "${lines[8]}")
use_trt_key=$(func_parser_key "${lines[9]}")
use_trt_list=$(func_parser_value "${lines[9]}")
precision_key=$(func_parser_key "${lines[10]}")
precision_list=$(func_parser_value "${lines[10]}")
infer_model_key=$(func_parser_key "${lines[11]}")
image_dir_key=$(func_parser_key "${lines[12]}")
infer_img_dir=$(func_parser_value "${lines[12]}")
save_log_key=$(func_parser_key "${lines[13]}")
benchmark_key=$(func_parser_key "${lines[14]}")
benchmark_value=$(func_parser_value "${lines[14]}")
infer_key1=$(func_parser_key "${lines[15]}")
infer_value1=$(func_parser_value "${lines[15]}")
fi
LOG_PATH="./tests/output" LOG_PATH="./tests/output"
mkdir -p ${LOG_PATH} mkdir -p ${LOG_PATH}
...@@ -158,16 +194,148 @@ function func_inference(){ ...@@ -158,16 +194,148 @@ function func_inference(){
done done
} }
if [ ${MODE} = "infer" ] || [ ${MODE} = "klquant_infer" ]; then
# set cuda device GPUID=$3
GPUID=$2 if [ ${#GPUID} -le 0 ];then
if [ ${#GPUID} -le 0 ];then env=" "
env=" " else
env="export CUDA_VISIBLE_DEVICES=${GPUID}"
fi
# set CUDA_VISIBLE_DEVICES
eval $env
export Count=0
IFS="|"
infer_run_exports=(${infer_export_list})
infer_quant_flag=(${infer_is_quant})
for infer_model in ${infer_model_dir_list[*]}; do
# run export
if [ ${infer_run_exports[Count]} != "null" ];then
save_infer_dir=$(dirname $infer_model)
set_export_weight=$(func_set_params "${export_weight}" "${infer_model}")
set_save_infer_key=$(func_set_params "${save_infer_key}" "${save_infer_dir}")
export_cmd="${python} ${infer_run_exports[Count]} ${set_export_weight} ${set_save_infer_key}"
echo ${infer_run_exports[Count]}
echo $export_cmd
eval $export_cmd
status_export=$?
status_check $status_export "${export_cmd}" "${status_log}"
else
save_infer_dir=${infer_model}
fi
#run inference
is_quant=${infer_quant_flag[Count]}
func_inference "${python}" "${inference_py}" "${save_infer_dir}" "${LOG_PATH}" "${infer_img_dir}" ${is_quant}
Count=$(($Count + 1))
done
else else
env="export CUDA_VISIBLE_DEVICES=${GPUID}" IFS="|"
fi export Count=0
set CUDA_VISIBLE_DEVICES USE_GPU_KEY=(${train_use_gpu_value})
eval $env for gpu in ${gpu_list[*]}; do
use_gpu=${USE_GPU_KEY[Count]}
Count=$(($Count + 1))
if [ ${gpu} = "-1" ];then
env=""
elif [ ${#gpu} -le 1 ];then
env="export CUDA_VISIBLE_DEVICES=${gpu}"
eval ${env}
elif [ ${#gpu} -le 15 ];then
IFS=","
array=(${gpu})
env="export CUDA_VISIBLE_DEVICES=${array[0]}"
IFS="|"
else
IFS=";"
array=(${gpu})
ips=${array[0]}
gpu=${array[1]}
IFS="|"
env=" "
fi
for autocast in ${autocast_list[*]}; do
for trainer in ${trainer_list[*]}; do
flag_quant=False
if [ ${trainer} = ${pact_key} ]; then
run_train=${pact_trainer}
run_export=${pact_export}
flag_quant=True
elif [ ${trainer} = "${fpgm_key}" ]; then
run_train=${fpgm_trainer}
run_export=${fpgm_export}
elif [ ${trainer} = "${distill_key}" ]; then
run_train=${distill_trainer}
run_export=${distill_export}
elif [ ${trainer} = ${trainer_key1} ]; then
run_train=${trainer_value1}
run_export=${export_value1}
elif [[ ${trainer} = ${trainer_key2} ]]; then
run_train=${trainer_value2}
run_export=${export_value2}
else
run_train=${norm_trainer}
run_export=${norm_export}
fi
if [ ${run_train} = "null" ]; then
continue
fi
set_autocast=$(func_set_params "${autocast_key}" "${autocast}")
set_epoch=$(func_set_params "${epoch_key}" "${epoch_num}")
set_pretrain=$(func_set_params "${pretrain_model_key}" "${pretrain_model_value}")
set_batchsize=$(func_set_params "${train_batch_key}" "${train_batch_value}")
set_train_params1=$(func_set_params "${train_param_key1}" "${train_param_value1}")
set_use_gpu=$(func_set_params "${train_use_gpu_key}" "${use_gpu}")
save_log="${LOG_PATH}/${trainer}_gpus_${gpu}_autocast_${autocast}"
# load pretrain from norm training if current trainer is pact or fpgm trainer
if [ ${trainer} = ${pact_key} ] || [ ${trainer} = ${fpgm_key} ]; then
set_pretrain="${load_norm_train_model}"
fi
set_save_model=$(func_set_params "${save_model_key}" "${save_log}")
if [ ${#gpu} -le 2 ];then # train with cpu or single gpu
cmd="${python} ${run_train} ${set_use_gpu} ${set_save_model} ${set_epoch} ${set_pretrain} ${set_autocast} ${set_batchsize} ${set_train_params1} "
elif [ ${#gpu} -le 15 ];then # train with multi-gpu
cmd="${python} -m paddle.distributed.launch --gpus=${gpu} ${run_train} ${set_save_model} ${set_epoch} ${set_pretrain} ${set_autocast} ${set_batchsize} ${set_train_params1}"
else # train with multi-machine
cmd="${python} -m paddle.distributed.launch --ips=${ips} --gpus=${gpu} ${run_train} ${set_save_model} ${set_pretrain} ${set_epoch} ${set_autocast} ${set_batchsize} ${set_train_params1}"
fi
# run train
eval "unset CUDA_VISIBLE_DEVICES"
eval $cmd
status_check $? "${cmd}" "${status_log}"
set_eval_pretrain=$(func_set_params "${pretrain_model_key}" "${save_log}/${train_model_name}")
# save norm trained models to set pretrain for pact training and fpgm training
if [ ${trainer} = ${trainer_norm} ]; then
load_norm_train_model=${set_eval_pretrain}
fi
# run eval
if [ ${eval_py} != "null" ]; then
set_eval_params1=$(func_set_params "${eval_key1}" "${eval_value1}")
eval_cmd="${python} ${eval_py} ${set_eval_pretrain} ${set_use_gpu} ${set_eval_params1}"
eval $eval_cmd
status_check $? "${eval_cmd}" "${status_log}"
fi
# run export model
if [ ${run_export} != "null" ]; then
# run export model
save_infer_path="${save_log}"
set_export_weight=$(func_set_params "${export_weight}" "${save_log}/${train_model_name}")
set_save_infer_key=$(func_set_params "${save_infer_key}" "${save_infer_path}")
export_cmd="${python} ${run_export} ${set_export_weight} ${set_save_infer_key}"
eval $export_cmd
status_check $? "${export_cmd}" "${status_log}"
#run inference
eval $env
save_infer_path="${save_log}"
func_inference "${python}" "${inference_py}" "${save_infer_path}" "${LOG_PATH}" "${train_infer_img_dir}" "${flag_quant}"
eval "unset CUDA_VISIBLE_DEVICES"
fi
done # done with: for trainer in ${trainer_list[*]}; do
done # done with: for autocast in ${autocast_list[*]}; do
done # done with: for gpu in ${gpu_list[*]}; do
fi # end if [ ${MODE} = "infer" ]; then
echo "################### run test ###################"
Markdown is supported
0% 或 .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑！
想要评论请 注册 或