diff --git a/deploy/paddle2onnx/images/lite_demo_onnx.png b/deploy/paddle2onnx/images/lite_demo_onnx.png new file mode 100644 index 0000000000000000000000000000000000000000..b096f6eef5cc40dde2d4524e54a54c592d44b083 Binary files /dev/null and b/deploy/paddle2onnx/images/lite_demo_onnx.png differ diff --git a/deploy/paddle2onnx/images/lite_demo_paddle.png b/deploy/paddle2onnx/images/lite_demo_paddle.png new file mode 100644 index 0000000000000000000000000000000000000000..b096f6eef5cc40dde2d4524e54a54c592d44b083 Binary files /dev/null and b/deploy/paddle2onnx/images/lite_demo_paddle.png differ diff --git a/deploy/paddle2onnx/readme.md b/deploy/paddle2onnx/readme.md index 2148b3262d43f374a4310a5bfaaac854e92f9a32..e08f2adee5d315cecba703ecdf515c09cd1569d2 100644 --- a/deploy/paddle2onnx/readme.md +++ b/deploy/paddle2onnx/readme.md @@ -1,10 +1,19 @@ # paddle2onnx 模型转化与预测 -本章节介绍 PaddleOCR 模型如何转化为 ONNX 模型,并基于 ONNX 引擎预测。 +本章节介绍 PaddleOCR 模型如何转化为 ONNX 模型,并基于 ONNXRuntime 引擎预测。 ## 1. 环境准备 -需要准备 Paddle2ONNX 模型转化环境,和 ONNX 模型预测环境 +需要准备 PaddleOCR、Paddle2ONNX 模型转化环境,和 ONNXRuntime 预测环境 + +### PaddleOCR + +克隆PaddleOCR的仓库,使用release/2.4分支,并进行安装,由于PaddleOCR仓库比较大,git clone速度比较慢,所以本教程已下载 + +``` +git clone -b release/2.4 https://github.com/PaddlePaddle/PaddleOCR.git +cd PaddleOCR && python3.7 setup.py install +``` ### Paddle2ONNX @@ -16,7 +25,7 @@ Paddle2ONNX 支持将 PaddlePaddle 模型格式转化到 ONNX 模型格式,算 python3.7 -m pip install paddle2onnx ``` -- 安装 ONNX +- 安装 ONNXRuntime ``` # 建议安装 1.9.0 版本,可根据环境更换版本号 python3.7 -m pip install onnxruntime==1.9.0 @@ -30,11 +39,17 @@ python3.7 -m pip install onnxruntime==1.9.0 有两种方式获取Paddle静态图模型:在 [model_list](../../doc/doc_ch/models_list.md) 中下载PaddleOCR提供的预测模型; 参考[模型导出说明](../../doc/doc_ch/inference.md#训练模型转inference模型)把训练好的权重转为 inference_model。 -以 ppocr 检测模型为例: +以 ppocr 中文检测、识别、分类模型为例: ``` -wget -nc -P ./inference https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_mobile_v2.0_det_infer.tar -cd ./inference && tar xf ch_ppocr_mobile_v2.0_det_infer.tar && cd .. +wget -nc -P ./inference https://paddleocr.bj.bcebos.com/PP-OCRv2/chinese/ch_PP-OCRv2_det_infer.tar +cd ./inference && tar xf ch_PP-OCRv2_det_infer.tar && cd .. + +wget -nc -P ./inference https://paddleocr.bj.bcebos.com/PP-OCRv2/chinese/ch_PP-OCRv2_rec_infer.tar +cd ./inference && tar xf ch_PP-OCRv2_rec_infer.tar && cd .. + +wget -nc -P ./inference https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_mobile_v2.0_cls_infer.tar +cd ./inference && tar xf ch_ppocr_mobile_v2.0_cls_infer.tar && cd .. ``` - 模型转换 @@ -42,35 +57,160 @@ cd ./inference && tar xf ch_ppocr_mobile_v2.0_det_infer.tar && cd .. 使用 Paddle2ONNX 将Paddle静态图模型转换为ONNX模型格式: ``` -paddle2onnx --model_dir=./inference/ch_ppocr_mobile_v2.0_det_infer/ \ ---model_filename=inference.pdmodel \ ---params_filename=inference.pdiparams \ ---save_file=./inference/det_mobile_onnx/model.onnx \ ---opset_version=10 \ ---input_shape_dict="{'x': [-1, 3, -1, -1]}" \ ---enable_onnx_checker=True +paddle2onnx --model_dir ./inference/ch_PP-OCRv2_det_infer \ +--model_filename inference.pdmodel \ +--params_filename inference.pdiparams \ +--save_file ./inference/det_onnx/model.onnx \ +--opset_version 10 \ +--input_shape_dict="{'x':[-1,3,-1,-1]}" \ +--enable_onnx_checker True + +paddle2onnx --model_dir ./inference/ch_PP-OCRv2_rec_infer \ +--model_filename inference.pdmodel \ +--params_filename inference.pdiparams \ +--save_file ./inference/rec_onnx/model.onnx \ +--opset_version 10 \ +--input_shape_dict="{'x':[-1,3,-1,-1]}" \ +--enable_onnx_checker True + +paddle2onnx --model_dir ./inference/ch_ppocr_mobile_v2.0_cls_infer \ +--model_filename ch_ppocr_mobile_v2.0_cls_infer/inference.pdmodel \ +--params_filename ch_ppocr_mobile_v2.0_cls_infer/inference.pdiparams \ +--save_file ./inferencecls_onnx/model.onnx \ +--opset_version 10 \ +--input_shape_dict="{'x':[-1,3,-1,-1]}" \ +--enable_onnx_checker True ``` -执行完毕后,ONNX 模型会被保存在 `./inference/det_mobile_onnx/` 路径下 +执行完毕后,ONNX 模型会被分别保存在 `./inference/det_onnx/`,`./inference/rec_onnx/`,`./inference/cls_onnx/`路径下 * 注意:对于OCR模型,转化过程中必须采用动态shape的形式,即加入选项--input_shape_dict="{'x': [-1, 3, -1, -1]}",否则预测结果可能与直接使用Paddle预测有细微不同。 另外,以下几个模型暂不支持转换为 ONNX 模型: NRTR、SAR、RARE、SRN -## 3. onnx 预测 +## 3. 推理预测 + +以中文OCR模型为例,使用 ONNXRuntime 预测可执行如下命令: + +``` +python3.7 tools/infer/predict_system.py --use_gpu=False --use_onnx=True \ +--det_model_dir=./inference/det_onnx/model.onnx \ +--rec_model_dir=./inference/rec_onnx/model.onnx \ +--cls_model_dir=./inference/cls_onnx/model.onnx \ +--image_dir=./deploy/lite/imgs/lite_demo.png +``` + +以中文OCR模型为例,使用 Paddle Inference 预测可执行如下命令: + +``` +python3.7 tools/infer/predict_system.py --use_gpu=False \ +--cls_model_dir=./inference/ch_ppocr_mobile_v2.0_cls_infer \ +--rec_model_dir=./inference/ch_PP-OCRv2_rec_infer \ +--det_model_dir=./inference/ch_PP-OCRv2_det_infer \ +--image_dir=./deploy/lite/imgs/lite_demo.png +``` + + +执行命令后在终端会打印出预测的识别信息,并在 `./inference_results/` 下保存可视化结果。 + +ONNXRuntime 执行效果: + +
+ +
+ +Paddle Inference 执行效果: + +
+ +
-以检测模型为例,使用 ONNX 预测可执行如下命令: +使用 ONNXRuntime 预测,终端输出: ``` -python3.7 ../../tools/infer/predict_det.py --use_gpu=False --use_onnx=True \ ---det_model_dir=./inference/det_mobile_onnx/model.onnx \ ---image_dir=../../doc/imgs/1.jpg +[2022/02/22 17:48:27] root DEBUG: dt_boxes num : 38, elapse : 0.043187856674194336 +[2022/02/22 17:48:27] root DEBUG: rec_res num : 38, elapse : 0.592170000076294 +[2022/02/22 17:48:27] root DEBUG: 0 Predict time of ./deploy/lite/imgs/lite_demo.png: 0.642s +[2022/02/22 17:48:27] root DEBUG: The, 0.984 +[2022/02/22 17:48:27] root DEBUG: visualized, 0.882 +[2022/02/22 17:48:27] root DEBUG: etect18片, 0.720 +[2022/02/22 17:48:27] root DEBUG: image saved in./vis.jpg, 0.947 +[2022/02/22 17:48:27] root DEBUG: 纯臻营养护发素0.993604, 0.996 +[2022/02/22 17:48:27] root DEBUG: 产品信息/参数, 0.922 +[2022/02/22 17:48:27] root DEBUG: 0.992728, 0.914 +[2022/02/22 17:48:27] root DEBUG: (45元/每公斤,100公斤起订), 0.926 +[2022/02/22 17:48:27] root DEBUG: 0.97417, 0.977 +[2022/02/22 17:48:27] root DEBUG: 每瓶22元,1000瓶起订)0.993976, 0.962 +[2022/02/22 17:48:27] root DEBUG: 【品牌】:代加工方式/0EMODM, 0.945 +[2022/02/22 17:48:27] root DEBUG: 0.985133, 0.980 +[2022/02/22 17:48:27] root DEBUG: 【品名】:纯臻营养护发素, 0.921 +[2022/02/22 17:48:27] root DEBUG: 0.995007, 0.883 +[2022/02/22 17:48:27] root DEBUG: 【产品编号】:YM-X-30110.96899, 0.955 +[2022/02/22 17:48:27] root DEBUG: 【净含量】:220ml, 0.943 +[2022/02/22 17:48:27] root DEBUG: Q.996577, 0.932 +[2022/02/22 17:48:27] root DEBUG: 【适用人群】:适合所有肤质, 0.913 +[2022/02/22 17:48:27] root DEBUG: 0.995842, 0.969 +[2022/02/22 17:48:27] root DEBUG: 【主要成分】:鲸蜡硬脂醇、燕麦B-葡聚, 0.883 +[2022/02/22 17:48:27] root DEBUG: 0.961928, 0.964 +[2022/02/22 17:48:27] root DEBUG: 10, 0.812 +[2022/02/22 17:48:27] root DEBUG: 糖、椰油酰胺丙基甜菜碱、泛醒, 0.866 +[2022/02/22 17:48:27] root DEBUG: 0.925898, 0.943 +[2022/02/22 17:48:27] root DEBUG: (成品包材), 0.974 +[2022/02/22 17:48:27] root DEBUG: 0.972573, 0.961 +[2022/02/22 17:48:27] root DEBUG: 【主要功能】:可紧致头发磷层,从而达到, 0.936 +[2022/02/22 17:48:27] root DEBUG: 0.994448, 0.952 +[2022/02/22 17:48:27] root DEBUG: 13, 0.998 +[2022/02/22 17:48:27] root DEBUG: 即时持久改善头发光泽的效果,给干燥的头, 0.994 +[2022/02/22 17:48:27] root DEBUG: 0.990198, 0.975 +[2022/02/22 17:48:27] root DEBUG: 14, 0.977 +[2022/02/22 17:48:27] root DEBUG: 发足够的滋养, 0.991 +[2022/02/22 17:48:27] root DEBUG: 0.997668, 0.918 +[2022/02/22 17:48:27] root DEBUG: 花费了0.457335秒, 0.901 +[2022/02/22 17:48:27] root DEBUG: The visualized image saved in ./inference_results/lite_demo.png +[2022/02/22 17:48:27] root INFO: The predict total time is 0.7003889083862305 ``` -执行命令后在终端会打印出预测的检测框坐标,并在 `./inference_results/` 下保存可视化结果。 +使用 Paddle Inference 预测,终端输出: ``` -root INFO: 1.jpg [[[291, 295], [334, 292], [348, 844], [305, 847]], [[344, 296], [379, 294], [387, 669], [353, 671]]] -The predict time of ../../doc/imgs/1.jpg: 0.06162881851196289 -The visualized image saved in ./inference_results/det_res_1.jpg +[2022/02/22 17:47:25] root DEBUG: dt_boxes num : 38, elapse : 0.11791276931762695 +[2022/02/22 17:47:27] root DEBUG: rec_res num : 38, elapse : 2.6206860542297363 +[2022/02/22 17:47:27] root DEBUG: 0 Predict time of ./deploy/lite/imgs/lite_demo.png: 2.746s +[2022/02/22 17:47:27] root DEBUG: The, 0.984 +[2022/02/22 17:47:27] root DEBUG: visualized, 0.882 +[2022/02/22 17:47:27] root DEBUG: etect18片, 0.720 +[2022/02/22 17:47:27] root DEBUG: image saved in./vis.jpg, 0.947 +[2022/02/22 17:47:27] root DEBUG: 纯臻营养护发素0.993604, 0.996 +[2022/02/22 17:47:27] root DEBUG: 产品信息/参数, 0.922 +[2022/02/22 17:47:27] root DEBUG: 0.992728, 0.914 +[2022/02/22 17:47:27] root DEBUG: (45元/每公斤,100公斤起订), 0.926 +[2022/02/22 17:47:27] root DEBUG: 0.97417, 0.977 +[2022/02/22 17:47:27] root DEBUG: 每瓶22元,1000瓶起订)0.993976, 0.962 +[2022/02/22 17:47:27] root DEBUG: 【品牌】:代加工方式/0EMODM, 0.945 +[2022/02/22 17:47:27] root DEBUG: 0.985133, 0.980 +[2022/02/22 17:47:27] root DEBUG: 【品名】:纯臻营养护发素, 0.921 +[2022/02/22 17:47:27] root DEBUG: 0.995007, 0.883 +[2022/02/22 17:47:27] root DEBUG: 【产品编号】:YM-X-30110.96899, 0.955 +[2022/02/22 17:47:27] root DEBUG: 【净含量】:220ml, 0.943 +[2022/02/22 17:47:27] root DEBUG: Q.996577, 0.932 +[2022/02/22 17:47:27] root DEBUG: 【适用人群】:适合所有肤质, 0.913 +[2022/02/22 17:47:27] root DEBUG: 0.995842, 0.969 +[2022/02/22 17:47:27] root DEBUG: 【主要成分】:鲸蜡硬脂醇、燕麦B-葡聚, 0.883 +[2022/02/22 17:47:27] root DEBUG: 0.961928, 0.964 +[2022/02/22 17:47:27] root DEBUG: 10, 0.812 +[2022/02/22 17:47:27] root DEBUG: 糖、椰油酰胺丙基甜菜碱、泛醒, 0.866 +[2022/02/22 17:47:27] root DEBUG: 0.925898, 0.943 +[2022/02/22 17:47:27] root DEBUG: (成品包材), 0.974 +[2022/02/22 17:47:27] root DEBUG: 0.972573, 0.961 +[2022/02/22 17:47:27] root DEBUG: 【主要功能】:可紧致头发磷层,从而达到, 0.936 +[2022/02/22 17:47:27] root DEBUG: 0.994448, 0.952 +[2022/02/22 17:47:27] root DEBUG: 13, 0.998 +[2022/02/22 17:47:27] root DEBUG: 即时持久改善头发光泽的效果,给干燥的头, 0.994 +[2022/02/22 17:47:27] root DEBUG: 0.990198, 0.975 +[2022/02/22 17:47:27] root DEBUG: 14, 0.977 +[2022/02/22 17:47:27] root DEBUG: 发足够的滋养, 0.991 +[2022/02/22 17:47:27] root DEBUG: 0.997668, 0.918 +[2022/02/22 17:47:27] root DEBUG: 花费了0.457335秒, 0.901 +[2022/02/22 17:47:27] root DEBUG: The visualized image saved in ./inference_results/lite_demo.png +[2022/02/22 17:47:27] root INFO: The predict total time is 2.8338775634765625 ```