diff --git a/deploy/lite/readme.md b/deploy/lite/readme.md index 4775c19ca209694a4c1b20f66a05f6d03f7f7ced..d09a7e884118179e2c502bc95f0d07d3d8e6cffa 100644 --- a/deploy/lite/readme.md +++ b/deploy/lite/readme.md @@ -11,8 +11,6 @@ Paddle Lite是飞桨轻量化推理引擎，为手机、IOT端提供高效推理 - 电脑（编译Paddle Lite） - 安卓手机（armv7或armv8） -***注意： PaddleOCR 移动端部署当前不支持动态图模型，只支持静态图保存的模型。当前PaddleOCR静态图的分支是`develop`。*** - ### 1.1 准备交叉编译环境 交叉编译环境用于编译 Paddle Lite 和 PaddleOCR 的C++ demo。 支持多种开发环境，不同开发环境的编译流程请参考对应文档。 @@ -31,7 +29,7 @@ Paddle Lite是飞桨轻量化推理引擎，为手机、IOT端提供高效推理 |Android|[arm7](https://github.com/PaddlePaddle/Paddle-Lite/releases/download/v2.8/inference_lite_lib.android.armv7.gcc.c++_shared.with_extra.with_cv.tar.gz) / [arm8](https://github.com/PaddlePaddle/Paddle-Lite/releases/download/v2.8/inference_lite_lib.android.armv8.gcc.c++_shared.with_extra.with_cv.tar.gz)| |IOS|[arm7](https://github.com/PaddlePaddle/Paddle-Lite/releases/download/v2.8/inference_lite_lib.ios.armv7.with_cv.with_extra.with_log.tiny_publish.tar.gz) / [arm8](https://github.com/PaddlePaddle/Paddle-Lite/releases/download/v2.8/inference_lite_lib.ios.armv8.with_cv.with_extra.with_log.tiny_publish.tar.gz)| - 注：1. 上述预测库为PaddleLite 2.8分支编译得到，有关PaddleLite 2.8 详细信息可参考[链接](https://github.com/PaddlePaddle/Paddle-Lite/releases/tag/v2.8)。 + 注：1. 上述预测库为PaddleLite 2.8分支编译得到，有关PaddleLite 2.8 详细信息可参考[链接](https://github.com/PaddlePaddle/Paddle-Lite/releases/tag/v2.8) 。 - 2. [推荐]编译Paddle-Lite得到预测库，Paddle-Lite的编译方式如下： ``` @@ -87,10 +85,7 @@ Paddle-Lite 提供了多种策略来自动优化原始的模型，其中包括 |模型版本|模型简介|模型大小|检测模型|文本方向分类模型|识别模型|Paddle-Lite版本| |---|---|---|---|---|---|---| -|V2.0|超轻量中文OCR 移动端模型|8.1M|[下载地址](https://paddleocr.bj.bcebos.com/20-09-22/mobile/lite/ch_ppocr_mobile_v1.1_det_opt.nb)|[下载地址](https://paddleocr.bj.bcebos.com/20-09-22/mobile/lite/ch_ppocr_mobile_v1.1_cls_opt.nb)|[下载地址](https://paddleocr.bj.bcebos.com/20-09-22/mobile/lite/ch_ppocr_mobile_v1.1_rec_opt.nb)|v2.8| -|V2.0(slim)|超轻量中文OCR 移动端模型|3.5M|[下载地址](https://paddleocr.bj.bcebos.com/20-09-22/mobile/lite/ch_ppocr_mobile_v1.1_det_prune_opt.nb)|[下载地址](https://paddleocr.bj.bcebos.com/20-09-22/mobile/lite/ch_ppocr_mobile_v1.1_cls_quant_opt.nb)|[下载地址](https://paddleocr.bj.bcebos.com/20-09-22/mobile/lite/ch_ppocr_mobile_v1.1_rec_quant_opt.nb)|v2.8| - -注意：V2.0 3.0M 轻量模型是使用PaddleSlim优化后的，需要配合Paddle-Lite最新预测库使用。 +|V2.0|超轻量中文OCR 移动端模型|7.8M|[下载地址](https://paddleocr.bj.bcebos.com/dygraph_v2.0/lite/ch_ppocr_mobile_v2.0_det_infer_nb.nb)|[下载地址](https://paddleocr.bj.bcebos.com/dygraph_v2.0/lite/ch_ppocr_mobile_v2.0_cls_infer_nb.nb)|[下载地址](https://paddleocr.bj.bcebos.com/dygraph_v2.0/lite/ch_ppocr_mobile_v2.0_rec_infer_nb.nb)|v2.8| 如果直接使用上述表格中的模型进行部署，可略过下述步骤，直接阅读 [2.2节](#2.2与手机联调)。 @@ -128,12 +123,16 @@ cd build.opt/lite/api/ ``` # 【推荐】 下载PaddleOCR V2.0版本的中英文 inference模型 -wget https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_mobile_v2.0_det_slim_infer.tar && tar xf ch_ppocr_mobile_v1.1_det_prune_infer.tar -wget https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_mobile_v2.0_cls_slim_infer.tar && tar xf ch_ppocr_mobile_v1.1_rec_quant_infer.tar +wget https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_mobile_v2.0_det_slim_infer.tar && tar xf ch_ppocr_mobile_v2.0_det_slim_infer.tar +wget https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_mobile_v2.0_rec_slim_infer.tar && tar xf ch_ppocr_mobile_v2.0_rec_slim_infer.tar +wget https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_mobile_v2.0_cls_slim_infer.tar && tar xf ch_ppocr_mobile_v2.0_cls_slim_infer.tar # 转换V2.0检测模型 -./opt --model_file=./ch_ppocr_mobile_v1.1_det_prune_infer/model --param_file=./ch_ppocr_mobile_v1.1_det_prune_infer/params --optimize_out=./ch_ppocr_mobile_v1.1_det_prune_opt --valid_targets=arm --optimize_out_type=naive_buffer +./opt --model_file=./ch_ppocr_mobile_v2.0_det_slim_infer/inference.pdmodel --param_file=./ch_ppocr_mobile_v2.0_det_slim_infer/inference.pdiparams --optimize_out=./ch_ppocr_mobile_v2.0_det_slim_opt --valid_targets=arm --optimize_out_type=naive_buffer # 转换V2.0识别模型 -./opt --model_file=./ch_ppocr_mobile_v1.1_rec_quant_infer/model --param_file=./ch_ppocr_mobile_v1.1_rec_quant_infer/params --optimize_out=./ch_ppocr_mobile_v1.1_rec_quant_opt --valid_targets=arm --optimize_out_type=naive_buffer +./opt --model_file=./ch_ppocr_mobile_v2.0_rec_slim_infer/inference.pdmodel --param_file=./ch_ppocr_mobile_v2.0_rec_slim_infer/inference.pdiparams --optimize_out=./ch_ppocr_mobile_v2.0_rec_slim_opt --valid_targets=arm --optimize_out_type=naive_buffer +# 转换V2.0方向分类器模型 +./opt --model_file=./ch_ppocr_mobile_v2.0_cls_slim_infer/inference.pdmodel --param_file=./ch_ppocr_mobile_v2.0_cls_slim_infer/inference.pdiparams --optimize_out=./ch_ppocr_mobile_v2.0_cls_slim_opt --valid_targets=arm --optimize_out_type=naive_buffer + ``` 转换成功后，当前目录下会多出`.nb`结尾的文件，即是转换成功的模型文件。 @@ -186,21 +185,23 @@ wget https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_mobile_v2.0_cls_s ``` 准备测试图像，以`PaddleOCR/doc/imgs/11.jpg`为例，将测试的图像复制到`demo/cxx/ocr/debug/`文件夹下。 - 准备lite opt工具优化后的模型文件，比如使用`ch_ppocr_mobile_v1.1_det_prune_opt.nb，ch_ppocr_mobile_v1.1_rec_quant_opt.nb, ch_ppocr_mobile_cls_quant_opt.nb`，模型文件放置在`demo/cxx/ocr/debug/`文件夹下。 + 准备lite opt工具优化后的模型文件，比如使用`ch_ppocr_mobile_v2.0_det_slim_opt.nb，ch_ppocr_mobile_v2.0_rec_slim_opt.nb, ch_ppocr_mobile_v2.0_cls_slim_opt.nb`，模型文件放置在`demo/cxx/ocr/debug/`文件夹下。 执行完成后，ocr文件夹下将有如下文件格式： ``` demo/cxx/ocr/ |-- debug/ -| |--ch_ppocr_mobile_v1.1_det_prune_opt.nb 优化后的检测模型文件 -| |--ch_ppocr_mobile_v1.1_rec_quant_opt.nb 优化后的识别模型文件 -| |--ch_ppocr_mobile_cls_quant_opt.nb 优化后的文字方向分类器模型文件 +| |--ch_ppocr_mobile_v2.0_det_slim_opt.nb 优化后的检测模型文件 +| |--ch_ppocr_mobile_v2.0_rec_slim_opt.nb 优化后的识别模型文件 +| |--ch_ppocr_mobile_v2.0_cls_slim_opt.nb 优化后的文字方向分类器模型文件 | |--11.jpg 待测试图像 | |--ppocr_keys_v1.txt 中文字典文件 | |--libpaddle_light_api_shared.so C++预测库文件 -| |--config.txt DB-CRNN超参数配置 -|-- config.txt DB-CRNN超参数配置 +| |--config.txt 超参数配置 +|-- config.txt 超参数配置 +|-- cls_process.cc 方向分类器的预处理和后处理文件 +|-- cls_process.h |-- crnn_process.cc 识别模型CRNN的预处理和后处理文件 |-- crnn_process.h |-- db_post_process.cc 检测模型DB的后处理文件 @@ -219,6 +220,7 @@ ic15_dict.txt # 英文字典 dict/japan_dict.txt # 日语字典 dict/korean_dict.txt # 韩语字典 ppocr_keys_v1.txt # 中文字典 +... ``` 2. `config.txt` 包含了检测器、分类器的超参数，如下： @@ -246,7 +248,7 @@ use_direction_classify 0 # 是否使用方向分类器，0表示不使用，1 adb shell cd /data/local/tmp/debug export LD_LIBRARY_PATH=${PWD}:$LD_LIBRARY_PATH - ./ocr_db_crnn ch_ppocr_mobile_v1.1_det_prune_opt.nb ch_ppocr_mobile_v1.1_rec_quant_opt.nb ch_ppocr_mobile_cls_quant_opt.nb ./11.jpg ppocr_keys_v1.txt + ./ocr_db_crnn ch_ppocr_mobile_v2.0_det_slim_opt.nbb ch_ppocr_mobile_v2.0_rec_slim_opt.nb ch_ppocr_mobile_v2.0_cls_slim_opt.nb ./11.jpg ppocr_keys_v1.txt ``` 如果对代码做了修改，则需要重新编译并push到手机上。 diff --git a/deploy/lite/readme_en.md b/deploy/lite/readme_en.md index 58f6a574c7b21d35d8774e9d3311ecd26a7d5b99..9e683f11e2c638b9b3fc3dfa2ae8d564d7db0335 100644 --- a/deploy/lite/readme_en.md +++ b/deploy/lite/readme_en.md @@ -1,46 +1,55 @@ - # Tutorial of PaddleOCR Mobile deployment -This tutorial will introduce how to use [paddle-lite](https://github.com/PaddlePaddle/Paddle-Lite) to deploy paddleOCR ultra-lightweight Chinese and English detection models on mobile phones. +This tutorial will introduce how to use [Paddle Lite](https://github.com/PaddlePaddle/Paddle-Lite) to deploy paddleOCR ultra-lightweight Chinese and English detection models on mobile phones. -paddle-lite is a lightweight inference engine for PaddlePaddle. -It provides efficient inference capabilities for mobile phones and IoTs, -and extensively integrates cross-platform hardware to provide lightweight -deployment solutions for end-side deployment issues. +paddle-lite is a lightweight inference engine for PaddlePaddle. It provides efficient inference capabilities for mobile phones and IoTs, and extensively integrates cross-platform hardware to provide lightweight deployment solutions for end-side deployment issues. ## 1. Preparation +### 运行准备 + - Computer (for Compiling Paddle Lite) - Mobile phone (arm7 or arm8) -***Note: PaddleOCR lite deployment currently does not support dynamic graph models, only models saved with static graph. The static branch of PaddleOCR is `develop`.*** +### 1.1 Prepare the cross-compilation environment +The cross-compilation environment is used to compile C++ demos of Paddle Lite and PaddleOCR. +Supports multiple development environments. + +For the compilation process of different development environments, please refer to the corresponding documents. -## 2. Build PaddleLite library 1. [Docker](https://paddle-lite.readthedocs.io/zh/latest/source_compile/compile_env.html#docker) 2. [Linux](https://paddle-lite.readthedocs.io/zh/latest/source_compile/compile_env.html#linux) 3. [MAC OS](https://paddle-lite.readthedocs.io/zh/latest/source_compile/compile_env.html#mac-os) -## 3. Prepare prebuild library for android and ios +### 1.2 Prepare Paddle-Lite library -### 3.1 Download prebuild library -|Platform|Prebuild library Download Link| -|---|---| -|Android|[arm7](https://github.com/PaddlePaddle/Paddle-Lite/releases/download/v2.8/inference_lite_lib.android.armv7.gcc.c++_shared.with_extra.with_cv.tar.gz) / [arm8](https://github.com/PaddlePaddle/Paddle-Lite/releases/download/v2.8/inference_lite_lib.android.armv8.gcc.c++_shared.with_extra.with_cv.tar.gz)| -|IOS|[arm7](https://github.com/PaddlePaddle/Paddle-Lite/releases/download/v2.8/inference_lite_lib.ios.armv7.with_cv.with_extra.with_log.tiny_publish.tar.gz) / [arm8](https://github.com/PaddlePaddle/Paddle-Lite/releases/download/v2.8/inference_lite_lib.ios.armv8.with_cv.with_extra.with_log.tiny_publish.tar.gz)| +There are two ways to obtain the Paddle-Lite library： +- 1. Download directly, the download link of the Paddle-Lite library is as follows： + + | Platform | Paddle-Lite library download link | + |---|---| + |Android|[arm7](https://github.com/PaddlePaddle/Paddle-Lite/releases/download/v2.8/inference_lite_lib.android.armv7.gcc.c++_shared.with_extra.with_cv.tar.gz) / [arm8](https://github.com/PaddlePaddle/Paddle-Lite/releases/download/v2.8/inference_lite_lib.android.armv8.gcc.c++_shared.with_extra.with_cv.tar.gz)| + |IOS|[arm7](https://github.com/PaddlePaddle/Paddle-Lite/releases/download/v2.8/inference_lite_lib.ios.armv7.with_cv.with_extra.with_log.tiny_publish.tar.gz) / [arm8](https://github.com/PaddlePaddle/Paddle-Lite/releases/download/v2.8/inference_lite_lib.ios.armv8.with_cv.with_extra.with_log.tiny_publish.tar.gz)| -note: The above pre-build inference library is compiled from the PaddleLite `release/v2.8` branch. For more information about PaddleLite 2.8, please refer to [link](https://github.com/PaddlePaddle/Paddle-Lite/releases/tag/v2.8). + Note: 1. The above Paddle-Lite library is compiled from the Paddle-Lite 2.8 branch. For more information about Paddle-Lite 2.8, please refer to [link](https://github.com/PaddlePaddle/Paddle-Lite/releases/tag/v2.8). -### 3.2 Compile prebuild library (Recommended) +- 2. [Recommended] Compile Paddle-Lite to get the prediction library. The compilation method of Paddle-Lite is as follows： ``` git clone https://github.com/PaddlePaddle/Paddle-Lite.git cd Paddle-Lite -# checkout to Paddle-Lite release/v2.8 branch +# Switch to Paddle-Lite release/v2.8 stable branch git checkout release/v2.8 ./lite/tools/build_android.sh --arch=armv8 --with_cv=ON --with_extra=ON ``` -The structure of the prediction library is as follows: +Note: When compiling Paddle-Lite to obtain the Paddle-Lite library, you need to turn on the two options `--with_cv=ON --with_extra=ON`, `--arch` means the `arm` version, here is designated as armv8, + +More compilation commands refer to the introduction [link](https://paddle-lite.readthedocs.io/zh/latest/user_guides/Compile/Android.html#id2) 。 + +After directly downloading the Paddle-Lite library and decompressing it, you can get the `inference_lite_lib.android.armv8/` folder, and the Paddle-Lite library obtained by compiling Paddle-Lite is located +`Paddle-Lite/build.lite.android.armv8.gcc/inference_lite_lib.android.armv8/` folder. +The structure of the prediction library is as follows: ``` inference_lite_lib.android.armv8/ |-- cxx C++ prebuild library @@ -52,103 +61,117 @@ inference_lite_lib.android.armv8/ | | |-- paddle_use_kernels.h | | |-- paddle_use_ops.h | | `-- paddle_use_passes.h -| `-- lib +| `-- lib C++ library | |-- libpaddle_api_light_bundled.a C++ static library | `-- libpaddle_light_api_shared.so C++ dynamic library -|-- java Java predict library +|-- java Java library | |-- jar | | `-- PaddlePredictor.jar | |-- so | | `-- libpaddle_lite_jni.so | `-- src -|-- demo C++ and java demo -| |-- cxx -| `-- java +|-- demo C++ and Java demo +| |-- cxx C++ demo +| `-- java Java demo ``` +## 2 Run -## 4. Inference Model Optimization +### 2.1 Inference Model Optimization Paddle Lite provides a variety of strategies to automatically optimize the original training model, including quantization, sub-graph fusion, hybrid scheduling, Kernel optimization and so on. In order to make the optimization process more convenient and easy to use, Paddle Lite provide opt tools to automatically complete the optimization steps and output a lightweight, optimal executable model. -If you have prepared the model file ending in `.nb`, you can skip this step. +If you have prepared the model file ending in .nb, you can skip this step. + +The following table also provides a series of models that can be deployed on mobile phones to recognize Chinese. You can directly download the optimized model. -The following table also provides a series of models that can be deployed on mobile phones to recognize Chinese. -You can directly download the optimized model. +|Version|Introduction|Model size|Detection model|Text Direction model|Recognition model|Paddle-Lite branch| +|---|---|---|---|---|---|---| +|V2.0|extra-lightweight chinese OCR optimized model|7.8M|[download link](https://paddleocr.bj.bcebos.com/dygraph_v2.0/lite/ch_ppocr_mobile_v2.0_det_infer_nb.nb)|[download lin](https://paddleocr.bj.bcebos.com/dygraph_v2.0/lite/ch_ppocr_mobile_v2.0_cls_infer_nb.nb)|[download lin](https://paddleocr.bj.bcebos.com/dygraph_v2.0/lite/ch_ppocr_mobile_v2.0_rec_infer_nb.nb)|v2.8| -| Version | Introduction | Model size | Detection model | Text Direction model | Recognition model | Paddle Lite branch | -| --- | --- | --- | --- | --- | --- | --- | -| V1.1 | extra-lightweight chinese OCR optimized model | 8.1M | [Download](https://paddleocr.bj.bcebos.com/20-09-22/mobile/lite/ch_ppocr_mobile_v1.1_det_opt.nb) | [Download](https://paddleocr.bj.bcebos.com/20-09-22/mobile/lite/ch_ppocr_mobile_v1.1_cls_opt.nb) | [Download](https://paddleocr.bj.bcebos.com/20-09-22/mobile/lite/ch_ppocr_mobile_v1.1_rec_opt.nb) | develop | -| [slim] V1.1 | extra-lightweight chinese OCR optimized model | 3.5M | [Download](https://paddleocr.bj.bcebos.com/20-09-22/mobile/lite/ch_ppocr_mobile_v1.1_det_prune_opt.nb) | [Download](https://paddleocr.bj.bcebos.com/20-09-22/mobile/lite/ch_ppocr_mobile_v1.1_cls_quant_opt.nb) | [Download](https://paddleocr.bj.bcebos.com/20-09-22/mobile/lite/ch_ppocr_mobile_v1.1_rec_quant_opt.nb) | develop | -| V1.0 | lightweight Chinese OCR optimized model | 8.6M | [Download](https://paddleocr.bj.bcebos.com/20-09-22/mobile/lite/ch_ppocr_mobile_v1.0_det_opt.nb) | - | [Download](https://paddleocr.bj.bcebos.com/20-09-22/mobile/lite/ch_ppocr_mobile_v1.0_rec_opt.nb) | develop | +If you directly use the model in the above table for deployment, you can skip the following steps and directly read [Section 2.2](#2.2 Run optimized model on Phone). If the model to be deployed is not in the above table, you need to follow the steps below to obtain the optimized model. +The `opt` tool can be obtained by compiling Paddle Lite. ``` git clone https://github.com/PaddlePaddle/Paddle-Lite.git cd Paddle-Lite -git checkout release/v2.7 +git checkout release/v2.8 ./lite/tools/build.sh build_optimize_tool ``` -The `opt` tool can be obtained by compiling Paddle Lite. - -After the compilation is complete, the opt file is located under `build.opt/lite/api/`. - -The `opt` can optimize the inference model saved by paddle.io.save_inference_model to get the model that the paddlelite API can use. +After the compilation is complete, the opt file is located under build.opt/lite/api/, You can view the operating options and usage of opt in the following ways: -The usage of opt is as follows： ``` -# 【Recommend】V1.1 is better than V1.0. steps for convert V1.1 model to nb file are as follows -wget https://paddleocr.bj.bcebos.com/20-09-22/mobile-slim/det/ch_ppocr_mobile_v1.1_det_prune_infer.tar && tar xf ch_ppocr_mobile_v1.1_det_prune_infer.tar -wget https://paddleocr.bj.bcebos.com/20-09-22/mobile-slim/rec/ch_ppocr_mobile_v1.1_rec_quant_infer.tar && tar xf ch_ppocr_mobile_v1.1_rec_quant_infer.tar +cd build.opt/lite/api/ +./opt +``` -./opt --model_file=./ch_ppocr_mobile_v1.1_det_prune_infer/model --param_file=./ch_ppocr_mobile_v1.1_det_prune_infer/params --optimize_out=./ch_ppocr_mobile_v1.1_det_prune_opt --valid_targets=arm -./opt --model_file=./ch_ppocr_mobile_v1.1_rec_quant_infer/model --param_file=./ch_ppocr_mobile_v1.1_rec_quant_infer/params --optimize_out=./ch_ppocr_mobile_v1.1_rec_quant_opt --valid_targets=arm +|Options|Description| +|---|---| +|--model_dir|The path of the PaddlePaddle model to be optimized (non-combined form)| +|--model_file|The network structure file path of the PaddlePaddle model (combined form) to be optimized| +|--param_file|The weight file path of the PaddlePaddle model (combined form) to be optimized| +|--optimize_out_type|Output model type, currently supports two types: protobuf and naive_buffer, among which naive_buffer is a more lightweight serialization/deserialization implementation. If you need to perform model prediction on the mobile side, please set this option to naive_buffer. The default is protobuf| +|--optimize_out|The output path of the optimized model| +|--valid_targets|The executable backend of the model, the default is arm. Currently it supports x86, arm, opencl, npu, xpu, multiple backends can be specified at the same time (separated by spaces), and Model Optimize Tool will automatically select the best method. If you need to support Huawei NPU (DaVinci architecture NPU equipped with Kirin 810/990 Soc), it should be set to npu, arm| +|--record_tailoring_info|When using the function of cutting library files according to the model, set this option to true to record the kernel and OP information contained in the optimized model. The default is false| -# or use V1.0 model -wget https://paddleocr.bj.bcebos.com/ch_models/ch_det_mv3_db_infer.tar && tar xf ch_det_mv3_db_infer.tar -wget https://paddleocr.bj.bcebos.com/ch_models/ch_rec_mv3_crnn_infer.tar && tar xf ch_rec_mv3_crnn_infer.tar +`--model_dir` is suitable for the non-combined mode of the model to be optimized, and the inference model of PaddleOCR is the combined mode, that is, the model structure and model parameters are stored in a single file. -./opt --model_file=./ch_det_mv3_db/model --param_file=./ch_det_mv3_db/params --optimize_out_type=naive_buffer --optimize_out=./ch_det_mv3_db_opt --valid_targets=arm -./opt --model_file=./ch_rec_mv3_crnn/model --param_file=./ch_rec_mv3_crnn/params --optimize_out_type=naive_buffer --optimize_out=./ch_rec_mv3_crnn_opt --valid_targets=arm +The following takes the ultra-lightweight Chinese model of PaddleOCR as an example to introduce the use of the compiled opt file to complete the conversion of the inference model to the Paddle-Lite optimized model ``` +# [Recommendation] Download the Chinese and English inference model of PaddleOCR V2.0 +wget https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_mobile_v2.0_det_slim_infer.tar && tar xf ch_ppocr_mobile_v2.0_det_slim_infer.tar +wget https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_mobile_v2.0_rec_slim_infer.tar && tar xf ch_ppocr_mobile_v2.0_rec_slim_infer.tar +wget https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_mobile_v2.0_cls_slim_infer.tar && tar xf ch_ppocr_mobile_v2.0_cls_slim_infer.tar +# Convert V2.0 detection model +./opt --model_file=./ch_ppocr_mobile_v2.0_det_slim_infer/inference.pdmodel --param_file=./ch_ppocr_mobile_v2.0_det_slim_infer/inference.pdiparams --optimize_out=./ch_ppocr_mobile_v2.0_det_slim_opt --valid_targets=arm --optimize_out_type=naive_buffer +# 转换V2.0识别模型 +# Convert V2.0 recognition model +./opt --model_file=./ch_ppocr_mobile_v2.0_rec_slim_infer/inference.pdmodel --param_file=./ch_ppocr_mobile_v2.0_rec_slim_infer/inference.pdiparams --optimize_out=./ch_ppocr_mobile_v2.0_rec_slim_opt --valid_targets=arm --optimize_out_type=naive_buffer +# Convert V2.0 angle classifier model +./opt --model_file=./ch_ppocr_mobile_v2.0_cls_slim_infer/inference.pdmodel --param_file=./ch_ppocr_mobile_v2.0_cls_slim_infer/inference.pdiparams --optimize_out=./ch_ppocr_mobile_v2.0_cls_slim_opt --valid_targets=arm --optimize_out_type=naive_buffer -When the above code command is completed, there will be two more files `.nb` in the current directory, which is the converted model file. +``` -## 5. Run optimized model on Phone +After the conversion is successful, there will be more files ending with `.nb` in the current directory, which is the successfully converted model file. -1. Prepare an Android phone with arm8. If the compiled prediction library and opt file are armv7, you need an arm7 phone and modify ARM_ABI = arm7 in the Makefile. + +### 2.2 Run optimized model on Phone -2. Make sure the phone is connected to the computer, open the USB debugging option of the phone, and select the file transfer mode. +Some preparatory work is required first. + 1. Prepare an Android phone with arm8. If the compiled prediction library and opt file are armv7, you need an arm7 phone and modify ARM_ABI = arm7 in the Makefile. + 2. Make sure the phone is connected to the computer, open the USB debugging option of the phone, and select the file transfer mode. + 3. Install the adb tool on the computer. -3. Install the adb tool on the computer. - 3.1 Install ADB for MAC + 3.1. Install ADB for MAC: ``` brew cask install android-platform-tools ``` - 3.2 Install ADB for Linux + 3.2. Install ADB for Linux ``` sudo apt update sudo apt install -y wget adb ``` - 3.3 Install ADB for windows - [Download Link](https://developer.android.com/studio) + 3.3. Install ADB for windows + + To install on win, you need to go to Google's Android platform to download the adb package for installation：[link](https://developer.android.com/studio) Verify whether adb is installed successfully + ``` + adb devices ``` - $ adb devices - - List of devices attached - 744be294 device + If there is device output, it means the installation is successful。 + ``` + List of devices attached + 744be294 device ``` - If there is `device` output, it means the installation was successful. - -4. Prepare optimized models, prediction library files, test images and dictionary files used. - -``` + 4. Prepare optimized models, prediction library files, test images and dictionary files used. + ``` git clone https://github.com/PaddlePaddle/PaddleOCR.git cd PaddleOCR/deploy/lite/ # run prepare.sh @@ -162,39 +185,35 @@ When the above code command is completed, there will be two more files `.nb` in cd inference_lite_lib.android.armv8/demo/cxx/ocr/ cp ../../../cxx/lib/libpaddle_light_api_shared.so ./debug/ + ``` -``` - -Prepare the test image, taking `PaddleOCR/doc/imgs/11.jpg` as an example, copy the image file to the `demo/cxx/ocr/debug/` folder. -Prepare the model files optimized by the lite opt tool, `ch_det_mv3_db_opt.nb, ch_rec_mv3_crnn_opt.nb`, -and place them under the `demo/cxx/ocr/debug/` folder. - +Prepare the test image, taking PaddleOCR/doc/imgs/11.jpg as an example, copy the image file to the demo/cxx/ocr/debug/ folder. Prepare the model files optimized by the lite opt tool, ch_det_mv3_db_opt.nb, ch_rec_mv3_crnn_opt.nb, and place them under the demo/cxx/ocr/debug/ folder. The structure of the OCR demo is as follows after the above command is executed: + ``` demo/cxx/ocr/ |-- debug/ -| |--ch_ppocr_mobile_v1.1_det_prune_opt.nb Detection model -| |--ch_ppocr_mobile_v1.1_rec_quant_opt.nb Recognition model -| |--ch_ppocr_mobile_cls_quant_opt.nb Text direction classification model +| |--ch_ppocr_mobile_v2.0_det_slim_opt.nb Detection model +| |--ch_ppocr_mobile_v2.0_rec_slim_opt.nb Recognition model +| |--ch_ppocr_mobile_v2.0_cls_slim_opt.nb Text direction classification model | |--11.jpg Image for OCR | |--ppocr_keys_v1.txt Dictionary file | |--libpaddle_light_api_shared.so C++ .so file | |--config.txt Config file -|-- config.txt -|-- crnn_process.cc +|-- config.txt Config file +|-- cls_process.cc Pre-processing and post-processing files for the angle classifier +|-- cls_process.h +|-- crnn_process.cc Pre-processing and post-processing files for the CRNN model |-- crnn_process.h -|-- db_post_process.cc +|-- db_post_process.cc Pre-processing and post-processing files for the DB model |-- db_post_process.h |-- Makefile -|-- ocr_db_crnn.cc - +|-- ocr_db_crnn.cc C++ main code ``` -#### Note: -1. ppocr_keys_v1.txt is a Chinese dictionary file. -If the nb model is used for English recognition or other language recognition, dictionary file should be replaced with a dictionary of the corresponding language. -PaddleOCR provides a variety of dictionaries under ppocr/utils/, including: +#### 注意： +1. `ppocr_keys_v1.txt` is a Chinese dictionary file. If the nb model is used for English recognition or other language recognition, dictionary file should be replaced with a dictionary of the corresponding language. PaddleOCR provides a variety of dictionaries under ppocr/utils/, including: ``` dict/french_dict.txt # french dict/german_dict.txt # german @@ -204,7 +223,7 @@ dict/korean_dict.txt # korean ppocr_keys_v1.txt # chinese ``` -2. `config.txt` of the detector and classifier, as shown below: +2. `config.txt` of the detector and classifier, as shown below: ``` max_side_len 960 # Limit the maximum image height and width to 960 det_db_thresh 0.3 # Used to filter the binarized image of DB prediction, setting 0.-0.3 has no obvious effect on the result @@ -213,19 +232,26 @@ det_db_unclip_ratio 1.6 # Indicates the compactness of the text box, the small use_direction_classify 0 # Whether to use the direction classifier, 0 means not to use, 1 means to use ``` -5. Run Model on phone + 5. Run Model on phone -``` -cd inference_lite_lib.android.armv8/demo/cxx/ocr/ -make -j -mv ocr_db_crnn ./debug/ -adb push debug /data/local/tmp/ -adb shell -cd /data/local/tmp/debug -export LD_LIBRARY_PATH=/data/local/tmp/debug:$LD_LIBRARY_PATH -# run model - ./ocr_db_crnn ch_ppocr_mobile_v1.1_det_prune_opt.nb ch_ppocr_mobile_v1.1_rec_quant_opt.nb ch_ppocr_mobile_cls_quant_opt.nb ./11.jpg ppocr_keys_v1.txt -``` +After the above steps are completed, you can use adb to push the file to the phone to run, the steps are as follows: + + ``` + # Execute the compilation and get the executable file ocr_db_crnn + # The use of ocr_db_crnn is: + # ./ocr_db_crnn Detection model file Orientation classifier model file Recognition model file Test image path Dictionary file path + make -j + # Move the compiled executable file to the debug folder + mv ocr_db_crnn ./debug/ + # Push the debug folder to the phone + adb push debug /data/local/tmp/ + adb shell + cd /data/local/tmp/debug + export LD_LIBRARY_PATH=${PWD}:$LD_LIBRARY_PATH + ./ocr_db_crnn ch_ppocr_mobile_v2.0_det_slim_opt.nbb ch_ppocr_mobile_v2.0_rec_slim_opt.nb ch_ppocr_mobile_v2.0_cls_slim_opt.nb ./11.jpg ppocr_keys_v1.txt + ``` + +If you modify the code, you need to recompile and push to the phone. The outputs are as follows: @@ -233,14 +259,17 @@ The outputs are as follows: + ## FAQ Q1: What if I want to change the model, do I need to run it again according to the process? + A1: If you have performed the above steps, you only need to replace the .nb model file to complete the model replacement. Q2: How to test with another picture? -A2: Replace the .jpg test image under `./debug` with the image you want to test, and run `adb push` to push new image to the phone. + +A2: Replace the .jpg test image under ./debug with the image you want to test, and run adb push to push new image to the phone. Q3: How to package it into the mobile APP? -A3: This demo aims to provide the core algorithm part that can run OCR on mobile phones. Further, -PaddleOCR/deploy/android_demo is an example of encapsulating this demo into a mobile app for reference. + +A3: This demo aims to provide the core algorithm part that can run OCR on mobile phones. Further, PaddleOCR/deploy/android_demo is an example of encapsulating this demo into a mobile app for reference.