diff --git a/README.md b/README.md index 93044501739ed6d91ca5c270c9dce4576d9f26fc..8c530acdc94ccf4703aec3a33540465b5744e464 100644 --- a/README.md +++ b/README.md @@ -54,7 +54,7 @@ Mobile DEMO experience (based on EasyEdge and Paddle-Lite, supports iOS and Andr | ------------------------------------------------------------ | ---------------------------- | ----------------- | ------------------------------------------------------------ | ------------------------------------------------------------ | ------------------------------------------------------------ | | Chinese and English ultra-lightweight OCR model (8.1M) | ch_ppocr_mobile_v1.1_xx | Mobile & server | [inference model](https://paddleocr.bj.bcebos.com/20-09-22/mobile/det/ch_ppocr_mobile_v1.1_det_infer.tar) / [pre-trained model](https://paddleocr.bj.bcebos.com/20-09-22/mobile/det/ch_ppocr_mobile_v1.1_det_train.tar) | [inference model](https://paddleocr.bj.bcebos.com/20-09-22/cls/ch_ppocr_mobile_v1.1_cls_infer.tar) / [pre-trained model](https://paddleocr.bj.bcebos.com/20-09-22/cls/ch_ppocr_mobile_v1.1_cls_train.tar) | [inference model](https://paddleocr.bj.bcebos.com/20-09-22/mobile/rec/ch_ppocr_mobile_v1.1_rec_infer.tar) / [pre-trained model](https://paddleocr.bj.bcebos.com/20-09-22/mobile/rec/ch_ppocr_mobile_v1.1_rec_pre.tar) | | Chinese and English general OCR model (155.1M) | ch_ppocr_server_v1.1_xx | Server | [inference model](https://paddleocr.bj.bcebos.com/20-09-22/server/det/ch_ppocr_server_v1.1_det_infer.tar) / [pre-trained model](https://paddleocr.bj.bcebos.com/20-09-22/server/det/ch_ppocr_server_v1.1_det_train.tar) | [inference model](https://paddleocr.bj.bcebos.com/20-09-22/cls/ch_ppocr_mobile_v1.1_cls_infer.tar) / [pre-trained model](https://paddleocr.bj.bcebos.com/20-09-22/cls/ch_ppocr_mobile_v1.1_cls_train.tar) | [inference model](https://paddleocr.bj.bcebos.com/20-09-22/server/rec/ch_ppocr_server_v1.1_rec_infer.tar) / [pre-trained model](https://paddleocr.bj.bcebos.com/20-09-22/server/rec/ch_ppocr_server_v1.1_rec_pre.tar) | -| Chinese and English ultra-lightweight compressed OCR model (3.5M) | ch_ppocr_mobile_slim_v1.1_xx | Mobile | [inference model](https://paddleocr.bj.bcebos.com/20-09-22/mobile-slim/det/ch_ppocr_mobile_v1.1_det_prune_infer.tar) / [slim model](https://paddleocr.bj.bcebos.com/20-09-22/mobile-slim/det/ch_ppocr_mobile_v1.1_det_prune_opt.nb) | [inference model](https://paddleocr.bj.bcebos.com/20-09-22/cls/ch_ppocr_mobile_v1.1_cls_quant_infer.tar) / [slim model](https://paddleocr.bj.bcebos.com/20-09-22/cls/ch_ppocr_mobile_cls_quant_opt.nb) | [inference model](https://paddleocr.bj.bcebos.com/20-09-22/mobile-slim/rec/ch_ppocr_mobile_v1.1_rec_quant_infer.tar) / [slim model](https://paddleocr.bj.bcebos.com/20-09-22/mobile-slim/rec/ch_ppocr_mobile_v1.1_rec_quant_opt.nb) | +| Chinese and English ultra-lightweight compressed OCR model (3.5M) | ch_ppocr_mobile_slim_v1.1_xx | Mobile | [inference model](https://paddleocr.bj.bcebos.com/20-09-22/mobile-slim/det/ch_ppocr_mobile_v1.1_det_prune_infer.tar) / [slim model](https://paddleocr.bj.bcebos.com/20-09-22/mobile-slim/det/ch_ppocr_mobile_v1.1_det_prune_opt.nb) | [inference model](https://paddleocr.bj.bcebos.com/20-09-22/cls/ch_ppocr_mobile_v1.1_cls_quant_infer.tar) / [slim model](https://paddleocr.bj.bcebos.com/20-09-22/cls/ch_ppocr_mobile_v1.1_cls_quant_opt.nb) | [inference model](https://paddleocr.bj.bcebos.com/20-09-22/mobile-slim/rec/ch_ppocr_mobile_v1.1_rec_quant_infer.tar) / [slim model](https://paddleocr.bj.bcebos.com/20-09-22/mobile-slim/rec/ch_ppocr_mobile_v1.1_rec_quant_opt.nb) | For more model downloads (including multiple languages), please refer to [PP-OCR v1.1 series model downloads](./doc/doc_en/models_list_en.md) diff --git a/README_ch.md b/README_ch.md index 7d5c61acb5138e74e5cb5e9276a652017017f56b..806c0f57a6564a9d212e494b2058bc25b319170b 100644 --- a/README_ch.md +++ b/README_ch.md @@ -54,7 +54,7 @@ PaddleOCR旨在打造一套丰富、领先、且实用的OCR工具库，助力 | ------------ | --------------- | ----------------|---- | ---------- | -------- | | 中英文超轻量OCR模型（8.1M） | ch_ppocr_mobile_v1.1_xx |移动端&服务器端|[推理模型](https://paddleocr.bj.bcebos.com/20-09-22/mobile/det/ch_ppocr_mobile_v1.1_det_infer.tar) / [预训练模型](https://paddleocr.bj.bcebos.com/20-09-22/mobile/det/ch_ppocr_mobile_v1.1_det_train.tar)|[推理模型](https://paddleocr.bj.bcebos.com/20-09-22/cls/ch_ppocr_mobile_v1.1_cls_infer.tar) / [预训练模型](https://paddleocr.bj.bcebos.com/20-09-22/cls/ch_ppocr_mobile_v1.1_cls_train.tar) |[推理模型](https://paddleocr.bj.bcebos.com/20-09-22/mobile/rec/ch_ppocr_mobile_v1.1_rec_infer.tar) / [预训练模型](https://paddleocr.bj.bcebos.com/20-09-22/mobile/rec/ch_ppocr_mobile_v1.1_rec_pre.tar) | | 中英文通用OCR模型（155.1M） |ch_ppocr_server_v1.1_xx|服务器端 |[推理模型](https://paddleocr.bj.bcebos.com/20-09-22/server/det/ch_ppocr_server_v1.1_det_infer.tar) / [预训练模型](https://paddleocr.bj.bcebos.com/20-09-22/server/det/ch_ppocr_server_v1.1_det_train.tar) |[推理模型](https://paddleocr.bj.bcebos.com/20-09-22/cls/ch_ppocr_mobile_v1.1_cls_infer.tar) / [预训练模型](https://paddleocr.bj.bcebos.com/20-09-22/cls/ch_ppocr_mobile_v1.1_cls_train.tar) |[推理模型](https://paddleocr.bj.bcebos.com/20-09-22/server/rec/ch_ppocr_server_v1.1_rec_infer.tar) / [预训练模型](https://paddleocr.bj.bcebos.com/20-09-22/server/rec/ch_ppocr_server_v1.1_rec_pre.tar) | -| 中英文超轻量压缩OCR模型（3.5M） | ch_ppocr_mobile_slim_v1.1_xx| 移动端 |[推理模型](https://paddleocr.bj.bcebos.com/20-09-22/mobile-slim/det/ch_ppocr_mobile_v1.1_det_prune_infer.tar) / [slim模型](https://paddleocr.bj.bcebos.com/20-09-22/mobile-slim/det/ch_ppocr_mobile_v1.1_det_prune_opt.nb) |[推理模型](https://paddleocr.bj.bcebos.com/20-09-22/cls/ch_ppocr_mobile_v1.1_cls_quant_infer.tar) / [slim模型](https://paddleocr.bj.bcebos.com/20-09-22/cls/ch_ppocr_mobile_cls_quant_opt.nb)| [推理模型](https://paddleocr.bj.bcebos.com/20-09-22/mobile-slim/rec/ch_ppocr_mobile_v1.1_rec_quant_infer.tar) / [slim模型](https://paddleocr.bj.bcebos.com/20-09-22/mobile-slim/rec/ch_ppocr_mobile_v1.1_rec_quant_opt.nb)| +| 中英文超轻量压缩OCR模型（3.5M） | ch_ppocr_mobile_slim_v1.1_xx| 移动端 |[推理模型](https://paddleocr.bj.bcebos.com/20-09-22/mobile-slim/det/ch_ppocr_mobile_v1.1_det_prune_infer.tar) / [slim模型](https://paddleocr.bj.bcebos.com/20-09-22/mobile-slim/det/ch_ppocr_mobile_v1.1_det_prune_opt.nb) |[推理模型](https://paddleocr.bj.bcebos.com/20-09-22/cls/ch_ppocr_mobile_v1.1_cls_quant_infer.tar) / [slim模型](https://paddleocr.bj.bcebos.com/20-09-22/cls/ch_ppocr_mobile_v1.1_cls_quant_opt.nb)| [推理模型](https://paddleocr.bj.bcebos.com/20-09-22/mobile-slim/rec/ch_ppocr_mobile_v1.1_rec_quant_infer.tar) / [slim模型](https://paddleocr.bj.bcebos.com/20-09-22/mobile-slim/rec/ch_ppocr_mobile_v1.1_rec_quant_opt.nb)| 更多模型下载（包括多语言），可以参考[PP-OCR v1.1 系列模型下载](./doc/doc_ch/models_list.md) diff --git a/deploy/lite/config.txt b/deploy/lite/config.txt index f08f8e49dfe45ab0f4e47c994d471e0515771df6..670b2ff00efd7bd289c34c460df4582dd72a2df6 100644 --- a/deploy/lite/config.txt +++ b/deploy/lite/config.txt @@ -1,4 +1,5 @@ max_side_len 960 det_db_thresh 0.3 det_db_box_thresh 0.5 -det_db_unclip_ratio 1.6 \ No newline at end of file +det_db_unclip_ratio 1.6 +use_direction_classify 0 \ No newline at end of file diff --git a/deploy/lite/ocr_db_crnn.cc b/deploy/lite/ocr_db_crnn.cc index 2786d15ad0581bf10f9e8de4b06f94cc5d4027aa..f3c43ebde93678e14609f0ddc90207b0a5ab965c 100644 --- a/deploy/lite/ocr_db_crnn.cc +++ b/deploy/lite/ocr_db_crnn.cc @@ -114,6 +114,7 @@ cv::Mat RunClsModel(cv::Mat img, std::shared_ptr predictor_cls, cv::Mat srcimg; img.copyTo(srcimg); cv::Mat crop_img; + img.copyTo(crop_img); cv::Mat resize_img; int index = 0; @@ -154,7 +155,8 @@ void RunRecModel(std::vector>> boxes, cv::Mat img, std::vector &rec_text, std::vector &rec_text_score, std::vector charactor_dict, - std::shared_ptr predictor_cls) { + std::shared_ptr predictor_cls, + int use_direction_classify) { std::vector mean = {0.5f, 0.5f, 0.5f}; std::vector scale = {1 / 0.5f, 1 / 0.5f, 1 / 0.5f}; @@ -166,7 +168,9 @@ void RunRecModel(std::vector>> boxes, cv::Mat img, int index = 0; for (int i = boxes.size() - 1; i >= 0; i--) { crop_img = GetRotateCropImage(srcimg, boxes[i]); - crop_img = RunClsModel(crop_img, predictor_cls); + if (use_direction_classify >= 1) { + crop_img = RunClsModel(crop_img, predictor_cls); + } float wh_ratio = static_cast(crop_img.cols) / static_cast(crop_img.rows); @@ -378,6 +382,7 @@ int main(int argc, char **argv) { //// load config from txt file auto Config = LoadConfigTxt("./config.txt"); + int use_direction_classify = int(Config["use_direction_classify"]); auto start = std::chrono::system_clock::now(); @@ -393,8 +398,9 @@ int main(int argc, char **argv) { std::vector rec_text; std::vector rec_text_score; + RunRecModel(boxes, srcimg, rec_predictor, rec_text, rec_text_score, - charactor_dict, cls_predictor); + charactor_dict, cls_predictor, use_direction_classify); auto end = std::chrono::system_clock::now(); auto duration = diff --git a/deploy/lite/readme.md b/deploy/lite/readme.md index 93f38e057258edad0ac5dd3bbbb7b3789011cefb..5b36d0dc8384b812913f70e377e74f97e52e8299 100644 --- a/deploy/lite/readme.md +++ b/deploy/lite/readme.md @@ -79,7 +79,9 @@ inference_lite_lib.android.armv8/ Paddle-Lite 提供了多种策略来自动优化原始的模型，其中包括量化、子图融合、混合调度、Kernel优选等方法，使用Paddle-lite的opt工具可以自动 对inference模型进行优化，优化后的模型更轻量，模型运行速度更快。 -下述表格中提供了优化好的超轻量中文模型： +如果已经准备好了 `.nb` 结尾的模型文件，可以跳过此步骤。 + +下述表格中也提供了一系列中文移动端模型： |模型版本|模型简介|模型大小|检测模型|文本方向分类模型|识别模型|Paddle-Lite版本| |-|-|-|-|-|-|-| @@ -141,11 +143,9 @@ wget https://paddleocr.bj.bcebos.com/ch_models/ch_rec_mv3_crnn_infer.tar && tar ./opt --model_file=./ch_rec_mv3_crnn/model --param_file=./ch_rec_mv3_crnn/params --optimize_out_type=naive_buffer --optimize_out=./ch_rec_mv3_crnn_opt --valid_targets=arm ``` -# 转换V1.1检测模型 - 转换成功后，当前目录下会多出`.nb`结尾的文件，即是转换成功的模型文件。 -注意：使用paddle-lite部署时，需要使用opt工具优化后的模型。 opt 转换的输入模型是paddle保存的inference模型 +注意：使用paddle-lite部署时，需要使用opt工具优化后的模型。 opt 工具的输入模型是paddle保存的inference模型 ### 2.2 与手机联调 @@ -204,7 +204,7 @@ demo/cxx/ocr/ | |--ch_ppocr_mobile_v1.1_rec_quant_opt.nb 优化后的识别模型文件 | |--ch_ppocr_mobile_cls_quant_opt.nb 优化后的文字方向分类器模型文件 | |--11.jpg 待测试图像 -| |--ppocr_keys_v1.txt 字典文件 +| |--ppocr_keys_v1.txt 中文字典文件 | |--libpaddle_light_api_shared.so C++预测库文件 | |--config.txt DB-CRNN超参数配置 |-- config.txt DB-CRNN超参数配置 @@ -214,7 +214,27 @@ demo/cxx/ocr/ |-- db_post_process.h |-- Makefile 编译文件 |-- ocr_db_crnn.cc C++预测源文件 +``` + +#### 注意： +1. ppocr_keys_v1.txt是中文字典文件，如果使用的 nb 模型是英文数字或其他语言的模型，需要更换为对应语言的字典。 +PaddleOCR 在ppocr/utils/下存放了多种字典，包括： +``` +french_dict.txt # 法语字典 +german_dict.txt # 德语字典 +ic15_dict.txt # 英文字典 +japan_dict.txt # 日语字典 +korean_dict.txt # 韩语字典 +ppocr_keys_v1.txt # 中文字典 +``` +2. `config.txt` 包含了检测器、分类器的超参数，如下： +``` +max_side_len 960 # 输入图像长宽大于960时，等比例缩放图像，使得图像最长边为960 +det_db_thresh 0.3 # 用于过滤DB预测的二值化图像，设置为0.-0.3对结果影响不明显 +det_db_box_thresh 0.5 # DB后处理过滤box的阈值，如果检测存在漏框情况，可酌情减小 +det_db_unclip_ratio 1.6 # 表示文本框的紧致程度，越小则文本框更靠近文本 +use_direction_classify 1 # 是否使用方向分类器，0表示不使用，1表示使用 ``` 5. 启动调试 @@ -224,7 +244,7 @@ demo/cxx/ocr/ ``` # 执行编译，得到可执行文件ocr_db_crnn # ocr_db_crnn可执行文件的使用方式为: - # ./ocr_db_crnn 检测模型文件 识别模型文件 测试图像路径 + # ./ocr_db_crnn 检测模型文件 方向分类器模型文件 识别模型文件 测试图像路径 字典文件路径 make -j # 将编译的可执行文件移动到debug文件夹中 mv ocr_db_crnn ./debug/ @@ -243,3 +263,14 @@ demo/cxx/ocr/
+ + +## FAQ +Q1：如果想更换模型怎么办，需要重新按照流程走一遍吗？ +A1：如果已经走通了上述步骤，更换模型只需要替换 .nb 模型文件即可，同时要注意字典更新 + +Q2：换一个图测试怎么做？ +A2：替换debug下的.jpg测试图像为你想要测试的图像，adb push 到手机上即可 + +Q3：如何封装到手机APP中？ +A3：此demo旨在提供能在手机上运行OCR的核心算法部分，PaddleOCR/deploy/android_demo是将这个demo封装到手机app的示例，供参考 diff --git a/deploy/lite/readme_en.md b/deploy/lite/readme_en.md index a62f0478843fb28b21404f282d981daf7a0c294f..64679a39a14830b0d9d637b32a0cafa87c847c3c 100644 --- a/deploy/lite/readme_en.md +++ b/deploy/lite/readme_en.md @@ -56,13 +56,14 @@ inference_lite_lib.android.armv8/ ``` - ## 4. Inference Model Optimization Paddle Lite provides a variety of strategies to automatically optimize the original training model, including quantization, sub-graph fusion, hybrid scheduling, Kernel optimization and so on. In order to make the optimization process more convenient and easy to use, Paddle Lite provide opt tools to automatically complete the optimization steps and output a lightweight, optimal executable model. -If you use PaddleOCR 8.6M OCR model to deploy, you can directly download the optimized model. +If you have prepared the model file ending in `.nb`, you can skip this step. +The following table also provides a series of models that can be deployed on mobile phones to recognize Chinese. +You can directly download the optimized model. |Version|Introduction|Model size|Detection model|Text Direction model|Recognition model|Paddle Lite branch | |-|-|-|-|-|-| @@ -178,6 +179,28 @@ demo/cxx/ocr/ ``` +#### Note: +1. ppocr_keys_v1.txt is a Chinese dictionary file. +If the nb model is used for English recognition or other language recognition, dictionary file should be replaced with a dictionary of the corresponding language. +PaddleOCR provides a variety of dictionaries under ppocr/utils/, including: +``` +french_dict.txt # french +german_dict.txt # german +ic15_dict.txt # english +japan_dict.txt # japan +korean_dict.txt # korean +ppocr_keys_v1.txt # chinese +``` + +2. `config.txt` of the detector and classifier, as shown below: +``` +max_side_len 960 # Limit the maximum image height and width to 960 +det_db_thresh 0.3 # Used to filter the binarized image of DB prediction, setting 0.-0.3 has no obvious effect on the result +det_db_box_thresh 0.5 # DDB post-processing filter box threshold, if there is a missing box detected, it can be reduced as appropriate +det_db_unclip_ratio 1.6 # Indicates the compactness of the text box, the smaller the value, the closer the text box to the text +use_direction_classify 1 # Whether to use the direction classifier, 0 means not to use, 1 means to use +``` + 5. Run Model on phone ``` @@ -197,3 +220,15 @@ The outputs are as follows:
+ +## FAQ + +Q1: What if I want to change the model, do I need to run it again according to the process? +A1: If you have performed the above steps, you only need to replace the .nb model file to complete the model replacement. + +Q2: How to test with another picture? +A2: Replace the .jpg test image under `./debug` with the image you want to test, and run `adb push` to push new image to the phone. + +Q3: How to package it into the mobile APP? +A3: This demo aims to provide the core algorithm part that can run OCR on mobile phones. Further, +PaddleOCR/deploy/android_demo is an example of encapsulating this demo into a mobile app for reference.