diff --git a/doc/datasets/doc.jpg b/doc/datasets/doc.jpg new file mode 100644 index 0000000000000000000000000000000000000000..f57e62abe187bab5b9d49d406e313c2439d0f89e Binary files /dev/null and b/doc/datasets/doc.jpg differ diff --git a/doc/doc_ch/benchmark.md b/doc/doc_ch/benchmark.md index 65d9a534a5fca0d31500db6ca4cf4cf7f360b958..7308ba2c1404dfc74a72aca339d0ab410325f850 100644 --- a/doc/doc_ch/benchmark.md +++ b/doc/doc_ch/benchmark.md @@ -1,29 +1,51 @@ # Benchmark -本文给出了PaddleOCR超轻é‡ä¸æ–‡æ¨¡åž‹ï¼ˆ8.6M）在å„å¹³å°çš„预测耗时benchmark。 +本文给出了ä¸è‹±æ–‡OCRç³»åˆ—æ¨¡åž‹ç²¾åº¦æŒ‡æ ‡å’Œåœ¨å„å¹³å°é¢„测耗时的benchmark。 ## æµ‹è¯•æ•°æ® -- 从ä¸æ–‡å…¬å¼€æ•°æ®é›†[ICDAR2017-RCTW](https://github.com/PaddlePaddle/PaddleOCR/blob/develop/doc/doc_ch/datasets.md#ICDAR2017-RCTW-17)ä¸éšæœºé‡‡æ ·**500**å¼ å›¾åƒã€‚ -该集åˆå¤§éƒ¨åˆ†å›¾ç‰‡æ˜¯é€šè¿‡æ‰‹æœºæ‘„åƒå¤´åœ¨é‡Žå¤–采集的。有些是截图。这些图片展示了å„ç§å„æ ·çš„åœºæ™¯ï¼ŒåŒ…æ‹¬è¡—æ™¯ã€æµ·æŠ¥ã€èœå•ã€å®¤å†…场景和手机应用程åºçš„截图。 +针对OCR实际应用场景，包括åˆåŒï¼Œè½¦ç‰Œï¼Œé“牌，ç«è½¦ç¥¨ï¼ŒåŒ–验å•ï¼Œè¡¨æ ¼ï¼Œè¯ä¹¦ï¼Œè¡—景文å—，å片，数ç 显示å±ç‰ï¼Œæ”¶é›†çš„300å¼ å›¾åƒï¼Œæ¯å¼ 图平å‡æœ‰17个文本框，下图给出了一些图åƒç¤ºä¾‹ã€‚ -## è¯„ä¼°æŒ‡æ ‡ -在四ç§å¹³å°ä¸Šçš„é¢„æµ‹è€—æ—¶æŒ‡æ ‡å¦‚ä¸‹ï¼š +<div align="center"> +<img src="../datasets/doc.jpg" width = "800" height = "200" /> +</div> -|长边尺寸(px)|T4(s)|V100(s)|Intel至强6148(s)|éªé¾™855(s)| -|-|-|-|-|-| -|960|0.092|0.057|0.319|0.354| -|640|0.067|0.045|0.198|0.236| -|480|0.057|0.043|0.151|0.175| +## è¯„ä¼°æŒ‡æ ‡ -说明： +说明： +- v1.0æ˜¯æœªæ·»åŠ ä¼˜åŒ–ç–略的DB+CRNN模型，v1.1æ˜¯æ·»åŠ å¤šç§ä¼˜åŒ–ç–略和方å‘分类器的PP-OCR模型。slim_v1.1是使用è£å‰ªæˆ–é‡åŒ–的模型。 +- 检测输入图åƒçš„的长边尺寸是960。 - 评估耗时阶段为图åƒè¾“入到结果输出的完整阶段，包括了图åƒçš„预处ç†å’ŒåŽå¤„ç†ã€‚ -- `Intel至强6148`为æœåŠ¡å™¨ç«¯CPUåž‹å·ï¼Œæµ‹è¯•ä¸ä½¿ç”¨Intel MKL-DNN åŠ é€ŸCPU预测速度，使用该æ“作需è¦ï¼š - - 更新到飞桨latest版本：https://www.paddlepaddle.org.cn/documentation/docs/zh/install/Tables.html#whl-dev ï¼Œè¯·æ ¹æ®è‡ªå·±çŽ¯å¢ƒçš„CUDA版本和Python版本选择相应的mkl版wheel包，如，CUDA10ã€Python3.7环境，应æ“作： - ```shell - # 获å–安装包 - wget https://paddle-wheel.bj.bcebos.com/0.0.0-gpu-cuda10-cudnn7-mkl/paddlepaddle_gpu-0.0.0-cp37-cp37m-linux_x86_64.whl - # 安装 - pip3.7 install paddlepaddle_gpu-0.0.0-cp37-cp37m-linux_x86_64.whl - ``` - - 预测时使用å‚æ•°æ‰“å¼€åŠ é€Ÿå¼€å…³ï¼š `--enable_mkldnn True` +- `Intel至强6148`为æœåŠ¡å™¨ç«¯CPUåž‹å·ï¼Œæµ‹è¯•ä¸ä½¿ç”¨Intel MKL-DNN åŠ é€Ÿã€‚ - `éªé¾™855`为移动端处ç†å¹³å°åž‹å·ã€‚ + +ä¸åŒé¢„测模型大å°å’Œæ•´ä½“识别精度对比 + +| 模型å称 | 整体模型<br>大å°\(M\) | 检测模型<br>大å°\(M\) | æ–¹å‘分类器<br>模型大å°\(M\) | 识别模型<br>大å°\(M\) | 整体识别<br>F\-score | +|:-:|:-:|:-:|:-:|:-:|:-:| +| ch\_ppocr\_mobile\_v1\.1 | 8\.1 | 2\.6 | 0\.9 | 4\.6 | 0\.5193 | +| ch\_ppocr\_server\_v1\.1 | 155\.1 | 47\.2 | 0\.9 | 107 | 0\.5414 | +| ch\_ppocr\_mobile\_v1\.0 | 8\.6 | 4\.1 | \- | 4\.5 | 0\.393 | +| ch\_ppocr\_server\_v1\.0 | 203\.8 | 98\.5 | \- | 105\.3 | 0\.4436 | + +ä¸åŒé¢„测模型在T4 GPU上预测速度对比，å•ä½ms + +| 模型å称 | 整体 | 检测 | æ–¹å‘分类器 | 识别 | +|:-:|:-:|:-:|:-:|:-:| +| ch\_ppocr\_mobile\_v1\.1 | 137 | 35 | 24 | 78 | +| ch\_ppocr\_server\_v1\.1 | 204 | 39 | 25 | 140 | +| ch\_ppocr\_mobile\_v1\.0 | 117 | 41 | \- | 76 | +| ch\_ppocr\_server\_v1\.0 | 199 | 52 | \- | 147 | + +ä¸åŒé¢„测模型在CPU上预测速度对比，å•ä½ms + +| 模型å称 | 整体 | 检测 | æ–¹å‘分类器 | 识别 | +|:-:|:-:|:-:|:-:|:-:| +| ch\_ppocr\_mobile\_v1\.1 | 421 | 164 | 51 | 206 | +| ch\_ppocr\_mobile\_v1\.0 | 398 | 219 | \- | 179 | + +è£å‰ªé‡åŒ–模型和原始模型模型大å°ï¼Œæ•´ä½“识别精度和在SD 855上预测速度对比 + +| 模型å称 | 整体模型<br>大å°\(M\) | 检测模型<br>大å°\(M\) | æ–¹å‘分类器<br>模型大å°\(M\) | 识别模型<br>大å°\(M\) | 整体识别<br>F\-score | SD 855<br>\(ms\) | +|:-:|:-:|:-:|:-:|:-:|:-:|:-:| +| ch\_ppocr\_mobile\_v1\.1 | 8\.1 | 2\.6 | 0\.9 | 4\.6 | 0\.5193 | 306 | +| ch\_ppocr\_mobile\_slim\_v1\.1 | 3\.5 | 1\.4 | 0\.5 | 1\.6 | 0\.521 | 268 | diff --git a/doc/doc_ch/benchmark_en.md b/doc/doc_ch/benchmark_en.md new file mode 100644 index 0000000000000000000000000000000000000000..1e8c31dff081214a82fc99aa161305a0c8da6b59 --- /dev/null +++ b/doc/doc_ch/benchmark_en.md @@ -0,0 +1,56 @@ +# BENCHMARK + +This document gives the performance of the series models for Chinese and English recognition. + +## TEST DATA + +We collected 300 images for different real application scenarios to evaluate the overall OCR system, including contract samples, license plates, nameplates, train tickets, test sheets, forms, certificates, street view images, business cards, digital meter, etc. The following figure shows some images of the test set. + +<div align="center"> +<img src="../datasets/doc.jpg" width = "800" height = "200" /> +</div> + +## MEASUREMENT + +Explanation: +- v1.0 indicates DB+CRNN models without the strategies. v1.1 indicates the PP-OCR models with the strategies and the direction classify. slim_v1.1 indicates the PP-OCR models with prunner or quantization. + +- The long size of the input for the text detector is 960. + +- The evaluation time-consuming stage is the complete stage from image input to result output, including image pre-processing and post-processing. + +- ```Intel Xeon 6148``` is the server-side CPU model. Intel MKL-DNN is used in the test to accelerate the CPU prediction speed. + +- ```Snapdragon 855``` is a mobile processing platform model. + +Compares the model size and F-score: + +| Model Name | Model Size <br> of the <br> Whole System\(M\) | Model Size <br>of the Text <br> Detector\(M\) | Model Size <br> of the Direction <br> Classifier\(M\) | Model Size<br>of the Text <br> Recognizer \(M\) | F\-score | +|:-:|:-:|:-:|:-:|:-:|:-:| +| ch\_ppocr\_mobile\_v1\.1 | 8\.1 | 2\.6 | 0\.9 | 4\.6 | 0\.5193 | +| ch\_ppocr\_server\_v1\.1 | 155\.1 | 47\.2 | 0\.9 | 107 | 0\.5414 | +| ch\_ppocr\_mobile\_v1\.0 | 8\.6 | 4\.1 | \- | 4\.5 | 0\.393 | +| ch\_ppocr\_server\_v1\.0 | 203\.8 | 98\.5 | \- | 105\.3 | 0\.4436 | + +Compares the time-consuming on T4 GPU (ms): + +| Model Name | Overall | Text Detector | Direction Classifier | Text Recognizer | +|:-:|:-:|:-:|:-:|:-:| +| ch\_ppocr\_mobile\_v1\.1 | 137 | 35 | 24 | 78 | +| ch\_ppocr\_server\_v1\.1 | 204 | 39 | 25 | 140 | +| ch\_ppocr\_mobile\_v1\.0 | 117 | 41 | \- | 76 | +| ch\_ppocr\_server\_v1\.0 | 199 | 52 | \- | 147 | + +Compares the time-consuming on CPU (ms): + +| Model Name | Overall | Text Detector | Direction Classifier | Text Recognizer | +|:-:|:-:|:-:|:-:|:-:| +| ch\_ppocr\_mobile\_v1\.1 | 421 | 164 | 51 | 206 | +| ch\_ppocr\_mobile\_v1\.0 | 398 | 219 | \- | 179 | + +Compares the model size, F-score, the time-consuming on SD 855 of between the slim models and the original models: + +| Model Name | Model Size <br> of the <br> Whole System\(M\) | Model Size <br>of the Text <br> Detector\(M\) | Model Size <br> of the Direction <br> Classifier\(M\) | Model Size<br>of the Text <br> Recognizer \(M\) | F\-score | SD 855<br>\(ms\) | +|:-:|:-:|:-:|:-:|:-:|:-:|:-:| +| ch\_ppocr\_mobile\_v1\.1 | 8\.1 | 2\.6 | 0\.9 | 4\.6 | 0\.5193 | 306 | +| ch\_ppocr\_mobile\_slim\_v1\.1 | 3\.5 | 1\.4 | 0\.5 | 1\.6 | 0\.521 | 268 | diff --git a/doc/doc_ch/framework.png b/doc/doc_ch/framework.png new file mode 100644 index 0000000000000000000000000000000000000000..db151a0e161771f33a438d2e98b49777c34950f8 Binary files /dev/null and b/doc/doc_ch/framework.png differ