@@ -137,13 +139,9 @@ paddleocr --image_dir ./doc/imgs_en/254.jpg --lang=en
The result is a list, each item contains a text box, text and recognition confidence
```text
-[('PHO CAPITAL', 0.95723116), [[66.0, 50.0], [327.0, 44.0], [327.0, 76.0], [67.0, 82.0]]]
-[('107 State Street', 0.96311164), [[72.0, 90.0], [451.0, 84.0], [452.0, 116.0], [73.0, 121.0]]]
-[('Montpelier Vermont', 0.97389287), [[69.0, 132.0], [501.0, 126.0], [501.0, 158.0], [70.0, 164.0]]]
-[('8022256183', 0.99810505), [[71.0, 175.0], [363.0, 170.0], [364.0, 202.0], [72.0, 207.0]]]
-[('REG 07-24-201706:59 PM', 0.93537045), [[73.0, 299.0], [653.0, 281.0], [654.0, 318.0], [74.0, 336.0]]]
-[('045555', 0.99346405), [[509.0, 331.0], [651.0, 325.0], [652.0, 356.0], [511.0, 362.0]]]
-[('CT1', 0.9988654), [[535.0, 367.0], [654.0, 367.0], [654.0, 406.0], [535.0, 406.0]]]
+[[[67.0, 51.0], [327.0, 46.0], [327.0, 74.0], [68.0, 80.0]], ('PHOCAPITAL', 0.9944712519645691)]
+[[[72.0, 92.0], [453.0, 84.0], [454.0, 114.0], [73.0, 122.0]], ('107 State Street', 0.9744491577148438)]
+[[[69.0, 135.0], [501.0, 125.0], [501.0, 156.0], [70.0, 165.0]], ('Montpelier Vermont', 0.9357033967971802)]
......
```
@@ -234,10 +232,10 @@ im_show.save('result.jpg')
Output will be a list, each item contains bounding box, text and recognition confidence
```bash
-[[[442.0, 173.0], [1169.0, 173.0], [1169.0, 225.0], [442.0, 225.0]], ['ACKNOWLEDGEMENTS', 0.99283075]]
-[[[393.0, 340.0], [1207.0, 342.0], [1207.0, 389.0], [393.0, 387.0]], ['We would like to thank all the designers and', 0.9357758]]
-[[[399.0, 398.0], [1204.0, 398.0], [1204.0, 433.0], [399.0, 433.0]], ['contributors whohave been involved in the', 0.9592447]]
-......
+[[[441.0, 174.0], [1166.0, 176.0], [1165.0, 222.0], [441.0, 221.0]], ('ACKNOWLEDGEMENTS', 0.9971134662628174)]
+ [[[403.0, 346.0], [1204.0, 348.0], [1204.0, 384.0], [402.0, 383.0]], ('We would like to thank all the designers and', 0.9761400818824768)]
+ [[[403.0, 396.0], [1204.0, 398.0], [1204.0, 434.0], [402.0, 433.0]], ('contributors who have been involved in the', 0.9791957139968872)]
+ ......
```
Visualization of results
diff --git a/doc/doc_en/recognition_en.md b/doc/doc_en/recognition_en.md
index a6b255f9501a0c2d34c31162385bcf03ff578aa3..60b4a1b26b373adc562ab9624e55ffe59a775a35 100644
--- a/doc/doc_en/recognition_en.md
+++ b/doc/doc_en/recognition_en.md
@@ -1,7 +1,7 @@
# Text Recognition
- [1. Data Preparation](#DATA_PREPARATION)
- * [1.1 Costom Dataset](#Costom_Dataset)
+ * [1.1 Custom Dataset](#Custom_Dataset)
* [1.2 Dataset Download](#Dataset_download)
* [1.3 Dictionary](#Dictionary)
* [1.4 Add Space Category](#Add_space_category)
@@ -28,7 +28,34 @@
To prepare datasets, refer to [ocr_datasets](./dataset/ocr_datasets.md) .
-If you want to reproduce the paper SAR, you need to download extra dataset [SynthAdd](https://pan.baidu.com/share/init?surl=uV0LtoNmcxbO-0YA7Ch4dg), extraction code: 627x. Besides, icdar2013, icdar2015, cocotext, IIIT5k datasets are also used to train. For specific details, please refer to the paper SAR.
+PaddleOCR provides label files for training the icdar2015 dataset, which can be downloaded in the following ways:
+
+```
+# Training set label
+wget -P ./train_data/ic15_data https://paddleocr.bj.bcebos.com/dataset/rec_gt_train.txt
+# Test Set Label
+wget -P ./train_data/ic15_data https://paddleocr.bj.bcebos.com/dataset/rec_gt_test.txt
+```
+
+PaddleOCR also provides a data format conversion script, which can convert ICDAR official website label to a data format
+supported by PaddleOCR. The data conversion tool is in `ppocr/utils/gen_label.py`, here is the training set as an example:
+
+```
+# convert the official gt to rec_gt_label.txt
+python gen_label.py --mode="rec" --input_path="{path/of/origin/label}" --output_label="rec_gt_label.txt"
+```
+
+The data format is as follows, (a) is the original picture, (b) is the Ground Truth text file corresponding to each picture:
+
+
+
+
+- Multilingual dataset
+
+The multi-language model training method is the same as the Chinese model. The training data set is 100w synthetic data. A small amount of fonts and test data can be downloaded using the following two methods.
+* [Baidu Netdisk](https://pan.baidu.com/s/1bS_u207Rm7YbY33wOECKDA) ,Extraction code:frgi.
+* [Google drive](https://drive.google.com/file/d/18cSWX7wXSy4G0tbKJ0d9PuIaiwRLHpjA/view)
+
### 1.2 Dictionary
@@ -101,11 +128,11 @@ First download the pretrain model, you can download the trained model to finetun
```
cd PaddleOCR/
-# Download the pre-trained model of MobileNetV3
-wget -P ./pretrain_models/ https://paddleocr.bj.bcebos.com/dygraph_v2.0/en/rec_mv3_none_bilstm_ctc_v2.0_train.tar
+# Download the pre-trained model of en_PP-OCRv3
+wget -P ./pretrain_models/ https://paddleocr.bj.bcebos.com/PP-OCRv3/english/en_PP-OCRv3_rec_train.tar
# Decompress model parameters
cd pretrain_models
-tar -xf rec_mv3_none_bilstm_ctc_v2.0_train.tar && rm -rf rec_mv3_none_bilstm_ctc_v2.0_train.tar
+tar -xf en_PP-OCRv3_rec_train.tar && rm -rf en_PP-OCRv3_rec_train.tar
```
Start training:
@@ -115,9 +142,10 @@ Start training:
# Training icdar15 English data and The training log will be automatically saved as train.log under "{save_model_dir}"
#specify the single card training(Long training time, not recommended)
-python3 tools/train.py -c configs/rec/rec_icdar15_train.yml
+python3 tools/train.py -c configs/rec/PP-OCRv3/en_PP-OCRv3_rec.yml -o Global.pretrained_model=en_PP-OCRv3_rec_train/best_accuracy
+
#specify the card number through --gpus
-python3 -m paddle.distributed.launch --gpus '0,1,2,3' tools/train.py -c configs/rec/rec_icdar15_train.yml
+python3 -m paddle.distributed.launch --gpus '0,1,2,3' tools/train.py -c configs/rec/PP-OCRv3/en_PP-OCRv3_rec.yml -o Global.pretrained_model=en_PP-OCRv3_rec_train/best_accuracy
```
@@ -125,31 +153,13 @@ PaddleOCR supports alternating training and evaluation. You can modify `eval_bat
If the evaluation set is large, the test will be time-consuming. It is recommended to reduce the number of evaluations, or evaluate after training.
-* Tip: You can use the `-c` parameter to select multiple model configurations under the `configs/rec/` path for training. The recognition algorithms supported by PaddleOCR are:
-
-
-| Configuration file | Algorithm | backbone | trans | seq | pred |
-| :--------: | :-------: | :-------: | :-------: | :-----: | :-----: |
-| [rec_chinese_lite_train_v2.0.yml](../../configs/rec/ch_ppocr_v2.0/rec_chinese_lite_train_v2.0.yml) | CRNN | Mobilenet_v3 small 0.5 | None | BiLSTM | ctc |
-| [rec_chinese_common_train_v2.0.yml](../../configs/rec/ch_ppocr_v2.0/rec_chinese_common_train_v2.0.yml) | CRNN | ResNet34_vd | None | BiLSTM | ctc |
-| rec_chinese_lite_train.yml | CRNN | Mobilenet_v3 small 0.5 | None | BiLSTM | ctc |
-| rec_chinese_common_train.yml | CRNN | ResNet34_vd | None | BiLSTM | ctc |
-| rec_icdar15_train.yml | CRNN | Mobilenet_v3 large 0.5 | None | BiLSTM | ctc |
-| rec_mv3_none_bilstm_ctc.yml | CRNN | Mobilenet_v3 large 0.5 | None | BiLSTM | ctc |
-| rec_mv3_none_none_ctc.yml | Rosetta | Mobilenet_v3 large 0.5 | None | None | ctc |
-| rec_r34_vd_none_bilstm_ctc.yml | CRNN | Resnet34_vd | None | BiLSTM | ctc |
-| rec_r34_vd_none_none_ctc.yml | Rosetta | Resnet34_vd | None | None | ctc |
-| rec_mv3_tps_bilstm_att.yml | CRNN | Mobilenet_v3 | TPS | BiLSTM | att |
-| rec_r34_vd_tps_bilstm_att.yml | CRNN | Resnet34_vd | TPS | BiLSTM | att |
-| rec_r50fpn_vd_none_srn.yml | SRN | Resnet50_fpn_vd | None | rnn | srn |
-| rec_mtb_nrtr.yml | NRTR | nrtr_mtb | None | transformer encoder | transformer decoder |
-| rec_r31_sar.yml | SAR | ResNet31 | None | LSTM encoder | LSTM decoder |
+* Tip: You can use the `-c` parameter to select multiple model configurations under the `configs/rec/` path for training. The recognition algorithms supported at [rec_algorithm](https://github.com/PaddlePaddle/PaddleOCR/blob/dygraph/doc/doc_en/algorithm_overview.md):
For training Chinese data, it is recommended to use
-[rec_chinese_lite_train_v2.0.yml](../../configs/rec/ch_ppocr_v2.0/rec_chinese_lite_train_v2.0.yml). If you want to try the result of other algorithms on the Chinese data set, please refer to the following instructions to modify the configuration file:
-co
-Take `rec_chinese_lite_train_v2.0.yml` as an example:
+[ch_PP-OCRv3_rec_distillation.yml](../../configs/rec/PP-OCRv3/ch_PP-OCRv3_rec_distillation.yml). If you want to try the result of other algorithms on the Chinese data set, please refer to the following instructions to modify the configuration file:
+
+Take `ch_PP-OCRv3_rec_distillation.yml` as an example:
```
Global:
...
@@ -183,7 +193,7 @@ Train:
...
- RecResizeImg:
# Modify image_shape to fit long text
- image_shape: [3, 32, 320]
+ image_shape: [3, 48, 320]
...
loader:
...
@@ -203,7 +213,7 @@ Eval:
...
- RecResizeImg:
# Modify image_shape to fit long text
- image_shape: [3, 32, 320]
+ image_shape: [3, 48, 320]
...
loader:
# Eval batch_size for Single card
@@ -380,11 +390,12 @@ Running on a DCU device requires setting the environment variable `export HIP_VI
### 3.1 Evaluation
-The model parameters during training are saved in the `Global.save_model_dir` directory by default. When evaluating indicators, you need to set `Global.checkpoints` to point to the saved parameter file. The evaluation dataset can be set by modifying the `Eval.dataset.label_file_list` field in the `configs/rec/rec_icdar15_train.yml` file.
+The model parameters during training are saved in the `Global.save_model_dir` directory by default. When evaluating indicators, you need to set `Global.checkpoints` to point to the saved parameter file. The evaluation dataset can be set by modifying the `Eval.dataset.label_file_list` field in the `configs/rec/PP-OCRv3/en_PP-OCRv3_rec.yml` file.
+
```
# GPU evaluation, Global.checkpoints is the weight to be tested
-python3 -m paddle.distributed.launch --gpus '0' tools/eval.py -c configs/rec/rec_icdar15_train.yml -o Global.checkpoints={path/to/weights}/best_accuracy
+python3 -m paddle.distributed.launch --gpus '0' tools/eval.py -c configs/rec/PP-OCRv3/en_PP-OCRv3_rec.yml -o Global.checkpoints={path/to/weights}/best_accuracy
```
@@ -417,7 +428,7 @@ Among them, best_accuracy.* is the best model on the evaluation set; iter_epoch_
```
# Predict English results
-python3 tools/infer_rec.py -c configs/rec/ch_ppocr_v2.0/rec_chinese_lite_train_v2.0.yml -o Global.pretrained_model={path/to/weights}/best_accuracy Global.load_static_weights=false Global.infer_img=doc/imgs_words/en/word_1.jpg
+python3 tools/infer_rec.py -c configs/rec/PP-OCRv3/en_PP-OCRv3_rec.yml -o Global.pretrained_model={path/to/weights}/best_accuracy Global.infer_img=doc/imgs_words/en/word_1.png
```
@@ -436,7 +447,7 @@ The configuration file used for prediction must be consistent with the training.
```
# Predict Chinese results
-python3 tools/infer_rec.py -c configs/rec/ch_ppocr_v2.0/rec_chinese_lite_train_v2.0.yml -o Global.pretrained_model={path/to/weights}/best_accuracy Global.load_static_weights=false Global.infer_img=doc/imgs_words/ch/word_1.jpg
+python3 tools/infer_rec.py -c configs/rec/ch_ppocr_v2.0/rec_chinese_lite_train_v2.0.yml -o Global.pretrained_model={path/to/weights}/best_accuracy Global.infer_img=doc/imgs_words/ch/word_1.jpg
```
Input image:
@@ -467,7 +478,7 @@ The recognition model is converted to the inference model in the same way as the
# Global.pretrained_model parameter Set the training model address to be converted without adding the file suffix .pdmodel, .pdopt or .pdparams.
# Global.save_inference_dir Set the address where the converted model will be saved.
-python3 tools/export_model.py -c configs/rec/ch_ppocr_v2.0/rec_chinese_lite_train_v2.0.yml -o Global.pretrained_model=./ch_lite/ch_ppocr_mobile_v2.0_rec_train/best_accuracy Global.save_inference_dir=./inference/rec_crnn/
+python3 tools/export_model.py -c configs/rec/PP-OCRv3/en_PP-OCRv3_rec.yml -o Global.pretrained_model=en_PP-OCRv3_rec_train/best_accuracy Global.save_inference_dir=./inference/en_PP-OCRv3_rec/
```
If you have a model trained on your own dataset with a different dictionary file, please make sure that you modify the `character_dict_path` in the configuration file to your dictionary file path.
@@ -475,7 +486,8 @@ If you have a model trained on your own dataset with a different dictionary file
After the conversion is successful, there are three files in the model save directory:
```
-inference/rec_crnn/
+
+inference/en_PP-OCRv3_rec/
├── inference.pdiparams # The parameter file of recognition inference model
├── inference.pdiparams.info # The parameter information of recognition inference model, which can be ignored
└── inference.pdmodel # The program file of recognition model
diff --git a/doc/doc_en/whl_en.md b/doc/doc_en/whl_en.md
index 35b2b1798ad8b566ee87e921e23be84a5ecccf24..40a2e122d19679a59e7e65df29dd59781b4a2143 100644
--- a/doc/doc_en/whl_en.md
+++ b/doc/doc_en/whl_en.md
@@ -172,40 +172,42 @@ show help information
paddleocr -h
```
+**Note**: The whl package uses the `PP-OCRv3` model by default, and the input shape used by the recognition model is `3,48,320`, so if you use the recognition function, you need to add the parameter `--rec_image_shape 3,48,320`, if you do not use the default `PP- OCRv3` model, you do not need to set this parameter.
+
* detection classification and recognition
```bash
-paddleocr --image_dir PaddleOCR/doc/imgs_en/img_12.jpg --use_angle_cls true --lang en
+paddleocr --image_dir PaddleOCR/doc/imgs_en/img_12.jpg --use_angle_cls true --lang en --rec_image_shape 3,48,320
```
Output will be a list, each item contains bounding box, text and recognition confidence
```bash
-[[[442.0, 173.0], [1169.0, 173.0], [1169.0, 225.0], [442.0, 225.0]], ['ACKNOWLEDGEMENTS', 0.99283075]]
-[[[393.0, 340.0], [1207.0, 342.0], [1207.0, 389.0], [393.0, 387.0]], ['We would like to thank all the designers and', 0.9357758]]
-[[[399.0, 398.0], [1204.0, 398.0], [1204.0, 433.0], [399.0, 433.0]], ['contributors whohave been involved in the', 0.9592447]]
+[[[441.0, 174.0], [1166.0, 176.0], [1165.0, 222.0], [441.0, 221.0]], ('ACKNOWLEDGEMENTS', 0.9971134662628174)]
+[[[403.0, 346.0], [1204.0, 348.0], [1204.0, 384.0], [402.0, 383.0]], ('We would like to thank all the designers and', 0.9761400818824768)]
+[[[403.0, 396.0], [1204.0, 398.0], [1204.0, 434.0], [402.0, 433.0]], ('contributors who have been involved in the', 0.9791957139968872)]
......
```
* detection and recognition
```bash
-paddleocr --image_dir PaddleOCR/doc/imgs_en/img_12.jpg --lang en
+paddleocr --image_dir PaddleOCR/doc/imgs_en/img_12.jpg --lang en --rec_image_shape 3,48,320
```
Output will be a list, each item contains bounding box, text and recognition confidence
```bash
-[[[442.0, 173.0], [1169.0, 173.0], [1169.0, 225.0], [442.0, 225.0]], ['ACKNOWLEDGEMENTS', 0.99283075]]
-[[[393.0, 340.0], [1207.0, 342.0], [1207.0, 389.0], [393.0, 387.0]], ['We would like to thank all the designers and', 0.9357758]]
-[[[399.0, 398.0], [1204.0, 398.0], [1204.0, 433.0], [399.0, 433.0]], ['contributors whohave been involved in the', 0.9592447]]
+[[[441.0, 174.0], [1166.0, 176.0], [1165.0, 222.0], [441.0, 221.0]], ('ACKNOWLEDGEMENTS', 0.9971134662628174)]
+[[[403.0, 346.0], [1204.0, 348.0], [1204.0, 384.0], [402.0, 383.0]], ('We would like to thank all the designers and', 0.9761400818824768)]
+[[[403.0, 396.0], [1204.0, 398.0], [1204.0, 434.0], [402.0, 433.0]], ('contributors who have been involved in the', 0.9791957139968872)]
......
```
* classification and recognition
```bash
-paddleocr --image_dir PaddleOCR/doc/imgs_words_en/word_10.png --use_angle_cls true --det false --lang en
+paddleocr --image_dir PaddleOCR/doc/imgs_words_en/word_10.png --use_angle_cls true --det false --lang en --rec_image_shape 3,48,320
```
Output will be a list, each item contains text and recognition confidence
```bash
-['PAIN', 0.990372]
+['PAIN', 0.9934559464454651]
```
* only detection
@@ -215,20 +217,20 @@ paddleocr --image_dir PaddleOCR/doc/imgs_en/img_12.jpg --rec false
Output will be a list, each item only contains bounding box
```bash
-[[756.0, 812.0], [805.0, 812.0], [805.0, 830.0], [756.0, 830.0]]
-[[820.0, 803.0], [1085.0, 801.0], [1085.0, 836.0], [820.0, 838.0]]
-[[393.0, 801.0], [715.0, 805.0], [715.0, 839.0], [393.0, 836.0]]
+[[397.0, 802.0], [1092.0, 802.0], [1092.0, 841.0], [397.0, 841.0]]
+[[397.0, 750.0], [1211.0, 750.0], [1211.0, 789.0], [397.0, 789.0]]
+[[397.0, 702.0], [1209.0, 698.0], [1209.0, 734.0], [397.0, 738.0]]
......
```
* only recognition
```bash
-paddleocr --image_dir PaddleOCR/doc/imgs_words_en/word_10.png --det false --lang en
+paddleocr --image_dir PaddleOCR/doc/imgs_words_en/word_10.png --det false --lang en --rec_image_shape 3,48,320
```
Output will be a list, each item contains text and recognition confidence
```bash
-['PAIN', 0.990372]
+['PAIN', 0.9934559464454651]
```
* only classification
@@ -366,5 +368,4 @@ im_show.save('result.jpg')
| cls | Enable classification when `ppocr.ocr` func exec((Use use_angle_cls in command line mode to control whether to start classification in the forward direction) | FALSE |
| show_log | Whether to print log| FALSE |
| type | Perform ocr or table structuring, the value is selected in ['ocr','structure'] | ocr |
-| ocr_version | OCR Model version number, the current model support list is as follows: PP-OCRv2 support Chinese detection and recognition model, PP-OCR support Chinese detection, recognition and direction classifier, multilingual recognition model | PP-OCRv2 |
-| structure_version | table structure Model version number, the current model support list is as follows: STRUCTURE support english table structure model | STRUCTURE |
+| ocr_version | OCR Model version number, the current model support list is as follows: PP-OCRv3 support Chinese and English detection and recognition model and direction classifier model, PP-OCRv2 support Chinese detection and recognition model, PP-OCR support Chinese detection, recognition and direction classifier, multilingual recognition model | PP-OCRv3 |
diff --git a/doc/features.png b/doc/features.png
index 7d6342c1eb83e0544df0045a0cfa71bc083022fe..6295c902adc007446f72a2ff08793198f9ac2c4a 100644
Binary files a/doc/features.png and b/doc/features.png differ
diff --git a/doc/features_en.png b/doc/features_en.png
index 9f0a66299bb5e922257e3327b0c6cf2d3ebfe05b..6ec1ed7bdb50cd05d1cd280c635b0aea19725cb2 100644
Binary files a/doc/features_en.png and b/doc/features_en.png differ
diff --git a/doc/imgs_results/system_res_00018069_v3.jpg b/doc/imgs_results/system_res_00018069_v3.jpg
new file mode 100644
index 0000000000000000000000000000000000000000..51808ca556b52239cad3602cd5602f4d5d0ab7ce
Binary files /dev/null and b/doc/imgs_results/system_res_00018069_v3.jpg differ
diff --git a/doc/ppocr_v3/GTC.png b/doc/ppocr_v3/GTC.png
new file mode 100644
index 0000000000000000000000000000000000000000..2af2261d51d2279f171727a5a0b5a8d974763d80
Binary files /dev/null and b/doc/ppocr_v3/GTC.png differ
diff --git a/doc/ppocr_v3/LKPAN.png b/doc/ppocr_v3/LKPAN.png
new file mode 100644
index 0000000000000000000000000000000000000000..ff0578f6901603185809e10c85793c212c40dc48
Binary files /dev/null and b/doc/ppocr_v3/LKPAN.png differ
diff --git a/doc/ppocr_v3/RSEFPN.png b/doc/ppocr_v3/RSEFPN.png
new file mode 100644
index 0000000000000000000000000000000000000000..87f7f69fb516d496c9357d81b97e5bdb750f808a
Binary files /dev/null and b/doc/ppocr_v3/RSEFPN.png differ
diff --git a/doc/ppocr_v3/SSL.png b/doc/ppocr_v3/SSL.png
new file mode 100644
index 0000000000000000000000000000000000000000..1344a2a77cf50c4ffe044e9ebc565215a2d7efbc
Binary files /dev/null and b/doc/ppocr_v3/SSL.png differ
diff --git a/doc/ppocr_v3/UDML.png b/doc/ppocr_v3/UDML.png
new file mode 100644
index 0000000000000000000000000000000000000000..3b59bc58bc1e7cb0ae0ffab30b41bc519410c794
Binary files /dev/null and b/doc/ppocr_v3/UDML.png differ
diff --git a/doc/ppocr_v3/ppocr_v3.png b/doc/ppocr_v3/ppocr_v3.png
new file mode 100644
index 0000000000000000000000000000000000000000..123c125acdcbc9e2ef6e4d6a0a1c92d01136ffde
Binary files /dev/null and b/doc/ppocr_v3/ppocr_v3.png differ
diff --git a/doc/ppocr_v3/recconaug.png b/doc/ppocr_v3/recconaug.png
new file mode 100644
index 0000000000000000000000000000000000000000..899bc430de897cbd8741e07bfa796a58bf2c7715
Binary files /dev/null and b/doc/ppocr_v3/recconaug.png differ
diff --git a/doc/ppocr_v3/svtr_g2.png b/doc/ppocr_v3/svtr_g2.png
new file mode 100644
index 0000000000000000000000000000000000000000..d589891d5897533243845a993bd56d8f75726cfc
Binary files /dev/null and b/doc/ppocr_v3/svtr_g2.png differ
diff --git a/doc/ppocr_v3/svtr_g4.png b/doc/ppocr_v3/svtr_g4.png
new file mode 100644
index 0000000000000000000000000000000000000000..234a85c44b2cc3d968942480a596b2be5e45f53d
Binary files /dev/null and b/doc/ppocr_v3/svtr_g4.png differ
diff --git a/doc/ppocr_v3/svtr_tiny.jpg b/doc/ppocr_v3/svtr_tiny.jpg
new file mode 100644
index 0000000000000000000000000000000000000000..26261047ef253e9802956f4c64449870d10de850
Binary files /dev/null and b/doc/ppocr_v3/svtr_tiny.jpg differ
diff --git a/doc/ppocr_v3/svtr_tiny.png b/doc/ppocr_v3/svtr_tiny.png
new file mode 100644
index 0000000000000000000000000000000000000000..91b3eacb9f1242806ad3520cc36252351fc7baf1
Binary files /dev/null and b/doc/ppocr_v3/svtr_tiny.png differ
diff --git a/doc/ppocrv3_framework.png b/doc/ppocrv3_framework.png
new file mode 100644
index 0000000000000000000000000000000000000000..c05398248fa7273382e9691a26d932bddc3cf84f
Binary files /dev/null and b/doc/ppocrv3_framework.png differ
diff --git "a/notebook/notebook_ch/1.introduction/OCR\346\212\200\346\234\257\345\257\274\350\256\272.ipynb" "b/notebook/notebook_ch/1.introduction/OCR\346\212\200\346\234\257\345\257\274\350\256\272.ipynb"
index 37098ab63ea0f5ca2d033b4dc6c46b14ef2eadd9..96c91b11b7bfe04e754b58b1204a339e983de9c1 100644
--- "a/notebook/notebook_ch/1.introduction/OCR\346\212\200\346\234\257\345\257\274\350\256\272.ipynb"
+++ "b/notebook/notebook_ch/1.introduction/OCR\346\212\200\346\234\257\345\257\274\350\256\272.ipynb"
@@ -164,7 +164,7 @@
"\n",
"
\n",
"\n",
- "一般的KIE方法基于命名实体识别(Named Entity Recognition,NER)[4]来研究，但是这类方法只利用了图像中的文本信息，缺少对视觉和结构信息的使用，因此精度不高。在此基础上，近几年的方法都开始将视觉和结构信息与文本信息融合到一起，按照对多模态信息进行融合时所采用的的原理可以将这些方法分为下面四种：\n",
+ "一般的KIE方法基于命名实体识别(Named Entity Recognition,NER)[4]来研究，但是这类方法只利用了图像中的文本信息，缺少对视觉和结构信息的使用，因此精度不高。在此基础上，近几年的方法都开始将视觉和结构信息与文本信息融合到一起，按照对多模态信息进行融合时所采用的原理可以将这些方法分为下面四种：\n",
"\n",
"- 基于Grid的方法\n",
"- 基于Token的方法\n",
diff --git "a/notebook/notebook_ch/3.text_recognition/\346\226\207\346\234\254\350\257\206\345\210\253\345\256\236\350\267\265\351\203\250\345\210\206.ipynb" "b/notebook/notebook_ch/3.text_recognition/\346\226\207\346\234\254\350\257\206\345\210\253\345\256\236\350\267\265\351\203\250\345\210\206.ipynb"
index 8a6cb2d788719c238fb14703a4cfb2616da1a95b..b48f90bf54f02a0f74e10505f56919c202fab859 100644
--- "a/notebook/notebook_ch/3.text_recognition/\346\226\207\346\234\254\350\257\206\345\210\253\345\256\236\350\267\265\351\203\250\345\210\206.ipynb"
+++ "b/notebook/notebook_ch/3.text_recognition/\346\226\207\346\234\254\350\257\206\345\210\253\345\256\236\350\267\265\351\203\250\345\210\206.ipynb"
@@ -2133,7 +2133,7 @@
"collapsed": false
},
"source": [
- "根据配置文件中设置的的 `save_model_dir` 字段，会有以下几种参数被保存下来：\n",
+ "根据配置文件中设置的 `save_model_dir` 字段，会有以下几种参数被保存下来：\n",
"\n",
"```\n",
"output/rec/ic15\n",
diff --git "a/notebook/notebook_ch/3.text_recognition/\346\226\207\346\234\254\350\257\206\345\210\253\347\220\206\350\256\272\351\203\250\345\210\206.ipynb" "b/notebook/notebook_ch/3.text_recognition/\346\226\207\346\234\254\350\257\206\345\210\253\347\220\206\350\256\272\351\203\250\345\210\206.ipynb"
index dd6af90a35945a7b2ffd79be6fa8d2fe6665d2c5..6913a65533a9e12b338b92b324d6ca4225b477ef 100644
--- "a/notebook/notebook_ch/3.text_recognition/\346\226\207\346\234\254\350\257\206\345\210\253\347\220\206\350\256\272\351\203\250\345\210\206.ipynb"
+++ "b/notebook/notebook_ch/3.text_recognition/\346\226\207\346\234\254\350\257\206\345\210\253\347\220\206\350\256\272\351\203\250\345\210\206.ipynb"
@@ -22,7 +22,7 @@
"\n",
"## 1 背景介绍\n",
"\n",
- "文本识别是OCR（Optical Character Recognition）的一个子任务，其任务为识别一个固定区域的的文本内容。在OCR的两阶段方法里，它接在文本检测后面，将图像信息转换为文字信息。\n",
+ "文本识别是OCR（Optical Character Recognition）的一个子任务，其任务为识别一个固定区域的文本内容。在OCR的两阶段方法里，它接在文本检测后面，将图像信息转换为文字信息。\n",
"\n",
"具体地，模型输入一张定位好的文本行，由模型预测出图片中的文字内容和置信度，可视化结果如下图所示：\n",
"\n",
diff --git "a/notebook/notebook_ch/4.ppcor_system_strategy/PP-OCR\347\263\273\347\273\237\345\217\212\344\274\230\345\214\226\347\255\226\347\225\245.ipynb" "b/notebook/notebook_ch/4.ppcor_system_strategy/PP-OCR\347\263\273\347\273\237\345\217\212\344\274\230\345\214\226\347\255\226\347\225\245.ipynb"
index 4160b077b3ef2939ffd2ac88867bb12c93438c61..17b7d9f5b845dd7ea07050bffd9ddc57c8ade721 100644
--- "a/notebook/notebook_ch/4.ppcor_system_strategy/PP-OCR\347\263\273\347\273\237\345\217\212\344\274\230\345\214\226\347\255\226\347\225\245.ipynb"
+++ "b/notebook/notebook_ch/4.ppcor_system_strategy/PP-OCR\347\263\273\347\273\237\345\217\212\344\274\230\345\214\226\347\255\226\347\225\245.ipynb"
@@ -2915,7 +2915,7 @@
"\n",
"```yaml\n",
"Architecture:\n",
- " model_type: &model_type \"rec\" # 模型类别，rec、det等，每个子网络的的模型类别都与\n",
+ " model_type: &model_type \"rec\" # 模型类别，rec、det等，每个子网络的模型类别都与\n",
" name: DistillationModel # 结构名称，蒸馏任务中，为DistillationModel，用于构建对应的结构\n",
" algorithm: Distillation # 算法名称\n",
" Models: # 模型，包含子网络的配置信息\n",
diff --git "a/notebook/notebook_ch/4.ppocr_system_strategy/PP-OCR\347\263\273\347\273\237\345\217\212\344\274\230\345\214\226\347\255\226\347\225\245.ipynb" "b/notebook/notebook_ch/4.ppocr_system_strategy/PP-OCR\347\263\273\347\273\237\345\217\212\344\274\230\345\214\226\347\255\226\347\225\245.ipynb"
index 4160b077b3ef2939ffd2ac88867bb12c93438c61..17b7d9f5b845dd7ea07050bffd9ddc57c8ade721 100644
--- "a/notebook/notebook_ch/4.ppocr_system_strategy/PP-OCR\347\263\273\347\273\237\345\217\212\344\274\230\345\214\226\347\255\226\347\225\245.ipynb"
+++ "b/notebook/notebook_ch/4.ppocr_system_strategy/PP-OCR\347\263\273\347\273\237\345\217\212\344\274\230\345\214\226\347\255\226\347\225\245.ipynb"
@@ -2915,7 +2915,7 @@
"\n",
"```yaml\n",
"Architecture:\n",
- " model_type: &model_type \"rec\" # 模型类别，rec、det等，每个子网络的的模型类别都与\n",
+ " model_type: &model_type \"rec\" # 模型类别，rec、det等，每个子网络的模型类别都与\n",
" name: DistillationModel # 结构名称，蒸馏任务中，为DistillationModel，用于构建对应的结构\n",
" algorithm: Distillation # 算法名称\n",
" Models: # 模型，包含子网络的配置信息\n",
diff --git "a/notebook/notebook_ch/5.ppocrv2_inference_deployment/PP-OCRv2\351\242\204\346\265\213\351\203\250\347\275\262\345\256\236\346\210\230.ipynb" "b/notebook/notebook_ch/5.ppocrv2_inference_deployment/PP-OCRv2\351\242\204\346\265\213\351\203\250\347\275\262\345\256\236\346\210\230.ipynb"
index c65627acc8d179a5459e3ceb1afa83325383c998..3b8550d339b571f9bc2a4fdabb57b7c29830b9ba 100644
--- "a/notebook/notebook_ch/5.ppocrv2_inference_deployment/PP-OCRv2\351\242\204\346\265\213\351\203\250\347\275\262\345\256\236\346\210\230.ipynb"
+++ "b/notebook/notebook_ch/5.ppocrv2_inference_deployment/PP-OCRv2\351\242\204\346\265\213\351\203\250\347\275\262\345\256\236\346\210\230.ipynb"
@@ -1876,11 +1876,11 @@
" rec_res)\n",
" filter_boxes, filter_rec_res = [], []\n",
" # 根据识别得分的阈值对结果进行过滤，如果得分小于阈值，就过滤掉\n",
- " for box, rec_reuslt in zip(dt_boxes, rec_res):\n",
- " text, score = rec_reuslt\n",
+ " for box, rec_result in zip(dt_boxes, rec_res):\n",
+ " text, score = rec_result\n",
" if score >= self.drop_score:\n",
" filter_boxes.append(box)\n",
- " filter_rec_res.append(rec_reuslt)\n",
+ " filter_rec_res.append(rec_result)\n",
" return filter_boxes, filter_rec_res\n",
"\n",
"def sorted_boxes(dt_boxes):\n",
diff --git "a/notebook/notebook_ch/6.document_analysis/\346\226\207\346\241\243\345\210\206\346\236\220\347\220\206\350\256\272.ipynb" "b/notebook/notebook_ch/6.document_analysis/\346\226\207\346\241\243\345\210\206\346\236\220\347\220\206\350\256\272.ipynb"
index ae49cbc9052b15b30b80aecd493dadfcc7ceac77..2ddd4776c1f7526aca1240b170be60a7d376453f 100644
--- "a/notebook/notebook_ch/6.document_analysis/\346\226\207\346\241\243\345\210\206\346\236\220\347\220\206\350\256\272.ipynb"
+++ "b/notebook/notebook_ch/6.document_analysis/\346\226\207\346\241\243\345\210\206\346\236\220\347\220\206\350\256\272.ipynb"
@@ -327,7 +327,7 @@
"
![](\"https://ai-studio-static-online.cdn.bcebos.com/899470ba601349fbbc402a4c83e6cdaee08aaa10b5004977b1f684f346ebe31f\")
\n",
"
图 18: SER,RE任务示例\n",
"\n",
- "一般的KIE方法基于命名实体识别(Named Entity Recognition,NER)[4]来研究，但是这类方法只利用了图像中的文本信息，缺少对视觉和结构信息的使用，因此精度不高。在此基础上，近几年的方法都开始将视觉和结构信息与文本信息融合到一起，按照对多模态信息进行融合时所采用的的原理可以将这些方法分为下面三种：\n",
+ "一般的KIE方法基于命名实体识别(Named Entity Recognition,NER)[4]来研究，但是这类方法只利用了图像中的文本信息，缺少对视觉和结构信息的使用，因此精度不高。在此基础上，近几年的方法都开始将视觉和结构信息与文本信息融合到一起，按照对多模态信息进行融合时所采用的原理可以将这些方法分为下面三种：\n",
"\n",
"1. 基于Grid的方法\n",
"1. 基于Token的方法\n",
diff --git a/notebook/notebook_en/2.text_detection/text_detection_theory.ipynb b/notebook/notebook_en/2.text_detection/text_detection_theory.ipynb
index 64a4c669f0598887b6557f66ea72cd1cb9a0773b..5916ebed9eeafcfc7ce146bf802df84ad32bc26e 100644
--- a/notebook/notebook_en/2.text_detection/text_detection_theory.ipynb
+++ b/notebook/notebook_en/2.text_detection/text_detection_theory.ipynb
@@ -136,7 +136,7 @@
"
Figure 11: LOMO frame diagram\n",
"\n",
"\n",
- "Contournet [18] is based on the proposed modeling of text contour points to obtain a curved text detection frame. This method first uses Adaptive-RPN to obtain the proposal features of the text area, and then designs a local orthogonal texture perception LOTM module to learn horizontal and vertical textures. The feature is represented by contour points. Finally, by considering the feature responses in two orthogonal directions at the same time, the Point Re-Scoring algorithm can effectively filter out the prediction of strong unidirectional or weak orthogonal activation, and the final text contour can be used as a A group of high-quality contour points are shown.\n",
+ "Contournet [18] is based on the proposed modeling of text contour points to obtain a curved text detection frame. This method first uses Adaptive-RPN to obtain the proposal features of the text area, and then designs a local orthogonal texture perception LOTM module to learn horizontal and vertical textures. The feature is represented by contour points. Finally, by considering the feature responses in two orthogonal directions at the same time, the Point Re-Scoring algorithm can effectively filter out the prediction of strong unidirectional or weak orthogonal activation, and the final text contour can be used as a group of high-quality contour points are shown.\n",
"
![](\"https://ai-studio-static-online.cdn.bcebos.com/1f59ab5db899412f8c70ba71e8dd31d4ea9480d6511f498ea492c97dd2152384\"\n",)
\n",
"
Figure 12: Contournet frame diagram\n",
diff --git a/notebook/notebook_en/5.ppocrv2_inference_deployment/ppocrv2_inference_deployment_practice.ipynb b/notebook/notebook_en/5.ppocrv2_inference_deployment/ppocrv2_inference_deployment_practice.ipynb
index 61cd4561514c52644648eb7fdd8b2473a0caa8cb..780f948579d188449f1b8d3da5967bb3072a2b49 100644
--- a/notebook/notebook_en/5.ppocrv2_inference_deployment/ppocrv2_inference_deployment_practice.ipynb
+++ b/notebook/notebook_en/5.ppocrv2_inference_deployment/ppocrv2_inference_deployment_practice.ipynb
@@ -1886,11 +1886,11 @@
" rec_res)\n",
" filter_boxes, filter_rec_res = [], []\n",
" #Filter the results according to the threshold of the recognition score, if the score is less than the threshold, filter out\n",
- " for box, rec_reuslt in zip(dt_boxes, rec_res):\n",
- " text, score = rec_reuslt\n",
+ " for box, rec_result in zip(dt_boxes, rec_res):\n",
+ " text, score = rec_result\n",
" if score >= self.drop_score:\n",
" filter_boxes.append(box)\n",
- " filter_rec_res.append(rec_reuslt)\n",
+ " filter_rec_res.append(rec_result)\n",
" return filter_boxes, filter_rec_res\n",
"\n",
"def sorted_boxes(dt_boxes):\n",
diff --git a/paddleocr.py b/paddleocr.py
index cb2c34f69f68d289b317d4737bd23385c77c3d95..417350839ac4d1e512c7396831f89ab4b2d6c724 100644
--- a/paddleocr.py
+++ b/paddleocr.py
@@ -47,16 +47,46 @@ __all__ = [
]
SUPPORT_DET_MODEL = ['DB']
-VERSION = '2.5'
+VERSION = '2.5.0.1'
SUPPORT_REC_MODEL = ['CRNN']
BASE_DIR = os.path.expanduser("~/.paddleocr/")
-DEFAULT_OCR_MODEL_VERSION = 'PP-OCR'
-SUPPORT_OCR_MODEL_VERSION = ['PP-OCR', 'PP-OCRv2']
-DEFAULT_STRUCTURE_MODEL_VERSION = 'STRUCTURE'
-SUPPORT_STRUCTURE_MODEL_VERSION = ['STRUCTURE']
+DEFAULT_OCR_MODEL_VERSION = 'PP-OCRv3'
+SUPPORT_OCR_MODEL_VERSION = ['PP-OCR', 'PP-OCRv2', 'PP-OCRv3']
+DEFAULT_STRUCTURE_MODEL_VERSION = 'PP-STRUCTURE'
+SUPPORT_STRUCTURE_MODEL_VERSION = ['PP-STRUCTURE']
MODEL_URLS = {
'OCR': {
+ 'PP-OCRv3': {
+ 'det': {
+ 'ch': {
+ 'url':
+ 'https://paddleocr.bj.bcebos.com/PP-OCRv3/chinese/ch_PP-OCRv3_det_infer.tar',
+ },
+ 'en': {
+ 'url':
+ 'https://paddleocr.bj.bcebos.com/PP-OCRv3/english/en_PP-OCRv3_det_infer.tar',
+ },
+ },
+ 'rec': {
+ 'ch': {
+ 'url':
+ 'https://paddleocr.bj.bcebos.com/PP-OCRv3/chinese/ch_PP-OCRv3_rec_infer.tar',
+ 'dict_path': './ppocr/utils/ppocr_keys_v1.txt'
+ },
+ 'en': {
+ 'url':
+ 'https://paddleocr.bj.bcebos.com/PP-OCRv3/english/en_PP-OCRv3_rec_infer.tar',
+ 'dict_path': './ppocr/utils/en_dict.txt'
+ },
+ },
+ 'cls': {
+ 'ch': {
+ 'url':
+ 'https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_mobile_v2.0_cls_infer.tar',
+ }
+ },
+ },
'PP-OCRv2': {
'det': {
'ch': {
@@ -72,7 +102,7 @@ MODEL_URLS = {
}
}
},
- DEFAULT_OCR_MODEL_VERSION: {
+ 'PP-OCR': {
'det': {
'ch': {
'url':
@@ -173,7 +203,7 @@ MODEL_URLS = {
}
},
'STRUCTURE': {
- DEFAULT_STRUCTURE_MODEL_VERSION: {
+ 'PP-STRUCTURE': {
'table': {
'en': {
'url':
@@ -198,16 +228,17 @@ def parse_args(mMain=True):
"--ocr_version",
type=str,
choices=SUPPORT_OCR_MODEL_VERSION,
- default='PP-OCRv2',
+ default='PP-OCRv3',
help='OCR Model version, the current model support list is as follows: '
- '1. PP-OCRv2 Support Chinese detection and recognition model. '
- '2. PP-OCR support Chinese detection, recognition and direction classifier and multilingual recognition model.'
+ '1. PP-OCRv3 Support Chinese and English detection and recognition model, and direction classifier model'
+ '2. PP-OCRv2 Support Chinese detection and recognition model. '
+ '3. PP-OCR support Chinese detection, recognition and direction classifier and multilingual recognition model.'
)
parser.add_argument(
"--structure_version",
type=str,
choices=SUPPORT_STRUCTURE_MODEL_VERSION,
- default='STRUCTURE',
+ default='PP-STRUCTURE',
help='Model version, the current model support list is as follows:'
' 1. STRUCTURE Support en table structure model.')
diff --git a/ppocr/data/imaug/__init__.py b/ppocr/data/imaug/__init__.py
index 20aaf48e119d68e6c37ce9246a87701fb149d5e7..548832fb0d116ba2de622bd97562b591d74501d8 100644
--- a/ppocr/data/imaug/__init__.py
+++ b/ppocr/data/imaug/__init__.py
@@ -23,7 +23,7 @@ from .random_crop_data import EastRandomCropData, RandomCropImgMask
from .make_pse_gt import MakePseGt
from .rec_img_aug import RecAug, RecConAug, RecResizeImg, ClsResizeImg, \
- SRNRecResizeImg, NRTRRecResizeImg, SARRecResizeImg, PRENResizeImg, SVTRRecResizeImg
+ SRNRecResizeImg, NRTRRecResizeImg, SARRecResizeImg, PRENResizeImg
from .ssl_img_aug import SSLRotateResize
from .randaugment import RandAugment
from .copy_paste import CopyPaste
diff --git a/ppocr/data/imaug/rec_img_aug.py b/ppocr/data/imaug/rec_img_aug.py
index 2f70b51a3b88422274353046209c6d0d4dc79489..7483dffe5b6d9a0a2204702757fcb49762a1cc7a 100644
--- a/ppocr/data/imaug/rec_img_aug.py
+++ b/ppocr/data/imaug/rec_img_aug.py
@@ -207,25 +207,6 @@ class PRENResizeImg(object):
return data
-class SVTRRecResizeImg(object):
- def __init__(self,
- image_shape,
- infer_mode=False,
- character_dict_path='./ppocr/utils/ppocr_keys_v1.txt',
- padding=True,
- **kwargs):
- self.image_shape = image_shape
- self.infer_mode = infer_mode
- self.character_dict_path = character_dict_path
- self.padding = padding
-
- def __call__(self, data):
- img = data['image']
- norm_img = resize_norm_img_svtr(img, self.image_shape, self.padding)
- data['image'] = norm_img
- return data
-
-
def resize_norm_img_sar(img, image_shape, width_downsample_ratio=0.25):
imgC, imgH, imgW_min, imgW_max = image_shape
h = img.shape[0]
@@ -344,58 +325,6 @@ def resize_norm_img_srn(img, image_shape):
return np.reshape(img_black, (c, row, col)).astype(np.float32)
-def resize_norm_img_svtr(img, image_shape, padding=False):
- imgC, imgH, imgW = image_shape
- h = img.shape[0]
- w = img.shape[1]
- if not padding:
- if h > 2.0 * w:
- image = Image.fromarray(img)
- image1 = image.rotate(90, expand=True)
- image2 = image.rotate(-90, expand=True)
- img1 = np.array(image1)
- img2 = np.array(image2)
- else:
- img1 = copy.deepcopy(img)
- img2 = copy.deepcopy(img)
-
- resized_image = cv2.resize(
- img, (imgW, imgH), interpolation=cv2.INTER_LINEAR)
- resized_image1 = cv2.resize(
- img1, (imgW, imgH), interpolation=cv2.INTER_LINEAR)
- resized_image2 = cv2.resize(
- img2, (imgW, imgH), interpolation=cv2.INTER_LINEAR)
- resized_w = imgW
- else:
- ratio = w / float(h)
- if math.ceil(imgH * ratio) > imgW:
- resized_w = imgW
- else:
- resized_w = int(math.ceil(imgH * ratio))
- resized_image = cv2.resize(img, (resized_w, imgH))
- resized_image = resized_image.astype('float32')
- resized_image1 = resized_image1.astype('float32')
- resized_image2 = resized_image2.astype('float32')
- if image_shape[0] == 1:
- resized_image = resized_image / 255
- resized_image = resized_image[np.newaxis, :]
- else:
- resized_image = resized_image.transpose((2, 0, 1)) / 255
- resized_image1 = resized_image1.transpose((2, 0, 1)) / 255
- resized_image2 = resized_image2.transpose((2, 0, 1)) / 255
- resized_image -= 0.5
- resized_image /= 0.5
- resized_image1 -= 0.5
- resized_image1 /= 0.5
- resized_image2 -= 0.5
- resized_image2 /= 0.5
- padding_im = np.zeros((3, imgC, imgH, imgW), dtype=np.float32)
- padding_im[0, :, :, 0:resized_w] = resized_image
- padding_im[1, :, :, 0:resized_w] = resized_image1
- padding_im[2, :, :, 0:resized_w] = resized_image2
- return padding_im
-
-
def srn_other_inputs(image_shape, num_heads, max_text_length):
imgC, imgH, imgW = image_shape
diff --git a/ppocr/metrics/det_metric.py b/ppocr/metrics/det_metric.py
index c9ec8dd2e9082d7fd00db1086a352a61f0239cb1..dca94c092790b121c46c4f1c0f0355391ef8dba9 100644
--- a/ppocr/metrics/det_metric.py
+++ b/ppocr/metrics/det_metric.py
@@ -64,9 +64,9 @@ class DetMetric(object):
}
"""
- metircs = self.evaluator.combine_results(self.results)
+ metrics = self.evaluator.combine_results(self.results)
self.reset()
- return metircs
+ return metrics
def reset(self):
self.results = [] # clear results
@@ -127,20 +127,20 @@ class DetFCEMetric(object):
'thr 0.9':'precision: 0 recall: 0 hmean: 0',
}
"""
- metircs = {}
+ metrics = {}
hmean = 0
for score_thr in self.results.keys():
- metirc = self.evaluator.combine_results(self.results[score_thr])
- # for key, value in metirc.items():
- # metircs['{}_{}'.format(key, score_thr)] = value
- metirc_str = 'precision:{:.5f} recall:{:.5f} hmean:{:.5f}'.format(
- metirc['precision'], metirc['recall'], metirc['hmean'])
- metircs['thr {}'.format(score_thr)] = metirc_str
- hmean = max(hmean, metirc['hmean'])
- metircs['hmean'] = hmean
+ metric = self.evaluator.combine_results(self.results[score_thr])
+ # for key, value in metric.items():
+ # metrics['{}_{}'.format(key, score_thr)] = value
+ metric_str = 'precision:{:.5f} recall:{:.5f} hmean:{:.5f}'.format(
+ metric['precision'], metric['recall'], metric['hmean'])
+ metrics['thr {}'.format(score_thr)] = metric_str
+ hmean = max(hmean, metric['hmean'])
+ metrics['hmean'] = hmean
self.reset()
- return metircs
+ return metrics
def reset(self):
self.results = {
diff --git a/ppocr/metrics/e2e_metric.py b/ppocr/metrics/e2e_metric.py
index 41b7ac2bad041295ad67a2de3461c109cf76a84a..2f8ba3b222022099501e6bda48266ef51a40e9db 100644
--- a/ppocr/metrics/e2e_metric.py
+++ b/ppocr/metrics/e2e_metric.py
@@ -78,9 +78,9 @@ class E2EMetric(object):
self.results.append(result)
def get_metric(self):
- metircs = combine_results(self.results)
+ metrics = combine_results(self.results)
self.reset()
- return metircs
+ return metrics
def reset(self):
self.results = [] # clear results
diff --git a/ppocr/metrics/kie_metric.py b/ppocr/metrics/kie_metric.py
index f3bce0411d6521b1756892cbd7b4c6fcb7bcfb6c..28ab22b807ffe4347ca95be5a44143539db4c97c 100644
--- a/ppocr/metrics/kie_metric.py
+++ b/ppocr/metrics/kie_metric.py
@@ -61,9 +61,9 @@ class KIEMetric(object):
def get_metric(self):
- metircs = self.combine_results(self.results)
+ metrics = self.combine_results(self.results)
self.reset()
- return metircs
+ return metrics
def reset(self):
self.results = [] # clear results
diff --git a/ppocr/metrics/vqa_token_ser_metric.py b/ppocr/metrics/vqa_token_ser_metric.py
index 92d80d0970dc2eab1d3fb82e2b4cfb8d930a60a0..286d8addaf32ba8e4fa12751c6e270c853775bd4 100644
--- a/ppocr/metrics/vqa_token_ser_metric.py
+++ b/ppocr/metrics/vqa_token_ser_metric.py
@@ -34,13 +34,13 @@ class VQASerTokenMetric(object):
def get_metric(self):
from seqeval.metrics import f1_score, precision_score, recall_score
- metircs = {
+ metrics = {
"precision": precision_score(self.gt_list, self.pred_list),
"recall": recall_score(self.gt_list, self.pred_list),
"hmean": f1_score(self.gt_list, self.pred_list),
}
self.reset()
- return metircs
+ return metrics
def reset(self):
self.pred_list = []
diff --git a/ppocr/modeling/architectures/base_model.py b/ppocr/modeling/architectures/base_model.py
index f5b29f94057d5b1f1fbec27686d5f1d679b15479..c6b50d4886daa9bfd2f863c1d8fd6dbc3d1e42c0 100644
--- a/ppocr/modeling/architectures/base_model.py
+++ b/ppocr/modeling/architectures/base_model.py
@@ -92,6 +92,9 @@ class BaseModel(nn.Layer):
else:
y["head_out"] = x
if self.return_all_feats:
- return y
+ if self.training:
+ return y
+ else:
+ return {"head_out": y["head_out"]}
else:
return x
diff --git a/ppocr/modeling/backbones/rec_svtrnet.py b/ppocr/modeling/backbones/rec_svtrnet.py
index 5ded74378c60e6f08a4adf68671afaa1168737b6..c57bf46345d6e08f23b9258358f77f2285366314 100644
--- a/ppocr/modeling/backbones/rec_svtrnet.py
+++ b/ppocr/modeling/backbones/rec_svtrnet.py
@@ -12,7 +12,6 @@
# See the License for the specific language governing permissions and
# limitations under the License.
-from collections import Callable
from paddle import ParamAttr
from paddle.nn.initializer import KaimingNormal
import numpy as np
@@ -170,17 +169,14 @@ class Attention(nn.Layer):
self.N = H * W
self.C = dim
if mixer == 'Local' and HW is not None:
-
hk = local_k[0]
wk = local_k[1]
- mask = np.ones([H * W, H * W])
- for h in range(H):
- for w in range(W):
- for kh in range(-(hk // 2), (hk // 2) + 1):
- for kw in range(-(wk // 2), (wk // 2) + 1):
- if H > (h + kh) >= 0 and W > (w + kw) >= 0:
- mask[h * W + w][(h + kh) * W + (w + kw)] = 0
- mask_paddle = paddle.to_tensor(mask, dtype='float32')
+ mask = paddle.ones([H * W, H + hk - 1, W + wk - 1], dtype='float32')
+ for h in range(0, H):
+ for w in range(0, W):
+ mask[h * W + w, h:h + hk, w:w + wk] = 0.
+ mask_paddle = mask[:, hk // 2:H + hk // 2, wk // 2:W + wk //
+ 2].flatten(1)
mask_inf = paddle.full([H * W, H * W], '-inf', dtype='float32')
mask = paddle.where(mask_paddle < 1, mask_paddle, mask_inf)
self.mask = mask.unsqueeze([0, 1])
@@ -228,11 +224,8 @@ class Block(nn.Layer):
super().__init__()
if isinstance(norm_layer, str):
self.norm1 = eval(norm_layer)(dim, epsilon=epsilon)
- elif isinstance(norm_layer, Callable):
- self.norm1 = norm_layer(dim)
else:
- raise TypeError(
- "The norm_layer must be str or paddle.nn.layer.Layer class")
+ self.norm1 = norm_layer(dim)
if mixer == 'Global' or mixer == 'Local':
self.mixer = Attention(
dim,
@@ -250,15 +243,11 @@ class Block(nn.Layer):
else:
raise TypeError("The mixer must be one of [Global, Local, Conv]")
- # NOTE: drop path for stochastic depth, we shall see if this is better than dropout here
self.drop_path = DropPath(drop_path) if drop_path > 0. else Identity()
if isinstance(norm_layer, str):
self.norm2 = eval(norm_layer)(dim, epsilon=epsilon)
- elif isinstance(norm_layer, Callable):
- self.norm2 = norm_layer(dim)
else:
- raise TypeError(
- "The norm_layer must be str or paddle.nn.layer.Layer class")
+ self.norm2 = norm_layer(dim)
mlp_hidden_dim = int(dim * mlp_ratio)
self.mlp_ratio = mlp_ratio
self.mlp = Mlp(in_features=dim,
@@ -330,8 +319,6 @@ class PatchEmbed(nn.Layer):
act=nn.GELU,
bias_attr=None),
ConvBNLayer(
- embed_dim // 2,
- embed_dim,
in_channels=embed_dim // 2,
out_channels=embed_dim,
kernel_size=3,
diff --git a/ppocr/modeling/transforms/stn.py b/ppocr/modeling/transforms/stn.py
index 1b15d5b8a7b7a1b1ab686d20acea750437463939..6f2bdda050f217d8253740001901fbff4065782a 100644
--- a/ppocr/modeling/transforms/stn.py
+++ b/ppocr/modeling/transforms/stn.py
@@ -128,8 +128,6 @@ class STN_ON(nn.Layer):
self.out_channels = in_channels
def forward(self, image):
- if len(image.shape)==5:
- image = image.reshape([0, image.shape[-3], image.shape[-2], image.shape[-1]])
stn_input = paddle.nn.functional.interpolate(
image, self.tps_inputsize, mode="bilinear", align_corners=True)
stn_img_feat, ctrl_points = self.stn_head(stn_input)
diff --git a/ppocr/optimizer/optimizer.py b/ppocr/optimizer/optimizer.py
index c450a3a3684eb44cdc758a2b27783b5a81945c38..dd8544e2e7d39be33a9096cad16c4d58eb58bcad 100644
--- a/ppocr/optimizer/optimizer.py
+++ b/ppocr/optimizer/optimizer.py
@@ -43,12 +43,15 @@ class Momentum(object):
self.grad_clip = grad_clip
def __call__(self, model):
+ train_params = [
+ param for param in model.parameters() if param.trainable is True
+ ]
opt = optim.Momentum(
learning_rate=self.learning_rate,
momentum=self.momentum,
weight_decay=self.weight_decay,
grad_clip=self.grad_clip,
- parameters=model.parameters())
+ parameters=train_params)
return opt
@@ -76,6 +79,9 @@ class Adam(object):
self.lazy_mode = lazy_mode
def __call__(self, model):
+ train_params = [
+ param for param in model.parameters() if param.trainable is True
+ ]
opt = optim.Adam(
learning_rate=self.learning_rate,
beta1=self.beta1,
@@ -85,7 +91,7 @@ class Adam(object):
grad_clip=self.grad_clip,
name=self.name,
lazy_mode=self.lazy_mode,
- parameters=model.parameters())
+ parameters=train_params)
return opt
@@ -118,6 +124,9 @@ class RMSProp(object):
self.grad_clip = grad_clip
def __call__(self, model):
+ train_params = [
+ param for param in model.parameters() if param.trainable is True
+ ]
opt = optim.RMSProp(
learning_rate=self.learning_rate,
momentum=self.momentum,
@@ -125,7 +134,7 @@ class RMSProp(object):
epsilon=self.epsilon,
weight_decay=self.weight_decay,
grad_clip=self.grad_clip,
- parameters=model.parameters())
+ parameters=train_params)
return opt
@@ -149,6 +158,9 @@ class Adadelta(object):
self.name = name
def __call__(self, model):
+ train_params = [
+ param for param in model.parameters() if param.trainable is True
+ ]
opt = optim.Adadelta(
learning_rate=self.learning_rate,
epsilon=self.epsilon,
@@ -156,7 +168,7 @@ class Adadelta(object):
weight_decay=self.weight_decay,
grad_clip=self.grad_clip,
name=self.name,
- parameters=model.parameters())
+ parameters=train_params)
return opt
@@ -190,17 +202,20 @@ class AdamW(object):
self.one_dim_param_no_weight_decay = one_dim_param_no_weight_decay
def __call__(self, model):
- parameters = model.parameters()
+ parameters = [
+ param for param in model.parameters() if param.trainable is True
+ ]
self.no_weight_decay_param_name_list = [
- p.name for n, p in model.named_parameters() if any(nd in n for nd in self.no_weight_decay_name_list)
+ p.name for n, p in model.named_parameters()
+ if any(nd in n for nd in self.no_weight_decay_name_list)
]
if self.one_dim_param_no_weight_decay:
self.no_weight_decay_param_name_list += [
- p.name for n, p in model.named_parameters() if len(p.shape) == 1
+ p.name for n, p in model.named_parameters() if len(p.shape) == 1
]
-
+
opt = optim.AdamW(
learning_rate=self.learning_rate,
beta1=self.beta1,
@@ -216,4 +231,4 @@ class AdamW(object):
return opt
def _apply_decay_param_fun(self, name):
- return name not in self.no_weight_decay_param_name_list
\ No newline at end of file
+ return name not in self.no_weight_decay_param_name_list
diff --git a/ppocr/postprocess/__init__.py b/ppocr/postprocess/__init__.py
index 390f6f4560f9814a3af757a4fd16c55fe93d01f9..f50b5f1c5f8e617066bb47636c8f4d2b171b6ecb 100644
--- a/ppocr/postprocess/__init__.py
+++ b/ppocr/postprocess/__init__.py
@@ -27,7 +27,7 @@ from .sast_postprocess import SASTPostProcess
from .fce_postprocess import FCEPostProcess
from .rec_postprocess import CTCLabelDecode, AttnLabelDecode, SRNLabelDecode, \
DistillationCTCLabelDecode, TableLabelDecode, NRTRLabelDecode, SARLabelDecode, \
- SEEDLabelDecode, PRENLabelDecode, SVTRLabelDecode
+ SEEDLabelDecode, PRENLabelDecode
from .cls_postprocess import ClsPostProcess
from .pg_postprocess import PGPostProcess
from .vqa_token_ser_layoutlm_postprocess import VQASerTokenLayoutLMPostProcess
@@ -42,7 +42,7 @@ def build_post_process(config, global_config=None):
'DistillationDBPostProcess', 'NRTRLabelDecode', 'SARLabelDecode',
'SEEDLabelDecode', 'VQASerTokenLayoutLMPostProcess',
'VQAReTokenLayoutLMPostProcess', 'PRENLabelDecode',
- 'DistillationSARLabelDecode', 'SVTRLabelDecode'
+ 'DistillationSARLabelDecode'
]
if config['name'] == 'PSEPostProcess':
diff --git a/ppocr/postprocess/rec_postprocess.py b/ppocr/postprocess/rec_postprocess.py
index 50f11f899fb4dd49da75199095772a92cc4a8d7b..bf0fd890bf25949361665d212bf8e1a657054e5b 100644
--- a/ppocr/postprocess/rec_postprocess.py
+++ b/ppocr/postprocess/rec_postprocess.py
@@ -752,40 +752,3 @@ class PRENLabelDecode(BaseRecLabelDecode):
return text
label = self.decode(label)
return text, label
-
-
-class SVTRLabelDecode(BaseRecLabelDecode):
- """ Convert between text-label and text-index """
-
- def __init__(self, character_dict_path=None, use_space_char=False,
- **kwargs):
- super(SVTRLabelDecode, self).__init__(character_dict_path,
- use_space_char)
-
- def __call__(self, preds, label=None, *args, **kwargs):
- if isinstance(preds, tuple):
- preds = preds[-1]
- if isinstance(preds, paddle.Tensor):
- preds = preds.numpy()
- preds_idx = preds.argmax(axis=-1)
- preds_prob = preds.max(axis=-1)
-
- text = self.decode(preds_idx, preds_prob, is_remove_duplicate=True)
- return_text = []
- for i in range(0, len(text), 3):
- text0 = text[i]
- text1 = text[i + 1]
- text2 = text[i + 2]
-
- text_pred = [text0[0], text1[0], text2[0]]
- text_prob = [text0[1], text1[1], text2[1]]
- id_max = text_prob.index(max(text_prob))
- return_text.append((text_pred[id_max], text_prob[id_max]))
- if label is None:
- return return_text
- label = self.decode(label)
- return return_text, label
-
- def add_special_char(self, dict_character):
- dict_character = ['blank'] + dict_character
- return dict_character
\ No newline at end of file
diff --git a/ppocr/utils/utility.py b/ppocr/utils/utility.py
index dc2a6e7405e4d01ddffabf4a23c98bcf3ea3c205..4a25ff8b2fa182faaf4f4ce8909c9ec2e9b55ccc 100755
--- a/ppocr/utils/utility.py
+++ b/ppocr/utils/utility.py
@@ -49,18 +49,23 @@ def get_check_global_params(mode):
return check_params
+def _check_image_file(path):
+ img_end = {'jpg', 'bmp', 'png', 'jpeg', 'rgb', 'tif', 'tiff', 'gif'}
+ return any([path.lower().endswith(e) for e in img_end])
+
+
def get_image_file_list(img_file):
imgs_lists = []
if img_file is None or not os.path.exists(img_file):
raise Exception("not found any img file in {}".format(img_file))
- img_end = {'jpg', 'bmp', 'png', 'jpeg', 'rgb', 'tif', 'tiff', 'gif', 'GIF'}
- if os.path.isfile(img_file) and imghdr.what(img_file) in img_end:
+ img_end = {'jpg', 'bmp', 'png', 'jpeg', 'rgb', 'tif', 'tiff', 'gif'}
+ if os.path.isfile(img_file) and _check_image_file(img_file):
imgs_lists.append(img_file)
elif os.path.isdir(img_file):
for single_file in os.listdir(img_file):
file_path = os.path.join(img_file, single_file)
- if os.path.isfile(file_path) and imghdr.what(file_path) in img_end:
+ if os.path.isfile(file_path) and _check_image_file(file_path):
imgs_lists.append(file_path)
if len(imgs_lists) == 0:
raise Exception("not found any img file in {}".format(img_file))
diff --git a/ppstructure/docs/quickstart.md b/ppstructure/docs/quickstart.md
index 6610035d1442f988ac69763724ce78f6db35ae20..31e59416247b4f0e6b6d82fb13e0d3841a113a5f 100644
--- a/ppstructure/docs/quickstart.md
+++ b/ppstructure/docs/quickstart.md
@@ -194,5 +194,6 @@ dict 里各个字段说明如下
| layout | 前向中是否执行版面分析 | True |
| table | 前向中是否执行表格识别 | True |
| ocr | 对于版面分析中的非表格区域，是否执行ocr。当layout为False时会被自动设置为False | True |
+| structure_version | 表格结构化模型版本，可选 PP-STRUCTURE。PP-STRUCTURE支持表格结构化模型 | PP-STRUCTURE |
大部分参数和PaddleOCR whl包保持一致，见 [whl包文档](../../doc/doc_ch/whl.md)
diff --git a/ppstructure/docs/quickstart_en.md b/ppstructure/docs/quickstart_en.md
index 853436ff07e665fb140a749e8ccbde4392ea5c13..1f78b43ea3334648a37a37745737a6a26e27ece3 100644
--- a/ppstructure/docs/quickstart_en.md
+++ b/ppstructure/docs/quickstart_en.md
@@ -194,5 +194,5 @@ Please refer to: [Documentation Visual Q&A](../vqa/README.md) .
| layout | Whether to perform layout analysis in forward | True |
| table | Whether to perform table recognition in forward | True |
| ocr | Whether to perform ocr for non-table areas in layout analysis. When layout is False, it will be automatically set to False | True |
-
+| structure_version | table structure Model version number, the current model support list is as follows: PP-STRUCTURE support english table structure model | PP-STRUCTURE |
Most of the parameters are consistent with the PaddleOCR whl package, see [whl package documentation](../../doc/doc_en/whl.md)
diff --git a/ppstructure/vqa/tools/eval_with_label_end2end.py b/ppstructure/vqa/tools/eval_with_label_end2end.py
index 3aa439acb269d74165543fac7e0042cfc213f08d..b13ffb568fd9610fee5d5a246c501ed5b90de91a 100644
--- a/ppstructure/vqa/tools/eval_with_label_end2end.py
+++ b/ppstructure/vqa/tools/eval_with_label_end2end.py
@@ -82,7 +82,7 @@ def polygon_iou(poly1, poly2):
except shapely.geos.TopologicalError:
# except Exception as e:
# print(e)
- print('shapely.geos.TopologicalError occured, iou set to 0')
+ print('shapely.geos.TopologicalError occurred, iou set to 0')
iou = 0
return iou
diff --git a/test_tipc/docs/jeston_test_train_inference_python.md b/test_tipc/docs/jeston_test_train_inference_python.md
index d96505985ea8a291b3579acb2aaee1b3d66c1baa..9e9d15fb674ca04558b1f8cb616dc4e44934dbb9 100644
--- a/test_tipc/docs/jeston_test_train_inference_python.md
+++ b/test_tipc/docs/jeston_test_train_inference_python.md
@@ -1,6 +1,6 @@
-# Jeston端基础训练预测功能测试
+# Jetson端基础训练预测功能测试
-Jeston端基础训练预测功能测试的主程序为`test_inference_inference.sh`，由于Jeston端CPU较差，Jeston只需要测试TIPC关于GPU和TensorRT预测推理的部分即可。
+Jetson端基础训练预测功能测试的主程序为`test_inference_inference.sh`，由于Jetson端CPU较差，Jetson只需要测试TIPC关于GPU和TensorRT预测推理的部分即可。
## 1. 测试结论汇总
@@ -42,7 +42,7 @@ Jeston端基础训练预测功能测试的主程序为`test_inference_inference.
先运行`prepare.sh`准备数据和模型，然后运行`test_inference_inference.sh`进行测试，最终在```test_tipc/output```目录下生成`python_infer_*.log`格式的日志文件。
-`test_inference_inference.sh`仅有一个模式`whole_infer`，在Jeston端，仅需要测试预测推理的模式即可：
+`test_inference_inference.sh`仅有一个模式`whole_infer`，在Jetson端，仅需要测试预测推理的模式即可：
```
- 模式3：whole_infer，不训练，全量数据预测，走通开源模型评估、动转静，检查inference model预测时间和精度;
@@ -51,7 +51,7 @@ bash test_tipc/prepare.sh ./test_tipc/configs/ch_ppocr_mobile_v2.0_det/model_lin
# 用法1:
bash test_tipc/test_inference_inference.sh ./test_tipc/configs/ch_ppocr_mobile_v2.0_det/model_linux_gpu_normal_normal_infer_python_jetson.txt 'whole_infer'
# 用法2: 指定GPU卡预测，第三个传入参数为GPU卡号
-bash test_tipc/test_inference_jeston.sh ./test_tipc/configs/ch_ppocr_mobile_v2.0_det/model_linux_gpu_normal_normal_infer_python_jetson.txt 'whole_infer' '1'
+bash test_tipc/test_inference_jetson.sh ./test_tipc/configs/ch_ppocr_mobile_v2.0_det/model_linux_gpu_normal_normal_infer_python_jetson.txt 'whole_infer' '1'
```
运行相应指令后，在`test_tipc/output`文件夹下自动会保存运行日志。如`whole_infer`模式下，会运行训练+inference的链条，因此，在`test_tipc/output`文件夹有以下文件：
diff --git a/tools/end2end/eval_end2end.py b/tools/end2end/eval_end2end.py
index 6e7573ca472503e5d2216723c056ddf42c77e0aa..dd37940845b60d55291b40b8cd50d6afd5b91f94 100644
--- a/tools/end2end/eval_end2end.py
+++ b/tools/end2end/eval_end2end.py
@@ -59,7 +59,7 @@ def polygon_iou(poly1, poly2):
except shapely.geos.TopologicalError:
# except Exception as e:
# print(e)
- print('shapely.geos.TopologicalError occured, iou set to 0')
+ print('shapely.geos.TopologicalError occurred, iou set to 0')
iou = 0
return iou
diff --git a/tools/eval.py b/tools/eval.py
index 7fd4fa7ada7b1550bcca8766f5acb9b4d4ed2049..cab28334396c54f1526f830044de0772b5402a11 100755
--- a/tools/eval.py
+++ b/tools/eval.py
@@ -20,8 +20,8 @@ import os
import sys
__dir__ = os.path.dirname(os.path.abspath(__file__))
-sys.path.append(__dir__)
-sys.path.append(os.path.abspath(os.path.join(__dir__, '..')))
+sys.path.insert(0, __dir__)
+sys.path.insert(0, os.path.abspath(os.path.join(__dir__, '..')))
from ppocr.data import build_dataloader
from ppocr.modeling.architectures import build_model
diff --git a/tools/export_model.py b/tools/export_model.py
index 1f9f29e396fe4960914ae802769b65d20c103bd3..e971f6cb20025d529d0387d287ec87a76abbdbe7 100755
--- a/tools/export_model.py
+++ b/tools/export_model.py
@@ -133,7 +133,7 @@ def main():
else:
config["Architecture"]["Models"][key]["Head"][
"out_channels"] = char_num
- # just one final tensor needs to to exported for inference
+ # just one final tensor needs to exported for inference
config["Architecture"]["Models"][key][
"return_all_feats"] = False
elif config['Architecture']['Head'][
diff --git a/tools/infer/predict_system.py b/tools/infer/predict_system.py
index 63b635c1114d079478e34682d7c94c433bd21ffa..534f08fbcb3b90eea6a371d9f0ad128e276874c4 100755
--- a/tools/infer/predict_system.py
+++ b/tools/infer/predict_system.py
@@ -195,7 +195,7 @@ def main(args):
text_sys.text_detector.autolog.report()
text_sys.text_recognizer.autolog.report()
- with open(os.path.join(draw_img_save_dir, "system_results.txt"), 'w') as f:
+ with open(os.path.join(draw_img_save_dir, "system_results.txt"), 'w', encoding='utf-8') as f:
f.writelines(save_results)
diff --git a/tools/infer/utility.py b/tools/infer/utility.py
index c92e8e152a9ee4d86d269aec7ff5645f23cad443..ce4e2d92c2851fc7c7ae2d2a371c755c82dc97e5 100644
--- a/tools/infer/utility.py
+++ b/tools/infer/utility.py
@@ -193,7 +193,7 @@ def create_predictor(args, mode, logger):
gpu_id = get_infer_gpuid()
if gpu_id is None:
logger.warning(
- "GPU is not found in current device by nvidia-smi. Please check your device or ignore it if run on jeston."
+ "GPU is not found in current device by nvidia-smi. Please check your device or ignore it if run on jetson."
)
config.enable_use_gpu(args.gpu_mem, 0)
if args.use_tensorrt:
diff --git a/tools/infer_e2e.py b/tools/infer_e2e.py
index f3d5712fdda21faf04b95b7a2d5d3092af1b5011..d3e6b28fca0a3ff32ea940747712d6c71aa290fd 100755
--- a/tools/infer_e2e.py
+++ b/tools/infer_e2e.py
@@ -104,7 +104,7 @@ def main():
preds = model(images)
post_result = post_process_class(preds, shape_list)
points, strs = post_result['points'], post_result['texts']
- # write resule
+ # write result
dt_boxes_json = []
for poly, str in zip(points, strs):
tmp_json = {"transcription": str}
diff --git a/tools/infer_rec.py b/tools/infer_rec.py
index 63d410b627f3868191c9299f9bc99e7fcab69d35..193e24a4de12392130d16b86a3407db74602e1f4 100755
--- a/tools/infer_rec.py
+++ b/tools/infer_rec.py
@@ -150,7 +150,7 @@ def main():
"label": post_result[key][0][0],
"score": float(post_result[key][0][1]),
}
- info = json.dumps(rec_info)
+ info = json.dumps(rec_info, ensure_ascii=False)
else:
if len(post_result[0]) >= 2:
info = post_result[0][0] + "\t" + str(post_result[0][1])
diff --git a/tools/infer_vqa_token_ser_re.py b/tools/infer_vqa_token_ser_re.py
index 2c7cb5e4251819c53e56ba74df530181836299a2..6210f7f3c24227c9d366b08ce93ccfe4df849ce1 100755
--- a/tools/infer_vqa_token_ser_re.py
+++ b/tools/infer_vqa_token_ser_re.py
@@ -193,7 +193,7 @@ if __name__ == '__main__':
result = result[0]
fout.write(img_path + "\t" + json.dumps(
{
- "ser_resule": result,
+ "ser_result": result,
}, ensure_ascii=False) + "\n")
img_res = draw_re_results(img_path, result)
cv2.imwrite(save_img_path, img_res)
+
+
+
+
+由于 MKLDNN 加速库支持的模型结构有限，SVTR 在CPU+MKLDNN上相比PP-OCRv2慢了10倍。
+
+PP-OCRv3 期望在提升模型精度的同时，不带来额外的推理耗时。通过分析发现，SVTR_Tiny结构的主要耗时模块为Mixing Block，因此我们对 SVTR_Tiny 的结构进行了一系列优化（详细速度数据请参考下方消融实验表格）:
+
+1. 将SVTR网络前半部分替换为PP-LCNet的前三个stage，保留4个 Global Mixing Block ，精度为76%，加速69%，网络结构如下所示：
+
+2. 将4个 Global Attenntion Block 减小到2个，精度为72.9%，加速69%，网络结构如下所示：
+
+3. 实验发现 Global Attention 的预测速度与输入其特征的shape有关，因此后移Global Mixing Block的位置到池化层之后，精度下降为71.9%，速度超越 CNN-base 的PP-OCRv2 22%，网络结构如下所示：
+
+
+为了提升模型精度同时不引入额外推理成本，PP-OCRv3参考GTC策略，使用Attention监督CTC训练，预测时完全去除Attention模块，在推理阶段不增加任何耗时, 精度提升3.8%，训练流程如下所示：
+
+
+在训练策略方面，PP-OCRv3参考 [SSL](https://github.com/ku21fan/STR-Fewer-Labels) 设计了文本方向任务，训练了适用于文本识别的预训练模型，加速模型收敛过程，精度提升了0.6%; 使用UDML蒸馏策略，进一步提升精度1.5%，训练流程所示：
+
+
+
+
+数据增强方面：
+
+1. 基于 [ConCLR](https://www.cse.cuhk.edu.hk/~byu/papers/C139-AAAI2022-ConCLR.pdf) 中的ConAug方法，设计了 RecConAug 数据增强方法，增强数据多样性，精度提升0.5%，增强可视化效果如下所示：
+
+
+2. 使用训练好的 SVTR_large 预测 120W 的 lsvt 无标注数据，取出其中得分大于0.95的数据，共得到81W识别数据加入到PP-OCRv3的训练数据中，精度提升1%。
+
+总体来讲PP-OCRv3识别从网络结构、训练策略、数据增强三个方向做了进一步优化:
+
+- 网络结构上：考虑[SVTR](https://arxiv.org/abs/2205.00159) 在中英文效果上的优越性，采用SVTR_Tiny作为base，选取Global Mixing Block和卷积组合提取特征，并将Global Mixing Block位置后移进行加速; 参考 [GTC](https://arxiv.org/pdf/2002.01276.pdf) 策略，使用注意力机制模块指导CTC训练，定位和识别字符，提升不规则文本的识别精度。
+- 训练策略上：参考 [SSL](https://github.com/ku21fan/STR-Fewer-Labels) 设计了方向分类前序任务，获取更优预训练模型，加速模型收敛过程，提升精度; 使用UDML蒸馏策略、监督attention、ctc两个分支得到更优模型。
+- 数据增强上：基于 [ConCLR](https://www.cse.cuhk.edu.hk/~byu/papers/C139-AAAI2022-ConCLR.pdf) 中的ConAug方法，改进得到 RecConAug 数据增广方法，支持随机结合任意多张图片，提升训练数据的上下文信息丰富度，增强模型鲁棒性；使用 SVTR_large 预测无标签数据，向训练集中补充81w高质量真实数据。
+
+基于上述策略，PP-OCRv3识别模型相比PP-OCRv2，在速度可比的情况下，精度进一步提升4.5%。 具体消融实验如下所示：
+
+实验细节：
+
+| id | 策略 | 模型大小 | 精度 | 速度（cpu + mkldnn)|
+|-----|-----|--------|----| --- |
+| 01 | PP-OCRv2 | 8M | 69.3% | 8.54ms |
+| 02 | SVTR_Tiny | 21M | 80.1% | 97ms |
+| 03 | LCNet_SVTR_G4 | 9.2M | 76% | 30ms |
+| 04 | LCNet_SVTR_G2 | 13M | 72.98% | 9.37ms |
+| 05 | PP-OCRv3 | 12M | 71.9% | 6.6ms |
+| 06 | + large input_shape | 12M | 73.98% | 7.6ms |
+| 06 | + GTC | 12M | 75.8% | 7.6ms |
+| 07 | + RecConAug | 12M | 76.3% | 7.6ms |
+| 08 | + SSL pretrain | 12M | 76.9% | 7.6ms |
+| 09 | + UDML | 12M | 78.4% | 7.6ms |
+| 10 | + unlabeled data | 12M | 79.4% | 7.6ms |
+
+注： 测试速度时，实验01-05输入图片尺寸均为(3,32,320)，06-10输入图片尺寸均为(3,48,320)
+
+
+## 4. 端到端评估
diff --git a/doc/doc_ch/add_new_algorithm.md b/doc/doc_ch/add_new_algorithm.md
index 79c29249dd7dd0b25ffa7625d11ed2378bfafec4..bb97e00aa66771e99feb799dc4945f3d13b5b247 100644
--- a/doc/doc_ch/add_new_algorithm.md
+++ b/doc/doc_ch/add_new_algorithm.md
@@ -246,7 +246,7 @@ class MyMetric(object):
def get_metric(self):
"""
- return metircs {
+ return metrics {
'acc': 0,
'norm_edit_dis': 0,
}
diff --git a/doc/doc_ch/algorithm_det_east.md b/doc/doc_ch/algorithm_det_east.md
new file mode 100644
index 0000000000000000000000000000000000000000..b89018e3468aa7772af69da469e81c16e9d43dc9
--- /dev/null
+++ b/doc/doc_ch/algorithm_det_east.md
@@ -0,0 +1,96 @@
+# EAST
+
+- [1. 算法简介](#1)
+- [2. 环境配置](#2)
+- [3. 模型训练、评估、预测](#3)
+ - [3.1 训练](#3-1)
+ - [3.2 评估](#3-2)
+ - [3.3 预测](#3-3)
+- [4. 推理部署](#4)
+ - [4.1 Python推理](#4-1)
+ - [4.2 C++推理](#4-2)
+ - [4.3 Serving服务化部署](#4-3)
+ - [4.4 更多推理部署](#4-4)
+- [5. FAQ](#5)
+
+
+## 1. 算法简介
+
+论文信息：
+> [EAST: An Efficient and Accurate Scene Text Detector](https://arxiv.org/abs/1704.03155)
+> Xinyu Zhou, Cong Yao, He Wen, Yuzhi Wang, Shuchang Zhou, Weiran He, Jiajun Liang
+> CVPR, 2017
+
+
+在ICDAR2015文本检测公开数据集上，算法复现效果如下：
+
+|模型|骨干网络|配置文件|precision|recall|Hmean|下载链接|
+| --- | --- | --- | --- | --- | --- | --- |
+|EAST|ResNet50_vd|88.71%| 81.36%| 84.88%| [训练模型](https://paddleocr.bj.bcebos.com/dygraph_v2.0/en/det_r50_vd_east_v2.0_train.tar)|
+|EAST| MobileNetV3| 78.2%| 79.1%| 78.65%| [训练模型](https://paddleocr.bj.bcebos.com/dygraph_v2.0/en/det_r50_vd_east_v2.0_train.tar)|
+
+
+
+## 2. 环境配置
+请先参考[《运行环境准备》](./environment.md)配置PaddleOCR运行环境，参考[《项目克隆》](./clone.md)克隆项目代码。
+
+
+
+## 3. 模型训练、评估、预测
+
+上表中的EAST训练模型使用ICDAR2015文本检测公开数据集训练得到，数据集下载可参考 [ocr_datasets](./dataset/ocr_datasets.md)。
+
+数据下载完成后，请参考[文本检测训练教程](./detection.md)进行训练。PaddleOCR对代码进行了模块化，训练不同的检测模型只需要**更换配置文件**即可。
+
+
+
+## 4. 推理部署
+
+
+### 4.1 Python推理
+
+首先将EAST文本检测训练过程中保存的模型，转换成inference model。以基于Resnet50_vd骨干网络，在ICDAR2015英文数据集训练的模型为例（[训练模型](https://paddleocr.bj.bcebos.com/dygraph_v2.0/en/det_r50_vd_east_v2.0_train.tar))，可以使用如下命令进行转换：
+
+```shell
+python3 tools/export_model.py -c configs/det/det_r50_vd_east.yml -o Global.pretrained_model=./det_r50_vd_east_v2.0_train/best_accuracy Global.save_inference_dir=./inference/det_r50_east/
+```
+
+EAST文本检测模型推理，需要设置参数--det_algorithm="EAST"，执行预测：
+```shell
+python3 tools/infer/predict_det.py --image_dir="./doc/imgs_en/img_10.jpg" --det_model_dir="./inference/det_r50_east/" --det_algorithm="EAST"
+```
+
+可视化文本检测结果默认保存到`./inference_results`文件夹里面，结果文件的名称前缀为'det_res'。
+
+
+
+
+### 4.2 C++推理
+
+由于后处理暂未使用CPP编写，EAST文本检测模型暂不支持CPP推理。
+
+
+### 4.3 Serving服务化部署
+
+暂未支持
+
+
+### 4.4 更多推理部署
+
+暂未支持
+
+
+## 5. FAQ
+
+
+## 引用
+
+```bibtex
+@inproceedings{zhou2017east,
+ title={East: an efficient and accurate scene text detector},
+ author={Zhou, Xinyu and Yao, Cong and Wen, He and Wang, Yuzhi and Zhou, Shuchang and He, Weiran and Liang, Jiajun},
+ booktitle={Proceedings of the IEEE conference on Computer Vision and Pattern Recognition},
+ pages={5551--5560},
+ year={2017}
+}
+```
diff --git a/doc/doc_ch/algorithm_det_sast.md b/doc/doc_ch/algorithm_det_sast.md
new file mode 100644
index 0000000000000000000000000000000000000000..038d73fc15f3203bbcc17997c1a8e1c208f80ba8
--- /dev/null
+++ b/doc/doc_ch/algorithm_det_sast.md
@@ -0,0 +1,115 @@
+# SAST
+
+- [1. 算法简介](#1)
+- [2. 环境配置](#2)
+- [3. 模型训练、评估、预测](#3)
+ - [3.1 训练](#3-1)
+ - [3.2 评估](#3-2)
+ - [3.3 预测](#3-3)
+- [4. 推理部署](#4)
+ - [4.1 Python推理](#4-1)
+ - [4.2 C++推理](#4-2)
+ - [4.3 Serving服务化部署](#4-3)
+ - [4.4 更多推理部署](#4-4)
+- [5. FAQ](#5)
+
+
+## 1. 算法简介
+
+论文信息：
+> [A Single-Shot Arbitrarily-Shaped Text Detector based on Context Attended Multi-Task Learning](https://arxiv.org/abs/1908.05498)
+> Wang, Pengfei and Zhang, Chengquan and Qi, Fei and Huang, Zuming and En, Mengyi and Han, Junyu and Liu, Jingtuo and Ding, Errui and Shi, Guangming
+> ACM MM, 2019
+
+在ICDAR2015文本检测公开数据集上，算法复现效果如下：
+
+|模型|骨干网络|配置文件|precision|recall|Hmean|下载链接|
+| --- | --- | --- | --- | --- | --- | --- |
+|SAST|ResNet50_vd|[configs/det/det_r50_vd_sast_icdar15.yml](../../configs/det/det_r50_vd_sast_icdar15.yml)|91.39%|83.77%|87.42%|[训练模型](https://paddleocr.bj.bcebos.com/dygraph_v2.0/en/det_r50_vd_sast_icdar15_v2.0_train.tar)|
+
+
+在Total-text文本检测公开数据集上，算法复现效果如下：
+
+|模型|骨干网络|配置文件|precision|recall|Hmean|下载链接|
+| --- | --- | --- | --- | --- | --- | --- |
+|SAST|ResNet50_vd|[configs/det/det_r50_vd_sast_totaltext.yml](../../configs/det/det_r50_vd_sast_totaltext.yml)|89.63%|78.44%|83.66%|[训练模型](https://paddleocr.bj.bcebos.com/dygraph_v2.0/en/det_r50_vd_sast_totaltext_v2.0_train.tar)|
+
+
+
+## 2. 环境配置
+请先参考[《运行环境准备》](./environment.md)配置PaddleOCR运行环境，参考[《项目克隆》](./clone.md)克隆项目代码。
+
+
+
+## 3. 模型训练、评估、预测
+
+请参考[文本检测训练教程](./detection.md)。PaddleOCR对代码进行了模块化，训练不同的检测模型只需要**更换配置文件**即可。
+
+
+
+## 4. 推理部署
+
+
+### 4.1 Python推理
+#### (1). 四边形文本检测模型（ICDAR2015）
+首先将SAST文本检测训练过程中保存的模型，转换成inference model。以基于Resnet50_vd骨干网络，在ICDAR2015英文数据集训练的模型为例([模型下载地址](https://paddleocr.bj.bcebos.com/dygraph_v2.0/en/det_r50_vd_sast_icdar15_v2.0_train.tar))，可以使用如下命令进行转换：
+```
+python3 tools/export_model.py -c configs/det/det_r50_vd_sast_icdar15.yml -o Global.pretrained_model=./det_r50_vd_sast_icdar15_v2.0_train/best_accuracy Global.save_inference_dir=./inference/det_sast_ic15
+
+```
+**SAST文本检测模型推理，需要设置参数`--det_algorithm="SAST"`**，可以执行如下命令：
+```
+python3 tools/infer/predict_det.py --det_algorithm="SAST" --image_dir="./doc/imgs_en/img_10.jpg" --det_model_dir="./inference/det_sast_ic15/"
+```
+可视化文本检测结果默认保存到`./inference_results`文件夹里面，结果文件的名称前缀为'det_res'。结果示例如下：
+
+
+
+#### (2). 弯曲文本检测模型（Total-Text）
+首先将SAST文本检测训练过程中保存的模型，转换成inference model。以基于Resnet50_vd骨干网络，在Total-Text英文数据集训练的模型为例（[模型下载地址](https://paddleocr.bj.bcebos.com/dygraph_v2.0/en/det_r50_vd_sast_totaltext_v2.0_train.tar))，可以使用如下命令进行转换：
+
+```
+python3 tools/export_model.py -c configs/det/det_r50_vd_sast_totaltext.yml -o Global.pretrained_model=./det_r50_vd_sast_totaltext_v2.0_train/best_accuracy Global.save_inference_dir=./inference/det_sast_tt
+
+```
+
+SAST文本检测模型推理，需要设置参数`--det_algorithm="SAST"`，同时，还需要增加参数`--det_sast_polygon=True`，可以执行如下命令：
+```
+python3 tools/infer/predict_det.py --det_algorithm="SAST" --image_dir="./doc/imgs_en/img623.jpg" --det_model_dir="./inference/det_sast_tt/" --det_sast_polygon=True
+```
+可视化文本检测结果默认保存到`./inference_results`文件夹里面，结果文件的名称前缀为'det_res'。结果示例如下：
+
+
+
+**注意**：本代码库中，SAST后处理Locality-Aware NMS有python和c++两种版本，c++版速度明显快于python版。由于c++版本nms编译版本问题，只有python3.5环境下会调用c++版nms，其他情况将调用python版nms。
+
+
+### 4.2 C++推理
+
+暂未支持
+
+
+### 4.3 Serving服务化部署
+
+暂未支持
+
+
+### 4.4 更多推理部署
+
+暂未支持
+
+
+## 5. FAQ
+
+
+## 引用
+
+```bibtex
+@inproceedings{wang2019single,
+ title={A Single-Shot Arbitrarily-Shaped Text Detector based on Context Attended Multi-Task Learning},
+ author={Wang, Pengfei and Zhang, Chengquan and Qi, Fei and Huang, Zuming and En, Mengyi and Han, Junyu and Liu, Jingtuo and Ding, Errui and Shi, Guangming},
+ booktitle={Proceedings of the 27th ACM International Conference on Multimedia},
+ pages={1277--1285},
+ year={2019}
+}
+```
diff --git a/doc/doc_ch/algorithm_inference.md b/doc/doc_ch/algorithm_inference.md
index 0222dec85a0be0973b2546cbb6e5347852242093..d8457ea364d9875d9f0796283d42c0b4b10aaa53 100755
--- a/doc/doc_ch/algorithm_inference.md
+++ b/doc/doc_ch/algorithm_inference.md
@@ -296,7 +296,7 @@ Predicts of ./doc/imgs_words_en/word_336.png:('super', 0.9999073)
**注意**：由于上述模型是参考[DTRB](https://arxiv.org/abs/1904.01906)文本识别训练和评估流程，与超轻量级中文识别模型训练有两方面不同：
-- 训练时采用的图像分辨率不同，训练上述模型采用的图像分辨率是[3，32，100]，而中文模型训练时，为了保证长文本的识别效果，训练时采用的图像分辨率是[3, 32, 320]。预测推理程序默认的的形状参数是训练中文采用的图像分辨率，即[3, 32, 320]。因此，这里推理上述英文模型时，需要通过参数rec_image_shape设置识别图像的形状。
+- 训练时采用的图像分辨率不同，训练上述模型采用的图像分辨率是[3，32，100]，而中文模型训练时，为了保证长文本的识别效果，训练时采用的图像分辨率是[3, 32, 320]。预测推理程序默认的形状参数是训练中文采用的图像分辨率，即[3, 32, 320]。因此，这里推理上述英文模型时，需要通过参数rec_image_shape设置识别图像的形状。
- 字符列表，DTRB论文中实验只是针对26个小写英文本母和10个数字进行实验，总共36个字符。所有大小字符都转成了小写字符，不在上面列表的字符都忽略，认为是空格。因此这里没有输入字符字典，而是通过如下命令生成字典.因此在推理时需要设置参数rec_char_dict_path，指定为英文字典"./ppocr/utils/ic15_dict.txt"。
diff --git a/doc/doc_ch/algorithm_rec_crnn.md b/doc/doc_ch/algorithm_rec_crnn.md
new file mode 100644
index 0000000000000000000000000000000000000000..490485ec10311b63032fadad599d764ea67e68f9
--- /dev/null
+++ b/doc/doc_ch/algorithm_rec_crnn.md
@@ -0,0 +1,140 @@
+# CRNN
+
+- [1. 算法简介](#1)
+- [2. 环境配置](#2)
+- [3. 模型训练、评估、预测](#3)
+ - [3.1 训练](#3-1)
+ - [3.2 评估](#3-2)
+ - [3.3 预测](#3-3)
+- [4. 推理部署](#4)
+ - [4.1 Python推理](#4-1)
+ - [4.2 C++推理](#4-2)
+ - [4.3 Serving服务化部署](#4-3)
+ - [4.4 更多推理部署](#4-4)
+- [5. FAQ](#5)
+
+
+## 1. 算法简介
+
+论文信息：
+> [An End-to-End Trainable Neural Network for Image-based Sequence Recognition and Its Application to Scene Text Recognition](https://arxiv.org/abs/1507.05717)
+
+> Baoguang Shi, Xiang Bai, Cong Yao
+
+> IEEE, 2015
+
+参考[DTRB](https://arxiv.org/abs/1904.01906) 文字识别训练和评估流程，使用MJSynth和SynthText两个文字识别数据集训练，在IIIT, SVT, IC03, IC13, IC15, SVTP, CUTE数据集上进行评估，算法效果如下：
+
+|模型|骨干网络|Avg Accuracy|配置文件|下载链接|
+|---|---|---|---|---|
+|CRNN|Resnet34_vd|81.04%|[configs/rec/rec_r34_vd_none_bilstm_ctc.yml](../../configs/rec/rec_r34_vd_none_bilstm_ctc.yml)|[训练模型](https://paddleocr.bj.bcebos.com/dygraph_v2.0/en/rec_r34_vd_none_bilstm_ctc_v2.0_train.tar)|
+|CRNN|MobileNetV3|77.95%|[configs/rec/rec_mv3_none_bilstm_ctc.yml](../../configs/rec/rec_mv3_none_bilstm_ctc.yml)|[训练模型](https://paddleocr.bj.bcebos.com/dygraph_v2.0/en/rec_mv3_none_bilstm_ctc_v2.0_train.tar)|
+
+
+
+## 2. 环境配置
+请先参考[《运行环境准备》](./environment.md)配置PaddleOCR运行环境，参考[《项目克隆》](./clone.md)克隆项目代码。
+
+
+
+## 3. 模型训练、评估、预测
+
+请参考[文本识别训练教程](./recognition.md)。PaddleOCR对代码进行了模块化，训练不同的识别模型只需要**更换配置文件**即可。
+
+- 训练
+
+在完成数据准备后，便可以启动训练，训练命令如下：
+
+```
+#单卡训练（训练周期长，不建议）
+python3 tools/train.py -c configs/rec/rec_r34_vd_none_bilstm_ctc.yml
+
+#多卡训练，通过--gpus参数指定卡号
+python3 -m paddle.distributed.launch --gpus '0,1,2,3' tools/train.py -c rec_r34_vd_none_bilstm_ctc.yml
+
+```
+
+- 评估
+
+```
+# GPU 评估， Global.pretrained_model 为待测权重
+python3 -m paddle.distributed.launch --gpus '0' tools/eval.py -c configs/rec/rec_r34_vd_none_bilstm_ctc.yml -o Global.pretrained_model={path/to/weights}/best_accuracy
+```
+
+- 预测：
+
+```
+# 预测使用的配置文件必须与训练一致
+python3 tools/infer_rec.py -c configs/rec/rec_r34_vd_none_bilstm_ctc.yml -o Global.pretrained_model={path/to/weights}/best_accuracy Global.infer_img=doc/imgs_words/en/word_1.png
+```
+
+
+## 4. 推理部署
+
+
+### 4.1 Python推理
+
+首先将 CRNN 文本识别训练过程中保存的模型，转换成inference model。以基于Resnet34_vd骨干网络，使用MJSynth和SynthText两个英文文本识别合成数据集训练的[模型](https://paddleocr.bj.bcebos.com/dygraph_v2.0/en/rec_r34_vd_none_bilstm_ctc_v2.0_train.tar) 为例，可以使用如下命令进行转换：
+```shell
+python3 tools/export_model.py -c configs/rec/rec_r34_vd_none_bilstm_ctc.yml -o Global.pretrained_model=./rec_r34_vd_none_bilstm_ctc_v2.0_train/best_accuracy Global.save_inference_dir=./inference/rec_crnn
+```
+CRNN 文本识别模型推理，可以执行如下命令：
+
+```shell
+python3 tools/infer/predict_rec.py --image_dir="./doc/imgs_words_en/word_336.png" --rec_model_dir="./inference/rec_crnn/" --rec_image_shape="3, 32, 100" --rec_char_dict_path="./ppocr/utils/ic15_dict.txt"
+```
+
+
+
+执行命令后，上面图像的识别结果如下：
+
+```bash
+Predicts of ./doc/imgs_words_en/word_336.png:('super', 0.9999073)
+```
+
+**注意**：由于上述模型是参考[DTRB](https://arxiv.org/abs/1904.01906)文本识别训练和评估流程，与超轻量级中文识别模型训练有两方面不同：
+
+- 训练时采用的图像分辨率不同，训练上述模型采用的图像分辨率是[3，32，100]，而中文模型训练时，为了保证长文本的识别效果，训练时采用的图像分辨率是[3, 32, 320]。预测推理程序默认的形状参数是训练中文采用的图像分辨率，即[3, 32, 320]。因此，这里推理上述英文模型时，需要通过参数rec_image_shape设置识别图像的形状。
+
+- 字符列表，DTRB论文中实验只是针对26个小写英文本母和10个数字进行实验，总共36个字符。所有大小字符都转成了小写字符，不在上面列表的字符都忽略，认为是空格。因此这里没有输入字符字典，而是通过如下命令生成字典.因此在推理时需要设置参数rec_char_dict_path，指定为英文字典"./ppocr/utils/ic15_dict.txt"。
+
+```
+self.character_str = "0123456789abcdefghijklmnopqrstuvwxyz"
+dict_character = list(self.character_str)
+```
+
+
+
+### 4.2 C++推理
+
+准备好推理模型后，参考[cpp infer](../../deploy/cpp_infer/)教程进行操作即可。
+
+
+### 4.3 Serving服务化部署
+
+准备好推理模型后，参考[pdserving](../../deploy/pdserving/)教程进行Serving服务化部署，包括Python Serving和C++ Serving两种模式。
+
+
+### 4.4 更多推理部署
+
+CRNN模型还支持以下推理部署方式：
+
+- Paddle2ONNX推理：准备好推理模型后，参考[paddle2onnx](../../deploy/paddle2onnx/)教程操作。
+
+
+## 5. FAQ
+
+
+## 引用
+
+```bibtex
+@ARTICLE{7801919,
+ author={Shi, Baoguang and Bai, Xiang and Yao, Cong},
+ journal={IEEE Transactions on Pattern Analysis and Machine Intelligence},
+ title={An End-to-End Trainable Neural Network for Image-Based Sequence Recognition and Its Application to Scene Text Recognition},
+ year={2017},
+ volume={39},
+ number={11},
+ pages={2298-2304},
+ doi={10.1109/TPAMI.2016.2646371}}
+```
diff --git a/doc/doc_ch/algorithm_rec_nrtr.md b/doc/doc_ch/algorithm_rec_nrtr.md
new file mode 100644
index 0000000000000000000000000000000000000000..d3b626d0241a6c36797cba20a58f007e383c7aee
--- /dev/null
+++ b/doc/doc_ch/algorithm_rec_nrtr.md
@@ -0,0 +1,154 @@
+# 场景文本识别算法-NRTR
+
+- [1. 算法简介](#1)
+- [2. 环境配置](#2)
+- [3. 模型训练、评估、预测](#3)
+ - [3.1 训练](#3-1)
+ - [3.2 评估](#3-2)
+ - [3.3 预测](#3-3)
+- [4. 推理部署](#4)
+ - [4.1 Python推理](#4-1)
+ - [4.2 C++推理](#4-2)
+ - [4.3 Serving服务化部署](#4-3)
+ - [4.4 更多推理部署](#4-4)
+- [5. FAQ](#5)
+
+
+## 1. 算法简介
+
+论文信息：
+> [NRTR: A No-Recurrence Sequence-to-Sequence Model For Scene Text Recognition](https://arxiv.org/abs/1806.00926)
+> Fenfen Sheng and Zhineng Chen and Bo Xu
+> ICDAR, 2019
+
+
+
+`NRTR`使用MJSynth和SynthText两个文字识别数据集训练，在IIIT, SVT, IC03, IC13, IC15, SVTP, CUTE数据集上进行评估，算法复现效果如下：
+
+|模型|骨干网络|配置文件|Acc|下载链接|
+| --- | --- | --- | --- | --- |
+|NRTR|MTB|[rec_mtb_nrtr.yml](../../configs/rec/rec_mtb_nrtr.yml)|84.21%|[训练模型](https://paddleocr.bj.bcebos.com/dygraph_v2.0/en/rec_mtb_nrtr_train.tar)|
+
+
+## 2. 环境配置
+请先参考[《运行环境准备》](./environment.md)配置PaddleOCR运行环境，参考[《项目克隆》](./clone.md)克隆项目代码。
+
+
+
+## 3. 模型训练、评估、预测
+
+
+### 3.1 模型训练
+
+请参考[文本识别训练教程](./recognition.md)。PaddleOCR对代码进行了模块化，训练`NRTR`识别模型时需要**更换配置文件**为`NRTR`的[配置文件](../../configs/rec/rec_mtb_nrtr.yml)。
+
+#### 启动训练
+
+
+具体地，在完成数据准备后，便可以启动训练，训练命令如下：
+```shell
+#单卡训练（训练周期长，不建议）
+python3 tools/train.py -c configs/rec/rec_mtb_nrtr.yml
+
+#多卡训练，通过--gpus参数指定卡号
+python3 -m paddle.distributed.launch --gpus '0,1,2,3' tools/train.py -c configs/rec/rec_mtb_nrtr.yml
+```
+
+
+### 3.2 评估
+
+可下载已训练完成的[模型文件](#model)，使用如下命令进行评估：
+
+```shell
+# 注意将pretrained_model的路径设置为本地路径。
+python3 -m paddle.distributed.launch --gpus '0' tools/eval.py -c configs/rec/rec_mtb_nrtr.yml -o Global.pretrained_model=./rec_mtb_nrtr_train/best_accuracy
+```
+
+
+### 3.3 预测
+
+使用如下命令进行单张图片预测：
+```shell
+# 注意将pretrained_model的路径设置为本地路径。
+python3 tools/infer_rec.py -c configs/rec/rec_mtb_nrtr.yml -o Global.infer_img='./doc/imgs_words_en/word_10.png' Global.pretrained_model=./rec_mtb_nrtr_train/best_accuracy
+# 预测文件夹下所有图像时，可修改infer_img为文件夹，如 Global.infer_img='./doc/imgs_words_en/'。
+```
+
+
+
+## 4. 推理部署
+
+
+### 4.1 Python推理
+首先将训练得到best模型，转换成inference model。这里以训练完成的模型为例（[模型下载地址](https://paddleocr.bj.bcebos.com/dygraph_v2.0/en/rec_mtb_nrtr_train.tar) )，可以使用如下命令进行转换：
+
+```shell
+# 注意将pretrained_model的路径设置为本地路径。
+python3 tools/export_model.py -c configs/rec/rec_mtb_nrtr.yml -o Global.pretrained_model=./rec_mtb_nrtr_train/best_accuracy Global.save_inference_dir=./inference/rec_mtb_nrtr/
+```
+**注意：**
+- 如果您是在自己的数据集上训练的模型，并且调整了字典文件，请注意修改配置文件中的`character_dict_path`是否是所需要的字典文件。
+- 如果您修改了训练时的输入大小，请修改`tools/export_model.py`文件中的对应NRTR的`infer_shape`。
+
+转换成功后，在目录下有三个文件：
+```
+/inference/rec_mtb_nrtr/
+ ├── inference.pdiparams # 识别inference模型的参数文件
+ ├── inference.pdiparams.info # 识别inference模型的参数信息，可忽略
+ └── inference.pdmodel # 识别inference模型的program文件
+```
+
+执行如下命令进行模型推理：
+
+```shell
+python3 tools/infer/predict_rec.py --image_dir='./doc/imgs_words_en/word_10.png' --rec_model_dir='./inference/rec_mtb_nrtr/' --rec_algorithm='NRTR' --rec_image_shape='1,32,100' --rec_char_dict_path='./ppocr/utils/EN_symbol_dict.txt'
+# 预测文件夹下所有图像时，可修改image_dir为文件夹，如 --image_dir='./doc/imgs_words_en/'。
+```
+
+
+
+执行命令后，上面图像的预测结果（识别的文本和得分）会打印到屏幕上，示例如下：
+结果如下：
+```shell
+Predicts of ./doc/imgs_words_en/word_10.png:('pain', 0.9265879392623901)
+```
+
+**注意**：
+
+- 训练上述模型采用的图像分辨率是[1，32，100]，需要通过参数`rec_image_shape`设置为您训练时的识别图像形状。
+- 在推理时需要设置参数`rec_char_dict_path`指定字典，如果您修改了字典，请修改该参数为您的字典文件。
+- 如果您修改了预处理方法，需修改`tools/infer/predict_rec.py`中NRTR的预处理为您的预处理方法。
+
+
+
+### 4.2 C++推理部署
+
+由于C++预处理后处理还未支持NRTR，所以暂未支持
+
+
+### 4.3 Serving服务化部署
+
+暂不支持
+
+
+### 4.4 更多推理部署
+
+暂不支持
+
+
+## 5. FAQ
+
+1. `NRTR`论文中使用Beam搜索进行解码字符，但是速度较慢，这里默认未使用Beam搜索，以贪婪搜索进行解码字符。
+
+## 引用
+
+```bibtex
+@article{Sheng2019NRTR,
+ title = {NRTR: A No-Recurrence Sequence-to-Sequence Model For Scene Text Recognition},
+ author = {Fenfen Sheng and Zhineng Chen andBo Xu},
+ booktitle = {ICDAR},
+ year = {2019},
+ url = {http://arxiv.org/abs/1806.00926},
+ pages = {781-786}
+}
+```
diff --git a/doc/doc_ch/algorithm_rec_rare.md b/doc/doc_ch/algorithm_rec_rare.md
new file mode 100644
index 0000000000000000000000000000000000000000..dddd27ef9880fc7f8735f32fa85a406a57a77c8e
--- /dev/null
+++ b/doc/doc_ch/algorithm_rec_rare.md
@@ -0,0 +1,121 @@
+# RARE
+
+- [1. 算法简介](#1)
+- [2. 环境配置](#2)
+- [3. 模型训练、评估、预测](#3)
+ - [3.1 训练](#3-1)
+ - [3.2 评估](#3-2)
+ - [3.3 预测](#3-3)
+- [4. 推理部署](#4)
+ - [4.1 Python推理](#4-1)
+ - [4.2 C++推理](#4-2)
+ - [4.3 Serving服务化部署](#4-3)
+ - [4.4 更多推理部署](#4-4)
+- [5. FAQ](#5)
+
+
+## 1. 算法简介
+
+论文信息：
+> [Robust Scene Text Recognition with Automatic Rectification](https://arxiv.org/abs/1603.03915v2)
+> Baoguang Shi, Xinggang Wang, Pengyuan Lyu, Cong Yao, Xiang Bai∗
+> CVPR, 2016
+
+使用MJSynth和SynthText两个文字识别数据集训练，在IIIT, SVT, IC03, IC13, IC15, SVTP, CUTE数据集上进行评估，算法复现效果如下：
+
+|模型|骨干网络|配置文件|Avg Accuracy|下载链接|
+| --- | --- | --- | --- | --- |
+|RARE|Resnet34_vd|[configs/rec/rec_r34_vd_tps_bilstm_att.yml](../../configs/rec/rec_r34_vd_tps_bilstm_att.yml)|83.6%|[训练模型](https://paddleocr.bj.bcebos.com/dygraph_v2.0/en/rec_r34_vd_tps_bilstm_att_v2.0_train.tar)|
+|RARE|MobileNetV3|[configs/rec/rec_mv3_tps_bilstm_att.yml](../../configs/rec/rec_mv3_tps_bilstm_att.yml)|82.5%|[训练模型](https://paddleocr.bj.bcebos.com/dygraph_v2.0/en/rec_mv3_tps_bilstm_att_v2.0_train.tar)|
+
+
+
+## 2. 环境配置
+请先参考[《运行环境准备》](./environment.md)配置PaddleOCR运行环境，参考[《项目克隆》](./clone.md)克隆项目代码。
+
+
+## 3. 模型训练、评估、预测
+
+请参考[文本识别训练教程](./recognition.md)。PaddleOCR对代码进行了模块化，训练不同的识别模型只需要**更换配置文件**即可。以基于Resnet34_vd骨干网络为例:
+
+
+### 3.1 训练
+
+```
+#单卡训练（训练周期长，不建议）
+python3 tools/train.py -c configs/rec/rec_r34_vd_tps_bilstm_att.yml
+#多卡训练，通过--gpus参数指定卡号
+python3 -m paddle.distributed.launch --gpus '0,1,2,3' tools/train.py -c configs/rec/rec_r34_vd_tps_bilstm_att.yml
+```
+
+
+### 3.2 评估
+
+```
+# GPU评估, Global.pretrained_model为待评估模型
+python3 -m paddle.distributed.launch --gpus '0' tools/eval.py -c configs/rec/rec_r34_vd_tps_bilstm_att.yml -o Global.pretrained_model={path/to/weights}/best_accuracy
+```
+
+
+### 3.3 预测
+
+```
+python3 tools/infer_rec.py -c configs/rec/rec_r34_vd_tps_bilstm_att.yml -o Global.pretrained_model={path/to/weights}/best_accuracy Global.infer_img=doc/imgs_words/en/word_1.png
+```
+
+
+## 4. 推理部署
+
+
+### 4.1 Python推理
+首先将RARE文本识别训练过程中保存的模型，转换成inference model。以基于Resnet34_vd骨干网络，在MJSynth和SynthText两个文字识别数据集训练得到的模型为例（ [模型下载地址](https://paddleocr.bj.bcebos.com/dygraph_v2.0/en/rec_r34_vd_tps_bilstm_att_v2.0_train.tar) )，可以使用如下命令进行转换：
+
+```shell
+python3 tools/export_model.py -c configs/rec/rec_r34_vd_tps_bilstm_att.yml -o Global.pretrained_model=./rec_r34_vd_tps_bilstm_att_v2.0_train/best_accuracy Global.save_inference_dir=./inference/rec_rare
+```
+
+RARE文本识别模型推理，可以执行如下命令：
+
+```shell
+python3 tools/infer/predict_rec.py --image_dir="doc/imgs_words/en/word_1.png" --rec_model_dir="./inference/rec_rare/" --rec_image_shape="3, 32, 100" --rec_char_dict_path="./ppocr/utils/ic15_dict.txt"
+```
+推理结果如下所示：
+
+
+
+```
+Predicts of doc/imgs_words/en/word_1.png:('joint ', 0.9999969601631165)
+```
+
+
+
+### 4.2 C++推理
+
+暂不支持
+
+
+### 4.3 Serving服务化部署
+
+暂不支持
+
+
+### 4.4 更多推理部署
+
+RARE模型还支持以下推理部署方式：
+
+- Paddle2ONNX推理：准备好推理模型后，参考[paddle2onnx](../../deploy/paddle2onnx/)教程操作。
+
+
+## 5. FAQ
+
+
+## 引用
+
+```bibtex
+@inproceedings{2016Robust,
+ title={Robust Scene Text Recognition with Automatic Rectification},
+ author={ Shi, B. and Wang, X. and Lyu, P. and Cong, Y. and Xiang, B. },
+ booktitle={2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
+ year={2016},
+}
+```
diff --git a/doc/doc_ch/algorithm_rec_rosetta.md b/doc/doc_ch/algorithm_rec_rosetta.md
new file mode 100644
index 0000000000000000000000000000000000000000..c1784cac8c073eaaffc0258df2c419239332bb65
--- /dev/null
+++ b/doc/doc_ch/algorithm_rec_rosetta.md
@@ -0,0 +1,123 @@
+# Rosetta
+
+- [1. 算法简介](#1)
+- [2. 环境配置](#2)
+- [3. 模型训练、评估、预测](#3)
+ - [3.1 训练](#3-1)
+ - [3.2 评估](#3-2)
+ - [3.3 预测](#3-3)
+- [4. 推理部署](#4)
+ - [4.1 Python推理](#4-1)
+ - [4.2 C++推理](#4-2)
+ - [4.3 Serving服务化部署](#4-3)
+ - [4.4 更多推理部署](#4-4)
+- [5. FAQ](#5)
+
+
+## 1. 算法简介
+
+论文信息：
+> [Rosetta: Large Scale System for Text Detection and Recognition in Images](https://arxiv.org/abs/1910.05085)
+> Borisyuk F , Gordo A , V Sivakumar
+> KDD, 2018
+
+使用MJSynth和SynthText两个文字识别数据集训练，在IIIT, SVT, IC03, IC13, IC15, SVTP, CUTE数据集上进行评估， 算法复现效果如下：
+
+|模型|骨干网络|配置文件|Avg Accuracy|下载链接|
+| --- | --- | --- | --- | --- |
+|Rosetta|Resnet34_vd|[configs/rec/rec_r34_vd_none_none_ctc.yml](../../configs/rec/rec_r34_vd_none_none_ctc.yml)|79.11%|[训练模型](https://paddleocr.bj.bcebos.com/dygraph_v2.0/en/rec_r34_vd_none_none_ctc_v2.0_train.tar)|
+|Rosetta|MobileNetV3|[configs/rec/rec_mv3_none_none_ctc.yml](../../configs/rec/rec_mv3_none_none_ctc.yml)|75.80%|[训练模型](https://paddleocr.bj.bcebos.com/dygraph_v2.0/en/rec_mv3_none_none_ctc_v2.0_train.tar)|
+
+
+
+## 2. 环境配置
+请先参考[《运行环境准备》](./environment.md)配置PaddleOCR运行环境，参考[《项目克隆》](./clone.md)克隆项目代码。
+
+
+
+## 3. 模型训练、评估、预测
+
+请参考[文本识别训练教程](./recognition.md)。PaddleOCR对代码进行了模块化，训练不同的识别模型只需要**更换配置文件**即可。 以基于Resnet34_vd骨干网络为例:
+
+
+### 3.1 训练
+
+```
+#单卡训练（训练周期长，不建议）
+python3 tools/train.py -c configs/rec/rec_r34_vd_none_none_ctc.yml
+#多卡训练，通过--gpus参数指定卡号
+python3 -m paddle.distributed.launch --gpus '0,1,2,3' tools/train.py -c configs/rec/rec_r34_vd_none_none_ctc.yml
+```
+
+
+### 3.2 评估
+
+```
+# GPU评估, Global.pretrained_model为待评估模型
+python3 -m paddle.distributed.launch --gpus '0' tools/eval.py -c configs/rec/rec_r34_vd_none_none_ctc.yml -o Global.pretrained_model={path/to/weights}/best_accuracy
+```
+
+
+### 3.3 预测
+
+```
+python3 tools/infer_rec.py -c configs/rec/rec_r34_vd_none_none_ctc.yml -o Global.pretrained_model={path/to/weights}/best_accuracy Global.infer_img=doc/imgs_words/en/word_1.png
+```
+
+
+
+## 4. 推理部署
+
+
+### 4.1 Python推理
+首先将Rosetta文本识别训练过程中保存的模型，转换成inference model。以基于Resnet34_vd骨干网络，在MJSynth和SynthText两个文字识别数据集训练得到的模型为例（ [模型下载地址](https://paddleocr.bj.bcebos.com/dygraph_v2.0/en/rec_r34_vd_none_none_ctc_v2.0_train.tar) )，可以使用如下命令进行转换：
+
+```shell
+python3 tools/export_model.py -c configs/rec/rec_r34_vd_none_none_ctc.yml -o Global.pretrained_model=./rec_r34_vd_none_none_ctc_v2.0_train/best_accuracy Global.save_inference_dir=./inference/rec_rosetta
+```
+
+Rosetta文本识别模型推理，可以执行如下命令：
+
+```shell
+python3 tools/infer/predict_rec.py --image_dir="doc/imgs_words/en/word_1.png" --rec_model_dir="./inference/rec_rosetta/" --rec_image_shape="3, 32, 100" --rec_char_dict_path="./ppocr/utils/ic15_dict.txt"
+```
+
+推理结果如下所示：
+
+
+
+```
+Predicts of doc/imgs_words/en/word_1.png:('joint', 0.9999982714653015)
+```
+
+
+### 4.2 C++推理
+
+暂不支持
+
+
+### 4.3 Serving服务化部署
+
+暂不支持
+
+
+### 4.4 更多推理部署
+
+Rosetta模型还支持以下推理部署方式：
+
+- Paddle2ONNX推理：准备好推理模型后，参考[paddle2onnx](../../deploy/paddle2onnx/)教程操作。
+
+
+## 5. FAQ
+
+
+## 引用
+
+```bibtex
+@inproceedings{2018Rosetta,
+ title={Rosetta: Large Scale System for Text Detection and Recognition in Images},
+ author={ Borisyuk, Fedor and Gordo, Albert and Sivakumar, Viswanath },
+ booktitle={the 24th ACM SIGKDD International Conference},
+ year={2018},
+}
+```
diff --git a/doc/doc_ch/algorithm_rec_sar.md b/doc/doc_ch/algorithm_rec_sar.md
index aedc16714518b2de220118f755e00c3ba6bc7a5e..b8304313994754480a89d708e39149d67f828c0d 100644
--- a/doc/doc_ch/algorithm_rec_sar.md
+++ b/doc/doc_ch/algorithm_rec_sar.md
@@ -24,7 +24,7 @@
使用MJSynth和SynthText两个文字识别数据集训练，在IIIT, SVT, IC03, IC13, IC15, SVTP, CUTE数据集上进行评估，算法复现效果如下：
|模型|骨干网络|配置文件|Acc|下载链接|
-| --- | --- | --- | --- | --- | --- | --- |
+| --- | --- | --- | --- | --- |
|SAR|ResNet31|[rec_r31_sar.yml](../../configs/rec/rec_r31_sar.yml)|87.20%|[训练模型](https://paddleocr.bj.bcebos.com/dygraph_v2.1/rec/rec_r31_sar_train.tar)|
注：除了使用MJSynth和SynthText两个文字识别数据集外，还加入了[SynthAdd](https://pan.baidu.com/share/init?surl=uV0LtoNmcxbO-0YA7Ch4dg)数据（提取码：627x），和部分真实数据，具体数据细节可以参考论文。
diff --git a/doc/doc_ch/algorithm_rec_seed.md b/doc/doc_ch/algorithm_rec_seed.md
new file mode 100644
index 0000000000000000000000000000000000000000..710e92272dc3169bf373d273534441a15c6be01c
--- /dev/null
+++ b/doc/doc_ch/algorithm_rec_seed.md
@@ -0,0 +1,113 @@
+# SEED
+
+- [1. 算法简介](#1)
+- [2. 环境配置](#2)
+- [3. 模型训练、评估、预测](#3)
+ - [3.1 训练](#3-1)
+ - [3.2 评估](#3-2)
+ - [3.3 预测](#3-3)
+- [4. 推理部署](#4)
+ - [4.1 Python推理](#4-1)
+ - [4.2 C++推理](#4-2)
+ - [4.3 Serving服务化部署](#4-3)
+ - [4.4 更多推理部署](#4-4)
+- [5. FAQ](#5)
+
+
+## 1. 算法简介
+
+论文信息：
+> [SEED: Semantics Enhanced Encoder-Decoder Framework for Scene Text Recognition](https://arxiv.org/pdf/2005.10977.pdf)
+
+> Qiao, Zhi and Zhou, Yu and Yang, Dongbao and Zhou, Yucan and Wang, Weiping
+
+> CVPR, 2020
+
+参考[DTRB](https://arxiv.org/abs/1904.01906) 文字识别训练和评估流程，使用MJSynth和SynthText两个文字识别数据集训练，在IIIT, SVT, IC03, IC13, IC15, SVTP, CUTE数据集上进行评估，算法效果如下：
+
+|模型|骨干网络|Avg Accuracy|配置文件|下载链接|
+|---|---|---|---|---|
+|SEED|Aster_Resnet| 85.2% | [configs/rec/rec_resnet_stn_bilstm_att.yml](../../configs/rec/rec_resnet_stn_bilstm_att.yml) | [训练模型](https://paddleocr.bj.bcebos.com/dygraph_v2.1/rec/rec_resnet_stn_bilstm_att.tar) |
+
+
+## 2. 环境配置
+请先参考[《运行环境准备》](./environment.md)配置PaddleOCR运行环境，参考[《项目克隆》](./clone.md)克隆项目代码。
+
+
+
+## 3. 模型训练、评估、预测
+
+请参考[文本识别训练教程](./recognition.md)。PaddleOCR对代码进行了模块化，训练不同的识别模型只需要**更换配置文件**即可。
+
+- 训练
+
+SEED模型需要额外加载FastText训练好的[语言模型](https://dl.fbaipublicfiles.com/fasttext/vectors-crawl/cc.en.300.bin.gz) ,并且安装 fasttext 依赖：
+
+```
+python3 -m pip install fasttext==0.9.1
+```
+
+然后，在完成数据准备后，便可以启动训练，训练命令如下：
+
+```
+#单卡训练（训练周期长，不建议）
+python3 tools/train.py -c configs/rec/rec_resnet_stn_bilstm_att.yml
+
+#多卡训练，通过--gpus参数指定卡号
+python3 -m paddle.distributed.launch --gpus '0,1,2,3' tools/train.py -c rec_resnet_stn_bilstm_att.yml
+
+```
+
+- 评估
+
+```
+# GPU 评估， Global.pretrained_model 为待测权重
+python3 -m paddle.distributed.launch --gpus '0' tools/eval.py -c configs/rec/rec_resnet_stn_bilstm_att.yml -o Global.pretrained_model={path/to/weights}/best_accuracy
+```
+
+- 预测：
+
+```
+# 预测使用的配置文件必须与训练一致
+python3 tools/infer_rec.py -c configs/rec/rec_resnet_stn_bilstm_att.yml -o Global.pretrained_model={path/to/weights}/best_accuracy Global.infer_img=doc/imgs_words/en/word_1.png
+```
+
+
+## 4. 推理部署
+
+
+### 4.1 Python推理
+
+coming soon
+
+
+
+### 4.2 C++推理
+
+coming soon
+
+
+### 4.3 Serving服务化部署
+
+coming soon
+
+
+### 4.4 更多推理部署
+
+coming soon
+
+
+## 5. FAQ
+
+
+## 引用
+
+```bibtex
+@inproceedings{qiao2020seed,
+ title={Seed: Semantics enhanced encoder-decoder framework for scene text recognition},
+ author={Qiao, Zhi and Zhou, Yu and Yang, Dongbao and Zhou, Yucan and Wang, Weiping},
+ booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
+ pages={13528--13537},
+ year={2020}
+}
+```
diff --git a/doc/doc_ch/algorithm_rec_srn.md b/doc/doc_ch/algorithm_rec_srn.md
index b124790761eed875434ecf509f7647fe23d1bc90..ca7961359eb902fafee959b26d02f324aece233a 100644
--- a/doc/doc_ch/algorithm_rec_srn.md
+++ b/doc/doc_ch/algorithm_rec_srn.md
@@ -24,7 +24,7 @@
使用MJSynth和SynthText两个文字识别数据集训练，在IIIT, SVT, IC03, IC13, IC15, SVTP, CUTE数据集上进行评估，算法复现效果如下：
|模型|骨干网络|配置文件|Acc|下载链接|
-| --- | --- | --- | --- | --- | --- | --- |
+| --- | --- | --- | --- | --- |
|SRN|Resnet50_vd_fpn|[rec_r50_fpn_srn.yml](../../configs/rec/rec_r50_fpn_srn.yml)|86.31%|[训练模型](https://paddleocr.bj.bcebos.com/dygraph_v2.0/en/rec_r50_vd_srn_train.tar)|
diff --git a/doc/doc_ch/algorithm_rec_starnet.md b/doc/doc_ch/algorithm_rec_starnet.md
new file mode 100644
index 0000000000000000000000000000000000000000..c1ca761a0780b6bb8d15c12c820902c27d11d029
--- /dev/null
+++ b/doc/doc_ch/algorithm_rec_starnet.md
@@ -0,0 +1,139 @@
+# STAR-Net
+
+- [1. 算法简介](#1)
+- [2. 环境配置](#2)
+- [3. 模型训练、评估、预测](#3)
+ - [3.1 训练](#3-1)
+ - [3.2 评估](#3-2)
+ - [3.3 预测](#3-3)
+- [4. 推理部署](#4)
+ - [4.1 Python推理](#4-1)
+ - [4.2 C++推理](#4-2)
+ - [4.3 Serving服务化部署](#4-3)
+ - [4.4 更多推理部署](#4-4)
+- [5. FAQ](#5)
+
+
+## 1. 算法简介
+
+论文信息：
+> [STAR-Net: a spatial attention residue network for scene text recognition.](http://www.bmva.org/bmvc/2016/papers/paper043/paper043.pdf)
+
+> Wei Liu, Chaofeng Chen, Kwan-Yee K. Wong, Zhizhong Su and Junyu Han.
+
+> BMVC, pages 43.1-43.13, 2016
+
+参考[DTRB](https://arxiv.org/abs/1904.01906) 文字识别训练和评估流程，使用MJSynth和SynthText两个文字识别数据集训练，在IIIT, SVT, IC03, IC13, IC15, SVTP, CUTE数据集上进行评估，算法效果如下：
+
+|模型|骨干网络|Avg Accuracy|配置文件|下载链接|
+|---|---|---|---|---|
+|StarNet|Resnet34_vd|84.44%|[configs/rec/rec_r34_vd_tps_bilstm_ctc.yml](../../configs/rec/rec_r34_vd_tps_bilstm_ctc.yml)|[训练模型](https://paddleocr.bj.bcebos.com/dygraph_v2.0/en/rec_r34_vd_tps_bilstm_ctc_v2.0_train.tar)|
+|StarNet|MobileNetV3|81.42%|[configs/rec/rec_mv3_tps_bilstm_ctc.yml](../../configs/rec/rec_mv3_tps_bilstm_ctc.yml)|[训练模型](https://paddleocr.bj.bcebos.com/dygraph_v2.0/en/rec_mv3_tps_bilstm_ctc_v2.0_train.tar)|
+
+
+
+## 2. 环境配置
+请先参考[《运行环境准备》](./environment.md)配置PaddleOCR运行环境，参考[《项目克隆》](./clone.md)克隆项目代码。
+
+
+
+## 3. 模型训练、评估、预测
+
+请参考[文本识别训练教程](./recognition.md)。PaddleOCR对代码进行了模块化，训练不同的识别模型只需要**更换配置文件**即可。
+
+- 训练
+
+在完成数据准备后，便可以启动训练，训练命令如下：
+
+```
+#单卡训练（训练周期长，不建议）
+python3 tools/train.py -c configs/rec/rec_r34_vd_tps_bilstm_ctc.yml
+
+#多卡训练，通过--gpus参数指定卡号
+python3 -m paddle.distributed.launch --gpus '0,1,2,3' tools/train.py -c rec_r34_vd_tps_bilstm_ctc.yml
+
+```
+
+- 评估
+
+```
+# GPU 评估， Global.pretrained_model 为待测权重
+python3 -m paddle.distributed.launch --gpus '0' tools/eval.py -c configs/rec/rec_r34_vd_tps_bilstm_ctc.yml -o Global.pretrained_model={path/to/weights}/best_accuracy
+```
+
+- 预测：
+
+```
+# 预测使用的配置文件必须与训练一致
+python3 tools/infer_rec.py -c configs/rec/rec_r34_vd_tps_bilstm_ctc.yml -o Global.pretrained_model={path/to/weights}/best_accuracy Global.infer_img=doc/imgs_words/en/word_1.png
+```
+
+
+## 4. 推理部署
+
+
+### 4.1 Python推理
+
+首先将 STAR-Net 文本识别训练过程中保存的模型，转换成inference model。以基于Resnet34_vd骨干网络，使用MJSynth和SynthText两个英文文本识别合成数据集训练的[模型](https://paddleocr.bj.bcebos.com/dygraph_v2.0/en/rec_r34_vd_none_bilstm_ctc_v2.0_train.tar) 为例，可以使用如下命令进行转换：
+```shell
+python3 tools/export_model.py -c configs/rec/rec_r34_vd_tps_bilstm_ctc.yml -o Global.pretrained_model=./rec_r34_vd_tps_bilstm_ctc_v2.0_train/best_accuracy Global.save_inference_dir=./inference/rec_starnet
+```
+STAR-Net 文本识别模型推理，可以执行如下命令：
+
+```shell
+python3 tools/infer/predict_rec.py --image_dir="./doc/imgs_words_en/word_336.png" --rec_model_dir="./inference/rec_starnet/" --rec_image_shape="3, 32, 100" --rec_char_dict_path="./ppocr/utils/ic15_dict.txt"
+```
+
+
+
+执行命令后，上面图像的识别结果如下：
+
+```bash
+Predicts of ./doc/imgs_words_en/word_336.png:('super', 0.9999073)
+```
+
+**注意**：由于上述模型是参考[DTRB](https://arxiv.org/abs/1904.01906)文本识别训练和评估流程，与超轻量级中文识别模型训练有两方面不同：
+
+- 训练时采用的图像分辨率不同，训练上述模型采用的图像分辨率是[3，32，100]，而中文模型训练时，为了保证长文本的识别效果，训练时采用的图像分辨率是[3, 32, 320]。预测推理程序默认的形状参数是训练中文采用的图像分辨率，即[3, 32, 320]。因此，这里推理上述英文模型时，需要通过参数rec_image_shape设置识别图像的形状。
+
+- 字符列表，DTRB论文中实验只是针对26个小写英文本母和10个数字进行实验，总共36个字符。所有大小字符都转成了小写字符，不在上面列表的字符都忽略，认为是空格。因此这里没有输入字符字典，而是通过如下命令生成字典.因此在推理时需要设置参数rec_char_dict_path，指定为英文字典"./ppocr/utils/ic15_dict.txt"。
+
+```
+self.character_str = "0123456789abcdefghijklmnopqrstuvwxyz"
+dict_character = list(self.character_str)
+```
+
+
+
+### 4.2 C++推理
+
+准备好推理模型后，参考[cpp infer](../../deploy/cpp_infer/)教程进行操作即可。
+
+
+### 4.3 Serving服务化部署
+
+准备好推理模型后，参考[pdserving](../../deploy/pdserving/)教程进行Serving服务化部署，包括Python Serving和C++ Serving两种模式。
+
+
+### 4.4 更多推理部署
+
+STAR-Net模型还支持以下推理部署方式：
+
+- Paddle2ONNX推理：准备好推理模型后，参考[paddle2onnx](../../deploy/paddle2onnx/)教程操作。
+
+
+## 5. FAQ
+
+
+## 引用
+
+```bibtex
+@inproceedings{liu2016star,
+ title={STAR-Net: a spatial attention residue network for scene text recognition.},
+ author={Liu, Wei and Chen, Chaofeng and Wong, Kwan-Yee K and Su, Zhizhong and Han, Junyu},
+ booktitle={BMVC},
+ volume={2},
+ pages={7},
+ year={2016}
+}
+```
diff --git a/doc/doc_ch/algorithm_rec_svtr.md b/doc/doc_ch/algorithm_rec_svtr.md
new file mode 100644
index 0000000000000000000000000000000000000000..41a22ca65c00d0668298cdcf1486c1c3afdef919
--- /dev/null
+++ b/doc/doc_ch/algorithm_rec_svtr.md
@@ -0,0 +1,176 @@
+# 场景文本识别算法-SVTR
+
+- [1. 算法简介](#1)
+- [2. 环境配置](#2)
+- [3. 模型训练、评估、预测](#3)
+ - [3.1 训练](#3-1)
+ - [3.2 评估](#3-2)
+ - [3.3 预测](#3-3)
+- [4. 推理部署](#4)
+ - [4.1 Python推理](#4-1)
+ - [4.2 C++推理](#4-2)
+ - [4.3 Serving服务化部署](#4-3)
+ - [4.4 更多推理部署](#4-4)
+- [5. FAQ](#5)
+
+
+## 1. 算法简介
+
+论文信息：
+> [SVTR: Scene Text Recognition with a Single Visual Model](https://arxiv.org/abs/2205.00159)
+> Yongkun Du and Zhineng Chen and Caiyan Jia Xiaoting Yin and Tianlun Zheng and Chenxia Li and Yuning Du and Yu-Gang Jiang
+> IJCAI, 2022
+
+场景文本识别旨在将自然图像中的文本转录为数字字符序列，从而传达对场景理解至关重要的高级语义。这项任务由于文本变形、字体、遮挡、杂乱背景等方面的变化具有一定的挑战性。先前的方法为提高识别精度做出了许多工作。然而文本识别器除了准确度外，还因为实际需求需要考虑推理速度等因素。
+
+### SVTR算法简介
+
+主流的场景文本识别模型通常包含两个模块：用于特征提取的视觉模型和用于文本转录的序列模型。这种架构虽然准确，但复杂且效率较低，限制了在实际场景中的应用。SVTR提出了一种用于场景文本识别的单视觉模型，该模型在patch-wise image tokenization框架内，完全摒弃了序列建模，在精度具有竞争力的前提下，模型参数量更少，速度更快，主要有以下几点贡献：
+1. 首次发现单视觉模型可以达到与视觉语言模型相媲美甚至更高的准确率，并且其具有效率高和适应多语言的优点，在实际应用中很有前景。
+2. SVTR从字符组件的角度出发，逐渐的合并字符组件，自下而上地完成字符的识别。
+3. SVTR引入了局部和全局Mixing，分别用于提取字符组件特征和字符间依赖关系，与多尺度的特征一起，形成多粒度特征描述。
+
+
+
+SVTR在场景文本识别公开数据集上的精度(%)和模型文件如下：
+
+* 中文数据集来自于[Chinese Benckmark](https://arxiv.org/abs/2112.15093) ，SVTR的中文训练评估策略遵循该论文。
+
+| 模型 |IC13
+ 
+
+ 
+
-
+
