inference_en.md 24.7 KB
Newer Older
K
Khanh Tran 已提交
1

qq_25193841's avatar
qq_25193841 已提交
2
# Inference based on Python Prediction Engine
K
Khanh Tran 已提交
3

W
WenmuZhou 已提交
4
The inference model (the model saved by `paddle.jit.save`) is generally a solidified model saved after the model training is completed, and is mostly used to give prediction in deployment.
K
Khanh Tran 已提交
5 6 7

The model saved during the training process is the checkpoints model, which saves the parameters of the model and is mostly used to resume training.

L
LDOUBLEV 已提交
8
Compared with the checkpoints model, the inference model will additionally save the structural information of the model. Therefore, it is easier to deploy because the model structure and model parameters are already solidified in the inference model file, and is suitable for integration with actual systems.
L
LDOUBLEV 已提交
9
For more details, please refer to the document [Classification Framework](https://github.com/PaddlePaddle/PaddleClas/blob/release%2F2.0/docs/zh_CN/extension/paddle_mobile_inference.md).
K
Khanh Tran 已提交
10

W
WenmuZhou 已提交
11
Next, we first introduce how to convert a trained model into an inference model, and then we will introduce text detection, text recognition, angle class, and the concatenation of them based on inference model.
K
Khanh Tran 已提交
12

qq_25193841's avatar
qq_25193841 已提交
13 14 15 16
- [1. Convert Training Model to Inference Model](#CONVERT)
    - [1.1 Convert Detection Model to Inference Model](#Convert_detection_model)
    - [1.2 Convert Recognition Model to Inference Model](#Convert_recognition_model)
    - [1.3 Convert Angle Classification Model to Inference Model](#Convert_angle_class_model)
W
WenmuZhou 已提交
17 18


qq_25193841's avatar
qq_25193841 已提交
19 20 21 22 23
- [2. Text Detection Model Inference](#DETECTION_MODEL_INFERENCE)
    - [2.1 Lightweight Chinese Detection Model Inference](#LIGHTWEIGHT_DETECTION)
    - [2.2 DB Text Detection Model Inference](#DB_DETECTION)
    - [2.3 East Text Detection Model Inference](#EAST_DETECTION)
    - [2.4 Sast Text Detection Model Inference](#SAST_DETECTION)
W
WenmuZhou 已提交
24 25
    - [5. Multilingual model inference](#Multilingual model inference)

qq_25193841's avatar
qq_25193841 已提交
26 27
- [3. Text Recognition Model Inference](#RECOGNITION_MODEL_INFERENCE)
    - [3.1 Lightweight Chinese Text Recognition Model Reference](#LIGHTWEIGHT_RECOGNITION)
L
licx 已提交
28
    - [2. CTC-BASED TEXT RECOGNITION MODEL INFERENCE](#CTC-BASED_RECOGNITION)
T
tink2123 已提交
29
    - [3. SRN-BASED TEXT RECOGNITION MODEL INFERENCE](#SRN-BASED_RECOGNITION)
W
WenmuZhou 已提交
30 31
    - [3. TEXT RECOGNITION MODEL INFERENCE USING CUSTOM CHARACTERS DICTIONARY](#USING_CUSTOM_CHARACTERS)
    - [4. MULTILINGUAL MODEL INFERENCE](MULTILINGUAL_MODEL_INFERENCE)
W
WenmuZhou 已提交
32 33 34 35 36

- [ANGLE CLASSIFICATION MODEL INFERENCE](#ANGLE_CLASS_MODEL_INFERENCE)
    - [1. ANGLE CLASSIFICATION MODEL INFERENCE](#ANGLE_CLASS_MODEL_INFERENCE)

- [TEXT DETECTION ANGLE CLASSIFICATION AND RECOGNITION INFERENCE CONCATENATION](#CONCATENATION)
L
licx 已提交
37 38
    - [1. LIGHTWEIGHT CHINESE MODEL](#LIGHTWEIGHT_CHINESE_MODEL)
    - [2. OTHER MODELS](#OTHER_MODELS)
W
WenmuZhou 已提交
39

L
licx 已提交
40
<a name="CONVERT"></a>
qq_25193841's avatar
qq_25193841 已提交
41
## 1. Convert Training Model to Inference Model
L
licx 已提交
42
<a name="Convert_detection_model"></a>
qq_25193841's avatar
qq_25193841 已提交
43
### 1.1 Convert Detection Model to Inference Model
K
Khanh Tran 已提交
44

X
xxxpsyduck 已提交
45
Download the lightweight Chinese detection model:
K
Khanh Tran 已提交
46
```
W
WenmuZhou 已提交
47
wget -P ./ch_lite/ https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_mobile_v2.0_det_train.tar && tar xf ./ch_lite/ch_ppocr_mobile_v2.0_det_train.tar -C ./ch_lite/
K
Khanh Tran 已提交
48
```
W
WenmuZhou 已提交
49

K
Khanh Tran 已提交
50 51
The above model is a DB algorithm trained with MobileNetV3 as the backbone. To convert the trained model into an inference model, just run the following command:
```
W
WenmuZhou 已提交
52 53
# -c Set the training algorithm yml configuration file
# -o Set optional parameters
W
WenmuZhou 已提交
54
# Global.pretrained_model parameter Set the training model address to be converted without adding the file suffix .pdmodel, .pdopt or .pdparams.
W
WenmuZhou 已提交
55
# Global.save_inference_dir Set the address where the converted model will be saved.
T
tink2123 已提交
56

57
python3 tools/export_model.py -c configs/det/ch_ppocr_v2.0/ch_det_mv3_db_v2.0.yml -o Global.pretrained_model=./ch_lite/ch_ppocr_mobile_v2.0_det_train/best_accuracy  Global.save_inference_dir=./inference/det_db/
K
Khanh Tran 已提交
58
```
W
WenmuZhou 已提交
59

W
WenmuZhou 已提交
60
When converting to an inference model, the configuration file used is the same as the configuration file used during training. In addition, you also need to set the `Global.pretrained_model` parameter in the configuration file.
W
WenmuZhou 已提交
61
After the conversion is successful, there are three files in the model save directory:
K
Khanh Tran 已提交
62 63
```
inference/det_db/
64 65 66
    ├── inference.pdiparams         # The parameter file of detection inference model
    ├── inference.pdiparams.info    # The parameter information of detection inference model, which can be ignored
    └── inference.pdmodel           # The program file of detection inference model
K
Khanh Tran 已提交
67 68
```

L
licx 已提交
69
<a name="Convert_recognition_model"></a>
qq_25193841's avatar
qq_25193841 已提交
70
### 1.2 Convert Recognition Model to Inference Model
K
Khanh Tran 已提交
71

X
xxxpsyduck 已提交
72
Download the lightweight Chinese recognition model:
K
Khanh Tran 已提交
73
```
W
WenmuZhou 已提交
74
wget -P ./ch_lite/ https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_mobile_v2.0_rec_train.tar && tar xf ./ch_lite/ch_ppocr_mobile_v2.0_rec_train.tar -C ./ch_lite/
K
Khanh Tran 已提交
75 76 77 78
```

The recognition model is converted to the inference model in the same way as the detection, as follows:
```
W
WenmuZhou 已提交
79 80
# -c Set the training algorithm yml configuration file
# -o Set optional parameters
W
WenmuZhou 已提交
81
# Global.pretrained_model parameter Set the training model address to be converted without adding the file suffix .pdmodel, .pdopt or .pdparams.
W
WenmuZhou 已提交
82
# Global.save_inference_dir Set the address where the converted model will be saved.
T
tink2123 已提交
83

84
python3 tools/export_model.py -c configs/rec/ch_ppocr_v2.0/rec_chinese_lite_train_v2.0.yml -o Global.pretrained_model=./ch_lite/ch_ppocr_mobile_v2.0_rec_train/best_accuracy  Global.save_inference_dir=./inference/rec_crnn/
K
Khanh Tran 已提交
85 86 87 88
```

If you have a model trained on your own dataset with a different dictionary file, please make sure that you modify the `character_dict_path` in the configuration file to your dictionary file path.

W
WenmuZhou 已提交
89
After the conversion is successful, there are three files in the model save directory:
K
Khanh Tran 已提交
90
```
W
WenmuZhou 已提交
91
inference/det_db/
92 93 94
    ├── inference.pdiparams         # The parameter file of recognition inference model
    ├── inference.pdiparams.info    # The parameter information of recognition inference model, which can be ignored
    └── inference.pdmodel           # The program file of recognition model
K
Khanh Tran 已提交
95 96
```

W
WenmuZhou 已提交
97
<a name="Convert_angle_class_model"></a>
qq_25193841's avatar
qq_25193841 已提交
98
### 1.3 Convert Angle Classification Model to Inference Model
W
WenmuZhou 已提交
99 100 101

Download the angle classification model:
```
W
WenmuZhou 已提交
102
wget -P ./ch_lite/ https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_mobile_v2.0_cls_train.tar && tar xf ./ch_lite/ch_ppocr_mobile_v2.0_cls_train.tar -C ./ch_lite/
W
WenmuZhou 已提交
103 104 105 106
```

The angle classification model is converted to the inference model in the same way as the detection, as follows:
```
W
WenmuZhou 已提交
107 108
# -c Set the training algorithm yml configuration file
# -o Set optional parameters
W
WenmuZhou 已提交
109
# Global.pretrained_model parameter Set the training model address to be converted without adding the file suffix .pdmodel, .pdopt or .pdparams.
W
WenmuZhou 已提交
110
# Global.save_inference_dir Set the address where the converted model will be saved.
W
WenmuZhou 已提交
111

112
python3 tools/export_model.py -c configs/cls/cls_mv3.yml -o Global.pretrained_model=./ch_lite/ch_ppocr_mobile_v2.0_cls_train/best_accuracy  Global.save_inference_dir=./inference/cls/
W
WenmuZhou 已提交
113 114 115 116
```

After the conversion is successful, there are two files in the directory:
```
W
WenmuZhou 已提交
117
inference/det_db/
118 119 120
    ├── inference.pdiparams         # The parameter file of angle class inference model
    ├── inference.pdiparams.info    # The parameter information of  angle class inference model, which can be ignored
    └── inference.pdmodel           # The program file of angle class model
W
WenmuZhou 已提交
121 122 123
```


L
licx 已提交
124
<a name="DETECTION_MODEL_INFERENCE"></a>
qq_25193841's avatar
qq_25193841 已提交
125
## 2. Text Detection Model Inference
K
Khanh Tran 已提交
126

T
tink2123 已提交
127 128
The following will introduce the lightweight Chinese detection model inference, DB text detection model inference and EAST text detection model inference. The default configuration is based on the inference setting of the DB text detection model.
Because EAST and DB algorithms are very different, when inference, it is necessary to **adapt the EAST text detection algorithm by passing in corresponding parameters**.
K
Khanh Tran 已提交
129

L
licx 已提交
130
<a name="LIGHTWEIGHT_DETECTION"></a>
qq_25193841's avatar
qq_25193841 已提交
131
### 2.1 Lightweight Chinese Detection Model Inference
K
Khanh Tran 已提交
132

X
xxxpsyduck 已提交
133
For lightweight Chinese detection model inference, you can execute the following commands:
K
Khanh Tran 已提交
134 135

```
L
LDOUBLEV 已提交
136 137 138 139
# download DB text detection inference model
wget  https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_mobile_v2.0_det_infer.tar
tar xf ch_ppocr_mobile_v2.0_det_infer.tar
# predict
L
LDOUBLEV 已提交
140
python3 tools/infer/predict_det.py --image_dir="./doc/imgs/00018069.jpg" --det_model_dir="./inference/det_db/"
K
Khanh Tran 已提交
141 142 143 144
```

The visual text detection results are saved to the ./inference_results folder by default, and the name of the result file is prefixed with'det_res'. Examples of results are as follows:

L
LDOUBLEV 已提交
145
![](../imgs_results/det_res_00018069.jpg)
K
Khanh Tran 已提交
146

L
LDOUBLEV 已提交
147
You can use the parameters `limit_type` and `det_limit_side_len` to limit the size of the input image,
M
MissPenguin 已提交
148
The optional parameters of `limit_type` are [`max`, `min`], and
L
LDOUBLEV 已提交
149
`det_limit_size_len` is a positive integer, generally set to a multiple of 32, such as 960.
K
Khanh Tran 已提交
150

L
LDOUBLEV 已提交
151 152 153 154 155
The default setting of the parameters is `limit_type='max', det_limit_side_len=960`. Indicates that the longest side of the network input image cannot exceed 960,
If this value is exceeded, the image will be resized with the same width ratio to ensure that the longest side is `det_limit_side_len`.
Set as `limit_type='min', det_limit_side_len=960`, it means that the shortest side of the image is limited to 960.

If the resolution of the input picture is relatively large and you want to use a larger resolution prediction, you can set det_limit_side_len to the desired value, such as 1216:
K
Khanh Tran 已提交
156
```
W
WenmuZhou 已提交
157
python3 tools/infer/predict_det.py --image_dir="./doc/imgs/1.jpg" --det_model_dir="./inference/det_db/" --det_limit_type=max --det_limit_side_len=1216
K
Khanh Tran 已提交
158 159 160 161
```

If you want to use the CPU for prediction, execute the command as follows
```
W
WenmuZhou 已提交
162
python3 tools/infer/predict_det.py --image_dir="./doc/imgs/1.jpg" --det_model_dir="./inference/det_db/" --use_gpu=False
K
Khanh Tran 已提交
163 164
```

L
licx 已提交
165
<a name="DB_DETECTION"></a>
qq_25193841's avatar
qq_25193841 已提交
166
### 2.2 DB Text Detection Model Inference
K
Khanh Tran 已提交
167

W
WenmuZhou 已提交
168
First, convert the model saved in the DB text detection training process into an inference model. Taking the model based on the Resnet50_vd backbone network and trained on the ICDAR2015 English dataset as an example ([model download link](https://paddleocr.bj.bcebos.com/dygraph_v2.0/en/det_r50_vd_db_v2.0_train.tar)), you can use the following command to convert:
K
Khanh Tran 已提交
169 170

```
171
python3 tools/export_model.py -c configs/det/det_r50_vd_db.yml -o Global.pretrained_model=./det_r50_vd_db_v2.0_train/best_accuracy  Global.save_inference_dir=./inference/det_db
K
Khanh Tran 已提交
172 173 174 175 176 177 178 179 180 181
```

DB text detection model inference, you can execute the following command:

```
python3 tools/infer/predict_det.py --image_dir="./doc/imgs_en/img_10.jpg" --det_model_dir="./inference/det_db/"
```

The visualized text detection results are saved to the `./inference_results` folder by default, and the name of the result file is prefixed with 'det_res'. Examples of results are as follows:

182
![](../imgs_results/det_res_img_10_db.jpg)
K
Khanh Tran 已提交
183 184 185

**Note**: Since the ICDAR2015 dataset has only 1,000 training images, mainly for English scenes, the above model has very poor detection result on Chinese text images.

L
licx 已提交
186
<a name="EAST_DETECTION"></a>
qq_25193841's avatar
qq_25193841 已提交
187
### 2.3 EAST TEXT DETECTION MODEL INFERENCE
K
Khanh Tran 已提交
188

M
MissPenguin 已提交
189
First, convert the model saved in the EAST text detection training process into an inference model. Taking the model based on the Resnet50_vd backbone network and trained on the ICDAR2015 English dataset as an example ([model download link](https://paddleocr.bj.bcebos.com/dygraph_v2.0/en/det_r50_vd_east_v2.0_train.tar)), you can use the following command to convert:
K
Khanh Tran 已提交
190 191

```
192
python3 tools/export_model.py -c configs/det/det_r50_vd_east.yml -o Global.pretrained_model=./det_r50_vd_east_v2.0_train/best_accuracy  Global.save_inference_dir=./inference/det_east
K
Khanh Tran 已提交
193
```
L
licx 已提交
194
**For EAST text detection model inference, you need to set the parameter ``--det_algorithm="EAST"``**, run the following command:
K
Khanh Tran 已提交
195 196 197 198

```
python3 tools/infer/predict_det.py --image_dir="./doc/imgs_en/img_10.jpg" --det_model_dir="./inference/det_east/" --det_algorithm="EAST"
```
L
licx 已提交
199

K
Khanh Tran 已提交
200 201
The visualized text detection results are saved to the `./inference_results` folder by default, and the name of the result file is prefixed with 'det_res'. Examples of results are as follows:

M
MissPenguin 已提交
202
![](../imgs_results/det_res_img_10_east.jpg)
K
Khanh Tran 已提交
203

L
licx 已提交
204 205 206 207
**Note**: EAST post-processing locality aware NMS has two versions: Python and C++. The speed of C++ version is obviously faster than that of Python version. Due to the compilation version problem of NMS of C++ version, C++ version NMS will be called only in Python 3.5 environment, and python version NMS will be called in other cases.


<a name="SAST_DETECTION"></a>
qq_25193841's avatar
qq_25193841 已提交
208
### 2.4 Sast Text Detection Model Inference
L
licx 已提交
209
#### (1). Quadrangle text detection model (ICDAR2015)  
M
MissPenguin 已提交
210
First, convert the model saved in the SAST text detection training process into an inference model. Taking the model based on the Resnet50_vd backbone network and trained on the ICDAR2015 English dataset as an example ([model download link](https://paddleocr.bj.bcebos.com/dygraph_v2.0/en/det_r50_vd_sast_icdar15_v2.0_train.tar)), you can use the following command to convert:
L
licx 已提交
211 212

```
213
python3 tools/export_model.py -c configs/det/det_r50_vd_sast_icdar15.yml -o Global.pretrained_model=./det_r50_vd_sast_icdar15_v2.0_train/best_accuracy  Global.save_inference_dir=./inference/det_sast_ic15
L
licx 已提交
214 215 216
```

**For SAST quadrangle text detection model inference, you need to set the parameter `--det_algorithm="SAST"`**, run the following command:
K
Khanh Tran 已提交
217

L
licx 已提交
218 219 220 221 222
```
python3 tools/infer/predict_det.py --det_algorithm="SAST" --image_dir="./doc/imgs_en/img_10.jpg" --det_model_dir="./inference/det_sast_ic15/"
```

The visualized text detection results are saved to the `./inference_results` folder by default, and the name of the result file is prefixed with 'det_res'. Examples of results are as follows:
K
Khanh Tran 已提交
223

M
MissPenguin 已提交
224
![](../imgs_results/det_res_img_10_sast.jpg)
L
licx 已提交
225 226

#### (2). Curved text detection model (Total-Text)  
M
MissPenguin 已提交
227
First, convert the model saved in the SAST text detection training process into an inference model. Taking the model based on the Resnet50_vd backbone network and trained on the Total-Text English dataset as an example ([model download link](https://paddleocr.bj.bcebos.com/dygraph_v2.0/en/det_r50_vd_sast_totaltext_v2.0_train.tar)), you can use the following command to convert:
L
licx 已提交
228 229

```
230
python3 tools/export_model.py -c configs/det/det_r50_vd_sast_totaltext.yml -o Global.pretrained_model=./det_r50_vd_sast_totaltext_v2.0_train/best_accuracy  Global.save_inference_dir=./inference/det_sast_tt
L
licx 已提交
231 232
```

W
opt doc  
WenmuZhou 已提交
233
For SAST curved text detection model inference, you need to set the parameter `--det_algorithm="SAST"` and `--det_sast_polygon=True`, run the following command:
L
licx 已提交
234 235 236 237 238 239 240

```
python3 tools/infer/predict_det.py --det_algorithm="SAST" --image_dir="./doc/imgs_en/img623.jpg" --det_model_dir="./inference/det_sast_tt/" --det_sast_polygon=True
```

The visualized text detection results are saved to the `./inference_results` folder by default, and the name of the result file is prefixed with 'det_res'. Examples of results are as follows:

M
MissPenguin 已提交
241
![](../imgs_results/det_res_img623_sast.jpg)
L
licx 已提交
242 243 244 245

**Note**: SAST post-processing locality aware NMS has two versions: Python and C++. The speed of C++ version is obviously faster than that of Python version. Due to the compilation version problem of NMS of C++ version, C++ version NMS will be called only in Python 3.5 environment, and python version NMS will be called in other cases.

<a name="RECOGNITION_MODEL_INFERENCE"></a>
qq_25193841's avatar
qq_25193841 已提交
246
## 3. Text Recognition Model Inference
K
Khanh Tran 已提交
247

X
xxxpsyduck 已提交
248
The following will introduce the lightweight Chinese recognition model inference, other CTC-based and Attention-based text recognition models inference. For Chinese text recognition, it is recommended to choose the recognition model based on CTC loss. In practice, it is also found that the result of the model based on Attention loss is not as good as the one based on CTC loss. In addition, if the characters dictionary is modified during training, make sure that you use the same characters set during inferencing. Please check below for details.
K
Khanh Tran 已提交
249 250


L
licx 已提交
251
<a name="LIGHTWEIGHT_RECOGNITION"></a>
qq_25193841's avatar
qq_25193841 已提交
252
### 3.1 Lightweight Chinese Text Recognition Model Reference
K
Khanh Tran 已提交
253

X
xxxpsyduck 已提交
254
For lightweight Chinese recognition model inference, you can execute the following commands:
K
Khanh Tran 已提交
255 256

```
W
WenmuZhou 已提交
257 258 259 260
# download CRNN text recognition inference model
wget  https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_mobile_v2.0_rec_infer.tar
tar xf ch_ppocr_mobile_v2.0_rec_infer.tar
python3 tools/infer/predict_rec.py --image_dir="./doc/imgs_words_en/word_10.png" --rec_model_dir="ch_ppocr_mobile_v2.0_rec_infer"
K
Khanh Tran 已提交
261 262
```

W
WenmuZhou 已提交
263
![](../imgs_words_en/word_10.png)
K
Khanh Tran 已提交
264 265 266

After executing the command, the prediction results (recognized text and score) of the above image will be printed on the screen.

W
WenmuZhou 已提交
267
```bash
W
WenmuZhou 已提交
268
Predicts of ./doc/imgs_words_en/word_10.png:('PAIN', 0.9897658)
W
WenmuZhou 已提交
269
```
K
Khanh Tran 已提交
270

L
licx 已提交
271
<a name="CTC-BASED_RECOGNITION"></a>
X
xxxpsyduck 已提交
272
### 2. CTC-BASED TEXT RECOGNITION MODEL INFERENCE
K
Khanh Tran 已提交
273

W
WenmuZhou 已提交
274
Taking CRNN as an example, we introduce the recognition model inference based on CTC loss. Rosetta and Star-Net are used in a similar way, No need to set the recognition algorithm parameter rec_algorithm.
K
Khanh Tran 已提交
275

W
WenmuZhou 已提交
276
First, convert the model saved in the CRNN text recognition training process into an inference model. Taking the model based on Resnet34_vd backbone network, using MJSynth and SynthText (two English text recognition synthetic datasets) for training, as an example ([model download address](https://paddleocr.bj.bcebos.com/dygraph_v2.0/en/rec_r34_vd_none_bilstm_ctc_v2.0_train.tar)). It can be converted as follow:
K
Khanh Tran 已提交
277 278

```
279
python3 tools/export_model.py -c configs/det/rec_r34_vd_none_bilstm_ctc.yml -o Global.pretrained_model=./rec_r34_vd_none_bilstm_ctc_v2.0_train/best_accuracy  Global.save_inference_dir=./inference/rec_crnn
K
Khanh Tran 已提交
280 281
```

W
WenmuZhou 已提交
282
For CRNN text recognition model inference, execute the following commands:
K
Khanh Tran 已提交
283 284 285 286

```
python3 tools/infer/predict_rec.py --image_dir="./doc/imgs_words_en/word_336.png" --rec_model_dir="./inference/starnet/" --rec_image_shape="3, 32, 100" --rec_char_type="en"
```
X
xxxpsyduck 已提交
287

W
WenmuZhou 已提交
288
![](../imgs_words_en/word_336.png)
K
Khanh Tran 已提交
289

W
WenmuZhou 已提交
290 291 292 293 294
After executing the command, the recognition result of the above image is as follows:

```bash
Predicts of ./doc/imgs_words_en/word_336.png:('super', 0.9999073)
```
X
xxxpsyduck 已提交
295
**Note**:Since the above model refers to [DTRB](https://arxiv.org/abs/1904.01906) text recognition training and evaluation process, it is different from the training of lightweight Chinese recognition model in two aspects:
K
Khanh Tran 已提交
296 297 298 299 300 301 302 303 304 305

- The image resolution used in training is different: the image resolution used in training the above model is [3,32,100], while during our Chinese model training, in order to ensure the recognition effect of long text, the image resolution used in training is [3, 32, 320]. The default shape parameter of the inference stage is the image resolution used in training phase, that is [3, 32, 320]. Therefore, when running inference of the above English model here, you need to set the shape of the recognition image through the parameter `rec_image_shape`.

- Character list: the experiment in the DTRB paper is only for 26 lowercase English characters and 10 numbers, a total of 36 characters. All upper and lower case characters are converted to lower case characters, and characters not in the above list are ignored and considered as spaces. Therefore, no characters dictionary file is used here, but a dictionary is generated by the below command. Therefore, the parameter `rec_char_type` needs to be set during inference, which is specified as "en" in English.

```
self.character_str = "0123456789abcdefghijklmnopqrstuvwxyz"
dict_character = list(self.character_str)
```

T
tink2123 已提交
306 307 308 309 310 311 312 313 314 315 316 317 318 319 320
<a name="SRN-BASED_RECOGNITION"></a>
### 3. SRN-BASED TEXT RECOGNITION MODEL INFERENCE

The recognition model based on SRN requires additional setting of the recognition algorithm parameter
--rec_algorithm="SRN". At the same time, it is necessary to ensure that the predicted shape is consistent
with the training, such as: --rec_image_shape="1, 64, 256"

```
python3 tools/infer/predict_rec.py --image_dir="./doc/imgs_words_en/word_336.png" \
                                    --rec_model_dir="./inference/srn/" \
                                    --rec_image_shape="1, 64, 256" \
                                    --rec_char_type="en" \
                                    --rec_algorithm="SRN"
```

L
licx 已提交
321
<a name="USING_CUSTOM_CHARACTERS"></a>
T
tink2123 已提交
322
### 4. TEXT RECOGNITION MODEL INFERENCE USING CUSTOM CHARACTERS DICTIONARY
W
WenmuZhou 已提交
323
If the text dictionary is modified during training, when using the inference model to predict, you need to specify the dictionary path used by `--rec_char_dict_path`, and set `rec_char_type=ch`
L
LDOUBLEV 已提交
324 325

```
W
WenmuZhou 已提交
326
python3 tools/infer/predict_rec.py --image_dir="./doc/imgs_words_en/word_336.png" --rec_model_dir="./your inference model" --rec_image_shape="3, 32, 100" --rec_char_type="ch" --rec_char_dict_path="your text dict path"
L
LDOUBLEV 已提交
327 328
```

W
WenmuZhou 已提交
329
<a name="MULTILINGUAL_MODEL_INFERENCE"></a>
T
tink2123 已提交
330
### 5. MULTILINGAUL MODEL INFERENCE
W
WenmuZhou 已提交
331
If you need to predict other language models, when using inference model prediction, you need to specify the dictionary path used by `--rec_char_dict_path`. At the same time, in order to get the correct visualization results,
T
tink2123 已提交
332
You need to specify the visual font path through `--vis_font_path`. There are small language fonts provided by default under the `doc/fonts` path, such as Korean recognition:
W
WenmuZhou 已提交
333 334

```
T
tink2123 已提交
335
python3 tools/infer/predict_rec.py --image_dir="./doc/imgs_words/korean/1.jpg" --rec_model_dir="./your inference model" --rec_char_type="korean" --rec_char_dict_path="ppocr/utils/dict/korean_dict.txt" --vis_font_path="doc/fonts/korean.ttf"
W
WenmuZhou 已提交
336 337 338 339 340 341
```
![](../imgs_words/korean/1.jpg)

After executing the command, the prediction result of the above figure is:

``` text
W
WenmuZhou 已提交
342
Predicts of ./doc/imgs_words/korean/1.jpg:('바탕으로', 0.9948904)
W
WenmuZhou 已提交
343 344 345 346 347 348 349 350 351 352 353 354 355 356
```

<a name="ANGLE_CLASSIFICATION_MODEL_INFERENCE"></a>
## ANGLE CLASSIFICATION MODEL INFERENCE

The following will introduce the angle classification model inference.


<a name="ANGLE_CLASS_MODEL_INFERENCE"></a>
### 1.ANGLE CLASSIFICATION MODEL INFERENCE

For angle classification model inference, you can execute the following commands:

```
W
WenmuZhou 已提交
357
python3 tools/infer/predict_cls.py --image_dir="./doc/imgs_words_en/word_10.png" --cls_model_dir="./inference/cls/"
W
WenmuZhou 已提交
358
```
W
WenmuZhou 已提交
359 360 361 362 363 364
```
# download text angle class inference model:
wget  https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_mobile_v2.0_cls_infer.tar
tar xf ch_ppocr_mobile_v2.0_cls_infer.tar
python3 tools/infer/predict_cls.py --image_dir="./doc/imgs_words_en/word_10.png" --cls_model_dir="ch_ppocr_mobile_v2.0_cls_infer"
```
W
WenmuZhou 已提交
365
![](../imgs_words_en/word_10.png)
W
WenmuZhou 已提交
366 367 368

After executing the command, the prediction results (classification angle and score) of the above image will be printed on the screen.

W
WenmuZhou 已提交
369
```
W
WenmuZhou 已提交
370
 Predicts of ./doc/imgs_words_en/word_10.png:['0', 0.9999995]
W
WenmuZhou 已提交
371
```
W
WenmuZhou 已提交
372

L
licx 已提交
373
<a name="CONCATENATION"></a>
W
WenmuZhou 已提交
374
## TEXT DETECTION ANGLE CLASSIFICATION AND RECOGNITION INFERENCE CONCATENATION
K
Khanh Tran 已提交
375

L
licx 已提交
376
<a name="LIGHTWEIGHT_CHINESE_MODEL"></a>
X
xxxpsyduck 已提交
377
### 1. LIGHTWEIGHT CHINESE MODEL
K
Khanh Tran 已提交
378

littletomatodonkey's avatar
littletomatodonkey 已提交
379
When performing prediction, you need to specify the path of a single image or a folder of images through the parameter `image_dir`, the parameter `det_model_dir` specifies the path to detect the inference model, the parameter `cls_model_dir` specifies the path to angle classification inference model and the parameter `rec_model_dir` specifies the path to identify the inference model. The parameter `use_angle_cls` is used to control whether to enable the angle classification model. The parameter `use_mp` specifies whether to use multi-process to infer `total_process_num` specifies process number when using multi-process. The parameter . The visualized recognition results are saved to the `./inference_results` folder by default.
K
Khanh Tran 已提交
380

littletomatodonkey's avatar
littletomatodonkey 已提交
381
```shell
W
WenmuZhou 已提交
382
# use direction classifier
W
WenmuZhou 已提交
383
python3 tools/infer/predict_system.py --image_dir="./doc/imgs/00018069.jpg" --det_model_dir="./inference/det_db/" --cls_model_dir="./inference/cls/" --rec_model_dir="./inference/rec_crnn/" --use_angle_cls=true
W
WenmuZhou 已提交
384 385

# not use use direction classifier
W
WenmuZhou 已提交
386
python3 tools/infer/predict_system.py --image_dir="./doc/imgs/00018069.jpg" --det_model_dir="./inference/det_db/" --rec_model_dir="./inference/rec_crnn/"
littletomatodonkey's avatar
littletomatodonkey 已提交
387 388 389 390

# use multi-process
python3 tools/infer/predict_system.py --image_dir="./doc/imgs/00018069.jpg" --det_model_dir="./inference/det_db/" --rec_model_dir="./inference/rec_crnn/" --use_angle_cls=false --use_mp=True --total_process_num=6
```
K
Khanh Tran 已提交
391 392 393 394
```

After executing the command, the recognition result image is as follows:

W
WenmuZhou 已提交
395
![](../imgs_results/system_res_00018069.jpg)
K
Khanh Tran 已提交
396

L
licx 已提交
397
<a name="OTHER_MODELS"></a>
X
xxxpsyduck 已提交
398
### 2. OTHER MODELS
K
Khanh Tran 已提交
399

L
licx 已提交
400 401 402 403 404
If you want to try other detection algorithms or recognition algorithms, please refer to the above text detection model inference and text recognition model inference, update the corresponding configuration and model.

**Note: due to the limitation of rotation logic of detected box, SAST curved text detection model (using the parameter `det_sast_polygon=True`) is not supported for model combination yet.**

The following command uses the combination of the EAST text detection and STAR-Net text recognition:
K
Khanh Tran 已提交
405 406 407 408 409 410 411

```
python3 tools/infer/predict_system.py --image_dir="./doc/imgs_en/img_10.jpg" --det_model_dir="./inference/det_east/" --det_algorithm="EAST" --rec_model_dir="./inference/starnet/" --rec_image_shape="3, 32, 100" --rec_char_type="en"
```

After executing the command, the recognition result image is as follows:

W
WenmuZhou 已提交
412
![](../imgs_results/img_10_east_starnet.jpg)