README_en.md 3.3 KB
Newer Older
B
baiyfbupt 已提交
1

qq_25193841's avatar
qq_25193841 已提交
2
# PP-OCR Models Quantization
B
baiyfbupt 已提交
3

fanruinet's avatar
fanruinet 已提交
4
Generally, a more complex model would achieve better performance in the task, but it also leads to some redundancy in the model.
B
baiyfbupt 已提交
5 6 7
Quantization is a technique that reduces this redundancy by reducing the full precision data to a fixed number,
so as to reduce model calculation complexity and improve model inference performance.

A
andyjpaddle 已提交
8
This example uses PaddleSlim provided [APIs of Quantization](https://github.com/PaddlePaddle/PaddleSlim/blob/develop/docs/zh_cn/api_cn/dygraph/quanter/qat.rst) to compress the OCR model.
B
baiyfbupt 已提交
9 10 11

It is recommended that you could understand following pages before reading this example:
- [The training strategy of OCR model](../../../doc/doc_en/quickstart_en.md)
A
andyjpaddle 已提交
12
- [PaddleSlim Document](https://github.com/PaddlePaddle/PaddleSlim/blob/develop/docs/zh_cn/api_cn/dygraph/quanter/qat.rst)
B
baiyfbupt 已提交
13 14 15 16 17 18 19 20 21 22 23 24 25 26 27

## Quick Start
Quantization is mostly suitable for the deployment of lightweight models on mobile terminals.
After training, if you want to further compress the model size and accelerate the prediction, you can use quantization methods to compress the model according to the following steps.

1. Install PaddleSlim
2. Prepare trained model
3. Quantization-Aware Training
4. Export inference model
5. Deploy quantization inference model


### 1. Install PaddleSlim

```bash
28
pip3 install paddleslim==2.3.2
B
baiyfbupt 已提交
29 30 31
```


fanruinet's avatar
fanruinet 已提交
32 33
### 2. Download Pre-trained Model
PaddleOCR provides a series of pre-trained [models](../../../doc/doc_en/models_list_en.md).
B
baiyfbupt 已提交
34 35 36 37 38
If the model to be quantified is not in the list, you need to follow the [Regular Training](../../../doc/doc_en/quickstart_en.md) method to get the trained model.


### 3. Quant-Aware Training
Quantization training includes offline quantization training and online quantization training.
fanruinet's avatar
fanruinet 已提交
39
Online quantization training is more effective. It is necessary to load the pre-trained model.
B
baiyfbupt 已提交
40 41
After the quantization strategy is defined, the model can be quantified.

42
The code for quantization training is located in `slim/quantization/quant.py`. For example, the training instructions of slim PPOCRv3 detection model are as follows:
L
LDOUBLEV 已提交
43 44 45
```
# download provided model
wget https://paddleocr.bj.bcebos.com/PP-OCRv3/chinese/ch_PP-OCRv3_det_distill_train.tar
46
tar xf ch_PP-OCRv3_det_distill_train.tar
L
LDOUBLEV 已提交
47

48
python deploy/slim/quantization/quant.py -c configs/det/ch_PP-OCRv3/ch_PP-OCRv3_det_cml.yml -o Global.pretrained_model='./ch_PP-OCRv3_det_distill_train/best_accuracy'   Global.save_model_dir=./output/quant_model_distill/
L
LDOUBLEV 已提交
49 50 51 52
```

If you want to quantify the text recognition model, you can modify the configuration file and loaded model parameters.

B
baiyfbupt 已提交
53 54
### 4. Export inference model

fanruinet's avatar
fanruinet 已提交
55
Once we got the model after pruning and fine-tuning, we can export it as an inference model for the deployment of predictive tasks:
B
baiyfbupt 已提交
56 57

```bash
58
python deploy/slim/quantization/export_model.py -c configs/det/ch_PP-OCRv3/ch_PP-OCRv3_det_cml.yml -o Global.checkpoints=output/quant_model/best_accuracy Global.save_inference_dir=./output/quant_inference_model
B
baiyfbupt 已提交
59 60 61 62 63 64
```

### 5. Deploy
The numerical range of the quantized model parameters derived from the above steps is still FP32, but the numerical range of the parameters is int8.
The derived model can be converted through the `opt tool` of PaddleLite.

65
For quantitative model deployment, please refer to [Mobile terminal model deployment](../../lite/readme.md)