diff --git a/README_en.md b/README_en.md index dbba2065a979e7b90405c9eda740ac43ca98271c..462b20855e8fe21d8cd2800765848394c9811606 100644 --- a/README_en.md +++ b/README_en.md @@ -63,13 +63,13 @@ Quick experience of **P**ractical **U**ltra **L**ight-weight image **C**lassific - [PULC Model Zoo](docs/en/PULC/PULC_model_list_en.md) - [PULC Classification Model of Someone or Nobody](docs/en/PULC/PULC_person_exists_en.md) - [PULC Recognition Model of Person Attribute](docs/en/PULC/PULC_person_attribute_en.md) - + - [PULC Classification Model of Wearing or Unwearing Safety Helmet](docs/en/PULC/PULC_safety_helmet_en.md) - [PULC Classification Model of Traffic Sing](docs/en/PULC/PULC_traffic_sign_en.md) - [PULC Recognition Model of Vehicle Attribute](docs/en/PULC/PULC_vehicle_attribute_en.md) - [PULC Classification Model of Containing or Uncontaining Car](docs/en/PULC/PULC_car_exists_en.md) - [PULC Classification Model of Text Image Orientation](docs/en/PULC/PULC_text_image_orientation_en.md) - - + - [PULC Classification Model of Textline Orientation](docs/en/PULC/PULC_textline_orientation_en.md) + - [PULC Classification Model of Language](docs/en/PULC/PULC_language_classification_en.md) - [Quick Start of Recognition](./docs/en/tutorials/quick_start_recognition_en.md) - [Quick Start of Recognition](./docs/en/quick_start/quick_start_recognition_en.md) - [Introduction to Image Recognition Systems](#Introduction_to_Image_Recognition_Systems) diff --git a/docs/en/PULC/PULC_car_exists_en.md b/docs/en/PULC/PULC_car_exists_en.md index 7284fbe3414b43ee22ca166d3348851d01126dd3..3ec2e9d147881a3e9874718478a5eb4fd5228552 100644 --- a/docs/en/PULC/PULC_car_exists_en.md +++ b/docs/en/PULC/PULC_car_exists_en.md @@ -51,7 +51,7 @@ The following table lists the relevant indicators of the model. The first two li | PPLCNet_x1_0 | 95.48 | 2.12 | 6.5 | using SSLD pretrained model + EDA strategy | | PPLCNet_x1_0 | 95.92 | 2.12 | 6.5 | using SSLD pretrained model + EDA strategy + SKL-UGI knowledge distillation strategy| -It can be seen that high Tpr can be getted when backbone is SwinTranformer_tiny, but the speed is slow. Replacing backbone with the lightweight model MobileNetV3_small_x0_35, the speed can be greatly improved, but the Tpr will be greatly reduced. Replacing backbone with faster backbone PPLCNet_x1_0, the Tpr is higher more 13 percentage points than MobileNetv3_small_x0_35. At the same time, the speed can be more than 20% faster. After additional using the SSLD pretrained model, the Tpr can be improved by about 0.7 percentage points without affecting the inference speed. Finally, after additional using the SKL-UGI knowledge distillation, the Tpr can be further improved by 0.44 percentage points. At this point, the Tpr close to that of SwinTranformer_tiny, but the speed is more than 40 times faster. The training method and deployment instructions of PULC will be introduced in detail below. +It can be seen that high Tpr can be getted when backbone is SwinTranformer_tiny, but the speed is slow. Replacing backbone with the lightweight model MobileNetV3_small_x0_35, the speed can be greatly improved, but the Tpr will be greatly reduced. Replacing backbone with faster backbone PPLCNet_x1_0, the Tpr is higher more 13 percentage points than MobileNetv3_small_x0_35. At the same time, the speed can be more than 20% faster. After additional using the SSLD pretrained model, the Tpr can be improved by about 0.7 percentage points without affecting the inference speed. Finally, after additional using the SKL-UGI knowledge distillation, the Tpr can be further improved by 0.44 percentage points. At this point, the Tpr is close to that of SwinTranformer_tiny, but the speed is more than 40 times faster. The training method and deployment instructions of PULC will be introduced in detail below. **Note**: diff --git a/docs/en/PULC/PULC_language_classification_en.md b/docs/en/PULC/PULC_language_classification_en.md new file mode 100644 index 0000000000000000000000000000000000000000..48362b316179686bf85e2e54f21339970dac8a43 --- /dev/null +++ b/docs/en/PULC/PULC_language_classification_en.md @@ -0,0 +1,470 @@ +# PULC Classification Model of Language + +------ + +## Catalogue + +- [1. Introduction](#1) +- [2. Quick Start](#2) + - [2.1 PaddlePaddle Installation](#2.1) + - [2.2 PaddleClas Installation](#2.2) + - [2.3 Prediction](#2.3) +- [3. Training, Evaluation and Inference](#3) + - [3.1 Installation](#3.1) + - [3.2 Dataset](#3.2) + - [3.2.1 Dataset Introduction](#3.2.1) + - [3.2.2 Getting Dataset](#3.2.2) + - [3.3 Training](#3.3) + - [3.4 Evaluation](#3.4) + - [3.5 Inference](#3.5) +- [4. Model Compression](#4) + - [4.1 SKL-UGI Knowledge Distillation](#4.1) + - [4.1.1 Teacher Model Training](#4.1.1) + - [4.1.2 Knowledge Distillation Training](#4.1.2) +- [5. SHAS](#5) +- [6. Inference Deployment](#6) + - [6.1 Getting Paddle Inference Model](#6.1) + - [6.1.1 Exporting Paddle Inference Model](#6.1.1) + - [6.1.2 Downloading Inference Model](#6.1.2) + - [6.2 Prediction with Python](#6.2) + - [6.2.1 Image Prediction](#6.2.1) + - [6.2.2 Images Prediction](#6.2.2) + - [6.3 Deployment with C++](#6.3) + - [6.4 Deployment as Service](#6.4) + - [6.5 Deployment on Mobile](#6.5) + - [6.6 Converting To ONNX and Deployment](#6.6) + + + +## 1. Introduction + +This case provides a way for users to quickly build a lightweight, high-precision and practical classification model of language in the image using PaddleClas PULC (Practical Ultra Lightweight Classification). The model can be widely used in various scenarios involving multilingual OCR processing, such as finance and government affairs. + +The following table lists the relevant indicators of the model. The first two lines means that using SwinTransformer_tiny and MobileNetV3_small_x0_35 as the backbone to training. The third to sixth lines means that the backbone is replaced by PPLCNet, additional use of EDA strategy and additional use of EDA strategy and SKL-UGI knowledge distillation strategy. When replacing the backbone with PPLCNet_x1_0, the input shape of model is changed to [192, 48], and the stride of the network is changed to [2, [2, 1], [2, 1], [2, 1]]. + +| Backbone | Top1-Acc(%) | Latency(ms) | Size(M)| Training Strategy | +| ----------------------- | --------- | -------- | ------- | ---------------------------------------------- | +| SwinTranformer_tiny | 98.12 | 89.09 | 107 | using ImageNet pretrained model | +| MobileNetV3_small_x0_35 | 95.92 | 2.98 | 17 | using ImageNet pretrained model | +| PPLCNet_x1_0 | 98.35 | 2.58 | 6.5 | using ImageNet pretrained model | +| PPLCNet_x1_0 | 98.7 | 2.58 | 6.5 | using SSLD pretrained model | +| PPLCNet_x1_0 | 99.12 | 2.58 | 6.5 | using SSLD pretrained model + EDA strategy | +| **PPLCNet_x1_0** | **99.26** | **2.58** | **6.5** | using SSLD pretrained model + EDA strategy + SKL-UGI knowledge distillation strategy| + +It can be seen that high accuracy can be getted when backbone is SwinTranformer_tiny, but the speed is slow. Replacing backbone with the lightweight model MobileNetV3_small_x0_35, the speed can be greatly improved, but the accuracy will be greatly reduced. Replacing backbone with faster backbone PPLCNet_x1_0 and changing the input shape and stride of network, the accuracy is higher more 2.43 percentage points than MobileNetv3_small_x0_35. At the same time, the speed can be more than 20% faster. After additional using the SSLD pretrained model, the accuracy can be improved by about 0.35 percentage points without affecting the inference speed. Further, additional using the EDA strategy, the accuracy can be increased by 0.42 percentage points. Finally, after additional using the SKL-UGI knowledge distillation, the accuracy can be further improved by 0.14 percentage points. At this point, the accuracy is higher than that of SwinTranformer_tiny, but the speed is more faster. The training method and deployment instructions of PULC will be introduced in detail below. + +**Note**: + +* The Latency is tested on Intel(R) Xeon(R) Gold 6148 CPU @ 2.40GHz. The MKLDNN is enabled and the number of threads is 10. +* About PP-LCNet, please refer to [PP-LCNet Introduction](../models/PP-LCNet_en.md) and [PP-LCNet Paper](https://arxiv.org/abs/2109.15099). + + + +## 2. Quick Start + + + +### 2.1 PaddlePaddle Installation + +- Run the following command to install if CUDA9 or CUDA10 is available. + +```bash +python3 -m pip install paddlepaddle-gpu -i https://mirror.baidu.com/pypi/simple +``` + +- Run the following command to install if GPU device is unavailable. + +```bash +python3 -m pip install paddlepaddle -i https://mirror.baidu.com/pypi/simple +``` + +Please refer to [PaddlePaddle Installation](https://www.paddlepaddle.org.cn/install/quick?docurl=/documentation/docs/en/install/pip/linux-pip_en.html) for more information about installation, for examples other versions. + + + +### 2.2 PaddleClas wheel Installation + +The command of PaddleClas installation as bellow: + +```bash +pip3 install paddleclas +``` + + + +### 2.3 Prediction + +First, please click [here](https://paddleclas.bj.bcebos.com/data/PULC/pulc_demo_imgs.zip) to download and unzip to get the test demo images. + +* Prediction with CLI + +```bash +paddleclas --model_name=language_classification --infer_imgs=pulc_demo_imgs/language_classification/word_35404.png +``` + +Results: + +``` +>>> result +class_ids: [4, 6], scores: [0.88672, 0.01434], label_names: ['japan', 'korean'], filename: pulc_demo_imgs/language_classification/word_35404.png +Predict complete! +``` + +**Note**: If you want to test other images, only need to specify the `--infer_imgs` argument, and the directory containing images is also supported. + +* Prediction in Python + +```python +import paddleclas +model = paddleclas.PaddleClas(model_name="language_classification") +result = model.predict(input_data="pulc_demo_imgs/language_classification/word_35404.png") +print(next(result)) +``` + +**Note**: The `result` returned by `model.predict()` is a generator, so you need to use the `next()` function to call it or `for` loop to loop it. And it will predict with `batch_size` size batch and return the prediction results when called. The default `batch_size` is 1, and you also specify the `batch_size` when instantiating, such as `model = paddleclas.PaddleClas(model_name="language_classification", batch_size=2)`. The result of demo above: + +``` +>>> result +[{'class_ids': [4, 6], 'scores': [0.88672, 0.01434], 'label_names': ['japan', 'korean'], 'filename': 'pulc_demo_imgs/language_classification/word_35404.png'}] +``` + + + +## 3. Training, Evaluation and Inference + + + +### 3.1 Installation + +Please refer to [Installation](../installation/install_paddleclas_en.md) to get the description about installation. + + + +### 3.2 Dataset + + + +#### 3.2.1 Dataset Introduction + +The models wo provided are trained with internal data, which is not open source yet. So it is suggested that constructing dataset based on open source dataset [Multi-lingual scene text detection and recognition](https://rrc.cvc.uab.es/?ch=15&com=downloads) to experience the this case. + +Some image of the processed dataset is as follows: + +![](../../images/PULC/docs/language_classification_original_data.png) + + + +#### 3.2.2 Getting Dataset + +The models provided support to classcify 10 languages, which as shown in the following list: + +`0` : means Arabic +`1` : means chinese_cht +`2` : means cyrillic +`3` : means devanagari +`4` : means Japanese +`5` : means ka +`6` : means Korean +`7` : means ta +`8` : means te +`9` : means Latin + +In the `Multi-lingual scene text detection and recognition`, only Arabic, Japanese, Korean and Latin data are included. 1600 images from each of the four languages are taken as the training data of this case, 300 images as the evaluation data, and 400 images as the supplementary data is used for the `SKL-UGI Knowledge Distillation`. + +Therefore, for the demo dataset in this case, the language categories are shown in following list: +`0` : means arabic +`4` : means japan +`6` : means korean +`9` : means latin + +**Note**: The images used in this task should be cropped by text from original image. Only the text line part is used as the image data. + +If you want to create your own dataset, you can collect and sort out the data of the required languages in your task as required. And you can also download the data processed directly. + +``` +cd path_to_PaddleClas +``` + +Enter the `dataset/` directory, download and unzip the dataset. + +```shell +cd dataset +wget https://paddleclas.bj.bcebos.com/data/PULC/language_classification.tar +tar -xf language_classification.tar +cd ../ +``` + +The datas under `language_classification` directory: + +``` +├── img +│ ├── word_1.png +│ ├── word_2.png +... +├── train_list.txt +├── train_list_for_distill.txt +├── test_list.txt +└── label_list.txt +``` + +Where `img/` is the directory including 9200 images in 4 languages. The `train_list.txt` and `test_list.txt` are label files of training data and validation data respectively. `label_list.txt` is the mapping file corresponding to the four languages. `train_list_for_distill.txt` is the label list of images used for `SKL-UGI Knowledge Distillation`. + +**Note**: + +* About the contents format of `train_list.txt` and `val_list.txt`, please refer to [Description about Classification Dataset in PaddleClas](../data_preparation/classification_dataset_en.md). +* About the `train_list_for_distill.txt`, please refer to [Knowledge Distillation Label](../advanced_tutorials/distillation/distillation_en.md). + + + +### 3.3 Training + +The details of training config in `ppcls/configs/PULC/person_exists/PPLCNet_x1_0.yaml`. The command about training as follows: + +```shell +export CUDA_VISIBLE_DEVICES=0,1,2,3 +python3 -m paddle.distributed.launch \ + --gpus="0,1,2,3" \ + tools/train.py \ + -c ./ppcls/configs/PULC/language_classification/PPLCNet_x1_0.yaml \ + -o Arch.class_num=4 +``` + +**Note**: Because the class num of demo dataset is 4, the argument `-o Arch.class_num=4` should be specifed to change the prediction class num of model to 4. + + + +### 3.4 Evaluation + +After training, you can use the following commands to evaluate the model. + +```bash +python3 tools/eval.py \ + -c ./ppcls/configs/PULC/language_classification/PPLCNet_x1_0.yaml \ + -o Global.pretrained_model="output/PPLCNet_x1_0/best_model" \ + -o Arch.class_num=4 +``` + +Among the above command, the argument `-o Global.pretrained_model="output/PPLCNet_x1_0/best_model"` specify the path of the best model weight file. You can specify other path if needed. + + + +### 3.5 Inference + +After training, you can use the model that trained to infer. Command is as follow: + +```bash +python3 tools/infer.py \ + -c ./ppcls/configs/PULC/language_classification/PPLCNet_x1_0.yaml \ + -o Global.pretrained_model="output/PPLCNet_x1_0/best_model" \ + -o Arch.class_num=4 +``` + +The results: + +``` +[{'class_ids': [4, 9], 'scores': [0.96809, 0.01001], 'file_name': 'deploy/images/PULC/language_classification/word_35404.png', 'label_names': ['japan', 'latin']}] +``` + +**Note**: + +* Among the above command, argument `-o Global.pretrained_model="output/PPLCNet_x1_0/best_model"` specify the path of the best model weight file. You can specify other path if needed. +* The default test image is `deploy/images/PULC/person_exists/objects365_02035329.jpg`. And you can test other image, only need to specify the argument `-o Infer.infer_imgs=path_to_test_image`. +* Among the prediction results, `japan` means japanese and `korean` means korean. + + + +## 4. Model Compression + + + +### 4.1 SKL-UGI Knowledge Distillation + +SKL-UGI is a simple but effective knowledge distillation algrithem proposed by PaddleClas. + + + + + + +#### 4.1.1 Teacher Model Training + +Training the teacher model with hyperparameters specified in `ppcls/configs/PULC/language_classification/PPLCNet/PPLCNet_x1_0.yaml`. The command is as follow: + +```shell +export CUDA_VISIBLE_DEVICES=0,1,2,3 +python3 -m paddle.distributed.launch \ + --gpus="0,1,2,3" \ + tools/train.py \ + -c ./ppcls/configs/PULC/language_classification/PPLCNet_x1_0.yaml \ + -o Arch.name=ResNet101_vd \ + -o Arch.class_num=4 +``` + +The best teacher model weight would be saved in file `output/ResNet101_vd/best_model.pdparams`. + +**Note**: Training the ResNet101_vd model requires more GPU memory. If the memory is not enough, you can reduce the learning rate and batch size in the same proportion. + + + +#### 4.1.2 Knowledge Distillation Training + +The training strategy, specified in training config file `ppcls/configs/PULC/language_classification/PPLCNet_x1_0_distillation.yaml`, the teacher model is `ResNet101_vd`, the student model is `PPLCNet_x1_0`. The command is as follow: + +```shell +export CUDA_VISIBLE_DEVICES=0,1,2,3 +python3 -m paddle.distributed.launch \ + --gpus="0,1,2,3" \ + tools/train.py \ + -c ./ppcls/configs/PULC/language_classification/PPLCNet_x1_0_distillation.yaml \ + -o Arch.models.0.Teacher.pretrained=output/ResNet101_vd/best_model \ + -o Arch.class_num=4 +``` + +The best student model weight would be saved in file `output/DistillationModel/best_model_student.pdparams`. + + + +## 5. Hyperparameters Searching + +The hyperparameters used by [3.2 section](#3.2) and [4.1 section](#4.1) are according by `Hyperparameters Searching` in PaddleClas. If you want to get better results on your own dataset, you can refer to [Hyperparameters Searching](PULC_train_en.md#4) to get better hyperparameters. + +**Note**: This section is optional. Because the search process will take a long time, you can selectively run according to your specific. If not replace the dataset, you can ignore this section. + + + +## 6. Inference Deployment + + + +### 6.1 Getting Paddle Inference Model + +Paddle Inference is the original Inference Library of the PaddlePaddle, provides high-performance inference for server deployment. And compared with directly based on the pretrained model, Paddle Inference can use tools to accelerate prediction, so as to achieve better inference performance. Please refer to [Paddle Inference](https://www.paddlepaddle.org.cn/documentation/docs/zh/guides/infer/inference/inference_cn.html) for more information. + +Paddle Inference need Paddle Inference Model to predict. Two process provided to get Paddle Inference Model. If want to use the provided by PaddleClas, you can download directly, click [Downloading Inference Model](#6.1.2). + + + +### 6.1.1 Exporting Paddle Inference Model + +The command about exporting Paddle Inference Model is as follow: + +```bash +python3 tools/export_model.py \ + -c ./ppcls/configs/PULC/language_classification/PPLCNet_x1_0.yaml \ + -o Global.pretrained_model=output/DistillationModel/best_model_student \ + -o Global.save_inference_dir=deploy/models/PPLCNet_x1_0_language_classification_infer +``` + +After running above command, the inference model files would be saved in `deploy/models/PPLCNet_x1_0_language_classification_infer`, as shown below: + +``` +├── PPLCNet_x1_0_language_classification_infer +│ ├── inference.pdiparams +│ ├── inference.pdiparams.info +│ └── inference.pdmodel +``` + +**Note**: The best model is from knowledge distillation training. If knowledge distillation training is not used, the best model would be saved in `output/PPLCNet_x1_0/best_model.pdparams`. + + + +### 6.1.2 Downloading Inference Model + +You can also download directly. + +``` +cd deploy/models +# download the inference model and decompression +wget https://paddleclas.bj.bcebos.com/models/PULC/language_classification_infer.tar && tar -xf language_classification_infer.tar +``` + +After decompression, the directory `models` should be shown below. + +``` +├── language_classification_infer +│ ├── inference.pdiparams +│ ├── inference.pdiparams.info +│ └── inference.pdmodel +``` + + + +### 6.2 Prediction with Python + + + +#### 6.2.1 Image Prediction + +Return the directory `deploy`: + +``` +cd ../ +``` + +Run the following command to classify language about the image `./images/PULC/language_classification/word_35404.png`. + +```shell +# Use the following command to predict with GPU. +python3.7 python/predict_cls.py -c configs/PULC/language_classification/inference_language_classification.yaml +# Use the following command to predict with CPU. +python3.7 python/predict_cls.py -c configs/PULC/language_classification/inference_language_classification.yaml -o Global.use_gpu=False +``` + +The prediction results: + +``` +word_35404.png: class id(s): [4, 6], score(s): [0.89, 0.01], label_name(s): ['japan', 'korean'] +``` + +**Note**: Among the prediction results, `japan` means japanese and `korean` means korean. + + + +#### 6.2.2 Images Prediction + +If you want to predict images in directory, please specify the argument `Global.infer_imgs` as directory path by `-o Global.infer_imgs`. The command is as follow. + +```shell +# Use the following command to predict with GPU. If want to replace with CPU, you can add argument -o Global.use_gpu=False +python3.7 python/predict_cls.py -c configs/PULC/language_classification/inference_language_classification.yaml -o Global.infer_imgs="./images/PULC/language_classification/" +``` + +All prediction results will be printed, as shown below. + +``` +word_17.png: class id(s): [9, 4], score(s): [0.80, 0.09], label_name(s): ['latin', 'japan'] +word_20.png: class id(s): [0, 4], score(s): [0.91, 0.02], label_name(s): ['arabic', 'japan'] +word_35404.png: class id(s): [4, 6], score(s): [0.89, 0.01], label_name(s): ['japan', 'korean'] +``` + +Among the prediction results above, `japan` means japanese, `latin` means latin, `arabic` means arabic and `korean` means korean. + + + +### 6.3 Deployment with C++ + +PaddleClas provides an example about how to deploy with C++. Please refer to [Deployment with C++](../inference_deployment/cpp_deploy_en.md). + + + +### 6.4 Deployment as Service + +Paddle Serving is a flexible, high-performance carrier for machine learning models, and supports different protocol, such as RESTful, gRPC, bRPC and so on, which provides different deployment solutions for a variety of heterogeneous hardware and operating system environments. Please refer [Paddle Serving](https://github.com/PaddlePaddle/Serving) for more information. + +PaddleClas provides an example about how to deploy as service by Paddle Serving. Please refer to [Paddle Serving Deployment](../inference_deployment/paddle_serving_deploy_en.md). + + + +### 6.5 Deployment on Mobile + +Paddle-Lite is an open source deep learning framework that designed to make easy to perform inference on mobile, embeded, and IoT devices. Please refer to [Paddle-Lite](https://github.com/PaddlePaddle/Paddle-Lite) for more information. + +PaddleClas provides an example of how to deploy on mobile by Paddle-Lite. Please refer to [Paddle-Lite deployment](../inference_deployment/paddle_lite_deploy_en.md). + + + +### 6.6 Converting To ONNX and Deployment + +Paddle2ONNX support convert Paddle Inference model to ONNX model. And you can deploy with ONNX model on different inference engine, such as TensorRT, OpenVINO, MNN/TNN, NCNN and so on. About Paddle2ONNX details, please refer to [Paddle2ONNX](https://github.com/PaddlePaddle/Paddle2ONNX). + +PaddleClas provides an example of how to convert Paddle Inference model to ONNX model by paddle2onnx toolkit and predict by ONNX model. You can refer to [paddle2onnx](../../../deploy/paddle2onnx/readme_en.md) for deployment details. diff --git a/docs/en/PULC/PULC_person_exists_en.md b/docs/en/PULC/PULC_person_exists_en.md index 93f583392b1890aa8a6c7d26affe33e4c4a7935a..21829e5544d8bdfdf2ecc01616f8b8912a48a08d 100644 --- a/docs/en/PULC/PULC_person_exists_en.md +++ b/docs/en/PULC/PULC_person_exists_en.md @@ -51,7 +51,7 @@ The following table lists the relevant indicators of the model. The first two li | PPLCNet_x1_0 | 93.43 | 2.12 | 6.5 | using SSLD pretrained model + EDA strategy | | PPLCNet_x1_0 | 95.60 | 2.12 | 6.5 | using SSLD pretrained model + EDA strategy + SKL-UGI knowledge distillation strategy| -It can be seen that high Tpr can be getted when backbone is SwinTranformer_tiny, but the speed is slow. Replacing backbone with the lightweight model MobileNetV3_small_x0_35, the speed can be greatly improved, but the Tpr will be greatly reduced. Replacing backbone with faster backbone PPLCNet_x1_0, the Tpr is higher more 20 percentage points than MobileNetv3_small_x0_35. At the same time, the speed can be more than 20% faster. After additional using the SSLD pretrained model, the Tpr can be improved by about 2.6 percentage points without affecting the inference speed. Further, additional using the EDA strategy, the Tpr can be increased by 1.3 percentage points. Finally, after additional using the SKL-UGI knowledge distillation, the Tpr can be further improved by 2.2 percentage points. At this point, the Tpr close to that of SwinTranformer_tiny, but the speed is more than 40 times faster. The training method and deployment instructions of PULC will be introduced in detail below. +It can be seen that high Tpr can be getted when backbone is SwinTranformer_tiny, but the speed is slow. Replacing backbone with the lightweight model MobileNetV3_small_x0_35, the speed can be greatly improved, but the Tpr will be greatly reduced. Replacing backbone with faster backbone PPLCNet_x1_0, the Tpr is higher more 20 percentage points than MobileNetv3_small_x0_35. At the same time, the speed can be more than 20% faster. After additional using the SSLD pretrained model, the Tpr can be improved by about 2.6 percentage points without affecting the inference speed. Further, additional using the EDA strategy, the Tpr can be increased by 1.3 percentage points. Finally, after additional using the SKL-UGI knowledge distillation, the Tpr can be further improved by 2.2 percentage points. At this point, the Tpr is close to that of SwinTranformer_tiny, but the speed is more than 40 times faster. The training method and deployment instructions of PULC will be introduced in detail below. **Note**: diff --git a/docs/en/PULC/PULC_safety_helmet_en.md b/docs/en/PULC/PULC_safety_helmet_en.md new file mode 100644 index 0000000000000000000000000000000000000000..91f8b76f68c8788dff1787b68fd72b14de33ccc6 --- /dev/null +++ b/docs/en/PULC/PULC_safety_helmet_en.md @@ -0,0 +1,432 @@ +# PULC Classification Model of Wheather Wearing Safety Helmet or Not + +----- + +## Catalogue + +- [1. Introduction](#1) +- [2. Quick Start](#2) + - [2.1 PaddlePaddle Installation](#2.1) + - [2.2 PaddleClas Installation](#2.2) + - [2.3 Prediction](#2.3) +- [3. Training, Evaluation and Inference](#3) + - [3.1 Installation](#3.1) + - [3.2 Dataset](#3.2) + - [3.2.1 Dataset Introduction](#3.2.1) + - [3.2.2 Getting Dataset](#3.2.2) + - [3.3 Training](#3.3) + - [3.4 Evaluation](#3.4) + - [3.5 Inference](#3.5) +- [4. Model Compression](#4) + - [4.1 SKL-UGI Knowledge Distillation](#4.1) + - [4.1.1 Teacher Model Training](#4.1.1) + - [4.1.2 Knowledge Distillation Training](#4.1.2) +- [5. SHAS](#5) +- [6. Inference Deployment](#6) + - [6.1 Getting Paddle Inference Model](#6.1) + - [6.1.1 Exporting Paddle Inference Model](#6.1.1) + - [6.1.2 Downloading Inference Model](#6.1.2) + - [6.2 Prediction with Python](#6.2) + - [6.2.1 Image Prediction](#6.2.1) + - [6.2.2 Images Prediction](#6.2.2) + - [6.3 Deployment with C++](#6.3) + - [6.4 Deployment as Service](#6.4) + - [6.5 Deployment on Mobile](#6.5) + - [6.6 Converting To ONNX and Deployment](#6.6) + + + +## 1. Introduction + +This case provides a way for users to quickly build a lightweight, high-precision and practical classification model of wheather wearing safety helmet using PaddleClas PULC (Practical Ultra Lightweight Classification). The model can be widely used in construction scenes, factory workshop scenes, traffic scenes and so on. + +The following table lists the relevant indicators of the model. The first two lines means that using SwinTransformer_tiny and MobileNetV3_small_x0_35 as the backbone to training. The third to seventh lines means that the backbone is replaced by PPLCNet, additional use of EDA strategy and additional use of EDA strategy and SKL-UGI knowledge distillation strategy. + +| Backbone | Tpr(%) | Latency(ms) | Size(M)| Training Strategy | +|-------|-----------|----------|---------------|---------------| +| SwinTranformer_tiny | 93.57 | 91.32 | 107 | using ImageNet pretrained model | +| Res2Net200_vd_26w_4s | 98.92 | 80.99 | 284 | using ImageNet pretrained model | +| MobileNetV3_small_x0_35 | 84.83 | 2.85 | 1.6 | using ImageNet pretrained model | +| PPLCNet_x1_0 | 93.27 | 2.03 | 6.5 | using ImageNet pretrained model | +| PPLCNet_x1_0 | 98.16 | 2.03 | 6.5 | using SSLD pretrained model | +| PPLCNet_x1_0 | 99.30 | 2.03 | 6.5 | using SSLD pretrained model + EDA strategy | +| PPLCNet_x1_0 | 99.38 | 2.03 | 6.5 | using SSLD pretrained model + EDA strategy + SKL-UGI knowledge distillation strategy| + +It can be seen that high Tpr can be getted when backbone is Res2Net200_vd_26w_4s, but the speed is slow. Replacing backbone with the lightweight model MobileNetV3_small_x0_35, the speed can be greatly improved, but the Tpr will be greatly reduced. Replacing backbone with faster backbone PPLCNet_x1_0, the Tpr is higher more 8.5 percentage points than MobileNetv3_small_x0_35. At the same time, the speed can be more than 20% faster. After additional using the SSLD pretrained model, the Tpr can be improved by about 4.9 percentage points without affecting the inference speed. Further, additional using the EDA strategy, the Tpr can be increased by 1.1 percentage points. Finally, after additional using the UDML knowledge distillation, the Tpr can be further improved by 2.2 percentage points. At this point, the Tpr is higher than that of Res2Net200_vd_26w_4s, but the speed is more than 70 times faster. The training method and deployment instructions of PULC will be introduced in detail below. + +**Note**: + +* About `Tpr` metric, please refer to [3.2 section](#3.2) for more information . +* The Latency is tested on Intel(R) Xeon(R) Gold 6148 CPU @ 2.40GHz. The MKLDNN is enabled and the number of threads is 10. +* About PP-LCNet, please refer to [PP-LCNet Introduction](../models/PP-LCNet_en.md) and [PP-LCNet Paper](https://arxiv.org/abs/2109.15099). + + + +## 2. Quick Start + + + +### 2.1 PaddlePaddle Installation + +- Run the following command to install if CUDA9 or CUDA10 is available. + +```bash +python3 -m pip install paddlepaddle-gpu -i https://mirror.baidu.com/pypi/simple +``` + +- Run the following command to install if GPU device is unavailable. + +```bash +python3 -m pip install paddlepaddle -i https://mirror.baidu.com/pypi/simple +``` + +Please refer to [PaddlePaddle Installation](https://www.paddlepaddle.org.cn/install/quick?docurl=/documentation/docs/en/install/pip/linux-pip_en.html) for more information about installation, for examples other versions. + + + +### 2.2 PaddleClas wheel Installation + +The command of PaddleClas installation as bellow: + +```bash +pip3 install paddleclas +``` + + + +### 2.3 Prediction + +First, please click [here](https://paddleclas.bj.bcebos.com/data/PULC/pulc_demo_imgs.zip) to download and unzip to get the test demo images. + +* Prediction with CLI + +```bash +paddleclas --model_name=safety_helmet --infer_imgs=pulc_demo_imgs/safety_helmet/safety_helmet_test_1.png +``` + +Results: + +``` +>>> result +class_ids: [1], scores: [0.9986255], label_names: ['unwearing_helmet'], filename: pulc_demo_imgs/safety_helmet/safety_helmet_test_1.png +Predict complete! +``` +**Note**: If you want to test other images, only need to specify the `--infer_imgs` argument, and the directory containing images is also supported. + +* Prediction in Python + +```python +import paddleclas +model = paddleclas.PaddleClas(model_name="safety_helmet") +result = model.predict(input_data="pulc_demo_imgs/safety_helmet/safety_helmet_test_1.png") +print(next(result)) +``` + +**Note**: The `result` returned by `model.predict()` is a generator, so you need to use the `next()` function to call it or `for` loop to loop it. And it will predict with `batch_size` size batch and return the prediction results when called. The default `batch_size` is 1, and you also specify the `batch_size` when instantiating, such as `model = paddleclas.PaddleClas(model_name="safety_helmet", batch_size=2)`. The result of demo above: + +``` +>>> result +[{'class_ids': [1], 'scores': [0.9986255], 'label_names': ['unwearing_helmet'], 'filename': 'pulc_demo_imgs/safety_helmet/safety_helmet_test_1.png'}] +``` + + + +## 3. Training, Evaluation and Inference + + + +### 3.1 Installation + +Please refer to [Installation](../installation/install_paddleclas_en.md) to get the description about installation. + + + +### 3.2 Dataset + + + +#### 3.2.1 Dataset Introduction + +All datasets used in this case are open source data. Train data is the subset of [Safety-Helmet-Wearing-Dataset](https://github.com/njvisionpower/Safety-Helmet-Wearing-Dataset), [hard-hat-detection](https://www.kaggle.com/datasets/andrewmvd/hard-hat-detection) and [Large-scale CelebFaces Attributes (CelebA) Dataset](https://mmlab.ie.cuhk.edu.hk/projects/CelebA.html). + + + +#### 3.2.2 Getting Dataset + +The data used in this case can be getted by processing the open source data. The detailed processes are as follows: + +* `Safety-Helmet-Wearing-Dataset`: according to the bbox label data, the image is cropped by enlarging width and height of bbox by 3 times. The label is 0 if wearing safety helmet in the image, and the label is 1 if not; +* `hard-hat-detection`: Only use the image that labeled "hat" and crop it with bbox. The label is 0; +* `CelebA`: Only use the image labeled "wearing_hat" and crop it with bbox. The label is 0; + +After processing, the dataset totals about 150000 images, of which the number of images with and without wearing safety helmet is about 28000 and 121000 respectively. Then 5600 images are randomly selected in the two labels as the valuation data, a total of about 11200 images, and about 138000 other images as the training data. + +Some image of the processed dataset is as follows: + +![](../../images/PULC/docs/safety_helmet_data_demo.jpg) + +And you can also download the data processed directly. + +``` +cd path_to_PaddleClas +``` + +Enter the `dataset/` directory, download and unzip the dataset. + +```shell +cd dataset +wget https://paddleclas.bj.bcebos.com/data/PULC/safety_helmet.tar +tar -xf safety_helmet.tar +cd ../ +``` + +The datas under `safety_helmet` directory: + +``` +├── images +│   ├── VOC2028_part2_001209_1.jpg +│   ├── HHD_hard_hat_workers23_1.jpg +│   ├── CelebA_077809.jpg +│   ├── ... +│   └── ... +├── train_list.txt +└── val_list.txt +``` + +The `train_list.txt` and `val_list.txt` are label files of training data and validation data respectively. All images in `images/` directory. + +**Note**: + +* About the contents format of `train_list.txt` and `val_list.txt`, please refer to [Description about Classification Dataset in PaddleClas](../data_preparation/classification_dataset_en.md). + + + +### 3.3 Training + +The details of training config in `ppcls/configs/PULC/person_exists/PPLCNet_x1_0.yaml`. The command about training as follows: + +```shell +export CUDA_VISIBLE_DEVICES=0,1,2,3 +python3 -m paddle.distributed.launch \ + --gpus="0,1,2,3" \ + tools/train.py \ + -c ./ppcls/configs/PULC/safety_helmet/PPLCNet_x1_0.yaml +``` + +The best metric of validation data is between `0.985` and `0.993`. There would be fluctuations because the data size is small. + +**Note**: + +* The metric Tpr, that describe the True Positive Rate when False Positive Rate is less than a certain threshold(1/10000 used in this case), is one of the commonly used metric for binary classification. About the details of Fpr and Tpr, please refer [here](https://en.wikipedia.org/wiki/Receiver_operating_characteristic). +* When evaluation, the best metric TprAtFpr will be printed that include `Fpr`, `Tpr` and the current `threshold`. The `Tpr` means the Recall rate under the current `Fpr`. The `Tpr` higher, the model better. The `threshold` would be used in deployment, which means the classification threshold under best `Fpr` metric. + + + +### 3.4 Evaluation + +After training, you can use the following commands to evaluate the model. + + +```bash +python3 tools/eval.py \ + -c ./ppcls/configs/PULC/safety_helmet/PPLCNet_x1_0.yaml \ + -o Global.pretrained_model=output/PPLCNet_x1_0/best_model +``` + +Among the above command, the argument `-o Global.pretrained_model="output/PPLCNet_x1_0/best_model"` specify the path of the best model weight file. You can specify other path if needed. + + + +### 3.5 Inference + +After training, you can use the model that trained to infer. Command is as follow: + +```python +python3 tools/infer.py \ + -c ./ppcls/configs/PULC/safety_helmet/PPLCNet_x1_0.yaml \ + -o Global.pretrained_model=output/PPLCNet_x1_0/best_model +``` + +The results: + +``` +[{'class_ids': [1], 'scores': [0.9524797], 'label_names': ['unwearing_helmet'], 'file_name': 'deploy/images/PULC/safety_helmet/safety_helmet_test_1.png'}] +``` + +**备注：** + +* Among the above command, argument `-o Global.pretrained_model="output/PPLCNet_x1_0/best_model"` specify the path of the best model weight file. You can specify other path if needed. +* The default test image is `deploy/images/PULC/safety_helmet/safety_helmet_test_1.png`. And you can test other image, only need to specify the argument `-o Infer.infer_imgs=path_to_test_image`. +* The default threshold is `0.5`. If needed, you can specify the argument `Infer.PostProcess.threshold`, such as: `-o Infer.PostProcess.threshold=0.9167`. And the argument `threshold` is needed to be specified according by specific case. The `0.9167` is the best threshold when `Fpr` is less than `1/10000` in this valuation dataset. + + + +## 4. Model Compression + + + +### 4.1 UDML Knowledge Distillation + +UDML is a simple but effective knowledge distillation algrithem proposed by PaddleClas. Please refer to [UDML 知识蒸馏](../advanced_tutorials/knowledge_distillation_en.md#1.2.3) for more details. + + + +#### 4.1.1 Knowledge Distillation Training + +Training with hyperparameters specified in `ppcls/configs/PULC/safety_helmet/PPLCNet_x1_0_distillation.yaml`. The command is as follow: + +```shell +export CUDA_VISIBLE_DEVICES=0,1,2,3 +python3 -m paddle.distributed.launch \ + --gpus="0,1,2,3" \ + tools/train.py \ + -c ./ppcls/configs/PULC/safety_helmet/PPLCNet_x1_0_distillation.yaml +``` + +The best metric is between `0.990` and `0.993`. The best student model weight would be saved in file `output/DistillationModel/best_model_student.pdparams`. + + + +## 5. Hyperparameters Searching + +The hyperparameters used by [3.2 section](#3.2) and [4.1 section](#4.1) are according by `Hyperparameters Searching` in PaddleClas. If you want to get better results on your own dataset, you can refer to [Hyperparameters Searching](PULC_train_en.md#4) to get better hyperparameters. + +**Note**: This section is optional. Because the search process will take a long time, you can selectively run according to your specific. If not replace the dataset, you can ignore this section. + + + +## 6. Inference Deployment + + + +### 6.1 Getting Paddle Inference Model + +Paddle Inference is the original Inference Library of the PaddlePaddle, provides high-performance inference for server deployment. And compared with directly based on the pretrained model, Paddle Inference can use tools to accelerate prediction, so as to achieve better inference performance. Please refer to [Paddle Inference](https://www.paddlepaddle.org.cn/documentation/docs/zh/guides/infer/inference/inference_cn.html) for more information. + +Paddle Inference need Paddle Inference Model to predict. Two process provided to get Paddle Inference Model. If want to use the provided by PaddleClas, you can download directly, click [Downloading Inference Model](#6.1.2). + + + +### 6.1.1 Exporting Paddle Inference Model + +The command about exporting Paddle Inference Model is as follow: + +```bash +python3 tools/export_model.py \ + -c ./ppcls/configs/PULC/safety_helmet/PPLCNet_x1_0.yaml \ + -o Global.pretrained_model=output/DistillationModel/best_model_student \ + -o Global.save_inference_dir=deploy/models/PPLCNet_x1_0_safety_helmet_infer +``` + +After running above command, the inference model files would be saved in `deploy/models/PPLCNet_x1_0_safety_helmet_infer`, as shown below: + +``` +├── PPLCNet_x1_0_safety_helmet_infer +│ ├── inference.pdiparams +│ ├── inference.pdiparams.info +│ └── inference.pdmodel +``` + +**Note**: The best model is from knowledge distillation training. If knowledge distillation training is not used, the best model would be saved in `output/PPLCNet_x1_0/best_model.pdparams`. + + + +### 6.1.2 Downloading Inference Model + +You can also download directly. + +``` +cd deploy/models +# download the inference model and decompression +wget https://paddleclas.bj.bcebos.com/models/PULC/safety_helmet_infer.tar && tar -xf safety_helmet_infer.tar +``` + +After decompression, the directory `models` should be shown below. + +``` +├── safety_helmet_infer +│ ├── inference.pdiparams +│ ├── inference.pdiparams.info +│ └── inference.pdmodel +``` + + + +### 6.2 Prediction with Python + + + +#### 6.2.1 Image Prediction + +Return the directory `deploy`: + +``` +cd ../ +``` + +Run the following command to classify whether wearing safety helmet about the image `./images/PULC/safety_helmet/safety_helmet_test_1.png`. + +```shell +# Use the following command to predict with GPU. +python3.7 python/predict_cls.py -c configs/PULC/safety_helmet/inference_safety_helmet.yaml +# Use the following command to predict with CPU. +python3.7 python/predict_cls.py -c configs/PULC/safety_helmet/inference_safety_helmet.yaml -o Global.use_gpu=False +``` + +The prediction results: + +``` +safety_helmet_test_1.png: class id(s): [1], score(s): [1.00], label_name(s): ['unwearing_helmet'] +``` + +**Note**: The default threshold is `0.5`. If needed, you can specify the argument `Infer.PostProcess.threshold`, such as: `-o Infer.PostProcess.threshold=0.9167`. And the argument `threshold` is needed to be specified according by specific case. The `0.9167` is the best threshold when `Fpr` is less than `1/10000` in this valuation dataset. Please refer to [3.3 section](#3.3) for details. + + + +#### 6.2.2 Images Prediction + +If you want to predict images in directory, please specify the argument `Global.infer_imgs` as directory path by `-o Global.infer_imgs`. The command is as follow. + +```shell +# Use the following command to predict with GPU. If want to replace with CPU, you can add argument -o Global.use_gpu=False +python3.7 python/predict_cls.py -c configs/PULC/safety_helmet/inference_safety_helmet.yaml -o Global.infer_imgs="./images/PULC/safety_helmet/" +``` + +All prediction results will be printed, as shown below. + +``` +safety_helmet_test_1.png: class id(s): [1], score(s): [1.00], label_name(s): ['unwearing_helmet'] +safety_helmet_test_2.png: class id(s): [0], score(s): [1.00], label_name(s): ['wearing_helmet'] +``` + +Among the prediction results above, `wearing_helmet` means that wearing safety helmet about the image, `unwearing_helmet` means not. + + + +### 6.3 Deployment with C++ + +PaddleClas provides an example about how to deploy with C++. Please refer to [Deployment with C++](../inference_deployment/cpp_deploy_en.md). + + + +### 6.4 Deployment as Service + +Paddle Serving is a flexible, high-performance carrier for machine learning models, and supports different protocol, such as RESTful, gRPC, bRPC and so on, which provides different deployment solutions for a variety of heterogeneous hardware and operating system environments. Please refer [Paddle Serving](https://github.com/PaddlePaddle/Serving) for more information. + +PaddleClas provides an example about how to deploy as service by Paddle Serving. Please refer to [Paddle Serving Deployment](../inference_deployment/paddle_serving_deploy_en.md). + + + +### 6.5 Deployment on Mobile + +Paddle-Lite is an open source deep learning framework that designed to make easy to perform inference on mobile, embeded, and IoT devices. Please refer to [Paddle-Lite](https://github.com/PaddlePaddle/Paddle-Lite) for more information. + +PaddleClas provides an example of how to deploy on mobile by Paddle-Lite. Please refer to [Paddle-Lite deployment](../inference_deployment/paddle_lite_deploy_en.md). + + + +### 6.6 Converting To ONNX and Deployment + +Paddle2ONNX support convert Paddle Inference model to ONNX model. And you can deploy with ONNX model on different inference engine, such as TensorRT, OpenVINO, MNN/TNN, NCNN and so on. About Paddle2ONNX details, please refer to [Paddle2ONNX](https://github.com/PaddlePaddle/Paddle2ONNX). + +PaddleClas provides an example of how to convert Paddle Inference model to ONNX model by paddle2onnx toolkit and predict by ONNX model. You can refer to [paddle2onnx](../../../deploy/paddle2onnx/readme_en.md) for deployment details. diff --git a/docs/en/PULC/PULC_text_image_orientation_en.md b/docs/en/PULC/PULC_text_image_orientation_en.md index 139e165f5350cadb670f933ea223e7fdf2cd111f..2cddf79fa785e8afab93626cd5a502545e55a8ba 100644 --- a/docs/en/PULC/PULC_text_image_orientation_en.md +++ b/docs/en/PULC/PULC_text_image_orientation_en.md @@ -48,7 +48,7 @@ The following table lists the relevant indicators of the model. The first two li | PPLCNet_x1_0 | 98.02 | 2.16 | 6.5 | using SSLD pretrained model | | **PPLCNet_x1_0** | **99.06** | **2.16** | **6.5** | using SSLD pretrained model + hyperparameters searching strategy | -It can be seen that high Tpr can be getted when backbone is SwinTranformer_tiny, but the speed is slow. Replacing backbone with the lightweight model MobileNetV3_small_x0_35, the speed can be greatly improved, but the Tpr will be greatly reduced. Replacing backbone with faster backbone PPLCNet_x1_0, the Tpr is higher more 14 percentage points than MobileNetv3_small_x0_35. At the same time, the speed can be more faster. After additional using the SSLD pretrained model, the accuracy can be improved by about 0.17 percentage points without affecting the inference speed. Finally, after additional using the hyperparameters searching strategy, the accuracy can be further improved by 1.04 percentage points. At this point, the Tpr close to that of SwinTranformer_tiny, but the speed is more faster. The training method and deployment instructions of PULC will be introduced in detail below. +It can be seen that high accuracy can be getted when backbone is SwinTranformer_tiny, but the speed is slow. Replacing backbone with the lightweight model MobileNetV3_small_x0_35, the speed can be greatly improved, but the accuracy will be greatly reduced. Replacing backbone with faster backbone PPLCNet_x1_0, the accuracy is higher more 14 percentage points than MobileNetv3_small_x0_35. At the same time, the speed can be more faster. After additional using the SSLD pretrained model, the accuracy can be improved by about 0.17 percentage points without affecting the inference speed. Finally, after additional using the hyperparameters searching strategy, the accuracy can be further improved by 1.04 percentage points. At this point, the accuracy is close to that of SwinTranformer_tiny, but the speed is more faster. The training method and deployment instructions of PULC will be introduced in detail below. **Note**: @@ -161,7 +161,6 @@ Some image of the processed dataset is as follows: And you can also download the data processed directly. - ``` cd path_to_PaddleClas ``` @@ -307,7 +306,7 @@ The best metric of validation data is about `0.996`. The best teacher model weig #### 4.1.2 Knowledge Distillation Training -The training strategy, specified in training config file `ppcls/configs/PULC/text_image_orientation/PPLCNet_x1_0_distillation.yaml`, the teacher model is `ResNet101_vd`, the student model is `PPLCNet_x1_0` and the additional unlabeled training data is validation data of ImageNet1k. +The training strategy, specified in training config file `ppcls/configs/PULC/text_image_orientation/PPLCNet_x1_0_distillation.yaml`, the teacher model is `ResNet101_vd` and the student model is `PPLCNet_x1_0`. The command is as follow: diff --git a/docs/en/PULC/PULC_textline_orientation_en.md b/docs/en/PULC/PULC_textline_orientation_en.md index 35274ff3f7aa9447e4f6cc5c396e6abc436e3e55..71ea3407b952296bf7b6dad5768fd8f38ee6d563 100644 --- a/docs/en/PULC/PULC_textline_orientation_en.md +++ b/docs/en/PULC/PULC_textline_orientation_en.md @@ -55,13 +55,11 @@ The following table lists the relevant indicators of the model. The first two li It can be seen that high accuracy can be getted when backbone is SwinTranformer_tiny, but the speed is slow. Replacing backbone with the lightweight model MobileNetV3_small_x0_35, the speed can be greatly improved, but the accuracy will be greatly reduced. Replacing backbone with faster backbone PPLCNet_x1_0, the accuracy is higher more 8.6 percentage points than MobileNetv3_small_x0_35. At the same time, the speed can be more than 10% faster. On this basis, by changing the resolution and stripe (refer to [PaddleOCR](https://github.com/PaddlePaddle/PaddleOCR)), the speed becomes 27% slower, but the accuracy can be improved by 4.5 percentage points. After additional using the SSLD pretrained model, the accuracy can be improved by about 0.05 percentage points without affecting the inference speed. Finally, additional using the EDA strategy, the accuracy can be increased by 1.9 percentage points. The training method and deployment instructions of PULC will be introduced in detail below. **Note**: - -* Among the above tabel, backbone name without \* means the resolution is 224x224, and with \* means the resolution is 48x192 (h\*w). The data augmentation is changed from the stripe in the network to ` [2, [2, 1], [2, 1], [2, 1]`. Each element in the outer list represents the stripe of the sampling layer under the network structure. The strategy is [paddleocr]（ https://github.com/PaddlePaddle/PaddleOCR ）Provided text line direction classifier scheme. The model with \ * \ * indicates that the resolution is 80x160 (h\*w), and the stripe in the network is changed to ` [2, [2, 1], [2, 1], [2, 1]]`. Each element in the outer list represents the stripe of the sampling layer under the network structure. This resolution is obtained through the search of [hyper parameter search strategy] (pulc_train.md\4- hyper parameter search). +* Backbone name without \* means the resolution is 224x224, and with \* means the resolution is 48x192 (h\*w). The stride of the network is changed to `[2, [2, 1], [2, 1], [2, 1]`. Please refer to [PaddleOCR]（ https://github.com/PaddlePaddle/PaddleOCR）for more details. +* Backbone name with \*\* means that the resolution is 80x160 (h\*w), and the stride of the network is changed to `[2, [2, 1], [2, 1], [2, 1]]`. This resolution is searched by [Hyperparameter Searching](pulc_train_en.md#4). * The Latency is tested on Intel(R) Xeon(R) Gold 6148 CPU @ 2.40GHz. The MKLDNN is enabled and the number of threads is 10. * About PP-LCNet, please refer to [PP-LCNet Introduction](../models/PP-LCNet_en.md) and [PP-LCNet Paper](https://arxiv.org/abs/2109.15099). -* 其中不带\*的模型表示分辨率为224x224，带\*的模型表示分辨率为48x192（h\*w）,数据增强从网络中的 stride 改为 `[2, [2, 1], [2, 1], [2, 1], [2, 1]]`，其中，外层列表中的每一个元素代表网络结构下采样层的stride，该策略为 [PaddleOCR](https://github.com/PaddlePaddle/PaddleOCR) 提供的文本行方向分类器方案。带\*\*的模型表示分辨率为80x160（h\*w）, 网络中的 stride 改为 `[2, [2, 1], [2, 1], [2, 1], [2, 1]]`，其中，外层列表中的每一个元素代表网络结构下采样层的stride，此分辨率是经过[超参数搜索策略](PULC_train.md#4-超参搜索)搜索得到的。 - ## 2. Quick Start @@ -292,7 +290,7 @@ The best metric of validation data is between `0.96` and `0.98`. The best teache #### 4.1.2 Knowledge Distillation Training -The training strategy, specified in training config file `ppcls/configs/PULC/textline_orientation/PPLCNet_x1_0_distillation.yaml`, the teacher model is `ResNet101_vd`, the student model is `PPLCNet_x1_0` and the additional unlabeled training data is validation data of ImageNet1k. The command is as follow: +The training strategy, specified in training config file `ppcls/configs/PULC/textline_orientation/PPLCNet_x1_0_distillation.yaml`, the teacher model is `ResNet101_vd` and the student model is `PPLCNet_x1_0`. The command is as follow: ```shell export CUDA_VISIBLE_DEVICES=0,1,2,3 diff --git a/docs/zh_CN/PULC/PULC_textline_orientation.md b/docs/zh_CN/PULC/PULC_textline_orientation.md index b4de6351e2fc3bce51f7bfde14afaaa9f5cb2a8f..d22d1a0cd41e89d762a70bb6f2f33fdf1d5aa9cb 100644 --- a/docs/zh_CN/PULC/PULC_textline_orientation.md +++ b/docs/zh_CN/PULC/PULC_textline_orientation.md @@ -42,7 +42,7 @@ 该案例提供了用户使用 PaddleClas 的超轻量图像分类方案（PULC，Practical Ultra Lightweight Classification）快速构建轻量级、高精度、可落地的文本行方向分类模型。该模型可以广泛应用于如文字矫正、文字识别等场景。 -下表列出了文本行方向分类模型的相关指标，前两行展现了使用 Res2Net200_vd 和 MobileNetV3_large_x1_0 作为 backbone 训练得到的模型的相关指标，第三行至第六行依次展现了替换 backbone 为 PPLCNet_x1_0、使用 SSLD 预训练模型、使用 SSLD 预训练模型 + EDA 策略、使用 SSLD 预训练模型 + EDA 策略 + SKL-UGI 知识蒸馏策略训练得到的模型的相关指标。 +下表列出了文本行方向分类模型的相关指标，前两行展现了使用 Res2Net200_vd 和 MobileNetV3_large_x1_0 作为 backbone 训练得到的模型的相关指标，第三行至第七行依次展现了替换 backbone 为 PPLCNet_x1_0、使用 SSLD 预训练模型、使用 SSLD 预训练模型 + EDA 策略、使用 SSLD 预训练模型 + EDA 策略 + SKL-UGI 知识蒸馏策略训练得到的模型的相关指标。 | 模型 | Top-1 Acc（%） | 延时（ms） | 存储（M） | 策略 |