SSLD is a simple semi-supervised distillation method proposed by Baidu in 2021. By designing an improved JS divergence as the loss function and combining the data mining strategy based on ImageNet22k dataset, the accuracy of the 18 backbone network models was improved by more than 3% on average.
For more information about the principle, model zoo and usage of SSLD, please refer to: [Introduction to SSLD](ssld.md).
<!-- For more information about the principle, model zoo and usage of SSLD, please refer to: [Introduction to SSLD](ssld_en.md). -->
##### 1.2.1.2 Configuration of SSLD
...
...
@@ -152,8 +152,8 @@ Performance on ImageNet1k is shown below.
* Note: Complete PPLCNet_x2_5 The model have been trained for 360 epochs. For comparison, both baseline and DML have been trained for 100 epochs. Therefore, the accuracy is lower than the model (76.60%) opened on the official website.
...
...
@@ -210,8 +210,8 @@ Performance on ImageNet1k is shown below.
**Note(:**`return_patterns` are specified in the network above. The function of returning middle layer features is based on TheseusLayer. For more information about usage of TheseusLayer, please refer to: [Usage of TheseusLayer](theseus_layer.md).
**Note(:**`return_patterns` are specified in the network above. The function of returning middle layer features is based on TheseusLayer.
<!-- TODO(gaotingquan) -->
<!-- For more information about usage of TheseusLayer, please refer to: [Usage of TheseusLayer](theseus_layer.md). -->
<aname='1.2.4'></a>
...
...
@@ -286,8 +289,8 @@ Performance on ImageNet1k is shown below.
Note: In order to keep alignment with the training configuration in the paper, the number of training iterations is set to be 100 epochs, so the baseline accuracy is lower than the open source model accuracy in PaddleClas (71.0%).
...
...
@@ -374,7 +377,10 @@ Loss:
weight:1.0
```
**Note(:**`return_patterns` are specified in the network above. The function of returning middle layer features is based on TheseusLayer. For more information about usage of TheseusLayer, please refer to: [Usage of TheseusLayer](theseus_layer.md).
**Note(:**`return_patterns` are specified in the network above. The function of returning middle layer features is based on TheseusLayer.
<!-- TODO(gaotingquan) -->
<!-- For more information about usage of TheseusLayer, please refer to: [Usage of TheseusLayer](theseus_layer.md). -->
<aname='1.2.5'></a>
...
...
@@ -397,8 +403,8 @@ Performance on ImageNet1k is shown below.
* Installation: Please refer to [Paddle Installation Tutorial](../installation/install_paddle.md) and [PaddleClas Installation Tutorial](../../installation.md) to configure the running environment.
* Installation: Please refer to [Paddle Installation Tutorial](../installation/install_paddle_en.md) and [PaddleClas Installation Tutorial](../installation/install_paddleclas_en.md) to configure the running environment.
* For more information about the format of `train_list.txt` and `val_list.txt`, you may refer to [Format Description of PaddleClas Classification Dataset](../single_label_classification/dataset.md#1-数据集格式说明) .
* For more information about the format of `train_list.txt` and `val_list.txt`, you may refer to [Format Description of PaddleClas Classification Dataset](../data_preparation/classification_dataset_en.md#1dataset-format) .
In this section, the process of model training, evaluation and prediction of knowledge distillation algorithm will be introduced using the SSLD knowledge distillation algorithm as an example. The configuration file is [PPLCNet_x2_5_ssld.yaml](../../../../ppcls/configs/ImageNet/Distillation/PPLCNet_x2_5_ssld.yaml). You can use the following command to complete the model training.
In this section, the process of model training, evaluation and prediction of knowledge distillation algorithm will be introduced using the SSLD knowledge distillation algorithm as an example. The configuration file is [PPLCNet_x2_5_ssld.yaml](../../../ppcls/configs/ImageNet/Distillation/PPLCNet_x2_5_ssld.yaml). You can use the following command to complete the model training.
@@ -39,7 +39,7 @@ Functions of the above modules :
#### 3.1 Backbone
The Backbone part adopts [PP-LCNetV2_base](../models/PP-LCNetV2.md), which is based on `PPLCNet_V1`, including Rep strategy, PW convolution, Shortcut, activation function improvement, SE module improvement After several optimization points, the final classification accuracy is similar to `PPLCNet_x2_5`, and the inference delay is reduced by 40%<sup>*</sup>. During the experiment, we made appropriate improvements to `PPLCNetV2_base`, so that it can achieve higher performance in recognition tasks while keeping the speed basically unchanged, including: removing `ReLU` and ` at the end of `PPLCNetV2_base` FC`, change the stride of the last stage (RepDepthwiseSeparable) to 1.
The Backbone part adopts [PP-LCNetV2_base](../models/PP-LCNetV2_en.md), which is based on `PPLCNet_V1`, including Rep strategy, PW convolution, Shortcut, activation function improvement, SE module improvement After several optimization points, the final classification accuracy is similar to `PPLCNet_x2_5`, and the inference delay is reduced by 40%<sup>*</sup>. During the experiment, we made appropriate improvements to `PPLCNetV2_base`, so that it can achieve higher performance in recognition tasks while keeping the speed basically unchanged, including: removing `ReLU` and ` at the end of `PPLCNetV2_base` FC`, change the stride of the last stage (RepDepthwiseSeparable) to 1.
**Note:**<sup>*</sup>The inference environment is based on Intel(R) Xeon(R) Gold 6271C CPU @ 2.60GHz hardware platform, OpenVINO inference platform.
@@ -124,7 +124,10 @@ Note: Since some decompression software has problems in decompressing the above
The demo data download path of this chapter is as follows: [drink_dataset_v2.0.tar (drink data)](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/data/drink_dataset_v2.0.tar),
The following takes **drink_dataset_v2.0.tar** as an example to introduce the PP-ShiTu quick start process on the PC. Users can also download and decompress the data of other scenarios to experience: [22 scenarios data download](../../zh_CN/introduction/ppshitu_application_scenarios.md#22-下载解压场景库数据).
The following takes **drink_dataset_v2.0.tar** as an example to introduce the PP-ShiTu quick start process on the PC.
<!-- TODO -->
<!-- Users can also download and decompress the data of other scenarios to experience: [22 scenarios data download](../../zh_CN/introduction/ppshitu_application_scenarios.md#22-下载解压场景库数据). -->
If you want to experience the server object detection and the recognition model of each scene, you can refer to [2.4 Server recognition model list](#24-list-of-server-identification-models)
