-PULC实用图像分类模型效果展示
+
+PP-ShiTuV2图像识别系统效果展示
-PULC实用图像分类模型效果展示
+
+PP-ShiTuV2图像识别系统效果展示
-PP-ShiTu图像识别系统效果展示
-
+
+PULC实用图像分类模型效果展示
-
+
+PP-ShiTuV2 Android Demo
+
-
-
+
+
-
-PULC demo images
+
+PP-ShiTuV2 demo images
+
-PP-ShiTu demo images
+PULC demo images
+PP-ShiTuV2 Android Demo
+
-Image recognition can be divided into three steps:
-- （1）Identify region proposal for target objects through a detection model；
-- （2）Extract features for each region proposal;
-- （3）Search features in the retrieval database and output results;
-
+PP-ShiTuV2 is a practical lightweight general image recognition system, which is mainly composed of three modules: mainbody detection model, feature extraction model and vector search tool. The system adopts a variety of strategies including backbone network, loss function, data augmentations, optimal hyperparameters, pre-training model, model pruning and quantization. Compared to V1, PP-ShiTuV2, Recall1 is improved by nearly 8 points. For more details, please refer to [PP-ShiTuV2 introduction](./docs/en/PPShiTu/PPShiTuV2_introduction.md).
For a new unknown category, there is no need to retrain the model, just prepare images of new category, extract features and update retrieval database and the category can be recognised.
-
-## PULC demo images
+
+## PP-ShiTuV2 Demo images
+
+- Drinks recognition
+
+
+
+
or the "file recognition" button
, you can take an image or select an image, then wait a few seconds, main object in the image will be marked and the predicted class and inference time will be shown below the image.
+Take the following image as an example:
-Demo data in this tutorial can be downloaded here: [download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/data/recognition_demo_data_en_v1.1.tar).
+
+The retrieval results obtained are visualized as follows:
-**Attention**
-1. If you do not have wget installed on Windows, you can download the model by copying the link into your browser and unzipping it in the appropriate folder; for Linux or macOS users, you can right-click and copy the download link to download it via the `wget` command.
-2. If you want to install `wget` on macOS, you can run the following command.
-3. The predict config file of the lightweight generic recognition model and the config file to build index database are used for the config of product recognition model of server-side. You can modify the path of the model to complete the index building and prediction.
+
-```shell
-# install homebrew
-ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)";
-# install wget
-brew install wget
-```
+#### 1.2.2 Update Index
+Click the "photo upload" button above 
or the "file upload" button 
, you can take an image or select an image and enter the class name of the uploaded image (such as `keyboard`), click the "OK" button, then the feature vector and classname corresponding to the image will be added to the index database.
-3. If you want to isntall `wget` on Windows, you can refer to [link](https://www.cnblogs.com/jeshy/p/10518062.html). If you want to install `tar` on Windows, you can refer to [link](https://www.cnblogs.com/chooperman/p/14190107.html).
+#### 1.2.3 Save Index
+Click the "save index" button above 
, you can save the current index database as `latest`.
+#### 1.2.4 Initialize Index
+Click the "initialize index" button above 
to initialize the current library to `original`.
-* You can download and unzip the data and models by following the command below
+#### 1.2.5 Preview Index
+Click the "class preview" button 
to view it in the pop-up window.
-```shell
-mkdir models
-cd models
-# Download and unzip the inference model
-wget {Models download link} && tar -xf {Name of the tar archive}
-cd ..
+
-# Download the demo data and unzip
-wget {Data download link} && tar -xf {Name of the tar archive}
-```
+### 1.3 Feature Details
+
+#### 1.3.1 Image Retrieval
+After selecting the image to be retrieved, firstly, the mainbody detection will be performed through the detection model to obtain the bounding box of the object in the image, and then the image will be cropped and is input into the feature extraction model to obtain the corresponding feature vector and retrieved in the index database, returns and displays the final search result.
+
+#### 1.3.2 Update Index
+After selecting the picture to be stored, firstly, the mainbody detection will be performed through the detection model to obtain the bounding box of the object in the image, and then the image will be cropped and is input into the feature extraction model to obtain the corresponding feature vector, and then added into index database.
+
+#### 1.3.3 Save Index
+Save the index database in the current program index database name of `latest`, and automatically switch to `latest`. The saving logic is similar to "Save As" in general software. If the current index is already `latest`, it will be automatically overwritten, or it will switch to `latest`.
+
+#### 1.3.4 Initialize Index
+When initializing the index database, it will automatically switch the search index database to `original.index` and `original.txt`, and automatically delete `latest.index` and `latest.txt` (if exists).
+
+#### 1.3.5 Preview Index
+One can preview it according to the instructions in [Function Experience - Preview Index](#125-preview-index).
+
+
+## 2. PP-ShiTu PC demo for quick start
+
+
+
+### 2.1 Environment configuration
+
+* Installation: Please refer to the document [Environment Preparation](../installation/install_paddleclas.md) to configure the PaddleClas operating environment.
+
+* Go to the `deploy` run directory. All the content and scripts in this section need to be run in the `deploy` directory, you can enter the `deploy` directory with the following scripts.
+
+ ```shell
+ cd deploy
+ ```
+
-
-### 2.1 Download and Unzip the Inference Model and Demo Data
+### 2.2 Image recognition experience
-Take the product recognition as an example, download the detection model, recognition model and product recognition demo data with the following commands.
+The lightweight general object detection model, lightweight general recognition model and configuration file are available in following table.
+
+
+
+| Model Introduction | Recommended Scenarios | Inference Model | Prediction Profile |
+| ------------------------------------------ | --------------------- | ------------------ | ------------------------------------------------------------------------ |
+| Lightweight General MainBody Detection Model | General Scene | [tar format download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/picodet_PPLCNet_x2_5_mainMainBody_lite_v1.0_infer.tar ) \| [zip format download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/picodet_PPLCNet_x2_5_mainMainBody_lite_v1.0_infer.zip) | - |
+| Lightweight General Recognition Model | General Scene | [tar format download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/PP-ShiTuV2/general_PPLCNetV2_base_pretrained_v1.0_infer.tar) \| [zip format download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/PP-ShiTuV2/general_PPLCNetV2_base_pretrained_v1.0_infer.zip) | [inference_general.yaml](../../../deploy/configs/inference_general.yaml) |
+
+Note: Since some decompression software has problems in decompressing the above `tar` format files, it is recommended that non-script line users download the `zip` format files and decompress them. `tar` format file is recommended to use the script `tar -xf xxx.tar`unzip.
+
+The demo data download path of this chapter is as follows: [drink_dataset_v2.0.tar (drink data)](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/data/drink_dataset_v2.0.tar),
+
+The following takes **drink_dataset_v2.0.tar** as an example to introduce the PP-ShiTu quick start process on the PC. Users can also download and decompress the data of other scenarios to experience: [22 scenarios data download](../../zh_CN/introduction/ppshitu_application_scenarios.md#22-下载解压场景库数据).
+
+If you want to experience the server object detection and the recognition model of each scene, you can refer to [2.4 Server recognition model list](#24-list-of-server-identification-models)
+
+**Notice**
+
+- If wget is not installed in the windows environment, you can install the `wget` and tar scripts according to the following steps, or you can copy the link to the browser to download the model, decompress it and place it in the corresponding directory.
+- If the `wget` script is not installed in the macOS environment, you can run the following script to install it.
+ ```shell
+ # install homebrew
+ ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)";
+ # install wget
+ brew install wget
+ ```
+- If you want to install `wget` in the windows environment, you can refer to: [link](https://www.cnblogs.com/jeshy/p/10518062.html); if you want to install the `tar` script in the windows environment, you can refer to: [Link](https://www.cnblogs.com/chooperman/p/14190107.html).
+
+
+
+#### 2.2.1 Download and unzip the inference model and demo data
+
+Download the demo dataset and the lightweight subject detection and recognition model. The scripts are as follows.
```shell
mkdir models
cd models
-# Download the generic detection inference model and unzip it
-wget https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/ppyolov2_r50vd_dcn_mainbody_v1.0_infer.tar && tar -xf ppyolov2_r50vd_dcn_mainbody_v1.0_infer.tar
-# Download and unpack the inference model
-wget https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/product_ResNet50_vd_aliproduct_v1.0_infer.tar && tar -xf product_ResNet50_vd_aliproduct_v1.0_infer.tar
-cd ..
-
-# Download the demo data and unzip it
-wget https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/data/recognition_demo_data_en_v1.1.tar && tar -xf recognition_demo_data_en_v1.1.tar
+# Download the mainbody detection inference model and unzip it
+wget https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/picodet_PPLCNet_x2_5_mainbody_lite_v1.0_infer.tar && tar -xf picodet_PPLCNet_x2_5_mainbody_lite_v1.0_infer.tar
+# Download the feature extraction inference model and unzip it
+wget https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/PP-ShiTuV2/general_PPLCNetV2_base_pretrained_v1.0_infer.tar && tar -xf general_PPLCNetV2_base_pretrained_v1.0_infer.tar
+
+cd ../
+# Download demo data and unzip it
+wget https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/data/drink_dataset_v2.0.tar && tar -xf drink_dataset_v2.0.tar
```
-Once unpacked, the `recognition_demo_data_v1.1` folder should have the following file structure.
+After decompression, the `drink_dataset_v2.0/` folder be structured as follows:
-```
-├── recognition_demo_data_v1.1
-│ ├── gallery_cartoon
-│ ├── gallery_logo
-│ ├── gallery_product
-│ ├── gallery_vehicle
-│ ├── test_cartoon
-│ ├── test_logo
-│ ├── test_product
-│ └── test_vehicle
+```log
+├── drink_dataset_v2.0/
+│ ├── gallery/
+│ ├── index/
+│ ├── index_all/
+│ └── test_images/
├── ...
```
-here, original images to build index are in folder `gallery_xxx`, test images are in folder `test_xxx`. You can also access specific folder for more details.
+The `gallery` folder stores the original images used to build the index database, `index` represents the index database constructed based on the original images, and the `test_images` folder stores the list of images for query.
-The `models` folder should have the following file structure.
+The `models` folder should be structured as follows:
-```
-├── product_ResNet50_vd_aliproduct_v1.0_infer
+```log
+├── general_PPLCNetV2_base_pretrained_v1.0_infer
│ ├── inference.pdiparams
│ ├── inference.pdiparams.info
│ └── inference.pdmodel
-├── ppyolov2_r50vd_dcn_mainbody_v1.0_infer
+├── picodet_PPLCNet_x2_5_mainbody_lite_v1.0_infer
│ ├── inference.pdiparams
│ ├── inference.pdiparams.info
│ └── inference.pdmodel
```
-**Attention**
-If you want to use the lightweight generic recognition model, you need to re-extract the features of the demo data and re-build the index. The way is as follows:
+**Notice**
+
+If the general feature extraction model is changed, the index for demo data must be rebuild, as follows:
```shell
-python3.7 python/build_gallery.py -c configs/build_product.yaml -o Global.rec_inference_model_dir=./models/general_PPLCNet_x2_5_lite_v1.0_infer
+python3.7 python/build_gallery.py \
+-c configs/inference_general.yaml \
+-o Global.rec_inference_model_dir=./models/general_PPLCNetV2_base_pretrained_v1.0_infer
```
-
-### 2.2 Product Recognition and Retrieval
-
-Take the product recognition demo as an example to show the recognition and retrieval process (if you wish to try other scenarios of recognition and retrieval, replace the corresponding configuration file after downloading and unzipping the corresponding demo data and model to complete the prediction).
-
-**Note:** `faiss` is used as search library. The installation method is as follows：
+
-```
-pip install faiss-cpu==1.7.1post2
-```
+#### 2.2.2 Drink recognition and retrieval
-If error happens when using `import faiss`, please uninstall `faiss` and reinstall it, especially on `Windows`.
+Take the drink recognition demo as an example to show the recognition and retrieval process.
-
+Note that this section will uses `faiss` as the retrieval tool, and the installation script is as follows:
-#### 2.2.1 Single Image Recognition
+```python
+python3.7 -m pip install faiss-cpu==1.7.1post2
+```
-Run the following command to identify and retrieve the image `./recognition_demo_data_v1.1/test_product/daoxiangcunjinzhubing_6.jpg` for recognition and retrieval
+If `faiss` cannot be importted, try reinstall it, especially for windows users.
-```shell
-# use the following command to predict using GPU.
-python3.7 python/predict_system.py -c configs/inference_product.yaml
-# use the following command to predict using CPU
-python3.7 python/predict_system.py -c configs/inference_product.yaml -o Global.use_gpu=False
-```
+
+##### 2.2.2.1 single image recognition
-The image to be retrieved is shown below.
+Run the following script to recognize the image `./drink_dataset_v2.0/test_images/100.jpeg`
-
+The images to be retrieved are as follows
+
-The final output is shown below.
+```shell
+# Use the script below to make predictions using the GPU
+python3.7 python/predict_system.py -c configs/inference_general.yaml
-```
-[{'bbox': [287, 129, 497, 326], 'rec_docs': 'Daoxaingcun Golden Piggie Cake', 'rec_scores': 0.8309420347213745}, {'bbox': [99, 242, 313, 426], 'rec_docs': 'Daoxaingcun Golden Piggie Cake', 'rec_scores': 0.7245651483535767}]
+# Use the following script to make predictions using the CPU
+python3.7 python/predict_system.py -c configs/inference_general.yaml -o Global.use_gpu=False
```
+The final output is as follows.
-where bbox indicates the location of the detected object, rec_docs indicates the labels corresponding to the label in the index dabase that are most similar to the detected object, and rec_scores indicates the corresponding confidence.
+```log
+[{'bbox': [437, 71, 660, 728], 'rec_docs': '元气森林', 'rec_scores': 0.7740249}, {'bbox': [221, 72, 449, 701], 'rec_docs' : '元气森林', 'rec_scores': 0.6950992}, {'bbox': [794, 104, 979, 652], 'rec_docs': '元气森林', 'rec_scores': 0.6305153}]
+```
+Where `bbox` represents the location of the detected object, `rec_docs` represents the most similar category to the detection box in the index database, and `rec_scores` represents the corresponding similarity.
-The detection result is also saved in the folder `output`, for this image, the visualization result is as follows.
+The visualization results of the recognition are saved in the `output` folder by default. For this image, the visualization of the recognition results is shown below.
-
+
+
-
-#### 2.2.2 Folder-based Batch Recognition
+##### 2.2.2.2 Folder-based batch recognition
-If you want to predict the images in the folder, you can directly modify the `Global.infer_imgs` field in the configuration file, or you can also modify the corresponding configuration through the following `-o` parameter.
+If you want to use multi images in the folder for prediction, you can modify the `Global.infer_imgs` field in the configuration file, or you can modify the corresponding configuration through the `-o` parameter below.
```shell
-# using the following command to predict using GPU, you can append `-o Global.use_gpu=False` to predict using CPU.
-python3.7 python/predict_system.py -c configs/inference_product.yaml -o Global.infer_imgs="./recognition_demo_data_v1.1/test_product/"
+# Use the following script to use GPU for prediction, if you want to use CPU prediction, you can add -o Global.use_gpu=False after the script
+python3.7 python/predict_system.py -c configs/inference_general.yaml -o Global.infer_imgs="./drink_dataset_v2.0/test_images/"
```
+The recognition results of all images in the folder will be output in the terminal, as shown below.
-The results on the screen are shown as following.
-
-```
+```log
...
-[{'bbox': [37, 29, 123, 89], 'rec_docs': 'Chanel Handbag', 'rec_scores': 0.6163763999938965}, {'bbox': [153, 96, 235, 175], 'rec_docs': 'Chanel Handbag', 'rec_scores': 0.5279821157455444}]
-[{'bbox': [735, 562, 1133, 851], 'rec_docs': 'Chanel Handbag', 'rec_scores': 0.5588355660438538}]
-[{'bbox': [124, 50, 230, 129], 'rec_docs': 'Chanel Handbag', 'rec_scores': 0.6980369687080383}]
-[{'bbox': [0, 0, 275, 183], 'rec_docs': 'Chanel Handbag', 'rec_scores': 0.5818190574645996}]
-[{'bbox': [400, 1179, 905, 1537], 'rec_docs': 'Chanel Handbag', 'rec_scores': 0.9814301133155823}, {'bbox': [295, 713, 820, 1046], 'rec_docs': 'Chanel Handbag', 'rec_scores': 0.9496176242828369}, {'bbox': [153, 236, 694, 614], 'rec_docs': 'Chanel Handbag', 'rec_scores': 0.8395382761955261}]
-[{'bbox': [544, 4, 1482, 932], 'rec_docs': 'Chanel Handbag', 'rec_scores': 0.5143815279006958}]
+[{'bbox': [0, 0, 600, 600], 'rec_docs': '红牛-强化型', 'rec_scores': 0.74081033}]
+Inference: 120.39852142333984 ms per batch image
+[{'bbox': [0, 0, 514, 436], 'rec_docs': '康师傅矿物质水', 'rec_scores': 0.6918598}]
+Inference: 32.045602798461914 ms per batch image
+[{'bbox': [138, 40, 573, 1198], 'rec_docs': '乐虎功能饮料', 'rec_scores': 0.68214047}]
+Inference: 113.41428756713867 ms per batch image
+[{'bbox': [328, 7, 467, 272], 'rec_docs': '脉动', 'rec_scores': 0.60406065}]
+Inference: 122.04337120056152 ms per batch image
+[{'bbox': [242, 82, 498, 726], 'rec_docs': '味全_每日C', 'rec_scores': 0.5428652}]
+Inference: 37.95266151428223 ms per batch image
+[{'bbox': [437, 71, 660, 728], 'rec_docs': '元气森林', 'rec_scores': 0.7740249}, {'bbox': [221, 72, 449, 701], 'rec_docs': '元气森林', 'rec_scores': 0.6950992}, {'bbox': [794, 104, 979, 652], 'rec_docs': '元气森林', 'rec_scores': 0.6305153}]
...
```
-All the visualization results are also saved in folder `output`.
+Visualizations of recognition results for all images are also saved in the `output` folder.
+
+Furthermore, you can change the path of the recognition inference model by modifying the `Global.rec_inference_model_dir` field, and change the path of the index database by modifying the `IndexProcess.index_dir` field.
+
-Furthermore, the recognition inference model path can be changed by modifying the `Global.rec_inference_model_dir` field, and the path of the index to the index databass can be changed by modifying the `IndexProcess.index_dir` field.
+### 2.3 Image of Unknown categories recognition experience
+Now we try to recognize the unseen image `./drink_dataset_v2.0/test_images/mosilian.jpeg`
-
-## 3. Recognize Images of Unknown Category
+The images to be retrieved are as follows
-To recognize the image `./recognition_demo_data_v1.1/test_product/anmuxi.jpg`, run the command as follows:
+
+
+Execute the following identification script
```shell
-python3.7 python/predict_system.py -c configs/inference_product.yaml -o Global.infer_imgs="./recognition_demo_data_v1.1/test_product/anmuxi.jpg"
+# Use the following script to use GPU for prediction, if you want to use CPU prediction, you can add -o Global.use_gpu=False after the script
+python3.7 python/predict_system.py -c configs/inference_general.yaml -o Global.infer_imgs="./drink_dataset_v2.0/test_images/mosilian.jpeg"
```
-The image to be retrieved is shown below.
+It can be found that the output result is empty
+
+Since the default index database does not contain the unknown category's information, the recognition result here is wrong. At this time, we can achieve the image recognition of unknown classes by building a new index database.
+
+When the images in the index database cannot cover the scene we actually recognize, i.e. recognizing an image of an unknown category, we need to add a similar image(at least one) belong the unknown category to the index database. This process does not require re-training the model. Take `mosilian.jpeg` as an example, just follow the steps below to rebuild a new index database.
+
+
+
+#### 2.3.1 Prepare new data and labels
-
+First, copy the image(s) belong to unknown category(except the query image) to the original image folder of the index database. Here we already put all the image data in the folder `drink_dataset_v2.0/gallery/`.
-The output is empty.
+Then we need to edit the text file that records the image path and label information. Here we already put the updated label information file in the `drink_dataset_v2.0/gallery/drink_label_all.txt` file. Comparing with the original `drink_dataset_v2.0/gallery/drink_label.txt` label file, it can be found that the index images of the bright and ternary series of milk have been added.
-Since the index infomation is not included in the corresponding index databse, the recognition result is empty or not proper. At this time, we can complete the image recognition of unknown categories by constructing a new index database.
+In each line of text, the first field represents the relative path of the image, and the second field represents the label information corresponding to the image, separated by the `\t` key (Note: some editors will automatically convert `tab` is `space`, in which case it will cause a file parsing error).
-When the index database cannot cover the scenes we actually recognise, i.e. when predicting images of unknown categories, we need to add similar images of the corresponding categories to the index databasey, thus completing the recognition of images of unknown categories ，which does not require retraining.
+
-
-### 3.1 Prepare for the new images and labels
+#### 2.3.2 Create a new index database
-First, you need to copy the images which are similar with the image to retrieval to the original images for the index database. The command is as follows.
+Build a new index database `index_all` with the following scripts.
```shell
-cp -r ../docs/images/recognition/product_demo/gallery/anmuxi ./recognition_demo_data_/gallery_product/gallery/
+python3.7 python/build_gallery.py -c configs/inference_general.yaml -o IndexProcess.data_file="./drink_dataset_v2.0/gallery/drink_label_all.txt" -o IndexProcess.index_dir="./drink_dataset_v2.0/index_all"
```
-Then you need to create a new label file which records the image path and label information. Use the following command to create a new file based on the original one.
+The final constructed new index database is saved in the folder `./drink_dataset_v2.0/index_all`. For specific instructions on yaml `yaml`, please refer to [Vector Search Documentation](../image_recognition_pipeline/vector_search.md).
+
+
+
+#### 2.3.3 Image recognition based on the new index database
+
+To re-recognize the `mosilian.jpeg` image using the new index database, run the following scripts.
```shell
-# copy the file
-cp recognition_demo_data_v1.1/gallery_product/data_file.txt recognition_demo_data_v1.1/gallery_product/data_file_update.txt
+# run the following script predict with GPU, if you want to use CPU, you can add -o Global.use_gpu=False after the script
+python3.7 python/predict_system.py -c configs/inference_general.yaml -o Global.infer_imgs="./drink_dataset_v2.0/test_images/mosilian.jpeg" -o IndexProcess.index_dir="./drink_dataset_v2.0/index_all"
```
-Then add some new lines into the new label file, which is shown as follows.
+The output is as follows.
-```
-gallery/anmuxi/001.jpg Anmuxi Ambrosial Yogurt
-gallery/anmuxi/002.jpg Anmuxi Ambrosial Yogurt
-gallery/anmuxi/003.jpg Anmuxi Ambrosial Yogurt
-gallery/anmuxi/004.jpg Anmuxi Ambrosial Yogurt
-gallery/anmuxi/005.jpg Anmuxi Ambrosial Yogurt
-gallery/anmuxi/006.jpg Anmuxi Ambrosial Yogurt
+```log
+[{'bbox': [290, 297, 564, 919], 'rec_docs': 'Bright_Mosleyan', 'rec_scores': 0.59137374}]
```
-Each line can be splited into two fields. The first field denotes the relative image path, and the second field denotes its label. The `delimiter` is `tab` here.
+The final recognition result is `光明_莫斯利安`, we can see the recognition result is correct now , and the visualization of the recognition result is shown below.
+
-
-### 3.2 Build a new Index Base Library
-Use the following command to build the index to accelerate the retrieval process after recognition.
+
-```shell
-python3.7 python/build_gallery.py -c configs/build_product.yaml -o IndexProcess.data_file="./recognition_demo_data_v1.1/gallery_product/data_file_update.txt" -o IndexProcess.index_dir="./recognition_demo_data_v1.1/gallery_product/index_update"
-```
+### 2.4 List of server recognition models
-Finally, the new index information is stored in the folder`./recognition_demo_data_v1.1/gallery_product/index_update`. Use the new index database for the above index.
+At present, we recommend to use model in [Lightweight General Object Detection Model and Lightweight General Recognition Model](#22-image-recognition-experience) to get better test results. However, if you want to experience the general recognition model, general object detection model and other recognition model for server, the test data download path, and the corresponding configuration file path are as follows.
+| Model Introduction | Recommended Scenarios | Inference Model | Prediction Profile |
+| --------------------------------- | --------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------- |
+| General Body Detection Model | General Scene | [Model download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/ppyolov2_r50vd_dcn_mainbody_v1.0_infer.tar) | - |
+| Logo Recognition Model | Logo Scene | [Model download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/logo_rec_ResNet50_Logo3K_v1.0_infer.tar) | [inference_logo. yaml](../../../deploy/configs/inference_logo.yaml) |
+| Anime Character Recognition Model | Anime Character Scene | [Model download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/cartoon_rec_ResNet50_iCartoon_v1.0_infer.tar) | [ inference_cartoon.yaml](../../../deploy/configs/inference_cartoon.yaml) |
+| Vehicle Subdivision Model | Vehicle Scene | [Model download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/vehicle_cls_ResNet50_CompCars_v1.0_infer.tar) | [inference_vehicle .yaml](../../../deploy/configs/inference_vehicle.yaml) |
+| Product Recognition Model | Product Scene | [Model Download Link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/product_ResNet50_vd_aliproduct_v1.0_infer.tar) | [inference_product. yaml](../../../deploy/configs/inference_product.yaml) |
+| Vehicle ReID Model | Vehicle ReID Scene | [Model download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/vehicle_reid_ResNet50_VERIWild_v1.0_infer.tar) | [inference_vehicle .yaml](../../../deploy/configs/inference_vehicle.yaml) |
-
-### 3.3 Recognize the Unknown Category Images
+The above models can be downloaded to the `deploy/models` folder by the following script for use in recognition tasks
+```shell
+cd ./deploy
+mkdir -p models
-To recognize the image `./recognition_demo_data_v1.1/test_product/anmuxi.jpg`, run the command as follows.
+cd ./models
+# Download the generic object detection model for server and unzip it
+wget https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/ppyolov2_r50vd_dcn_mainbody_v1.0_infer.tar && tar -xf ppyolov2_r50vd_dcn_mainbody_v1.0_infer.tar
+# Download the generic recognition model and unzip it
+wget {recognize model download link path} && tar -xf {name of compressed package}
+```
+
+Then use the following scripts to download the test data for other recognition scenario:
```shell
-# using the following command to predict using GPU, you can append `-o Global.use_gpu=False` to predict using CPU.
-python3.7 python/predict_system.py -c configs/inference_product.yaml -o Global.infer_imgs="./recognition_demo_data_v1.1/test_product/anmuxi.jpg" -o IndexProcess.index_dir="./recognition_demo_data_v1.1/gallery_product/index_update"
+# Go back to the deploy directory
+cd..
+# Download test data and unzip
+wget https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/data/recognition_demo_data_en_v1.1.tar && tar -xf recognition_demo_data_en_v1.1.tar
```
-The output is as follows:
+After decompression, the `recognition_demo_data_v1.1` folder should have the following file structure:
-```
-[{'bbox': [243, 80, 523, 522], 'rec_docs': 'Anmuxi Ambrosial Yogurt', 'rec_scores': 0.5570770502090454}]
+```log
+├── recognition_demo_data_v1.1
+│ ├── gallery_cartoon
+│ ├── gallery_logo
+│ ├── gallery_product
+│ ├── gallery_vehicle
+│ ├── test_cartoon
+│ ├── test_logo
+│ ├── test_product
+│ └── test_vehicle
+├── ...
```
-The final recognition result is `Anmuxi Ambrosial Yogurt`, which is corrrect, the visualization result is as follows.
+After downloading the model and test data according to the above steps, you can re-build the index database and test the relevant recognition model.
-
-
+* For more introduction to object detection, please refer to: [Object Detection Tutorial Document](../image_recognition_pipeline/mainbody_detection.md); for the introduction of feature extraction, please refer to: [Feature Extraction Tutorial Document](../image_recognition_pipeline/feature_extraction.md); for the introduction to vector search, please refer to: [vector search tutorial document](../image_recognition_pipeline/vector_search.md).
diff --git a/docs/images/deep_hash/DCH.png b/docs/images/deep_hash/DCH.png
new file mode 100644
index 0000000000000000000000000000000000000000..63cf004dac575c4b1badeca8883d9371c6a45eb9
Binary files /dev/null and b/docs/images/deep_hash/DCH.png differ
diff --git a/docs/images/deep_hash/DSHSD.png b/docs/images/deep_hash/DSHSD.png
new file mode 100644
index 0000000000000000000000000000000000000000..b4d0406990f7280d191c4d2f23be12a968d7b350
Binary files /dev/null and b/docs/images/deep_hash/DSHSD.png differ
diff --git a/docs/images/deep_hash/LCDSH.png b/docs/images/deep_hash/LCDSH.png
new file mode 100644
index 0000000000000000000000000000000000000000..4717283981f518a7764e28fe7c66102ee4c47ca0
Binary files /dev/null and b/docs/images/deep_hash/LCDSH.png differ
diff --git a/docs/images/det/PaddleDetection_config.png b/docs/images/det/PaddleDetection_config.png
index d18932b66cc148b7796fe4b319ad9eb82c2a2868..53248227535c9089ca1f70fbba0d69f224fd8a54 100644
Binary files a/docs/images/det/PaddleDetection_config.png and b/docs/images/det/PaddleDetection_config.png differ
diff --git a/docs/images/ppshitu_application_scenarios/100sports.jpg b/docs/images/ppshitu_application_scenarios/100sports.jpg
new file mode 100644
index 0000000000000000000000000000000000000000..baca0eb2e2a57abfa3c90c89e95ffb59ebbde994
Binary files /dev/null and b/docs/images/ppshitu_application_scenarios/100sports.jpg differ
diff --git a/docs/images/ppshitu_application_scenarios/104flowers.jpeg b/docs/images/ppshitu_application_scenarios/104flowers.jpeg
new file mode 100644
index 0000000000000000000000000000000000000000..e420fcc655836e9bf3e214c7561ccd39b71333bf
Binary files /dev/null and b/docs/images/ppshitu_application_scenarios/104flowers.jpeg differ
diff --git a/docs/images/ppshitu_application_scenarios/AID.jpg b/docs/images/ppshitu_application_scenarios/AID.jpg
new file mode 100644
index 0000000000000000000000000000000000000000..569c4c426b37ae4947e376e8d084e31ff896a69f
Binary files /dev/null and b/docs/images/ppshitu_application_scenarios/AID.jpg differ
diff --git a/docs/images/ppshitu_application_scenarios/AnimalImageDataset.jpg b/docs/images/ppshitu_application_scenarios/AnimalImageDataset.jpg
new file mode 100644
index 0000000000000000000000000000000000000000..da88a61b9c8201a136316ce719193d65cf99e918
Binary files /dev/null and b/docs/images/ppshitu_application_scenarios/AnimalImageDataset.jpg differ
diff --git a/docs/images/ppshitu_application_scenarios/Ball.jpg b/docs/images/ppshitu_application_scenarios/Ball.jpg
new file mode 100644
index 0000000000000000000000000000000000000000..6d8e2ff079b9aaf6fd89eb273eed7522d47a4311
Binary files /dev/null and b/docs/images/ppshitu_application_scenarios/Ball.jpg differ
diff --git a/docs/images/ppshitu_application_scenarios/Bird400.jpg b/docs/images/ppshitu_application_scenarios/Bird400.jpg
new file mode 100644
index 0000000000000000000000000000000000000000..c86abd202132e63965492d05147c6a6d2acdccbf
Binary files /dev/null and b/docs/images/ppshitu_application_scenarios/Bird400.jpg differ
diff --git a/docs/images/ppshitu_application_scenarios/Boat.jpg b/docs/images/ppshitu_application_scenarios/Boat.jpg
new file mode 100644
index 0000000000000000000000000000000000000000..f3510422648326766252a83ed29b36ecdd934f4d
Binary files /dev/null and b/docs/images/ppshitu_application_scenarios/Boat.jpg differ
diff --git a/docs/images/ppshitu_application_scenarios/Butterfly.jpg b/docs/images/ppshitu_application_scenarios/Butterfly.jpg
new file mode 100644
index 0000000000000000000000000000000000000000..b17373a74008ec5c40e57d060dce67b06620660c
Binary files /dev/null and b/docs/images/ppshitu_application_scenarios/Butterfly.jpg differ
diff --git a/docs/images/ppshitu_application_scenarios/DogBreeds.jpg b/docs/images/ppshitu_application_scenarios/DogBreeds.jpg
new file mode 100644
index 0000000000000000000000000000000000000000..4d82a2ca2213b420b5cb372ffbf388d527bc065d
Binary files /dev/null and b/docs/images/ppshitu_application_scenarios/DogBreeds.jpg differ
diff --git a/docs/images/ppshitu_application_scenarios/FashionProductsImage.jpg b/docs/images/ppshitu_application_scenarios/FashionProductsImage.jpg
new file mode 100644
index 0000000000000000000000000000000000000000..4e6a7df8d359063cdec0e499e5c8a89f1630b287
Binary files /dev/null and b/docs/images/ppshitu_application_scenarios/FashionProductsImage.jpg differ
diff --git a/docs/images/ppshitu_application_scenarios/Garbage12.jpg b/docs/images/ppshitu_application_scenarios/Garbage12.jpg
new file mode 100644
index 0000000000000000000000000000000000000000..1543820443254f63a9a307c771e3479ba1b0a9f9
Binary files /dev/null and b/docs/images/ppshitu_application_scenarios/Garbage12.jpg differ
diff --git a/docs/images/ppshitu_application_scenarios/Gemstones.jpg b/docs/images/ppshitu_application_scenarios/Gemstones.jpg
new file mode 100644
index 0000000000000000000000000000000000000000..c025330aeaf27483d4f79cf852a6b8e6cc1e38c5
Binary files /dev/null and b/docs/images/ppshitu_application_scenarios/Gemstones.jpg differ
diff --git a/docs/images/ppshitu_application_scenarios/Logo3K.jpg b/docs/images/ppshitu_application_scenarios/Logo3K.jpg
new file mode 100644
index 0000000000000000000000000000000000000000..465ac49a7ddded3a8f4ab66d0657aae1988eec1c
Binary files /dev/null and b/docs/images/ppshitu_application_scenarios/Logo3K.jpg differ
diff --git a/docs/images/ppshitu_application_scenarios/MusicInstruments.jpg b/docs/images/ppshitu_application_scenarios/MusicInstruments.jpg
new file mode 100644
index 0000000000000000000000000000000000000000..0a6c02b1103d987e59c7fe8d7aa27b9ef6abf474
Binary files /dev/null and b/docs/images/ppshitu_application_scenarios/MusicInstruments.jpg differ
diff --git a/docs/images/ppshitu_application_scenarios/Paris.jpg b/docs/images/ppshitu_application_scenarios/Paris.jpg
new file mode 100644
index 0000000000000000000000000000000000000000..724ae3a85389adeaffbfc4014938c733383a5bb5
Binary files /dev/null and b/docs/images/ppshitu_application_scenarios/Paris.jpg differ
diff --git a/docs/images/ppshitu_application_scenarios/Pokemon.png b/docs/images/ppshitu_application_scenarios/Pokemon.png
new file mode 100644
index 0000000000000000000000000000000000000000..ca7d438a9c86d305931a3a572cc06da7226bd80f
Binary files /dev/null and b/docs/images/ppshitu_application_scenarios/Pokemon.png differ
diff --git a/docs/images/ppshitu_application_scenarios/Shoes.jpeg b/docs/images/ppshitu_application_scenarios/Shoes.jpeg
new file mode 100644
index 0000000000000000000000000000000000000000..d788e679cf3a46e5f836d9c5cb84d165d3623c53
Binary files /dev/null and b/docs/images/ppshitu_application_scenarios/Shoes.jpeg differ
diff --git a/docs/images/ppshitu_application_scenarios/TreeNuts.jpg b/docs/images/ppshitu_application_scenarios/TreeNuts.jpg
new file mode 100644
index 0000000000000000000000000000000000000000..3211a8d0c76da60b7aeb340a62f0804c6cc43513
Binary files /dev/null and b/docs/images/ppshitu_application_scenarios/TreeNuts.jpg differ
diff --git a/docs/images/ppshitu_application_scenarios/Vechicles.jpg b/docs/images/ppshitu_application_scenarios/Vechicles.jpg
new file mode 100644
index 0000000000000000000000000000000000000000..c22686864fa8128067ed6acb24c234afe97a5271
Binary files /dev/null and b/docs/images/ppshitu_application_scenarios/Vechicles.jpg differ
diff --git a/docs/images/ppshitu_application_scenarios/Veg200.jpg b/docs/images/ppshitu_application_scenarios/Veg200.jpg
new file mode 100644
index 0000000000000000000000000000000000000000..d64331217966f354b0de2befaa3be575a03dd8d7
Binary files /dev/null and b/docs/images/ppshitu_application_scenarios/Veg200.jpg differ
diff --git a/docs/images/ppshitu_application_scenarios/WeatherImageRecognition.jpg b/docs/images/ppshitu_application_scenarios/WeatherImageRecognition.jpg
new file mode 100644
index 0000000000000000000000000000000000000000..c0636d7f9e2e3dc08c56050008a010e6a4ce4b54
Binary files /dev/null and b/docs/images/ppshitu_application_scenarios/WeatherImageRecognition.jpg differ
diff --git a/docs/images/ppshitu_application_scenarios/WildEdiblePlants.jpg b/docs/images/ppshitu_application_scenarios/WildEdiblePlants.jpg
new file mode 100644
index 0000000000000000000000000000000000000000..feb7cbb09750dec537ecb40dc5faaa9576afca40
Binary files /dev/null and b/docs/images/ppshitu_application_scenarios/WildEdiblePlants.jpg differ
diff --git a/docs/images/ppshitu_application_scenarios/systerm_result.jpg b/docs/images/ppshitu_application_scenarios/systerm_result.jpg
new file mode 100644
index 0000000000000000000000000000000000000000..ab9bc1cb523d62dcd36864ef3487bbcff42e2ae3
Binary files /dev/null and b/docs/images/ppshitu_application_scenarios/systerm_result.jpg differ
diff --git a/docs/images/quick_start/android_demo/PPShiTu_qrcode.png b/docs/images/quick_start/android_demo/PPShiTu_qrcode.png
new file mode 100644
index 0000000000000000000000000000000000000000..6b75ce4f3d460b6bb49e8c3eb13af963002eac83
Binary files /dev/null and b/docs/images/quick_start/android_demo/PPShiTu_qrcode.png differ
diff --git a/docs/images/quick_start/android_demo/android_nongfu_spring.JPG b/docs/images/quick_start/android_demo/android_nongfu_spring.JPG
new file mode 100644
index 0000000000000000000000000000000000000000..9c1d4ffc87082dfb73c5634f3d5454f72f8c1d98
Binary files /dev/null and b/docs/images/quick_start/android_demo/android_nongfu_spring.JPG differ
diff --git a/docs/images/quick_start/android_demo/baocunxiugai_100.png b/docs/images/quick_start/android_demo/baocunxiugai_100.png
new file mode 100644
index 0000000000000000000000000000000000000000..94f7ba7f16b418a5a461ab3d903a976b4fd86120
Binary files /dev/null and b/docs/images/quick_start/android_demo/baocunxiugai_100.png differ
diff --git a/docs/images/quick_start/android_demo/bendishangchuan_100.png b/docs/images/quick_start/android_demo/bendishangchuan_100.png
new file mode 100644
index 0000000000000000000000000000000000000000..a9263a32e88ef3d9b29268874ac5beb87f836e63
Binary files /dev/null and b/docs/images/quick_start/android_demo/bendishangchuan_100.png differ
diff --git a/docs/images/quick_start/android_demo/bendishibie_100.png b/docs/images/quick_start/android_demo/bendishibie_100.png
new file mode 100644
index 0000000000000000000000000000000000000000..ee1715d8aed47eeb36f90848416353ce6624a22e
Binary files /dev/null and b/docs/images/quick_start/android_demo/bendishibie_100.png differ
diff --git a/docs/images/quick_start/android_demo/leibiechaxun_100.png b/docs/images/quick_start/android_demo/leibiechaxun_100.png
new file mode 100644
index 0000000000000000000000000000000000000000..f8e71b864cd21896795ffdaf3869e1b098e0a5bb
Binary files /dev/null and b/docs/images/quick_start/android_demo/leibiechaxun_100.png differ
diff --git a/docs/images/quick_start/android_demo/paizhaoshangchuan_100.png b/docs/images/quick_start/android_demo/paizhaoshangchuan_100.png
new file mode 100644
index 0000000000000000000000000000000000000000..3f3b5544b303353b1ccf486bba207ad4bfe027e5
Binary files /dev/null and b/docs/images/quick_start/android_demo/paizhaoshangchuan_100.png differ
diff --git a/docs/images/quick_start/android_demo/paizhaoshibie_100.png b/docs/images/quick_start/android_demo/paizhaoshibie_100.png
new file mode 100644
index 0000000000000000000000000000000000000000..13df35310c415273abd3577df2c38d2ff82fedb9
Binary files /dev/null and b/docs/images/quick_start/android_demo/paizhaoshibie_100.png differ
diff --git a/docs/images/quick_start/android_demo/reset_100.png b/docs/images/quick_start/android_demo/reset_100.png
new file mode 100644
index 0000000000000000000000000000000000000000..93f6f6223a890bd442bf9a6dd5bbb5ba2514138c
Binary files /dev/null and b/docs/images/quick_start/android_demo/reset_100.png differ
diff --git a/docs/images/recognition/drink_data_demo/output/mosilian.jpeg b/docs/images/recognition/drink_data_demo/output/mosilian.jpeg
index ca7f2dbb0575001ff79b81a9fad2827cbd5261cb..909ddd91fb36e2fddb794ef96e62059dc331370d 100644
Binary files a/docs/images/recognition/drink_data_demo/output/mosilian.jpeg and b/docs/images/recognition/drink_data_demo/output/mosilian.jpeg differ
diff --git a/docs/images/recognition/drink_data_demo/test_images/100.jpeg b/docs/images/recognition/drink_data_demo/test_images/100.jpeg
new file mode 100644
index 0000000000000000000000000000000000000000..e5f845ed2842f67352ffda26380b59a35d0449f9
Binary files /dev/null and b/docs/images/recognition/drink_data_demo/test_images/100.jpeg differ
diff --git a/docs/images/shitu_index_manager/all_menu.png b/docs/images/shitu_index_manager/all_menu.png
new file mode 100644
index 0000000000000000000000000000000000000000..768041a77d89e76c77fdc445bfb1bafd7e64d1fa
Binary files /dev/null and b/docs/images/shitu_index_manager/all_menu.png differ
diff --git a/docs/images/shitu_index_manager/creat_images.png b/docs/images/shitu_index_manager/creat_images.png
new file mode 100644
index 0000000000000000000000000000000000000000..8a2d7914c754b4b9d810ed3ac094bbae57dc0758
Binary files /dev/null and b/docs/images/shitu_index_manager/creat_images.png differ
diff --git a/docs/images/shitu_index_manager/image_operation.png b/docs/images/shitu_index_manager/image_operation.png
new file mode 100644
index 0000000000000000000000000000000000000000..86457fe5e439d793cd7920a746e9500f758f9bf5
Binary files /dev/null and b/docs/images/shitu_index_manager/image_operation.png differ
diff --git a/docs/images/shitu_index_manager/main_page.png b/docs/images/shitu_index_manager/main_page.png
new file mode 100644
index 0000000000000000000000000000000000000000..2b69ca7f4efb12ff1724a2a2598a8b7f8b0ca241
Binary files /dev/null and b/docs/images/shitu_index_manager/main_page.png differ
diff --git a/docs/images/shitu_index_manager/menu.png b/docs/images/shitu_index_manager/menu.png
new file mode 100644
index 0000000000000000000000000000000000000000..90dd4c709a85b76f3f284069c04ff74d1ed25720
Binary files /dev/null and b/docs/images/shitu_index_manager/menu.png differ
diff --git a/docs/images/shituv2.gif b/docs/images/shituv2.gif
new file mode 100644
index 0000000000000000000000000000000000000000..5a9a4b84232f813ea866e5c0f283c6768823374b
Binary files /dev/null and b/docs/images/shituv2.gif differ
diff --git a/docs/images/structure.jpg b/docs/images/structure.jpg
index 1d7f5a17b5377606a0e69a6fb45fdab1652af5d0..14948bd4214f4cb321e9a6b9819f2a42ba3ac385 100644
Binary files a/docs/images/structure.jpg and b/docs/images/structure.jpg differ
diff --git a/docs/zh_CN/PPShiTu/PPShiTuV2_introduction.md b/docs/zh_CN/PPShiTu/PPShiTuV2_introduction.md
new file mode 100644
index 0000000000000000000000000000000000000000..bac4f8acbaf6da54e1bb15a44d151a9a37a20769
--- /dev/null
+++ b/docs/zh_CN/PPShiTu/PPShiTuV2_introduction.md
@@ -0,0 +1,254 @@
+## PP-ShiTu V2图像识别系统
+
+## 目录
+
+- [1. PP-ShiTu V2模型和应用场景介绍](#1-pp-shituv2模型和应用场景介绍)
+- [2. 模型快速体验](#2-模型快速体验)
+ - [2.1 PP-ShiTu android demo 快速体验](#21-pp-shitu-android-demo-快速体验)
+ - [2.2 命令行代码快速体验](#22-命令行代码快速体验)
+- [3 模块介绍与训练](#3-模块介绍与训练)
+ - [3.1 主体检测](#31-主体检测)
+ - [3.2 特征提取](#32-特征提取)
+ - [3.3 向量检索](#33-向量检索)
+- [4. 推理部署](#4-推理部署)
+ - [4.1 推理模型准备](#41-推理模型准备)
+ - [4.1.1 基于训练得到的权重导出 inference 模型](#411-基于训练得到的权重导出-inference-模型)
+ - [4.1.2 直接下载 inference 模型](#412-直接下载-inference-模型)
+ - [4.2 测试数据准备](#42-测试数据准备)
+ - [4.3 基于 Python 预测引擎推理](#43-基于-python-预测引擎推理)
+ - [4.3.1 预测单张图像](#431-预测单张图像)
+ - [4.3.2 基于文件夹的批量预测](#432-基于文件夹的批量预测)
+ - [4.4 基于 C++ 预测引擎推理](#44-基于-c-预测引擎推理)
+ - [4.5 服务化部署](#45-服务化部署)
+ - [4.6 端侧部署](#46-端侧部署)
+ - [4.7 Paddle2ONNX 模型转换与预测](#47-paddle2onnx-模型转换与预测)
+- [参考文献](#参考文献)
+
+## 1. PP-ShiTuV2模型和应用场景介绍
+
+PP-ShiTuV2 是基于 PP-ShiTuV1 改进的一个实用轻量级通用图像识别系统，由主体检测、特征提取、向量检索三个模块构成，相比 PP-ShiTuV1 具有更高的识别精度、更强的泛化能力以及相近的推理速度*。主要针对训练数据集、特征提取两个部分进行优化，使用了更优的骨干网络、损失函数与训练策略，使得 PP-ShiTuV2 在多个实际应用场景上的检索性能有显著提升。
+
+**本文档提供了用户使用 PaddleClas 的 PP-ShiTuV2 图像识别方案进行快速构建轻量级、高精度、可落地的图像识别pipeline。该pipeline可以广泛应用于商场商品识别场景、安防人脸或行人识别场景、海量图像检索过滤等场景中。**
+
+
+
+
+
+
+界面按钮展示
+
+主要功能展示
+
+新建库
+
+打开库
+
+图像操作
+
|各种球类识别 | 0.9769 | [Balls](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/data/PP-ShiTuV2_application_dataset/Balls.tar) | [原数据下载地址](https://www.kaggle.com/datasets/gpiosenka/balls-image-classification) |
+| 狗识别 | 
| 狗细分类识别，包括69种狗的图像 | 0.9606 | [DogBreeds](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/data/PP-ShiTuV2_application_dataset/DogBreeds.tar) | [原数据下载地址](https://www.kaggle.com/datasets/gpiosenka/70-dog-breedsimage-data-set) |
+| 宝石 | 
| 宝石种类识别 | 0.9653 | [Gemstones](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/data/PP-ShiTuV2_application_dataset/Gemstones.tar) | [原数据下载地址](https://www.kaggle.com/datasets/lsind18/gemstones-images) |
+| 动物 | 
|各种动物识别 | 0.9078 | [AnimalImageDataset](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/data/PP-ShiTuV2_application_dataset/AnimalImageDataset.tar) | [原数据下载地址](https://www.kaggle.com/datasets/iamsouravbanerjee/animal-image-dataset-90-different-animals) |
+| 鸟类 | 
|鸟细分类识别，包括400种各种姿态的鸟类图像 | 0.9673 | [Bird400](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/data/PP-ShiTuV2_application_dataset/Bird400.tar) | [原数据下载地址](https://www.kaggle.com/datasets/gpiosenka/100-bird-species) |
+| 交通工具 | 
|车、船等交通工具粗分类识别 | 0.9307 | [Vechicles](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/data/PP-ShiTuV2_application_dataset/Vechicles.tar) | [原数据下载地址](https://www.kaggle.com/datasets/rishabkoul1/vechicle-dataset) |
+| 花 | 
|104种花细分类识别 | 0.9788 | [104flowers](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/data/PP-ShiTuV2_application_dataset/104flowrs.tar) | [原数据下载地址](https://www.kaggle.com/datasets/msheriey/104-flowers-garden-of-eden) |
+| 运动种类 | 
|100种运动图像识别 | 0.9413 | [100sports](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/data/PP-ShiTuV2_application_dataset/100sports.tar) | [原数据下载地址](https://www.kaggle.com/datasets/gpiosenka/sports-classification) |
+| 乐器 | 
|30种不同乐器种类识别 | 0.9467 | [MusicInstruments](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/data/PP-ShiTuV2_application_dataset/MusicInstruments.tar) | [原数据下载地址](https://www.kaggle.com/datasets/gpiosenka/musical-instruments-image-classification) |
+| 宝可梦 | 
|宝可梦神奇宝贝识别 | 0.9236 | [Pokemon](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/data/PP-ShiTuV2_application_dataset/Pokemon.tar) | [原数据下载地址](https://www.kaggle.com/datasets/lantian773030/pokemonclassification) |
+| 船 | 
|船种类识别 |0.9242 | [Boat](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/data/PP-ShiTuV2_application_dataset/Boat.tar) | [原数据下载地址](https://www.kaggle.com/datasets/imsparsh/dockship-boat-type-classification) |
+| 鞋子 | 
|鞋子种类识别，包括靴子、拖鞋等 | 0.9000 | [Shoes](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/data/PP-ShiTuV2_application_dataset/Shoes.tar) | [原数据下载地址](https://www.kaggle.com/datasets/noobyogi0100/shoe-dataset) |
+| 巴黎建筑 | 
|巴黎著名建筑景点识别，如：巴黎铁塔、圣母院等 | 1.000 | [Paris](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/data/PP-ShiTuV2_application_dataset/Paris.tar) | [原数据下载地址](https://www.kaggle.com/datasets/skylord/oxbuildings) |
+| 蝴蝶 | 
|75种蝴蝶细分类识别 | 0.9360 | [Butterfly](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/data/PP-ShiTuV2_application_dataset/Butterfly.tar) | [原数据下载地址](https://www.kaggle.com/datasets/gpiosenka/butterfly-images40-species) |
+| 野外植物 | 
|野外植物识别 | 0.9758 | [WildEdiblePlants](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/data/PP-ShiTuV2_application_dataset/WildEdiblePlants.tar) | [原数据下载地址](https://www.kaggle.com/datasets/ryanpartridge01/wild-edible-plants) |
+| 天气 | 
|各种天气场景识别，如：雨天、打雷、下雪等 | 0.9924 | [WeatherImageRecognition](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/data/PP-ShiTuV2_application_dataset/WeatherImageRecognition.tar) | [原数据下载地址](https://www.kaggle.com/datasets/jehanbhathena/weather-dataset) |
+| 坚果 | 
|各种坚果种类识别 | 0.9412 | [TreeNuts](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/data/PP-ShiTuV2_application_dataset/TreeNuts.tar) | [原数据下载地址](https://www.kaggle.com/datasets/gpiosenka/tree-nuts-image-classification) |
+| 时装 | 
|首饰、挎包、化妆品等时尚商品识别 | 0.9555 | [FashionProductImageSmall](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/data/PP-ShiTuV2_application_dataset/FashionProductImageSmall.tar) | [原数据下载地址](https://www.kaggle.com/datasets/paramaggarwal/fashion-product-images-small) |
+| 垃圾 | 
|12种垃圾分类识别 | 0.9845 | [Garbage12](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/data/PP-ShiTuV2_application_dataset/Garbage12.tar) | [原数据下载地址](https://www.kaggle.com/datasets/mostafaabla/garbage-classification) |
+| 航拍场景 | 
|各种航拍场景识别，如机场、火车站等 | 0.9797 | [AID](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/data/PP-ShiTuV2_application_dataset/AID.tar) | [原数据下载地址](https://www.kaggle.com/datasets/jiayuanchengala/aid-scene-classification-datasets) |
+| 蔬菜 | 
|各种蔬菜识别 | 0.8929 | [Veg200](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/data/PP-ShiTuV2_application_dataset/Veg200.tar) | [原数据下载地址](https://www.kaggle.com/datasets/zhaoyj688/vegfru) |
+| 商标 | 
|两千多种logo识别 | 0.9313 | [Logo3k](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/data/PP-ShiTuV2_application_dataset/Logo3k.tar) | [原数据下载地址](https://github.com/Wangjing1551/LogoDet-3K-Dataset) |
+
+
+
+
+
+## 2. 使用说明
+
+
+
+### 2.1 环境配置
+- 安装：请先参考文档[环境准备](../installation/install_paddleclas.md)配置PaddleClas运行环境
+- 进入`deploy`运行目录，本部分所有内容与命令均需要在`deploy`目录下运行，可以通过下面命令进入`deploy`目录。
+```shell
+cd deploy
+```
+
+
+
+
+### 2.2 下载、解压场景库数据
+首先创建存放场景库的地址`deploy/datasets`:
+
+```shell
+mkdir datasets
+```
+下载并解压对应场景库到`deploy/datasets`中。
+```shell
+cd datasets
+
+# 下载并解压场景库数据
+wget {场景库下载链接} && tar -xf {压缩包的名称}
+```
+以`dataset_name`为例，解压完毕后，`datasets/dataset_name`文件夹下应有如下文件结构：
+```shel
+├── dataset_name/
+│ ├── Gallery/
+│ ├── Index/
+│ ├── Query/
+│ ├── gallery_list.txt/
+│ ├── query_list.txt/
+│ ├── label_list.txt/
+├── ...
+```
+其中，`Gallery`文件夹中存放的是用于构建索引库的原始图像，`Index`表示基于原始图像构建得到的索引库信息，`Query`文件夹存放的是用于检索的图像列表，`gallery_list.txt`和`query_list.txt`分别为索引库和检索图像的标签文件，`label_list.txt`是标签的中英文对照文件（注意：商标场景库文件不包含中英文对照文件）。
+
+
+
+### 2.3 准备模型
+创建存放模型的文件夹`deploy/models`，并下载轻量级主体检测、识别模型，命令如下：
+```shell
+cd ..
+mkdir models
+cd models
+
+# 下载检测模型并解压
+# wget {检测模型下载链接} && tar -xf {检测模型压缩包名称}
+wget https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/PP-ShiTuV2/general_PPLCNetV2_base_pretrained_v1.0_infer.tar && tar -xf general_PPLCNetV2_base_pretrained_v1.0_infer.tar
+
+# 下载识别 inference 模型并解压
+#wget {识别模型下载链接} && tar -xf {识别模型压缩包名称}
+wget https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/picodet_PPLCNet_x2_5_mainbody_lite_v1.0_infer.tar && tar -xf picodet_PPLCNet_x2_5_mainbody_lite_v1.0_infer.tar
+```
+
+解压完成后，`models`文件夹下有如下文件结构：
+```
+├── inference_model_name
+│ ├── inference.pdiparams
+│ ├── inference.pdiparams.info
+│ └── inference.pdmodel
+└── det_model_name
+ ├── inference.pdiparams
+ ├── inference.pdiparams.info
+ └── inference.pdmodel
+```
+
+
+
+### 2.4 场景库识别与检索
+
+以`动物识别`场景为例，展示识别和检索过程（如果希望尝试其他场景库的识别与检索效果，在下载解压好对应的场景库数据和模型后，替换对应的配置文件即可完成预测）。
+
+注意，此部分使用了`faiss`作为检索库，安装方法如下：
+```shell
+pip install faiss-cpu==1.7.1post2
+```
+
+若使用时，不能正常引用，则`uninstall`之后，重新`install`，尤其是在windows下。
+
+
+
+#### 2.4.1 识别单张图像
+
+假设需要测试`./datasets/AnimalImageDataset/Query/羚羊/0a37838e99.jpg`这张图像识别和检索效果。
+
+首先分别修改配置文件`./configs/inference_general.yaml`中的`Global.det_inference_model_dir`和`Global.rec_inference_model_dir`字段为对应的检测和识别模型文件夹，以及修改测试图像地址字段`Global.infer_imgs`示例如下：
+
+```shell
+Global:
+ infer_imgs: './datasets/AnimalImageDataset/Query/羚羊/0a37838e99.jpg'
+ det_inference_model_dir: './models/picodet_PPLCNet_x2_5_mainbody_lite_v1.0_infer.tar'
+ rec_inference_model_dir: './models/general_PPLCNetV2_base_pretrained_v1.0_infer.tar'
+```
+
+并修改配置文件`./configs/inference_general.yaml`中的`IndexProcess.index_dir`字段为对应场景index库地址：
+
+```shell
+IndexProcess:
+ index_dir:'./datasets/AnimalImageDataset/Index/'
+```
+
+
+运行下面的命令，对图像`./datasets/AnimalImageDataset/Query/羚羊/0a37838e99.jpg`进行识别与检索
+
+```shell
+# 使用下面的命令使用 GPU 进行预测
+python3.7 python/predict_system.py -c configs/inference_general.yaml
+
+# 使用下面的命令使用 CPU 进行预测
+python3.7 python/predict_system.py -c configs/inference_general.yaml -o Global.use_gpu=False
+```
+
+最终输出结果如下：
+```
+[{'bbox': [609, 70, 1079, 629], 'rec_docs': '羚羊', 'rec_scores': 0.6571544}]
+```
+其中`bbox`表示检测出的主体所在位置，`rec_docs`表示索引库中与检测框最为相似的类别，`rec_scores`表示对应的置信度。
+检测的可视化结果也保存在`output`文件夹下，对于本张图像，识别结果可视化如下所示。
+
+
+
+
+
+#### 2.4.2 基于文件夹的批量识别
+
+如果希望预测文件夹内的图像，可以直接修改配置文件中`Global.infer_imgs`字段，也可以通过下面的`-o`参数修改对应的配置。
+
+```shell
+# 使用下面的命令使用 GPU 进行预测，如果希望使用 CPU 预测，可以在命令后面添加 -o Global.use_gpu=False
+python3.7 python/predict_system.py -c configs/inference_general.yaml -o Global.infer_imgs="./datasets/AnimalImageDataset/Query/羚羊"
+```
+终端中会输出该文件夹内所有图像的识别结果，如下所示。
+```
+...
+[{'bbox': [0, 0, 1200, 675], 'rec_docs': '羚羊', 'rec_scores': 0.6153812}]
+[{'bbox': [0, 0, 275, 183], 'rec_docs': '羚羊', 'rec_scores': 0.77218026}]
+[{'bbox': [264, 79, 1088, 850], 'rec_docs': '羚羊', 'rec_scores': 0.81452656}]
+[{'bbox': [0, 0, 188, 268], 'rec_docs': '羚羊', 'rec_scores': 0.637074}]
+[{'bbox': [118, 41, 235, 161], 'rec_docs': '羚羊', 'rec_scores': 0.67315465}]
+[{'bbox': [0, 0, 175, 287], 'rec_docs': '羚羊', 'rec_scores': 0.68271667}]
+[{'bbox': [0, 0, 310, 163], 'rec_docs': '羚羊', 'rec_scores': 0.6706451}]
+...
+```
+所有图像的识别结果可视化图像也保存在`output`文件夹内。
diff --git a/docs/zh_CN/models/LeViT.md b/docs/zh_CN/models/LeViT.md
index 5f0e480047adc612850fdd1be9e8de8e978e898e..d8aaa744b4c7312d8c6a5c186ae48a0e671171c2 100644
--- a/docs/zh_CN/models/LeViT.md
+++ b/docs/zh_CN/models/LeViT.md
@@ -18,7 +18,7 @@ LeViT 是一种快速推理的、用于图像分类任务的混合神经网络
| Models | Top1 | Top5 | Reference
-
-体验整体图像识别系统，或查看特征库建立方法，详见[图像识别快速开始文档](../quick_start/quick_start_recognition.md)。其中，图像识别快速开始文档主要讲解整体流程的使用过程。以下内容，主要对上述三个步骤的训练部分进行介绍。
+在Android端或PC端体验整体图像识别系统，或查看特征库建立方法，可以参考 [图像识别快速开始文档](../quick_start/quick_start_recognition.md)。
-首先，请参考[安装指南](../installation/install_paddleclas.md)配置运行环境。
+以下内容，主要对上述三个步骤的训练部分进行介绍。
-## 目录
+在训练开始之前，请参考 [安装指南](../installation/install_paddleclas.md) 配置运行环境。
-- [1. 主体检测](#1)
-- [2. 特征模型训练](#2)
- - [2.1. 特征模型数据准备与处理](#2.1)
- - [2. 2 特征模型基于单卡 GPU 上的训练与评估](#2.2)
- - [2.2.1 特征模型训练](#2.2.2)
- - [2.2.2 特征模型恢复训练](#2.2.2)
- - [2.2.3 特征模型评估](#2.2.3)
- - [2.3 特征模型导出 inference 模型](#2.3)
-- [3. 特征检索](#3)
-- [4. 基础知识](#4)
+## 目录
-
+- [1. 主体检测](#1-主体检测)
+- [2. 特征提取模型训练](#2-特征提取模型训练)
+ - [2.1 特征提取模型数据的准备与处理](#21-特征提取模型数据的准备与处理)
+ - [2.2 特征提取模型在 GPU 上的训练与评估](#22-特征提取模型在-gpu-上的训练与评估)
+ - [2.2.1 特征提取模型训练](#221-特征提取模型训练)
+ - [2.2.2 特征提取模型恢复训练](#222-特征提取模型恢复训练)
+ - [2.2.3 特征提取模型评估](#223-特征提取模型评估)
+ - [2.3 特征提取模型导出 inference 模型](#23-特征提取模型导出-inference-模型)
+- [3. 特征检索](#3-特征检索)
+- [4. 基础知识](#4-基础知识)
+
+
## 1. 主体检测
@@ -38,142 +43,143 @@
[{u'id': 1, u'name': u'foreground', u'supercategory': u'foreground'}]
```
-关于主体检测训练方法可以参考： [PaddleDetection 训练教程](https://github.com/PaddlePaddle/PaddleDetection/blob/develop/docs/tutorials/GETTING_STARTED_cn.md#4-%E8%AE%AD%E7%BB%83)。
+关于主体检测数据集构造与模型训练方法可以参考： [30分钟快速上手PaddleDetection](https://github.com/PaddlePaddle/PaddleDetection/blob/develop/docs/tutorials/GETTING_STARTED_cn.md#30%E5%88%86%E9%92%9F%E5%BF%AB%E9%80%9F%E4%B8%8A%E6%89%8Bpaddledetection)。
更多关于 PaddleClas 中提供的主体检测的模型介绍与下载请参考：[主体检测教程](../image_recognition_pipeline/mainbody_detection.md)。
-
+
-## 2. 特征模型训练
+## 2. 特征提取模型训练
-
+为了快速体验 PaddleClas 图像检索模块，以下使用经典的200类鸟类细粒度分类数据集 [CUB_200_2011](http://vision.ucsd.edu/sites/default/files/WelinderEtal10_CUB-200.pdf) 为例，介绍特征提取模型训练过程。CUB_200_2011 下载方式请参考 [CUB_200_2011官网](https://www.vision.caltech.edu/datasets/cub_200_2011/)
-### 2.1 特征模型数据的准备与处理
+
-* 进入 `PaddleClas` 目录。
+### 2.1 特征提取模型数据的准备与处理
-```bash
-## linux or mac， $path_to_PaddleClas 表示 PaddleClas 的根目录，用户需要根据自己的真实目录修改
-cd $path_to_PaddleClas
-```
+* 进入 `PaddleClas` 目录
-* 进入 `dataset` 目录，为了快速体验 PaddleClas 图像检索模块，此处使用的数据集为 [CUB_200_2011](http://vision.ucsd.edu/sites/default/files/WelinderEtal10_CUB-200.pdf)，其是一个包含 200 类鸟的细粒度鸟类数据集。首先，下载 CUB_200_2011 数据集，下载方式请参考[官网](http://www.vision.caltech.edu/visipedia/CUB-200-2011.html)。
+ ```shell
+ cd PaddleClas
+ ```
-```shell
-# linux or mac
-cd dataset
+* 进入 `dataset` 目录
-# 将下载后的数据拷贝到此目录
-cp {数据存放的路径}/CUB_200_2011.tgz .
+ ```shell
+ # 进入dataset目录
+ cd dataset
-# 解压
-tar -xzvf CUB_200_2011.tgz
+ # 将下载后的数据拷贝到dataset目录下
+ cp {数据存放的路径}/CUB_200_2011.tgz ./
-#进入 CUB_200_2011 目录
-cd CUB_200_2011
-```
+ # 解压该数据集
+ tar -xzvf CUB_200_2011.tgz
-该数据集在用作图像检索任务时，通常将前 100 类当做训练集，后 100 类当做测试集，所以此处需要将下载的数据集做一些后处理，来更好的适应 PaddleClas 的图像检索训练。
+ #进入 CUB_200_2011 目录
+ cd CUB_200_2011
+ ```
-```shell
-#新建 train 和 test 目录
-mkdir train && mkdir test
+* 该数据集在用作图像检索任务时，通常将前 100 类当做训练集，后 100 类当做测试集，所以此处需要将下载的数据集做一些后处理，来更好的适应 PaddleClas 的图像检索训练。
-#将数据分成训练集和测试集，前 100 类作为训练集，后 100 类作为测试集
-ls images | awk -F "." '{if(int($1)<101)print "mv images/"$0" train/"int($1)}' | sh
-ls images | awk -F "." '{if(int($1)>100)print "mv images/"$0" test/"int($1)}' | sh
+ ```shell
+ #新建 train 和 test 目录
+ mkdir train
+ mkdir test
-#生成 train_list 和 test_list
-tree -r -i -f train | grep jpg | awk -F "/" '{print $0" "int($2) " "NR}' > train_list.txt
-tree -r -i -f test | grep jpg | awk -F "/" '{print $0" "int($2) " "NR}' > test_list.txt
-```
-
-至此，现在已经得到 `CUB_200_2011` 的训练集（`train` 目录）、测试集（`test` 目录）、`train_list.txt`、`test_list.txt`。
+ #将数据分成训练集和测试集，前 100 类作为训练集，后 100 类作为测试集
+ ls images | awk -F "." '{if(int($1)<101)print "mv images/"$0" train/"int($1)}' | sh
+ ls images | awk -F "." '{if(int($1)>100)print "mv images/"$0" test/"int($1)}' | sh
-数据处理完毕后，`CUB_200_2011` 中的 `train` 目录下应有如下结构：
+ #生成 train_list 和 test_list
+ tree -r -i -f train | grep jpg | awk -F "/" '{print $0" "int($2) " "NR}' > train_list.txt
+ tree -r -i -f test | grep jpg | awk -F "/" '{print $0" "int($2) " "NR}' > test_list.txt
+ ```
-```
-├── 1
-│ ├── Black_Footed_Albatross_0001_796111.jpg
-│ ├── Black_Footed_Albatross_0002_55.jpg
- ...
-├── 10
-│ ├── Red_Winged_Blackbird_0001_3695.jpg
-│ ├── Red_Winged_Blackbird_0005_5636.jpg
-...
-```
+ 至此，现在已经得到 `CUB_200_2011` 的训练集（`train` 目录）、测试集（`test` 目录）、`train_list.txt`、`test_list.txt`。
-`train_list.txt` 应为：
+ 数据处理完毕后，`CUB_200_2011` 中的 `train` 目录下应有如下结构：
-```
-train/99/Ovenbird_0137_92639.jpg 99 1
-train/99/Ovenbird_0136_92859.jpg 99 2
-train/99/Ovenbird_0135_93168.jpg 99 3
-train/99/Ovenbird_0131_92559.jpg 99 4
-train/99/Ovenbird_0130_92452.jpg 99 5
-...
-```
-其中，分隔符为空格" ", 三列数据的含义分别是训练数据的路径、训练数据的 label 信息、训练数据的 unique id。
+ ```
+ CUB_200_2011/train/
+ ├── 1
+ │ ├── Black_Footed_Albatross_0001_796111.jpg
+ │ ├── Black_Footed_Albatross_0002_55.jpg
+ ...
+ ├── 10
+ │ ├── Red_Winged_Blackbird_0001_3695.jpg
+ │ ├── Red_Winged_Blackbird_0005_5636.jpg
+ ...
+ ```
-测试集格式与训练集格式相同。
+ `train_list.txt` 应为：
-**注意**：
+ ```
+ train/99/Ovenbird_0137_92639.jpg 99 1
+ train/99/Ovenbird_0136_92859.jpg 99 2
+ train/99/Ovenbird_0135_93168.jpg 99 3
+ train/99/Ovenbird_0131_92559.jpg 99 4
+ train/99/Ovenbird_0130_92452.jpg 99 5
+ ...
+ ```
+ 其中，分隔符为空格`" "`, 三列数据的含义分别是`训练数据的相对路径`、`训练数据的 label 标签`、`训练数据的 unique id`。测试集格式与训练集格式相同。
-* 当 gallery dataset 和 query dataset 相同时，为了去掉检索得到的第一个数据（检索图片本身无须评估），每个数据需要对应一个 unique id，用于后续评测 mAP、recall@1 等指标。关于 gallery dataset 与 query dataset 的解析请参考[图像检索数据集介绍](#图像检索数据集介绍), 关于 mAP、recall@1 等评测指标请参考[图像检索评价指标](#图像检索评价指标)。
+* 构建完毕后返回 `PaddleClas` 根目录
-返回 `PaddleClas` 根目录
+ ```shell
+ # linux or mac
+ cd ../../
+ ```
-```shell
-# linux or mac
-cd ../../
-```
+**注意**：
-
+* 当 gallery dataset 和 query dataset 相同时，为了去掉检索得到的第一个数据（检索图片本身不能出现在gallery中），每个数据需要对应一个 unique id（一般使用从1开始的自然数为unique id，如1,2,3,...），用于后续评测 `mAP`、`recall@1` 等指标。关于 gallery dataset 与 query dataset 的解析请参考[图像检索数据集介绍](#图像检索数据集介绍), 关于 `mAP`、`recall@1` 等评测指标请参考[图像检索评价指标](#图像检索评价指标)。
-### 2.2 特征模型 GPU 上的训练与评估
+
-在基于单卡 GPU 上训练与评估，推荐使用 `tools/train.py` 与 `tools/eval.py` 脚本。
+### 2.2 特征提取模型在 GPU 上的训练与评估
-PaddleClas 支持使用 VisualDL 可视化训练过程。VisualDL 是飞桨可视化分析工具，以丰富的图表呈现训练参数变化趋势、模型结构、数据样本、高维数据分布等。可帮助用户更清晰直观地理解深度学习模型训练过程及模型结构，进而实现高效的模型优化。更多细节请查看[VisualDL](../others/VisualDL.md)。
+下面以 MobileNetV1 模型为例，介绍特征提取模型在 GPU 上的训练与评估流程
-#### 2.2.1 特征模型训练
+#### 2.2.1 特征提取模型训练
准备好配置文件之后，可以使用下面的方式启动图像检索任务的训练。PaddleClas 训练图像检索任务的方法是度量学习，关于度量学习的解析请参考[度量学习](#度量学习)。
```shell
# 单卡 GPU
-python3 tools/train.py \
- -c ./ppcls/configs/quick_start/MobileNetV1_retrieval.yaml \
- -o Arch.Backbone.pretrained=True \
- -o Global.device=gpu
+python3.7 tools/train.py \
+-c ./ppcls/configs/quick_start/MobileNetV1_retrieval.yaml \
+-o Arch.Backbone.pretrained=True \
+-o Global.device=gpu
+
# 多卡 GPU
export CUDA_VISIBLE_DEVICES=0,1,2,3
-python3 -m paddle.distributed.launch tools/train.py \
- -c ./ppcls/configs/quick_start/MobileNetV1_retrieval.yaml \
- -o Arch.Backbone.pretrained=True \
- -o Global.device=gpu
+python3.7 -m paddle.distributed.launch tools/train.py \
+-c ./ppcls/configs/quick_start/MobileNetV1_retrieval.yaml \
+-o Arch.Backbone.pretrained=True \
+-o Global.device=gpu
```
-其中，`-c` 用于指定配置文件的路径，`-o` 用于指定需要修改或者添加的参数，其中 `-o Arch.Backbone.pretrained=True` 表示 Backbone 部分使用预训练模型，此外，`Arch.Backbone.pretrained` 也可以指定具体的模型权重文件的地址，使用时需要换成自己的预训练模型权重文件的路径。`-o Global.device=gpu` 表示使用 GPU 进行训练。如果希望使用 CPU 进行训练，则需要将 `Global.device` 设置为 `cpu`。
+**注**：其中，`-c` 用于指定配置文件的路径，`-o` 用于指定需要修改或者添加的参数，其中 `-o Arch.Backbone.pretrained=True` 表示 Backbone 在训练开始前会加载预训练模型；`-o Arch.Backbone.pretrained` 也可以指定为模型权重文件的路径，使用时换成自己的预训练模型权重文件的路径即可；`-o Global.device=gpu` 表示使用 GPU 进行训练。如果希望使用 CPU 进行训练，则设置 `-o Global.device=cpu`即可。
更详细的训练配置，也可以直接修改模型对应的配置文件。具体配置参数参考[配置文档](config_description.md)。
-运行上述命令，可以看到输出日志，示例如下：
+运行上述训练命令，可以看到输出日志，示例如下：
- ```
- ...
- [Train][Epoch 1/50][Avg]CELoss: 6.59110, TripletLossV2: 0.54044, loss: 7.13154
- ...
- [Eval][Epoch 1][Avg]recall1: 0.46962, recall5: 0.75608, mAP: 0.21238
- ...
- ```
-此处配置文件的 Backbone 是 MobileNetV1，如果想使用其他 Backbone，可以重写参数 `Arch.Backbone.name`，比如命令中增加 `-o Arch.Backbone.name={其他 Backbone}`。此外，由于不同模型 `Neck` 部分的输入维度不同，更换 Backbone 后可能需要改写此处的输入大小，改写方式类似替换 Backbone 的名字。
+ ```log
+ ...
+ [Train][Epoch 1/50][Avg]CELoss: 6.59110, TripletLossV2: 0.54044, loss: 7.13154
+ ...
+ [Eval][Epoch 1][Avg]recall1: 0.46962, recall5: 0.75608, mAP: 0.21238
+ ...
+ ```
+
+此处配置文件的 Backbone 是 MobileNetV1，如果想使用其他 Backbone，可以重写参数 `Arch.Backbone.name`，比如命令中增加 `-o Arch.Backbone.name={其他 Backbone 的名字}`。此外，由于不同模型 `Neck` 部分的输入维度不同，更换 Backbone 后可能需要改写 `Neck` 的输入大小，改写方式类似替换 Backbone 的名字。
在训练 Loss 部分，此处使用了 [CELoss](../../../ppcls/loss/celoss.py) 和 [TripletLossV2](../../../ppcls/loss/triplet.py)，配置文件如下：
-```
+```yaml
Loss:
Train:
- CELoss:
@@ -183,110 +189,113 @@ Loss:
margin: 0.5
```
-最终的总 Loss 是所有 Loss 的加权和，其中 weight 定义了特定 Loss 在最终总 Loss 的权重。如果想替换其他 Loss，也可以在配置文件中更改 Loss 字段，目前支持的 Loss 请参考 [Loss](../../../ppcls/loss)。
+最终的总 Loss 是所有 Loss 的加权和，其中 weight 定义了特定 Loss 在最终总 Loss 的权重。如果想替换其他 Loss，也可以在配置文件中更改 Loss 字段，目前支持的 Loss 请参考 [Loss](../../../ppcls/loss/__init__.py)。
-#### 2.2.2 特征模型恢复训练
+#### 2.2.2 特征提取模型恢复训练
-如果训练任务因为其他原因被终止，也可以加载断点权重文件，继续训练：
+如果训练任务因为其他原因被终止，且训练过程中有保存权重文件，可以加载断点权重文件，继续训练：
```shell
-# 单卡
-python3 tools/train.py \
- -c ./ppcls/configs/quick_start/MobileNetV1_retrieval.yaml \
- -o Global.checkpoints="./output/RecModel/epoch_5" \
- -o Global.device=gpu
-# 多卡
+# 单卡恢复训练
+python33.7 tools/train.py \
+-c ./ppcls/configs/quick_start/MobileNetV1_retrieval.yaml \
+-o Global.checkpoints="./output/RecModel/epoch_5" \
+-o Global.device=gpu
+
+# 多卡恢复训练
export CUDA_VISIBLE_DEVICES=0,1,2,3
-python3 -m paddle.distributed.launch tools/train.py \
- -c ./ppcls/configs/quick_start/MobileNetV1_retrieval.yaml \
- -o Global.checkpoints="./output/RecModel/epoch_5" \
- -o Global.device=gpu
+python3.7 -m paddle.distributed.launch tools/train.py \
+-c ./ppcls/configs/quick_start/MobileNetV1_retrieval.yaml \
+-o Global.checkpoints="./output/RecModel/epoch_5" \
+-o Global.device=gpu
```
其中配置文件不需要做任何修改，只需要在继续训练时设置 `Global.checkpoints` 参数即可，表示加载的断点权重文件路径，使用该参数会同时加载保存的断点权重和学习率、优化器等信息。
**注意**：
-* `-o Global.checkpoints` 参数无需包含断点权重文件的后缀名，上述训练命令会在训练过程中生成如下所示的断点权重文件，若想从断点 `5` 继续训练，则 `Global.checkpoints` 参数只需设置为 `"./output/RecModel/epoch_5"`，PaddleClas 会自动补充后缀名。
-
- ```shell
- output/
- └── RecModel
- ├── best_model.pdopt
- ├── best_model.pdparams
- ├── best_model.pdstates
- ├── epoch_1.pdopt
- ├── epoch_1.pdparams
- ├── epoch_1.pdstates
- .
- .
- .
- ```
+* `-o Global.checkpoints` 后的参数无需包含断点权重文件的后缀名，上述训练命令会在训练过程中生成如下所示的断点权重文件，若想从断点 `epoch_5` 继续训练，则 `Global.checkpoints` 参数只需设置为 `"./output/RecModel/epoch_5"`，PaddleClas 会自动补充后缀名。
+
+ `epoch_5.pdparams`所在目录如下所示：
+
+ ```log
+ output/
+ └── RecModel
+ ├── best_model.pdopt
+ ├── best_model.pdparams
+ ├── best_model.pdstates
+ ├── epoch_5.pdopt
+ ├── epoch_5.pdparams
+ ├── epoch_5.pdstates
+ .
+ .
+ .
+ ```
-#### 2.2.3 特征模型评估
+#### 2.2.3 特征提取模型评估
-可以通过以下命令进行模型评估。
+可以通过以下命令进行指定模型进行评估。
```bash
-# 单卡
-python3 tools/eval.py \
- -c ./ppcls/configs/quick_start/MobileNetV1_retrieval.yaml \
- -o Global.pretrained_model=./output/RecModel/best_model
-# 多卡
+# 单卡评估
+python3.7 tools/eval.py \
+-c ./ppcls/configs/quick_start/MobileNetV1_retrieval.yaml \
+-o Global.pretrained_model=./output/RecModel/best_model
+
+# 多卡评估
export CUDA_VISIBLE_DEVICES=0,1,2,3
-python3 -m paddle.distributed.launch tools/eval.py \
- -c ./ppcls/configs/quick_start/MobileNetV1_retrieval.yaml \
- -o Global.pretrained_model=./output/RecModel/best_model
+python3.7 -m paddle.distributed.launch tools/eval.py \
+-c ./ppcls/configs/quick_start/MobileNetV1_retrieval.yaml \
+-o Global.pretrained_model=./output/RecModel/best_model
```
-上述命令将使用 `./configs/quick_start/MobileNetV1_retrieval.yaml` 作为配置文件，对上述训练得到的模型 `./output/RecModel/best_model` 进行评估。你也可以通过更改配置文件中的参数来设置评估，也可以通过 `-o` 参数更新配置，如上所示。
+上述命令将使用 `./configs/quick_start/MobileNetV1_retrieval.yaml` 作为配置文件，对上述训练得到的模型 `./output/RecModel/best_model.pdparams` 进行评估。你也可以通过更改配置文件中的参数来设置评估，也可以通过 `-o` 参数更新配置，如上所示。
可配置的部分评估参数说明如下：
-* `Arch.name`：模型名称
* `Global.pretrained_model`：待评估的模型的预训练模型文件路径，不同于 `Global.Backbone.pretrained`，此处的预训练模型是整个模型的权重，而 `Global.Backbone.pretrained` 只是 Backbone 部分的权重。当需要做模型评估时，需要加载整个模型的权重。
-* `Metric.Eval`：待评估的指标，默认评估 recall@1、recall@5、mAP。当你不准备评测某一项指标时，可以将对应的试标从配置文件中删除；当你想增加某一项评测指标时，也可以参考 [Metric](../../../ppcls/metric/metrics.py) 部分在配置文件 `Metric.Eval` 中添加相关的指标。
+* `Metric.Eval`：待评估的指标，默认评估 `recall@1`、`recall@5`、`mAP`。当你不准备评测某一项指标时，可以将对应的试标从配置文件中删除；当你想增加某一项评测指标时，也可以参考 [Metric](../../../ppcls/metric/metrics.py) 部分在配置文件 `Metric.Eval` 中添加相关的指标。
**注意：**
-* 在加载待评估模型时，需要指定模型文件的路径，但无需包含文件后缀名，PaddleClas 会自动补齐 `.pdparams` 的后缀，如 [2.2.2 特征模型恢复训练](#2.2.2)。
+* 在加载待评估模型时，需要指定模型文件的路径，但无需包含文件后缀名，PaddleClas 会自动补齐 `.pdparams` 的后缀，如 [2.2.2 特征提取模型恢复训练](#2.2.2)。
-* Metric learning 任务一般不评测 TopkAcc。
+* Metric learning 任务一般不评测 `TopkAcc` 指标。
-### 2.3 特征模型导出 inference 模型
+### 2.3 特征提取模型导出 inference 模型
通过导出 inference 模型，PaddlePaddle 支持使用预测引擎进行预测推理。对训练好的模型进行转换：
```bash
-python3 tools/export_model.py \
- -c ./ppcls/configs/quick_start/MobileNetV1_retrieval.yaml \
- -o Global.pretrained_model=output/RecModel/best_model \
- -o Global.save_inference_dir=./inference
+python3.7 tools/export_model.py \
+-c ./ppcls/configs/quick_start/MobileNetV1_retrieval.yaml \
+-o Global.pretrained_model=output/RecModel/best_model \
+-o Global.save_inference_dir=./inference
```
-其中，`Global.pretrained_model` 用于指定模型文件路径，该路径仍无需包含模型文件后缀名（如[2.2.2 特征模型恢复训练](#2.2.2)）。当执行后，会在当前目录下生成 `./inference` 目录，目录下包含 `inference.pdiparams`、`inference.pdiparams.info`、`inference.pdmodel` 文件。`Global.save_inference_dir` 可以指定导出 inference 模型的路径。此处保存的 inference 模型在 embedding 特征层做了截断，即模型最终的输出为 n 维 embedding 特征。
+其中，`Global.pretrained_model` 用于指定模型文件路径，该路径仍无需包含模型文件后缀名（如[2.2.2 特征提取模型恢复训练](#2.2.2)）。当执行后，会在当前目录下生成 `./inference` 目录，目录下包含 `inference.pdiparams`、`inference.pdiparams.info`、`inference.pdmodel` 文件。`Global.save_inference_dir` 可以指定导出 inference 模型文件夹的路径。此处保存的 inference 模型在 embedding 特征层做了截断，即模型的推理输出为 n 维特征。
-上述命令将生成模型结构文件(`inference.pdmodel`)和模型权重文件(`inference.pdiparams`)，然后可以使用预测引擎进行推理。使用 inference 模型推理的流程可以参考[基于 Python 预测引擎预测推理](../inference_deployment/python_deploy.md)。
+有了上述命令将生成的模型结构文件(`inference.pdmodel`)和模型权重文件(`inference.pdiparams`)，接下来就可以使用预测引擎进行推理。使用 inference 模型推理的流程可以参考[基于 Python 预测引擎预测推理](../inference_deployment/python_deploy.md)。
-
+
## 3. 特征检索
PaddleClas 图像检索部分目前支持的环境如下：
-```shell
-└── CPU/单卡 GPU
- ├── Linux
- ├── MacOS
- └── Windows
-```
+| 操作系统 | 推理硬件 |
+| :------- | :------- |
+| Linux | CPU/GPU |
+| Windows | CPU/GPU |
+| MacOS | CPU/GPU |
-此部分使用了 [Faiss](https://github.com/facebookresearch/faiss) 作为检索库，其是一个高效的特征检索及聚类的库。此库中集成了多种相似度检索算法，以满足不同的检索场景。在 PaddleClas 中，支持三种检索算法：
+
+此部分使用了第三方开源库 [Faiss](https://github.com/facebookresearch/faiss) 作为检索工具，它是一个高效的特征检索与聚类的库，集成了多种相似度检索算法，以满足不同的检索场景。PaddleClas 目前支持三种检索算法：
- **HNSW32**: 一种图索引方法。检索精度较高，速度较快。但是特征库只支持添加图像功能，不支持删除图像特征功能。（默认方法）
- **IVF**：倒排索引检索方法。速度较快，但是精度略低。特征库支持增加、删除图像特功能。
@@ -296,22 +305,27 @@ PaddleClas 图像检索部分目前支持的环境如下：
具体安装方法如下：
-```python
-pip install faiss-cpu==1.7.1post2
+```shell
+python3.7 -m pip install faiss-cpu==1.7.1post2
```
-若使用时，不能正常引用，则 `uninstall` 之后，重新 `install`，尤其是 `windows` 下。
+若无法正常使用faiss，可以按以下命令先将其卸载，然后重新安装（Windows系统中该问题比较常见）。
+
+```shell
+python3.7 -m pip uninstall faiss-cpu
+python3.7 -m pip install faiss-cpu==1.7.1post2
+```
## 4. 基础知识
-图像检索指的是给定一个包含特定实例(例如特定目标、场景、物品等)的查询图像，图像检索旨在从数据库图像中找到包含相同实例的图像。不同于图像分类，图像检索解决的是一个开集问题，训练集中可能不包含被识别的图像的类别。图像检索的整体流程为：首先将图像中表示为一个合适的特征向量，其次，对这些图像的特征向量用欧式距离或余弦距离进行最近邻搜索以找到底库中相似的图像，最后，可以使用一些后处理技术对检索结果进行微调，确定被识别图像的类别等信息。所以，决定一个图像检索算法性能的关键在于图像对应的特征向量的好坏。
+图像检索指的是给定一个包含特定实例(例如特定目标、场景、物品等)的查询图像，图像检索旨在从数据库图像中找到包含相同实例的图像。不同于图像分类，图像检索解决的是一个开集问题，训练集中可能不包含被识别的图像的类别。图像检索的整体流程为：首先将图像中表示为一个合适的特征向量，其次对这些图像的特征向量用合适的距离度量函数进行最近邻搜索以找到数据库图像中相似的图像，最后，可能会使用一些后处理对检索结果进行进一步优化，得到待识别图像的类别、相似度等信息。所以，图像检索算法性能的关键在于图像提取的特征向量的表示能力强弱。
- 度量学习（Metric Learning）
-度量学习研究如何在一个特定的任务上学习一个距离函数，使得该距离函数能够帮助基于近邻的算法（kNN、k-means 等）取得较好的性能。深度度量学习(Deep Metric Learning)是度量学习的一种方法，它的目标是学习一个从原始特征到低维稠密的向量空间（嵌入空间，embedding space）的映射，使得同类对象在嵌入空间上使用常用的距离函数（欧氏距离、cosine 距离等）计算的距离比较近，而不同类的对象之间的距离则比较远。深度度量学习在计算机视觉领域取得了非常多的成功的应用，比如人脸识别、商品识别、图像检索、行人重识别等。更详细的介绍请参考[此文档](../algorithm_introduction/metric_learning.md)。
+ 度量学习研究如何在一个特定的任务上学习一个距离函数，使得该距离函数能够帮助基于近邻的算法（kNN、k-means 等）取得较好的性能。深度度量学习(Deep Metric Learning)是度量学习的一种方法，它的目标是学习一个从原始特征到低维稠密的向量空间（嵌入空间，embedding space）的映射，使得同类对象在嵌入空间上使用常用的距离函数（欧氏距离、cosine 距离等）计算的距离比较近，而不同类的对象之间的距离则比较远。深度度量学习在计算机视觉领域取得了非常多的成功的应用，比如人脸识别、商品识别、图像检索、行人重识别等。更详细的介绍请参考[此文档](../algorithm_introduction/metric_learning.md)。
@@ -319,19 +333,17 @@ pip install faiss-cpu==1.7.1post2
- 训练集合(train dataset)：用来训练模型，使模型能够学习该集合的图像特征。
- 底库数据集合(gallery dataset)：用来提供图像检索任务中的底库数据，该集合可与训练集或测试集相同，也可以不同，当与训练集相同时，测试集的类别体系应与训练集的类别体系相同。
- - 测试集合(query dataset)：用来测试模型的好坏，通常要对测试集的每一张测试图片进行特征提取，之后和底库数据的特征进行距离匹配，得到识别结果，后根据识别结果计算整个测试集的指标。
+ - 测试集合(query dataset)：用来测试模型的检索性能，通常要对测试集的每一张测试图片进行特征提取，之后和底库数据的特征进行距离匹配，得到检索结果，后根据检索结果计算模型在整个测试集上的性能指标。
- 图像检索评价指标
- 召回率(recall)：表示预测为正例且标签为正例的个数 / 标签为正例的个数
-
- - recall@1：检索的 top-1 中预测正例且标签为正例的个数 / 标签为正例的个数
- - recall@5：检索的 top-5 中所有预测正例且标签为正例的个数 / 标签为正例的个数
+ - `recall@k`：检索的 top-k 结果中预测为正例且标签为正例的个数 / 标签为正例的个数
- 平均检索精度(mAP)
- - AP: AP 指的是不同召回率上的正确率的平均值
- - mAP: 测试集中所有图片对应的 AP 的平均值
+ - `AP`: AP 指的是不同召回率上的正确率的平均值
+ - `mAP`: 测试集中所有图片对应的 AP 的平均值
diff --git a/docs/zh_CN/others/update_history.md b/docs/zh_CN/others/update_history.md
index 5ea649e52a9d53eb3aab5b8b9322d1a87920fefa..0e88a51362d7d04db960c966db72a5ec3a0ee787 100644
--- a/docs/zh_CN/others/update_history.md
+++ b/docs/zh_CN/others/update_history.md
@@ -1,5 +1,6 @@
# 更新日志
+- 2022.4.21 新增 CVPR2022 oral论文 [MixFormer](https://arxiv.org/pdf/2204.02557.pdf) 相关[代码](https://github.com/PaddlePaddle/PaddleClas/pull/1820/files)。
- 2021.11.1 发布[PP-ShiTu技术报告](https://arxiv.org/pdf/2111.00775.pdf)，新增饮料识别demo。
- 2021.10.23 发布轻量级图像识别系统PP-ShiTu，CPU上0.2s即可完成在10w+库的图像识别。[点击这里](../quick_start/quick_start_recognition.md)立即体验。
- 2021.09.17 发布PP-LCNet系列超轻量骨干网络模型, 在Intel CPU上，单张图像预测速度约5ms，ImageNet-1K数据集上Top1识别准确率达到80.82%，超越ResNet152的模型效果。PP-LCNet的介绍可以参考[论文](https://arxiv.org/pdf/2109.15099.pdf), 或者[PP-LCNet模型介绍](../models/PP-LCNet.md)，相关指标和预训练权重可以从 [这里](../algorithm_introduction/ImageNet_models.md)下载。
diff --git a/docs/zh_CN/quick_start/quick_start_recognition.md b/docs/zh_CN/quick_start/quick_start_recognition.md
index 38803ec9be510d3a4a96117fce3a1ccf537d3af9..550e455228f65b6886b13d8627413d4dd1387990 100644
--- a/docs/zh_CN/quick_start/quick_start_recognition.md
+++ b/docs/zh_CN/quick_start/quick_start_recognition.md
@@ -1,87 +1,131 @@
-# 图像识别快速开始
+## 图像识别快速体验
-本文档包含 3 个部分：环境配置、图像识别体验、未知类别的图像识别体验。
+本文档包含 2 个部分：PP-ShiTu android端 demo 快速体验与PP-ShiTu PC端 demo 快速体验。
如果图像类别已经存在于图像索引库中，那么可以直接参考[图像识别体验](#图像识别体验)章节，完成图像识别过程；如果希望识别未知类别的图像，即图像类别之前不存在于索引库中，那么可以参考[未知类别的图像识别体验](#未知类别的图像识别体验)章节，完成建立索引并识别的过程。
## 目录
-* [1. 环境配置](#环境配置)
-* [2. 图像识别体验](#图像识别体验)
- * [2.1 下载、解压 inference 模型与 demo 数据](#2.1)
- * [2.2 瓶装饮料识别与检索](#瓶装饮料识别与检索)
- * [2.2.1 识别单张图像](#识别单张图像)
- * [2.2.2 基于文件夹的批量识别](#基于文件夹的批量识别)
-* [3. 未知类别的图像识别体验](#未知类别的图像识别体验)
- * [3.1 准备新的数据与标签](#准备新的数据与标签)
- * [3.2 建立新的索引库](#建立新的索引库)
- * [3.3 基于新的索引库的图像识别](#基于新的索引库的图像识别)
-* [4. 服务端识别模型列表](#4)
+- [1. PP-ShiTu android demo 快速体验](#1-pp-shitu-android-demo-快速体验)
+ - [1.1 安装 PP-ShiTu android demo](#11-安装-pp-shitu-android-demo)
+ - [1.2 操作说明](#12-操作说明)
+- [2. PP-ShiTu PC端 demo 快速体验](#2-pp-shitu-pc端-demo-快速体验)
+ - [2.1 环境配置](#21-环境配置)
+ - [2.2 图像识别体验](#22-图像识别体验)
+ - [2.2.1 下载、解压 inference 模型与 demo 数据](#221-下载解压-inference-模型与-demo-数据)
+ - [2.2.2 瓶装饮料识别与检索](#222-瓶装饮料识别与检索)
+ - [2.2.2.1 识别单张图像](#2221-识别单张图像)
+ - [2.2.2.2 基于文件夹的批量识别](#2222-基于文件夹的批量识别)
+ - [2.3 未知类别的图像识别体验](#23-未知类别的图像识别体验)
+ - [2.3.1 准备新的数据与标签](#231-准备新的数据与标签)
+ - [2.3.2 建立新的索引库](#232-建立新的索引库)
+ - [2.3.3 基于新的索引库的图像识别](#233-基于新的索引库的图像识别)
+ - [2.4 服务端识别模型列表](#24-服务端识别模型列表)
+
+
+
+## 1. PP-ShiTu android demo 快速体验
+
+
+
+### 1.1 安装 PP-ShiTu android demo
+
+可以通过扫描二维码或者[点击链接](https://paddle-imagenet-models-name.bj.bcebos.com/demos/PP-ShiTu.apk)下载并安装APP
+
+或者“本地识别”按钮，即可拍摄一张图像或者选中一张图像，然后等待几秒钟，APP便会将图像中的主体框标注出来并且在图像下方给出预测的类别以及预测时间等信息。
+
+在选择好要检索的图片之后，首先会通过检测模型进行主体检测，得到图像中的物体的区域，然后将这块区域裁剪出来输入到识别模型中，得到对应的特征向量并在检索库中检索，返回并显示最终的检索结果。
+
+假设待检索的图像如下：
+
+
+
+得到的检索结果可视化如下：
+
+
+
+#### （2）向检索库中添加新的类别或物体
+点击上方的“拍照上传”按钮或者“本地上传”按钮，即可拍摄一张图像或从图库中选择一张图像，然后再输入这张图像的类别名字（比如`keyboard`），点击“确定”按钮，即可将图片对应的特征向量与标签加入检索库。
+
+在选择好要入库的图片之后，首先会通过检测模型进行主体检测，得到图像中的物体的区域，然后将这块区域裁剪出来输入到识别模型中，得到对应的特征向量，再与用户输入的图像标签一起加入到检索库中。
+
+**温馨提示：** 使用安卓demo管理类别主要用于功能体验，如果您有较为重要的数据要生成检索库，推荐使用[检索库管理工具](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.5/docs/zh_CN/inference_deployment/shitu_gallery_manager.md)
+
+#### (3) 保存检索库
+点击上方的“保存修改”按钮，即可将当前库以 `latest` 的库名保存下来。
+再次打开程序时，将会自动选择使用`latest`库。app仅存在一个自定义库，每次保存时会覆盖之前的库。
+
+#### (4) 检索库恢复出厂设置
+**警告：本操作无法撤销，初始化后自定义的标签和类别都会被删除，请谨慎操作**
+
+点击上方的“初始化 ”按钮，删除所有自定义的标签和类别，恢复出厂特征库。
+
+初始化库时会删掉`latest`库（如果存在），自动将检索库和标签库切换成 `original.index` 和 `original.txt`。不管是否有保存过，自定义的标签和类别都会被清空。
+
+#### (5) 查看当前检索库中的类别列表
+点击“类别查询”按钮
，即可在弹窗中查看。
+
+当检索标签库过多（如本demo自带的196类检索标签库）时，可在弹窗中滑动查看。
+
+
+## 2. PP-ShiTu PC端 demo 快速体验
-## 1. 环境配置
+### 2.1 环境配置
* 安装：请先参考文档 [环境准备](../installation/install_paddleclas.md) 配置 PaddleClas 运行环境。
* 进入 `deploy` 运行目录。本部分所有内容与命令均需要在 `deploy` 目录下运行，可以通过下面的命令进入 `deploy` 目录。
- ```
+ ```shell
cd deploy
```
-## 2. 图像识别体验
+### 2.2 图像识别体验
轻量级通用主体检测模型与轻量级通用识别模型和配置文件下载方式如下表所示。
-| 模型简介 | 推荐场景 | inference 模型 | 预测配置文件 |
-| ------------ | ------------- | -------- | ------- |
-| 轻量级通用主体检测模型 | 通用场景 |[tar 格式下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/picodet_PPLCNet_x2_5_mainbody_lite_v1.0_infer.tar) [zip 格式文件下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/picodet_PPLCNet_x2_5_mainbody_lite_v1.0_infer.zip) | - |
-| 轻量级通用识别模型 | 通用场景 | [tar 格式下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/general_PPLCNet_x2_5_lite_v1.0_infer.tar) [zip 格式文件下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/general_PPLCNet_x2_5_lite_v1.0_infer.zip) | [inference_general.yaml](../../../deploy/configs/inference_general.yaml) |
-| 轻量级通用识别二值模型 | 检索库很大， 存储受限场景 | [tar 格式下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/general_PPLCNet_x2_5_lite_binary_v1.0_infer.tar) [zip 格式文件下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/general_PPLCNet_x2_5_lite_binary_v1.0_infer.zip)| [inference_general_binary.yaml](../../../deploy/configs/inference_general_binary.yaml) |
+| 模型简介 | 推荐场景 | inference 模型 | 预测配置文件 |
+| ---------------------- | -------- | ----------- | ------------ |
+| 轻量级通用主体检测模型 | 通用场景 | [tar 格式下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/picodet_PPLCNet_x2_5_mainbody_lite_v1.0_infer.tar) \| [zip 格式下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/picodet_PPLCNet_x2_5_mainbody_lite_v1.0_infer.zip) | - |
+| 轻量级通用识别模型 | 通用场景 | [tar 格式下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/PP-ShiTuV2/general_PPLCNetV2_base_pretrained_v1.0_infer.tar) \| [zip 格式下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/PP-ShiTuV2/general_PPLCNetV2_base_pretrained_v1.0_infer.zip) | [inference_general.yaml](../../../deploy/configs/inference_general.yaml) |
-注意：由于部分解压缩软件在解压上述 `tar` 格式文件时存在问题，建议非命令行用户下载 `zip` 格式文件并解压。`tar` 格式文件建议使用命令 `tar xf xxx.tar` 解压。
+注意：由于部分解压缩软件在解压上述 `tar` 格式文件时存在问题，建议非命令行用户下载 `zip` 格式文件并解压。`tar` 格式文件建议使用命令 `tar -xf xxx.tar` 解压。
-本章节 demo 数据下载地址如下: [瓶装饮料数据下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/data/drink_dataset_v1.0.tar)。
+本章节 demo 数据下载地址如下: [drink_dataset_v2.0.tar(瓶装饮料数据)](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/data/drink_dataset_v2.0.tar)，
+下面以 **drink_dataset_v2.0.tar** 为例介绍PC端的 PP-ShiTu 快速体验流程。用户也可以自行下载并解压其它场景的数据进行体验：[22种场景数据下载](../introduction/ppshitu_application_scenarios.md#1-应用场景介绍)。
-如果希望体验服务端主体检测和各垂类方向的识别模型，可以参考[第4章](#4)。
+如果希望体验服务端主体检测和各垂类方向的识别模型，可以参考 [2.4 服务端识别模型列表](#24-服务端识别模型列表)
**注意**
-1. windows 环境下如果没有安装 wget, 可以按照下面的步骤安装 wget 与 tar 命令，也可以在下载模型时将链接复制到浏览器中下载，并解压放置在相应目录下； linux 或者 macOS 用户可以右键点击，然后复制下载链接，即可通过 `wget` 命令下载。
-2. 如果 macOS 环境下没有安装 `wget` 命令，可以运行下面的命令进行安装。
-
-```shell
-# 安装 homebrew
-ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)";
-# 安装 wget
-brew install wget
-```
-
-4. 如果希望在 windows 环境下安装 wget，可以参考：[链接](https://www.cnblogs.com/jeshy/p/10518062.html)；如果希望在 windows 环境中安装 tar 命令，可以参考：[链接](https://www.cnblogs.com/chooperman/p/14190107.html)。
+- windows 环境下如果没有安装 wget, 可以按照下面的步骤安装 wget 与 tar 命令，也可以在下载模型时将链接复制到浏览器中下载，并解压放置在相应目录下； linux 或者 macOS 用户可以右键点击，然后复制下载链接，即可通过 `wget` 命令下载。
+- 如果 macOS 环境下没有安装 `wget` 命令，可以运行下面的命令进行安装。
+ ```shell
+ # 安装 homebrew
+ ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)";
+ # 安装 wget
+ brew install wget
+ ```
+- 如果希望在 windows 环境下安装 wget，可以参考：[链接](https://www.cnblogs.com/jeshy/p/10518062.html)；如果希望在 windows 环境中安装 tar 命令，可以参考：[链接](https://www.cnblogs.com/chooperman/p/14190107.html)。
+
-* 可以按照下面的命令下载并解压数据与模型
-
-```shell
-mkdir models
-cd models
-# 下载识别 inference 模型并解压
-wget {模型下载链接地址} && tar -xf {压缩包的名称}
-cd ..
-
-# 下载 demo 数据并解压
-wget {数据下载链接地址} && tar -xf {压缩包的名称}
-```
-
-
-
-### 2.1 下载、解压 inference 模型与 demo 数据
+#### 2.2.1 下载、解压 inference 模型与 demo 数据
下载 demo 数据集以及轻量级主体检测、识别模型，命令如下。
@@ -91,30 +135,30 @@ cd models
# 下载通用检测 inference 模型并解压
wget https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/picodet_PPLCNet_x2_5_mainbody_lite_v1.0_infer.tar && tar -xf picodet_PPLCNet_x2_5_mainbody_lite_v1.0_infer.tar
# 下载识别 inference 模型并解压
-wget https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/general_PPLCNet_x2_5_lite_v1.0_infer.tar && tar -xf general_PPLCNet_x2_5_lite_v1.0_infer.tar
+wget https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/PP-ShiTuV2/general_PPLCNetV2_base_pretrained_v1.0_infer.tar && tar -xf general_PPLCNetV2_base_pretrained_v1.0_infer.tar
cd ../
# 下载 demo 数据并解压
-wget https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/data/drink_dataset_v1.0.tar && tar -xf drink_dataset_v1.0.tar
+wget https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/data/drink_dataset_v2.0.tar && tar -xf drink_dataset_v2.0.tar
```
-解压完毕后，`drink_dataset_v1.0/` 文件夹下应有如下文件结构：
+解压完毕后，`drink_dataset_v2.0/` 文件夹下应有如下文件结构：
-```
-├── drink_dataset_v1.0/
+```log
+├── drink_dataset_v2.0/
│ ├── gallery/
│ ├── index/
-│ ├── test_images/
+│ ├── index_all/
+│ └── test_images/
├── ...
```
其中 `gallery` 文件夹中存放的是用于构建索引库的原始图像，`index` 表示基于原始图像构建得到的索引库信息，`test_images` 文件夹中存放的是用于测试识别效果的图像列表。
-
`models` 文件夹下应有如下文件结构：
-```
-├── general_PPLCNet_x2_5_lite_v1.0_infer
+```log
+├── general_PPLCNetV2_base_pretrained_v1.0_infer
│ ├── inference.pdiparams
│ ├── inference.pdiparams.info
│ └── inference.pdmodel
@@ -129,183 +173,185 @@ wget https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/data/drink_da
如果使用服务端通用识别模型，Demo 数据需要重新提取特征、够建索引，方式如下：
```shell
-# 下面是使用下载的服务端商品识别模型进行索引库构建
-python3.7 python/build_gallery.py -c configs/inference_general.yaml -o Global.rec_inference_model_dir=./models/general_PPLCNet_x2_5_lite_v1.0_infer
+python3.7 python/build_gallery.py \
+-c configs/inference_general.yaml \
+-o Global.rec_inference_model_dir=./models/general_PPLCNetV2_base_pretrained_v1.0_infer
```
-### 2.2 瓶装饮料识别与检索
+#### 2.2.2 瓶装饮料识别与检索
以瓶装饮料识别 demo 为例，展示识别与检索过程（如果希望尝试其他方向的识别与检索效果，在下载解压好对应的 demo 数据与模型之后，替换对应的配置文件即可完成预测）。
注意，此部分使用了 `faiss` 作为检索库，安装方法如下：
```python
-pip install faiss-cpu==1.7.1post2
+python3.7 -m pip install faiss-cpu==1.7.1post2
```
若使用时，不能正常引用，则 `uninstall` 之后，重新 `install`，尤其是 windows 下。
-#### 2.2.1 识别单张图像
+##### 2.2.2.1 识别单张图像
+
+运行下面的命令，对图像 `./drink_dataset_v2.0/test_images/100.jpeg` 进行识别与检索
-运行下面的命令，对图像 `./drink_dataset_v1.0/test_images/nongfu_spring.jpeg` 进行识别与检索
+待检索图像如下所示
+
+
```shell
# 使用下面的命令使用 GPU 进行预测
python3.7 python/predict_system.py -c configs/inference_general.yaml
+
# 使用下面的命令使用 CPU 进行预测
python3.7 python/predict_system.py -c configs/inference_general.yaml -o Global.use_gpu=False
```
-待检索图像如下所示。
-
-
-
最终输出结果如下。
-```
-[{'bbox': [244, 49, 509, 964], 'rec_docs': '农夫山泉-饮用天然水', 'rec_scores': 0.7585664}]
+```log
+[{'bbox': [437, 71, 660, 728], 'rec_docs': '元气森林', 'rec_scores': 0.7740249}, {'bbox': [221, 72, 449, 701], 'rec_docs': '元气森林', 'rec_scores': 0.6950992}, {'bbox': [794, 104, 979, 652], 'rec_docs': '元气森林', 'rec_scores': 0.6305153}]
```
其中 `bbox` 表示检测出的主体所在位置，`rec_docs` 表示索引库中与检测框最为相似的类别，`rec_scores` 表示对应的置信度。
-检测的可视化结果也保存在 `output` 文件夹下，对于本张图像，识别结果可视化如下所示。
+检测的可视化结果默认保存在 `output` 文件夹下，对于本张图像，识别结果可视化如下所示。
-
+
-#### 2.2.2 基于文件夹的批量识别
+
+##### 2.2.2.2 基于文件夹的批量识别
如果希望预测文件夹内的图像，可以直接修改配置文件中的 `Global.infer_imgs` 字段，也可以通过下面的 `-o` 参数修改对应的配置。
```shell
# 使用下面的命令使用 GPU 进行预测，如果希望使用 CPU 预测，可以在命令后面添加 -o Global.use_gpu=False
-python3.7 python/predict_system.py -c configs/inference_general.yaml -o Global.infer_imgs="./drink_dataset_v1.0/test_images/"
+python3.7 python/predict_system.py -c configs/inference_general.yaml -o Global.infer_imgs="./drink_dataset_v2.0/test_images/"
```
终端中会输出该文件夹内所有图像的识别结果，如下所示。
-```
+```log
...
-[{'bbox': [345, 95, 524, 586], 'rec_docs': '红牛-强化型', 'rec_scores': 0.80164653}]
-Inference: 23.43583106994629 ms per batch image
-[{'bbox': [233, 0, 372, 436], 'rec_docs': '康师傅矿物质水', 'rec_scores': 0.72513914}]
-Inference: 117.95639991760254 ms per batch image
-[{'bbox': [138, 40, 573, 1198], 'rec_docs': '乐虎功能饮料', 'rec_scores': 0.7855944}]
-Inference: 22.172927856445312 ms per batch image
-[{'bbox': [328, 7, 467, 272], 'rec_docs': '脉动', 'rec_scores': 0.5829516}]
-Inference: 118.08514595031738 ms per batch image
-[{'bbox': [242, 82, 498, 726], 'rec_docs': '味全_每日C', 'rec_scores': 0.75581443}]
-Inference: 150.06470680236816 ms per batch image
-[{'bbox': [437, 71, 660, 728], 'rec_docs': '元气森林', 'rec_scores': 0.8478892}, {'bbox': [221, 72, 449, 701], 'rec_docs': '元气森林', 'rec_scores': 0.6790612}, {'bbox': [794, 104, 979, 652], 'rec_docs': '元气森林', 'rec_scores': 0.6292581}]
+[{'bbox': [0, 0, 600, 600], 'rec_docs': '红牛-强化型', 'rec_scores': 0.74081033}]
+Inference: 120.39852142333984 ms per batch image
+[{'bbox': [0, 0, 514, 436], 'rec_docs': '康师傅矿物质水', 'rec_scores': 0.6918598}]
+Inference: 32.045602798461914 ms per batch image
+[{'bbox': [138, 40, 573, 1198], 'rec_docs': '乐虎功能饮料', 'rec_scores': 0.68214047}]
+Inference: 113.41428756713867 ms per batch image
+[{'bbox': [328, 7, 467, 272], 'rec_docs': '脉动', 'rec_scores': 0.60406065}]
+Inference: 122.04337120056152 ms per batch image
+[{'bbox': [242, 82, 498, 726], 'rec_docs': '味全_每日C', 'rec_scores': 0.5428652}]
+Inference: 37.95266151428223 ms per batch image
+[{'bbox': [437, 71, 660, 728], 'rec_docs': '元气森林', 'rec_scores': 0.7740249}, {'bbox': [221, 72, 449, 701], 'rec_docs': '元气森林', 'rec_scores': 0.6950992}, {'bbox': [794, 104, 979, 652], 'rec_docs': '元气森林', 'rec_scores': 0.6305153}]
...
```
所有图像的识别结果可视化图像也保存在 `output` 文件夹内。
-
更多地，可以通过修改 `Global.rec_inference_model_dir` 字段来更改识别 inference 模型的路径，通过修改 `IndexProcess.index_dir` 字段来更改索引库索引的路径。
-## 3. 未知类别的图像识别体验
+### 2.3 未知类别的图像识别体验
-对图像 `./drink_dataset_v1.0/test_images/mosilian.jpeg` 进行识别，命令如下
-
-```shell
-# 使用下面的命令使用 GPU 进行预测，如果希望使用 CPU 预测，可以在命令后面添加 -o Global.use_gpu=False
-python3.7 python/predict_system.py -c configs/inference_general.yaml -o Global.infer_imgs="./drink_dataset_v1.0/test_images/mosilian.jpeg"
-```
+对图像 `./drink_dataset_v2.0/test_images/mosilian.jpeg` 进行识别
-待检索图像如下所示。
+待检索图像如下
+执行如下识别命令
+
+```shell
+# 使用下面的命令使用 GPU 进行预测，如果希望使用 CPU 预测，可以在命令后面添加 -o Global.use_gpu=False
+python3.7 python/predict_system.py -c configs/inference_general.yaml -o Global.infer_imgs="./drink_dataset_v2.0/test_images/mosilian.jpeg"
+```
-输出结果为空。
+可以发现输出结果为空
由于默认的索引库中不包含对应的索引信息，所以这里的识别结果有误，此时我们可以通过构建新的索引库的方式，完成未知类别的图像识别。
-当索引库中的图像无法覆盖我们实际识别的场景时，即在预测未知类别的图像时，我们需要将对应类别的相似图像添加到索引库中，从而完成对未知类别的图像识别，这一过程是不需要重新训练的。
+当索引库中的图像无法覆盖我们实际识别的场景时，即识别未知类别的图像前，我们需要将该未知类别的相似图像（至少一张）添加到索引库中，从而完成对未知类别的图像识别。这一过程不需要重新训练模型，以识别 `mosilian.jpeg` 为例，只需按以下步骤重新构建新的索引库即可。
-### 3.1 准备新的数据与标签
+#### 2.3.1 准备新的数据与标签
-首先需要将与待检索图像相似的图像列表拷贝到索引库原始图像的文件夹。这里 PaddleClas 已经将所有的图像数据都放在文件夹 `drink_dataset_v1.0/gallery/` 中。
-
-然后需要编辑记录了图像路径和标签信息的文本文件，这里 PaddleClas 将更正后的标签信息文件放在了 `drink_dataset_v1.0/gallery/drink_label_all.txt` 文件中。可以与默认的 `drink_dataset_v1.0/gallery/drink_label.txt` 标签文件进行对比，添加了光明和三元系列牛奶的索引图像。
+首先需要将与待检索图像相似的图像列表拷贝到索引库原始图像的文件夹中。这里 PaddleClas 已经将所有的图像数据都放在文件夹 `drink_dataset_v2.0/gallery/` 中。
+然后需要编辑记录了图像路径和标签信息的文本文件，这里 PaddleClas 将更新后的标签信息文件放在了 `drink_dataset_v2.0/gallery/drink_label_all.txt` 文件中。与原始的 `drink_dataset_v2.0/gallery/drink_label.txt` 标签文件进行对比，可以发现新增了光明和三元系列牛奶的索引图像。
每一行的文本中，第一个字段表示图像的相对路径，第二个字段表示图像对应的标签信息，中间用 `\t` 键分隔开（注意：有些编辑器会将 `tab` 自动转换为 `空格`，这种情况下会导致文件解析报错）。
-### 3.2 建立新的索引库
+#### 2.3.2 建立新的索引库
-使用下面的命令构建 `index` 索引，加速识别后的检索过程。
+使用下面的命令构建新的索引库 `index_all`。
```shell
-python3.7 python/build_gallery.py -c configs/inference_general.yaml -o IndexProcess.data_file="./drink_dataset_v1.0/gallery/drink_label_all.txt" -o IndexProcess.index_dir="./drink_dataset_v1.0/index_all"
+python3.7 python/build_gallery.py -c configs/inference_general.yaml -o IndexProcess.data_file="./drink_dataset_v2.0/gallery/drink_label_all.txt" -o IndexProcess.index_dir="./drink_dataset_v2.0/index_all"
```
-最终新的索引信息保存在文件夹 `./drink_dataset_v1.0/index_all` 中。具体 `yaml` 请参考[向量检索文档](../image_recognition_pipeline/vector_search.md)。
+最终构建完毕的新的索引库保存在文件夹 `./drink_dataset_v2.0/index_all` 下。具体 `yaml` 请参考[向量检索文档](../image_recognition_pipeline/vector_search.md)。
-### 3.3 基于新的索引库的图像识别
+#### 2.3.3 基于新的索引库的图像识别
-使用新的索引库，对上述图像进行识别，运行命令如下。
+使用新的索引库，重新对 `mosilian.jpeg` 图像进行识别，运行命令如下。
```shell
# 使用下面的命令使用 GPU 进行预测，如果希望使用 CPU 预测，可以在命令后面添加 -o Global.use_gpu=False
-python3.7 python/predict_system.py -c configs/inference_general.yaml -o Global.infer_imgs="././drink_dataset_v1.0/test_images/mosilian.jpeg" -o IndexProcess.index_dir="./drink_dataset_v1.0/index_all"
+python3.7 python/predict_system.py -c configs/inference_general.yaml -o Global.infer_imgs="./drink_dataset_v2.0/test_images/mosilian.jpeg" -o IndexProcess.index_dir="./drink_dataset_v2.0/index_all"
```
输出结果如下。
-```
-[{'bbox': [396, 553, 508, 621], 'rec_docs': '光明_莫斯利安', 'rec_scores': 0.5921005}]
+```log
+[{'bbox': [290, 297, 564, 919], 'rec_docs': '光明_莫斯利安', 'rec_scores': 0.59137374}]
```
-最终识别结果为`光明_莫斯利安`，识别正确，识别结果可视化如下所示。
+最终识别结果为 `光明_莫斯利安` ，识别正确，识别结果可视化如下所示。
-
-## 4. 服务端识别模型列表
+
+
+### 2.4 服务端识别模型列表
目前，我们更推荐您使用[轻量级通用主体检测模型与轻量级通用识别模型](#轻量级通用主体检测模型与轻量级通用识别模型)，以获得更好的测试结果。但是如果您希望体验服务端识别模型，服务器端通用主体检测模型与各方向识别模型、测试数据下载地址以及对应的配置文件地址如下。
-| 模型简介 | 推荐场景 | inference 模型 | 预测配置文件 |
-| ------------ | ------------- | -------- | ------- |
-| 通用主体检测模型 | 通用场景 |[模型下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/ppyolov2_r50vd_dcn_mainbody_v1.0_infer.tar) | - |
-| Logo 识别模型 | Logo 场景 | [模型下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/logo_rec_ResNet50_Logo3K_v1.0_infer.tar) | [inference_logo.yaml](../../../deploy/configs/inference_logo.yaml) |
-| 动漫人物识别模型 | 动漫人物场景 | [模型下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/cartoon_rec_ResNet50_iCartoon_v1.0_infer.tar) | [inference_cartoon.yaml](../../../deploy/configs/inference_cartoon.yaml) |
-| 车辆细分类模型 | 车辆场景 | [模型下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/vehicle_cls_ResNet50_CompCars_v1.0_infer.tar) | [inference_vehicle.yaml](../../../deploy/configs/inference_vehicle.yaml) |
-| 商品识别模型 | 商品场景 | [模型下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/product_ResNet50_vd_aliproduct_v1.0_infer.tar) | [inference_product.yaml](../../../deploy/configs/inference_product.yaml) |
-| 车辆 ReID 模型 | 车辆 ReID 场景 | [模型下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/vehicle_reid_ResNet50_VERIWild_v1.0_infer.tar) | [inference_vehicle.yaml](../../../deploy/configs/inference_vehicle.yaml) |
+| 模型简介 | 推荐场景 | inference 模型 | 预测配置文件 |
+| ---------------- | -------------- | ------------ | ----------- |
+| 通用主体检测模型 | 通用场景 | [模型下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/ppyolov2_r50vd_dcn_mainbody_v1.0_infer.tar) | - |
+| Logo 识别模型 | Logo 场景 | [模型下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/logo_rec_ResNet50_Logo3K_v1.0_infer.tar) | [inference_logo.yaml](../../../deploy/configs/inference_logo.yaml) |
+| 动漫人物识别模型 | 动漫人物场景 | [模型下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/cartoon_rec_ResNet50_iCartoon_v1.0_infer.tar) | [inference_cartoon.yaml](../../../deploy/configs/inference_cartoon.yaml) |
+| 车辆细分类模型 | 车辆场景 | [模型下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/vehicle_cls_ResNet50_CompCars_v1.0_infer.tar) | [inference_vehicle.yaml](../../../deploy/configs/inference_vehicle.yaml) |
+| 商品识别模型 | 商品场景 | [模型下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/product_ResNet50_vd_aliproduct_v1.0_infer.tar) | [inference_product.yaml](../../../deploy/configs/inference_product.yaml) |
+| 车辆 ReID 模型 | 车辆 ReID 场景 | [模型下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/vehicle_reid_ResNet50_VERIWild_v1.0_infer.tar) | [inference_vehicle.yaml](../../../deploy/configs/inference_vehicle.yaml) |
+可以按照如下命令下载上述模型到 `deploy/models` 文件夹中，以供识别任务使用
```shell
-cd PaddleClas/deploy/
+cd ./deploy
mkdir -p models
-```
-```shell
cd ./models
-# 下载通用主体检测模型并解压
+# 下载服务器端通用主体检测模型并解压
wget https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/ppyolov2_r50vd_dcn_mainbody_v1.0_infer.tar && tar -xf ppyolov2_r50vd_dcn_mainbody_v1.0_infer.tar
-# 下载识别模型并解压
+# 下载通用识别模型并解压
wget {识别模型下载链接地址} && tar -xf {压缩包的名称}
```
-使用如下命令下载各方向识别模型的测试数据：
+然后使用如下命令下载各个识别场景的测试数据：
```shell
# 回到 deploy 目录下
@@ -316,7 +362,7 @@ wget https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/data/recognit
解压完毕后，`recognition_demo_data_v1.1` 文件夹下应有如下文件结构：
-```
+```log
├── recognition_demo_data_v1.1
│ ├── gallery_cartoon
│ ├── gallery_logo
@@ -329,6 +375,6 @@ wget https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/data/recognit
├── ...
```
-按照上述步骤下载模型和测试数据后，您可以进行相关方向识别模型的测试。
+按照上述步骤下载模型和测试数据后，您可以重新建立索引库，并进行相关方向识别模型的测试。
* 更多关于主体检测的介绍可以参考：[主体检测教程文档](../image_recognition_pipeline/mainbody_detection.md)；关于特征提取的介绍可以参考：[特征提取教程文档](../image_recognition_pipeline/feature_extraction.md)；关于向量检索的介绍可以参考：[向量检索教程文档](../image_recognition_pipeline/vector_search.md)。
diff --git a/docs/zh_CN/samples/Fresh_Food_Recogniiton/README.md b/docs/zh_CN/samples/Fresh_Food_Recogniiton/README.md
new file mode 100644
index 0000000000000000000000000000000000000000..cca051cec89d9ef5a86a3edc2cb8745a5bc69a2e
--- /dev/null
+++ b/docs/zh_CN/samples/Fresh_Food_Recogniiton/README.md
@@ -0,0 +1,21 @@
+## 生鲜品自主结算
+
+在超市等无人零售场景中，目前主要是结算方式，主要有以下几种
+
+- 条形码方式
+- RFID等射频码
+- 称重方法
+
+但是以上几种方法存在如下缺点： 1）针对条形码方式，对于成品包装的商品，较为成熟，但是对与生鲜产品等商品，并不能满足需求。 2）RFID等方式，虽然对生鲜等产品能够支持，但是额外生成标签，增加成本 3）称重方法，对于相同重量的山商品，不能很好的区分，同时重量称等精密仪器在长时间的负重和使用过程中，精度会发生变化，需要工作人员定期调教，以满足精度需求。
+
+因此，如何选择一种既能大规模支持各种商品识别，又能方便管理，同时维护成本不高的识别系统，显得尤为重要。
+
+深圳市银歌云技术有限公司基于飞桨的图像识别开发套件PaddleClas，提供了一套基于计算机视觉的完整生鲜品自主结算方案，其通过结算平台的摄像头拍摄的图像，自动的识别称上的商品，整个流程在1秒内完成，无需售卖人员的操作及称重。整个流程，实现了精度高、速度快，无需人工干预的自动结算效果。减少人工成本的同时，大大提高了效率和用户体验。
+
+本案例使用了飞桨图像分类开发套件中的通用图像识别系统[PP-ShiTuV2](../../PPShiTu/PPShiTuV2_introduction.md)。
+
+
+
+
+**注**: AI Studio在线运行代码请参考[生鲜品自主结算](https://aistudio.baidu.com/aistudio/projectdetail/4486158)
+
diff --git a/docs/zh_CN/samples/Fresh_Food_Recogniiton/imgs/yingeo.png b/docs/zh_CN/samples/Fresh_Food_Recogniiton/imgs/yingeo.png
new file mode 100644
index 0000000000000000000000000000000000000000..e84b06a0591e3892d37a8e7068ab439d365246f0
Binary files /dev/null and b/docs/zh_CN/samples/Fresh_Food_Recogniiton/imgs/yingeo.png differ
diff --git a/paddleclas.py b/paddleclas.py
index b11b343d42ca210e0363c71b4522c294db08fa41..1463b80d28cdcf7b89ed72c3089aa7f5e084793a 100644
--- a/paddleclas.py
+++ b/paddleclas.py
@@ -32,6 +32,7 @@ from .ppcls.arch import backbone
from .ppcls.utils import logger
from .deploy.python.predict_cls import ClsPredictor
+from .deploy.python.predict_system import SystemPredictor
from .deploy.utils.get_image_list import get_image_list
from .deploy.utils import config
@@ -50,6 +51,11 @@ BASE_IMAGES_DIR = os.path.join(BASE_DIR, "images")
IMN_MODEL_BASE_DOWNLOAD_URL = "https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/{}_infer.tar"
IMN_MODEL_SERIES = {
"AlexNet": ["AlexNet"],
+ "CSWinTransformer": [
+ "CSWinTransformer_tiny_224", "CSWinTransformer_small_224",
+ "CSWinTransformer_base_224", "CSWinTransformer_base_384",
+ "CSWinTransformer_large_224", "CSWinTransformer_large_384"
+ ],
"DarkNet": ["DarkNet53"],
"DeiT": [
"DeiT_base_distilled_patch16_224", "DeiT_base_distilled_patch16_384",
@@ -81,6 +87,8 @@ IMN_MODEL_SERIES = {
"HRNet_W48_C_ssld"
],
"Inception": ["GoogLeNet", "InceptionV3", "InceptionV4"],
+ "LeViT":
+ ["LeViT_128S", "LeViT_128", "LeViT_192", "LeViT_256", "LeViT_384"],
"MixNet": ["MixNet_S", "MixNet_M", "MixNet_L"],
"MobileNetV1": [
"MobileNetV1_x0_25", "MobileNetV1_x0_5", "MobileNetV1_x0_75",
@@ -99,6 +107,7 @@ IMN_MODEL_SERIES = {
"MobileNetV3_large_x1_0", "MobileNetV3_large_x1_25",
"MobileNetV3_small_x1_0_ssld", "MobileNetV3_large_x1_0_ssld"
],
+ "MobileViT": ["MobileViT_XXS", "MobileViT_XS", "MobileViT_S"],
"PPHGNet": [
"PPHGNet_tiny",
"PPHGNet_small",
@@ -110,6 +119,10 @@ IMN_MODEL_SERIES = {
"PPLCNet_x1_0", "PPLCNet_x1_5", "PPLCNet_x2_0", "PPLCNet_x2_5"
],
"PPLCNetV2": ["PPLCNetV2_base"],
+ "PVTV2": [
+ "PVT_V2_B0", "PVT_V2_B1", "PVT_V2_B2", "PVT_V2_B2_Linear", "PVT_V2_B3",
+ "PVT_V2_B4", "PVT_V2_B5"
+ ],
"RedNet": ["RedNet26", "RedNet38", "RedNet50", "RedNet101", "RedNet152"],
"RegNet": ["RegNetX_4GF"],
"Res2Net": [
@@ -162,6 +175,7 @@ IMN_MODEL_SERIES = {
"pcpvt_small", "pcpvt_base", "pcpvt_large", "alt_gvt_small",
"alt_gvt_base", "alt_gvt_large"
],
+ "TNT": ["TNT_small"],
"VGG": ["VGG11", "VGG13", "VGG16", "VGG19"],
"VisionTransformer": [
"ViT_base_patch16_224", "ViT_base_patch16_384", "ViT_base_patch32_384",
@@ -178,7 +192,16 @@ PULC_MODEL_BASE_DOWNLOAD_URL = "https://paddleclas.bj.bcebos.com/models/PULC/inf
PULC_MODELS = [
"car_exists", "language_classification", "person_attribute",
"person_exists", "safety_helmet", "text_image_orientation",
- "textline_orientation", "traffic_sign", "vehicle_attribute"
+ "textline_orientation", "traffic_sign", "vehicle_attribute",
+ "table_attribute"
+]
+
+SHITU_MODEL_BASE_DOWNLOAD_URL = "https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/{}_infer.tar"
+SHITU_MODELS = [
+ # "picodet_PPLCNet_x2_5_mainbody_lite_v1.0", # ShiTuV1(V2)_mainbody_det
+ # "general_PPLCNet_x2_5_lite_v1.0" # ShiTuV1_general_rec
+ # "PP-ShiTuV2/general_PPLCNetV2_base_pretrained_v1.0", # ShiTuV2_general_rec TODO(hesensen): add lite model
+ "PP-ShiTuV2"
]
@@ -200,12 +223,24 @@ class InputModelError(Exception):
def init_config(model_type, model_name, inference_model_dir, **kwargs):
- cfg_path = f"deploy/configs/PULC/{model_name}/inference_{model_name}.yaml" if model_type == "pulc" else "deploy/configs/inference_cls.yaml"
+ if model_type == "pulc":
+ cfg_path = f"deploy/configs/PULC/{model_name}/inference_{model_name}.yaml"
+ elif model_type == "shitu":
+ cfg_path = "deploy/configs/inference_general.yaml"
+ else:
+ cfg_path = "deploy/configs/inference_cls.yaml"
+
__dir__ = os.path.dirname(__file__)
cfg_path = os.path.join(__dir__, cfg_path)
cfg = config.get_config(cfg_path, show=False)
-
- cfg.Global.inference_model_dir = inference_model_dir
+ if cfg.Global.get("inference_model_dir"):
+ cfg.Global.inference_model_dir = inference_model_dir
+ else:
+ cfg.Global.rec_inference_model_dir = os.path.join(
+ inference_model_dir,
+ "PP-ShiTuV2/general_PPLCNetV2_base_pretrained_v1.0")
+ cfg.Global.det_inference_model_dir = os.path.join(
+ inference_model_dir, "picodet_PPLCNet_x2_5_mainbody_lite_v1.0")
if "batch_size" in kwargs and kwargs["batch_size"]:
cfg.Global.batch_size = kwargs["batch_size"]
@@ -219,6 +254,10 @@ def init_config(model_type, model_name, inference_model_dir, **kwargs):
if "infer_imgs" in kwargs and kwargs["infer_imgs"]:
cfg.Global.infer_imgs = kwargs["infer_imgs"]
+ if "index_dir" in kwargs and kwargs["index_dir"]:
+ cfg.IndexProcess.index_dir = kwargs["index_dir"]
+ if "data_file" in kwargs and kwargs["data_file"]:
+ cfg.IndexProcess.data_file = kwargs["data_file"]
if "enable_mkldnn" in kwargs and kwargs["enable_mkldnn"]:
cfg.Global.enable_mkldnn = kwargs["enable_mkldnn"]
if "cpu_num_threads" in kwargs and kwargs["cpu_num_threads"]:
@@ -240,25 +279,45 @@ def init_config(model_type, model_name, inference_model_dir, **kwargs):
if "thresh" in kwargs and kwargs[
"thresh"] and "ThreshOutput" in cfg.PostProcess:
cfg.PostProcess.ThreshOutput.thresh = kwargs["thresh"]
- if "Topk" in cfg.PostProcess:
- if "topk" in kwargs and kwargs["topk"]:
- cfg.PostProcess.Topk.topk = kwargs["topk"]
- if "class_id_map_file" in kwargs and kwargs["class_id_map_file"]:
- cfg.PostProcess.Topk.class_id_map_file = kwargs[
- "class_id_map_file"]
- else:
- class_id_map_file_path = os.path.relpath(
- cfg.PostProcess.Topk.class_id_map_file, "../")
- cfg.PostProcess.Topk.class_id_map_file = os.path.join(
- __dir__, class_id_map_file_path)
- if "VehicleAttribute" in cfg.PostProcess:
- if "color_threshold" in kwargs and kwargs["color_threshold"]:
- cfg.PostProcess.VehicleAttribute.color_threshold = kwargs[
- "color_threshold"]
- if "type_threshold" in kwargs and kwargs["type_threshold"]:
- cfg.PostProcess.VehicleAttribute.type_threshold = kwargs[
- "type_threshold"]
+ if cfg.get("PostProcess"):
+ if "Topk" in cfg.PostProcess:
+ if "topk" in kwargs and kwargs["topk"]:
+ cfg.PostProcess.Topk.topk = kwargs["topk"]
+ if "class_id_map_file" in kwargs and kwargs["class_id_map_file"]:
+ cfg.PostProcess.Topk.class_id_map_file = kwargs[
+ "class_id_map_file"]
+ else:
+ class_id_map_file_path = os.path.relpath(
+ cfg.PostProcess.Topk.class_id_map_file, "../")
+ cfg.PostProcess.Topk.class_id_map_file = os.path.join(
+ __dir__, class_id_map_file_path)
+ if "VehicleAttribute" in cfg.PostProcess:
+ if "color_threshold" in kwargs and kwargs["color_threshold"]:
+ cfg.PostProcess.VehicleAttribute.color_threshold = kwargs[
+ "color_threshold"]
+ if "type_threshold" in kwargs and kwargs["type_threshold"]:
+ cfg.PostProcess.VehicleAttribute.type_threshold = kwargs[
+ "type_threshold"]
+ if "TableAttribute" in cfg.PostProcess:
+ if "source_threshold" in kwargs and kwargs["source_threshold"]:
+ cfg.PostProcess.VehicleAttribute.color_threshold = kwargs[
+ "source_threshold"]
+ if "number_threshold" in kwargs and kwargs["number_threshold"]:
+ cfg.PostProcess.VehicleAttribute.color_threshold = kwargs[
+ "number_threshold"]
+ if "color_threshold" in kwargs and kwargs["color_threshold"]:
+ cfg.PostProcess.VehicleAttribute.color_threshold = kwargs[
+ "color_threshold"]
+ if "clarity_threshold" in kwargs and kwargs["clarity_threshold"]:
+ cfg.PostProcess.VehicleAttribute.color_threshold = kwargs[
+ "clarity_threshold"]
+ if "obstruction_threshold" in kwargs and kwargs["obstruction_threshold"]:
+ cfg.PostProcess.VehicleAttribute.color_threshold = kwargs[
+ "obstruction_threshold"]
+ if "angle_threshold" in kwargs and kwargs["angle_threshold"]:
+ cfg.PostProcess.VehicleAttribute.color_threshold = kwargs[
+ "angle_threshold"]
if "save_dir" in kwargs and kwargs["save_dir"]:
cfg.PostProcess.SavePreLabel.save_dir = kwargs["save_dir"]
@@ -282,6 +341,13 @@ def args_cfg():
type=str,
help="The directory of model files. Valid when model_name not specifed."
)
+ parser.add_argument(
+ "--index_dir",
+ type=str,
+ required=False,
+ help="The index directory path.")
+ parser.add_argument(
+ "--data_file", type=str, required=False, help="The label file path.")
parser.add_argument("--use_gpu", type=str2bool, help="Whether use GPU.")
parser.add_argument(
"--gpu_mem",
@@ -334,6 +400,7 @@ def print_info():
"""
imn_table = PrettyTable(["IMN Model Series", "Model Name"])
pulc_table = PrettyTable(["PULC Models"])
+ shitu_table = PrettyTable(["PP-ShiTu Models"])
try:
sz = os.get_terminal_size()
total_width = sz.columns
@@ -352,11 +419,16 @@ def print_info():
textwrap.fill(
" ".join(PULC_MODELS), width=total_width).center(table_width - 4)
])
+ shitu_table.add_row([
+ textwrap.fill(
+ " ".join(SHITU_MODELS), width=total_width).center(table_width - 4)
+ ])
print("{}".format("-" * table_width))
print("Models supported by PaddleClas".center(table_width))
print(imn_table)
print(pulc_table)
+ print(shitu_table)
print("Powered by PaddlePaddle!".rjust(table_width))
print("{}".format("-" * table_width))
@@ -412,6 +484,10 @@ def check_model_file(model_type, model_name):
storage_directory = partial(os.path.join, BASE_INFERENCE_MODEL_DIR,
"PULC", model_name)
url = PULC_MODEL_BASE_DOWNLOAD_URL.format(model_name)
+ elif model_type == "shitu":
+ storage_directory = partial(os.path.join, BASE_INFERENCE_MODEL_DIR,
+ "PP-ShiTu", model_name)
+ url = SHITU_MODEL_BASE_DOWNLOAD_URL.format(model_name)
else:
storage_directory = partial(os.path.join, BASE_INFERENCE_MODEL_DIR,
"IMN", model_name)
@@ -472,8 +548,10 @@ class PaddleClas(object):
model_name, inference_model_dir)
self._config = init_config(self.model_type, model_name,
inference_model_dir, **kwargs)
-
- self.cls_predictor = ClsPredictor(self._config)
+ if self.model_type == "shitu":
+ self.predictor = SystemPredictor(self._config)
+ else:
+ self.predictor = ClsPredictor(self._config)
def get_config(self):
"""Get the config.
@@ -485,6 +563,7 @@ class PaddleClas(object):
"""
all_imn_model_names = get_imn_model_names()
all_pulc_model_names = PULC_MODELS
+ all_shitu_model_names = SHITU_MODELS
if model_name:
if model_name in all_imn_model_names:
@@ -493,6 +572,15 @@ class PaddleClas(object):
elif model_name in all_pulc_model_names:
inference_model_dir = check_model_file("pulc", model_name)
return "pulc", inference_model_dir
+ elif model_name in all_shitu_model_names:
+ inference_model_dir = check_model_file(
+ "shitu",
+ "PP-ShiTuV2/general_PPLCNetV2_base_pretrained_v1.0")
+ inference_model_dir = check_model_file(
+ "shitu", "picodet_PPLCNet_x2_5_mainbody_lite_v1.0")
+ inference_model_dir = os.path.abspath(
+ os.path.dirname(inference_model_dir))
+ return "shitu", inference_model_dir
else:
similar_imn_names = similar_model_names(model_name,
all_imn_model_names)
@@ -513,12 +601,13 @@ class PaddleClas(object):
raise InputModelError(err)
return "custom", inference_model_dir
else:
- err = f"Please specify the model name supported by PaddleClas or directory contained model files(inference.pdmodel, inference.pdiparams)."
+ err = "Please specify the model name supported by PaddleClas or directory contained model files(inference.pdmodel, inference.pdiparams)."
raise InputModelError(err)
return None
- def predict(self, input_data: Union[str, np.array],
- print_pred: bool=False) -> Generator[list, None, None]:
+ def predict_cls(self,
+ input_data: Union[str, np.array],
+ print_pred: bool=False) -> Generator[list, None, None]:
"""Predict input_data.
Args:
@@ -538,7 +627,7 @@ class PaddleClas(object):
"""
if isinstance(input_data, np.ndarray):
- yield self.cls_predictor.predict(input_data)
+ yield self.predictor.predict(input_data)
elif isinstance(input_data, str):
if input_data.startswith("http") or input_data.startswith("https"):
image_storage_dir = partial(os.path.join, BASE_IMAGES_DIR)
@@ -570,7 +659,7 @@ class PaddleClas(object):
cnt += 1
if cnt % batch_size == 0 or (idx_img + 1) == len(image_list):
- preds = self.cls_predictor.predict(img_list)
+ preds = self.predictor.predict(img_list)
if preds:
for idx_pred, pred in enumerate(preds):
@@ -587,6 +676,77 @@ class PaddleClas(object):
raise ImageTypeError(err)
return
+ def predict_shitu(self,
+ input_data: Union[str, np.array],
+ print_pred: bool=False) -> Generator[list, None, None]:
+ """Predict input_data.
+ Args:
+ input_data (Union[str, np.array]):
+ When the type is str, it is the path of image, or the directory containing images, or the URL of image from Internet.
+ When the type is np.array, it is the image data whose channel order is RGB.
+ print_pred (bool, optional): Whether print the prediction result. Defaults to False.
+
+ Raises:
+ ImageTypeError: Illegal input_data.
+
+ Yields:
+ Generator[list, None, None]:
+ The prediction result(s) of input_data by batch_size. For every one image,
+ prediction result(s) is zipped as a dict, that includs topk "class_ids", "scores" and "label_names".
+ The format of batch prediction result(s) is as follow: [{"class_ids": [...], "scores": [...], "label_names": [...]}, ...]
+ """
+ if isinstance(input_data, np.ndarray):
+ yield self.predictor.predict(input_data)
+ elif isinstance(input_data, str):
+ if input_data.startswith("http") or input_data.startswith("https"):
+ image_storage_dir = partial(os.path.join, BASE_IMAGES_DIR)
+ if not os.path.exists(image_storage_dir()):
+ os.makedirs(image_storage_dir())
+ image_save_path = image_storage_dir("tmp.jpg")
+ download_with_progressbar(input_data, image_save_path)
+ logger.info(
+ f"Image to be predicted from Internet: {input_data}, has been saved to: {image_save_path}"
+ )
+ input_data = image_save_path
+ image_list = get_image_list(input_data)
+
+ cnt = 0
+ for idx_img, img_path in enumerate(image_list):
+ img = cv2.imread(img_path)
+ if img is None:
+ logger.warning(
+ f"Image file failed to read and has been skipped. The path: {img_path}"
+ )
+ continue
+ img = img[:, :, ::-1]
+ cnt += 1
+
+ preds = self.predictor.predict(
+ img) # [dict1, dict2, ..., dictn]
+ if preds:
+ if print_pred:
+ logger.info(f"{preds}, filename: {img_path}")
+
+ yield preds
+ else:
+ err = "Please input legal image! The type of image supported by PaddleClas are: NumPy.ndarray and string of local path or Ineternet URL"
+ raise ImageTypeError(err)
+ return
+
+ def predict(self,
+ input_data: Union[str, np.array],
+ print_pred: bool=False,
+ predict_type="cls"):
+ if predict_type == "cls":
+ return self.predict_cls(input_data, print_pred)
+ elif predict_type == "shitu":
+ assert not isinstance(input_data, (
+ list, tuple
+ )), "PP-ShiTu predictor only support single image as input now."
+ return self.predict_shitu(input_data, print_pred)
+ else:
+ raise ModuleNotFoundError
+
# for CLI
def main():
@@ -595,7 +755,10 @@ def main():
print_info()
cfg = args_cfg()
clas_engine = PaddleClas(**cfg)
- res = clas_engine.predict(cfg["infer_imgs"], print_pred=True)
+ res = clas_engine.predict(
+ cfg["infer_imgs"],
+ print_pred=True,
+ predict_type="cls" if "PP-ShiTu" not in cfg["model_name"] else "shitu")
for _ in res:
pass
logger.info("Predict complete!")
diff --git a/ppcls/arch/backbone/__init__.py b/ppcls/arch/backbone/__init__.py
index 545725f71c23cfb0fa7198dd121fd1ff865fc760..49d47bb7e522ca782087807d31e736e087189441 100644
--- a/ppcls/arch/backbone/__init__.py
+++ b/ppcls/arch/backbone/__init__.py
@@ -69,10 +69,12 @@ from .model_zoo.repvgg import RepVGG_A0, RepVGG_A1, RepVGG_A2, RepVGG_B0, RepVGG
from .model_zoo.van import VAN_tiny
from .model_zoo.peleenet import PeleeNet
from .model_zoo.convnext import ConvNeXt_tiny
+from .model_zoo.cae import cae_base_patch16_224, cae_large_patch16_224
from .variant_models.resnet_variant import ResNet50_last_stage_stride1
from .variant_models.vgg_variant import VGG19Sigmoid
from .variant_models.pp_lcnet_variant import PPLCNet_x2_5_Tanh
+from .variant_models.pp_lcnetv2_variant import PPLCNetV2_base_ShiTu
from .model_zoo.adaface_ir_net import AdaFace_IR_18, AdaFace_IR_34, AdaFace_IR_50, AdaFace_IR_101, AdaFace_IR_152, AdaFace_IR_SE_50, AdaFace_IR_SE_101, AdaFace_IR_SE_152, AdaFace_IR_SE_200
diff --git a/ppcls/arch/backbone/base/theseus_layer.py b/ppcls/arch/backbone/base/theseus_layer.py
index a533cdc77bd1a124030fde66184144bdd3025f37..616e728943d81c0e2b0a404906ad691a176f9c51 100644
--- a/ppcls/arch/backbone/base/theseus_layer.py
+++ b/ppcls/arch/backbone/base/theseus_layer.py
@@ -103,7 +103,7 @@ class TheseusLayer(nn.Layer):
return new_layer
net = paddleclas.MobileNetV1()
- res = net.replace_sub(layer_name_pattern=["blocks[11].depthwise_conv.conv", "blocks[12].depthwise_conv.conv"], handle_func=rep_func)
+ res = net.upgrade_sublayer(layer_name_pattern=["blocks[11].depthwise_conv.conv", "blocks[12].depthwise_conv.conv"], handle_func=rep_func)
print(res)
# {'blocks[11].depthwise_conv.conv': the corresponding new_layer, 'blocks[12].depthwise_conv.conv': the corresponding new_layer}
"""
@@ -117,18 +117,26 @@ class TheseusLayer(nn.Layer):
layer_list = parse_pattern_str(pattern=pattern, parent_layer=self)
if not layer_list:
continue
+
sub_layer_parent = layer_list[-2]["layer"] if len(
layer_list) > 1 else self
-
sub_layer = layer_list[-1]["layer"]
sub_layer_name = layer_list[-1]["name"]
- sub_layer_index = layer_list[-1]["index"]
+ sub_layer_index_list = layer_list[-1]["index_list"]
new_sub_layer = handle_func(sub_layer, pattern)
- if sub_layer_index:
- getattr(sub_layer_parent,
- sub_layer_name)[sub_layer_index] = new_sub_layer
+ if sub_layer_index_list:
+ if len(sub_layer_index_list) > 1:
+ sub_layer_parent = getattr(
+ sub_layer_parent,
+ sub_layer_name)[sub_layer_index_list[0]]
+ for sub_layer_index in sub_layer_index_list[1:-1]:
+ sub_layer_parent = sub_layer_parent[sub_layer_index]
+ sub_layer_parent[sub_layer_index_list[-1]] = new_sub_layer
+ else:
+ getattr(sub_layer_parent, sub_layer_name)[
+ sub_layer_index_list[0]] = new_sub_layer
else:
setattr(sub_layer_parent, sub_layer_name, new_sub_layer)
@@ -151,8 +159,8 @@ class TheseusLayer(nn.Layer):
parent_layer = self
for layer_dict in layer_list:
- name, index = layer_dict["name"], layer_dict["index"]
- if not set_identity(parent_layer, name, index):
+ name, index_list = layer_dict["name"], layer_dict["index_list"]
+ if not set_identity(parent_layer, name, index_list):
msg = f"Failed to set the layers that after stop_layer_name('{stop_layer_name}') to IdentityLayer. The error layer's name is '{name}'."
logger.warning(msg)
return False
@@ -208,13 +216,13 @@ def save_sub_res_hook(layer, input, output):
def set_identity(parent_layer: nn.Layer,
layer_name: str,
- layer_index: str=None) -> bool:
- """set the layer specified by layer_name and layer_index to Indentity.
+ layer_index_list: str=None) -> bool:
+ """set the layer specified by layer_name and layer_index_list to Indentity.
Args:
- parent_layer (nn.Layer): The parent layer of target layer specified by layer_name and layer_index.
+ parent_layer (nn.Layer): The parent layer of target layer specified by layer_name and layer_index_list.
layer_name (str): The name of target layer to be set to Indentity.
- layer_index (str, optional): The index of target layer to be set to Indentity in parent_layer. Defaults to None.
+ layer_index_list (str, optional): The index of target layer to be set to Indentity in parent_layer. Defaults to None.
Returns:
bool: True if successfully, False otherwise.
@@ -228,16 +236,19 @@ def set_identity(parent_layer: nn.Layer,
if sub_layer_name == layer_name:
stop_after = True
- if layer_index and stop_after:
- stop_after = False
- for sub_layer_index in parent_layer._sub_layers[
- layer_name]._sub_layers:
- if stop_after:
- parent_layer._sub_layers[layer_name][
- sub_layer_index] = Identity()
- continue
- if layer_index == sub_layer_index:
- stop_after = True
+ if layer_index_list and stop_after:
+ layer_container = parent_layer._sub_layers[layer_name]
+ for num, layer_index in enumerate(layer_index_list):
+ stop_after = False
+ for i in range(num):
+ layer_container = layer_container[layer_index_list[i]]
+ for sub_layer_index in layer_container._sub_layers:
+ if stop_after:
+ parent_layer._sub_layers[layer_name][
+ sub_layer_index] = Identity()
+ continue
+ if layer_index == sub_layer_index:
+ stop_after = True
return stop_after
@@ -269,10 +280,12 @@ def parse_pattern_str(pattern: str, parent_layer: nn.Layer) -> Union[
while len(pattern_list) > 0:
if '[' in pattern_list[0]:
target_layer_name = pattern_list[0].split('[')[0]
- target_layer_index = pattern_list[0].split('[')[1].split(']')[0]
+ target_layer_index_list = list(
+ index.split(']')[0]
+ for index in pattern_list[0].split('[')[1:])
else:
target_layer_name = pattern_list[0]
- target_layer_index = None
+ target_layer_index_list = None
target_layer = getattr(parent_layer, target_layer_name, None)
@@ -281,21 +294,22 @@ def parse_pattern_str(pattern: str, parent_layer: nn.Layer) -> Union[
logger.warning(msg)
return None
- if target_layer_index and target_layer:
- if int(target_layer_index) < 0 or int(target_layer_index) >= len(
- target_layer):
- msg = f"Not found layer by index('{target_layer_index}') specifed in pattern('{pattern}'). The index should < {len(target_layer)} and > 0."
- logger.warning(msg)
- return None
-
- target_layer = target_layer[target_layer_index]
+ if target_layer_index_list:
+ for target_layer_index in target_layer_index_list:
+ if int(target_layer_index) < 0 or int(
+ target_layer_index) >= len(target_layer):
+ msg = f"Not found layer by index('{target_layer_index}') specifed in pattern('{pattern}'). The index should < {len(target_layer)} and > 0."
+ logger.warning(msg)
+ return None
+ target_layer = target_layer[target_layer_index]
layer_list.append({
"layer": target_layer,
"name": target_layer_name,
- "index": target_layer_index
+ "index_list": target_layer_index_list
})
pattern_list = pattern_list[1:]
parent_layer = target_layer
+
return layer_list
diff --git a/ppcls/arch/backbone/legendary_models/pp_lcnet_v2.py b/ppcls/arch/backbone/legendary_models/pp_lcnet_v2.py
index 40264092a47deb1e11ed11d2edbda7135f0b5a75..ea24489c16c9d1281b3555546c5786a2168a8a38 100644
--- a/ppcls/arch/backbone/legendary_models/pp_lcnet_v2.py
+++ b/ppcls/arch/backbone/legendary_models/pp_lcnet_v2.py
@@ -126,6 +126,8 @@ class RepDepthwiseSeparable(TheseusLayer):
use_se=False,
use_shortcut=False):
super().__init__()
+ self.in_channels = in_channels
+ self.out_channels = out_channels
self.is_repped = False
self.dw_size = dw_size
@@ -306,8 +308,8 @@ class PPLCNetV2(TheseusLayer):
self.dropout = Dropout(p=dropout_prob, mode="downscale_in_infer")
self.flatten = nn.Flatten(start_axis=1, stop_axis=-1)
- in_features = self.class_expand if self.use_last_conv else NET_CONFIG[
- "stage4"][0] * 2 * scale
+ in_features = self.class_expand if self.use_last_conv else make_divisible(
+ NET_CONFIG["stage4"][0] * 2 * scale)
self.fc = Linear(in_features, class_num)
def forward(self, x):
diff --git a/ppcls/arch/backbone/model_zoo/cae.py b/ppcls/arch/backbone/model_zoo/cae.py
new file mode 100644
index 0000000000000000000000000000000000000000..b6262e9f6418aca93e4c486daf1ae64c36537748
--- /dev/null
+++ b/ppcls/arch/backbone/model_zoo/cae.py
@@ -0,0 +1,860 @@
+# copyright (c) 2021 PaddlePaddle Authors. All Rights Reserve.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# Code was heavily based on https://github.com/PaddlePaddle/VIMER/blob/main/CAE/models/modeling_finetune.py
+# reference: https://arxiv.org/abs/2202.03026
+
+import collections
+from itertools import repeat
+import math
+import numpy as np
+from functools import partial
+
+import paddle
+import paddle.nn as nn
+import paddle.nn.functional as F
+
+from ....utils.download import get_weights_path_from_url
+
+MODEL_URLS = {
+ "cae_base_patch16_224":
+ "https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/cae_base_patch16_224_pretrained.pdparams",
+ "cae_large_patch16_224":
+ "https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/cae_large_patch16_224_pretrained.pdparams"
+}
+
+__all__ = list(MODEL_URLS.keys())
+
+
+def _ntuple(n):
+ def parse(x):
+ if isinstance(x, collections.abc.Iterable):
+ return x
+ return tuple(repeat(x, n))
+
+ return parse
+
+
+def trunc_normal_(tensor, mean=0., std=1.):
+ nn.initializer.TruncatedNormal(mean=mean, std=std)(tensor)
+
+
+def drop_path(x, drop_prob: float=0., training: bool=False):
+ """Drop paths (Stochastic Depth) per sample (when applied in main path of residual blocks).
+
+ This is the same as the DropConnect impl I created for EfficientNet, etc networks, however,
+ the original name is misleading as 'Drop Connect' is a different form of dropout in a separate paper...
+ See discussion: https://github.com/tensorflow/tpu/issues/494#issuecomment-532968956 ... I've opted for
+ changing the layer and argument names to 'drop path' rather than mix DropConnect as a layer name and use
+ 'survival rate' as the argument.
+
+ """
+ if drop_prob == 0. or not training:
+ return x
+ keep_prob = 1 - drop_prob
+ shape = (x.shape[0], ) + (1, ) * (
+ x.ndim - 1) # work with diff dim tensors, not just 2D ConvNets
+ random_tensor = keep_prob + paddle.rand(shape, dtype=x.dtype)
+ random_tensor.floor_() # binarize
+ output = x / keep_prob * random_tensor
+ return output
+
+
+class DropPath(nn.Layer):
+ """Drop paths (Stochastic Depth) per sample (when applied in main path of residual blocks).
+ """
+
+ def __init__(self, drop_prob=None):
+ super(DropPath, self).__init__()
+ self.drop_prob = drop_prob
+
+ def forward(self, x):
+ return drop_path(x, self.drop_prob, self.training)
+
+ def extra_repr(self) -> str:
+ return 'p={}'.format(self.drop_prob)
+
+
+class Mlp(nn.Layer):
+ def __init__(self,
+ in_features,
+ hidden_features=None,
+ out_features=None,
+ act_layer=nn.GELU,
+ drop=0.):
+ super().__init__()
+ out_features = out_features or in_features
+ hidden_features = hidden_features or in_features
+ self.fc1 = nn.Linear(in_features, hidden_features, bias_attr=True)
+ self.act = act_layer()
+ self.fc2 = nn.Linear(hidden_features, out_features, bias_attr=True)
+ self.drop = nn.Dropout(drop)
+
+ def forward(self, x):
+ x = self.fc1(x)
+ x = self.act(x)
+ # x = self.drop(x)
+ # commit this for the orignal BERT implement
+ x = self.fc2(x)
+ x = self.drop(x)
+ return x
+
+
+class Attention(nn.Layer):
+ def __init__(self,
+ dim,
+ num_heads=8,
+ qkv_bias=False,
+ qk_scale=None,
+ attn_drop=0.,
+ proj_drop=0.,
+ window_size=None,
+ attn_head_dim=None):
+ super().__init__()
+
+ self.num_heads = num_heads
+ head_dim = dim // num_heads
+ if attn_head_dim is not None:
+ head_dim = attn_head_dim
+ all_head_dim = head_dim * self.num_heads
+ self.scale = qk_scale or head_dim**-0.5
+
+ self.zeros_ = nn.initializer.Constant(value=0.)
+
+ self.qkv = nn.Linear(dim, all_head_dim * 3, bias_attr=False)
+ if qkv_bias:
+ self.q_bias = self.create_parameter(
+ [all_head_dim], default_initializer=self.zeros_)
+ self.v_bias = self.create_parameter(
+ [all_head_dim], default_initializer=self.zeros_)
+ else:
+ self.q_bias = None
+ self.v_bias = None
+
+ if window_size:
+ self.window_size = window_size
+ self.num_relative_distance = (2 * window_size[0] - 1) * (
+ 2 * window_size[1] - 1) + 3
+ self.relative_position_bias_table = self.create_parameter(
+ [self.num_relative_distance, num_heads],
+ default_initializer=self.zeros_) # 2*Wh-1 * 2*Ww-1, nH
+ # cls to token & token 2 cls & cls to cls
+
+ # get pair-wise relative position index for each token inside the window
+ coords_h = paddle.arange(window_size[0])
+ coords_w = paddle.arange(window_size[1])
+ coords = paddle.stack(paddle.meshgrid(
+ [coords_h, coords_w])) # 2, Wh, Ww
+ coords_flatten = paddle.flatten(coords, 1) # 2, Wh*Ww
+ relative_coords = coords_flatten[:, :,
+ None] - coords_flatten[:,
+ None, :] # 2, Wh*Ww, Wh*Ww
+ relative_coords = relative_coords.transpose(
+ [1, 2, 0]) # Wh*Ww, Wh*Ww, 2
+ relative_coords[:, :, 0] += window_size[
+ 0] - 1 # shift to start from 0
+ relative_coords[:, :, 1] += window_size[1] - 1
+ relative_coords[:, :, 0] *= 2 * window_size[1] - 1
+ relative_position_index = \
+ paddle.zeros((window_size[0] * window_size[1] + 1, ) * 2, dtype=relative_coords.dtype)
+ relative_position_index[1:, 1:] = relative_coords.sum(
+ -1) # Wh*Ww, Wh*Ww
+ relative_position_index[0, 0:] = self.num_relative_distance - 3
+ relative_position_index[0:, 0] = self.num_relative_distance - 2
+ relative_position_index[0, 0] = self.num_relative_distance - 1
+
+ self.register_buffer("relative_position_index",
+ relative_position_index)
+ else:
+ self.window_size = None
+ self.relative_position_bias_table = None
+ self.relative_position_index = None
+
+ self.attn_drop = nn.Dropout(attn_drop)
+ self.proj = nn.Linear(all_head_dim, dim, bias_attr=True)
+ self.proj_drop = nn.Dropout(proj_drop)
+
+ def forward(self, x, rel_pos_bias=None):
+ B, N, C = x.shape
+ qkv_bias = None
+ if self.q_bias is not None:
+ k_bias = paddle.zeros_like(self.v_bias)
+ k_bias.stop_gradient = True
+ qkv_bias = paddle.concat((self.q_bias, k_bias, self.v_bias))
+ # qkv = self.qkv(x).reshape([B, N, 3, self.num_heads, C // self.num_heads]).transpose([2, 0, 3, 1, 4])
+ qkv = F.linear(x=x, weight=self.qkv.weight, bias=qkv_bias)
+ qkv = qkv.reshape([B, N, 3, self.num_heads, -1]).transpose(
+ [2, 0, 3, 1, 4])
+ q, k, v = qkv[0], qkv[1], qkv[
+ 2] # make torchscript happy (cannot use tensor as tuple)
+
+ q = q * self.scale
+ attn = (q @k.transpose([0, 1, 3, 2]))
+
+ if self.relative_position_bias_table is not None:
+ relative_position_bias = \
+ self.relative_position_bias_table[self.relative_position_index.reshape([-1])].reshape([
+ self.window_size[0] * self.window_size[1] + 1,
+ self.window_size[0] * self.window_size[1] + 1, -1]) # Wh*Ww,Wh*Ww,nH
+ relative_position_bias = relative_position_bias.transpose(
+ [2, 0, 1]) # nH, Wh*Ww, Wh*Ww
+ attn = attn + relative_position_bias.unsqueeze(0)
+
+ if rel_pos_bias is not None:
+ attn = attn + rel_pos_bias
+
+ attn = F.softmax(attn, axis=-1)
+ attn = self.attn_drop(attn)
+
+ x = (attn @v).transpose([0, 2, 1, 3]).reshape([B, N, -1])
+ x = self.proj(x)
+ x = self.proj_drop(x)
+ return x
+
+
+class Block(nn.Layer):
+ def __init__(self,
+ dim,
+ num_heads,
+ mlp_ratio=4.,
+ qkv_bias=False,
+ qk_scale=None,
+ drop=0.,
+ attn_drop=0.,
+ drop_path=0.,
+ init_values=None,
+ act_layer=nn.GELU,
+ norm_layer=nn.LayerNorm,
+ window_size=None,
+ attn_head_dim=None):
+ super().__init__()
+ self.norm1 = norm_layer(dim)
+ self.attn = Attention(
+ dim,
+ num_heads=num_heads,
+ qkv_bias=qkv_bias,
+ qk_scale=qk_scale,
+ attn_drop=attn_drop,
+ proj_drop=drop,
+ window_size=window_size,
+ attn_head_dim=attn_head_dim)
+ # NOTE: drop path for stochastic depth, we shall see if this is better than dropout here
+ self.drop_path = DropPath(
+ drop_path) if drop_path > 0. else nn.Identity()
+ self.norm2 = norm_layer(dim)
+ mlp_hidden_dim = int(dim * mlp_ratio)
+ self.mlp = Mlp(in_features=dim,
+ hidden_features=mlp_hidden_dim,
+ act_layer=act_layer,
+ drop=drop)
+
+ if init_values > 0:
+ self.gamma_1 = self.create_parameter(
+ [dim],
+ default_initializer=nn.initializer.Constant(value=init_values))
+ self.gamma_2 = self.create_parameter(
+ [dim],
+ default_initializer=nn.initializer.Constant(value=init_values))
+ else:
+ self.gamma_1, self.gamma_2 = None, None
+
+ def forward(self, x, rel_pos_bias=None):
+ if self.gamma_1 is None:
+ x = x + self.drop_path(
+ self.attn(
+ self.norm1(x), rel_pos_bias=rel_pos_bias))
+ x = x + self.drop_path(self.mlp(self.norm2(x)))
+ else:
+ x = x + self.drop_path(self.gamma_1 * self.attn(
+ self.norm1(x), rel_pos_bias=rel_pos_bias))
+ x = x + self.drop_path(self.gamma_2 * self.mlp(self.norm2(x)))
+ return x
+
+
+class PatchEmbed(nn.Layer):
+ """ Image to Patch Embedding
+ """
+
+ def __init__(self, img_size=224, patch_size=16, in_chans=3, embed_dim=768):
+ super().__init__()
+ to_2tuple = _ntuple(2)
+ img_size = to_2tuple(img_size)
+ patch_size = to_2tuple(patch_size)
+ num_patches = (img_size[1] // patch_size[1]) * (img_size[0] //
+ patch_size[0])
+ self.patch_shape = (img_size[0] // patch_size[0],
+ img_size[1] // patch_size[1])
+ self.img_size = img_size
+ self.patch_size = patch_size
+ self.num_patches = num_patches
+ self.in_chans = in_chans
+ self.out_chans = embed_dim
+ self.proj = nn.Conv2D(
+ in_chans,
+ embed_dim,
+ kernel_size=patch_size,
+ stride=patch_size,
+ bias_attr=True)
+
+ def forward(self, x, **kwargs):
+ B, C, H, W = x.shape
+ # FIXME look at relaxing size constraints
+ assert H == self.img_size[0] and W == self.img_size[1], \
+ f"Input image size ({H}*{W}) doesn't match model ({self.img_size[0]}*{self.img_size[1]})."
+ x = self.proj(x).flatten(2).transpose([0, 2, 1])
+ return x
+
+ def _init_weights(self):
+ fan_out = self.out_chans
+ fan_in = self.patch_size[0] * self.patch_size[1] * self.in_chans
+ weight_attr = paddle.ParamAttr(
+ initializer=nn.initializer.XavierUniform(fan_in, fan_out)) # MAE
+ bias_attr = paddle.ParamAttr(initializer=nn.initializer.Constant(0.0))
+ return weight_attr, bias_attr
+
+
+class RelativePositionBias(nn.Layer):
+ def __init__(self, window_size, num_heads):
+ super().__init__()
+ self.window_size = window_size
+ self.num_relative_distance = (2 * window_size[0] - 1) * (
+ 2 * window_size[1] - 1) + 3
+ self.zeros_ = nn.initializer.Constant(value=0.)
+ self.relative_position_bias_table = self.create_parameter(
+ [self.num_relative_distance, num_heads],
+ default_initializer=self.zeros_) # 2*Wh-1 * 2*Ww-1, nH
+ # cls to token & token 2 cls & cls to cls
+
+ # get pair-wise relative position index for each token inside the window
+ coords_h = paddle.arange(window_size[0])
+ coords_w = paddle.arange(window_size[1])
+ coords = paddle.stack(paddle.meshgrid(
+ [coords_h, coords_w])) # 2, Wh, Ww
+ coords_flatten = paddle.flatten(coords, 1) # 2, Wh*Ww
+ relative_coords = coords_flatten[:, :,
+ None] - coords_flatten[:,
+ None, :] # 2, Wh*Ww, Wh*Ww
+ relative_coords = relative_coords.transpose(
+ [1, 2, 0]) # Wh*Ww, Wh*Ww, 2
+ relative_coords[:, :, 0] += window_size[0] - 1 # shift to start from 0
+ relative_coords[:, :, 1] += window_size[1] - 1
+ relative_coords[:, :, 0] *= 2 * window_size[1] - 1
+ relative_position_index = \
+ paddle.zeros((window_size[0] * window_size[1] + 1,) * 2, dtype=relative_coords.dtype)
+ relative_position_index[1:, 1:] = relative_coords.sum(
+ -1) # Wh*Ww, Wh*Ww
+ relative_position_index[0, 0:] = self.num_relative_distance - 3
+ relative_position_index[0:, 0] = self.num_relative_distance - 2
+ relative_position_index[0, 0] = self.num_relative_distance - 1
+
+ self.register_buffer("relative_position_index",
+ relative_position_index)
+
+ def forward(self):
+ relative_position_bias = \
+ self.relative_position_bias_table[self.relative_position_index.reshape([-1])].reshape([
+ self.window_size[0] * self.window_size[1] + 1,
+ self.window_size[0] * self.window_size[1] + 1, -1]) # Wh*Ww,Wh*Ww,nH
+ return relative_position_bias.transpose([2, 0, 1]) # nH, Wh*Ww, Wh*Ww
+
+
+def get_sinusoid_encoding_table(n_position, d_hid, token=False):
+ ''' Sinusoid position encoding table '''
+
+ def get_position_angle_vec(position):
+ return [
+ position / np.power(10000, 2 * (hid_j // 2) / d_hid)
+ for hid_j in range(d_hid)
+ ]
+
+ sinusoid_table = np.array(
+ [get_position_angle_vec(pos_i) for pos_i in range(n_position)])
+ sinusoid_table[:, 0::2] = np.sin(sinusoid_table[:, 0::2]) # dim 2i
+ sinusoid_table[:, 1::2] = np.cos(sinusoid_table[:, 1::2]) # dim 2i+1
+
+ if token:
+ sinusoid_table = np.concatenate(
+ [sinusoid_table, np.zeros([1, d_hid])], dim=0)
+
+ return paddle.to_tensor(sinusoid_table).unsqueeze(0)
+
+
+class VisionTransformer(nn.Layer):
+ """ Vision Transformer with support for patch or hybrid CNN input stage
+ """
+
+ def __init__(self,
+ img_size=224,
+ patch_size=16,
+ in_chans=3,
+ class_num=1000,
+ embed_dim=768,
+ depth=12,
+ num_heads=12,
+ mlp_ratio=4.,
+ qkv_bias=False,
+ qk_scale=None,
+ drop_rate=0.,
+ attn_drop_rate=0.,
+ drop_path_rate=0.,
+ norm_layer=nn.LayerNorm,
+ init_values=None,
+ use_abs_pos_emb=True,
+ use_rel_pos_bias=False,
+ use_shared_rel_pos_bias=False,
+ use_mean_pooling=True,
+ init_scale=0.001,
+ lin_probe=False,
+ sin_pos_emb=True,
+ args=None):
+ super().__init__()
+ self.class_num = class_num
+ self.num_features = self.embed_dim = embed_dim # num_features for consistency with other models
+ self.use_mean_pooling = use_mean_pooling
+
+ self.patch_embed = PatchEmbed(
+ img_size=img_size,
+ patch_size=patch_size,
+ in_chans=in_chans,
+ embed_dim=embed_dim)
+ num_patches = self.patch_embed.num_patches
+
+ self.zeros_ = nn.initializer.Constant(value=0.)
+ self.ones_ = nn.initializer.Constant(value=1.)
+
+ self.cls_token = self.create_parameter(
+ [1, 1, embed_dim], default_initializer=self.zeros_)
+
+ self.use_abs_pos_emb = use_abs_pos_emb
+ if use_abs_pos_emb:
+ self.pos_embed = self.create_parameter(
+ [1, num_patches + 1, embed_dim],
+ default_initializer=self.zeros_)
+ elif sin_pos_emb:
+ # sine-cosine positional embeddings is on the way
+ self.pos_embed = self.create_parameter(
+ [1, num_patches + 1, embed_dim],
+ default_initializer=self.zeros_)
+ self.pos_embed.set_value(
+ self.build_2d_sincos_position_embedding(embed_dim))
+ self.pos_embed.stop_gradient = True # fixed sin-cos embedding
+ else:
+ self.pos_embed = None
+
+ self.pos_drop = nn.Dropout(p=drop_rate)
+
+ if use_shared_rel_pos_bias:
+ self.rel_pos_bias = RelativePositionBias(
+ window_size=self.patch_embed.patch_shape, num_heads=num_heads)
+ else:
+ self.rel_pos_bias = None
+
+ dpr = [x.item() for x in paddle.linspace(0, drop_path_rate, depth)
+ ] # stochastic depth decay rule
+ self.use_rel_pos_bias = use_rel_pos_bias
+ self.blocks = nn.LayerList([
+ Block(
+ dim=embed_dim,
+ num_heads=num_heads,
+ mlp_ratio=mlp_ratio,
+ qkv_bias=qkv_bias,
+ qk_scale=qk_scale,
+ drop=drop_rate,
+ attn_drop=attn_drop_rate,
+ drop_path=dpr[i],
+ norm_layer=norm_layer,
+ init_values=init_values,
+ window_size=self.patch_embed.patch_shape
+ if use_rel_pos_bias else None) for i in range(depth)
+ ])
+ self.norm = nn.Identity() if use_mean_pooling else norm_layer(
+ embed_dim)
+
+ self.lin_probe = lin_probe
+ # NOTE: batch norm
+ if lin_probe:
+ # TODO
+ from models.lincls_bn import LP_BatchNorm
+ self.fc_norm = LP_BatchNorm(embed_dim, affine=False)
+ else:
+ if use_mean_pooling:
+ self.fc_norm = norm_layer(embed_dim)
+ else:
+ self.fc_norm = None
+ self.head = nn.Linear(embed_dim,
+ class_num) if class_num > 0 else nn.Identity()
+
+ if self.pos_embed is not None and use_abs_pos_emb:
+ trunc_normal_(self.pos_embed, std=.02)
+ trunc_normal_(self.cls_token, std=.02)
+ # trunc_normal_(self.mask_token, std=.02)
+ trunc_normal_(self.head.weight, std=.02)
+ self.apply(self._init_weights)
+ self.fix_init_weight()
+
+ self.head.weight.set_value(self.head.weight * init_scale)
+ self.head.bias.set_value(self.head.bias * init_scale)
+
+ def build_2d_sincos_position_embedding(self,
+ embed_dim=768,
+ temperature=10000.):
+ h, w = self.patch_embed.patch_shape
+ grid_w = paddle.arange(w, dtype=paddle.float32)
+ grid_h = paddle.arange(h, dtype=paddle.float32)
+ grid_w, grid_h = paddle.meshgrid(grid_w, grid_h)
+ assert embed_dim % 4 == 0, 'Embed dimension must be divisible by 4 for 2D sin-cos position embedding'
+ pos_dim = embed_dim // 4
+ omega = paddle.arange(pos_dim, dtype=paddle.float32) / pos_dim
+ omega = 1. / (temperature**omega)
+ out_w = paddle.einsum('m,d->md', grid_w.flatten(), omega)
+ out_h = paddle.einsum('m,d->md', grid_h.flatten(), omega)
+ pos_emb = paddle.concat(
+ [
+ paddle.sin(out_w), paddle.cos(out_w), paddle.sin(out_h),
+ paddle.cos(out_h)
+ ],
+ axis=1)[None, :, :]
+
+ # if not self.use_mean_pooling:
+ pe_token = paddle.zeros([1, 1, embed_dim], dtype=paddle.float32)
+ pos_emb = paddle.concat([pe_token, pos_emb], axis=1)
+ return pos_emb
+
+ def fix_init_weight(self):
+ def rescale(param, layer_id):
+ param.set_value(param / math.sqrt(2.0 * layer_id))
+
+ for layer_id, layer in enumerate(self.blocks):
+ rescale(layer.attn.proj.weight, layer_id + 1)
+ rescale(layer.mlp.fc2.weight, layer_id + 1)
+
+ def _init_weights(self, m):
+ if isinstance(m, nn.Linear):
+ trunc_normal_(m.weight, std=.02)
+ if isinstance(m, nn.Linear) and m.bias is not None:
+ self.zeros_(m.bias)
+ elif isinstance(m, nn.LayerNorm):
+ self.zeros_(m.bias)
+ self.ones_(m.weight)
+
+ def get_num_layers(self):
+ return len(self.blocks)
+
+ def no_weight_decay(self):
+ return {'pos_embed', 'cls_token'}
+
+ def get_classifier(self):
+ return self.head
+
+ def reset_classifier(self, class_num, global_pool=''):
+ self.class_num = class_num
+ self.head = nn.Linear(self.embed_dim,
+ class_num) if class_num > 0 else nn.Identity()
+
+ def forward_features(self, x, is_train=True):
+ x = self.patch_embed(x)
+ batch_size, seq_len, _ = x.shape
+
+ cls_tokens = self.cls_token.expand(
+ [batch_size, -1,
+ -1]) # stole cls_tokens impl from Phil Wang, thanks
+ x = paddle.concat((cls_tokens, x), axis=1)
+ if self.pos_embed is not None:
+ if self.use_abs_pos_emb:
+ x = x + self.pos_embed.expand(
+ [batch_size, -1, -1]).astype(x.dtype).clone().detach()
+ else:
+ x = x + self.pos_embed.expand(
+ [batch_size, -1, -1]).astype(x.dtype).clone().detach()
+
+ x = self.pos_drop(x)
+
+ rel_pos_bias = self.rel_pos_bias(
+ ) if self.rel_pos_bias is not None else None
+ for blk in self.blocks:
+ x = blk(x, rel_pos_bias=rel_pos_bias)
+
+ x = self.norm(x)
+ if self.fc_norm is not None:
+ t = x[:, 1:, :]
+ if self.lin_probe:
+ if self.use_mean_pooling:
+ return self.fc_norm(t.mean(1), is_train=is_train)
+ else:
+ return self.fc_norm(x[:, 0], is_train=is_train)
+ else:
+ return self.fc_norm(t.mean(1))
+
+ else:
+ return x[:, 0]
+
+ def forward(self, x, is_train=True):
+ x = self.forward_features(x, is_train)
+ x = self.head(x)
+ return x
+
+
+def _enable_linear_eval(model):
+ zeros_ = nn.initializer.Constant(value=0.)
+ normal_ = nn.initializer.Normal(mean=0.0, std=0.01)
+ linear_keyword = 'head'
+ head_norm = 'fc_norm'
+ requires_grad = []
+ for name, param in model.named_parameters():
+ if name not in [
+ '%s.weight' % linear_keyword, '%s.bias' % linear_keyword
+ ] and head_norm not in name:
+ param.stop_gradient = True
+ else:
+ requires_grad.append(name)
+ # init the fc layer
+ normal_(getattr(model, linear_keyword).weight)
+ zeros_(getattr(model, linear_keyword).bias)
+
+ return
+
+
+def _load_pretrained(pretrained,
+ pretrained_url,
+ model,
+ model_keys,
+ model_ema_configs,
+ abs_pos_emb,
+ rel_pos_bias,
+ use_ssld=False):
+ if pretrained is False:
+ pass
+ elif pretrained is True:
+ local_weight_path = get_weights_path_from_url(pretrained_url).replace(
+ ".pdparams", "")
+ checkpoint = paddle.load(local_weight_path + ".pdparams")
+ elif isinstance(pretrained, str):
+ checkpoint = paddle.load(local_weight_path + ".pdparams")
+
+ checkpoint_model = None
+ for model_key in model_keys.split('|'):
+ if model_key in checkpoint:
+ checkpoint_model = checkpoint[model_key]
+ break
+
+ if checkpoint_model is None:
+ checkpoint_model = checkpoint
+ state_dict = model.state_dict()
+ all_keys = list(checkpoint_model.keys())
+ # NOTE: remove all decoder keys
+ all_keys = [key for key in all_keys if key.startswith('encoder.')]
+ for key in all_keys:
+ new_key = key.replace('encoder.', '')
+ checkpoint_model[new_key] = checkpoint_model[key]
+ checkpoint_model.pop(key)
+
+ for key in list(checkpoint_model.keys()):
+ if key.startswith('regressor_and_decoder.'):
+ checkpoint_model.pop(key)
+ if key.startswith('teacher_network.'):
+ checkpoint_model.pop(key)
+
+ # NOTE: replace norm with fc_norm
+ for key in list(checkpoint_model.keys()):
+ if key.startswith('norm.'):
+ new_key = key.replace('norm.', 'fc_norm.')
+ checkpoint_model[new_key] = checkpoint_model[key]
+ checkpoint_model.pop(key)
+
+ for k in ['head.weight', 'head.bias']:
+ if k in checkpoint_model and checkpoint_model[k].shape != state_dict[
+ k].shape:
+ del checkpoint_model[k]
+
+ if model.use_rel_pos_bias and "rel_pos_bias.relative_position_bias_table" in checkpoint_model:
+ num_layers = model.get_num_layers()
+ rel_pos_bias = checkpoint_model[
+ "rel_pos_bias.relative_position_bias_table"]
+ for i in range(num_layers):
+ checkpoint_model["blocks.%d.attn.relative_position_bias_table" %
+ i] = rel_pos_bias.clone()
+
+ checkpoint_model.pop("rel_pos_bias.relative_position_bias_table")
+
+ all_keys = list(checkpoint_model.keys())
+
+ for key in all_keys:
+ if "relative_position_index" in key:
+ checkpoint_model.pop(key)
+
+ if "relative_position_bias_table" in key and rel_pos_bias:
+ rel_pos_bias = checkpoint_model[key]
+ src_num_pos, num_attn_heads = rel_pos_bias.size()
+ dst_num_pos, _ = model.state_dict()[key].size()
+ dst_patch_shape = model.patch_embed.patch_shape
+ if dst_patch_shape[0] != dst_patch_shape[1]:
+ raise NotImplementedError()
+ num_extra_tokens = dst_num_pos - (dst_patch_shape[0] * 2 - 1) * (
+ dst_patch_shape[1] * 2 - 1)
+ src_size = int((src_num_pos - num_extra_tokens)**0.5)
+ dst_size = int((dst_num_pos - num_extra_tokens)**0.5)
+ if src_size != dst_size:
+ extra_tokens = rel_pos_bias[-num_extra_tokens:, :]
+ rel_pos_bias = rel_pos_bias[:-num_extra_tokens, :]
+
+ def geometric_progression(a, r, n):
+ return a * (1.0 - r**n) / (1.0 - r)
+
+ left, right = 1.01, 1.5
+ while right - left > 1e-6:
+ q = (left + right) / 2.0
+ gp = geometric_progression(1, q, src_size // 2)
+ if gp > dst_size // 2:
+ right = q
+ else:
+ left = q
+
+ dis = []
+ cur = 1
+ for i in range(src_size // 2):
+ dis.append(cur)
+ cur += q**(i + 1)
+
+ r_ids = [-_ for _ in reversed(dis)]
+
+ x = r_ids + [0] + dis
+ y = r_ids + [0] + dis
+
+ t = dst_size // 2.0
+ dx = np.arange(-t, t + 0.1, 1.0)
+ dy = np.arange(-t, t + 0.1, 1.0)
+
+ all_rel_pos_bias = []
+
+ for i in range(num_attn_heads):
+ z = rel_pos_bias[:, i].view(src_size,
+ src_size).float().numpy()
+ f = interpolate.interp2d(x, y, z, kind='cubic')
+ all_rel_pos_bias.append(
+ paddle.Tensor(f(dx, dy)).contiguous().view(-1, 1).to(
+ rel_pos_bias.device))
+
+ rel_pos_bias = paddle.concat(all_rel_pos_bias, axis=-1)
+
+ new_rel_pos_bias = paddle.concat(
+ (rel_pos_bias, extra_tokens), axis=0)
+ checkpoint_model[key] = new_rel_pos_bias
+
+ # interpolate position embedding
+ if 'pos_embed' in checkpoint_model and abs_pos_emb:
+ pos_embed_checkpoint = checkpoint_model['pos_embed']
+ embedding_size = pos_embed_checkpoint.shape[-1]
+ num_patches = model.patch_embed.num_patches
+ num_extra_tokens = model.pos_embed.shape[-2] - num_patches
+ # height (== width) for the checkpoint position embedding
+ orig_size = int((pos_embed_checkpoint.shape[-2] - num_extra_tokens)**
+ 0.5)
+ # height (== width) for the new position embedding
+ new_size = int(num_patches**0.5)
+ # class_token and dist_token are kept unchanged
+ if orig_size != new_size:
+ extra_tokens = pos_embed_checkpoint[:, :num_extra_tokens]
+ # only the position tokens are interpolated
+ pos_tokens = pos_embed_checkpoint[:, num_extra_tokens:]
+ pos_tokens = pos_tokens.reshape(-1, orig_size, orig_size,
+ embedding_size).permute(0, 3, 1, 2)
+ pos_tokens = paddle.nn.functional.interpolate(
+ pos_tokens,
+ size=(new_size, new_size),
+ mode='bicubic',
+ align_corners=False)
+ pos_tokens = pos_tokens.permute(0, 2, 3, 1).flatten(1, 2)
+ new_pos_embed = paddle.concat((extra_tokens, pos_tokens), axis=1)
+ checkpoint_model['pos_embed'] = new_pos_embed
+ msg = model.set_state_dict(checkpoint_model)
+
+ model_without_ddp = model
+ n_parameters = sum(p.numel() for p in model.parameters()
+ if not p.stop_gradient).item()
+
+ return
+
+
+def cae_base_patch16_224(pretrained=True, use_ssld=False, **kwargs):
+ config = kwargs.copy()
+ enable_linear_eval = config.pop('enable_linear_eval')
+ model_keys = config.pop('model_key')
+ model_ema_configs = config.pop('model_ema')
+ abs_pos_emb = config.pop('abs_pos_emb')
+ rel_pos_bias = config.pop('rel_pos_bias')
+ if pretrained in config:
+ pretrained = config.pop('pretrained')
+
+ model = VisionTransformer(
+ patch_size=16,
+ embed_dim=768,
+ depth=12,
+ num_heads=12,
+ mlp_ratio=4,
+ qkv_bias=True,
+ norm_layer=partial(
+ nn.LayerNorm, epsilon=1e-6),
+ **config)
+
+ if enable_linear_eval:
+ _enable_linear_eval(model)
+
+ _load_pretrained(
+ pretrained,
+ MODEL_URLS["cae_base_patch16_224"],
+ model,
+ model_keys,
+ model_ema_configs,
+ abs_pos_emb,
+ rel_pos_bias,
+ use_ssld=False)
+
+ return model
+
+
+def cae_large_patch16_224(pretrained=True, use_ssld=False, **kwargs):
+ config = kwargs.copy()
+ enable_linear_eval = config.pop('enable_linear_eval')
+ model_keys = config.pop('model_key')
+ model_ema_configs = config.pop('model_ema')
+ abs_pos_emb = config.pop('abs_pos_emb')
+ rel_pos_bias = config.pop('rel_pos_bias')
+ if pretrained in config:
+ pretrained = config.pop('pretrained')
+
+ model = VisionTransformer(
+ patch_size=16,
+ embed_dim=1024,
+ depth=24,
+ num_heads=16,
+ mlp_ratio=4,
+ qkv_bias=True,
+ norm_layer=partial(
+ nn.LayerNorm, epsilon=1e-6),
+ **config)
+
+ if enable_linear_eval:
+ _enable_linear_eval(model)
+
+ _load_pretrained(
+ pretrained,
+ MODEL_URLS["cae_large_patch16_224"],
+ model,
+ model_keys,
+ model_ema_configs,
+ abs_pos_emb,
+ rel_pos_bias,
+ use_ssld=False)
+
+ return model
diff --git a/ppcls/arch/backbone/variant_models/__init__.py b/ppcls/arch/backbone/variant_models/__init__.py
index 75cf29ffa9c59b744972a9e82fba7a506219e83b..d2fcd0bdd9b83c6e87a6a0684382c380e5fff93a 100644
--- a/ppcls/arch/backbone/variant_models/__init__.py
+++ b/ppcls/arch/backbone/variant_models/__init__.py
@@ -1,3 +1,4 @@
from .resnet_variant import ResNet50_last_stage_stride1
from .vgg_variant import VGG19Sigmoid
from .pp_lcnet_variant import PPLCNet_x2_5_Tanh
+from .pp_lcnetv2_variant import PPLCNetV2_base_ShiTu
diff --git a/ppcls/arch/backbone/variant_models/pp_lcnetv2_variant.py b/ppcls/arch/backbone/variant_models/pp_lcnetv2_variant.py
new file mode 100644
index 0000000000000000000000000000000000000000..6acccdc8e5c115cf4e1e6b213ab3ea3ffcc710b3
--- /dev/null
+++ b/ppcls/arch/backbone/variant_models/pp_lcnetv2_variant.py
@@ -0,0 +1,56 @@
+from paddle.nn import Conv2D, Identity
+
+from ..legendary_models.pp_lcnet_v2 import MODEL_URLS, PPLCNetV2_base, RepDepthwiseSeparable, _load_pretrained
+
+__all__ = ["PPLCNetV2_base_ShiTu"]
+
+
+def PPLCNetV2_base_ShiTu(pretrained=False, use_ssld=False, **kwargs):
+ """
+ An variant network of PPLCNetV2_base
+ 1. remove ReLU layer after last_conv
+ 2. add bias to last_conv
+ 3. change stride to 1 in last two RepDepthwiseSeparable Block
+ """
+ model = PPLCNetV2_base(pretrained=False, use_ssld=use_ssld, **kwargs)
+
+ def remove_ReLU_function(conv, pattern):
+ new_conv = Identity()
+ return new_conv
+
+ def add_bias_last_conv(conv, pattern):
+ new_conv = Conv2D(
+ in_channels=conv._in_channels,
+ out_channels=conv._out_channels,
+ kernel_size=conv._kernel_size,
+ stride=conv._stride,
+ padding=conv._padding,
+ groups=conv._groups,
+ bias_attr=True)
+ return new_conv
+
+ def last_stride_function(rep_block, pattern):
+ new_conv = RepDepthwiseSeparable(
+ in_channels=rep_block.in_channels,
+ out_channels=rep_block.out_channels,
+ stride=1,
+ dw_size=rep_block.dw_size,
+ split_pw=rep_block.split_pw,
+ use_rep=rep_block.use_rep,
+ use_se=rep_block.use_se,
+ use_shortcut=rep_block.use_shortcut)
+ return new_conv
+
+ pattern_act = ["act"]
+ pattern_lastconv = ["last_conv"]
+ pattern_last_stride = [
+ "stages[3][0]",
+ "stages[3][1]",
+ ]
+ model.upgrade_sublayer(pattern_act, remove_ReLU_function)
+ model.upgrade_sublayer(pattern_lastconv, add_bias_last_conv)
+ model.upgrade_sublayer(pattern_last_stride, last_stride_function)
+
+ # load params again after upgrade some layers
+ _load_pretrained(pretrained, model, MODEL_URLS["PPLCNetV2_base"], use_ssld)
+ return model
diff --git a/ppcls/configs/CAE/cae_base_patch16_224_finetune.yaml b/ppcls/configs/CAE/cae_base_patch16_224_finetune.yaml
new file mode 100644
index 0000000000000000000000000000000000000000..7ec1c9c4693fce359f8bd44fdc38f50baf6465fc
--- /dev/null
+++ b/ppcls/configs/CAE/cae_base_patch16_224_finetune.yaml
@@ -0,0 +1,167 @@
+# global configs
+Global:
+ checkpoints: null
+ pretrained_model: null
+ output_dir: ./output/
+ device: gpu
+ save_interval: 20
+ eval_during_train: True
+ eval_interval: 1
+ epochs: 100
+ print_batch_step: 10
+ use_visualdl: False
+ # used for static mode and model export
+ image_shape: [3, 224, 224]
+ save_inference_dir: ./inference
+
+# model architecture
+Arch:
+ name: cae_base_patch16_224
+ class_num: 102
+ drop_rate: 0.0
+ drop_path_rate: 0.1
+ attn_drop_rate: 0.0
+
+ use_mean_pooling: True
+ init_scale: 0.001
+ use_rel_pos_bias: True
+ use_abs_pos_emb: False
+ init_values: 0.1
+ lin_probe: False
+
+ sin_pos_emb: True
+
+ abs_pos_emb: False
+ enable_linear_eval: False
+ model_key: model|module|state_dict
+ rel_pos_bias: True
+ model_ema:
+ enable_model_ema: False
+ model_ema_decay: 0.9999
+ model_ema_force_cpu: False
+ pretrained: True
+
+# loss function config for traing/eval process
+Loss:
+ Train:
+ - SoftTargetCrossEntropy:
+ weight: 1.0
+ Eval:
+ - CELoss:
+ weight: 1.0
+
+
+Optimizer:
+ name: AdamWDL
+ beta1: 0.9
+ beta2: 0.999
+ epsilon: 1e-8
+ weight_decay: 0.05
+ layerwise_decay: 0.65
+ lr:
+ name: Cosine
+ learning_rate: 0.001
+ eta_min: 1e-6
+ warmup_epoch: 10
+ warmup_start_lr: 1e-6
+
+
+# data loader for train and eval
+DataLoader:
+ Train:
+ dataset:
+ name: ImageNetDataset
+ image_root: ./dataset/flowers102/
+ cls_label_path: ./dataset/flowers102/train_list.txt
+ batch_transform_ops:
+ - MixupCutmixHybrid:
+ mixup_alpha: 0.8
+ cutmix_alpha: 1.0
+ switch_prob: 0.5
+ num_classes: 102
+ transform_ops:
+ - DecodeImage:
+ to_rgb: True
+ channel_first: False
+ - RandCropImage:
+ size: 224
+ interpolation: bilinear
+ - RandFlipImage:
+ flip_code: 1
+ - RandAugment:
+ - NormalizeImage:
+ scale: 1.0/255.0
+ mean: [0.485, 0.456, 0.406]
+ std: [0.229, 0.224, 0.225]
+ order: ''
+ - RandomErasing:
+ EPSILON: 0.5
+ sl: 0.02
+ sh: 0.3
+ r1: 0.3
+
+ sampler:
+ name: DistributedBatchSampler
+ batch_size: 16
+ drop_last: True
+ shuffle: True
+ loader:
+ num_workers: 4
+ use_shared_memory: True
+
+ Eval:
+ dataset:
+ name: ImageNetDataset
+ image_root: ./dataset/flowers102/
+ cls_label_path: ./dataset/flowers102/val_list.txt
+ transform_ops:
+ - DecodeImage:
+ to_rgb: True
+ channel_first: False
+ - ResizeImage:
+ resize_short: 256
+ - CropImage:
+ size: 224
+ - NormalizeImage:
+ scale: 1.0/255.0
+ mean: [0.485, 0.456, 0.406]
+ std: [0.229, 0.224, 0.225]
+ order: ''
+ sampler:
+ name: DistributedBatchSampler
+ batch_size: 16
+ drop_last: False
+ shuffle: False
+ loader:
+ num_workers: 4
+ use_shared_memory: True
+
+Infer:
+ infer_imgs: docs/images/inference_deployment/whl_demo.jpg
+ batch_size: 10
+ transforms:
+ - DecodeImage:
+ to_rgb: True
+ channel_first: False
+ - ResizeImage:
+ resize_short: 256
+ - CropImage:
+ size: 224
+ - NormalizeImage:
+ scale: 1.0/255.0
+ mean: [0.5, 0.5, 0.5]
+ std: [0.5, 0.5, 0.5]
+ order: ''
+ - ToCHWImage:
+ PostProcess:
+ name: Topk
+ topk: 5
+ class_id_map_file: ppcls/utils/imagenet1k_label_list.txt
+
+Metric:
+ Train:
+ - TopkAcc:
+ topk: [1, 5]
+ Eval:
+ - TopkAcc:
+ topk: [1, 5]
diff --git a/ppcls/configs/CAE/cae_large_patch16_224_finetune.yaml b/ppcls/configs/CAE/cae_large_patch16_224_finetune.yaml
new file mode 100644
index 0000000000000000000000000000000000000000..f8f7edc58a1e161ecaf64e5bd6cb7cd5b62a6c40
--- /dev/null
+++ b/ppcls/configs/CAE/cae_large_patch16_224_finetune.yaml
@@ -0,0 +1,167 @@
+# global configs
+Global:
+ checkpoints: null
+ pretrained_model: null
+ output_dir: ./output/
+ device: gpu
+ save_interval: 20
+ eval_during_train: True
+ eval_interval: 1
+ epochs: 100
+ print_batch_step: 10
+ use_visualdl: False
+ # used for static mode and model export
+ image_shape: [3, 224, 224]
+ save_inference_dir: ./inference
+
+# model architecture
+Arch:
+ name: cae_large_patch16_224
+ class_num: 102
+ drop_rate: 0.0
+ drop_path_rate: 0.2
+ attn_drop_rate: 0.0
+
+ use_mean_pooling: True
+ init_scale: 0.001
+ use_rel_pos_bias: True
+ use_abs_pos_emb: False
+ init_values: 0.1
+ lin_probe: False
+
+ sin_pos_emb: True
+
+ abs_pos_emb: False
+ enable_linear_eval: False
+ model_key: model|module|state_dict
+ rel_pos_bias: True
+ model_ema:
+ enable_model_ema: False
+ model_ema_decay: 0.9999
+ model_ema_force_cpu: False
+ pretrained: True
+
+# loss function config for traing/eval process
+Loss:
+ Train:
+ - SoftTargetCrossEntropy:
+ weight: 1.0
+ Eval:
+ - CELoss:
+ weight: 1.0
+
+
+Optimizer:
+ name: AdamWDL
+ beta1: 0.9
+ beta2: 0.999
+ epsilon: 1e-8
+ weight_decay: 0.05
+ layerwise_decay: 0.75
+ lr:
+ name: Cosine
+ learning_rate: 0.001
+ eta_min: 1e-6
+ warmup_epoch: 10
+ warmup_start_lr: 1e-6
+
+
+# data loader for train and eval
+DataLoader:
+ Train:
+ dataset:
+ name: ImageNetDataset
+ image_root: ./dataset/flowers102/
+ cls_label_path: ./dataset/flowers102/train_list.txt
+ batch_transform_ops:
+ - MixupCutmixHybrid:
+ mixup_alpha: 0.8
+ cutmix_alpha: 1.0
+ switch_prob: 0.5
+ num_classes: 102
+ transform_ops:
+ - DecodeImage:
+ to_rgb: True
+ channel_first: False
+ - RandCropImage:
+ size: 224
+ interpolation: bilinear
+ - RandFlipImage:
+ flip_code: 1
+ - RandAugment:
+ - NormalizeImage:
+ scale: 1.0/255.0
+ mean: [0.485, 0.456, 0.406]
+ std: [0.229, 0.224, 0.225]
+ order: ''
+ - RandomErasing:
+ EPSILON: 0.5
+ sl: 0.02
+ sh: 0.3
+ r1: 0.3
+
+ sampler:
+ name: DistributedBatchSampler
+ batch_size: 16
+ drop_last: True
+ shuffle: True
+ loader:
+ num_workers: 4
+ use_shared_memory: True
+
+ Eval:
+ dataset:
+ name: ImageNetDataset
+ image_root: ./dataset/flowers102/
+ cls_label_path: ./dataset/flowers102/val_list.txt
+ transform_ops:
+ - DecodeImage:
+ to_rgb: True
+ channel_first: False
+ - ResizeImage:
+ resize_short: 256
+ - CropImage:
+ size: 224
+ - NormalizeImage:
+ scale: 1.0/255.0
+ mean: [0.485, 0.456, 0.406]
+ std: [0.229, 0.224, 0.225]
+ order: ''
+ sampler:
+ name: DistributedBatchSampler
+ batch_size: 16
+ drop_last: False
+ shuffle: False
+ loader:
+ num_workers: 4
+ use_shared_memory: True
+
+Infer:
+ infer_imgs: docs/images/inference_deployment/whl_demo.jpg
+ batch_size: 10
+ transforms:
+ - DecodeImage:
+ to_rgb: True
+ channel_first: False
+ - ResizeImage:
+ resize_short: 256
+ - CropImage:
+ size: 224
+ - NormalizeImage:
+ scale: 1.0/255.0
+ mean: [0.5, 0.5, 0.5]
+ std: [0.5, 0.5, 0.5]
+ order: ''
+ - ToCHWImage:
+ PostProcess:
+ name: Topk
+ topk: 5
+ class_id_map_file: ppcls/utils/imagenet1k_label_list.txt
+
+Metric:
+ Train:
+ - TopkAcc:
+ topk: [1, 5]
+ Eval:
+ - TopkAcc:
+ topk: [1, 5]
diff --git a/ppcls/configs/GeneralRecognition/GeneralRecognition_PPLCNet_x2_5.yaml b/ppcls/configs/GeneralRecognition/GeneralRecognition_PPLCNet_x2_5.yaml
index 626dd7c2ee05ffac21320b948855d12442e64eff..70daa639bebad9fc6050276cf6303bb5a8b4b47a 100644
--- a/ppcls/configs/GeneralRecognition/GeneralRecognition_PPLCNet_x2_5.yaml
+++ b/ppcls/configs/GeneralRecognition/GeneralRecognition_PPLCNet_x2_5.yaml
@@ -23,7 +23,7 @@ Arch:
infer_output_key: features
infer_add_softmax: False
- Backbone:
+ Backbone:
name: PPLCNet_x2_5
pretrained: True
use_ssld: True
@@ -34,7 +34,7 @@ Arch:
embedding_size: 1280
class_num: 512
Head:
- name: ArcMargin
+ name: ArcMargin
embedding_size: 512
class_num: 185341
margin: 0.2
@@ -57,10 +57,9 @@ Optimizer:
learning_rate: 0.04
warmup_epoch: 5
regularizer:
- name: 'L2'
+ name: "L2"
coeff: 0.00001
-
# data loader for train and eval
DataLoader:
Train:
@@ -80,7 +79,7 @@ DataLoader:
scale: 1.0/255.0
mean: [0.485, 0.456, 0.406]
std: [0.229, 0.224, 0.225]
- order: ''
+ order: ""
sampler:
name: DistributedBatchSampler
@@ -93,7 +92,7 @@ DataLoader:
Eval:
Query:
- dataset:
+ dataset:
name: VeriWild
image_root: ./dataset/Aliproduct/
cls_label_path: ./dataset/Aliproduct/val_list.txt
@@ -107,7 +106,7 @@ DataLoader:
scale: 0.00392157
mean: [0.485, 0.456, 0.406]
std: [0.229, 0.224, 0.225]
- order: ''
+ order: ""
sampler:
name: DistributedBatchSampler
batch_size: 64
@@ -118,7 +117,7 @@ DataLoader:
use_shared_memory: True
Gallery:
- dataset:
+ dataset:
name: VeriWild
image_root: ./dataset/Aliproduct/
cls_label_path: ./dataset/Aliproduct/val_list.txt
@@ -132,7 +131,7 @@ DataLoader:
scale: 0.00392157
mean: [0.485, 0.456, 0.406]
std: [0.229, 0.224, 0.225]
- order: ''
+ order: ""
sampler:
name: DistributedBatchSampler
batch_size: 64
@@ -146,3 +145,4 @@ Metric:
Eval:
- Recallk:
topk: [1, 5]
+ - mAP: {}
diff --git a/ppcls/configs/GeneralRecognitionV2/GeneralRecognitionV2_PPLCNetV2_base.yaml b/ppcls/configs/GeneralRecognitionV2/GeneralRecognitionV2_PPLCNetV2_base.yaml
new file mode 100644
index 0000000000000000000000000000000000000000..e6dfde7cdde9b88772ac414cb0de1646daf9c304
--- /dev/null
+++ b/ppcls/configs/GeneralRecognitionV2/GeneralRecognitionV2_PPLCNetV2_base.yaml
@@ -0,0 +1,205 @@
+# global configs
+Global:
+ checkpoints: null
+ pretrained_model: null
+ output_dir: ./output
+ device: gpu
+ save_interval: 1
+ eval_during_train: True
+ eval_interval: 1
+ epochs: 100
+ print_batch_step: 20
+ use_visualdl: False
+ eval_mode: retrieval
+ retrieval_feature_from: features # 'backbone' or 'features'
+ re_ranking: False
+ use_dali: False
+ # used for static mode and model export
+ image_shape: [3, 224, 224]
+ save_inference_dir: ./inference
+
+AMP:
+ scale_loss: 65536
+ use_dynamic_loss_scaling: True
+ # O1: mixed fp16
+ level: O1
+
+# model architecture
+Arch:
+ name: RecModel
+ infer_output_key: features
+ infer_add_softmax: False
+
+ Backbone:
+ name: PPLCNetV2_base_ShiTu
+ pretrained: True
+ use_ssld: True
+ class_expand: &feat_dim 512
+ BackboneStopLayer:
+ name: flatten
+ Neck:
+ name: BNNeck
+ num_features: *feat_dim
+ weight_attr:
+ initializer:
+ name: Constant
+ value: 1.0
+ bias_attr:
+ initializer:
+ name: Constant
+ value: 0.0
+ learning_rate: 1.0e-20 # NOTE: Temporarily set lr small enough to freeze the bias to zero
+ Head:
+ name: FC
+ embedding_size: *feat_dim
+ class_num: 192612
+ weight_attr:
+ initializer:
+ name: Normal
+ std: 0.001
+ bias_attr: False
+
+# loss function config for traing/eval process
+Loss:
+ Train:
+ - CELoss:
+ weight: 1.0
+ epsilon: 0.1
+ - TripletAngularMarginLoss:
+ weight: 1.0
+ feature_from: features
+ margin: 0.5
+ reduction: mean
+ add_absolute: True
+ absolute_loss_weight: 0.1
+ normalize_feature: True
+ ap_value: 0.8
+ an_value: 0.4
+ Eval:
+ - CELoss:
+ weight: 1.0
+
+Optimizer:
+ name: Momentum
+ momentum: 0.9
+ lr:
+ name: Cosine
+ learning_rate: 0.06 # for 8gpu x 256bs
+ warmup_epoch: 5
+ regularizer:
+ name: L2
+ coeff: 0.00001
+
+# data loader for train and eval
+DataLoader:
+ Train:
+ dataset:
+ name: ImageNetDataset
+ image_root: ./dataset/
+ cls_label_path: ./dataset/train_reg_all_data_v2.txt
+ relabel: True
+ transform_ops:
+ - DecodeImage:
+ to_rgb: True
+ channel_first: False
+ - ResizeImage:
+ size: [224, 224]
+ return_numpy: False
+ interpolation: bilinear
+ backend: cv2
+ - RandFlipImage:
+ flip_code: 1
+ - Pad:
+ padding: 10
+ backend: cv2
+ - RandCropImageV2:
+ size: [224, 224]
+ - RandomRotation:
+ prob: 0.5
+ degrees: 90
+ interpolation: bilinear
+ - ResizeImage:
+ size: [224, 224]
+ return_numpy: False
+ interpolation: bilinear
+ backend: cv2
+ - NormalizeImage:
+ scale: 1.0/255.0
+ mean: [0.485, 0.456, 0.406]
+ std: [0.229, 0.224, 0.225]
+ order: hwc
+ sampler:
+ name: PKSampler
+ batch_size: 256
+ sample_per_id: 4
+ drop_last: False
+ shuffle: True
+ sample_method: "id_avg_prob"
+ id_list: [50030, 80700, 92019, 96015] # be careful when set relabel=True
+ ratio: [4, 4]
+ loader:
+ num_workers: 4
+ use_shared_memory: True
+
+ Eval:
+ Query:
+ dataset:
+ name: VeriWild
+ image_root: ./dataset/Aliproduct/
+ cls_label_path: ./dataset/Aliproduct/val_list.txt
+ transform_ops:
+ - DecodeImage:
+ to_rgb: True
+ channel_first: False
+ - ResizeImage:
+ size: [224, 224]
+ return_numpy: False
+ interpolation: bilinear
+ backend: cv2
+ - NormalizeImage:
+ scale: 1.0/255.0
+ mean: [0.485, 0.456, 0.406]
+ std: [0.229, 0.224, 0.225]
+ order: hwc
+ sampler:
+ name: DistributedBatchSampler
+ batch_size: 64
+ drop_last: False
+ shuffle: False
+ loader:
+ num_workers: 4
+ use_shared_memory: True
+
+ Gallery:
+ dataset:
+ name: VeriWild
+ image_root: ./dataset/Aliproduct/
+ cls_label_path: ./dataset/Aliproduct/val_list.txt
+ transform_ops:
+ - DecodeImage:
+ to_rgb: True
+ channel_first: False
+ - ResizeImage:
+ size: [224, 224]
+ return_numpy: False
+ interpolation: bilinear
+ backend: cv2
+ - NormalizeImage:
+ scale: 1.0/255.0
+ mean: [0.485, 0.456, 0.406]
+ std: [0.229, 0.224, 0.225]
+ order: hwc
+ sampler:
+ name: DistributedBatchSampler
+ batch_size: 64
+ drop_last: False
+ shuffle: False
+ loader:
+ num_workers: 4
+ use_shared_memory: True
+
+Metric:
+ Eval:
+ - Recallk:
+ topk: [1, 5]
+ - mAP: {}
diff --git a/ppcls/configs/ImageNet/Distillation/resnet34_distill_resnet18_wsl.yaml b/ppcls/configs/ImageNet/Distillation/resnet34_distill_resnet18_wsl.yaml
new file mode 100644
index 0000000000000000000000000000000000000000..7822a2bea425ac657dc9a10644aa2cdf1fd4273c
--- /dev/null
+++ b/ppcls/configs/ImageNet/Distillation/resnet34_distill_resnet18_wsl.yaml
@@ -0,0 +1,152 @@
+# global configs
+Global:
+ checkpoints: null
+ pretrained_model: null
+ output_dir: ./output/r34_r18_wsl
+ device: "gpu"
+ save_interval: 1
+ eval_during_train: True
+ eval_interval: 1
+ epochs: 100
+ print_batch_step: 10
+ use_visualdl: False
+ # used for static mode and model export
+ image_shape: [3, 224, 224]
+ save_inference_dir: "./inference"
+
+# model architecture
+Arch:
+ name: "DistillationModel"
+ # if not null, its lengths should be same as models
+ pretrained_list:
+ # if not null, its lengths should be same as models
+ freeze_params_list:
+ - True
+ - False
+ models:
+ - Teacher:
+ name: ResNet34
+ pretrained: True
+
+ - Student:
+ name: ResNet18
+ pretrained: False
+
+ infer_model_name: "Student"
+
+
+# loss function config for traing/eval process
+Loss:
+ Train:
+ - DistillationGTCELoss:
+ weight: 1.0
+ model_names: ["Student"]
+ - DistillationWSLLoss:
+ weight: 2.5
+ model_name_pairs: [["Student", "Teacher"]]
+ temperature: 2
+ Eval:
+ - CELoss:
+ weight: 1.0
+
+
+Optimizer:
+ name: Momentum
+ momentum: 0.9
+ weight_decay: 1e-4
+ lr:
+ name: MultiStepDecay
+ learning_rate: 0.1
+ milestones: [30, 60, 90]
+ step_each_epoch: 1
+ gamma: 0.1
+
+
+# data loader for train and eval
+DataLoader:
+ Train:
+ dataset:
+ name: ImageNetDataset
+ image_root: "./dataset/ILSVRC2012/"
+ cls_label_path: "./dataset/ILSVRC2012/train_list.txt"
+ transform_ops:
+ - DecodeImage:
+ to_rgb: True
+ channel_first: False
+ - RandCropImage:
+ size: 224
+ - RandFlipImage:
+ flip_code: 1
+ - NormalizeImage:
+ scale: 0.00392157
+ mean: [0.485, 0.456, 0.406]
+ std: [0.229, 0.224, 0.225]
+ order: ''
+
+ sampler:
+ name: DistributedBatchSampler
+ batch_size: 64
+ drop_last: False
+ shuffle: True
+ loader:
+ num_workers: 8
+ use_shared_memory: True
+
+ Eval:
+ dataset:
+ name: ImageNetDataset
+ image_root: "./dataset/ILSVRC2012/"
+ cls_label_path: "./dataset/ILSVRC2012/val_list.txt"
+ transform_ops:
+ - DecodeImage:
+ to_rgb: True
+ channel_first: False
+ - ResizeImage:
+ resize_short: 256
+ - CropImage:
+ size: 224
+ - NormalizeImage:
+ scale: 0.00392157
+ mean: [0.485, 0.456, 0.406]
+ std: [0.229, 0.224, 0.225]
+ order: ''
+ sampler:
+ name: DistributedBatchSampler
+ batch_size: 64
+ drop_last: False
+ shuffle: False
+ loader:
+ num_workers: 4
+ use_shared_memory: True
+
+Infer:
+ infer_imgs: "docs/images/inference_deployment/whl_demo.jpg"
+ batch_size: 10
+ transforms:
+ - DecodeImage:
+ to_rgb: True
+ channel_first: False
+ - ResizeImage:
+ resize_short: 256
+ - CropImage:
+ size: 224
+ - NormalizeImage:
+ scale: 1.0/255.0
+ mean: [0.485, 0.456, 0.406]
+ std: [0.229, 0.224, 0.225]
+ order: ''
+ - ToCHWImage:
+ PostProcess:
+ name: Topk
+ topk: 5
+ class_id_map_file: "ppcls/utils/imagenet1k_label_list.txt"
+
+Metric:
+ Train:
+ - DistillationTopkAcc:
+ model_key: "Student"
+ topk: [1, 5]
+ Eval:
+ - DistillationTopkAcc:
+ model_key: "Student"
+ topk: [1, 5]
diff --git a/ppcls/configs/ImageNet/PPHGNet/PPHGNet_base.yaml b/ppcls/configs/ImageNet/PPHGNet/PPHGNet_base.yaml
index 5e07692b01715ffa8196e5ded4604f9294d1ed07..8941894ad707bf6e34a9b9b83b446b72cd83ca1c 100644
--- a/ppcls/configs/ImageNet/PPHGNet/PPHGNet_base.yaml
+++ b/ppcls/configs/ImageNet/PPHGNet/PPHGNet_base.yaml
@@ -142,6 +142,8 @@ Infer:
channel_first: False
- ResizeImage:
resize_short: 236
+ interpolation: bicubic
+ backend: pil
- CropImage:
size: 224
- NormalizeImage:
diff --git a/ppcls/configs/ImageNet/PPHGNet/PPHGNet_small.yaml b/ppcls/configs/ImageNet/PPHGNet/PPHGNet_small.yaml
index eabccd4b712ab48886c74caf6b784b4c193f6913..227450fca21cf2feecb9c616a4ac3d17ce881a60 100644
--- a/ppcls/configs/ImageNet/PPHGNet/PPHGNet_small.yaml
+++ b/ppcls/configs/ImageNet/PPHGNet/PPHGNet_small.yaml
@@ -142,6 +142,8 @@ Infer:
channel_first: False
- ResizeImage:
resize_short: 236
+ interpolation: bicubic
+ backend: pil
- CropImage:
size: 224
- NormalizeImage:
diff --git a/ppcls/configs/ImageNet/PPHGNet/PPHGNet_tiny.yaml b/ppcls/configs/ImageNet/PPHGNet/PPHGNet_tiny.yaml
index e423c866b131aefda13b0186eca7ac27d3c84733..f8332a0430a89da37208915d9df177f16f8d0cda 100644
--- a/ppcls/configs/ImageNet/PPHGNet/PPHGNet_tiny.yaml
+++ b/ppcls/configs/ImageNet/PPHGNet/PPHGNet_tiny.yaml
@@ -142,6 +142,8 @@ Infer:
channel_first: False
- ResizeImage:
resize_short: 232
+ interpolation: bicubic
+ backend: pil
- CropImage:
size: 224
- NormalizeImage:
diff --git a/ppcls/configs/PULC/table_attribute/PPLCNet_x1_0.yaml b/ppcls/configs/PULC/table_attribute/PPLCNet_x1_0.yaml
new file mode 100644
index 0000000000000000000000000000000000000000..2c1e9b253cb3b60afa635ebb6bb94dbdcf5bb886
--- /dev/null
+++ b/ppcls/configs/PULC/table_attribute/PPLCNet_x1_0.yaml
@@ -0,0 +1,133 @@
+# global configs
+Global:
+ checkpoints: null
+ pretrained_model: null
+ output_dir: "./output/"
+ device: "gpu"
+ save_interval: 1
+ eval_during_train: True
+ eval_interval: 1
+ epochs: 20
+ print_batch_step: 10
+ use_visualdl: False
+ # used for static mode and model export
+ image_shape: [3, 224, 224]
+ save_inference_dir: "./inference"
+ use_multilabel: True
+
+# model architecture
+Arch:
+ name: "PPLCNet_x1_0"
+ pretrained: True
+ use_ssld: True
+ class_num: 6
+
+
+# loss function config for traing/eval process
+Loss:
+ Train:
+ - MultiLabelLoss:
+ weight: 1.0
+ weight_ratio: True
+ size_sum: True
+ Eval:
+ - MultiLabelLoss:
+ weight: 1.0
+ weight_ratio: True
+ size_sum: True
+
+Optimizer:
+ name: Momentum
+ momentum: 0.9
+ lr:
+ name: Cosine
+ learning_rate: 0.01
+ warmup_epoch: 5
+ regularizer:
+ name: 'L2'
+ coeff: 0.0005
+
+# data loader for train and eval
+DataLoader:
+ Train:
+ dataset:
+ name: MultiLabelDataset
+ image_root: "dataset/table_attribute/"
+ cls_label_path: "dataset/table_attribute/train_list.txt"
+ label_ratio: True
+ transform_ops:
+ - DecodeImage:
+ to_rgb: True
+ channel_first: False
+ - ResizeImage:
+ size: [224, 224]
+ - RandFlipImage:
+ flip_code: 1
+ - NormalizeImage:
+ scale: 1.0/255.0
+ mean: [0.485, 0.456, 0.406]
+ std: [0.229, 0.224, 0.225]
+ order: ''
+ sampler:
+ name: DistributedBatchSampler
+ batch_size: 64
+ drop_last: True
+ shuffle: True
+ loader:
+ num_workers: 4
+ use_shared_memory: True
+ Eval:
+ dataset:
+ name: MultiLabelDataset
+ image_root: "dataset/table_attribute/"
+ cls_label_path: "dataset/table_attribute/val_list.txt"
+ label_ratio: True
+ transform_ops:
+ - DecodeImage:
+ to_rgb: True
+ channel_first: False
+ - ResizeImage:
+ size: [224, 224]
+ - NormalizeImage:
+ scale: 1.0/255.0
+ mean: [0.485, 0.456, 0.406]
+ std: [0.229, 0.224, 0.225]
+ order: ''
+ sampler:
+ name: DistributedBatchSampler
+ batch_size: 64
+ drop_last: False
+ shuffle: False
+ loader:
+ num_workers: 4
+ use_shared_memory: True
+
+Infer:
+ infer_imgs: deploy/images/PULC/table_attribute/val_3610.jpg
+ batch_size: 10
+ transforms:
+ - DecodeImage:
+ to_rgb: True
+ channel_first: False
+ - ResizeImage:
+ size: [224, 224]
+ - NormalizeImage:
+ scale: 1.0/255.0
+ mean: [0.485, 0.456, 0.406]
+ std: [0.229, 0.224, 0.225]
+ order: ''
+ - ToCHWImage:
+ PostProcess:
+ name: TableAttribute
+ source_threshold: 0.5
+ number_threshold: 0.5
+ color_threshold: 0.5
+ clarity_threshold : 0.5
+ obstruction_threshold: 0.5
+ angle_threshold: 0.5
+
+Metric:
+ Eval:
+ - ATTRMetric:
+
+
diff --git a/ppcls/configs/quick_start/MobileNetV1_retrieval.yaml b/ppcls/configs/quick_start/MobileNetV1_retrieval.yaml
index f088e1cd9c9024fc80a3a171cee8d6865abd7dda..bac477392c3b1fe45a6cbe7e643a4a5aec96ac3e 100644
--- a/ppcls/configs/quick_start/MobileNetV1_retrieval.yaml
+++ b/ppcls/configs/quick_start/MobileNetV1_retrieval.yaml
@@ -20,8 +20,8 @@ Arch:
name: RecModel
infer_output_key: features
infer_add_softmax: False
-
- Backbone:
+
+ Backbone:
name: MobileNetV1
pretrained: False
BackboneStopLayer:
@@ -31,12 +31,12 @@ Arch:
embedding_size: 1024
class_num: 512
Head:
- name: ArcMargin
+ name: ArcMargin
embedding_size: 512
class_num: 101
margin: 0.15
scale: 30
-
+
# loss function config for traing/eval process
Loss:
Train:
@@ -60,7 +60,7 @@ Optimizer:
verbose: False
last_epoch: -1
regularizer:
- name: 'L2'
+ name: "L2"
coeff: 0.0005
# data loader for train and eval
@@ -82,7 +82,7 @@ DataLoader:
scale: 0.00392157
mean: [0.485, 0.456, 0.406]
std: [0.229, 0.224, 0.225]
- order: ''
+ order: ""
- RandomErasing:
EPSILON: 0.5
sl: 0.02
@@ -98,10 +98,10 @@ DataLoader:
loader:
num_workers: 4
use_shared_memory: True
-
+
Eval:
Query:
- dataset:
+ dataset:
name: VeriWild
image_root: ./dataset/CUB_200_2011/
cls_label_path: ./dataset/CUB_200_2011/test_list.txt
@@ -115,7 +115,7 @@ DataLoader:
scale: 0.00392157
mean: [0.485, 0.456, 0.406]
std: [0.229, 0.224, 0.225]
- order: ''
+ order: ""
sampler:
name: DistributedBatchSampler
batch_size: 64
@@ -126,7 +126,7 @@ DataLoader:
use_shared_memory: True
Gallery:
- dataset:
+ dataset:
name: VeriWild
image_root: ./dataset/CUB_200_2011/
cls_label_path: ./dataset/CUB_200_2011/test_list.txt
@@ -140,7 +140,7 @@ DataLoader:
scale: 1.0/255.0
mean: [0.485, 0.456, 0.406]
std: [0.229, 0.224, 0.225]
- order: ''
+ order: ""
sampler:
name: DistributedBatchSampler
batch_size: 64
@@ -155,4 +155,3 @@ Metric:
- Recallk:
topk: [1, 5]
- mAP: {}
-
diff --git a/ppcls/data/__init__.py b/ppcls/data/__init__.py
index 80cf3bc9af826e935fe0fe6ccf8cad8d6924d370..5f73e7d832a7da4f5733e7c20c1c481a8fb5d09b 100644
--- a/ppcls/data/__init__.py
+++ b/ppcls/data/__init__.py
@@ -72,7 +72,12 @@ def build_dataloader(config, mode, device, use_dali=False, seed=None):
# build dataset
if use_dali:
from ppcls.data.dataloader.dali import dali_dataloader
- return dali_dataloader(config, mode, paddle.device.get_device(), seed)
+ return dali_dataloader(
+ config,
+ mode,
+ paddle.device.get_device(),
+ num_threads=config[mode]['loader']["num_workers"],
+ seed=seed)
class_num = config.get("class_num", None)
config_dataset = config[mode]['dataset']
diff --git a/ppcls/data/dataloader/dali.py b/ppcls/data/dataloader/dali.py
index faef45e26b3dee2e17464a502f42f9886eac6518..a0b91a9a9dd468b0bb4ba0dd314341f067f6e3f5 100644
--- a/ppcls/data/dataloader/dali.py
+++ b/ppcls/data/dataloader/dali.py
@@ -143,7 +143,7 @@ class HybridValPipe(Pipeline):
return self.epoch_size("Reader")
-def dali_dataloader(config, mode, device, seed=None):
+def dali_dataloader(config, mode, device, num_threads=4, seed=None):
assert "gpu" in device, "gpu training is required for DALI"
device_id = int(device.split(':')[1])
config_dataloader = config[mode]
@@ -248,6 +248,7 @@ def dali_dataloader(config, mode, device, seed=None):
device_id,
shard_id,
num_shards,
+ num_threads=num_threads,
seed=seed + shard_id,
pad_output=pad_output,
output_dtype=output_dtype)
@@ -270,6 +271,7 @@ def dali_dataloader(config, mode, device, seed=None):
device_id=device_id,
shard_id=0,
num_shards=1,
+ num_threads=num_threads,
seed=seed,
pad_output=pad_output,
output_dtype=output_dtype)
@@ -298,6 +300,7 @@ def dali_dataloader(config, mode, device, seed=None):
device_id=device_id,
shard_id=shard_id,
num_shards=num_shards,
+ num_threads=num_threads,
pad_output=pad_output,
output_dtype=output_dtype)
else:
@@ -311,6 +314,7 @@ def dali_dataloader(config, mode, device, seed=None):
mean,
std,
device_id=device_id,
+ num_threads=num_threads,
pad_output=pad_output,
output_dtype=output_dtype)
pipe.build()
diff --git a/ppcls/data/dataloader/imagenet_dataset.py b/ppcls/data/dataloader/imagenet_dataset.py
index 0e2b1c8a0b4cbc7e2f251ed2363fca71ec3239e7..51f1fbec3df24164a8c7c41a29a0ad91128120aa 100644
--- a/ppcls/data/dataloader/imagenet_dataset.py
+++ b/ppcls/data/dataloader/imagenet_dataset.py
@@ -21,27 +21,54 @@ from .common_dataset import CommonDataset
class ImageNetDataset(CommonDataset):
- def __init__(
- self,
- image_root,
- cls_label_path,
- transform_ops=None,
- delimiter=None):
+ """ImageNetDataset
+
+ Args:
+ image_root (str): image root, path to `ILSVRC2012`
+ cls_label_path (str): path to annotation file `train_list.txt` or 'val_list.txt`
+ transform_ops (list, optional): list of transform op(s). Defaults to None.
+ delimiter (str, optional): delimiter. Defaults to None.
+ relabel (bool, optional): whether do relabel when original label do not starts from 0 or are discontinuous. Defaults to False.
+ """
+ def __init__(self,
+ image_root,
+ cls_label_path,
+ transform_ops=None,
+ delimiter=None,
+ relabel=False):
self.delimiter = delimiter if delimiter is not None else " "
- super(ImageNetDataset, self).__init__(image_root, cls_label_path, transform_ops)
+ self.relabel = relabel
+ super(ImageNetDataset, self).__init__(image_root, cls_label_path,
+ transform_ops)
def _load_anno(self, seed=None):
- assert os.path.exists(self._cls_path)
- assert os.path.exists(self._img_root)
+ assert os.path.exists(
+ self._cls_path), f"path {self._cls_path} does not exist."
+ assert os.path.exists(
+ self._img_root), f"path {self._img_root} does not exist."
self.images = []
self.labels = []
with open(self._cls_path) as fd:
lines = fd.readlines()
+ if self.relabel:
+ label_set = set()
+ for line in lines:
+ line = line.strip().split(self.delimiter)
+ label_set.add(np.int64(line[1]))
+ label_map = {
+ oldlabel: newlabel
+ for newlabel, oldlabel in enumerate(label_set)
+ }
+
if seed is not None:
np.random.RandomState(seed).shuffle(lines)
- for l in lines:
- l = l.strip().split(self.delimiter)
- self.images.append(os.path.join(self._img_root, l[0]))
- self.labels.append(np.int64(l[1]))
- assert os.path.exists(self.images[-1])
+ for line in lines:
+ line = line.strip().split(self.delimiter)
+ self.images.append(os.path.join(self._img_root, line[0]))
+ if self.relabel:
+ self.labels.append(label_map[np.int64(line[1])])
+ else:
+ self.labels.append(np.int64(line[1]))
+ assert os.path.exists(self.images[
+ -1]), f"path {self.images[-1]} does not exist."
diff --git a/ppcls/data/dataloader/pk_sampler.py b/ppcls/data/dataloader/pk_sampler.py
index 69d1a7c83001e0ea326b30082093fee2f83d3b8a..a4081b5c31f3fe37ae18bd9793cc030e479a77ab 100644
--- a/ppcls/data/dataloader/pk_sampler.py
+++ b/ppcls/data/dataloader/pk_sampler.py
@@ -32,17 +32,23 @@ class PKSampler(DistributedBatchSampler):
batch_size (int): batch size
sample_per_id (int): number of instance(s) within an class
shuffle (bool, optional): _description_. Defaults to True.
+ id_list(list): list of (start_id, end_id, start_id, end_id) for set of ids to duplicated.
+ ratio(list): list of (ratio1, ratio2..) the duplication number for ids in id_list.
drop_last (bool, optional): whether to discard the data at the end. Defaults to True.
sample_method (str, optional): sample method when generating prob_list. Defaults to "sample_avg_prob".
"""
+
def __init__(self,
dataset,
batch_size,
sample_per_id,
shuffle=True,
drop_last=True,
+ id_list=None,
+ ratio=None,
sample_method="sample_avg_prob"):
- super().__init__(dataset, batch_size, shuffle=shuffle, drop_last=drop_last)
+ super().__init__(
+ dataset, batch_size, shuffle=shuffle, drop_last=drop_last)
assert batch_size % sample_per_id == 0, \
f"PKSampler configs error, sample_per_id({sample_per_id}) must be a divisor of batch_size({batch_size})."
assert hasattr(self.dataset,
@@ -67,6 +73,16 @@ class PKSampler(DistributedBatchSampler):
logger.error(
"PKSampler only support id_avg_prob and sample_avg_prob sample method, "
"but receive {}.".format(self.sample_method))
+
+ if id_list and ratio:
+ assert len(id_list) % 2 == 0 and len(id_list) == len(ratio) * 2
+ for i in range(len(self.prob_list)):
+ for j in range(len(ratio)):
+ if i >= id_list[j * 2] and i <= id_list[j * 2 + 1]:
+ self.prob_list[i] = self.prob_list[i] * ratio[j]
+ break
+ self.prob_list = self.prob_list / sum(self.prob_list)
+
diff = np.abs(sum(self.prob_list) - 1)
if diff > 0.00000001:
self.prob_list[-1] = 1 - sum(self.prob_list[:-1])
@@ -74,8 +90,8 @@ class PKSampler(DistributedBatchSampler):
logger.error("PKSampler prob list error")
else:
logger.info(
- "PKSampler: sum of prob list not equal to 1, diff is {}, change the last prob".format(diff)
- )
+ "PKSampler: sum of prob list not equal to 1, diff is {}, change the last prob".
+ format(diff))
def __iter__(self):
label_per_batch = self.batch_size // self.sample_per_label
diff --git a/ppcls/data/dataloader/vehicle_dataset.py b/ppcls/data/dataloader/vehicle_dataset.py
index 2981a57a0516aa25145f39479a34635b3be063f8..e4fbcad6a48d31b02a6fac6063ccb10d4dccdb48 100644
--- a/ppcls/data/dataloader/vehicle_dataset.py
+++ b/ppcls/data/dataloader/vehicle_dataset.py
@@ -89,11 +89,7 @@ class CompCars(Dataset):
class VeriWild(Dataset):
- def __init__(
- self,
- image_root,
- cls_label_path,
- transform_ops=None, ):
+ def __init__(self, image_root, cls_label_path, transform_ops=None):
self._img_root = image_root
self._cls_path = cls_label_path
if transform_ops:
@@ -102,19 +98,23 @@ class VeriWild(Dataset):
self._load_anno()
def _load_anno(self):
- assert os.path.exists(self._cls_path)
- assert os.path.exists(self._img_root)
+ assert os.path.exists(
+ self._cls_path), f"path {self._cls_path} does not exist."
+ assert os.path.exists(
+ self._img_root), f"path {self._img_root} does not exist."
self.images = []
self.labels = []
self.cameras = []
with open(self._cls_path) as fd:
lines = fd.readlines()
- for l in lines:
- l = l.strip().split()
- self.images.append(os.path.join(self._img_root, l[0]))
- self.labels.append(np.int64(l[1]))
- self.cameras.append(np.int64(l[2]))
+ for line in lines:
+ line = line.strip().split()
+ self.images.append(os.path.join(self._img_root, line[0]))
+ self.labels.append(np.int64(line[1]))
+ if len(line) >= 3:
+ self.cameras.append(np.int64(line[2]))
assert os.path.exists(self.images[-1])
+ self.has_camera = len(self.cameras) > 0
def __getitem__(self, idx):
try:
@@ -123,7 +123,10 @@ class VeriWild(Dataset):
if self._transform_ops:
img = transform(img, self._transform_ops)
img = img.transpose((2, 0, 1))
- return (img, self.labels[idx], self.cameras[idx])
+ if self.has_camera:
+ return (img, self.labels[idx], self.cameras[idx])
+ else:
+ return (img, self.labels[idx])
except Exception as ex:
logger.error("Exception occured when parse line: {} with msg: {}".
format(self.images[idx], ex))
diff --git a/ppcls/data/postprocess/__init__.py b/ppcls/data/postprocess/__init__.py
index 7e53832342564233a069444e97a0f25f560076ef..130ff22595c3df78261a8bb79292f5cf763feafd 100644
--- a/ppcls/data/postprocess/__init__.py
+++ b/ppcls/data/postprocess/__init__.py
@@ -21,6 +21,7 @@ from .threshoutput import ThreshOutput, MultiLabelThreshOutput
from .attr_rec import VehicleAttribute, PersonAttribute
+
def build_postprocess(config):
config = copy.deepcopy(config)
model_name = config.pop("name")
diff --git a/ppcls/data/postprocess/attr_rec.py b/ppcls/data/postprocess/attr_rec.py
index a8d492501833ac4ccd83d3aea108e7e34c46cadf..2a3de779e32653d43035b3bd3b3549ba147546ca 100644
--- a/ppcls/data/postprocess/attr_rec.py
+++ b/ppcls/data/postprocess/attr_rec.py
@@ -71,7 +71,6 @@ class VehicleAttribute(object):
return batch_res
-
class PersonAttribute(object):
def __init__(self,
threshold=0.5,
@@ -171,3 +170,58 @@ class PersonAttribute(object):
batch_res.append({"attributes": label_res, "output": pred_res})
return batch_res
+
+class TableAttribute(object):
+ def __init__(
+ self,
+ source_threshold=0.5,
+ number_threshold=0.5,
+ color_threshold=0.5,
+ clarity_threshold=0.5,
+ obstruction_threshold=0.5,
+ angle_threshold=0.5, ):
+ self.source_threshold = source_threshold
+ self.number_threshold = number_threshold
+ self.color_threshold = color_threshold
+ self.clarity_threshold = clarity_threshold
+ self.obstruction_threshold = obstruction_threshold
+ self.angle_threshold = angle_threshold
+
+ def __call__(self, x, file_names=None):
+ if isinstance(x, dict):
+ x = x['logits']
+ assert isinstance(x, paddle.Tensor)
+ if file_names is not None:
+ assert x.shape[0] == len(file_names)
+ x = F.sigmoid(x).numpy()
+
+ # postprocess output of predictor
+ batch_res = []
+ for idx, res in enumerate(x):
+ res = res.tolist()
+ label_res = []
+ source = 'Scanned' if res[0] > self.source_threshold else 'Photo'
+ number = 'Little' if res[1] > self.number_threshold else 'Numerous'
+ color = 'Black-and-White' if res[
+ 2] > self.color_threshold else 'Multicolor'
+ clarity = 'Clear' if res[3] > self.clarity_threshold else 'Blurry'
+ obstruction = 'Without-Obstacles' if res[
+ 4] > self.number_threshold else 'With-Obstacles'
+ angle = 'Horizontal' if res[
+ 5] > self.number_threshold else 'Tilted'
+
+ label_res = [source, number, color, clarity, obstruction, angle]
+
+ threshold_list = [
+ self.source_threshold, self.number_threshold,
+ self.color_threshold, self.clarity_threshold,
+ self.obstruction_threshold, self.angle_threshold
+ ]
+ pred_res = (np.array(res) > np.array(threshold_list)
+ ).astype(np.int8).tolist()
+ batch_res.append({
+ "attributes": label_res,
+ "output": pred_res,
+ "file_name": file_names[idx]
+ })
+ return batch_res
diff --git a/ppcls/data/preprocess/__init__.py b/ppcls/data/preprocess/__init__.py
index d0cfcf2409d2d890adcf03ef0e03b2475625ead8..8f9ea02834d2851116d5b40fd4fc91d59aac3152 100644
--- a/ppcls/data/preprocess/__init__.py
+++ b/ppcls/data/preprocess/__init__.py
@@ -38,9 +38,11 @@ from ppcls.data.preprocess.ops.operators import CropWithPadding
from ppcls.data.preprocess.ops.operators import RandomInterpolationAugment
from ppcls.data.preprocess.ops.operators import ColorJitter
from ppcls.data.preprocess.ops.operators import RandomCropImage
+from ppcls.data.preprocess.ops.operators import RandomRotation
from ppcls.data.preprocess.ops.operators import Padv2
from ppcls.data.preprocess.batch_ops.batch_operators import MixupOperator, CutmixOperator, OpSampler, FmixOperator
+from ppcls.data.preprocess.batch_ops.batch_operators import MixupCutmixHybrid
import numpy as np
from PIL import Image
diff --git a/ppcls/data/preprocess/batch_ops/batch_operators.py b/ppcls/data/preprocess/batch_ops/batch_operators.py
index 6f0abb864574f8bead1d1ad6461460b4ececc7a7..c9563e2294d43df8183e2cd32894c96da1f48f8e 100644
--- a/ppcls/data/preprocess/batch_ops/batch_operators.py
+++ b/ppcls/data/preprocess/batch_ops/batch_operators.py
@@ -23,6 +23,9 @@ import numpy as np
from ppcls.utils import logger
from ppcls.data.preprocess.ops.fmix import sample_mask
+import paddle
+import paddle.nn.functional as F
+
class BatchOperator(object):
""" BatchOperator """
@@ -229,3 +232,270 @@ class OpSampler(object):
list(self.ops.keys()), weights=list(self.ops.values()), k=1)[0]
# return batch directly when None Op
return op(batch) if op else batch
+
+
+class MixupCutmixHybrid(object):
+ """ Mixup/Cutmix that applies different params to each element or whole batch
+
+ Args:
+ mixup_alpha (float): mixup alpha value, mixup is active if > 0.
+ cutmix_alpha (float): cutmix alpha value, cutmix is active if > 0.
+ cutmix_minmax (List[float]): cutmix min/max image ratio, cutmix is active and uses this vs alpha if not None.
+ prob (float): probability of applying mixup or cutmix per batch or element
+ switch_prob (float): probability of switching to cutmix instead of mixup when both are active
+ mode (str): how to apply mixup/cutmix params (per 'batch', 'pair' (pair of elements), 'elem' (element)
+ correct_lam (bool): apply lambda correction when cutmix bbox clipped by image borders
+ label_smoothing (float): apply label smoothing to the mixed target tensor
+ num_classes (int): number of classes for target
+ """
+
+ def __init__(self,
+ mixup_alpha=1.,
+ cutmix_alpha=0.,
+ cutmix_minmax=None,
+ prob=1.0,
+ switch_prob=0.5,
+ mode='batch',
+ correct_lam=True,
+ label_smoothing=0.1,
+ num_classes=4):
+ self.mixup_alpha = mixup_alpha
+ self.cutmix_alpha = cutmix_alpha
+ self.cutmix_minmax = cutmix_minmax
+ if self.cutmix_minmax is not None:
+ assert len(self.cutmix_minmax) == 2
+ # force cutmix alpha == 1.0 when minmax active to keep logic simple & safe
+ self.cutmix_alpha = 1.0
+ self.mix_prob = prob
+ self.switch_prob = switch_prob
+ self.label_smoothing = label_smoothing
+ self.num_classes = num_classes
+ self.mode = mode
+ self.correct_lam = correct_lam # correct lambda based on clipped area for cutmix
+ self.mixup_enabled = True # set to false to disable mixing (intended tp be set by train loop)
+
+ def _one_hot(self, x, num_classes, on_value=1., off_value=0.):
+ x = paddle.cast(x, dtype='int64')
+ on_value = paddle.full([x.shape[0], num_classes], on_value)
+ off_value = paddle.full([x.shape[0], num_classes], off_value)
+ return paddle.where(
+ F.one_hot(x, num_classes) == 1, on_value, off_value)
+
+ def _mixup_target(self, target, num_classes, lam=1., smoothing=0.0):
+ off_value = smoothing / num_classes
+ on_value = 1. - smoothing + off_value
+ y1 = self._one_hot(
+ target,
+ num_classes,
+ on_value=on_value,
+ off_value=off_value, )
+ y2 = self._one_hot(
+ target.flip(0),
+ num_classes,
+ on_value=on_value,
+ off_value=off_value)
+ return y1 * lam + y2 * (1. - lam)
+
+ def _rand_bbox(self, img_shape, lam, margin=0., count=None):
+ """ Standard CutMix bounding-box
+ Generates a random square bbox based on lambda value. This impl includes
+ support for enforcing a border margin as percent of bbox dimensions.
+
+ Args:
+ img_shape (tuple): Image shape as tuple
+ lam (float): Cutmix lambda value
+ margin (float): Percentage of bbox dimension to enforce as margin (reduce amount of box outside image)
+ count (int): Number of bbox to generate
+ """
+ ratio = np.sqrt(1 - lam)
+ img_h, img_w = img_shape[-2:]
+ cut_h, cut_w = int(img_h * ratio), int(img_w * ratio)
+ margin_y, margin_x = int(margin * cut_h), int(margin * cut_w)
+ cy = np.random.randint(0 + margin_y, img_h - margin_y, size=count)
+ cx = np.random.randint(0 + margin_x, img_w - margin_x, size=count)
+ yl = np.clip(cy - cut_h // 2, 0, img_h)
+ yh = np.clip(cy + cut_h // 2, 0, img_h)
+ xl = np.clip(cx - cut_w // 2, 0, img_w)
+ xh = np.clip(cx + cut_w // 2, 0, img_w)
+ return yl, yh, xl, xh
+
+ def _rand_bbox_minmax(self, img_shape, minmax, count=None):
+ """ Min-Max CutMix bounding-box
+ Inspired by Darknet cutmix impl, generates a random rectangular bbox
+ based on min/max percent values applied to each dimension of the input image.
+
+ Typical defaults for minmax are usually in the .2-.3 for min and .8-.9 range for max.
+
+ Args:
+ img_shape (tuple): Image shape as tuple
+ minmax (tuple or list): Min and max bbox ratios (as percent of image size)
+ count (int): Number of bbox to generate
+ """
+ assert len(minmax) == 2
+ img_h, img_w = img_shape[-2:]
+ cut_h = np.random.randint(
+ int(img_h * minmax[0]), int(img_h * minmax[1]), size=count)
+ cut_w = np.random.randint(
+ int(img_w * minmax[0]), int(img_w * minmax[1]), size=count)
+ yl = np.random.randint(0, img_h - cut_h, size=count)
+ xl = np.random.randint(0, img_w - cut_w, size=count)
+ yu = yl + cut_h
+ xu = xl + cut_w
+ return yl, yu, xl, xu
+
+ def _cutmix_bbox_and_lam(self,
+ img_shape,
+ lam,
+ ratio_minmax=None,
+ correct_lam=True,
+ count=None):
+ """ Generate bbox and apply lambda correction.
+ """
+ if ratio_minmax is not None:
+ yl, yu, xl, xu = self._rand_bbox_minmax(
+ img_shape, ratio_minmax, count=count)
+ else:
+ yl, yu, xl, xu = self._rand_bbox(img_shape, lam, count=count)
+ if correct_lam or ratio_minmax is not None:
+ bbox_area = (yu - yl) * (xu - xl)
+ lam = 1. - bbox_area / float(img_shape[-2] * img_shape[-1])
+ return (yl, yu, xl, xu), lam
+
+ def _params_per_elem(self, batch_size):
+ lam = np.ones(batch_size, dtype=np.float32)
+ use_cutmix = np.zeros(batch_size, dtype=np.bool)
+ if self.mixup_enabled:
+ if self.mixup_alpha > 0. and self.cutmix_alpha > 0.:
+ use_cutmix = np.random.rand(batch_size) < self.switch_prob
+ lam_mix = np.where(
+ use_cutmix,
+ np.random.beta(
+ self.cutmix_alpha, self.cutmix_alpha, size=batch_size),
+ np.random.beta(
+ self.mixup_alpha, self.mixup_alpha, size=batch_size))
+ elif self.mixup_alpha > 0.:
+ lam_mix = np.random.beta(
+ self.mixup_alpha, self.mixup_alpha, size=batch_size)
+ elif self.cutmix_alpha > 0.:
+ use_cutmix = np.ones(batch_size, dtype=np.bool)
+ lam_mix = np.random.beta(
+ self.cutmix_alpha, self.cutmix_alpha, size=batch_size)
+ else:
+ assert False, "One of mixup_alpha > 0., cutmix_alpha > 0., cutmix_minmax not None should be true."
+ lam = np.where(
+ np.random.rand(batch_size) < self.mix_prob,
+ lam_mix.astype(np.float32), lam)
+ return lam, use_cutmix
+
+ def _params_per_batch(self):
+ lam = 1.
+ use_cutmix = False
+ if self.mixup_enabled and np.random.rand() < self.mix_prob:
+ if self.mixup_alpha > 0. and self.cutmix_alpha > 0.:
+ use_cutmix = np.random.rand() < self.switch_prob
+ lam_mix = np.random.beta(self.cutmix_alpha, self.cutmix_alpha) if use_cutmix else \
+ np.random.beta(self.mixup_alpha, self.mixup_alpha)
+ elif self.mixup_alpha > 0.:
+ lam_mix = np.random.beta(self.mixup_alpha, self.mixup_alpha)
+ elif self.cutmix_alpha > 0.:
+ use_cutmix = True
+ lam_mix = np.random.beta(self.cutmix_alpha, self.cutmix_alpha)
+ else:
+ assert False, "One of mixup_alpha > 0., cutmix_alpha > 0., cutmix_minmax not None should be true."
+ lam = float(lam_mix)
+ return lam, use_cutmix
+
+ def _mix_elem(self, x):
+ batch_size = len(x)
+ lam_batch, use_cutmix = self._params_per_elem(batch_size)
+ x_orig = x.clone(
+ ) # need to keep an unmodified original for mixing source
+ for i in range(batch_size):
+ j = batch_size - i - 1
+ lam = lam_batch[i]
+ if lam != 1.:
+ if use_cutmix[i]:
+ (yl, yh, xl, xh), lam = self._cutmix_bbox_and_lam(
+ x[i].shape,
+ lam,
+ ratio_minmax=self.cutmix_minmax,
+ correct_lam=self.correct_lam)
+ if yl < yh and xl < xh:
+ x[i][:, yl:yh, xl:xh] = x_orig[j][:, yl:yh, xl:xh]
+ lam_batch[i] = lam
+ else:
+ x[i] = x[i] * lam + x_orig[j] * (1 - lam)
+ return paddle.to_tensor(lam_batch, dtype=x.dtype).unsqueeze(1)
+
+ def _mix_pair(self, x):
+ batch_size = len(x)
+ lam_batch, use_cutmix = self._params_per_elem(batch_size // 2)
+ x_orig = x.clone(
+ ) # need to keep an unmodified original for mixing source
+ for i in range(batch_size // 2):
+ j = batch_size - i - 1
+ lam = lam_batch[i]
+ if lam != 1.:
+ if use_cutmix[i]:
+ (yl, yh, xl, xh), lam = self._cutmix_bbox_and_lam(
+ x[i].shape,
+ lam,
+ ratio_minmax=self.cutmix_minmax,
+ correct_lam=self.correct_lam)
+ if yl < yh and xl < xh:
+ x[i][:, yl:yh, xl:xh] = x_orig[j][:, yl:yh, xl:xh]
+ x[j][:, yl:yh, xl:xh] = x_orig[i][:, yl:yh, xl:xh]
+ lam_batch[i] = lam
+ else:
+ x[i] = x[i] * lam + x_orig[j] * (1 - lam)
+ x[j] = x[j] * lam + x_orig[i] * (1 - lam)
+ lam_batch = np.concatenate((lam_batch, lam_batch[::-1]))
+ return paddle.to_tensor(lam_batch, dtype=x.dtype).unsqueeze(1)
+
+ def _mix_batch(self, x):
+ lam, use_cutmix = self._params_per_batch()
+ if lam == 1.:
+ return 1.
+ if use_cutmix:
+ (yl, yh, xl, xh), lam = self._cutmix_bbox_and_lam(
+ x.shape,
+ lam,
+ ratio_minmax=self.cutmix_minmax,
+ correct_lam=self.correct_lam)
+ if yl < yh and xl < xh:
+ x[:, :, yl:yh, xl:xh] = x.flip(0)[:, :, yl:yh, xl:xh]
+
+ else:
+ x_flipped = x.flip(0) * (1. - lam)
+ x[:] = x * lam + x_flipped
+ return lam
+
+ def _unpack(self, batch):
+ """ _unpack """
+ assert isinstance(batch, list), \
+ 'batch should be a list filled with tuples (img, label)'
+ bs = len(batch)
+ assert bs > 0, 'size of the batch data should > 0'
+ #imgs, labels = list(zip(*batch))
+ imgs = []
+ labels = []
+ for item in batch:
+ imgs.append(item[0])
+ labels.append(item[1])
+ return np.array(imgs), np.array(labels), bs
+
+ def __call__(self, batch):
+ x, target, bs = self._unpack(batch)
+ x = paddle.to_tensor(x)
+ target = paddle.to_tensor(target)
+ assert len(x) % 2 == 0, 'Batch size should be even when using this'
+ if self.mode == 'elem':
+ lam = self._mix_elem(x)
+ elif self.mode == 'pair':
+ lam = self._mix_pair(x)
+ else:
+ lam = self._mix_batch(x)
+ target = self._mixup_target(target, self.num_classes, lam,
+ self.label_smoothing)
+
+ return list(zip(x.numpy(), target.numpy()))
diff --git a/ppcls/data/preprocess/ops/operators.py b/ppcls/data/preprocess/ops/operators.py
index e617b8a71afffeb9e18e4be412f5a3374bd387ec..c70b9cb723dce77755189aad16f3839046673f35 100644
--- a/ppcls/data/preprocess/ops/operators.py
+++ b/ppcls/data/preprocess/ops/operators.py
@@ -26,6 +26,7 @@ import cv2
import numpy as np
from PIL import Image, ImageOps, __version__ as PILLOW_VERSION
from paddle.vision.transforms import ColorJitter as RawColorJitter
+from paddle.vision.transforms import RandomRotation as RawRandomRotation
from paddle.vision.transforms import ToTensor, Normalize, RandomHorizontalFlip, RandomResizedCrop
from paddle.vision.transforms import functional as F
from .autoaugment import ImageNetPolicy
@@ -181,7 +182,8 @@ class DecodeImage(object):
img = np.asarray(img)[:, :, ::-1] # BRG
if self.to_rgb:
- assert img.shape[2] == 3, f"invalid shape of image[{img.shape}]"
+ assert img.shape[
+ 2] == 3, f"invalid shape of image[{img.shape}]"
img = img[:, :, ::-1]
if self.channel_first:
@@ -495,7 +497,13 @@ class RandFlipImage(object):
if isinstance(img, np.ndarray):
return cv2.flip(img, self.flip_code)
else:
- return img.transpose(Image.FLIP_LEFT_RIGHT)
+ if self.flip_code == 1:
+ return img.transpose(Image.FLIP_LEFT_RIGHT)
+ elif self.flip_code == 0:
+ return img.transpose(Image.FLIP_TOP_BOTTOM)
+ else:
+ return img.transpose(Image.FLIP_LEFT_RIGHT).transpose(
+ Image.FLIP_LEFT_RIGHT)
else:
return img
@@ -653,17 +661,38 @@ class ColorJitter(RawColorJitter):
return img
+class RandomRotation(RawRandomRotation):
+ """RandomRotation.
+ """
+
+ def __init__(self, prob=0.5, *args, **kwargs):
+ super().__init__(*args, **kwargs)
+ self.prob = prob
+
+ def __call__(self, img):
+ if np.random.random() < self.prob:
+ img = super()._apply_image(img)
+ return img
+
+
class Pad(object):
"""
Pads the given PIL.Image on all sides with specified padding mode and fill value.
adapted from: https://pytorch.org/vision/stable/_modules/torchvision/transforms/transforms.html#Pad
"""
- def __init__(self, padding: int, fill: int=0,
- padding_mode: str="constant"):
+ def __init__(self,
+ padding: int,
+ fill: int=0,
+ padding_mode: str="constant",
+ backend: str="pil"):
self.padding = padding
self.fill = fill
self.padding_mode = padding_mode
+ self.backend = backend
+ assert backend in [
+ "pil", "cv2"
+ ], f"backend must in ['pil', 'cv2'], but got {backend}"
def _parse_fill(self, fill, img, min_pil_version, name="fillcolor"):
# Process fill color for affine transforms
@@ -698,11 +727,21 @@ class Pad(object):
return {name: fill}
def __call__(self, img):
- opts = self._parse_fill(self.fill, img, "2.3.0", name="fill")
- if img.mode == "P":
- palette = img.getpalette()
- img = ImageOps.expand(img, border=self.padding, **opts)
- img.putpalette(palette)
+ if self.backend == "pil":
+ opts = self._parse_fill(self.fill, img, "2.3.0", name="fill")
+ if img.mode == "P":
+ palette = img.getpalette()
+ img = ImageOps.expand(img, border=self.padding, **opts)
+ img.putpalette(palette)
+ return img
+ return ImageOps.expand(img, border=self.padding, **opts)
+ else:
+ img = cv2.copyMakeBorder(
+ img,
+ self.padding,
+ self.padding,
+ self.padding,
+ self.padding,
+ cv2.BORDER_CONSTANT,
+ value=(self.fill, self.fill, self.fill))
return img
-
- return ImageOps.expand(img, border=self.padding, **opts)
diff --git a/ppcls/engine/engine.py b/ppcls/engine/engine.py
index 50683bac357d96f2080886425dcae7ae0001ce33..e6977110afe35418d77467f2d3e383532977221c 100644
--- a/ppcls/engine/engine.py
+++ b/ppcls/engine/engine.py
@@ -114,6 +114,7 @@ class Engine(object):
#TODO(gaotingquan): support rec
class_num = config["Arch"].get("class_num", None)
self.config["DataLoader"].update({"class_num": class_num})
+
# build dataloader
if self.mode == 'train':
self.train_dataloader = build_dataloader(
diff --git a/ppcls/engine/evaluation/retrieval.py b/ppcls/engine/evaluation/retrieval.py
index 02cae1670bbe1255a84fcf80c3097c5c020c917f..ef4bbd24c9d4db6440b3ca0d6e8b948bef4b53f7 100644
--- a/ppcls/engine/evaluation/retrieval.py
+++ b/ppcls/engine/evaluation/retrieval.py
@@ -25,32 +25,35 @@ from ppcls.utils import logger
def retrieval_eval(engine, epoch_id=0):
engine.model.eval()
- # step1. build gallery
+ # step1. build query & gallery
if engine.gallery_query_dataloader is not None:
gallery_feas, gallery_img_id, gallery_unique_id = cal_feature(
engine, name='gallery_query')
- query_feas, query_img_id, query_query_id = gallery_feas, gallery_img_id, gallery_unique_id
+ query_feas, query_img_id, query_unique_id = gallery_feas, gallery_img_id, gallery_unique_id
else:
gallery_feas, gallery_img_id, gallery_unique_id = cal_feature(
engine, name='gallery')
- query_feas, query_img_id, query_query_id = cal_feature(
+ query_feas, query_img_id, query_unique_id = cal_feature(
engine, name='query')
- # step2. do evaluation
+ # step2. split data into blocks so as to save memory
sim_block_size = engine.config["Global"].get("sim_block_size", 64)
sections = [sim_block_size] * (len(query_feas) // sim_block_size)
if len(query_feas) % sim_block_size:
sections.append(len(query_feas) % sim_block_size)
+
fea_blocks = paddle.split(query_feas, num_or_sections=sections)
- if query_query_id is not None:
- query_id_blocks = paddle.split(
- query_query_id, num_or_sections=sections)
- image_id_blocks = paddle.split(query_img_id, num_or_sections=sections)
+ if query_unique_id is not None:
+ query_unique_id_blocks = paddle.split(
+ query_unique_id, num_or_sections=sections)
+ query_img_id_blocks = paddle.split(query_img_id, num_or_sections=sections)
metric_key = None
+ # step3. do evaluation
if engine.eval_loss_func is None:
metric_dict = {metric_key: 0.}
else:
+ # do evaluation with re-ranking(k-reciprocal)
reranking_flag = engine.config['Global'].get('re_ranking', False)
logger.info(f"re_ranking={reranking_flag}")
metric_dict = dict()
@@ -70,9 +73,9 @@ def retrieval_eval(engine, epoch_id=0):
query_feas, gallery_feas, k1=20, k2=6, lambda_value=0.3)
# compute keep mask
- query_id_mask = (query_query_id != gallery_unique_id.t())
+ unique_id_mask = (query_unique_id != gallery_unique_id.t())
image_id_mask = (query_img_id != gallery_img_id.t())
- keep_mask = paddle.logical_or(query_id_mask, image_id_mask)
+ keep_mask = paddle.logical_or(image_id_mask, unique_id_mask)
# set inf(1e9) distance to those exist in gallery
distmat = distmat * keep_mask.astype("float32")
@@ -85,24 +88,27 @@ def retrieval_eval(engine, epoch_id=0):
for key in metric_tmp:
metric_dict[key] = metric_tmp[key]
else:
+ # do evaluation without re-ranking
for block_idx, block_fea in enumerate(fea_blocks):
similarity_matrix = paddle.matmul(
block_fea, gallery_feas, transpose_y=True) # [n,m]
- if query_query_id is not None:
- query_id_block = query_id_blocks[block_idx]
- query_id_mask = (query_id_block != gallery_unique_id.t())
+ if query_unique_id is not None:
+ query_unique_id_block = query_unique_id_blocks[block_idx]
+ unique_id_mask = (
+ query_unique_id_block != gallery_unique_id.t())
- image_id_block = image_id_blocks[block_idx]
- image_id_mask = (image_id_block != gallery_img_id.t())
+ query_img_id_block = query_img_id_blocks[block_idx]
+ image_id_mask = (query_img_id_block != gallery_img_id.t())
- keep_mask = paddle.logical_or(query_id_mask, image_id_mask)
+ keep_mask = paddle.logical_or(image_id_mask,
+ unique_id_mask)
similarity_matrix = similarity_matrix * keep_mask.astype(
"float32")
else:
keep_mask = None
metric_tmp = engine.eval_metric_func(
- similarity_matrix, image_id_blocks[block_idx],
+ similarity_matrix, query_img_id_blocks[block_idx],
gallery_img_id, keep_mask)
for key in metric_tmp:
diff --git a/ppcls/loss/__init__.py b/ppcls/loss/__init__.py
index 489aea7fb1ee4b4a8ff9388f3984e64965a1eac7..019eff71949972d1f72e02948fcce65ff96298ac 100644
--- a/ppcls/loss/__init__.py
+++ b/ppcls/loss/__init__.py
@@ -12,10 +12,12 @@ from .msmloss import MSMLoss
from .npairsloss import NpairsLoss
from .trihardloss import TriHardLoss
from .triplet import TripletLoss, TripletLossV2
+from .tripletangularmarginloss import TripletAngularMarginLoss
from .supconloss import SupConLoss
from .pairwisecosface import PairwiseCosface
from .dmlloss import DMLLoss
from .distanceloss import DistanceLoss
+from .softtargetceloss import SoftTargetCrossEntropy
from .distillationloss import DistillationCELoss
from .distillationloss import DistillationGTCELoss
@@ -24,6 +26,7 @@ from .distillationloss import DistillationDistanceLoss
from .distillationloss import DistillationRKDLoss
from .distillationloss import DistillationKLDivLoss
from .distillationloss import DistillationDKDLoss
+from .distillationloss import DistillationWSLLoss
from .distillationloss import DistillationMultiLabelLoss
from .distillationloss import DistillationDISTLoss
from .distillationloss import DistillationPairLoss
diff --git a/ppcls/loss/distillationloss.py b/ppcls/loss/distillationloss.py
index 5a924afe72fd7c6627b9f3b3be8ce3553932b535..af24f83015928984ccb30ae864adf73928657e5e 100644
--- a/ppcls/loss/distillationloss.py
+++ b/ppcls/loss/distillationloss.py
@@ -22,6 +22,7 @@ from .distanceloss import DistanceLoss
from .rkdloss import RKdAngle, RkdDistance
from .kldivloss import KLDivLoss
from .dkdloss import DKDLoss
+from .wslloss import WSLLoss
from .dist_loss import DISTLoss
from .multilabelloss import MultiLabelLoss
from .mgd_loss import MGDLoss
@@ -262,6 +263,34 @@ class DistillationDKDLoss(DKDLoss):
return loss_dict
+class DistillationWSLLoss(WSLLoss):
+ """
+ DistillationWSLLoss
+ """
+
+ def __init__(self,
+ model_name_pairs=[],
+ key=None,
+ temperature=2.0,
+ name="wsl_loss"):
+ super().__init__(temperature)
+ self.model_name_pairs = model_name_pairs
+ self.key = key
+ self.name = name
+
+ def forward(self, predicts, batch):
+ loss_dict = dict()
+ for idx, pair in enumerate(self.model_name_pairs):
+ out1 = predicts[pair[0]]
+ out2 = predicts[pair[1]]
+ if self.key is not None:
+ out1 = out1[self.key]
+ out2 = out2[self.key]
+ loss = super().forward(out1, out2, batch)
+ loss_dict[f"{self.name}_{pair[0]}_{pair[1]}"] = loss
+ return loss_dict
+
+
class DistillationMultiLabelLoss(MultiLabelLoss):
"""
DistillationMultiLabelLoss
diff --git a/ppcls/loss/softtargetceloss.py b/ppcls/loss/softtargetceloss.py
new file mode 100644
index 0000000000000000000000000000000000000000..351db50e3434d333814a9ea49f1b0463b80432a7
--- /dev/null
+++ b/ppcls/loss/softtargetceloss.py
@@ -0,0 +1,16 @@
+import paddle
+import paddle.nn as nn
+import paddle.nn.functional as F
+
+
+class SoftTargetCrossEntropy(nn.Layer):
+ def __init__(self):
+ super().__init__()
+
+ def forward(self, x, target):
+ loss = paddle.sum(-target * F.log_softmax(x, axis=-1), axis=-1)
+ loss = loss.mean()
+ return {"SoftTargetCELoss": loss}
+
+ def __str__(self, ):
+ return type(self).__name__
diff --git a/ppcls/loss/tripletangularmarginloss.py b/ppcls/loss/tripletangularmarginloss.py
new file mode 100644
index 0000000000000000000000000000000000000000..3a91d2d499fa22aadc7ca15322f4048b978fb19d
--- /dev/null
+++ b/ppcls/loss/tripletangularmarginloss.py
@@ -0,0 +1,115 @@
+# Copyright (c) 2018 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from __future__ import absolute_import
+from __future__ import division
+from __future__ import print_function
+
+import paddle
+import paddle.nn as nn
+
+
+class TripletAngularMarginLoss(nn.Layer):
+ """A more robust triplet loss with hard positive/negative mining on angular margin instead of relative distance between d(a,p) and d(a,n).
+
+ Args:
+ margin (float, optional): angular margin. Defaults to 0.5.
+ normalize_feature (bool, optional): whether to apply L2-norm in feature before computing distance(cos-similarity). Defaults to True.
+ reduction (str, optional): reducing option within an batch . Defaults to "mean".
+ add_absolute (bool, optional): whether add absolute loss within d(a,p) or d(a,n). Defaults to False.
+ absolute_loss_weight (float, optional): weight for absolute loss. Defaults to 1.0.
+ ap_value (float, optional): weight for d(a, p). Defaults to 0.9.
+ an_value (float, optional): weight for d(a, n). Defaults to 0.5.
+ feature_from (str, optional): which key feature from. Defaults to "features".
+ """
+
+ def __init__(self,
+ margin=0.5,
+ normalize_feature=True,
+ reduction="mean",
+ add_absolute=False,
+ absolute_loss_weight=1.0,
+ ap_value=0.9,
+ an_value=0.5,
+ feature_from="features"):
+ super(TripletAngularMarginLoss, self).__init__()
+ self.margin = margin
+ self.feature_from = feature_from
+ self.ranking_loss = paddle.nn.loss.MarginRankingLoss(
+ margin=margin, reduction=reduction)
+ self.normalize_feature = normalize_feature
+ self.add_absolute = add_absolute
+ self.ap_value = ap_value
+ self.an_value = an_value
+ self.absolute_loss_weight = absolute_loss_weight
+
+ def forward(self, input, target):
+ """
+ Args:
+ inputs: feature matrix with shape (batch_size, feat_dim)
+ target: ground truth labels with shape (num_classes)
+ """
+ inputs = input[self.feature_from]
+
+ if self.normalize_feature:
+ inputs = paddle.divide(
+ inputs, paddle.norm(
+ inputs, p=2, axis=-1, keepdim=True))
+
+ bs = inputs.shape[0]
+
+ # compute distance(cos-similarity)
+ dist = paddle.matmul(inputs, inputs.t())
+
+ # hard negative mining
+ is_pos = paddle.expand(target, (
+ bs, bs)).equal(paddle.expand(target, (bs, bs)).t())
+ is_neg = paddle.expand(target, (
+ bs, bs)).not_equal(paddle.expand(target, (bs, bs)).t())
+
+ # `dist_ap` means distance(anchor, positive)
+ # both `dist_ap` and `relative_p_inds` with shape [N, 1]
+ dist_ap = paddle.min(paddle.reshape(
+ paddle.masked_select(dist, is_pos), (bs, -1)),
+ axis=1,
+ keepdim=True)
+ # `dist_an` means distance(anchor, negative)
+ # both `dist_an` and `relative_n_inds` with shape [N, 1]
+ dist_an = paddle.max(paddle.reshape(
+ paddle.masked_select(dist, is_neg), (bs, -1)),
+ axis=1,
+ keepdim=True)
+ # shape [N]
+ dist_ap = paddle.squeeze(dist_ap, axis=1)
+ dist_an = paddle.squeeze(dist_an, axis=1)
+
+ # Compute ranking hinge loss
+ y = paddle.ones_like(dist_an)
+ loss = self.ranking_loss(dist_ap, dist_an, y)
+
+ if self.add_absolute:
+ absolut_loss_ap = self.ap_value - dist_ap
+ absolut_loss_ap = paddle.where(absolut_loss_ap > 0,
+ absolut_loss_ap,
+ paddle.zeros_like(absolut_loss_ap))
+
+ absolut_loss_an = dist_an - self.an_value
+ absolut_loss_an = paddle.where(absolut_loss_an > 0,
+ absolut_loss_an,
+ paddle.ones_like(absolut_loss_an))
+
+ loss = (absolut_loss_an.mean() + absolut_loss_ap.mean()
+ ) * self.absolute_loss_weight + loss.mean()
+
+ return {"TripletAngularMarginLoss": loss}
diff --git a/ppcls/loss/wslloss.py b/ppcls/loss/wslloss.py
new file mode 100644
index 0000000000000000000000000000000000000000..8bdfaf8ccdacb475f209497bd96859b4c16dd760
--- /dev/null
+++ b/ppcls/loss/wslloss.py
@@ -0,0 +1,66 @@
+# copyright (c) 2022 PaddlePaddle Authors. All Rights Reserve.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import paddle
+import paddle.nn as nn
+import paddle.nn.functional as F
+
+
+class WSLLoss(nn.Layer):
+ """
+ Weighted Soft Labels Loss
+ paper: https://arxiv.org/pdf/2102.00650.pdf
+ code reference: https://github.com/bellymonster/Weighted-Soft-Label-Distillation
+ """
+
+ def __init__(self, temperature=2.0, use_target_as_gt=False):
+ super().__init__()
+ self.temperature = temperature
+ self.use_target_as_gt = use_target_as_gt
+
+ def forward(self, logits_student, logits_teacher, target=None):
+ """Compute weighted soft labels loss.
+ Args:
+ logits_student: student's logits with shape (batch_size, num_classes)
+ logits_teacher: teacher's logits with shape (batch_size, num_classes)
+ target: ground truth labels with shape (batch_size)
+ """
+ if target is None or self.use_target_as_gt:
+ target = logits_teacher.argmax(axis=-1)
+
+ target = F.one_hot(
+ target.reshape([-1]), num_classes=logits_student[0].shape[0])
+
+ s_input_for_softmax = logits_student / self.temperature
+ t_input_for_softmax = logits_teacher / self.temperature
+
+ ce_loss_s = -paddle.sum(target *
+ F.log_softmax(logits_student.detach()),
+ axis=1)
+ ce_loss_t = -paddle.sum(target *
+ F.log_softmax(logits_teacher.detach()),
+ axis=1)
+
+ ratio = ce_loss_s / (ce_loss_t + 1e-7)
+ ratio = paddle.maximum(ratio, paddle.zeros_like(ratio))
+
+ kd_loss = -paddle.sum(F.softmax(t_input_for_softmax) *
+ F.log_softmax(s_input_for_softmax),
+ axis=1)
+ weight = 1 - paddle.exp(-ratio)
+
+ weighted_kd_loss = (self.temperature**2) * paddle.mean(kd_loss *
+ weight)
+
+ return weighted_kd_loss
diff --git a/ppcls/metric/metrics.py b/ppcls/metric/metrics.py
index 0c803ccfdbb29216381625ea3df4a4540c7b56c0..b6dc934f31c04b0df2a90e63fed48973dddff1ca 100644
--- a/ppcls/metric/metrics.py
+++ b/ppcls/metric/metrics.py
@@ -12,6 +12,7 @@
# See the License for the specific language governing permissions and
# limitations under the License.
+from cmath import nan
import numpy as np
import paddle
import paddle.nn as nn
@@ -97,6 +98,11 @@ class mAP(nn.Layer):
num_rel = paddle.greater_than(num_rel, paddle.to_tensor(0.))
num_rel_index = paddle.nonzero(num_rel.astype("int"))
num_rel_index = paddle.reshape(num_rel_index, [num_rel_index.shape[0]])
+
+ if paddle.numel(num_rel_index).item() == 0:
+ metric_dict["mAP"] = np.nan
+ return metric_dict
+
equal_flag = paddle.index_select(equal_flag, num_rel_index, axis=0)
acc_sum = paddle.cumsum(equal_flag, axis=1)
diff --git a/ppcls/optimizer/optimizer.py b/ppcls/optimizer/optimizer.py
index c0403cf95cdaf442b6fdaeea54d21a2382e3858b..c446fc1dec9e01fe85dfac68dee829082655826a 100644
--- a/ppcls/optimizer/optimizer.py
+++ b/ppcls/optimizer/optimizer.py
@@ -272,3 +272,145 @@ class AdamW(object):
def _apply_decay_param_fun(self, name):
return name not in self.no_weight_decay_param_name_list
+
+
+class AdamWDL(object):
+ """
+ The AdamWDL optimizer is implemented based on the AdamW Optimization with dynamic lr setting.
+ Generally it's used for transformer model.
+ """
+
+ def __init__(self,
+ learning_rate=0.001,
+ beta1=0.9,
+ beta2=0.999,
+ epsilon=1e-8,
+ weight_decay=None,
+ multi_precision=False,
+ grad_clip=None,
+ layerwise_decay=None,
+ filter_bias_and_bn=True,
+ **args):
+ self.learning_rate = learning_rate
+ self.beta1 = beta1
+ self.beta2 = beta2
+ self.epsilon = epsilon
+ self.grad_clip = grad_clip
+ self.weight_decay = weight_decay
+ self.multi_precision = multi_precision
+ self.layerwise_decay = layerwise_decay
+ self.filter_bias_and_bn = filter_bias_and_bn
+
+ class AdamWDLImpl(optim.AdamW):
+ def __init__(self,
+ learning_rate=0.001,
+ beta1=0.9,
+ beta2=0.999,
+ epsilon=1e-8,
+ parameters=None,
+ weight_decay=0.01,
+ apply_decay_param_fun=None,
+ grad_clip=None,
+ lazy_mode=False,
+ multi_precision=False,
+ layerwise_decay=1.0,
+ n_layers=12,
+ name_dict=None,
+ name=None):
+ if not isinstance(layerwise_decay, float) and \
+ not isinstance(layerwise_decay, fluid.framework.Variable):
+ raise TypeError("coeff should be float or Tensor.")
+ self.layerwise_decay = layerwise_decay
+ self.name_dict = name_dict
+ self.n_layers = n_layers
+ self.set_param_lr_fun = self._layerwise_lr_decay
+ super().__init__(
+ learning_rate=learning_rate,
+ parameters=parameters,
+ beta1=beta1,
+ beta2=beta2,
+ epsilon=epsilon,
+ grad_clip=grad_clip,
+ name=name,
+ apply_decay_param_fun=apply_decay_param_fun,
+ weight_decay=weight_decay,
+ lazy_mode=lazy_mode,
+ multi_precision=multi_precision)
+
+ def _append_optimize_op(self, block, param_and_grad):
+ if self.set_param_lr_fun is None:
+ return super(AdamLW, self)._append_optimize_op(block,
+ param_and_grad)
+
+ self._append_decoupled_weight_decay(block, param_and_grad)
+ prev_lr = param_and_grad[0].optimize_attr["learning_rate"]
+ self.set_param_lr_fun(self.layerwise_decay, self.name_dict,
+ self.n_layers, param_and_grad[0])
+ # excute Adam op
+ res = super(optim.AdamW, self)._append_optimize_op(block,
+ param_and_grad)
+ param_and_grad[0].optimize_attr["learning_rate"] = prev_lr
+ return res
+
+ # Layerwise decay
+ def _layerwise_lr_decay(self, decay_rate, name_dict, n_layers, param):
+ """
+ Args:
+ decay_rate (float):
+ The layer-wise decay ratio.
+ name_dict (dict):
+ The keys of name_dict is dynamic name of model while the value
+ of name_dict is static name.
+ Use model.named_parameters() to get name_dict.
+ n_layers (int):
+ Total number of layers in the transformer encoder.
+ """
+ ratio = 1.0
+ static_name = name_dict[param.name]
+ if "blocks" in static_name:
+ idx = static_name.find("blocks.")
+ layer = int(static_name[idx:].split(".")[1])
+ ratio = decay_rate**(n_layers - layer)
+ elif "embed" in static_name:
+ ratio = decay_rate**(n_layers + 1)
+ param.optimize_attr["learning_rate"] *= ratio
+
+ def __call__(self, model_list):
+ model = model_list[0]
+ if self.weight_decay and self.filter_bias_and_bn:
+ skip = {}
+ if hasattr(model, 'no_weight_decay'):
+ skip = model.no_weight_decay()
+ decay_dict = {
+ param.name: not (len(param.shape) == 1 or
+ name.endswith(".bias") or name in skip)
+ for name, param in model.named_parameters()
+ if not 'teacher' in name
+ }
+ parameters = [
+ param for param in model.parameters()
+ if 'teacher' not in param.name
+ ]
+ weight_decay = 0.
+ else:
+ parameters = model.parameters()
+
+ opt_args = dict(
+ learning_rate=self.learning_rate, weight_decay=self.weight_decay)
+ opt_args['parameters'] = parameters
+ if decay_dict is not None:
+ opt_args['apply_decay_param_fun'] = lambda n: decay_dict[n]
+ opt_args['epsilon'] = self.epsilon
+ opt_args['beta1'] = self.beta1
+ opt_args['beta2'] = self.beta2
+ if self.layerwise_decay and self.layerwise_decay < 1.0:
+ opt_args['layerwise_decay'] = self.layerwise_decay
+ name_dict = dict()
+ for n, p in model.named_parameters():
+ name_dict[p.name] = n
+ opt_args['name_dict'] = name_dict
+ opt_args['n_layers'] = model.get_num_layers()
+
+ optimizer = self.AdamWDLImpl(**opt_args)
+
+ return optimizer
diff --git a/test_tipc/README.md b/test_tipc/README.md
index 5a3426bcbbb98f7f4f891cd0b72119939f4769a2..e110f475ade01c64c72d6bbbdd1df816e0cb5d13 100644
--- a/test_tipc/README.md
+++ b/test_tipc/README.md
@@ -110,7 +110,6 @@ bash test_tipc/test_train_inference_python.sh ./test_tipc/configs/MobileNetV3/Mo
- [test_train_pact_inference_python 使用](docs/test_train_pact_inference_python.md)：测试基于Python的模型PACT在线量化等基本功能。
- [test_train_ptq_inference_python 使用](docs/test_train_ptq_inference_python.md)：测试基于Python的模型KL离线量化等基本功能。
- [test_inference_cpp 使用](docs/test_inference_cpp.md) ：测试基于C++的模型推理。
-- [test_serving 使用](docs/test_serving.md) ：测试基于Paddle Serving的服务化部署功能。
- [test_lite_arm_cpu_cpp 使用](docs/test_lite_arm_cpu_cpp.md): 测试基于Paddle-Lite的ARM CPU端c++预测部署功能.
- [test_paddle2onnx 使用](docs/test_paddle2onnx.md)：测试Paddle2ONNX的模型转化功能，并验证正确性。
- [test_serving_infer_python 使用](docs/test_serving_infer_python.md)：测试python serving功能。
diff --git a/test_tipc/benchmark_train.sh b/test_tipc/benchmark_train.sh
index 1ecae126f5b3298c74d4379c2eb78faf5e3798f7..ad952424c13b7bf9738304d759a6c049693e2e61 100644
--- a/test_tipc/benchmark_train.sh
+++ b/test_tipc/benchmark_train.sh
@@ -179,6 +179,11 @@ for batch_size in ${batch_size_list[*]}; do
func_sed_params "$FILENAME" "${line_epoch}" "$epoch"
gpu_id=$(set_gpu_id $device_num)
+ # It is needed that using dali, NHWC and 4 channels when training ResNet50 with AMPO2
+ if [[ $model_name == "ResNet50" && $precision == "fp16" ]]; then
+ sed -i "s/ResNet50.yaml/ResNet50_amp_O2_ultra.yaml/g" $FILENAME
+ fi
+
# if bs is big, then copy train_list.txt to generate more train log
# At least 25 log number would be good to calculate ips for benchmark system.
# So the copy number for train_list is as follows:
diff --git a/test_tipc/common_func.sh b/test_tipc/common_func.sh
index 4aa3db6ca1d8091c99bf9a50a946417d94b7a791..0b8a660d5434937ac87ef08b076da61fa868e0a0 100644
--- a/test_tipc/common_func.sh
+++ b/test_tipc/common_func.sh
@@ -77,9 +77,10 @@ function status_check(){
run_command=$2
run_log=$3
model_name=$4
+ log_path=$5
if [ $last_status -eq 0 ]; then
- echo -e "\033[33m Run successfully with command - ${model_name} - ${run_command}! \033[0m" | tee -a ${run_log}
+ echo -e "\033[33m Run successfully with command - ${model_name} - ${run_command} - ${log_path} ! \033[0m" | tee -a ${run_log}
else
- echo -e "\033[33m Run failed with command - ${model_name} - ${run_command}! \033[0m" | tee -a ${run_log}
+ echo -e "\033[33m Run failed with command - ${model_name} - ${run_command} - ${log_path} ! \033[0m" | tee -a ${run_log}
fi
}
diff --git a/test_tipc/configs/GeneralRecognition/GeneralRecognition_PPLCNet_x2_5_train_custom_sampler.txt b/test_tipc/configs/GeneralRecognition/GeneralRecognition_PPLCNet_x2_5_train_custom_sampler.txt
index c1dbc89617fba27a3abfc80d9eca461f4aee212c..f3c78bbd2b5321260969acc9ea1a79fa6f39b1cf 100644
--- a/test_tipc/configs/GeneralRecognition/GeneralRecognition_PPLCNet_x2_5_train_custom_sampler.txt
+++ b/test_tipc/configs/GeneralRecognition/GeneralRecognition_PPLCNet_x2_5_train_custom_sampler.txt
@@ -13,14 +13,14 @@ train_infer_img_dir:./dataset/ILSVRC2012/val
null:null
##
trainer:norm_train
-norm_train:tools/train.py -c ppcls/configs/GeneralRecognition/GeneralRecognition_PPLCNet_x2_5.yaml -o DataLoader.Train.sampler.name="PKSampler" -o DataLoader.Train.sampler.sample_per_id=2
+norm_train:tools/train.py -c ppcls/configs/GeneralRecognition/GeneralRecognition_PPLCNet_x2_5.yaml -o DataLoader.Train.sampler.name="PKSampler" -o DataLoader.Train.sampler.sample_per_id=2
pact_train:null
fpgm_train:null
distill_train:null
null:null
null:null
##
-===========================eval_params===========================
+===========================eval_params===========================
eval:null
null:null
##
diff --git a/test_tipc/configs/GeneralRecognition/GeneralRecognition_PPLCNet_x2_5_train_infer_python.txt b/test_tipc/configs/GeneralRecognition/GeneralRecognition_PPLCNet_x2_5_train_infer_python.txt
index ff3bd088ddf22e20ca2b40aee1175c07961017f3..bdefd9476489bc61b870beb93a056dc2416d5213 100644
--- a/test_tipc/configs/GeneralRecognition/GeneralRecognition_PPLCNet_x2_5_train_infer_python.txt
+++ b/test_tipc/configs/GeneralRecognition/GeneralRecognition_PPLCNet_x2_5_train_infer_python.txt
@@ -20,7 +20,7 @@ distill_train:null
null:null
null:null
##
-===========================eval_params===========================
+===========================eval_params===========================
eval:tools/eval.py -c ppcls/configs/GeneralRecognition/GeneralRecognition_PPLCNet_x2_5.yaml
null:null
##
@@ -37,7 +37,7 @@ pretrained_model_url:https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/r
infer_model:../inference/
infer_export:True
infer_quant:Fasle
-inference:python/predict_rec.py -c configs/inference_rec.yaml
+inference:python/predict_rec.py -c configs/inference_rec.yaml -o Global.rec_inference_model_dir="./models/general_PPLCNet_x2_5_lite_v1.0_infer"
-o Global.use_gpu:True|False
-o Global.enable_mkldnn:False
-o Global.cpu_num_threads:1
diff --git a/test_tipc/configs/GeneralRecognition/GeneralRecognition_PPLCNet_x2_5_train_linux_gpu_fleet_normal_infer_python_linux_gpu_cpu.txt b/test_tipc/configs/GeneralRecognition/GeneralRecognition_PPLCNet_x2_5_train_linux_gpu_fleet_normal_infer_python_linux_gpu_cpu.txt
index fdcc052e9bd53af443095586ab858af0049bd75b..2c7788ac6e31424061309e2e45116ea27fedf9a1 100644
--- a/test_tipc/configs/GeneralRecognition/GeneralRecognition_PPLCNet_x2_5_train_linux_gpu_fleet_normal_infer_python_linux_gpu_cpu.txt
+++ b/test_tipc/configs/GeneralRecognition/GeneralRecognition_PPLCNet_x2_5_train_linux_gpu_fleet_normal_infer_python_linux_gpu_cpu.txt
@@ -37,7 +37,7 @@ pretrained_model_url:https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/r
infer_model:../inference/
infer_export:True
infer_quant:Fasle
-inference:python/predict_rec.py -c configs/inference_rec.yaml
+inference:python/predict_rec.py -c configs/inference_rec.yaml -o Global.rec_inference_model_dir="./models/general_PPLCNet_x2_5_lite_v1.0_infer"
-o Global.use_gpu:True|False
-o Global.enable_mkldnn:False
-o Global.cpu_num_threads:1
diff --git a/test_tipc/configs/GeneralRecognition/GeneralRecognition_PPLCNet_x2_5_train_linux_gpu_normal_amp_infer_python_linux_gpu_cpu.txt b/test_tipc/configs/GeneralRecognition/GeneralRecognition_PPLCNet_x2_5_train_linux_gpu_normal_amp_infer_python_linux_gpu_cpu.txt
index c433838faa65875fdb48679ae7fca47466a2f5c9..b2510e269d8af8018dac9745190f66d3a80615e7 100644
--- a/test_tipc/configs/GeneralRecognition/GeneralRecognition_PPLCNet_x2_5_train_linux_gpu_normal_amp_infer_python_linux_gpu_cpu.txt
+++ b/test_tipc/configs/GeneralRecognition/GeneralRecognition_PPLCNet_x2_5_train_linux_gpu_normal_amp_infer_python_linux_gpu_cpu.txt
@@ -37,7 +37,7 @@ pretrained_model_url:https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/r
infer_model:../inference/
infer_export:True
infer_quant:Fasle
-inference:python/predict_rec.py -c configs/inference_rec.yaml
+inference:python/predict_rec.py -c configs/inference_rec.yaml -o Global.rec_inference_model_dir="./models/general_PPLCNet_x2_5_lite_v1.0_infer"
-o Global.use_gpu:True|False
-o Global.enable_mkldnn:False
-o Global.cpu_num_threads:6
diff --git a/test_tipc/configs/GeneralRecognition/GeneralRecognition_PPLCNet_x2_5_train_multicard_eval.txt b/test_tipc/configs/GeneralRecognition/GeneralRecognition_PPLCNet_x2_5_train_multicard_eval.txt
index 165cfa9fb60d9d578ccfe7a9bd0fba5548bed3be..a4ef01b886183e9c18f54d90ab01ad528c908116 100644
--- a/test_tipc/configs/GeneralRecognition/GeneralRecognition_PPLCNet_x2_5_train_multicard_eval.txt
+++ b/test_tipc/configs/GeneralRecognition/GeneralRecognition_PPLCNet_x2_5_train_multicard_eval.txt
@@ -20,7 +20,7 @@ distill_train:null
null:null
null:null
##
-===========================eval_params===========================
+===========================eval_params===========================
eval:null
null:null
##
diff --git a/test_tipc/configs/GeneralRecognition/GeneralRecognition_PPLCNet_x2_5_train_no_eval.txt b/test_tipc/configs/GeneralRecognition/GeneralRecognition_PPLCNet_x2_5_train_no_eval.txt
index 1e167518433c4cfe42a931f4314361406941a87f..ba8ef39caaf7140aac059ae2b8a1aa9c962c301f 100644
--- a/test_tipc/configs/GeneralRecognition/GeneralRecognition_PPLCNet_x2_5_train_no_eval.txt
+++ b/test_tipc/configs/GeneralRecognition/GeneralRecognition_PPLCNet_x2_5_train_no_eval.txt
@@ -20,7 +20,7 @@ distill_train:null
null:null
null:null
##
-===========================eval_params===========================
+===========================eval_params===========================
eval:null
null:null
##
diff --git a/test_tipc/configs/GeneralRecognition/GeneralRecognition_PPLCNet_x2_5_train_pact_infer_python.txt b/test_tipc/configs/GeneralRecognition/GeneralRecognition_PPLCNet_x2_5_train_pact_infer_python.txt
index f557b258f42d7a793825e5a237de960a8c2e1fd8..5e299948b7876381371668f50ed1cbc9b47a3a93 100644
--- a/test_tipc/configs/GeneralRecognition/GeneralRecognition_PPLCNet_x2_5_train_pact_infer_python.txt
+++ b/test_tipc/configs/GeneralRecognition/GeneralRecognition_PPLCNet_x2_5_train_pact_infer_python.txt
@@ -37,7 +37,7 @@ pretrained_model_url:https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/r
infer_model:../inference/
infer_export:True
infer_quant:Fasle
-inference:python/predict_rec.py -c configs/inference_rec.yaml
+inference:python/predict_rec.py -c configs/inference_rec.yaml -o Global.rec_inference_model_dir="./models/general_PPLCNet_x2_5_lite_v1.0_infer"
-o Global.use_gpu:True|False
-o Global.enable_mkldnn:False
-o Global.cpu_num_threads:1
diff --git a/test_tipc/configs/GeneralRecognition/GeneralRecognition_PPLCNet_x2_5_train_ptq_infer_python.txt b/test_tipc/configs/GeneralRecognition/GeneralRecognition_PPLCNet_x2_5_train_ptq_infer_python.txt
index d5863d8c06a2647b2c52bee77fac02368023f52c..925ee91757acb4f48458933fa2b9163dc9c4557a 100644
--- a/test_tipc/configs/GeneralRecognition/GeneralRecognition_PPLCNet_x2_5_train_ptq_infer_python.txt
+++ b/test_tipc/configs/GeneralRecognition/GeneralRecognition_PPLCNet_x2_5_train_ptq_infer_python.txt
@@ -20,7 +20,7 @@ distill_train:null
null:null
null:null
##
-===========================eval_params===========================
+===========================eval_params===========================
eval:tools/eval.py -c ppcls/configs/GeneralRecognition/GeneralRecognition_PPLCNet_x2_5.yaml
null:null
##
@@ -37,7 +37,7 @@ pretrained_model_url:https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/r
infer_model:./general_PPLCNet_x2_5_lite_v1.0_infer/
infer_export:True
infer_quant:Fasle
-inference:python/predict_rec.py -c configs/inference_rec.yaml
+inference:python/predict_rec.py -c configs/inference_rec.yaml -o Global.rec_inference_model_dir="./models/general_PPLCNet_x2_5_lite_v1.0_infer"
-o Global.use_gpu:True|False
-o Global.enable_mkldnn:False
-o Global.cpu_num_threads:1
diff --git a/test_tipc/configs/GeneralRecognitionV2/GeneralRecognitionV2_PPLCNetV2_base_linux_gpu_normal_normal_paddle2onnx_python_linux_cpu.txt b/test_tipc/configs/GeneralRecognitionV2/GeneralRecognitionV2_PPLCNetV2_base_linux_gpu_normal_normal_paddle2onnx_python_linux_cpu.txt
new file mode 100644
index 0000000000000000000000000000000000000000..f22f55a52f869fe7811c7722da5bc63c8111120f
--- /dev/null
+++ b/test_tipc/configs/GeneralRecognitionV2/GeneralRecognitionV2_PPLCNetV2_base_linux_gpu_normal_normal_paddle2onnx_python_linux_cpu.txt
@@ -0,0 +1,16 @@
+===========================paddle2onnx_params===========================
+model_name:GeneralRecognitionV2_PPLCNetV2_base
+python:python3.7
+2onnx: paddle2onnx
+--model_dir:./deploy/models/general_PPLCNetV2_base_pretrained_v1.0_infer/
+--model_filename:inference.pdmodel
+--params_filename:inference.pdiparams
+--save_file:./deploy/models/general_PPLCNetV2_base_pretrained_v1.0_infer/inference.onnx
+--opset_version:10
+--enable_onnx_checker:True
+inference_model_url:https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/PP-ShiTuV2/general_PPLCNetV2_base_pretrained_v1.0_infer.tar
+inference:./python/predict_rec.py
+Global.use_onnx:True
+Global.rec_inference_model_dir:./models/general_PPLCNetV2_base_pretrained_v1.0_infer
+Global.use_gpu:False
+-c:configs/inference_rec.yaml
\ No newline at end of file
diff --git a/test_tipc/configs/GeneralRecognitionV2/GeneralRecognitionV2_PPLCNetV2_base_train_infer_python.txt b/test_tipc/configs/GeneralRecognitionV2/GeneralRecognitionV2_PPLCNetV2_base_train_infer_python.txt
new file mode 100644
index 0000000000000000000000000000000000000000..81b59edbf4e18ac29579c4b45464d1f8a85a2024
--- /dev/null
+++ b/test_tipc/configs/GeneralRecognitionV2/GeneralRecognitionV2_PPLCNetV2_base_train_infer_python.txt
@@ -0,0 +1,60 @@
+===========================train_params===========================
+model_name:GeneralRecognitionV2_PPLCNetV2_base
+python:python3.7
+gpu_list:0|0,1
+-o Global.device:gpu
+-o Global.auto_cast:null
+-o Global.epochs:lite_train_lite_infer=2|whole_train_whole_infer=120
+-o Global.output_dir:./output/
+-o DataLoader.Train.sampler.batch_size:8
+-o Global.pretrained_model:null
+train_model_name:latest
+train_infer_img_dir:./dataset/ILSVRC2012/val
+null:null
+##
+trainer:norm_train
+norm_train:tools/train.py -c ppcls/configs/GeneralRecognitionV2/GeneralRecognitionV2_PPLCNetV2_base.yaml -o Global.seed=1234 -o DataLoader.Train.sampler.shuffle=False -o DataLoader.Train.loader.num_workers=0 -o DataLoader.Train.loader.use_shared_memory=False -o Global.eval_during_train=False -o Global.save_interval=2 -o DataLoader.Train.dataset.cls_label_path=./dataset/train_reg_all_data.txt -o DataLoader.Train.loader.sampler.batch_size=8
+pact_train:null
+fpgm_train:null
+distill_train:null
+null:null
+null:null
+##
+===========================eval_params===========================
+eval:tools/eval.py -c ppcls/configs/GeneralRecognitionV2/GeneralRecognitionV2_PPLCNetV2_base.yaml
+null:null
+##
+===========================infer_params==========================
+-o Global.save_inference_dir:./inference
+-o Global.pretrained_model:
+norm_export:tools/export_model.py -c ppcls/configs/GeneralRecognitionV2/GeneralRecognitionV2_PPLCNetV2_base.yaml
+quant_export:null
+fpgm_export:null
+distill_export:null
+kl_quant:null
+export2:null
+pretrained_model_url:https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/pretrain/PPShiTuV2/general_PPLCNetV2_base_pretrained_v1.0.pdparams
+infer_model:../inference/
+infer_export:True
+infer_quant:Fasle
+inference:python/predict_rec.py -c configs/inference_rec.yaml
+-o Global.use_gpu:True|False
+-o Global.enable_mkldnn:False
+-o Global.cpu_num_threads:1
+-o Global.batch_size:1
+-o Global.use_tensorrt:False
+-o Global.use_fp16:False
+-o Global.rec_inference_model_dir:../inference
+-o Global.infer_imgs:../dataset/Aliproduct/demo_test/
+-o Global.save_log_path:null
+-o Global.benchmark:False
+null:null
+null:null
+===========================train_benchmark_params==========================
+batch_size:256
+fp_items:fp32|fp16
+epoch:1
+--profiler_options:batch_range=[10,20];state=GPU;tracer_option=Default;profile_path=model.profile
+flags:FLAGS_eager_delete_tensor_gb=0.0;FLAGS_fraction_of_gpu_memory_to_use=0.98;FLAGS_conv_workspace_size_limit=4096
+===========================infer_benchmark_params==========================
+random_infer_input:[{float32,[3,224,224]}]
\ No newline at end of file
diff --git a/test_tipc/configs/GeneralRecognitionV2/GeneralRecognitionV2_PPLCNetV2_base_train_linux_gpu_fleet_normal_infer_python_linux_gpu_cpu.txt b/test_tipc/configs/GeneralRecognitionV2/GeneralRecognitionV2_PPLCNetV2_base_train_linux_gpu_fleet_normal_infer_python_linux_gpu_cpu.txt
new file mode 100644
index 0000000000000000000000000000000000000000..f1f32751d6ee296c341c0e8382e5fdae2b9335bb
--- /dev/null
+++ b/test_tipc/configs/GeneralRecognitionV2/GeneralRecognitionV2_PPLCNetV2_base_train_linux_gpu_fleet_normal_infer_python_linux_gpu_cpu.txt
@@ -0,0 +1,54 @@
+===========================train_params===========================
+model_name:GeneralRecognition_PPLCNet_x2_5
+python:python3.7
+gpu_list:192.168.0.1,192.168.0.2;0,1
+-o Global.device:gpu
+-o Global.auto_cast:null
+-o Global.epochs:lite_train_lite_infer=2|whole_train_whole_infer=120
+-o Global.output_dir:./output/
+-o DataLoader.Train.sampler.batch_size:8
+-o Global.pretrained_model:null
+train_model_name:latest
+train_infer_img_dir:./dataset/ILSVRC2012/val
+null:null
+##
+trainer:norm_train
+norm_train:tools/train.py -c ppcls/configs/GeneralRecognitionV2/GeneralRecognitionV2_PPLCNetV2_base.yaml -o Global.seed=1234 -o DataLoader.Train.sampler.shuffle=False -o DataLoader.Train.loader.num_workers=0 -o DataLoader.Train.loader.use_shared_memory=False -o Global.eval_during_train=False -o Global.save_interval=2 -o DataLoader.Train.dataset.cls_label_path=./dataset/train_reg_all_data.txt -o DataLoader.Train.loader.sampler.batch_size=8
+pact_train:null
+fpgm_train:null
+distill_train:null
+null:null
+null:null
+##
+===========================eval_params===========================
+eval:tools/eval.py -c ppcls/configs/GeneralRecognitionV2/GeneralRecognitionV2_PPLCNetV2_base.yaml
+null:null
+##
+===========================infer_params==========================
+-o Global.save_inference_dir:./inference
+-o Global.pretrained_model:
+norm_export:tools/export_model.py -c ppcls/configs/GeneralRecognitionV2/GeneralRecognitionV2_PPLCNetV2_base.yaml
+quant_export:null
+fpgm_export:null
+distill_export:null
+kl_quant:null
+export2:null
+pretrained_model_url:https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/pretrain/PPShiTuV2/general_PPLCNetV2_base_pretrained_v1.0.pdparams
+infer_model:../inference/
+infer_export:True
+infer_quant:Fasle
+inference:python/predict_rec.py -c configs/inference_rec.yaml
+-o Global.use_gpu:True|False
+-o Global.enable_mkldnn:False
+-o Global.cpu_num_threads:1
+-o Global.batch_size:1
+-o Global.use_tensorrt:False
+-o Global.use_fp16:False
+-o Global.rec_inference_model_dir:../inference
+-o Global.infer_imgs:../dataset/Aliproduct/demo_test/
+-o Global.save_log_path:null
+-o Global.benchmark:False
+null:null
+null:null
+===========================infer_benchmark_params==========================
+random_infer_input:[{float32,[3,224,224]}]
\ No newline at end of file
diff --git a/test_tipc/configs/GeneralRecognitionV2/GeneralRecognitionV2_PPLCNetV2_base_train_linux_gpu_normal_amp_infer_python_linux_gpu_cpu.txt b/test_tipc/configs/GeneralRecognitionV2/GeneralRecognitionV2_PPLCNetV2_base_train_linux_gpu_normal_amp_infer_python_linux_gpu_cpu.txt
new file mode 100644
index 0000000000000000000000000000000000000000..6e436de4063ac5e4917815b639ae5684fce2831b
--- /dev/null
+++ b/test_tipc/configs/GeneralRecognitionV2/GeneralRecognitionV2_PPLCNetV2_base_train_linux_gpu_normal_amp_infer_python_linux_gpu_cpu.txt
@@ -0,0 +1,52 @@
+===========================train_params===========================
+model_name:GeneralRecognitionV2_PPLCNetV2_base
+python:python3.7
+gpu_list:0|0,1
+-o Global.device:gpu
+-o Global.auto_cast:null
+-o Global.epochs:lite_train_lite_infer=2|whole_train_whole_infer=100
+-o Global.output_dir:./output/
+-o DataLoader.Train.sampler.batch_size:8
+-o Global.pretrained_model:null
+train_model_name:latest
+train_infer_img_dir:./dataset/ILSVRC2012/val
+null:null
+##
+trainer:amp_train
+amp_train:tools/train.py -c ppcls/configs/GeneralRecognitionV2/GeneralRecognitionV2_PPLCNetV2_base.yaml -o Global.seed=1234 -o DataLoader.Train.sampler.shuffle=False -o DataLoader.Train.loader.num_workers=0 -o DataLoader.Train.loader.use_shared_memory=False -o AMP.scale_loss=65536 -o AMP.use_dynamic_loss_scaling=True -o AMP.level=O2 -o Optimizer.multi_precision=True -o Global.eval_during_train=False -o DataLoader.Train.dataset.cls_label_path=./dataset/train_reg_all_data.txt -o DataLoader.Train.loader.sampler.batch_size=8
+pact_train:null
+fpgm_train:null
+distill_train:null
+null:null
+null:null
+##
+===========================eval_params===========================
+eval:tools/eval.py -c ppcls/configs/GeneralRecognitionV2/GeneralRecognitionV2_PPLCNetV2_base.yaml
+null:null
+##
+===========================infer_params==========================
+-o Global.save_inference_dir:./inference
+-o Global.pretrained_model:
+norm_export:tools/export_model.py -c ppcls/configs/GeneralRecognitionV2/GeneralRecognitionV2_PPLCNetV2_base.yaml
+quant_export:null
+fpgm_export:null
+distill_export:null
+kl_quant:null
+export2:null
+pretrained_model_url:https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/pretrain/PPShiTuV2/general_PPLCNetV2_base_pretrained_v1.0.pdparams
+infer_model:../inference/
+infer_export:True
+infer_quant:Fasle
+inference:python/predict_rec.py -c configs/inference_rec.yaml
+-o Global.use_gpu:True|False
+-o Global.enable_mkldnn:False
+-o Global.cpu_num_threads:6
+-o Global.batch_size:1
+-o Global.use_tensorrt:False
+-o Global.use_fp16:False
+-o Global.rec_inference_model_dir:../inference
+-o Global.infer_imgs:../dataset/Aliproduct/demo_test/
+-o Global.save_log_path:null
+-o Global.benchmark:False
+null:null
+null:null
diff --git a/test_tipc/configs/GeneralRecognitionV2/GeneralRecognitionV2_PPLCNetV2_base_train_pact_infer_python.txt b/test_tipc/configs/GeneralRecognitionV2/GeneralRecognitionV2_PPLCNetV2_base_train_pact_infer_python.txt
new file mode 100644
index 0000000000000000000000000000000000000000..68e4c4ac8ec00cb688fefa7d0a0d27bab9fedbe1
--- /dev/null
+++ b/test_tipc/configs/GeneralRecognitionV2/GeneralRecognitionV2_PPLCNetV2_base_train_pact_infer_python.txt
@@ -0,0 +1,54 @@
+===========================train_params===========================
+model_name:GeneralRecognitionV2_PPLCNetV2_base
+python:python3.7
+gpu_list:0
+-o Global.device:gpu
+-o Global.auto_cast:null
+-o Global.epochs:lite_train_lite_infer=2|whole_train_whole_infer=100
+-o Global.output_dir:./output/
+-o DataLoader.Train.sampler.batch_size:8
+-o Global.pretrained_model:null
+train_model_name:latest
+train_infer_img_dir:./dataset/ILSVRC2012/val
+null:null
+##
+trainer:pact_train
+norm_train:null
+pact_train:tools/train.py -c ppcls/configs/GeneralRecognitionV2/GeneralRecognitionV2_PPLCNetV2_base.yaml -o Global.seed=1234 -o DataLoader.Train.sampler.shuffle=False -o DataLoader.Train.loader.num_workers=0 -o DataLoader.Train.loader.use_shared_memory=False -o Slim.quant.name=pact -o Optimizer.lr.learning_rate=0.006 -o Global.pretrained_model="pretrained_model/general_PPLCNetV2_base_pretrained_v1.0" -o DataLoader.Train.dataset.cls_label_path=./dataset/train_reg_all_data.txt -o AMP=None -o DataLoader.Train.sampler.batch_size=8
+fpgm_train:null
+distill_train:null
+null:null
+null:null
+##
+===========================eval_params===========================
+eval:tools/eval.py -c ppcls/configs/GeneralRecognitionV2/GeneralRecognitionV2_PPLCNetV2_base.yaml -o Slim.quant.name=pact
+null:null
+##
+===========================infer_params==========================
+-o Global.save_inference_dir:./inference
+-o Global.pretrained_model:
+norm_export:null
+quant_export:tools/export_model.py -c ppcls/configs/GeneralRecognitionV2/GeneralRecognitionV2_PPLCNetV2_base.yaml -o Slim.quant.name=pact
+fpgm_export:null
+distill_export:null
+kl_quant:null
+export2:null
+pretrained_model_url:https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/pretrain/PPShiTuV2/general_PPLCNetV2_base_pretrained_v1.0.pdparams
+infer_model:../inference/
+infer_export:True
+infer_quant:Fasle
+inference:python/predict_rec.py -c configs/inference_rec.yaml
+-o Global.use_gpu:True|False
+-o Global.enable_mkldnn:False
+-o Global.cpu_num_threads:1
+-o Global.batch_size:1
+-o Global.use_tensorrt:False
+-o Global.use_fp16:False
+-o Global.rec_inference_model_dir:../inference
+-o Global.infer_imgs:../dataset/Aliproduct/demo_test/
+-o Global.save_log_path:null
+-o Global.benchmark:True
+null:null
+null:null
+===========================infer_benchmark_params==========================
+random_infer_input:[{float32,[3,224,224]}]
\ No newline at end of file
diff --git a/test_tipc/configs/GeneralRecognitionV2/GeneralRecognitionV2_PPLCNetV2_base_train_ptq_infer_python.txt b/test_tipc/configs/GeneralRecognitionV2/GeneralRecognitionV2_PPLCNetV2_base_train_ptq_infer_python.txt
new file mode 100644
index 0000000000000000000000000000000000000000..d82e871fb30aa70093b1f91aab3e0327fd16eedf
--- /dev/null
+++ b/test_tipc/configs/GeneralRecognitionV2/GeneralRecognitionV2_PPLCNetV2_base_train_ptq_infer_python.txt
@@ -0,0 +1,54 @@
+===========================train_params===========================
+model_name:GeneralRecognitionV2_PPLCNetV2_base
+python:python3.7
+gpu_list:0
+-o Global.device:gpu
+-o Global.auto_cast:null
+-o Global.epochs:lite_train_lite_infer=2|whole_train_whole_infer=100
+-o Global.output_dir:./output/
+-o DataLoader.Train.sampler.batch_size:8
+-o Global.pretrained_model:null
+train_model_name:latest
+train_infer_img_dir:./dataset/ILSVRC2012/val
+null:null
+##
+trainer:pact_train
+norm_train:null
+pact_train:tools/train.py -c ppcls/configs/GeneralRecognitionV2/GeneralRecognitionV2_PPLCNetV2_base.yaml -o Global.seed=1234 -o DataLoader.Train.sampler.shuffle=False -o DataLoader.Train.loader.num_workers=0 -o DataLoader.Train.loader.use_shared_memory=False
+fpgm_train:null
+distill_train:null
+null:null
+null:null
+##
+===========================eval_params===========================
+eval:tools/eval.py -c ppcls/configs/GeneralRecognitionV2/GeneralRecognitionV2_PPLCNetV2_base.yaml
+null:null
+##
+===========================infer_params==========================
+-o Global.save_inference_dir:./inference
+-o Global.pretrained_model:
+norm_export:null
+quant_export:tools/export_model.py -c ppcls/configs/GeneralRecognitionV2/GeneralRecognitionV2_PPLCNetV2_base.yaml
+fpgm_export:null
+distill_export:null
+kl_quant:deploy/slim/quant_post_static.py -c ppcls/configs/GeneralRecognitionV2/GeneralRecognitionV2_PPLCNetV2_base.yaml -o Global.save_inference_dir=./general_PPLCNetV2_base_pretrained_v1.0_infer
+export2:null
+pretrained_model_url:https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/PP-ShiTuV2/general_PPLCNetV2_base_pretrained_v1.0_infer.tar
+infer_model:./general_PPLCNetV2_base_pretrained_v1.0_infer
+infer_export:True
+infer_quant:Fasle
+inference:python/predict_rec.py -c configs/inference_rec.yaml
+-o Global.use_gpu:True|False
+-o Global.enable_mkldnn:False
+-o Global.cpu_num_threads:1
+-o Global.batch_size:1
+-o Global.use_tensorrt:False
+-o Global.use_fp16:False
+-o Global.rec_inference_model_dir:../inference
+-o Global.infer_imgs:../dataset/Aliproduct/demo_test/
+-o Global.save_log_path:null
+-o Global.benchmark:False
+null:null
+null:null
+===========================infer_benchmark_params==========================
+random_infer_input:[{float32,[3,224,224]}]
\ No newline at end of file
diff --git a/test_tipc/configs/PP-ShiTuV2/PPShiTuV2_linux_gpu_normal_normal_infer_cpp_linux_gpu_cpu.txt b/test_tipc/configs/PP-ShiTuV2/PPShiTuV2_linux_gpu_normal_normal_infer_cpp_linux_gpu_cpu.txt
new file mode 100644
index 0000000000000000000000000000000000000000..52a88a3183e1dde7e75177160e868c2617dc75cb
--- /dev/null
+++ b/test_tipc/configs/PP-ShiTuV2/PPShiTuV2_linux_gpu_normal_normal_infer_cpp_linux_gpu_cpu.txt
@@ -0,0 +1,19 @@
+===========================cpp_infer_params===========================
+model_name:PPShiTuV2
+cpp_infer_type:shitu
+feature_inference_model_dir:./general_PPLCNetV2_base_pretrained_v1.0_infer/
+det_inference_model_dir:./picodet_PPLCNet_x2_5_mainbody_lite_v1.0_infer/
+cls_inference_url:https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/PP-ShiTuV2/general_PPLCNetV2_base_pretrained_v1.0_infer.tar
+det_inference_url:https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/picodet_PPLCNet_x2_5_mainbody_lite_v1.0_infer.tar
+infer_quant:False
+inference_cmd:./deploy/cpp_shitu/build/pp_shitu -c inference_drink.yaml
+use_gpu:True|False
+enable_mkldnn:False
+cpu_threads:1
+batch_size:1
+use_tensorrt:False
+precision:fp32
+data_dir:./dataset/drink_dataset_v2.0
+benchmark:True
+generate_yaml_cmd:python3.7 test_tipc/generate_cpp_yaml.py
+transform_index_cmd:python3.7 deploy/cpp_shitu/tools/transform_id_map.py -c inference_drink.yaml
diff --git a/test_tipc/configs/PP-ShiTuV2/PPShiTuV2_linux_gpu_normal_normal_serving_cpp_linux_gpu_cpu.txt b/test_tipc/configs/PP-ShiTuV2/PPShiTuV2_linux_gpu_normal_normal_serving_cpp_linux_gpu_cpu.txt
new file mode 100644
index 0000000000000000000000000000000000000000..bc0c086b5c7c713392bbb09a34994a631035dc26
--- /dev/null
+++ b/test_tipc/configs/PP-ShiTuV2/PPShiTuV2_linux_gpu_normal_normal_serving_cpp_linux_gpu_cpu.txt
@@ -0,0 +1,18 @@
+===========================serving_params===========================
+model_name:PPShiTuV2
+python:python3.7
+cls_inference_model_url:https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/PP-ShiTuV2/general_PPLCNetV2_base_pretrained_v1.0_infer.tar
+det_inference_model_url:https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/picodet_PPLCNet_x2_5_mainbody_lite_v1.0_infer.tar
+trans_model:-m paddle_serving_client.convert
+--dirname:./models/general_PPLCNetV2_base_pretrained_v1.0_infer/
+--dirname:./models/picodet_PPLCNet_x2_5_mainbody_lite_v1.0_infer/
+--model_filename:inference.pdmodel
+--params_filename:inference.pdiparams
+--serving_server:./models/general_PPLCNetV2_base_pretrained_v1.0_serving/
+--serving_client:./models/general_PPLCNetV2_base_pretrained_v1.0_client/
+--serving_server:./models/picodet_PPLCNet_x2_5_mainbody_lite_v1.0_serving/
+--serving_client:./models/picodet_PPLCNet_x2_5_mainbody_lite_v1.0_client/
+serving_dir:./paddleserving/recognition
+web_service:null
+--use_gpu:0|null
+pipline:test_cpp_serving_client.py
diff --git a/test_tipc/configs/PP-ShiTuV2/PPShiTuV2_linux_gpu_normal_normal_serving_python_linux_gpu_cpu.txt b/test_tipc/configs/PP-ShiTuV2/PPShiTuV2_linux_gpu_normal_normal_serving_python_linux_gpu_cpu.txt
new file mode 100644
index 0000000000000000000000000000000000000000..1c08a4cf3b21a9edbb2847b6ca4589f50136b60f
--- /dev/null
+++ b/test_tipc/configs/PP-ShiTuV2/PPShiTuV2_linux_gpu_normal_normal_serving_python_linux_gpu_cpu.txt
@@ -0,0 +1,18 @@
+===========================serving_params===========================
+model_name:PPShiTuV2
+python:python3.7
+cls_inference_model_url:https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/PP-ShiTuV2/general_PPLCNetV2_base_pretrained_v1.0_infer.tar
+det_inference_model_url:https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/picodet_PPLCNet_x2_5_mainbody_lite_v1.0_infer.tar
+trans_model:-m paddle_serving_client.convert
+--dirname:./models/general_PPLCNetV2_base_pretrained_v1.0_infer/
+--dirname:./models/picodet_PPLCNet_x2_5_mainbody_lite_v1.0_infer/
+--model_filename:inference.pdmodel
+--params_filename:inference.pdiparams
+--serving_server:./models/general_PPLCNetV2_base_pretrained_v1.0_serving/
+--serving_client:./models/general_PPLCNetV2_base_pretrained_v1.0_client/
+--serving_server:./models/picodet_PPLCNet_x2_5_mainbody_lite_v1.0_serving/
+--serving_client:./models/picodet_PPLCNet_x2_5_mainbody_lite_v1.0_client/
+serving_dir:./paddleserving/recognition
+web_service:recognition_web_service.py
+--use_gpu:0|null
+pipline:pipeline_http_client.py
diff --git a/test_tipc/configs/PP-ShiTuV2/PPShiTuV2_mainbody_det_linux_gpu_normal_normal_paddle2onnx_python_linux_cpu.txt b/test_tipc/configs/PP-ShiTuV2/PPShiTuV2_mainbody_det_linux_gpu_normal_normal_paddle2onnx_python_linux_cpu.txt
new file mode 100644
index 0000000000000000000000000000000000000000..bfd24bb4106245d7b279d0e7c07ffbc39f28fe83
--- /dev/null
+++ b/test_tipc/configs/PP-ShiTuV2/PPShiTuV2_mainbody_det_linux_gpu_normal_normal_paddle2onnx_python_linux_cpu.txt
@@ -0,0 +1,16 @@
+===========================paddle2onnx_params===========================
+model_name:PP-ShiTu_mainbody_det
+python:python3.7
+2onnx: paddle2onnx
+--model_dir:./deploy/models/picodet_lcnet_x2_5_640_mainbody_infer/
+--model_filename:inference.pdmodel
+--params_filename:inference.pdiparams
+--save_file:./deploy/models/picodet_lcnet_x2_5_640_mainbody_infer/inference.onnx
+--opset_version:11
+--enable_onnx_checker:True
+inference_model_url:https://paddledet.bj.bcebos.com/models/picodet_lcnet_x2_5_640_mainbody_infer.tar
+inference:null
+Global.use_onnx:null
+Global.inference_model_dir:null
+Global.use_gpu:null
+-c:null
\ No newline at end of file
diff --git a/test_tipc/docs/test_inference_cpp.md b/test_tipc/docs/test_inference_cpp.md
index 256e6a5f4d10d2e64e71778fe43f84c1784b3000..86a486ae93ca9f6664f9db708406e840609e2948 100644
--- a/test_tipc/docs/test_inference_cpp.md
+++ b/test_tipc/docs/test_inference_cpp.md
@@ -12,6 +12,7 @@ Linux GPU/CPU C++ 推理功能测试的主程序为`test_inference_cpp.sh`，可
| MobileNetV3 | MobileNetV3_large_x1_0_KL | 支持 | 支持 |
| MobileNetV3 | MobileNetV3_large_x1_0_PACT | 支持 | 支持 |
| PP-ShiTu | PPShiTu_general_rec、PPShiTu_mainbody_det | 支持 | 支持 |
+| PP-ShiTuV2 | PPShiTuV2_general_rec、PPShiTu_mainbody_det | 支持 | 支持 |
| PP-ShiTu | GeneralRecognition_PPLCNet_x2_5_KL | 支持 | 支持 |
| PP-ShiTu | GeneralRecognition_PPLCNet_x2_5_PACT | 支持 | 支持 |
| PPHGNet | PPHGNet_small | 支持 | 支持 |
diff --git a/test_tipc/docs/test_serving_infer_cpp.md b/test_tipc/docs/test_serving_infer_cpp.md
index 3ddd0c253b9d596697da8108e9cf563a21bf0cba..d1dc02ac375a46a888a66ed240304f241dee25cb 100644
--- a/test_tipc/docs/test_serving_infer_cpp.md
+++ b/test_tipc/docs/test_serving_infer_cpp.md
@@ -15,6 +15,7 @@ Linux GPU/CPU C++ 服务化部署测试的主程序为`test_serving_infer_cpp.sh
| PP-ShiTu | PPShiTu_general_rec、PPShiTu_mainbody_det | 支持 | 支持 |
| PP-ShiTu | GeneralRecognition_PPLCNet_x2_5_KL | 支持 | 支持 |
| PP-ShiTu | GeneralRecognition_PPLCNet_x2_5_PACT | 支持 | 支持 |
+| PP-ShiTuV2 | PPShiTuV2_general_rec、PPShiTu_mainbody_det | 支持 | 支持 |
| PPHGNet | PPHGNet_small | 支持 | 支持 |
| PPHGNet | PPHGNet_small_KL | 支持 | 支持 |
| PPHGNet | PPHGNet_small_PACT | 支持 | 支持 |
diff --git a/test_tipc/docs/test_serving_infer_python.md b/test_tipc/docs/test_serving_infer_python.md
index 9bd9dc4c2d65f2d55a0335d012fdef37b7097f54..6314031c6963662a0df2fd81dca57fd86f1ae9b0 100644
--- a/test_tipc/docs/test_serving_infer_python.md
+++ b/test_tipc/docs/test_serving_infer_python.md
@@ -15,6 +15,7 @@ Linux GPU/CPU PYTHON 服务化部署测试的主程序为`test_serving_infer_pyt
| PP-ShiTu | PPShiTu_general_rec、PPShiTu_mainbody_det | 支持 | 支持 |
| PP-ShiTu | GeneralRecognition_PPLCNet_x2_5_KL | 支持 | 支持 |
| PP-ShiTu | GeneralRecognition_PPLCNet_x2_5_PACT | 支持 | 支持 |
+| PP-ShiTuV2 | PPShiTuV2_general_rec、PPShiTu_mainbody_det | 支持 | 支持 |
| PPHGNet | PPHGNet_small | 支持 | 支持 |
| PPHGNet | PPHGNet_small_KL | 支持 | 支持 |
| PPHGNet | PPHGNet_small_PACT | 支持 | 支持 |
diff --git a/test_tipc/docs/test_train_amp_inference_python.md b/test_tipc/docs/test_train_amp_inference_python.md
index a6a5897d3f1afeab333ad40d7b8b8f926b92c68e..b9fd6cb68981938120b2b2e610a517cc47592f61 100644
--- a/test_tipc/docs/test_train_amp_inference_python.md
+++ b/test_tipc/docs/test_train_amp_inference_python.md
@@ -10,6 +10,7 @@ Linux GPU/CPU 混合精度训练推理测试的主程序为`test_train_inference
| :-------------: | :-------------------------------------: | :----------: | :----------: |
| MobileNetV3 | MobileNetV3_large_x1_0 | 混合精度训练 | 混合精度训练 |
| PP-ShiTu | GeneralRecognition_PPLCNet_x2_5 | 混合精度训练 | 混合精度训练 |
+| PP-ShiTuV2 | GeneralRecognitionV2_PPLCNetV2_base | 混合精度训练 | 混合精度训练 |
| PPHGNet | PPHGNet_small | 混合精度训练 | 混合精度训练 |
| PPHGNet | PPHGNet_tiny | 混合精度训练 | 混合精度训练 |
| PPLCNet | PPLCNet_x0_25 | 混合精度训练 | 混合精度训练 |
@@ -31,6 +32,7 @@ Linux GPU/CPU 混合精度训练推理测试的主程序为`test_train_inference
| :-------------: | :-------------------------------------: | :--------: | :--------: | :-------: |
| MobileNetV3 | MobileNetV3_large_x1_0 | 支持 | 支持 | 1 |
| PP-ShiTu | GeneralRecognition_PPLCNet_x2_5 | 支持 | 支持 | 1 |
+| PP-ShiTuV2 | GeneralRecognitionV2_PPLCNetV2_base | 支持 | 支持 | 1 |
| PPHGNet | PPHGNet_small | 支持 | 支持 | 1 |
| PPHGNet | PPHGNet_tiny | 支持 | 支持 | 1 |
| PPLCNet | PPLCNet_x0_25 | 支持 | 支持 | 1 |
diff --git a/test_tipc/docs/test_train_pact_inference_python.md b/test_tipc/docs/test_train_pact_inference_python.md
index 6aeecad78a27172ea54aa2bbe318f68a5d0ee188..795c70c6447be933674b603130f93aed98c27e08 100644
--- a/test_tipc/docs/test_train_pact_inference_python.md
+++ b/test_tipc/docs/test_train_pact_inference_python.md
@@ -10,6 +10,7 @@ Linux GPU/CPU PACT量化训练推理测试的主程序为`test_train_inference_p
| :-------------: | :-------------------------------------: | :----------: |
| MobileNetV3 | MobileNetV3_large_x1_0 | PACT量化训练 |
| PP-ShiTu | GeneralRecognition_PPLCNet_x2_5 | PACT量化训练 |
+| PP-ShiTuV2 | GeneralRecognitionV2_PPLCNetV2_base | PACT量化训练 |
| PPHGNet | PPHGNet_small | PACT量化训练 |
| PPHGNet | PPHGNet_tiny | PACT量化训练 |
| PPLCNet | PPLCNet_x0_25 | PACT量化训练 |
@@ -31,6 +32,7 @@ Linux GPU/CPU PACT量化训练推理测试的主程序为`test_train_inference_p
| :-------------: | :-------------------------------------: | :--------: | :--------: | :-------: |
| MobileNetV3 | MobileNetV3_large_x1_0 | 支持 | 支持 | 1 |
| PP-ShiTu | GeneralRecognition_PPLCNet_x2_5 | 支持 | 支持 | 1 |
+| PP-ShiTuV2 | GeneralRecognitionV2_PPLCNetV2_base | 支持 | 支持 | 1 |
| PPHGNet | PPHGNet_small | 支持 | 支持 | 1 |
| PPHGNet | PPHGNet_tiny | 支持 | 支持 | 1 |
| PPLCNet | PPLCNet_x0_25 | 支持 | 支持 | 1 |
diff --git a/test_tipc/docs/test_train_ptq_inference_python.md b/test_tipc/docs/test_train_ptq_inference_python.md
index 29d5b9f59b31d96dd5fe2b325853fee911a87c97..18eddd127934fc63d82140c196913248572ae321 100644
--- a/test_tipc/docs/test_train_ptq_inference_python.md
+++ b/test_tipc/docs/test_train_ptq_inference_python.md
@@ -10,6 +10,7 @@ Linux GPU/CPU KL离线量化推理测试的主程序为`test_ptq_inference_pytho
| :-------------: | :-------------------------------------: | :----------: |
| MobileNetV3 | MobileNetV3_large_x1_0 | KL离线量化 |
| PP-ShiTu | GeneralRecognition_PPLCNet_x2_5 | KL离线量化 |
+| PP-ShiTuV2 | GeneralRecognitionV2_PPLCNetV2_base | KL离线量化 |
| PPHGNet | PPHGNet_small | KL离线量化 |
| PPHGNet | PPHGNet_tiny | KL离线量化 |
| PPLCNet | PPLCNet_x0_25 | KL离线量化 |
@@ -31,6 +32,7 @@ Linux GPU/CPU KL离线量化推理测试的主程序为`test_ptq_inference_pytho
| :-------------: | :-------------------------------------: | :----------: |
| MobileNetV3 | MobileNetV3_large_x1_0 | KL离线量化 |
| PP-ShiTu | GeneralRecognition_PPLCNet_x2_5 | KL离线量化 |
+| PP-ShiTuV2 | GeneralRecognitionV2_PPLCNetV2_base | KL离线量化 |
| PPHGNet | PPHGNet_small | KL离线量化 |
| PPHGNet | PPHGNet_tiny | KL离线量化 |
| PPLCNet | PPLCNet_x0_25 | KL离线量化 |
diff --git a/test_tipc/prepare.sh b/test_tipc/prepare.sh
index 0ccb8a95878ab1f406586bc07b5bf8929601349f..81841b612d602b7d45768768ded4faa3ae6d781e 100644
--- a/test_tipc/prepare.sh
+++ b/test_tipc/prepare.sh
@@ -42,6 +42,10 @@ function func_get_url_file_name() {
model_name=$(func_parser_value "${lines[1]}")
+# install paddleclas whl
+python_name=$(func_parser_value "${lines[2]}")
+${python_name} setup.py install
+
if [[ ${MODE} = "cpp_infer" ]]; then
if [ -d "./deploy/cpp/opencv-3.4.7/opencv3/" ] && [ $(md5sum ./deploy/cpp/opencv-3.4.7.tar.gz | awk -F ' ' '{print $1}') = "faa2b5950f8bee3f03118e600c74746a" ]; then
echo "################### build opencv skipped ###################"
@@ -139,6 +143,8 @@ if [[ ${MODE} = "cpp_infer" ]]; then
cd dataset
wget -nc https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/data/drink_dataset_v1.0.tar
tar -xf drink_dataset_v1.0.tar
+ wget -nc https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/data/drink_dataset_v2.0.tar
+ tar -xf drink_dataset_v2.0.tar
else
echo "Wrong cpp type in config file in line 3. only support cls, shitu"
fi
@@ -167,8 +173,9 @@ if [[ $model_name == *ShiTu* ]]; then
ln -s demo_test.txt val_list.txt
cd ../../
eval "wget -nc $model_url_value --no-check-certificate"
- mv general_PPLCNet_x2_5_pretrained_v1.0.pdparams GeneralRecognition_PPLCNet_x2_5_pretrained.pdparams
- exit 0
+ if [[ -d "./general_PPLCNet_x2_5_pretrained_v1.0.pdparams" ]]; then
+ mv general_PPLCNet_x2_5_pretrained_v1.0.pdparams GeneralRecognition_PPLCNet_x2_5_pretrained.pdparams
+ fi
fi
if [[ $FILENAME == *use_dali* ]]; then
@@ -240,12 +247,12 @@ elif [[ ${MODE} = "whole_infer" ]]; then
cd ../../
fi
# download inference or pretrained model
- eval "wget -nc $model_url_value"
+ eval "wget -nc ${model_url_value}"
if [[ ${model_url_value} =~ ".tar" ]]; then
tar_name=$(func_get_url_file_name "${model_url_value}")
- echo $tar_name
- rm -rf {tar_name}
- tar xf ${tar_name}
+ echo ${tar_name}
+ eval "tar -xf ${tar_name}"
+ rm -f ${tar_name}
fi
if [[ $model_name == "SwinTransformer_large_patch4_window7_224" || $model_name == "SwinTransformer_large_patch4_window12_384" ]]; then
cmd="mv ${model_name}_22kto1k_pretrained.pdparams ${model_name}_pretrained.pdparams"
@@ -275,7 +282,7 @@ fi
if [[ ${MODE} = "serving_infer" ]]; then
# prepare serving env
python_name=$(func_parser_value "${lines[2]}")
- if [[ ${model_name} = "PPShiTu" ]]; then
+ if [[ ${model_name} =~ "PPShiTu" ]]; then
cls_inference_model_url=$(func_parser_value "${lines[3]}")
cls_tar_name=$(func_get_url_file_name "${cls_inference_model_url}")
det_inference_model_url=$(func_parser_value "${lines[4]}")
@@ -283,6 +290,8 @@ if [[ ${MODE} = "serving_infer" ]]; then
cd ./deploy
wget -nc https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/data/drink_dataset_v1.0.tar --no-check-certificate
tar -xf drink_dataset_v1.0.tar
+ wget -nc https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/data/drink_dataset_v2.0.tar --no-check-certificate
+ tar -xf drink_dataset_v2.0.tar
mkdir models
cd models
wget -nc ${cls_inference_model_url} && tar xf ${cls_tar_name}
@@ -314,8 +323,9 @@ if [[ ${MODE} = "paddle2onnx_infer" ]]; then
# prepare paddle2onnx env
python_name=$(func_parser_value "${lines[2]}")
inference_model_url=$(func_parser_value "${lines[10]}")
- tar_name=${inference_model_url##*/}
-
+ tar_name=$(func_get_url_file_name "$inference_model_url")
+
+ ${python_name} -m pip install onnx
${python_name} -m pip install paddle2onnx
${python_name} -m pip install onnxruntime
if [[ ${model_name} =~ "GeneralRecognition" ]]; then
@@ -332,14 +342,12 @@ if [[ ${MODE} = "paddle2onnx_infer" ]]; then
rm -rf val_list.txt
ln -s demo_test.txt val_list.txt
cd ../../
- eval "wget -nc $model_url_value --no-check-certificate"
- mv general_PPLCNet_x2_5_pretrained_v1.0.pdparams GeneralRecognition_PPLCNet_x2_5_pretrained.pdparams
fi
cd deploy
mkdir models
cd models
wget -nc ${inference_model_url}
- tar xf ${tar_name}
+ eval "tar -xf ${tar_name}"
cd ../../
fi
diff --git a/test_tipc/static/ResNet50/N1C1/ResNet50_bs128_amp_fp16_DP.sh b/test_tipc/static/ResNet50/N1C1/ResNet50_bs128_amp_fp16_DP.sh
index fabbb9fe62d23aea28d9b09a76bf3ce9174c3fb3..59bb145171eb739bd8f07897d150fb955c02fa50 100644
--- a/test_tipc/static/ResNet50/N1C1/ResNet50_bs128_amp_fp16_DP.sh
+++ b/test_tipc/static/ResNet50/N1C1/ResNet50_bs128_amp_fp16_DP.sh
@@ -4,7 +4,7 @@ fp_item=amp_fp16
run_mode=DP
device_num=N1C1
max_epochs=1
-num_workers=8
+num_workers=4
# get data
bash test_tipc/static/${model_item}/benchmark_common/prepare.sh
diff --git a/test_tipc/static/ResNet50/N1C1/ResNet50_bs128_pure_fp16_DP.sh b/test_tipc/static/ResNet50/N1C1/ResNet50_bs128_pure_fp16_DP.sh
index 86242916f4fbf915f835bcc3f09178ed0fed5722..d4cc04fa451aef18dd856ea8cb8cd7ba7f67752c 100644
--- a/test_tipc/static/ResNet50/N1C1/ResNet50_bs128_pure_fp16_DP.sh
+++ b/test_tipc/static/ResNet50/N1C1/ResNet50_bs128_pure_fp16_DP.sh
@@ -4,7 +4,7 @@ fp_item=pure_fp16
run_mode=DP
device_num=N1C1
max_epochs=1
-num_workers=8
+num_workers=4
# get data
bash test_tipc/static/${model_item}/benchmark_common/prepare.sh
diff --git a/test_tipc/static/ResNet50/N1C1/ResNet50_bs256_amp_fp16_DP.sh b/test_tipc/static/ResNet50/N1C1/ResNet50_bs256_amp_fp16_DP.sh
index 878cd119e1b2b108d6d7329b4ee650c9f7e9c926..05b924d2eb95c6a1b67e731e59fd77694a23bec0 100644
--- a/test_tipc/static/ResNet50/N1C1/ResNet50_bs256_amp_fp16_DP.sh
+++ b/test_tipc/static/ResNet50/N1C1/ResNet50_bs256_amp_fp16_DP.sh
@@ -4,7 +4,7 @@ fp_item=amp_fp16
run_mode=DP
device_num=N1C1
max_epochs=1
-num_workers=8
+num_workers=4
# get data
bash test_tipc/static/${model_item}/benchmark_common/prepare.sh
diff --git a/test_tipc/static/ResNet50/N1C1/ResNet50_bs256_pure_fp16_DP.sh b/test_tipc/static/ResNet50/N1C1/ResNet50_bs256_pure_fp16_DP.sh
index 47e94fb4b8df66fcf14dcd9f0a9e644f47ed4f92..1eb86bbac5c76812c57b1d7d4f4f9131a9284938 100644
--- a/test_tipc/static/ResNet50/N1C1/ResNet50_bs256_pure_fp16_DP.sh
+++ b/test_tipc/static/ResNet50/N1C1/ResNet50_bs256_pure_fp16_DP.sh
@@ -4,7 +4,7 @@ fp_item=pure_fp16
run_mode=DP
device_num=N1C1
max_epochs=1
-num_workers=8
+num_workers=4
# get data
bash test_tipc/static/${model_item}/benchmark_common/prepare.sh
diff --git a/test_tipc/static/ResNet50/N1C1/ResNet50_bs64_amp_fp16_DP.sh b/test_tipc/static/ResNet50/N1C1/ResNet50_bs64_amp_fp16_DP.sh
index 2d6f17c87589963bbadfdad64d5a698b9258c03f..39d0aa9248260398f6423fd4336540d78c5b1639 100644
--- a/test_tipc/static/ResNet50/N1C1/ResNet50_bs64_amp_fp16_DP.sh
+++ b/test_tipc/static/ResNet50/N1C1/ResNet50_bs64_amp_fp16_DP.sh
@@ -4,7 +4,7 @@ fp_item=amp_fp16
run_mode=DP
device_num=N1C1
max_epochs=1
-num_workers=8
+num_workers=4
# get data
bash test_tipc/static/${model_item}/benchmark_common/prepare.sh
diff --git a/test_tipc/static/ResNet50/N1C1/ResNet50_bs64_pure_fp16_DP.sh b/test_tipc/static/ResNet50/N1C1/ResNet50_bs64_pure_fp16_DP.sh
index 52b8ccc5a3c18f45062d8c06b20fc8b523eec48d..285255a3deb2d8b5edaf83e4bc578d78d08ccba1 100644
--- a/test_tipc/static/ResNet50/N1C1/ResNet50_bs64_pure_fp16_DP.sh
+++ b/test_tipc/static/ResNet50/N1C1/ResNet50_bs64_pure_fp16_DP.sh
@@ -4,7 +4,7 @@ fp_item=pure_fp16
run_mode=DP
device_num=N1C1
max_epochs=1
-num_workers=8
+num_workers=4
# get data
bash test_tipc/static/${model_item}/benchmark_common/prepare.sh
diff --git a/test_tipc/static/ResNet50/N1C8/ResNet50_bs128_amp_fp16_DP.sh b/test_tipc/static/ResNet50/N1C8/ResNet50_bs128_amp_fp16_DP.sh
index e12494ff5e3ddaed236835cc7d6d6cbae33b69af..b16379ca65295d46754df3d2167e7158a3b3f740 100644
--- a/test_tipc/static/ResNet50/N1C8/ResNet50_bs128_amp_fp16_DP.sh
+++ b/test_tipc/static/ResNet50/N1C8/ResNet50_bs128_amp_fp16_DP.sh
@@ -4,7 +4,7 @@ fp_item=amp_fp16
run_mode=DP
device_num=N1C8
max_epochs=8
-num_workers=8
+num_workers=4
# get data
bash test_tipc/static/${model_item}/benchmark_common/prepare.sh
diff --git a/test_tipc/static/ResNet50/N1C8/ResNet50_bs128_pure_fp16_DP.sh b/test_tipc/static/ResNet50/N1C8/ResNet50_bs128_pure_fp16_DP.sh
index 08503177f89e75365ba0af8c26d69452067b9146..1965cd48b47008c2e140c149956c0358d1af9350 100644
--- a/test_tipc/static/ResNet50/N1C8/ResNet50_bs128_pure_fp16_DP.sh
+++ b/test_tipc/static/ResNet50/N1C8/ResNet50_bs128_pure_fp16_DP.sh
@@ -4,7 +4,7 @@ fp_item=pure_fp16
run_mode=DP
device_num=N1C8
max_epochs=8
-num_workers=8
+num_workers=4
# get data
bash test_tipc/static/${model_item}/benchmark_common/prepare.sh
diff --git a/test_tipc/static/ResNet50/N1C8/ResNet50_bs256_amp_fp16_DP.sh b/test_tipc/static/ResNet50/N1C8/ResNet50_bs256_amp_fp16_DP.sh
index ecabf01177abc7781ad04690f6dfdb26aa1fdc3e..f3f95f6987641b67d087fa2ac8cdd6d92889b0ca 100644
--- a/test_tipc/static/ResNet50/N1C8/ResNet50_bs256_amp_fp16_DP.sh
+++ b/test_tipc/static/ResNet50/N1C8/ResNet50_bs256_amp_fp16_DP.sh
@@ -4,7 +4,7 @@ fp_item=amp_fp16
run_mode=DP
device_num=N1C8
max_epochs=8
-num_workers=8
+num_workers=4
# get data
bash test_tipc/static/${model_item}/benchmark_common/prepare.sh
diff --git a/test_tipc/static/ResNet50/N1C8/ResNet50_bs256_pure_fp16_DP.sh b/test_tipc/static/ResNet50/N1C8/ResNet50_bs256_pure_fp16_DP.sh
index 0296cb7b200b959795840359bc734653471cb788..3f02f44576b5cf18e1932342bdc50e8b74b4d7d1 100644
--- a/test_tipc/static/ResNet50/N1C8/ResNet50_bs256_pure_fp16_DP.sh
+++ b/test_tipc/static/ResNet50/N1C8/ResNet50_bs256_pure_fp16_DP.sh
@@ -4,7 +4,7 @@ fp_item=pure_fp16
run_mode=DP
device_num=N1C8
max_epochs=8
-num_workers=8
+num_workers=4
# get data
bash test_tipc/static/${model_item}/benchmark_common/prepare.sh
diff --git a/test_tipc/static/ResNet50/N1C8/ResNet50_bs64_amp_fp16_DP.sh b/test_tipc/static/ResNet50/N1C8/ResNet50_bs64_amp_fp16_DP.sh
index ba666114be5fd7053efafc0ab8a6bcdc43513457..b5b8e4c685d2e1032b1f9c5d9a505b896862f5cd 100644
--- a/test_tipc/static/ResNet50/N1C8/ResNet50_bs64_amp_fp16_DP.sh
+++ b/test_tipc/static/ResNet50/N1C8/ResNet50_bs64_amp_fp16_DP.sh
@@ -4,7 +4,7 @@ fp_item=amp_fp16
run_mode=DP
device_num=N1C8
max_epochs=8
-num_workers=8
+num_workers=4
# get data
bash test_tipc/static/${model_item}/benchmark_common/prepare.sh
diff --git a/test_tipc/static/ResNet50/N1C8/ResNet50_bs64_pure_fp16_DP.sh b/test_tipc/static/ResNet50/N1C8/ResNet50_bs64_pure_fp16_DP.sh
index a7cd97ea80d30165729d19fa9f0b0a89cdf7e093..50b6fd7f886beb55174b254ef0af0871842ece73 100644
--- a/test_tipc/static/ResNet50/N1C8/ResNet50_bs64_pure_fp16_DP.sh
+++ b/test_tipc/static/ResNet50/N1C8/ResNet50_bs64_pure_fp16_DP.sh
@@ -4,7 +4,7 @@ fp_item=pure_fp16
run_mode=DP
device_num=N1C8
max_epochs=8
-num_workers=8
+num_workers=4
# get data
bash test_tipc/static/${model_item}/benchmark_common/prepare.sh
diff --git a/test_tipc/static/ResNet50/N4C32/ResNet50_bs128_amp_fp16_DP.sh b/test_tipc/static/ResNet50/N4C32/ResNet50_bs128_amp_fp16_DP.sh
index c96d5603d03ebacafa02c3fb49e0c8bb6b89f531..e5a1d7e5f8b4363c2d936723b1b14f4cd44d6058 100644
--- a/test_tipc/static/ResNet50/N4C32/ResNet50_bs128_amp_fp16_DP.sh
+++ b/test_tipc/static/ResNet50/N4C32/ResNet50_bs128_amp_fp16_DP.sh
@@ -4,7 +4,7 @@ fp_item=amp_fp16
run_mode=DP
device_num=N4C32
max_epochs=32
-num_workers=8
+num_workers=4
# get data
bash test_tipc/static/${model_item}/benchmark_common/prepare.sh
diff --git a/test_tipc/static/ResNet50/N4C32/ResNet50_bs128_pure_fp16_DP.sh b/test_tipc/static/ResNet50/N4C32/ResNet50_bs128_pure_fp16_DP.sh
index ef3876f1d3348c21df1de3526e162cc2ef34d8d1..7beb72e877a12e17f5c0eaacec15262cfa88ab3b 100644
--- a/test_tipc/static/ResNet50/N4C32/ResNet50_bs128_pure_fp16_DP.sh
+++ b/test_tipc/static/ResNet50/N4C32/ResNet50_bs128_pure_fp16_DP.sh
@@ -4,7 +4,7 @@ fp_item=pure_fp16
run_mode=DP
device_num=N4C32
max_epochs=32
-num_workers=8
+num_workers=4
# get data
bash test_tipc/static/${model_item}/benchmark_common/prepare.sh
diff --git a/test_tipc/static/ResNet50/N4C32/ResNet50_bs256_amp_fp16_DP.sh b/test_tipc/static/ResNet50/N4C32/ResNet50_bs256_amp_fp16_DP.sh
index f9f2f76665cb030fbf11414d4f313345a539b12c..a0ebe2d0e42d79938b0aa36f3a0995e3d24f2278 100644
--- a/test_tipc/static/ResNet50/N4C32/ResNet50_bs256_amp_fp16_DP.sh
+++ b/test_tipc/static/ResNet50/N4C32/ResNet50_bs256_amp_fp16_DP.sh
@@ -4,7 +4,7 @@ fp_item=amp_fp16
run_mode=DP
device_num=N4C32
max_epochs=32
-num_workers=8
+num_workers=4
# get data
bash test_tipc/static/${model_item}/benchmark_common/prepare.sh
diff --git a/test_tipc/static/ResNet50/N4C32/ResNet50_bs256_pure_fp16_DP.sh b/test_tipc/static/ResNet50/N4C32/ResNet50_bs256_pure_fp16_DP.sh
index bef8186ea5e10feda6d62e1df01a41303b5c3469..054693b123df98993d0dac490e8832be9a1a74fc 100644
--- a/test_tipc/static/ResNet50/N4C32/ResNet50_bs256_pure_fp16_DP.sh
+++ b/test_tipc/static/ResNet50/N4C32/ResNet50_bs256_pure_fp16_DP.sh
@@ -4,7 +4,7 @@ fp_item=pure_fp16
run_mode=DP
device_num=N4C32
max_epochs=32
-num_workers=8
+num_workers=4
# get data
bash test_tipc/static/${model_item}/benchmark_common/prepare.sh
diff --git a/test_tipc/static/ResNet50/N4C32/ResNet50_bs64_amp_fp16_DP.sh b/test_tipc/static/ResNet50/N4C32/ResNet50_bs64_amp_fp16_DP.sh
index 58f68d234d91bd98f4027d739070b045cd33a235..cd318ff71d3c58c3cb138baa595098cfbf3b9f2f 100644
--- a/test_tipc/static/ResNet50/N4C32/ResNet50_bs64_amp_fp16_DP.sh
+++ b/test_tipc/static/ResNet50/N4C32/ResNet50_bs64_amp_fp16_DP.sh
@@ -4,7 +4,7 @@ fp_item=amp_fp16
run_mode=DP
device_num=N4C32
max_epochs=32
-num_workers=8
+num_workers=4
# get data
bash test_tipc/static/${model_item}/benchmark_common/prepare.sh
diff --git a/test_tipc/static/ResNet50/N4C32/ResNet50_bs64_pure_fp16_DP.sh b/test_tipc/static/ResNet50/N4C32/ResNet50_bs64_pure_fp16_DP.sh
index c764a79e574bfaa5356d180321537dcb6d1fb5bb..75ca5d05270b09e39363a2a372909a43106a3a4e 100644
--- a/test_tipc/static/ResNet50/N4C32/ResNet50_bs64_pure_fp16_DP.sh
+++ b/test_tipc/static/ResNet50/N4C32/ResNet50_bs64_pure_fp16_DP.sh
@@ -4,7 +4,7 @@ fp_item=pure_fp16
run_mode=DP
device_num=N4C32
max_epochs=32
-num_workers=8
+num_workers=4
# get data
bash test_tipc/static/${model_item}/benchmark_common/prepare.sh
diff --git a/test_tipc/test_inference_cpp.sh b/test_tipc/test_inference_cpp.sh
index 24d406b8f06e6fa38385b2252d37d35a87ecffbb..c56bc69602819cbbe3e7eeced13841cec0156085 100644
--- a/test_tipc/test_inference_cpp.sh
+++ b/test_tipc/test_inference_cpp.sh
@@ -37,7 +37,8 @@ cpp_benchmark_value=$(func_parser_value "${lines[16]}")
generate_yaml_cmd=$(func_parser_value "${lines[17]}")
transform_index_cmd=$(func_parser_value "${lines[18]}")
-LOG_PATH="./test_tipc/output/${model_name}/${MODE}"
+CLS_ROOT_PATH=$(pwd)
+LOG_PATH="${CLS_ROOT_PATH}/test_tipc/output/${model_name}/${MODE}"
mkdir -p ${LOG_PATH}
status_log="${LOG_PATH}/results_cpp.log"
# generate_yaml_cmd="python3 test_tipc/generate_cpp_yaml.py"
@@ -70,7 +71,7 @@ function func_shitu_cpp_inference(){
command="${_script} > ${_save_log_path} 2>&1"
eval $command
last_status=${PIPESTATUS[0]}
- status_check $last_status "${command}" "${status_log}" "${model_name}"
+ status_check $last_status "${command}" "${status_log}" "${model_name}" "${_save_log_path}"
done
done
done
@@ -94,7 +95,7 @@ function func_shitu_cpp_inference(){
command="${_script} > ${_save_log_path} 2>&1"
eval $command
last_status=${PIPESTATUS[0]}
- status_check $last_status "${command}" "${status_log}" "${model_name}"
+ status_check $last_status "${command}" "${status_log}" "${model_name}" "${_save_log_path}"
done
done
done
@@ -126,13 +127,12 @@ function func_cls_cpp_inference(){
precison="int8"
fi
_save_log_path="${_log_path}/cpp_infer_cpu_usemkldnn_${use_mkldnn}_threads_${threads}_precision_${precision}_batchsize_${batch_size}.log"
-
command="${generate_yaml_cmd} --type cls --batch_size ${batch_size} --mkldnn ${use_mkldnn} --gpu ${use_gpu} --cpu_thread ${threads} --tensorrt False --precision ${precision} --data_dir ${_img_dir} --benchmark True --cls_model_dir ${cpp_infer_model_dir} --gpu_id ${GPUID}"
eval $command
command1="${_script} > ${_save_log_path} 2>&1"
eval ${command1}
last_status=${PIPESTATUS[0]}
- status_check $last_status "${command1}" "${status_log}" "${model_name}"
+ status_check $last_status "${command1}" "${status_log}" "${model_name}" "${_save_log_path}"
done
done
done
@@ -155,7 +155,7 @@ function func_cls_cpp_inference(){
command="${_script} > ${_save_log_path} 2>&1"
eval $command
last_status=${PIPESTATUS[0]}
- status_check $last_status "${command}" "${status_log}" "${model_name}"
+ status_check $last_status "${command}" "${status_log}" "${model_name}" "${_save_log_path}"
done
done
done
diff --git a/test_tipc/test_inference_jeston.sh b/test_tipc/test_inference_jeston.sh
index 56845003908c1a9cc8ac1b76e40ec108d33e8478..7fc8adf5b772264237bbd0070b7d98d1aee13027 100644
--- a/test_tipc/test_inference_jeston.sh
+++ b/test_tipc/test_inference_jeston.sh
@@ -42,7 +42,8 @@ infer_key1=$(func_parser_key "${lines[17]}")
infer_value1=$(func_parser_value "${lines[17]}")
-LOG_PATH="./test_tipc/output"
+CLS_ROOT_PATH=$(pwd)
+LOG_PATH="${CLS_ROOT_PATH}/test_tipc/output"
mkdir -p ${LOG_PATH}
status_log="${LOG_PATH}/results_python.log"
@@ -71,7 +72,7 @@ if [ ${MODE} = "whole_infer" ]; then
echo $export_cmd
eval $export_cmd
status_export=$?
- status_check $status_export "${export_cmd}" "${status_log}" "${model_name}"
+ status_check $status_export "${export_cmd}" "${status_log}" "${model_name}" ""
else
save_infer_dir=${infer_model}
fi
diff --git a/test_tipc/test_lite_arm_cpu_cpp.sh b/test_tipc/test_lite_arm_cpu_cpp.sh
index 919226eea5ce38b82fad6c2130a7c6467b6ee041..07fcbe209de9d8eb88580130e4077e8c0fa6063d 100644
--- a/test_tipc/test_lite_arm_cpu_cpp.sh
+++ b/test_tipc/test_lite_arm_cpu_cpp.sh
@@ -1,6 +1,5 @@
#!/bin/bash
source test_tipc/common_func.sh
-current_path=$PWD
IFS=$'\n'
@@ -33,7 +32,8 @@ num_threads_list=$(func_parser_value_lite "${tipc_lines[5]}" ":")
batch_size_list=$(func_parser_value_lite "${tipc_lines[6]}" ":")
precision_list=$(func_parser_value_lite "${tipc_lines[7]}" ":")
-LOG_PATH=${current_path}"/output"
+CLS_ROOT_PATH=$(pwd)
+LOG_PATH="${CLS_ROOT_PATH}/output"
mkdir -p ${LOG_PATH}
status_log="${LOG_PATH}/results.log"
@@ -65,9 +65,9 @@ function func_test_tipc(){
real_inference_cmd=$(echo ${inference_cmd} | awk -F " " '{print path $1" "path $2" "path $3}' path="$lite_arm_work_path")
command1="adb push ${_basic_config} ${lite_arm_work_path}"
eval ${command1}
- command2="adb shell 'export LD_LIBRARY_PATH=${lite_arm_work_path}; ${real_inference_cmd}' > ${_save_log_path} 2>&1"
+ command2="adb shell 'export LD_LIBRARY_PATH=${lite_arm_work_path}; ${real_inference_cmd}' > ${_save_log_path} 2>&1"
eval ${command2}
- status_check $? "${command2}" "${status_log}" "${model_name}"
+ status_check $? "${command2}" "${status_log}" "${model_name}" "${_save_log_path}"
done
done
done
diff --git a/test_tipc/test_paddle2onnx.sh b/test_tipc/test_paddle2onnx.sh
index d025fb2efd672baab42e4617a13dd127d90a73bc..d5687e1546a94bce1304575a7bcf50f718cfdada 100644
--- a/test_tipc/test_paddle2onnx.sh
+++ b/test_tipc/test_paddle2onnx.sh
@@ -2,7 +2,7 @@
source test_tipc/common_func.sh
FILENAME=$1
-MODE=$2
+MODE="paddle2onnx_infer"
# parser params
dataline=$(awk 'NR==1, NR==16{print}' $FILENAME)
@@ -36,7 +36,8 @@ inference_hardware_value=$(func_parser_value "${lines[14]}")
inference_config_key=$(func_parser_key "${lines[15]}")
inference_config_value=$(func_parser_value "${lines[15]}")
-LOG_PATH="./test_tipc/output/${model_name}/${MODE}"
+CLS_ROOT_PATH=$(pwd)
+LOG_PATH="${CLS_ROOT_PATH}/test_tipc/output/${model_name}/${MODE}"
mkdir -p ${LOG_PATH}
status_log="${LOG_PATH}/results_paddle2onnx.log"
@@ -46,27 +47,29 @@ function func_paddle2onnx(){
_script=$1
# paddle2onnx
- _save_log_path=".${LOG_PATH}/paddle2onnx_infer_cpu.log"
set_dirname=$(func_set_params "${infer_model_dir_key}" "${infer_model_dir_value}")
set_model_filename=$(func_set_params "${model_filename_key}" "${model_filename_value}")
set_params_filename=$(func_set_params "${params_filename_key}" "${params_filename_value}")
set_save_model=$(func_set_params "${save_file_key}" "${save_file_value}")
set_opset_version=$(func_set_params "${opset_version_key}" "${opset_version_value}")
set_enable_onnx_checker=$(func_set_params "${enable_onnx_checker_key}" "${enable_onnx_checker_value}")
- trans_model_cmd="${padlle2onnx_cmd} ${set_dirname} ${set_model_filename} ${set_params_filename} ${set_save_model} ${set_opset_version} ${set_enable_onnx_checker}"
+ trans_log="${LOG_PATH}/trans_model.log"
+ trans_model_cmd="${padlle2onnx_cmd} ${set_dirname} ${set_model_filename} ${set_params_filename} ${set_save_model} ${set_opset_version} ${set_enable_onnx_checker} --enable_dev_version=False > ${trans_log} 2>&1"
eval $trans_model_cmd
last_status=${PIPESTATUS[0]}
- status_check $last_status "${trans_model_cmd}" "${status_log}" "${model_name}"
+ status_check $last_status "${trans_model_cmd}" "${status_log}" "${model_name}" "${trans_log}"
# python inference
if [[ ${inference_py} != "null" ]]; then
+ _save_log_path="${LOG_PATH}/paddle2onnx_infer_cpu.log"
set_model_dir=$(func_set_params "${inference_model_dir_key}" "${inference_model_dir_value}")
set_use_onnx=$(func_set_params "${use_onnx_key}" "${use_onnx_value}")
set_hardware=$(func_set_params "${inference_hardware_key}" "${inference_hardware_value}")
set_inference_config=$(func_set_params "${inference_config_key}" "${inference_config_value}")
+
infer_model_cmd="cd deploy && ${python} ${inference_py} -o ${set_model_dir} -o ${set_use_onnx} -o ${set_hardware} ${set_inference_config} > ${_save_log_path} 2>&1 && cd ../"
eval $infer_model_cmd
- status_check $last_status "${infer_model_cmd}" "${status_log}" "${model_name}"
+ status_check $last_status "${infer_model_cmd}" "${status_log}" "${model_name}" "${_save_log_path}"
fi
}
@@ -75,4 +78,4 @@ echo "################### run test ###################"
export Count=0
IFS="|"
-func_paddle2onnx
\ No newline at end of file
+func_paddle2onnx
diff --git a/test_tipc/test_ptq_inference_python.sh b/test_tipc/test_ptq_inference_python.sh
index 82c9816478f9ea993b2e53f8a685766e8dbf81d7..e6801f640b07df5f4694ef943404974f4fbf07fc 100644
--- a/test_tipc/test_ptq_inference_python.sh
+++ b/test_tipc/test_ptq_inference_python.sh
@@ -94,7 +94,8 @@ if [[ $MODE = 'benchmark_train' ]]; then
epoch_num=1
fi
-LOG_PATH="./test_tipc/output/${model_name}/${MODE}"
+CLS_ROOT_PATH=$(pwd)
+LOG_PATH="${CLS_ROOT_PATH}/test_tipc/output/${model_name}/${MODE}"
mkdir -p ${LOG_PATH}
status_log="${LOG_PATH}/results_python.log"
@@ -123,7 +124,7 @@ function func_inference() {
eval $command
last_status=${PIPESTATUS[0]}
eval "cat ${_save_log_path}"
- status_check $last_status "${command}" "../${status_log}" "${model_name}"
+ status_check $last_status "${command}" "${status_log}" "${model_name}"
done
done
done
@@ -145,7 +146,7 @@ function func_inference() {
eval $command
last_status=${PIPESTATUS[0]}
eval "cat ${_save_log_path}"
- status_check $last_status "${command}" "../${status_log}" "${model_name}"
+ status_check $last_status "${command}" "${status_log}" "${model_name}"
done
done
done
@@ -168,6 +169,6 @@ if [ ${kl_quant_cmd_value} != "null" ] && [ ${kl_quant_cmd_value} != "False" ];
ln -s __params__ inference.pdiparams
cd ../../deploy
is_quant=True
- func_inference "${python}" "${inference_py}" "../${infer_model_dir_list}/quant_post_static_model" "../${LOG_PATH}" "${infer_img_dir}" ${is_quant}
+ func_inference "${python}" "${inference_py}" "../${infer_model_dir_list}/quant_post_static_model" "${LOG_PATH}" "${infer_img_dir}" ${is_quant}
cd ..
fi
diff --git a/test_tipc/test_serving_infer_cpp.sh b/test_tipc/test_serving_infer_cpp.sh
index fdb7ef186bafd9dcd879150188e1f0450ca87211..01e5601cf10f2cf86c9396883f9baef011ca6ba0 100644
--- a/test_tipc/test_serving_infer_cpp.sh
+++ b/test_tipc/test_serving_infer_cpp.sh
@@ -38,10 +38,10 @@ pipeline_py=$(func_parser_value "${lines[13]}")
function func_serving_cls(){
- LOG_PATH="test_tipc/output/${model_name}"
+ CLS_ROOT_PATH=$(pwd)
+ LOG_PATH="${CLS_ROOT_PATH}/test_tipc/output/${model_name}/serving_infer"
mkdir -p ${LOG_PATH}
- LOG_PATH="../../${LOG_PATH}"
- status_log="${LOG_PATH}/results_serving.log"
+ status_log="${LOG_PATH}/results_cpp_serving.log"
IFS='|'
# pdserving
@@ -53,8 +53,11 @@ function func_serving_cls(){
for python_ in ${python[*]}; do
if [[ ${python_} =~ "python" ]]; then
- trans_model_cmd="${python_} ${trans_model_py} ${set_dirname} ${set_model_filename} ${set_params_filename} ${set_serving_server} ${set_serving_client}"
+ trans_log="${LOG_PATH}/cpp_trans_model.log"
+ trans_model_cmd="${python_} ${trans_model_py} ${set_dirname} ${set_model_filename} ${set_params_filename} ${set_serving_server} ${set_serving_client} > ${trans_log} 2>&1"
eval ${trans_model_cmd}
+ last_status=${PIPESTATUS[0]}
+ status_check $last_status "${trans_model_cmd}" "${status_log}" "${model_name}" "${trans_log}"
break
fi
done
@@ -102,32 +105,34 @@ function func_serving_cls(){
for use_gpu in ${web_use_gpu_list[*]}; do
if [[ ${use_gpu} = "null" ]]; then
- web_service_cpp_cmd="${python_} -m paddle_serving_server.serve --model ${serving_server_dir_name} --op GeneralClasOp --port 9292 &"
+ server_log_path="${LOG_PATH}/cpp_server_cpu.log"
+ web_service_cpp_cmd="nohup ${python_} -m paddle_serving_server.serve --model ${serving_server_dir_name} --op GeneralClasOp --port 9292 > ${server_log_path} 2>&1 &"
eval ${web_service_cpp_cmd}
last_status=${PIPESTATUS[0]}
- status_check $last_status "${web_service_cpp_cmd}" "${status_log}" "${model_name}"
+ status_check $last_status "${web_service_cpp_cmd}" "${status_log}" "${model_name}" "${server_log_path}"
sleep 5s
- _save_log_path="${LOG_PATH}/server_infer_cpp_cpu_pipeline_batchsize_1.log"
+ _save_log_path="${LOG_PATH}/cpp_client_cpu.log"
pipeline_cmd="${python_} test_cpp_serving_client.py > ${_save_log_path} 2>&1 "
eval ${pipeline_cmd}
last_status=${PIPESTATUS[0]}
eval "cat ${_save_log_path}"
- status_check ${last_status} "${pipeline_cmd}" "${status_log}" "${model_name}"
+ status_check ${last_status} "${pipeline_cmd}" "${status_log}" "${model_name}" "${_save_log_path}"
eval "${python_} -m paddle_serving_server.serve stop"
sleep 5s
else
- web_service_cpp_cmd="${python_} -m paddle_serving_server.serve --model ${serving_server_dir_name} --op GeneralClasOp --port 9292 --gpu_id=${use_gpu} &"
+ server_log_path="${LOG_PATH}/cpp_server_gpu.log"
+ web_service_cpp_cmd="nohup ${python_} -m paddle_serving_server.serve --model ${serving_server_dir_name} --op GeneralClasOp --port 9292 --gpu_id=${use_gpu} > ${server_log_path} 2>&1 &"
eval ${web_service_cpp_cmd}
last_status=${PIPESTATUS[0]}
- status_check $last_status "${web_service_cpp_cmd}" "${status_log}" "${model_name}"
+ status_check $last_status "${web_service_cpp_cmd}" "${status_log}" "${model_name}" "${server_log_path}"
sleep 8s
- _save_log_path="${LOG_PATH}/server_infer_cpp_gpu_pipeline_batchsize_1.log"
+ _save_log_path="${LOG_PATH}/cpp_client_gpu.log"
pipeline_cmd="${python_} test_cpp_serving_client.py > ${_save_log_path} 2>&1 "
eval ${pipeline_cmd}
last_status=${PIPESTATUS[0]}
eval "cat ${_save_log_path}"
- status_check ${last_status} "${pipeline_cmd}" "${status_log}" "${model_name}"
+ status_check ${last_status} "${pipeline_cmd}" "${status_log}" "${model_name}" "${_save_log_path}"
sleep 5s
eval "${python_} -m paddle_serving_server.serve stop"
fi
@@ -136,10 +141,11 @@ function func_serving_cls(){
function func_serving_rec(){
- LOG_PATH="test_tipc/output/${model_name}"
+ CLS_ROOT_PATH=$(pwd)
+ LOG_PATH="${CLS_ROOT_PATH}/test_tipc/output/${model_name}/serving_infer"
mkdir -p ${LOG_PATH}
- LOG_PATH="../../../${LOG_PATH}"
- status_log="${LOG_PATH}/results_serving.log"
+ status_log="${LOG_PATH}/results_cpp_serving.log"
+
trans_model_py=$(func_parser_value "${lines[5]}")
cls_infer_model_dir_key=$(func_parser_key "${lines[6]}")
cls_infer_model_dir_value=$(func_parser_value "${lines[6]}")
@@ -181,20 +187,36 @@ function func_serving_rec(){
set_params_filename=$(func_set_params "${params_filename_key}" "${params_filename_value}")
set_serving_server=$(func_set_params "${cls_serving_server_key}" "${cls_serving_server_value}")
set_serving_client=$(func_set_params "${cls_serving_client_key}" "${cls_serving_client_value}")
- cls_trans_model_cmd="${python_interp} ${trans_model_py} ${set_dirname} ${set_model_filename} ${set_params_filename} ${set_serving_server} ${set_serving_client}"
+ trans_cls_log="${LOG_PATH}/cpp_trans_model_cls.log"
+ cls_trans_model_cmd="${python_interp} ${trans_model_py} ${set_dirname} ${set_model_filename} ${set_params_filename} ${set_serving_server} ${set_serving_client} > ${trans_cls_log} 2>&1"
eval ${cls_trans_model_cmd}
+ last_status=${PIPESTATUS[0]}
+ status_check $last_status "${cls_trans_model_cmd}" "${status_log}" "${model_name}" "${trans_cls_log}"
set_dirname=$(func_set_params "${det_infer_model_dir_key}" "${det_infer_model_dir_value}")
set_model_filename=$(func_set_params "${model_filename_key}" "${model_filename_value}")
set_params_filename=$(func_set_params "${params_filename_key}" "${params_filename_value}")
set_serving_server=$(func_set_params "${det_serving_server_key}" "${det_serving_server_value}")
set_serving_client=$(func_set_params "${det_serving_client_key}" "${det_serving_client_value}")
- det_trans_model_cmd="${python_interp} ${trans_model_py} ${set_dirname} ${set_model_filename} ${set_params_filename} ${set_serving_server} ${set_serving_client}"
+ trans_det_log="${LOG_PATH}/cpp_trans_model_det.log"
+ det_trans_model_cmd="${python_interp} ${trans_model_py} ${set_dirname} ${set_model_filename} ${set_params_filename} ${set_serving_server} ${set_serving_client} > ${trans_det_log} 2>&1"
eval ${det_trans_model_cmd}
-
- cp_prototxt_cmd="cp ./paddleserving/recognition/preprocess/general_PPLCNet_x2_5_lite_v1.0_serving/*.prototxt ${cls_serving_server_value}"
+ last_status=${PIPESTATUS[0]}
+ status_check $last_status "${det_trans_model_cmd}" "${status_log}" "${model_name}" "${trans_det_log}"
+
+ OLD_IFS="${IFS}"
+ IFS='/'
+ tmp_arr=($cls_serving_server_value)
+ lastIndex=$((${#tmp_arr[@]}-1))
+ cls_serving_server_dirname="${tmp_arr[lastIndex]}"
+ tmp_arr=($cls_serving_client_value)
+ lastIndex=$((${#tmp_arr[@]}-1))
+ cls_serving_client_dirname="${tmp_arr[lastIndex]}"
+ IFS="${OLD_IFS}"
+
+ cp_prototxt_cmd="cp ./paddleserving/recognition/preprocess/${cls_serving_server_dirname}/*.prototxt ${cls_serving_server_value}"
eval ${cp_prototxt_cmd}
- cp_prototxt_cmd="cp ./paddleserving/recognition/preprocess/general_PPLCNet_x2_5_lite_v1.0_client/*.prototxt ${cls_serving_client_value}"
+ cp_prototxt_cmd="cp ./paddleserving/recognition/preprocess/${cls_serving_client_dirname}/*.prototxt ${cls_serving_client_value}"
eval ${cp_prototxt_cmd}
cp_prototxt_cmd="cp ./paddleserving/recognition/preprocess/picodet_PPLCNet_x2_5_mainbody_lite_v1.0_client/*.prototxt ${det_serving_client_value}"
eval ${cp_prototxt_cmd}
@@ -215,32 +237,34 @@ function func_serving_rec(){
for use_gpu in ${web_use_gpu_list[*]}; do
if [ ${use_gpu} = "null" ]; then
det_serving_server_dir_name=$(func_get_url_file_name "$det_serving_server_value")
- web_service_cpp_cmd="${python_interp} -m paddle_serving_server.serve --model ../../${det_serving_server_value} ../../${cls_serving_server_value} --op GeneralPicodetOp GeneralFeatureExtractOp --port 9400 &"
+ server_log_path="${LOG_PATH}/cpp_server_cpu.log"
+ web_service_cpp_cmd="nohup ${python_interp} -m paddle_serving_server.serve --model ../../${det_serving_server_value} ../../${cls_serving_server_value} --op GeneralPicodetOp GeneralFeatureExtractOp --port 9400 > ${server_log_path} 2>&1 &"
eval ${web_service_cpp_cmd}
last_status=${PIPESTATUS[0]}
- status_check $last_status "${web_service_cpp_cmd}" "${status_log}" "${model_name}"
+ status_check $last_status "${web_service_cpp_cmd}" "${status_log}" "${model_name}" "${server_log_path}"
sleep 5s
- _save_log_path="${LOG_PATH}/server_infer_cpp_cpu_batchsize_1.log"
+ _save_log_path="${LOG_PATH}/cpp_client_cpu.log"
pipeline_cmd="${python_interp} ${pipeline_py} > ${_save_log_path} 2>&1 "
eval ${pipeline_cmd}
last_status=${PIPESTATUS[0]}
eval "cat ${_save_log_path}"
- status_check ${last_status} "${pipeline_cmd}" "${status_log}" "${model_name}"
+ status_check ${last_status} "${pipeline_cmd}" "${status_log}" "${model_name}" "${_save_log_path}"
eval "${python_} -m paddle_serving_server.serve stop"
sleep 5s
else
det_serving_server_dir_name=$(func_get_url_file_name "$det_serving_server_value")
+ server_log_path="${LOG_PATH}/cpp_server_gpu.log"
web_service_cpp_cmd="${python_interp} -m paddle_serving_server.serve --model ../../${det_serving_server_value} ../../${cls_serving_server_value} --op GeneralPicodetOp GeneralFeatureExtractOp --port 9400 --gpu_id=${use_gpu} &"
eval ${web_service_cpp_cmd}
last_status=${PIPESTATUS[0]}
- status_check $last_status "${web_service_cpp_cmd}" "${status_log}" "${model_name}"
+ status_check $last_status "${web_service_cpp_cmd}" "${status_log}" "${model_name}" ${server_log_path}
sleep 5s
- _save_log_path="${LOG_PATH}/server_infer_cpp_gpu_batchsize_1.log"
+ _save_log_path="${LOG_PATH}/cpp_client_gpu.log"
pipeline_cmd="${python_interp} ${pipeline_py} > ${_save_log_path} 2>&1 "
eval ${pipeline_cmd}
last_status=${PIPESTATUS[0]}
eval "cat ${_save_log_path}"
- status_check ${last_status} "${pipeline_cmd}" "${status_log}" "${model_name}"
+ status_check ${last_status} "${pipeline_cmd}" "${status_log}" "${model_name}" "${_save_log_path}"
eval "${python_} -m paddle_serving_server.serve stop"
sleep 5s
fi
diff --git a/test_tipc/test_serving_infer_python.sh b/test_tipc/test_serving_infer_python.sh
index 050c3c89c9a454eb7f973f405ded37a3f1df042a..2c5a15e0592c9668a004aef2bf828ddc7df80935 100644
--- a/test_tipc/test_serving_infer_python.sh
+++ b/test_tipc/test_serving_infer_python.sh
@@ -36,13 +36,16 @@ web_service_py=$(func_parser_value "${lines[11]}")
web_use_gpu_key=$(func_parser_key "${lines[12]}")
web_use_gpu_list=$(func_parser_value "${lines[12]}")
pipeline_py=$(func_parser_value "${lines[13]}")
+use_mkldnn="False"
+threads="1"
function func_serving_cls(){
- LOG_PATH="test_tipc/output/${model_name}/${MODE}"
+ CLS_ROOT_PATH=$(pwd)
+ LOG_PATH="${CLS_ROOT_PATH}/test_tipc/output/${model_name}/${MODE}"
mkdir -p ${LOG_PATH}
- LOG_PATH="../../${LOG_PATH}"
status_log="${LOG_PATH}/results_serving.log"
+
IFS='|'
# pdserving
@@ -54,8 +57,11 @@ function func_serving_cls(){
for python_ in ${python[*]}; do
if [[ ${python_} =~ "python" ]]; then
- trans_model_cmd="${python_} ${trans_model_py} ${set_dirname} ${set_model_filename} ${set_params_filename} ${set_serving_server} ${set_serving_client}"
+ trans_log="${LOG_PATH}/python_trans_model.log"
+ trans_model_cmd="${python_} ${trans_model_py} ${set_dirname} ${set_model_filename} ${set_params_filename} ${set_serving_server} ${set_serving_client} > ${trans_log} 2>&1"
eval ${trans_model_cmd}
+ last_status=${PIPESTATUS[0]}
+ status_check $last_status "${trans_model_cmd}" "${status_log}" "${model_name}" "${trans_log}"
break
fi
done
@@ -96,19 +102,19 @@ function func_serving_cls(){
devices_line=27
set_devices_cmd="sed -i '${devices_line}s/devices: .*/devices: \"\"/' config.yml"
eval ${set_devices_cmd}
-
- web_service_cmd="${python_} ${web_service_py} &"
+ server_log_path="${LOG_PATH}/python_server_cpu.log"
+ web_service_cmd="nohup ${python_} ${web_service_py} > ${server_log_path} 2>&1 &"
eval ${web_service_cmd}
last_status=${PIPESTATUS[0]}
- status_check $last_status "${web_service_cmd}" "${status_log}" "${model_name}"
+ status_check $last_status "${web_service_cmd}" "${status_log}" "${model_name}" "${server_log_path}"
sleep 5s
for pipeline in ${pipeline_py[*]}; do
- _save_log_path="${LOG_PATH}/server_infer_cpu_${pipeline%_client*}_batchsize_1.log"
+ _save_log_path="${LOG_PATH}/python_client_cpu_${pipeline%_client*}_usemkldnn_${use_mkldnn}_threads_${threads}_batchsize_1.log"
pipeline_cmd="${python_} ${pipeline} > ${_save_log_path} 2>&1 "
eval ${pipeline_cmd}
last_status=${PIPESTATUS[0]}
eval "cat ${_save_log_path}"
- status_check $last_status "${pipeline_cmd}" "${status_log}" "${model_name}"
+ status_check $last_status "${pipeline_cmd}" "${status_log}" "${model_name}" "${_save_log_path}"
sleep 5s
done
eval "${python_} -m paddle_serving_server.serve stop"
@@ -130,19 +136,19 @@ function func_serving_cls(){
devices_line=27
set_devices_cmd="sed -i '${devices_line}s/devices: .*/devices: \"${use_gpu}\"/' config.yml"
eval ${set_devices_cmd}
-
- web_service_cmd="${python_} ${web_service_py} & "
+ server_log_path="${LOG_PATH}/python_server_gpu_usetrt_${use_trt}_precision_${precision}.log"
+ web_service_cmd="nohup ${python_} ${web_service_py} > ${server_log_path} 2>&1 &"
eval ${web_service_cmd}
last_status=${PIPESTATUS[0]}
- status_check $last_status "${web_service_cmd}" "${status_log}" "${model_name}"
+ status_check $last_status "${web_service_cmd}" "${status_log}" "${model_name}" "${server_log_path}"
sleep 5s
for pipeline in ${pipeline_py[*]}; do
- _save_log_path="${LOG_PATH}/server_infer_gpu_${pipeline%_client*}_batchsize_1.log"
+ _save_log_path="${LOG_PATH}/python_client_gpu_${pipeline%_client*}_usetrt_${use_trt}_precision_${precision}_batchsize_1.log"
pipeline_cmd="${python_} ${pipeline} > ${_save_log_path} 2>&1"
eval ${pipeline_cmd}
last_status=${PIPESTATUS[0]}
eval "cat ${_save_log_path}"
- status_check $last_status "${pipeline_cmd}" "${status_log}" "${model_name}"
+ status_check $last_status "${pipeline_cmd}" "${status_log}" "${model_name}" "${_save_log_path}"
sleep 5s
done
eval "${python_} -m paddle_serving_server.serve stop"
@@ -154,10 +160,11 @@ function func_serving_cls(){
function func_serving_rec(){
- LOG_PATH="test_tipc/output/${model_name}/${MODE}"
+ CLS_ROOT_PATH=$(pwd)
+ LOG_PATH="${CLS_ROOT_PATH}/test_tipc/output/${model_name}/${MODE}"
mkdir -p ${LOG_PATH}
- LOG_PATH="../../../${LOG_PATH}"
status_log="${LOG_PATH}/results_serving.log"
+
trans_model_py=$(func_parser_value "${lines[5]}")
cls_infer_model_dir_key=$(func_parser_key "${lines[6]}")
cls_infer_model_dir_value=$(func_parser_value "${lines[6]}")
@@ -199,16 +206,22 @@ function func_serving_rec(){
set_params_filename=$(func_set_params "${params_filename_key}" "${params_filename_value}")
set_serving_server=$(func_set_params "${cls_serving_server_key}" "${cls_serving_server_value}")
set_serving_client=$(func_set_params "${cls_serving_client_key}" "${cls_serving_client_value}")
- cls_trans_model_cmd="${python_interp} ${trans_model_py} ${set_dirname} ${set_model_filename} ${set_params_filename} ${set_serving_server} ${set_serving_client}"
+ trans_cls_log="${LOG_PATH}/python_trans_model_cls.log"
+ cls_trans_model_cmd="${python_interp} ${trans_model_py} ${set_dirname} ${set_model_filename} ${set_params_filename} ${set_serving_server} ${set_serving_client} > ${trans_cls_log} 2>&1"
eval ${cls_trans_model_cmd}
+ last_status=${PIPESTATUS[0]}
+ status_check $last_status "${cls_trans_model_cmd}" "${status_log}" "${model_name}" "${trans_cls_log}"
set_dirname=$(func_set_params "${det_infer_model_dir_key}" "${det_infer_model_dir_value}")
set_model_filename=$(func_set_params "${model_filename_key}" "${model_filename_value}")
set_params_filename=$(func_set_params "${params_filename_key}" "${params_filename_value}")
set_serving_server=$(func_set_params "${det_serving_server_key}" "${det_serving_server_value}")
set_serving_client=$(func_set_params "${det_serving_client_key}" "${det_serving_client_value}")
- det_trans_model_cmd="${python_interp} ${trans_model_py} ${set_dirname} ${set_model_filename} ${set_params_filename} ${set_serving_server} ${set_serving_client}"
+ trans_det_log="${LOG_PATH}/python_trans_model_det.log"
+ det_trans_model_cmd="${python_interp} ${trans_model_py} ${set_dirname} ${set_model_filename} ${set_params_filename} ${set_serving_server} ${set_serving_client} > ${trans_det_log} 2>&1"
eval ${det_trans_model_cmd}
+ last_status=${PIPESTATUS[0]}
+ status_check $last_status "${det_trans_model_cmd}" "${status_log}" "${model_name}" "${trans_det_log}"
# modify the alias_name of fetch_var to "outputs"
server_fetch_var_line_cmd="sed -i '/fetch_var/,/is_lod_tensor/s/alias_name: .*/alias_name: \"features\"/' $cls_serving_server_value/serving_server_conf.prototxt"
@@ -239,19 +252,19 @@ function func_serving_rec(){
devices_line=27
set_devices_cmd="sed -i '${devices_line}s/devices: .*/devices: \"\"/' config.yml"
eval ${set_devices_cmd}
-
- web_service_cmd="${python} ${web_service_py} &"
+ server_log_path="${LOG_PATH}/python_server_cpu.log"
+ web_service_cmd="nohup ${python} ${web_service_py} > ${server_log_path} 2>&1 &"
eval ${web_service_cmd}
last_status=${PIPESTATUS[0]}
- status_check $last_status "${web_service_cmd}" "${status_log}" "${model_name}"
+ status_check $last_status "${web_service_cmd}" "${status_log}" "${model_name}" "${server_log_path}"
sleep 5s
for pipeline in ${pipeline_py[*]}; do
- _save_log_path="${LOG_PATH}/server_infer_cpu_${pipeline%_client*}_batchsize_1.log"
- pipeline_cmd="${python} ${pipeline} > ${_save_log_path} 2>&1 "
+ _save_log_path="${LOG_PATH}/python_client_cpu_${pipeline%_client*}_usemkldnn_${use_mkldnn}_threads_${threads}_batchsize_1.log"
+ pipeline_cmd="${python} ${pipeline} > ${_save_log_path} 2>&1"
eval ${pipeline_cmd}
last_status=${PIPESTATUS[0]}
eval "cat ${_save_log_path}"
- status_check $last_status "${pipeline_cmd}" "${status_log}" "${model_name}"
+ status_check $last_status "${pipeline_cmd}" "${status_log}" "${model_name}" "${_save_log_path}"
sleep 5s
done
eval "${python_} -m paddle_serving_server.serve stop"
@@ -273,19 +286,19 @@ function func_serving_rec(){
devices_line=27
set_devices_cmd="sed -i '${devices_line}s/devices: .*/devices: \"${use_gpu}\"/' config.yml"
eval ${set_devices_cmd}
-
- web_service_cmd="${python} ${web_service_py} & "
+ server_log_path="${LOG_PATH}/python_server_gpu_usetrt_${use_trt}_precision_${precision}.log"
+ web_service_cmd="nohup ${python} ${web_service_py} > ${server_log_path} 2>&1 &"
eval ${web_service_cmd}
last_status=${PIPESTATUS[0]}
- status_check $last_status "${web_service_cmd}" "${status_log}" "${model_name}"
+ status_check $last_status "${web_service_cmd}" "${status_log}" "${model_name}" "${server_log_path}"
sleep 10s
for pipeline in ${pipeline_py[*]}; do
- _save_log_path="${LOG_PATH}/server_infer_gpu_${pipeline%_client*}_batchsize_1.log"
+ _save_log_path="${LOG_PATH}/python_client_gpu_${pipeline%_client*}_usetrt_${use_trt}_precision_${precision}_batchsize_1.log"
pipeline_cmd="${python} ${pipeline} > ${_save_log_path} 2>&1"
eval ${pipeline_cmd}
last_status=${PIPESTATUS[0]}
eval "cat ${_save_log_path}"
- status_check $last_status "${pipeline_cmd}" "${status_log}" "${model_name}"
+ status_check $last_status "${pipeline_cmd}" "${status_log}" "${model_name}" "${_save_log_path}"
sleep 10s
done
eval "${python_} -m paddle_serving_server.serve stop"
@@ -311,7 +324,7 @@ echo "################### run test ###################"
export Count=0
IFS="|"
-if [[ ${model_name} = "PPShiTu" ]]; then
+if [[ ${model_name} =~ "PPShiTu" ]]; then
func_serving_rec
else
func_serving_cls
diff --git a/test_tipc/test_train_inference_python.sh b/test_tipc/test_train_inference_python.sh
index 9ec79bb29ce69b908fb7c003086e013c55de2517..88274731aabb60a55bbfd0d0103269fd523e2224 100644
--- a/test_tipc/test_train_inference_python.sh
+++ b/test_tipc/test_train_inference_python.sh
@@ -95,7 +95,8 @@ if [[ $MODE = 'benchmark_train' ]]; then
epoch_num=1
fi
-LOG_PATH="./test_tipc/output/${model_name}/${MODE}"
+CLS_ROOT_PATH=$(pwd)
+LOG_PATH="${CLS_ROOT_PATH}/test_tipc/output/${model_name}/${MODE}"
mkdir -p ${LOG_PATH}
status_log="${LOG_PATH}/results_python.log"
@@ -107,24 +108,27 @@ function func_inference() {
_log_path=$4
_img_dir=$5
_flag_quant=$6
+ _gpu=$7
# inference
for use_gpu in ${use_gpu_list[*]}; do
if [ ${use_gpu} = "False" ] || [ ${use_gpu} = "cpu" ]; then
for use_mkldnn in ${use_mkldnn_list[*]}; do
for threads in ${cpu_threads_list[*]}; do
for batch_size in ${batch_size_list[*]}; do
- _save_log_path="${_log_path}/infer_cpu_usemkldnn_${use_mkldnn}_threads_${threads}_batchsize_${batch_size}.log"
- set_infer_data=$(func_set_params "${image_dir_key}" "${_img_dir}")
- set_benchmark=$(func_set_params "${benchmark_key}" "${benchmark_value}")
- set_batchsize=$(func_set_params "${batch_size_key}" "${batch_size}")
- set_cpu_threads=$(func_set_params "${cpu_threads_key}" "${threads}")
- set_model_dir=$(func_set_params "${infer_model_key}" "${_model_dir}")
- set_infer_params1=$(func_set_params "${infer_key1}" "${infer_value1}")
- command="${_python} ${_script} ${use_gpu_key}=${use_gpu} ${use_mkldnn_key}=${use_mkldnn} ${set_cpu_threads} ${set_model_dir} ${set_batchsize} ${set_infer_data} ${set_benchmark} ${set_infer_params1} > ${_save_log_path} 2>&1 "
- eval $command
- last_status=${PIPESTATUS[0]}
- eval "cat ${_save_log_path}"
- status_check $last_status "${command}" "../${status_log}" "${model_name}"
+ for precision in ${precision_list[*]}; do
+ _save_log_path="${_log_path}/python_infer_cpu_gpus_${_gpu}_usemkldnn_${use_mkldnn}_threads_${threads}_precision_${precision}_batchsize_${batch_size}.log"
+ set_infer_data=$(func_set_params "${image_dir_key}" "${_img_dir}")
+ set_benchmark=$(func_set_params "${benchmark_key}" "${benchmark_value}")
+ set_batchsize=$(func_set_params "${batch_size_key}" "${batch_size}")
+ set_cpu_threads=$(func_set_params "${cpu_threads_key}" "${threads}")
+ set_model_dir=$(func_set_params "${infer_model_key}" "${_model_dir}")
+ set_infer_params1=$(func_set_params "${infer_key1}" "${infer_value1}")
+ command="${_python} ${_script} ${use_gpu_key}=${use_gpu} ${use_mkldnn_key}=${use_mkldnn} ${set_cpu_threads} ${set_model_dir} ${set_batchsize} ${set_infer_data} ${set_benchmark} ${set_infer_params1} > ${_save_log_path} 2>&1 "
+ eval $command
+ last_status=${PIPESTATUS[0]}
+ eval "cat ${_save_log_path}"
+ status_check $last_status "${command}" "${status_log}" "${model_name}" "${_save_log_path}"
+ done
done
done
done
@@ -135,7 +139,7 @@ function func_inference() {
continue
fi
for batch_size in ${batch_size_list[*]}; do
- _save_log_path="${_log_path}/infer_gpu_usetrt_${use_trt}_precision_${precision}_batchsize_${batch_size}.log"
+ _save_log_path="${_log_path}/python_infer_gpu_gpus_${_gpu}_usetrt_${use_trt}_precision_${precision}_batchsize_${batch_size}.log"
set_infer_data=$(func_set_params "${image_dir_key}" "${_img_dir}")
set_benchmark=$(func_set_params "${benchmark_key}" "${benchmark_value}")
set_batchsize=$(func_set_params "${batch_size_key}" "${batch_size}")
@@ -146,7 +150,7 @@ function func_inference() {
eval $command
last_status=${PIPESTATUS[0]}
eval "cat ${_save_log_path}"
- status_check $last_status "${command}" "../${status_log}" "${model_name}"
+ status_check $last_status "${command}" "${status_log}" "${model_name}" "${_save_log_path}"
done
done
done
@@ -161,17 +165,19 @@ if [[ ${MODE} = "whole_infer" ]]; then
# for kl_quant
if [ ${kl_quant_cmd_value} != "null" ] && [ ${kl_quant_cmd_value} != "False" ]; then
echo "kl_quant"
- command="${python} ${kl_quant_cmd_value}"
+ log_path="${LOG_PATH}/export.log"
+ command="${python} ${kl_quant_cmd_value} > ${log_path} 2>&1"
echo ${command}
eval $command
last_status=${PIPESTATUS[0]}
- status_check $last_status "${command}" "${status_log}" "${model_name}"
+ status_check $last_status "${command}" "${status_log}" "${model_name}" "${log_path}"
cd ${infer_model_dir_list}/quant_post_static_model
- ln -s __model__ inference.pdmodel
- ln -s __params__ inference.pdiparams
+ ln -s model.pdmodel inference.pdmodel
+ ln -s model.pdiparams inference.pdiparams
cd ../../deploy
is_quant=True
- func_inference "${python}" "${inference_py}" "../${infer_model_dir_list}/quant_post_static_model" "../${LOG_PATH}" "${infer_img_dir}" ${is_quant}
+ gpu=0
+ func_inference "${python}" "${inference_py}" "../${infer_model_dir_list}/quant_post_static_model" "${LOG_PATH}" "${infer_img_dir}" "${is_quant}" "${gpu}"
cd ..
fi
else
@@ -240,7 +246,7 @@ else
if [ ${#ips} -le 15 ]; then
# if length of ips >= 15, then it is seen as multi-machine
# 15 is the min length of ips info for multi-machine: 0.0.0.0,0.0.0.0
- save_log="${LOG_PATH}/${trainer}_gpus_${gpu}_autocast_${autocast}"
+ save_log="${LOG_PATH}/${trainer}_gpus_${gpu}_autocast_${autocast}_nodes_1"
nodes=1
else
IFS=","
@@ -257,18 +263,23 @@ else
set_save_model=$(func_set_params "${save_model_key}" "${save_log}")
if [ ${#gpu} -le 2 ]; then # train with cpu or single gpu
- cmd="${python} ${run_train} ${set_use_gpu} ${set_save_model} ${set_epoch} ${set_pretrain} ${set_autocast} ${set_batchsize} ${set_train_params1} "
+ cmd="${python} ${run_train} ${set_use_gpu} ${set_save_model} ${set_epoch} ${set_pretrain} ${set_autocast} ${set_batchsize} ${set_train_params1} "
elif [ ${#ips} -le 15 ]; then # train with multi-gpu
- cmd="${python} -m paddle.distributed.launch --gpus=${gpu} ${run_train} ${set_use_gpu} ${set_save_model} ${set_epoch} ${set_pretrain} ${set_autocast} ${set_batchsize} ${set_train_params1}"
+ cmd="${python} -m paddle.distributed.launch --devices=${gpu} ${run_train} ${set_use_gpu} ${set_save_model} ${set_epoch} ${set_pretrain} ${set_autocast} ${set_batchsize} ${set_train_params1}"
else # train with multi-machine
- cmd="${python} -m paddle.distributed.launch --ips=${ips} --gpus=${gpu} ${run_train} ${set_use_gpu} ${set_save_model} ${set_pretrain} ${set_epoch} ${set_autocast} ${set_batchsize} ${set_train_params1}"
+ cmd="${python} -m paddle.distributed.launch --ips=${ips} --devices=${gpu} ${run_train} ${set_use_gpu} ${set_save_model} ${set_pretrain} ${set_epoch} ${set_autocast} ${set_batchsize} ${set_train_params1}"
fi
# run train
eval "unset CUDA_VISIBLE_DEVICES"
# export FLAGS_cudnn_deterministic=True
sleep 5
eval $cmd
- status_check $? "${cmd}" "${status_log}" "${model_name}"
+ if [[ $FILENAME == *GeneralRecognition* ]]; then
+ eval "cat ${save_log}/RecModel/train.log >> ${save_log}.log"
+ else
+ eval "cat ${save_log}/${model_name}/train.log >> ${save_log}.log"
+ fi
+ status_check $? "${cmd}" "${status_log}" "${model_name}" "${save_log}.log"
sleep 5
if [[ $FILENAME == *GeneralRecognition* ]]; then
@@ -283,9 +294,10 @@ else
# run eval
if [ ${eval_py} != "null" ]; then
set_eval_params1=$(func_set_params "${eval_key1}" "${eval_value1}")
- eval_cmd="${python} ${eval_py} ${set_eval_pretrain} ${set_use_gpu} ${set_eval_params1}"
+ eval_log_path="${LOG_PATH}/${trainer}_gpus_${gpu}_autocast_${autocast}_nodes_${nodes}_eval.log"
+ eval_cmd="${python} ${eval_py} ${set_eval_pretrain} ${set_use_gpu} ${set_eval_params1} > ${eval_log_path} 2>&1"
eval $eval_cmd
- status_check $? "${eval_cmd}" "${status_log}" "${model_name}"
+ status_check $? "${eval_cmd}" "${status_log}" "${model_name}" "${eval_log_path}"
sleep 5
fi
# run export model
@@ -298,15 +310,16 @@ else
set_export_weight=$(func_set_params "${export_weight}" "${save_log}/${model_name}/${train_model_name}")
fi
set_save_infer_key=$(func_set_params "${save_infer_key}" "${save_infer_path}")
- export_cmd="${python} ${run_export} ${set_export_weight} ${set_save_infer_key}"
+ export_log_path="${LOG_PATH}/${trainer}_gpus_${gpu}_autocast_${autocast}_nodes_${nodes}_export.log"
+ export_cmd="${python} ${run_export} ${set_export_weight} ${set_save_infer_key} > ${export_log_path} 2>&1"
eval $export_cmd
- status_check $? "${export_cmd}" "${status_log}" "${model_name}"
+ status_check $? "${export_cmd}" "${status_log}" "${model_name}" "${export_log_path}"
- #run inference
+ # run inference
eval $env
save_infer_path="${save_log}"
cd deploy
- func_inference "${python}" "${inference_py}" "../${save_infer_path}" "../${LOG_PATH}" "${infer_img_dir}" "${flag_quant}"
+ func_inference "${python}" "${inference_py}" "${save_infer_path}" "${LOG_PATH}" "${infer_img_dir}" "${flag_quant}" "${gpu}"
cd ..
fi
eval "unset CUDA_VISIBLE_DEVICES"
diff --git a/test_tipc/test_train_inference_python_npu.sh b/test_tipc/test_train_inference_python_npu.sh
new file mode 100644
index 0000000000000000000000000000000000000000..e933eff5b3518e717de503932a901efdfcfd17c9
--- /dev/null
+++ b/test_tipc/test_train_inference_python_npu.sh
@@ -0,0 +1,49 @@
+#!/bin/bash
+source test_tipc/common_func.sh
+
+function readlinkf() {
+ perl -MCwd -e 'print Cwd::abs_path shift' "$1";
+}
+
+function func_parser_config() {
+ strs=$1
+ IFS=" "
+ array=(${strs})
+ tmp=${array[2]}
+ echo ${tmp}
+}
+
+BASEDIR=$(dirname "$0")
+REPO_ROOT_PATH=$(readlinkf ${BASEDIR}/../)
+
+FILENAME=$1
+
+# change gpu to npu in tipc txt configs
+sed -i "s/Global.device:gpu/Global.device:npu/g" $FILENAME
+sed -i "s/Global.use_gpu/Global.use_npu/g" $FILENAME
+dataline=`cat $FILENAME`
+
+# parser params
+IFS=$'\n'
+lines=(${dataline})
+
+# replace inference config file
+inference_py=$(func_parser_value "${lines[39]}")
+inference_config=$(func_parser_config ${inference_py})
+sed -i 's/use_gpu: True/use_npu: True/g' "$REPO_ROOT_PATH/deploy/$inference_config"
+
+# replace training config file
+grep -n 'tools/.*yaml' $FILENAME | cut -d ":" -f 1 \
+| while read line_num ; do
+ train_cmd=$(func_parser_value "${lines[line_num-1]}")
+ trainer_config=$(func_parser_config ${train_cmd})
+ sed -i 's/device: gpu/device: npu/g' "$REPO_ROOT_PATH/$trainer_config"
+done
+
+# change gpu to npu in execution script
+sed -i "s/\"gpu\"/\"npu\"/g" test_tipc/test_train_inference_python.sh
+
+# pass parameters to test_train_inference_python.sh
+cmd="bash test_tipc/test_train_inference_python.sh ${FILENAME} $2"
+echo $cmd
+eval $cmd
diff --git a/test_tipc/test_train_inference_python_xpu.sh b/test_tipc/test_train_inference_python_xpu.sh
new file mode 100644
index 0000000000000000000000000000000000000000..124d39935c9cf618e8952ac46cb4568614f5cedd
--- /dev/null
+++ b/test_tipc/test_train_inference_python_xpu.sh
@@ -0,0 +1,49 @@
+#!/bin/bash
+source test_tipc/common_func.sh
+
+function readlinkf() {
+ perl -MCwd -e 'print Cwd::abs_path shift' "$1";
+}
+
+function func_parser_config() {
+ strs=$1
+ IFS=" "
+ array=(${strs})
+ tmp=${array[2]}
+ echo ${tmp}
+}
+
+BASEDIR=$(dirname "$0")
+REPO_ROOT_PATH=$(readlinkf ${BASEDIR}/../)
+
+FILENAME=$1
+
+# change gpu to xpu in tipc txt configs
+sed -i "s/Global.device:gpu/Global.device:xpu/g" $FILENAME
+sed -i "s/Global.use_gpu/Global.use_xpu/g" $FILENAME
+dataline=`cat $FILENAME`
+
+# parser params
+IFS=$'\n'
+lines=(${dataline})
+
+# replace inference config file
+inference_py=$(func_parser_value "${lines[39]}")
+inference_config=$(func_parser_config ${inference_py})
+sed -i 's/use_gpu: True/use_xpu: True/g' "$REPO_ROOT_PATH/deploy/$inference_config"
+
+# replace training config file
+grep -n 'tools/.*yaml' $FILENAME | cut -d ":" -f 1 \
+| while read line_num ; do
+ train_cmd=$(func_parser_value "${lines[line_num-1]}")
+ trainer_config=$(func_parser_config ${train_cmd})
+ sed -i 's/device: gpu/device: xpu/g' "$REPO_ROOT_PATH/$trainer_config"
+done
+
+# change gpu to npu in execution script
+sed -i "s/\"gpu\"/\"npu\"/g" test_tipc/test_train_inference_python.sh
+
+# pass parameters to test_train_inference_python.sh
+cmd="bash test_tipc/test_train_inference_python.sh ${FILENAME} $2"
+echo $cmd
+eval $cmd