diff --git a/docs/en/algorithm_introduction/reid.md b/docs/en/algorithm_introduction/reid.md index 0a74d71a44f537112b6e73af2bf9ef283b62a1ae..2f8d3c28bece50efe230a2bd36b65332b34d8f7a 100644 --- a/docs/en/algorithm_introduction/reid.md +++ b/docs/en/algorithm_introduction/reid.md @@ -6,27 +6,28 @@ English | [简体中文](../../zh_CN/algorithm_introduction/reid.md) - [1. Introduction to algorithms/application scenarios](#1-introduction-to-algorithmsapplication-scenarios) - [2. ReID algorithm](#2-reid-algorithm) -- [2.1 ReID strong-baseline](#21-reid-strong-baseline) - - [2.1.1 Principle introduction](#211-principle-introduction) - - [2.1.2 Accuracy Index](#212-accuracy-index) - - [2.1.3 Data Preparation](#213-data-preparation) - - [2.1.4 Model training](#214-model-training) - - [2.1.5 Model Evaluation](#215-model-evaluation) - - [2.1.6 Model Inference Deployment](#216-model-inference-deployment) - - [2.1.6.1 Inference model preparation](#2161-inference-model-preparation) - - [2.1.6.2 Inference based on Python prediction engine](#2162-inference-based-on-python-prediction-engine) - - [2.1.6.3 Inference based on C++ prediction engine](#2163-inference-based-on-c-prediction-engine) - - [2.1.7 Service deployment](#217-service-deployment) - - [2.1.8 Device side deployment](#218-device-side-deployment) - - [2.1.9 Paddle2ONNX Model Conversion and Prediction](#219-paddle2onnx-model-conversion-and-prediction) -- [3. Summary](#3-summary) -- [3.1 Method summary and comparison](#31-method-summary-and-comparison) -- [3.2 Usage advice/FAQ](#32-usage-advicefaq) + - [2.1 ReID strong-baseline](#21-reid-strong-baseline) + - [2.1.1 Principle introduction](#211-principle-introduction) + - [2.1.2 Accuracy Index](#212-accuracy-index) + - [2.1.3 Data Preparation](#213-data-preparation) + - [2.1.4 Model training](#214-model-training) +- [3. Model evaluation and inference deployment](#3-model-evaluation-and-inference-deployment) + - [3.1 Model Evaluation](#31-model-evaluation) + - [3.1 Model Inference and Deployment](#31-model-inference-and-deployment) + - [3.1.1 Inference model preparation](#311-inference-model-preparation) + - [3.1.2 Inference based on Python prediction engine](#312-inference-based-on-python-prediction-engine) + - [3.1.3 Inference based on C++ prediction engine](#313-inference-based-on-c-prediction-engine) + - [3.2 Service Deployment](#32-service-deployment) + - [3.3 Device side deployment](#33-device-side-deployment) + - [3.4 Paddle2ONNX model conversion and prediction](#34-paddle2onnx-model-conversion-and-prediction) +- [4. Summary](#4-summary) + - [4.1 Method summary and comparison](#41-method-summary-and-comparison) + - [4.2 Usage advice/FAQ](#42-usage-advicefaq) - [4. References](#4-references) ### 1. Introduction to algorithms/application scenarios -Pedestrian re-identification (Person re-identification), also known as pedestrian re-identification, is the use of [computer vision](https://baike.baidu.com/item/computervision/2803351) technology to judge [image](https://baike.baidu.com/item/image/773234) or whether there is a technique of a particular pedestrian in the video sequence. Widely regarded as a sub-problem of [Image Retrieval](https://baike.baidu.com/item/image_retrieval/1150910). Given a surveillance pedestrian image, retrieve the pedestrian image across devices. It aims to make up for the visual limitations of fixed cameras, and can be combined with [pedestrian detection](https://baike.baidu.com/item/pedestriandetection/20590256)/pedestrian tracking technology, which can be widely used in [intelligent video surveillance](https://baike.baidu.com/item/intelligentvideosurveillance/10717227), intelligent security and other fields. +Pedestrian re-identification, also known as pedestrian re-identification, is a technology that uses computer vision technology to determine whether there is a specific pedestrian in an image or video sequence. Widely regarded as a sub-problem of image retrieval. Given a surveillance pedestrian image, retrieve the pedestrian image across devices. It is designed to make up for the visual limitations of fixed cameras, and can be combined with pedestrian detection/pedestrian tracking technology, which can be widely used in intelligent video surveillance, intelligent security and other fields. The common person re-identification method extracts the local/global, single-granularity/multi-granularity features of the input image through the feature extraction module, and then obtains a high-dimensional feature vector through the fusion module. Use the classification head to convert the feature vector into the probability of each category during training to optimize the feature extraction model in the way of classification tasks; directly use the high-dimensional feature vector as the image description vector in the retrieval vector library during testing or inference search to get the search results. The ReID strong-baseline algorithm proposes several methods to effectively optimize training and retrieval to improve the overall model performance. @@ -55,13 +56,13 @@ Based on the commonly used person re-identification model based on ResNet50, the The following table summarizes the accuracy metrics of the 3 configurations of the recurring ReID strong-baseline on the Market1501 dataset, -| Profile | recall@1 | mAP | Reference recall@1 | Reference mAP | -| ------------------------ | -------- | ----- | ------------------ | ------------- | -| baseline.yaml | 88.21 | 74.12 | 87.7 | 74.0 | -| softmax.yaml | 94.18 | 85.76 | 94.1 | 85.7 | -| softmax_with_center.yaml | 94.19 | 85.80 | 94.1 | 85.7 | +| Configuration file | recall@1 | mAP | Refer to recall@1 | Refer to mAP | Pre-trained model download | Inference Model Download Address | +| -------------------------------- | -------- | ----- | ----------------- | ------------ | -------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------- | +| baseline.yaml | 88.45 | 74.37 | 87.7 | 74.0 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/reid/pretrain/baseline_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/reid/inference/baseline_infer.tar) | +| softmax_triplet.yaml | 94.29 | 85.57 | 94.1 | 85.7 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/reid/pretrain/softmax_triplet_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/reid/inference/softmax_triplet_infer.tar) | +| softmax_triplet_with_center.yaml | 94.50 | 85.82 | 94.5 | 85.9 | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/reid/pretrain/softmax_triplet_with_center_pretrained.pdparams) | [Download link](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/reid/inference/softmax_triplet_with_center_infer.tar) | -Note: The above reference indicators are obtained by using the author's open source code to train on our equipment for many times. Due to different system environment, torch version, CUDA version and other reasons, there may be slight differences with the indicators provided by the author. +Note: The above reference indicators are obtained by using the author's open source code to train on our equipment for many times. Due to different system environments, torch versions, and CUDA versions, there may be slight differences with the indicators provided by the author. Next, we mainly take the `softmax_triplet_with_center.yaml` configuration and trained model file as an example to show the process of training, testing, and inference on the Market1501 dataset. @@ -72,15 +73,15 @@ Download the [Market-1501-v15.09.15.zip](https://pan.baidu.com/s/1ntIi2Op?_at_=1 ```shell PaddleClas/dataset/market1501 └── Market-1501-v15.09.15/ - ├── bounding_box_test/ - ├── bounding_box_train/ + ├── bounding_box_test/ # gallery set pictures + ├── bounding_box_train/ # training set image ├── gt_bbox/ ├── gt_query/ - ├── query/ + ├── query/ # query set image ├── generate_anno.py - ├── bounding_box_test.txt - ├── bounding_box_train.txt - ├── query.txt + ├── bounding_box_test.txt # gallery set path + ├── bounding_box_train.txt # training set path + ├── query.txt # query set path └── readme.txt ``` @@ -88,19 +89,40 @@ Download the [Market-1501-v15.09.15.zip](https://pan.baidu.com/s/1ntIi2Op?_at_=1 1. Execute the following command to start training + Single card training: ```shell python3.7 tools/train.py -c ./ppcls/configs/reid/strong_baseline/softmax_triplet_with_center.yaml ``` + Doka training: + + For Doka training, you need to modify the sampler field of the training configuration as follows: + ```yaml + sampler: + name: PKSampler + batch_size: 64 + sample_per_id: 4 + drop_last: False + sample_method: id_avg_prob + shuffle: True + ``` + Then execute the following command(example of 4 gpus below): + ```shell + export CUDA_VISIBLE_DEVICES=0,1,2,3 + python3.7 -m paddle.distributed.launch --gpus="0,1,2,3" tools/train.py \ + -c ./ppcls/configs/reid/strong_baseline/softmax_triplet_with_center.yaml + ``` Note: Single card training takes about 1 hour. 2. View training logs and saved model parameter files During the training process, indicator information such as loss will be printed on the screen in real time, and the log file `train.log`, model parameter file `*.pdparams`, optimizer parameter file `*.pdopt` and other contents will be saved to `Global.output_dir` `Under the specified folder, the default is under the `PaddleClas/output/RecModel/` folder. -##### 2.1.5 Model Evaluation +### 3. Model evaluation and inference deployment -Prepare the `*.pdparams` model parameter file for evaluation. You can use the trained model or the model saved in [2.1.4 Model training](#214-model-training). +#### 3.1 Model Evaluation + +Prepare the `*.pdparams` model parameter file for evaluation. You can use the trained model or the model saved in [2.1.4 Model training] (#214-model training). - Take the `latest.pdparams` saved during training as an example, execute the following command to evaluate. @@ -110,19 +132,19 @@ Prepare the `*.pdparams` model parameter file for evaluation. You can use the tr -o Global.pretrained_model="./output/RecModel/latest" ``` -- Take the trained model as an example, download [reid_strong_baseline_softmax_with_center.epoch_120.pdparams](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/reid/pretrain/reid_strong_baseline_softmax_with_center.epoch_120.pdparams) Go to the `PaddleClas/pretrained_models` folder and execute the following command to evaluate. +- to train wellTake the model as an example, download [softmax_triplet_with_center_pretrained.pdparams](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/reid/pretrain/softmax_triplet_with_center_pretrained.pdparams) to `PaddleClas/ In the pretrained_models` folder, execute the following command to evaluate. ```shell # download model cd PaddleClas mkdir pretrained_models cd pretrained_models - wget https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/reid/pretrain/reid_strong_baseline_softmax_with_center.epoch_120.pdparams + wget https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/reid/pretrain/softmax_triplet_with_center_pretrained.pdparams cd.. # Evaluate python3.7 tools/eval.py \ -c ppcls/configs/reid/strong_baseline/softmax_triplet_with_center.yaml \ - -o Global.pretrained_model="pretrained_models/reid_strong_baseline_softmax_with_center.epoch_120" + -o Global.pretrained_model="pretrained_models/softmax_triplet_with_center_pretrained" ``` Note: The address filled after `pretrained_model` does not need to be suffixed with `.pdparams`, it will be added automatically when the program is running. @@ -130,6 +152,8 @@ Prepare the `*.pdparams` model parameter file for evaluation. You can use the tr ```log ... ... + ppcls INFO: unique_endpoints {''} + ppcls INFO: Found /root/.paddleclas/weights/resnet50-19c8e357_torch2paddle.pdparams ppcls INFO: gallery feature calculation process: [0/125] ppcls INFO: gallery feature calculation process: [20/125] ppcls INFO: gallery feature calculation process: [40/125] @@ -142,38 +166,39 @@ Prepare the `*.pdparams` model parameter file for evaluation. You can use the tr ppcls INFO: query feature calculation process: [20/27] ppcls INFO: Build query done, all feat shape: [3368, 2048], begin to eval.. ppcls INFO: re_ranking=False - ppcls INFO: [Eval][Epoch 0][Avg]recall1: 0.94270, recall5: 0.98189, mAP: 0.85799 + ppcls INFO: [Eval][Epoch 0][Avg]recall1: 0.94507, recall5: 0.98248, mAP: 0.85827 ``` - The default evaluation log is saved in `PaddleClas/output/RecModel/eval.log`. You can see that the evaluation metrics of the `reid_strong_baseline_softmax_with_center.epoch_120.pdparams` model we provided on the Market1501 dataset are recall@1=0.94270, recall@5 =0.98189, mAP=0.85799 + The default evaluation log is saved in `PaddleClas/output/RecModel/eval.log`. You can see that the evaluation metrics of the `softmax_triplet_with_center_pretrained.pdparams` model we provided on the Market1501 dataset are recall@1=0.94507, recall@5 =0.98248, mAP=0.85827 + +#### 3.1 Model Inference and Deployment -##### 2.1.6 Model Inference Deployment +##### 3.1.1 Inference model preparation -###### 2.1.6.1 Inference model preparation -You can choose to use the model file saved during the training process to convert into an inference model and inference, or use the converted inference model we provide for direct inference - - Convert the model file saved during the training process into an inference model, also take `latest.pdparams` as an example, execute the following command to convert +You can convert the model file saved during training into an inference model and inference, or use the converted inference model we provide for direct inference + - Convert the model file saved during the training process to an inference model, also take `latest.pdparams` as an example, execute the following command to convert ```shell python3.7 tools/export_model.py \ -c ppcls/configs/reid/strong_baseline/softmax_triplet_with_center.yaml \ -o Global.pretrained_model="output/RecModel/latest" \ - -o Global.save_inference_dir="./deploy/reid_srong_baseline_softmax_with_center" + -o Global.save_inference_dir="./deploy/softmax_triplet_with_center_infer" ``` - Or download and unzip the inference model we provide ```shell cd PaddleClas/deploy - wget https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/reid/inference/reid_srong_baseline_softmax_with_center.tar - tar xf reid_srong_baseline_softmax_with_center.tar + wget https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/reid/inference/softmax_triplet_with_center_infer.tar + tar xf softmax_triplet_with_center_infer.tar cd ../ ``` -###### 2.1.6.2 Inference based on Python prediction engine +##### 3.1.2 Inference based on Python prediction engine - 1. Modify `PaddleClas/deploy/configs/inference_rec.yaml`. Change the field after `infer_imgs:` to any image path under the query folder in Market1501 (the code below uses the image path of `0294_c1s1_066631_00.jpg`); change the field after `rec_inference_model_dir:` to the extracted one reid_srong_baseline_softmax_with_center folder path; change the preprocessing configuration under the `transform_ops` field to the preprocessing configuration under `Eval.Query.dataset` in `softmax_triplet_with_center.yaml`. As follows + 1. Modify `PaddleClas/deploy/configs/inference_rec.yaml`. Change the field after `infer_imgs:` to any image path under the query folder in Market1501 (the configuration below uses the path of the `0294_c1s1_066631_00.jpg` image); change the field after `rec_inference_model_dir:` to extract it softmax_triplet_with_center_infer folder path; change the preprocessing configuration under the `transform_ops` field to the preprocessing configuration under `Eval.Query.dataset` in `softmax_triplet_with_center.yaml`. As follows ```yaml Global: infer_imgs: "../dataset/market1501/Market-1501-v15.09.15/query/0294_c1s1_066631_00.jpg" - rec_inference_model_dir: "./reid_srong_baseline_softmax_with_center" + rec_inference_model_dir: "./softmax_triplet_with_center_infer" batch_size: 1 use_gpu: False enable_mkldnn: True @@ -209,44 +234,43 @@ You can choose to use the model file saved during the training process to conver 3. Check the output result, the actual result is a vector of length 2048, which represents the feature vector obtained after the input image is transformed by the model - ```shell + ```log 0294_c1s1_066631_00.jpg: [ 0.01806974 0.00476423 -0.00508293 ... 0.03925538 0.00377574 -0.00849029] ``` The output vector for inference is stored in the `result_dict` variable in [predict_rec.py](../../../deploy/python/predict_rec.py#L134-L135). - 4. Batch prediction - Change the path after `infer_imgs:` in the configuration file to a folder, such as `../dataset/market1501/Market-1501-v15.09.15/query`, it will predict and output all images under query. Feature vector. + 4. For batch prediction, change the path after `infer_imgs:` in the configuration file to a folder, such as `../dataset/market1501/Market-1501-v15.09.15/query`, it will predict and output The feature vector of all images under query. -###### 2.1.6.3 Inference based on C++ prediction engine +##### 3.1.3 Inference based on C++ prediction engine -PaddleClas provides an example of inference based on the C++ prediction engine, you can refer to [Server-side C++ prediction](../inference_deployment/cpp_deploy_en.md) to complete the corresponding inference deployment. If you are using the Windows platform, you can refer to the Visual Studio 2019 Community CMake Compilation Guide to complete the corresponding prediction library compilation and model prediction work. +PaddleClas provides an example of inference based on C++ prediction engine, you can refer to [C++ prediction](../inference_deployment/cpp_deploy_en.md) to complete the corresponding inference deployment. If you are using the Windows platform, you can refer to the Visual Studio 2019 Community CMake Compilation Guide to complete the corresponding prediction library compilation and model prediction work. -##### 2.1.7 Service deployment +##### 3.2 Service Deployment Paddle Serving provides high-performance, flexible and easy-to-use industrial-grade online inference services. Paddle Serving supports RESTful, gRPC, bRPC and other protocols, and provides inference solutions in a variety of heterogeneous hardware and operating system environments. For more introduction to Paddle Serving, please refer to the Paddle Serving code repository. -PaddleClas provides an example of model serving deployment based on Paddle Serving. You can refer to [Model serving deployment](../inference_deployment/paddle_serving_deploy_en.md) to complete the corresponding deployment. +PaddleClas provides an example of model serving deployment based on Paddle Serving. You can refer to [Model serving deployment](../inference_deployment/paddle_serving_deploy_en.md) to complete the corresponding deployment work. -##### 2.1.8 Device side deployment +##### 3.3 Lite deployment -Paddle Lite is a high-performance, lightweight, flexible and easily extensible deep learning inference framework, positioned to support multiple hardware platforms including mobile, embedded and server. For more introduction to Paddle Lite, please refer to the Paddle Lite code repository. +Paddle Lite is a high-performance, lightweight, flexible and easily extensible deep learning inference framework, positioned to support mobileMultiple hardware platforms including client, embedded and server. For more introduction to Paddle Lite, please refer to the Paddle Lite code repository. PaddleClas provides an example of deploying models based on Paddle Lite. You can refer to [Deployment](../inference_deployment/paddle_lite_deploy_en.md) to complete the corresponding deployment. -##### 2.1.9 Paddle2ONNX Model Conversion and Prediction +##### 3.4 Paddle2ONNX model conversion and prediction -Paddle2ONNX supports converting PaddlePaddle model format to ONNX model format. The deployment of Paddle models to various inference engines can be completed through ONNX, including TensorRT/OpenVINO/MNN/TNN/NCNN, and other inference engines or hardware that support the ONNX open source format. For more information about Paddle2ONNX, please refer to the Paddle2ONNX code repository. +Paddle2ONNX supports converting PaddlePaddle model format to ONNX model format. The deployment of Paddle models to various inference engines can be completed through ONNX, including TensorRT/OpenVINO/MNN/TNN/NCNN, as well as other inference engines or hardware that support the ONNX open source format. For more information about Paddle2ONNX, please refer to the Paddle2ONNX code repository. PaddleClas provides an example of converting an inference model to an ONNX model and making inference prediction based on Paddle2ONNX. You can refer to [Paddle2ONNX model conversion and prediction](../../../deploy/paddle2onnx/readme.md) to complete the corresponding deployment work. -### 3. Summary +### 4. Summary -#### 3.1 Method summary and comparison +#### 4.1 Method summary and comparison The above algorithm can be quickly migrated to most ReID models, which can further improve the performance of ReID models. -#### 3.2 Usage advice/FAQ +#### 4.2 Usage advice/FAQ The Market1501 dataset is relatively small, so you can try to train multiple times to get the highest accuracy. @@ -254,5 +278,5 @@ The Market1501 dataset is relatively small, so you can try to train multiple tim 1. [Bag of Tricks and A Strong Baseline for Deep Person Re-identification](https://openaccess.thecvf.com/content_CVPRW_2019/papers/TRMTMCT/Luo_Bag_of_Tricks_and_a_Strong_Baseline_for_Deep_Person_CVPRW_2019_paper.pdf) 2. [michuanhaohao/reid-strong-baseline](https://github.com/michuanhaohao/reid-strong-baseline) -3. [Pedestrian Re-ID dataset Market1501Data set _star_function's blog - CSDN blog _market1501 data set](https://blog.csdn.net/qq_39220334/article/details/121470106) +3. [Pedestrian Re-ID dataset Market1501 dataset _star_function blog-CSDN blog _market1501 dataset](https://blog.csdn.net/qq_39220334/article/details/121470106) 4. [Deep Learning for Person Re-identification: A Survey and Outlook](https://arxiv.org/abs/2001.04193) diff --git a/docs/zh_CN/algorithm_introduction/reid.md b/docs/zh_CN/algorithm_introduction/reid.md index 42d0693290d85cecb3ac21e07e86ff977f6d1471..7a2702a3273810cef48e0165639ececa17c90080 100644 --- a/docs/zh_CN/algorithm_introduction/reid.md +++ b/docs/zh_CN/algorithm_introduction/reid.md @@ -11,22 +11,23 @@ - [2.1.2 精度指标](#212-精度指标) - [2.1.3 数据准备](#213-数据准备) - [2.1.4 模型训练](#214-模型训练) - - [2.1.5 模型评估](#215-模型评估) - - [2.1.6 模型推理部署](#216-模型推理部署) - - [2.1.6.1 推理模型准备](#2161-推理模型准备) - - [2.1.6.2 基于 Python 预测引擎推理](#2162-基于-python-预测引擎推理) - - [2.1.6.3 基于 C++ 预测引擎推理](#2163-基于-c-预测引擎推理) - - [2.1.7 服务化部署](#217-服务化部署) - - [2.1.8 端侧部署](#218-端侧部署) - - [2.1.9 Paddle2ONNX 模型转换与预测](#219-paddle2onnx-模型转换与预测) -- [3. 总结](#3-总结) - - [3.1 方法总结与对比](#31-方法总结与对比) - - [3.2 使用建议/FAQ](#32-使用建议faq) +- [3. 模型评估与推理部署](#3-模型评估与推理部署) + - [3.1 模型评估](#31-模型评估) + - [3.1 模型推理与部署](#31-模型推理与部署) + - [3.1.1 推理模型准备](#311-推理模型准备) + - [3.1.2 基于 Python 预测引擎推理](#312-基于-python-预测引擎推理) + - [3.1.3 基于 C++ 预测引擎推理](#313-基于-c-预测引擎推理) + - [3.2 服务化部署](#32-服务化部署) + - [3.3 端侧部署](#33-端侧部署) + - [3.4 Paddle2ONNX 模型转换与预测](#34-paddle2onnx-模型转换与预测) +- [4. 总结](#4-总结) + - [4.1 方法总结与对比](#41-方法总结与对比) + - [4.2 使用建议/FAQ](#42-使用建议faq) - [4. 参考资料](#4-参考资料) ### 1. 算法/应用场景简介 -行人重识别(Person re-identification)也称行人再识别,是利用[计算机视觉](https://baike.baidu.com/item/计算机视觉/2803351)技术判断[图像](https://baike.baidu.com/item/图像/773234)或者视频序列中是否存在特定行人的技术。广泛被认为是一个[图像检索](https://baike.baidu.com/item/图像检索/1150910)的子问题。给定一个监控行人图像,检索跨设备下的该行人图像。旨在弥补固定的摄像头的视觉局限,并可与[行人检测](https://baike.baidu.com/item/行人检测/20590256)/行人跟踪技术相结合,可广泛应用于[智能视频监控](https://baike.baidu.com/item/智能视频监控/10717227)、智能安保等领域。 +行人重识别(Person re-identification)也称行人再识别,是利用计算机视觉技术判断图像或者视频序列中是否存在特定行人的技术。广泛被认为是一个图像检索的子问题。给定一个监控行人图像,检索跨设备下的该行人图像。旨在弥补固定的摄像头的视觉局限,并可与行人检测/行人跟踪技术相结合,可广泛应用于智能视频监控、智能安保等领域。 常见的行人重识别方法通过特征提取模块来提取输入图片的局部/全局、单粒度/多粒度特征,再经过融合模块得到一个高维的特征向量。在训练时使用分类头将该特征向量转换成属于每个类别的概率从而以分类任务的方式来优化特征提取模型;在测试或推理时直接将高维的特征向量作为图片描述向量在检索向量库中进行检索,以得到检索结果。而ReID strong-baseline算法则是提出了多个有效优化训练与检索的方法来提升整体模型性能。 @@ -55,11 +56,11 @@ 以下表格总结了复现的ReID strong-baseline的3种配置在 Market1501 数据集上的精度指标, -| 配置文件 | recall@1 | mAP | 参考recall@1 | 参考mAP | -| ------------------------ | -------- | ----- | ------------ | ------- | -| baseline.yaml | 88.21 | 74.12 | 87.7 | 74.0 | -| softmax.yaml | 94.18 | 85.76 | 94.1 | 85.7 | -| softmax_with_center.yaml | 94.19 | 85.80 | 94.1 | 85.7 | +| 配置文件 | recall@1 | mAP | 参考recall@1 | 参考mAP | 预训练模型下载地址 | inference模型下载地址 | +| -------------------------------- | -------- | ----- | ------------ | ------- | --------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------- | +| baseline.yaml | 88.45 | 74.37 | 87.7 | 74.0 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/reid/pretrain/baseline_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/reid/inference/baseline_infer.tar) | +| softmax_triplet.yaml | 94.29 | 85.57 | 94.1 | 85.7 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/reid/pretrain/softmax_triplet_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/reid/inference/softmax_triplet_infer.tar) | +| softmax_triplet_with_center.yaml | 94.50 | 85.82 | 94.5 | 85.9 | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/reid/pretrain/softmax_triplet_with_center_pretrained.pdparams) | [下载链接](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/reid/inference/softmax_triplet_with_center_infer.tar) | 注:上述参考指标由使用作者开源的代码在我们的设备上训练多次得到,由于系统环境、torch版本、CUDA版本不同等原因,与作者提供的指标可能存在略微差异。 @@ -72,15 +73,15 @@ ```shell PaddleClas/dataset/market1501 └── Market-1501-v15.09.15/ - ├── bounding_box_test/ - ├── bounding_box_train/ + ├── bounding_box_test/ # gallery集图片 + ├── bounding_box_train/ # 训练集图片 ├── gt_bbox/ ├── gt_query/ - ├── query/ + ├── query/ # query集图片 ├── generate_anno.py - ├── bounding_box_test.txt - ├── bounding_box_train.txt - ├── query.txt + ├── bounding_box_test.txt # gallery集路径 + ├── bounding_box_train.txt # 训练集路径 + ├── query.txt # query集路径 └── readme.txt ``` @@ -88,17 +89,38 @@ 1. 执行以下命令开始训练 + 单卡训练: ```shell python3.7 tools/train.py -c ./ppcls/configs/reid/strong_baseline/softmax_triplet_with_center.yaml ``` + 多卡训练: + + 多卡训练需修改训练配置的采样器字段以适配分布式训练,如下所示: + ```yaml + sampler: + name: PKSampler + batch_size: 64 + sample_per_id: 4 + drop_last: False + sample_method: id_avg_prob + shuffle: True + ``` + 然后执行以下命令: + ```shell + export CUDA_VISIBLE_DEVICES=0,1,2,3 + python3.7 -m paddle.distributed.launch --gpus="0,1,2,3" tools/train.py \ + -c ./ppcls/configs/reid/strong_baseline/softmax_triplet_with_center.yaml + ``` 注:单卡训练大约需要1个小时。 2. 查看训练日志和保存的模型参数文件 训练过程中会在屏幕上实时打印loss等指标信息,同时会保存日志文件`train.log`、模型参数文件`*.pdparams`、优化器参数文件`*.pdopt`等内容到`Global.output_dir`指定的文件夹下,默认在`PaddleClas/output/RecModel/`文件夹下。 -##### 2.1.5 模型评估 +### 3. 模型评估与推理部署 + +#### 3.1 模型评估 准备用于评估的`*.pdparams`模型参数文件,可以使用训练好的模型,也可以使用[2.1.4 模型训练](#214-模型训练)中保存的模型。 @@ -110,19 +132,19 @@ -o Global.pretrained_model="./output/RecModel/latest" ``` -- 以训练好的模型为例,下载 [reid_strong_baseline_softmax_with_center.epoch_120.pdparams](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/reid/pretrain/reid_strong_baseline_softmax_with_center.epoch_120.pdparams) 到 `PaddleClas/pretrained_models` 文件夹中,执行如下命令即可进行评估。 +- 以训练好的模型为例,下载 [softmax_triplet_with_center_pretrained.pdparams](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/reid/pretrain/softmax_triplet_with_center_pretrained.pdparams) 到 `PaddleClas/pretrained_models` 文件夹中,执行如下命令即可进行评估。 ```shell # 下载模型 cd PaddleClas mkdir pretrained_models cd pretrained_models - wget https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/reid/pretrain/reid_strong_baseline_softmax_with_center.epoch_120.pdparams + wget https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/reid/pretrain/softmax_triplet_with_center_pretrained.pdparams cd .. # 评估 python3.7 tools/eval.py \ -c ppcls/configs/reid/strong_baseline/softmax_triplet_with_center.yaml \ - -o Global.pretrained_model="pretrained_models/reid_strong_baseline_softmax_with_center.epoch_120" + -o Global.pretrained_model="pretrained_models/softmax_triplet_with_center_pretrained" ``` 注:`pretrained_model` 后填入的地址不需要加 `.pdparams` 后缀,在程序运行时会自动补上。 @@ -130,6 +152,8 @@ ```log ... ... + ppcls INFO: unique_endpoints {''} + ppcls INFO: Found /root/.paddleclas/weights/resnet50-19c8e357_torch2paddle.pdparams ppcls INFO: gallery feature calculation process: [0/125] ppcls INFO: gallery feature calculation process: [20/125] ppcls INFO: gallery feature calculation process: [40/125] @@ -142,43 +166,47 @@ ppcls INFO: query feature calculation process: [20/27] ppcls INFO: Build query done, all feat shape: [3368, 2048], begin to eval.. ppcls INFO: re_ranking=False - ppcls INFO: [Eval][Epoch 0][Avg]recall1: 0.94270, recall5: 0.98189, mAP: 0.85799 + ppcls INFO: [Eval][Epoch 0][Avg]recall1: 0.94507, recall5: 0.98248, mAP: 0.85827 ``` - 默认评估日志保存在`PaddleClas/output/RecModel/eval.log`中,可以看到我们提供的 `reid_strong_baseline_softmax_with_center.epoch_120.pdparams` 模型在 Market1501 数据集上的评估指标为recall@1=0.94270,recall@5=0.98189,mAP=0.85799 + 默认评估日志保存在`PaddleClas/output/RecModel/eval.log`中,可以看到我们提供的 `softmax_triplet_with_center_pretrained.pdparams` 模型在 Market1501 数据集上的评估指标为recall@1=0.94507,recall@5=0.98248,mAP=0.85827 + +#### 3.1 模型推理与部署 -##### 2.1.6 模型推理部署 +##### 3.1.1 推理模型准备 -###### 2.1.6.1 推理模型准备 -可以选择使用训练过程中保存的模型文件转换成 inference 模型并推理,或者使用我们提供的转换好的 inference 模型直接进行推理 - - 将训练过程中保存的模型文件转换成 inference 模型,同样以`latest.pdparams`为例,执行以下命令进行转换 +可以将训练过程中保存的模型文件转换成 inference 模型并推理,或者使用我们提供的转换好的 inference 模型直接进行推理 + - 将训练过程中保存的模型文件转换成 inference 模型,同样以 `latest.pdparams` 为例,执行以下命令进行转换 ```shell python3.7 tools/export_model.py \ -c ppcls/configs/reid/strong_baseline/softmax_triplet_with_center.yaml \ -o Global.pretrained_model="output/RecModel/latest" \ - -o Global.save_inference_dir="./deploy/reid_srong_baseline_softmax_with_center" + -o Global.save_inference_dir="./deploy/softmax_triplet_with_center_infer" ``` - 或者下载并解压我们提供的 inference 模型 ```shell cd PaddleClas/deploy - wget https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/reid/inference/reid_srong_baseline_softmax_with_center.tar - tar xf reid_srong_baseline_softmax_with_center.tar + wget https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/reid/inference/softmax_triplet_with_center_infer.tar + tar xf softmax_triplet_with_center_infer.tar cd ../ ``` -###### 2.1.6.2 基于 Python 预测引擎推理 +##### 3.1.2 基于 Python 预测引擎推理 - 1. 修改 `PaddleClas/deploy/configs/inference_rec.yaml`。将 `infer_imgs:` 后的字段改为 Market1501 中 query 文件夹下的任意一张图片路径(下方配置使用的是`0294_c1s1_066631_00.jpg`图片的路径);将 `rec_inference_model_dir:` 后的字段改为解压出来的 reid_srong_baseline_softmax_with_center 文件夹路径;将 `transform_ops` 字段下的预处理配置改为 `softmax_triplet_with_center.yaml` 中`Eval.Query.dataset` 下的预处理配置。如下所示 + 1. 修改 `PaddleClas/deploy/configs/inference_rec.yaml` + - 将 `infer_imgs:` 后的路径段改为 Market1501 中 query 文件夹下的任意一张图片路径(下方配置使用的是`0294_c1s1_066631_00.jpg`图片的路径) + - 将 `rec_inference_model_dir:` 后的字段改为解压出来的 softmax_triplet_with_center_infer 文件夹路径 + - 将 `transform_ops:` 字段下的预处理配置改为 `softmax_triplet_with_center.yaml` 中`Eval.Query.dataset` 下的预处理配置 ```yaml Global: infer_imgs: "../dataset/market1501/Market-1501-v15.09.15/query/0294_c1s1_066631_00.jpg" - rec_inference_model_dir: "./reid_srong_baseline_softmax_with_center" + rec_inference_model_dir: "./softmax_triplet_with_center_infer" batch_size: 1 use_gpu: False enable_mkldnn: True cpu_num_threads: 10 - enable_benchmark: True + enable_benchmark: False use_fp16: False ir_optim: True use_tensorrt: False @@ -209,43 +237,43 @@ 3. 查看输出结果,实际结果为一个长度2048的向量,表示输入图片经过模型转换后得到的特征向量 - ```shell + ```log 0294_c1s1_066631_00.jpg: [ 0.01806974 0.00476423 -0.00508293 ... 0.03925538 0.00377574 -0.00849029] ``` 推理时的输出向量储存在[predict_rec.py](../../../deploy/python/predict_rec.py#L134-L135)的`result_dict`变量中。 - 4. 批量预测,将配置文件中`infer_imgs:`后的路径改为为文件夹即可,如`../dataset/market1501/Market-1501-v15.09.15/query`,则会预测并输出出query下所有图片的特征向量。 + 4. 批量预测,将配置文件中`infer_imgs:`后的路径改为为文件夹即可,如`../dataset/market1501/Market-1501-v15.09.15/query`,会预测并逐个输出query下所有图片的特征向量。 -###### 2.1.6.3 基于 C++ 预测引擎推理 +##### 3.1.3 基于 C++ 预测引擎推理 PaddleClas 提供了基于 C++ 预测引擎推理的示例,您可以参考[服务器端 C++ 预测](../inference_deployment/cpp_deploy.md)来完成相应的推理部署。如果您使用的是 Windows 平台,可以参考基于 Visual Studio 2019 Community CMake 编译指南完成相应的预测库编译和模型预测工作。 -##### 2.1.7 服务化部署 +##### 3.2 服务化部署 Paddle Serving 提供高性能、灵活易用的工业级在线推理服务。Paddle Serving 支持 RESTful、gRPC、bRPC 等多种协议,提供多种异构硬件和多种操作系统环境下推理解决方案。更多关于Paddle Serving 的介绍,可以参考Paddle Serving 代码仓库。 PaddleClas 提供了基于 Paddle Serving 来完成模型服务化部署的示例,您可以参考[模型服务化部署](../inference_deployment/paddle_serving_deploy.md)来完成相应的部署工作。 -##### 2.1.8 端侧部署 +##### 3.3 端侧部署 Paddle Lite 是一个高性能、轻量级、灵活性强且易于扩展的深度学习推理框架,定位于支持包括移动端、嵌入式以及服务器端在内的多硬件平台。更多关于 Paddle Lite 的介绍,可以参考Paddle Lite 代码仓库。 PaddleClas 提供了基于 Paddle Lite 来完成模型端侧部署的示例,您可以参考[端侧部署](../inference_deployment/paddle_lite_deploy.md)来完成相应的部署工作。 -##### 2.1.9 Paddle2ONNX 模型转换与预测 +##### 3.4 Paddle2ONNX 模型转换与预测 Paddle2ONNX 支持将 PaddlePaddle 模型格式转化到 ONNX 模型格式。通过 ONNX 可以完成将 Paddle 模型到多种推理引擎的部署,包括TensorRT/OpenVINO/MNN/TNN/NCNN,以及其它对 ONNX 开源格式进行支持的推理引擎或硬件。更多关于 Paddle2ONNX 的介绍,可以参考Paddle2ONNX 代码仓库。 PaddleClas 提供了基于 Paddle2ONNX 来完成 inference 模型转换 ONNX 模型并作推理预测的示例,您可以参考[Paddle2ONNX 模型转换与预测](../../../deploy/paddle2onnx/readme.md)来完成相应的部署工作。 -### 3. 总结 +### 4. 总结 -#### 3.1 方法总结与对比 +#### 4.1 方法总结与对比 上述算法能快速地迁移至多数的ReID模型中,能进一步提升ReID模型的性能。 -#### 3.2 使用建议/FAQ +#### 4.2 使用建议/FAQ Market1501 数据集比较小,可以尝试训练多次取最高精度。