From 3ef5343832085458bc654998fbc9f572c868d5bc Mon Sep 17 00:00:00 2001 From: JYChen Date: Mon, 22 Aug 2022 17:35:51 +0800 Subject: [PATCH] add develop doc for vehicle attr (#6699) --- deploy/pipeline/README.md | 2 +- .../customization/pphuman_attribute.md | 2 +- .../customization/pphuman_attribute_en.md | 4 +- .../customization/ppvehicle_attribute.md | 257 +++++++++++++++++ .../customization/ppvehicle_attribute_en.md | 271 ++++++++++++++++++ 5 files changed, 532 insertions(+), 4 deletions(-) create mode 100644 docs/advanced_tutorials/customization/ppvehicle_attribute.md create mode 100644 docs/advanced_tutorials/customization/ppvehicle_attribute_en.md diff --git a/deploy/pipeline/README.md b/deploy/pipeline/README.md index cbb2d86fb..195341947 100644 --- a/deploy/pipeline/README.md +++ b/deploy/pipeline/README.md @@ -128,7 +128,7 @@ - [快速开始](docs/tutorials/ppvehicle_attribute.md) -- 二次开发教程 +- [二次开发教程](../../docs/advanced_tutorials/customization/ppvehicle_attribute.md) #### 违章检测 diff --git a/docs/advanced_tutorials/customization/pphuman_attribute.md b/docs/advanced_tutorials/customization/pphuman_attribute.md index 692a9dba5..75cdd4aeb 100644 --- a/docs/advanced_tutorials/customization/pphuman_attribute.md +++ b/docs/advanced_tutorials/customization/pphuman_attribute.md @@ -1,6 +1,6 @@ 简体中文 | [English](./pphuman_attribute_en.md) -# 属性识别任务二次开发 +# 行人属性识别任务二次开发 ## 数据准备 diff --git a/docs/advanced_tutorials/customization/pphuman_attribute_en.md b/docs/advanced_tutorials/customization/pphuman_attribute_en.md index fdfe74eb2..1dd3813ec 100644 --- a/docs/advanced_tutorials/customization/pphuman_attribute_en.md +++ b/docs/advanced_tutorials/customization/pphuman_attribute_en.md @@ -1,6 +1,6 @@ [简体中文](pphuman_attribute.md) | English -# Customized attribute recognition +# Customized Pedestrian Attribute Recognition ## Data Preparation @@ -51,7 +51,7 @@ After the annotation is completed, the model will use the detection box to inter -Model Training +## Model Training Once the data is annotated, it can be used for model training to complete the optimization of the customized model. diff --git a/docs/advanced_tutorials/customization/ppvehicle_attribute.md b/docs/advanced_tutorials/customization/ppvehicle_attribute.md new file mode 100644 index 000000000..c40c9cb30 --- /dev/null +++ b/docs/advanced_tutorials/customization/ppvehicle_attribute.md @@ -0,0 +1,257 @@ +简体中文 | [English](./ppvehicle_attribute_en.md) + +# 车辆属性识别任务二次开发 + +## 数据准备 + +### 数据格式 + +车辆属性模型采用VeRi数据集的属性，共计10种车辆颜色及9种车型, 具体如下： +``` +# 车辆颜色 +- "yellow" +- "orange" +- "green" +- "gray" +- "red" +- "blue" +- "white" +- "golden" +- "brown" +- "black" + +# 车型 +- "sedan" +- "suv" +- "van" +- "hatchback" +- "mpv" +- "pickup" +- "bus" +- "truck" +- "estate" +``` + +在标注文件中使用长度为19的序列来表示上述属性。 + +举例： + +[1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0] + +前10位中，位序号0的值为1，表示车辆颜色为`"yellow"`。 + +后9位中，位序号11的值为1，表示车型为`"suv"`。 + + +### 数据标注 + +理解了上面`数据格式`的含义后，就可以进行数据标注的工作。其本质是：每张车辆的图片，建立一组长度为19的标注项，分别对应各项属性值。 + +举例： + +对于一张原始图片， + +1） 使用检测框，标注图片中每台车辆的位置。 + +2） 每一个检测框（对应每辆车），包含一组19位的属性值数组，数组的每一位以0或1表示。对应上述19个属性分类。例如，如果颜色是'orange'，则数组索引为1的位置值为1，如果车型是'sedan'，则数组索引为10的位置值为1。 + +标注完成后利用检测框将每辆车截取成只包含单辆车的图片，则图片与19位属性标注建立了对应关系。也可先截取再进行标注，效果相同。 + + +## 模型训练 + +数据标注完成后，就可以拿来做模型的训练，完成自定义模型的优化工作。 + +其主要有两步工作需要完成：1）将数据与标注数据整理成训练格式。2）修改配置文件开始训练。 + +### 训练数据格式 + +训练数据包括训练使用的图片和一个训练列表train.txt，其具体位置在训练配置中指定，其放置方式示例如下： +``` +Attribute/ +|-- data 训练图片文件夹 +| |-- 00001.jpg +| |-- 00002.jpg +| `-- 0000x.jpg +`-- train.txt 训练数据列表 + +``` + +train.txt文件内为所有训练图片名称（相对于根路径的文件路径）+ 19个标注值 + +其每一行表示一辆车的图片和标注结果。其格式为： + +``` +00001.jpg 0,0,1,0,.... +``` + +注意：1)图片与标注值之间是以Tab[\t]符号隔开, 2)标注值之间是以逗号[,]隔开。该格式不能错，否则解析失败。 + +### 修改配置开始训练 + +首先执行以下命令下载训练代码（更多环境问题请参考[Install_PaddleClas](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.4/docs/en/installation/install_paddleclas_en.md)）: + +```shell +git clone https://github.com/PaddlePaddle/PaddleClas +``` + +需要在[配置文件](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.4/ppcls/configs/PULC/vehicle_attribute/PPLCNet_x1_0.yaml)中，修改的配置项如下： +```yaml +DataLoader: + Train: + dataset: + name: MultiLabelDataset + image_root: "dataset/VeRi/" # the root path of training images + cls_label_path: "dataset/VeRi/train_list.txt" # the location of the training list file + label_ratio: True + transform_ops: + ... + + Eval: + dataset: + name: MultiLabelDataset + image_root: "dataset/VeRi/" # the root path of evaluation images + cls_label_path: "dataset/VeRi/val_list.txt" # the location of the evaluation list file + label_ratio: True + transform_ops: + ... +``` + +注意： +1. 这里image_root路径+train.txt中图片相对路径，对应图片的完整路径位置。 +2. 如果有修改属性数量，则还需修改内容配置项中属性种类数量： +```yaml +# model architecture +Arch: + name: "PPLCNet_x1_0" + pretrained: True + use_ssld: True + class_num: 19 #属性种类数量 +``` + +然后运行以下命令开始训练。 + +``` +#多卡训练 +export CUDA_VISIBLE_DEVICES=0,1,2,3 +python3 -m paddle.distributed.launch \ + --gpus="0,1,2,3" \ + tools/train.py \ + -c ./ppcls/configs/PULC/vehicle_attribute/PPLCNet_x1_0.yaml + +#单卡训练 +python3 tools/train.py \ + -c ./ppcls/configs/PULC/vehicle_attribute/PPLCNet_x1_0.yaml +``` + +训练完成后可以执行以下命令进行性能评估： +``` +#多卡评估 +export CUDA_VISIBLE_DEVICES=0,1,2,3 +python3 -m paddle.distributed.launch \ + --gpus="0,1,2,3" \ + tools/eval.py \ + -c ./ppcls/configs/PULC/vehicle_attribute/PPLCNet_x1_0.yaml \ + -o Global.pretrained_model=./output/PPLCNet_x1_0/best_model + +#单卡评估 +python3 tools/eval.py \ + -c ./ppcls/configs/PULC/vehicle_attribute/PPLCNet_x1_0.yaml \ + -o Global.pretrained_model=./output/PPLCNet_x1_0/best_model +``` + + +### 模型导出 + +使用下述命令将训练好的模型导出为预测部署模型。 + +``` +python3 tools/export_model.py \ + -c ./ppcls/configs/PULC/vehicle_attribute/PPLCNet_x1_0.yaml \ + -o Global.pretrained_model=output/PPLCNet_x1_0/best_model \ + -o Global.save_inference_dir=deploy/models/PPLCNet_x1_0_vehicle_attribute_model +``` + +导出模型后，如果希望在PP-Vehicle中使用，则需要下载[预测部署模型](https://bj.bcebos.com/v1/paddledet/models/pipeline/vehicle_attribute_model.zip)，解压并将其中的配置文件`infer_cfg.yml`文件，放置到导出的模型文件夹`PPLCNet_x1_0_vehicle_attribute_model`中。 + +使用时在PP-Vehicle中的配置文件`./deploy/pipeline/config/infer_cfg_ppvehicle.yml`中修改新的模型路径`model_dir`项，并开启功能`enable: True`。 +``` +VEHICLE_ATTR: + model_dir: [YOUR_DEPLOY_MODEL_DIR]/PPLCNet_x1_0_vehicle_attribute_infer/ #新导出的模型路径位置 + enable: True #开启功能 +``` +然后可以使用-->至此即完成新增属性类别识别任务。 + +## 属性增减 + +该过程与行人属性的增减过程相似，如果需要增加、减少属性数量，则需要： + +1)标注时需增加新属性类别信息或删减属性类别信息； + +2)对应修改训练中train.txt所使用的属性数量和名称； + +3)修改训练配置，例如``PaddleClas/blob/develop/ppcls/configs/PULC/vehicle_attribute/PPLCNet_x1_0.yaml``文件中的属性数量,详细见上述`修改配置开始训练`部分。 + +增加属性示例： + +1. 在标注数据时在19位后继续增加新的属性标注数值； +2. 在train.txt文件的标注数值中也增加新的属性数值。 +3. 注意属性类型在train.txt中属性数值列表中的位置的对应关系需要固定。 + +
+ +
+ +删减属性同理。 + + +## 修改后处理代码 + +修改了属性定义后，pipeline后处理部分也需要做相应修改，主要影响结果可视化时的显示结果。 + +相应代码在[文件](https://github.com/PaddlePaddle/PaddleDetection/blob/develop/deploy/pipeline/ppvehicle/vehicle_attr.py#L108)中`postprocess`函数。 + +其函数实现说明如下： + +```python + # 在类的初始化函数中，定义了颜色/车型的名称 + self.color_list = [ + "yellow", "orange", "green", "gray", "red", "blue", "white", + "golden", "brown", "black" + ] + self.type_list = [ + "sedan", "suv", "van", "hatchback", "mpv", "pickup", "bus", "truck", + "estate" + ] + + ... + + def postprocess(self, inputs, result): + # postprocess output of predictor + im_results = result['output'] + batch_res = [] + for res in im_results: + res = res.tolist() + attr_res = [] + color_res_str = "Color: " + type_res_str = "Type: " + color_idx = np.argmax(res[:10]) # 前10项表示各项颜色得分，取得分最大项作为颜色结果 + type_idx = np.argmax(res[10:]) # 后9项表示各项车型得分，取得分最大项作为车型结果 + + # 颜色和车型的得分都需要超过对应阈值，否则视为'UnKnown' + if res[color_idx] >= self.color_threshold: + color_res_str += self.color_list[color_idx] + else: + color_res_str += "Unknown" + attr_res.append(color_res_str) + + if res[type_idx + 10] >= self.type_threshold: + type_res_str += self.type_list[type_idx] + else: + type_res_str += "Unknown" + attr_res.append(type_res_str) + + batch_res.append(attr_res) + result = {'output': batch_res} + return result +``` diff --git a/docs/advanced_tutorials/customization/ppvehicle_attribute_en.md b/docs/advanced_tutorials/customization/ppvehicle_attribute_en.md new file mode 100644 index 000000000..69fbc18fb --- /dev/null +++ b/docs/advanced_tutorials/customization/ppvehicle_attribute_en.md @@ -0,0 +1,271 @@ +[简体中文](ppvehicle_attribute.md) | English + +# Customized Vehicle Attribute Recognition + +## Data Preparation + +### Data Format + +We use the VeRi attribute annotation format, with a total of 10 color and 9 model attributes shown as follows. + +``` +# colors +- "yellow" +- "orange" +- "green" +- "gray" +- "red" +- "blue" +- "white" +- "golden" +- "brown" +- "black" + +# models +- "sedan" +- "suv" +- "van" +- "hatchback" +- "mpv" +- "pickup" +- "bus" +- "truck" +- "estate" +``` + +A sequence of length 19 is used in the annotation file to represent the above attributes. + +Examples: + +[1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0] + +In the first 10 bits, the value of bit index 0 is 1, indicating that the vehicle color is `"yellow"`. + +In the last 9 bits, the value of bit index 11 is 1, indicating that the model is `"suv"`. + + +### Data Annotation + +After knowing the purpose of the above `Data format`, we can start to annotate data. The essence is that each single-vehicle image creates a set of 19 annotation items, corresponding to the attribute values at 19 positions. + +Examples: + +For an original image: + +1) Using bounding boxes to annotate the position of each vehicle in the picture. + +2) Each detection box (corresponding to each vehicle) contains 19 attribute values which are represented by 0 or 1. It corresponds to the above 19 attributes. For example, if the color is 'orange', then the index 1 bit of the array is 1. If the model is 'sedan', then the index 10 bit of the array is 1. + +After the annotation is completed, the model will use the detection box to intercept each vehicle into a single-vehicle picture, and its picture establishes a corresponding relationship with the 19 attribute annotation. It is also possible to cut into a single-vehicle image first and then annotate it. The results are the same. + + + +## Model Training + +Once the data is annotated, it can be used for model training to complete the optimization of the customized model. + +There are two main steps: 1) Organize the data and annotated data into the training format. 2) Modify the configuration file to start training. + +### Training Data Format + +The training data includes the images used for training and a training list called train.txt. Its location is specified in the training configuration, with the following example: + +``` +Attribute/ +|-- data Training images folder +|-- 00001.jpg +|-- 00002.jpg +| `-- 0000x.jpg +train.txt List of training data +``` + +train.txt file contains the names of all training images (file path relative to the root path) + 19 annotation values + +Each line of it represents a vehicle's image and annotation result. The format is as follows: + +``` +00001.jpg 0,0,1,0,.... +``` + +Note 1) The images are separated by Tab[\t], 2) The annotated values are separated by commas [,]. If the format is wrong, the parsing will fail. + + +### Modify The Configuration To Start Training + +First run the following command to download the training code (for more environmental issues, please refer to [Install_PaddleClas](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.4/docs/en/installation/ install_paddleclas_en.md)): + +``` +git clone https://github.com/PaddlePaddle/PaddleClas +``` + +You need to modify the following configuration in the configuration file `PaddleClas/blob/develop/ppcls/configs/PULC/vehicle_attribute/PPLCNet_x1_0.yaml` + +```yaml +DataLoader: + Train: + dataset: + name: MultiLabelDataset + image_root: "dataset/VeRi/" # the root path of training images + cls_label_path: "dataset/VeRi/train_list.txt" # the location of the training list file + label_ratio: True + transform_ops: + ... + + Eval: + dataset: + name: MultiLabelDataset + image_root: "dataset/VeRi/" # the root path of evaluation images + cls_label_path: "dataset/VeRi/val_list.txt" # the location of the training list file + label_ratio: True + transform_ops: + ... +``` + +Note: + +1. here image_root path and the relative path of the image in train.txt, corresponding to the full path of the image. +2. If you modify the number of attributes, the number of attribute types in the content configuration item should also be modified accordingly. + +```yaml +# model architecture +Arch: +name: "PPLCNet_x1_0" +pretrained: True +use_ssld: True +class_num: 19 # Number of attribute classes +``` + +Then run the following command to start training: + +```bash +#Multi-card training +export CUDA_VISIBLE_DEVICES=0,1,2,3 +python3 -m paddle.distributed.launch \ + --gpus="0,1,2,3" \ + tools/train.py \ + -c ./ppcls/configs/PULC/vehicle_attribute/PPLCNet_x1_0.yaml + +#Single card training +python3 tools/train.py \ + -c ./ppcls/configs/PULC/vehicle_attribute/PPLCNet_x1_0.yaml +``` + +You can run the following commands for performance evaluation after the training is completed: + +``` +#Multi-card evaluation +export CUDA_VISIBLE_DEVICES=0,1,2,3 +python3 -m paddle.distributed.launch \ + --gpus="0,1,2,3" \ + tools/eval.py \ + -c ./ppcls/configs/PULC/vehicle_attribute/PPLCNet_x1_0.yaml \ + -o Global.pretrained_model=./output/PPLCNet_x1_0/best_model + +#Single card evaluation +python3 tools/eval.py \ + -c ./ppcls/configs/PULC/vehicle_attribute/PPLCNet_x1_0.yaml \ + -o Global.pretrained_model=./output/PPLCNet_x1_0/best_model +``` + +### Model Export + +Use the following command to export the trained model as an inference deployment model. + +``` +python3 tools/export_model.py \ + -c ./ppcls/configs/PULC/vehicle_attribute/PPLCNet_x1_0.yaml \ + -o Global.pretrained_model=output/PPLCNet_x1_0/best_model \ + -o Global.save_inference_dir=deploy/models/PPLCNet_x1_0_vehicle_attribute_model +``` + +After exporting the model, if want to use it in PP-Vehicle, you need to download the [deploy infer model](https://bj.bcebos.com/v1/paddledet/models/pipeline/vehicle_attribute_model.zip) and copy `infer_cfg.yml` into the exported model folder `PPLCNet_x1_0_vehicle_attribute_model` . + +When you use the model, you need to modify the new model path `model_dir` entry and set `enable: True` in the configuration file of PP-Vehicle `. /deploy/pipeline/config/infer_cfg_ppvehicle.yml` . + +``` +VEHICLE_ATTR: + model_dir: [YOUR_DEPLOY_MODEL_DIR]/PPLCNet_x1_0_vehicle_attribute_infer/ #The exported model location + enable: True #Whether to enable the function +``` + +To this point, a new attribute category recognition task is completed. + + + +## Adding or deleting attributes + +This is similar to the increase and decrease process of pedestrian attributes. + +If the attributes need to be added or deleted, you need to + +1) New attribute category information needs to be added or deleted when annotating the data. + +2) Modify the number and name of attributes used in train.txt corresponding to the training. + +3) Modify the training configuration, for example, the number of attributes in the ``PaddleClas/blob/develop/ppcls/configs/PULC/vehicle_attribute/PPLCNet_x1_0.yaml`` file, for details, please see the ``Modify configuration to start training`` section above. + +Example of adding attributes. + +1. Continue to add new attribute annotation values after 19 values when annotating the data. +2. Add new attribute values to the annotated values in the train.txt file as well. +3. The above is the annotation and training process with 19 attributes. + +
+ +
+ + +The same applies to the deletion of attributes. + + +## Modifications to post-processing code + +After modifying the attribute definition, the post-processing part of the pipeline also needs to be modified accordingly, which mainly affects the display results when the results are visualized. + + +The code is at [file](https://github.com/PaddlePaddle/PaddleDetection/blob/develop/deploy/pipeline/ppvehicle/vehicle_attr.py#L108), that is, the `postprocess` function. + +The function implementation is described as follows: + +```python + # The name of the color/model is defined in the initialization function of the class + self.color_list = [ + "yellow", "orange", "green", "gray", "red", "blue", "white", + "golden", "brown", "black" + ] + self.type_list = [ + "sedan", "suv", "van", "hatchback", "mpv", "pickup", "bus", "truck", + "estate" + ] + + ... + + def postprocess(self, inputs, result): + # postprocess output of predictor + im_results = result['output'] + batch_res = [] + for res in im_results: + res = res.tolist() + attr_res = [] + color_res_str = "Color: " + type_res_str = "Type: " + color_idx = np.argmax(res[:10]) # The first 10 items represent the color scores, and the item with the largest score is used as the color result + type_idx = np.argmax(res[10:]) # The last 9 items represent the model scores, and the item with the largest score is used as the model result. + + # The score of color and model need to be larger than the corresponding threshold, otherwise it will be regarded as 'UnKnown' + if res[color_idx] >= self.color_threshold: + color_res_str += self.color_list[color_idx] + else: + color_res_str += "Unknown" + attr_res.append(color_res_str) + + if res[type_idx + 10] >= self.type_threshold: + type_res_str += self.type_list[type_idx] + else: + type_res_str += "Unknown" + attr_res.append(type_res_str) + + batch_res.append(attr_res) + result = {'output': batch_res} + return result +``` -- GitLab