Skip to content

P
PaddleDetection

PaddlePaddle / PaddleDetection
1 年多 前同步成功

通知 696
Star 11112
Fork 2696
P
PaddleDetection
未验证 提交 8a1efe25 编写于 作者: W wangguanzhong 提交者： GitHub
浏览文件
操作

[cherry-pick] update ppvehicle en doc (#6919) 

* update ppvehicle en doc, test=document_fix

* refine vehicle doc, test=document_fix
上级 ec77c58e
显示空白变更内容
内联 并排
Showing with 681 addition and 22 deletion
deploy/pipeline/README_en.md
浏览文件 @ 8a1efe25
......@@ -69,22 +69,22 @@ Click to download the model, then unzip and save it in the `. /output_inference`
## 📚 Doc Tutorials
### [A Quick Start](docs/tutorials/PPHuman_QUICK_STARTED.md)
### [A Quick Start](docs/tutorials/PPHuman_QUICK_STARTED_en.md)
### Pedestrian attribute/feature recognition
* [A quick start](docs/tutorials/pphuman_attribute.md)
* [Customized release/2.5ment tutorials](../../docs/advanced_tutorials/customization/pphuman_attribute.md)
* [A quick start](docs/tutorials/pphuman_attribute_en.md)
* [Customized development tutorials](../../docs/advanced_tutorials/customization/pphuman_attribute_en.md)
* Data Preparation
* Model Optimization
* New Attributes
### Behavior detection
* [A quick start](docs/tutorials/pphuman_action.md)
* [A quick start](docs/tutorials/pphuman_action_en.md)
* Falling detection
* Fighting detection
* [Customized release/2.5ment tutorials](../../docs/advanced_tutorials/customization/action_recognotion/README.md)
* [Customized development tutorials](../../docs/advanced_tutorials/customization/action_recognotion/README_en.md)
* Solution Selection
* Data Preparation
* Model Optimization
......@@ -92,17 +92,17 @@ Click to download the model, then unzip and save it in the `. /output_inference`
### ReID
* [A quick start](docs/tutorials/pphuman_mtmct.md)
* [Customized release/2.5ment tutorials](../../docs/advanced_tutorials/customization/pphuman_mtmct.md)
* [A quick start](docs/tutorials/pphuman_mtmct_en.md)
* [Customized development tutorials](../../docs/advanced_tutorials/customization/pphuman_mtmct_en.md)
* Data Preparation
* Model Optimization
### Pedestrian tracking, visitor traffic statistics, trace records
* [A quick start](docs/tutorials/pphuman_mot.md)
* [A quick start](docs/tutorials/pphuman_mot_en.md)
* Pedestrian tracking,
* Visitor traffic statistics
* Regional intrusion diagnosis and counting
* [Customized release/2.5ment tutorials](../../docs/advanced_tutorials/customization/pphuman_mot.md)
* [Customized development tutorials](../../docs/advanced_tutorials/customization/pphuman_mot_en.md)
* Data Preparation
* Model Optimization
deploy/pipeline/docs/tutorials/PPHuman_QUICK_STARTED.md
浏览文件 @ 8a1efe25
# 快速开始
[English](PPHuman_QUICK_STARTED_en.md) | 简体中文
# PP-Human快速开始
## 目录
......
deploy/pipeline/docs/tutorials/PPHuman_QUICK_STARTED_en.md 0 → 100644
浏览文件 @ 8a1efe25
English | [简体中文](PPHuman_QUICK_STARTED.md)
# Quick Start for PP-Human
## Contents
- [Environment Preparation](#Environment-Preparation)
- [Model Download](#Model-Download)
- [Configuration](#Configuration)
- [Inference Deployment](#Inference-Deployment)
- [Parameters](#Parameters)
- [Solutions](#Solutions)
- [Pedestrian Detection](#edestrian-Detection)
- [Pedestrian Tracking](#Pedestrian-Tracking)
- [Multi-camera & multi-pedestrain tracking](#Multi-camera-&-multi-pedestrain-tracking)
- [Attribute Recognition](#Attribute-Recognition)
- [Behavior Recognition](#Behavior-Recognition)
## Environment Preparation
Environment requirements： PaddleDetection>= release/2.4 or develop version
Installation of PaddlePaddle and PaddleDetection
```
# PaddlePaddle CUDA10.1
python -m pip install paddlepaddle-gpu==2.2.2.post101 -f https://www.paddlepaddle.org.cn/whl/linux/mkl/avx/stable.html
# PaddlePaddle CPU
python -m pip install paddlepaddle -i https://mirror.baidu.com/pypi/simple
#Clone PaddleDetection repositories
cd <path/to/clone/PaddleDetection>
git clone https://github.com/PaddlePaddle/PaddleDetection.git
# Install dependencies
cd PaddleDetection
pip install -r requirements.txt
```
1. For installation details, please refer to [Installation Tutorials](../../../../docs/tutorials/INSTALL.md)
2. If you need TensorRT inference acceleration (speed measurement), please install PaddlePaddle with `TensorRT version`. You can download and install it from the [PaddlePaddle Installation Package](https://paddleinference.paddlepaddle.org.cn/v2.2/user_guides/download_lib.html#python) or follow the [Instructions](https://www. paddlepaddle.org.cn/inference/master/optimize/paddle_trt.html) or use docker, or self-compiling to prepare the environment.
## Model Download
PP-Human provides object detection, attribute recognition, behaviour recognition and ReID pre-trained models for different applications. Developers can download them directly.
| Task | End-to（ms） | Model Solution | Model Size |
|:--------------------------------------:|:--------------------:|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------:|
| Pedestrian Detection (high precision) | 25.1ms | [Multi-Object Tracking](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_l_36e_pipeline.zip) | 182M |
| Pedestrian Detection (Lightweight) | 16.2ms | [Multi-Object Tracking](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_s_36e_pipeline.zip) | 27M |
| Pedestrian Tracking (high precision) | 31.8ms | [Multi-Object Tracking](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_l_36e_pipeline.zip) | 182M |
| Pedestrian Tracking (Lightweight) | 21.0ms | [Multi-Object Tracking](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_s_36e_pipeline.zip) | 27M |
| Attribute Recognition (high precision) | Single Person 8.5ms | [Object Detection](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_l_36e_pipeline.zip)<br> [Attribute Recognition](https://bj.bcebos.com/v1/paddledet/models/pipeline/PPHGNet_small_person_attribute_954_infer.zip) | Object Detection：182M<br>Attribute Recogniton：86M |
| Attribute Recognition (Lightweight) | Single Person 7.1ms | [Object Detection](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_l_36e_pipeline.zip)<br> [Attribute Recogniton](https://bj.bcebos.com/v1/paddledet/models/pipeline/PPLCNet_x1_0_person_attribute_945_infer.zip) | Object Detection：182M<br>Attribute Recogniton：86M |
| Falling Detection | Single Person 10ms | [Multi-Object Tracking](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_l_36e_pipeline.zip) <br> [Keypoint Detection](https://bj.bcebos.com/v1/paddledet/models/pipeline/dark_hrnet_w32_256x192.zip) <br> [Skeleton Based Action Recognition](https://bj.bcebos.com/v1/paddledet/models/pipeline/STGCN.zip) | Multi-Object Tracking：182M<br>Keypoint Detection：101M<br>Skeleton Based Action Recognition：21.8M |
| Breaking-In Detection | 31.8ms | [Multi-Object Tracking](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_l_36e_pipeline.zip) | Multi-Object Tracking：182M |
| Fighting Detection | 19.7ms | [Video Classification](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_l_36e_pipeline.zip) | 90M |
| Smoking Detection | Single Person 15.1ms | [Object Detection](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_l_36e_pipeline.zip)<br>[Object Detection Based On Body ID](https://bj.bcebos.com/v1/paddledet/models/pipeline/ppyoloe_crn_s_80e_smoking_visdrone.zip) | Object Detection：182M<br>Object Detection Based On Body ID：27M |
| Phone-calling Detection | Single Person 6.0ms | [Object Detection](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_l_36e_pipeline.zip)<br>[Image Classification Based On Body ID](https://bj.bcebos.com/v1/paddledet/models/pipeline/PPHGNet_tiny_calling_halfbody.zip) | Object Detection：182M<br>Image Classification Based On Body ID：45M |
Download the model and unzip it into the `. /output_inference` folder.
In the configuration file, the model path defaults to the download path of the model. If the user does not change it, the corresponding model will be downloaded automatically upon inference.
**Note:**
- Model accuracy is tested on fused datasets, which contain both open source and enterprise datasets.
- ReID model accuracy is tested on the Market1501 dataset
- Prediction speed is obtained at T4 with TensorRT FP16 enabled, which includes data pre-processing, model inference and post-processing.
## Configuration
The PP-Human-related configuration is located in ``deploy/pipeline/config/infer_cfg_pphuman.yml``, and this configuration file contains all the features currently supported by PP-Human. If you want to see the configuration for a specific feature, please refer to the relevant configuration in ``deploy/pipeline/config/examples/``. In addition, the contents of the configuration file can be modified with the `-o`command line parameter. E.g. to modify the model directory of an attribute, developers can run ```-o ATTR.model_dir="DIR_PATH"``.
The features and corresponding task types are as follows.
| Input | Feature | Task | Config |
| ------------------- | --------------------- | ---------------------------------------------------------- | ----------------------- |
| Image | Attribute Recognition | Object Detection Attribute Recognition | DET ATTR |
| Single-camera video | Attribute Recognition | Multi-Object Tracking Attribute Recognition | MOT ATTR |
| Single-camera video | Behaviour Recognition | Multi-Object Tracking Keypoint Detection Falling detection | MOT KPT SKELETON_ACTION |
Take attribute recognition based on video input as an example: Its task type includes multi-object tracking and attributes recognition. The specific configuration is as follows.
```
crop_thresh: 0.5
attr_thresh: 0.5
visual: True
MOT:
model_dir: https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_l_36e_pipeline.zip
tracker_config: deploy/pipeline/config/tracker_config.yml
batch_size: 1
enable: True
ATTR:
model_dir: https://bj.bcebos.com/v1/paddledet/models/pipeline/PPLCNet_x1_0_person_attribute_945_infer.zip
batch_size: 8
enable: True
```
**Note:**
- If developer needs to carry out different tasks, set the corresponding enables option to be True in the configuration file.
## Inference Deployment
1. Use the default configuration directly or the configuration file in examples, or modify the configuration in `infer_cfg_pphuman.yml`
```
# Example: In pedestrian detection model, specify configuration file path and test image, and image input opens detection model by default
python deploy/pipeline/pipeline.py --config deploy/pipeline/config/infer_cfg_pphuman.yml --image_file=test_image.jpg --device=gpu
# Example: In pedestrian attribute recognition, directly configure the examples
python deploy/pipeline/pipeline.py --config deploy/pipeline/config/examples/infer_cfg_human_attr.yml --video_file=test_video.mp4 --device=gpu
```
2. Use the command line to enable functions or change the model path.
```
# Example: Pedestrian tracking, specify config file path, model path and test video. The specified model path on the command line has a higher priority than the config file.
python deploy/pipeline/pipeline.py --config deploy/pipeline/config/infer_cfg_pphuman.yml -o MOT.enable=True MOT.model_dir=ppyoloe_infer/ --video_file=test_video.mp4 --device=gpu
# Example: In behaviour recognition, with fall recognition as an example, enable the SKELETON_ACTION model on the command line
python deploy/pipeline/pipeline.py --config deploy/pipeline/config/infer_cfg_pphuman.yml -o SKELETON_ACTION.enbale=True --video_file=test_video.mp4 --device=gpu
```
3. For rtsp stream, use --rtsp RTSP [RTSP ...] parameter to specify one or more rtsp streams. Separate the multiple addresses with a space, or replace the video address directly after the video_file with the rtsp stream address), examples as follows
```
# Example: Single video stream for pedestrian attribute recognition
python deploy/pipeline/pipeline.py --config deploy/pipeline/config/examples/infer_cfg_human_attr.yml -o visual=False --rtsp rtsp://[YOUR_RTSP_SITE] --device=gpu
# Example: Multiple-video stream for pedestrian attribute recognition
python deploy/pipeline/pipeline.py --config deploy/pipeline/config/examples/infer_cfg_human_attr.yml -o visual=False --rtsp rtsp://[YOUR_RTSP_SITE1] rtsp://[YOUR_RTSP_SITE2] --device=gpu |
```
### Jetson Deployment
Due to the large gap in computing power of the Jetson platform compared to the server, we suggest:
1. choose a lightweight model, especially for tracking model, `ppyoloe_s: https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_s_36e_pipeline.zip` is recommended
2. For frame skipping of tracking; we recommend 2 or 3: `skip_frame_num: 3`
With this recommended configuration, it is possible to achieve higher speeds on the TX2 platform. It has been tested with attribute case, with speeds up to 20fps. The configuration file can be modified directly (recommended) or from the command line (not recommended due to its long fields).
### Parameters
| Parameters | Necessity | Implications |
| ---------------------- | --------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| --config | Yes | Path to configuration file |
| -o | Option | Overwrite the corresponding configuration in the configuration file |
| --image_file | Option | Images to be predicted |
| --image_dir | Option | Path to the images folder to be predicted |
| --video_file | Option | Video to be predicted, or rtsp stream address (rtsp parameter recommended) |
| --rtsp | Option | rtsp video stream address, supports one or more simultaneous streams input |
| --camera_id | Option | The camera ID for prediction, default is -1 ( for no camera prediction, can be set to 0 - (number of cameras - 1) ), press `q` in the visualization interface during the prediction process to output the prediction result to: output/output.mp4 |
| --device | Option | Running device, options include `CPU/GPU/XPU`, and the default is `CPU`. |
| --output_dir | Option | The root directory for the visualization results, and the default is output/ |
| --run_mode | Option | For GPU, the default is paddle, with (paddle/trt_fp32/trt_fp16/trt_int8) as optional |
| --enable_mkldnn | Option | Whether to enable MKLDNN acceleration in CPU prediction, the default is False |
| --cpu_threads | Option | Set the number of cpu threads, and the default is 1 |
| --trt_calib_mode | Option | Whether TensorRT uses the calibration function, and the default is False; set to True when using TensorRT's int8 function and False when using the PaddleSlim quantized model |
| --do_entrance_counting | Option | Whether to count entrance/exit traffic flows, the default is False |
| --draw_center_traj | Option | Whether to map the trace, the default is False |
| --region_type | Option | 'horizontal' (default), 'vertical': traffic count direction; 'custom': set break-in area |
| --region_polygon | Option | Set the coordinates of the polygon multipoint in the break-in area. No default. |
| --do_break_in_counting | Option | Area break-in checks |
## Solutions
The overall solution for PP-Human v2 is shown in the graph below:
### Pedestrian detection
- Take PP-YOLOE L as the object detection model
- For detailed documentation, please refer to [PP-YOLOE](... /... /... /... /configs/ppyoloe/) and [Multiple-Object-Tracking](pphuman_mot_en.md)
### Pedestrian tracking
- Vehicle tracking by SDE solution
- Adopt PP-YOLOE L (high precision) and S (lightweight) for detection models
- Adopt the OC-SORT solution for racking module
- Refer to [OC-SORT](... /... /... /... /configs/mot/ocsort) and [Multi-Object Tracking](pphuman_mot_en.md) for details
### Multi-camera & multi-pedestrain tracking
- Use PP-YOLOE & OC-SORT to acquire single-camera multi-object tracking trajectory
- Extract features for each frame using ReID (StrongBaseline network).
- Match multi-camera trajectory features to obtain multi-camera tracking results.
- Refer to [Multi-camera & multi-pedestrain tracking](pphuman_mtmct_en.md) for details.
### Attribute Recognition
- Use PP-YOLOE + OC-SORT to track the human body.
- Use PP-HGNet, PP-LCNet (multi-classification model) to complete the attribute recognition. Main attributes include age, gender, hat, eyes, top and bottom dressing style, backpack.
- Refer to [attribute recognition](pphuman_attribute_en.md) for details.
### Behaviour Recognition:
- Four behaviour recognition solutions are provided:
- 1. Behaviour recognition based on skeletal points, e.g. falling recognition
- 2. Behaviour recognition based on image classification, e.g. phone call recognition
- 3. Behaviour recognition based on detection, e.g. smoking recognition
- 4. Behaviour recognition based on Video classification, e.g. fighting recognition
- For details, please refer to [Behaviour Recognition](pphuman_action_en.md)
deploy/pipeline/docs/tutorials/PPVehicle_QUICK_STARTED.md
浏览文件 @ 8a1efe25
# 快速开始
[English](PPVehicle_QUICK_STARTED_en.md) | 简体中文
# PP-Vehicle快速开始
## 目录
......@@ -12,7 +14,7 @@
- [车辆跟踪](#车辆跟踪)
- [车牌识别](#车牌识别)
- [属性识别](#属性识别)
- [违法停车识别](#违法停车识别)
- [违章停车识别](#违章停车识别)
## 环境准备
......@@ -118,7 +120,7 @@ python deploy/pipeline/pipeline.py --config deploy/pipeline/config/examples/infe
# 例：车辆跟踪，指定配置文件路径和测试视频，命令行中开启MOT模型并修改模型路径，命令行中指定的模型路径优先级高于配置文件
python deploy/pipeline/pipeline.py --config deploy/pipeline/config/infer_cfg_ppvehicle.yml -o MOT.enable=True MOT.model_dir=ppyoloe_infer/ --video_file=test_video.mp4 --device=gpu
# 例：车辆违法分析，指定配置文件和测试视频，命令行中指定违停区域设置、违停时间判断。
# 例：车辆违章分析，指定配置文件和测试视频，命令行中指定违停区域设置、违停时间判断。
python deploy/pipeline/pipeline.py --config deploy/pipeline/config/examples/infer_cfg_illegal_parking.yml \
--video_file=../car_test.mov \
--device=gpu \
......@@ -200,6 +202,6 @@ PP-Vehicle 整体方案如下图所示:
- 使用PaddleOCR特色模型ch_PP-OCRv3_det+ch_PP-OCRv3_rec模型，识别车牌号码
- 详细文档参考[车牌识别](ppvehicle_plate.md)
### 违法停车识别
- 车辆跟踪模型使用高精度模型PP-YOLOE L，根据车辆的跟踪轨迹以及指定的违停区域判断是否违法停车，如果存在则展示违法停车车牌号。
- 详细文档参考[违法停车识别](ppvehicle_illegal_parking.md)
### 违章停车识别
- 车辆跟踪模型使用高精度模型PP-YOLOE L，根据车辆的跟踪轨迹以及指定的违停区域判断是否违章停车，如果存在则展示违章停车车牌号。
- 详细文档参考[违章停车识别](ppvehicle_illegal_parking.md)
deploy/pipeline/docs/tutorials/PPVehicle_QUICK_STARTED_en.md 0 → 100644
浏览文件 @ 8a1efe25
English | [简体中文](PPVehicle_QUICK_STARTED.md)
# Quick Start for PP-Vehicle
## Content
- [Environment Preparation](#Environment-Preparation)
- [Model Download](#Model-Download)
- [Configuration](#Configuration)
- [Inference Deployment](#Inference-Deployment)
- [Parameters](#Parameters)
- [Solutions](#Solutions)
- [Vehicle Detection](#Vehicle-Detection)
- [Vehicle Tracking](#Vehicle-Tracking)
- [License Plate Recognition](#License-Plate-Recognition)
- [Attribute Recognition](#Attribute-Recognition)
- [Illegal Parking Detection](#Illegal-Parking-Detection)
## Environment Preparation
Environment Preparation: PaddleDetection version >= release/2.4 or release/2.5
Installation of PaddlePaddle and PaddleDetection
```
# PaddlePaddle CUDA10.1
python -m pip install paddlepaddle-gpu==2.2.2.post101 -f https://www.paddlepaddle.org.cn/whl/linux/mkl/avx/stable.html
# PaddlePaddle CPU
python -m pip install paddlepaddle -i https://mirror.baidu.com/pypi/simple
# Clone PaddleDetectionrepositories
cd <path/to/clone/PaddleDetection>
git clone https://github.com/PaddlePaddle/PaddleDetection.git
# Installing dependencies
cd PaddleDetection
pip install -r requirements.txt
```
1. For installation details, please refer to [Installation Tutorials](../../../../docs/tutorials/INSTALL.md)
2. If you need TensorRT inference acceleration (speed measurement), please install PaddlePaddle with `TensorRT version`. You can download and install it from the [PaddlePaddle Installation Package](https://paddleinference.paddlepaddle.org.cn/v2.2/user_guides/download_lib.html#python) or follow the [Instructions]([https://www](https://www). paddlepaddle.org.cn/inference/master/optimize/paddle_trt.html) or use docker, or self-compiling to prepare the environment.
## Model Download
PP-Vehicle provides object detection, attribute recognition, behaviour recognition and ReID pre-trained models for different applications. Developers can download them directly.
| Task | End-to（ms） | Model Solution | Model Size |
|:---------------------------------:|:----------:|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------:|:---------------------------------------------------------------------:|
| Vehicle Detection（high precision） | 25.7ms | [Multi-Object Tracking](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_l_36e_ppvehicle.zip) | 182M |
| Vehicle Detection（Lightweight） | 13.2ms | [Multi-Object Tracking](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_s_36e_ppvehicle.zip) | 27M |
| Vehicle Tracking（high precision） | 40ms | [Multi-Object Tracking](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_l_36e_ppvehicle.zip) | 182M |
| Vehicle Tracking（Lightweight） | 25ms | [Multi-Object Tracking](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_s_36e_ppvehicle.zip) | 27M |
| License plate recognition | 4.68ms | [License plate recognition](https://bj.bcebos.com/v1/paddledet/models/pipeline/ch_PP-OCRv3_det_infer.tar.gz) <br> [License plate character recognition](https://bj.bcebos.com/v1/paddledet/models/pipeline/ch_PP-OCRv3_rec_infer.tar.gz) | Vehicle Detection：3.9M <br> License plate character recognition： 12M |
| Vehicle Attribute Recognition | 7.31ms | [Vehicle Attribute](https://bj.bcebos.com/v1/paddledet/models/pipeline/vehicle_attribute_model.zip) | 7.2M |
Download the model and unzip it into the `. /output_inference` folder.
In the configuration file, the model path defaults to the download path of the model. If the user does not change it, the corresponding model will be downloaded automatically upon inference.
**Notes：**
- The accuracy of detection tracking model is obtained from the joint dataset PPVehicle (integration of the public dataset BDD100K-MOT and UA-DETRAC). For more details, please refer to [PP-Vehicle](... /... /... /... /configs/ppvehicle)
- Inference speed is obtained at T4 with TensorRT FP16 enabled, which includes data pre-processing, model inference and post-processing.
## Configuration
PP-Vehicle related configuration locates in ``deploy/pipeline/config/infer_cfg_ppvehicle.yml``. Developers need to set specific task types to use different features.
The features and corresponding task types are as follows.
| Input | Feature | Task | Config |
| ------------------- | --------------------- | ------------------------------------------- | ---------------- |
| Image | Attribute Recognition | Object Detection Attribute Recognition | DET ATTR |
| Single-camera video | Attribute Recognition | Multi-Object Tracking Attribute Recognition | MOT ATTR |
| Single-camera video | Attribute Recognition | Multi-Object Tracking Attribute Recognition | MOT VEHICLEPLATE |
Take attribute recognition based on video input as an example: Its task type includes multi-object tracking and attributes recognition. The specific configuration is as follows.
```
crop_thresh: 0.5
visual: True
warmup_frame: 50
MOT:
model_dir: https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_l_36e_ppvehicle.zip
tracker_config: deploy/pipeline/config/tracker_config.yml
batch_size: 1
enable: True
VEHICLE_ATTR:
model_dir: https://bj.bcebos.com/v1/paddledet/models/pipeline/vehicle_attribute_model.zip
batch_size: 8
color_threshold: 0.5
type_threshold: 0.5
enable: True
```
**Notes：**
- If the developer needs to carry out different tasks, set the corresponding enables option to be True in the configuration file.
- If the developer only needs to modify the model file path, run the command line with `-o MOT.model_dir=ppyoloe/` after --config, or manually modify the corresponding model path in the configuration file. For more details, please refer to the following parameter descriptions
## Inference Deployment
1. Use the default configuration directly or the configuration file in examples, or modify the configuration in `infer_cfg_ppvehicle.yml`
```
# Example：In vehicle detection，specify configuration file path and test image
python deploy/pipeline/pipeline.py --config deploy/pipeline/config/infer_cfg_ppvehicle.yml --image_file=test_image.jpg --device=gpu
# Example：In license plate recognition，directly configure the examples
python deploy/pipeline/pipeline.py --config deploy/pipeline/config/examples/infer_cfg_vehicle_plate.yml --video_file=test_video.mp4 --device=gpu
```
2. Use the command line to enable functions or change the model path.
```
# Example：In vehicle tracking，specify configuration file path and test video, Turn on the MOT model and modify the model path on the command line, the model path specified on the command line has higher priority than the configuration file
python deploy/pipeline/pipeline.py --config deploy/pipeline/config/infer_cfg_ppvehicle.yml -o MOT.enable=True MOT.model_dir=ppyoloe_infer/ --video_file=test_video.mp4 --device=gpu
# Example：In vehicle illegal action analysis，specify configuration file path and test video, 命令行中指定违停区域设置、违停时间判断。
python deploy/pipeline/pipeline.py --config deploy/pipeline/config/examples/infer_cfg_illegal_parking.yml \
--video_file=../car_test.mov \
--device=gpu \
--draw_center_traj \
--illegal_parking_time=3 \
--region_type=custom \
--region_polygon 600 300 1300 300 1300 800 600 800
```
3. For rtsp stream, use --rtsp RTSP [RTSP ...] parameter to specify one or more rtsp streams. Separate the multiple addresses with a space, or replace the video address directly after the video_file with the rtsp stream address), examples as follows
```
# Example: Single video stream for pedestrian attribute recognition
python deploy/pipeline/pipeline.py --config deploy/pipeline/config/examples/infer_cfg_human_attr.yml -o visual=False --rtsp rtsp://[YOUR_RTSP_SITE] --device=gpu
# Example: Multiple-video stream for pedestrian attribute recognition
python deploy/pipeline/pipeline.py --config deploy/pipeline/config/examples/infer_cfg_human_attr.yml -o visual=False --rtsp rtsp://[YOUR_RTSP_SITE1] rtsp://[YOUR_RTSP_SITE2] --device=gpu |
```
### Jetson Deployment
Due to the large gap in computing power of the Jetson platform compared to the server, we suggest:
1. choose a lightweight model, especially for tracking model, `ppyoloe_s: https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_s_36e_pipeline.zip` is recommended
2. For frame skipping of tracking; we recommend 2 or 3: `skip_frame_num: 3`
With this recommended configuration, it is possible to achieve higher speeds on the TX2 platform. It has been tested with attribute case, with speeds up to 20fps. The configuration file can be modified directly (recommended) or from the command line (not recommended due to its long fields).
### Parameters
#
| Parameters | Necessity | Implications |
| ---------------------- | --------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| --config | Yes | Path to configuration file |
| -o | Option | Overwrite the corresponding configuration in the configuration file |
| --image_file | Option | Images to be predicted |
| --image_dir | Option | Path to the images folder to be predicted |
| --video_file | Option | Video to be predicted, or rtsp stream address (rtsp parameter recommended) |
| --rtsp | Option | rtsp video stream address, supports one or more simultaneous streams input |
| --camera_id | Option | The camera ID for prediction, default is -1 ( for no camera prediction, can be set to 0 - (number of cameras - 1) ), press `q` in the visualization interface during the prediction process to output the prediction result to: output/output.mp4 |
| --device | Option | Running device, options include `CPU/GPU/XPU`, and the default is `CPU`. |
| --output_dir | Option | The root directory for the visualization results, and the default is output/ |
| --run_mode | Option | For GPU, the default is paddle, with (paddle/trt_fp32/trt_fp16/trt_int8) as optional |
| --enable_mkldnn | Option | Whether to enable MKLDNN acceleration in CPU prediction, the default is False |
| --cpu_threads | Option | Set the number of cpu threads, and the default is 1 |
| --trt_calib_mode | Option | Whether TensorRT uses the calibration function, and the default is False; set to True when using TensorRT's int8 function and False when using the PaddleSlim quantized model |
| --do_entrance_counting | Option | Whether to count entrance/exit traffic flows, the default is False |
| --draw_center_traj | Option | Whether to draw center trajectory, the default is False |
| --region_type | Option | 'horizontal' (default), 'vertical': traffic count direction; 'custom': set illegal parking area |
| --region_polygon | Option | Set the coordinates of the polygon multipoint in the illegal parking area. No default. |
| --illegal_parking_time | Option | Set the time threshold for illegal parking in seconds (s), -1 (default) indicates no check |
## Solutions
The overall solution for PP-Vehicle v2 is shown in the graph below:
<div width="1000" align="center">
<img src="../../../../docs/images/ppvehicle.png"/>
</div>
###
### Vehicle detection
- Take PP-YOLOE L as the object detection model
- For detailed documentation, please refer to [PP-YOLOE](... /... /... /... /configs/ppyoloe/) and [Multiple-Object-Tracking](ppvehicle_mot_en.md)
### Vehicle tracking
- Vehicle tracking by SDE solution
- Adopt PP-YOLOE L (high precision) and S (lightweight) for detection models
- Adopt the OC-SORT solution for racking module
- Refer to [OC-SORT](... /... /... /... /configs/mot/ocsort) and [Multi-Object Tracking](ppvehicle_mot_en.md) for details
### Attribute Recognition
- Use PP-LCNet provided by PaddleClas to recognize vehicle colours and model attributes.
- For details, please refer to [Attribute Recognition](ppvehicle_attribute_en.md)
### License plate recognition
- Use ch_PP-OCRv3_det+ch_PP-OCRv3_rec model to recognize license plate number
- For details, please refer to [Plate Recognition](ppvehicle_plate_en.md)
### Illegal Parking Detection
- Use vehicle tracking model (high precision) PP-YOLOE L to determine whether the parking is illegal based on the vehicle's trajectory and the designated illegal parking area. If it is illegal parking, display the illegal parking plate number.
- For details, please refer to [Illegal Parking Detection](ppvehicle_illegal_parking_en.md)
deploy/pipeline/docs/tutorials/ppvehicle_illegal_parking.md
浏览文件 @ 8a1efe25
# PP-Vehicle违法停车识别模块
# PP-Vehicle违章停车识别模块
禁停区域违法停车识别在车辆应用场景中有着非常广泛的应用，借助AI的力量可以减轻人力投入，精准快速识别出违停车辆并进一步采取如广播驱离行为。PP-Vehicle中基于车辆跟踪模型、车牌检测模型和车牌识别模型实现了违法停车识别功能，具体模型信息如下：
禁停区域违章停车识别在车辆应用场景中有着非常广泛的应用，借助AI的力量可以减轻人力投入，精准快速识别出违停车辆并进一步采取如广播驱离行为。PP-Vehicle中基于车辆跟踪模型、车牌检测模型和车牌识别模型实现了违章停车识别功能，具体模型信息如下：
| 任务 | 算法 | 精度 | 预测速度(ms) |预测模型下载链接 |
|:---------------------|:---------:|:------:|:------:| :---------------------------------------------------------------------------------: |
......@@ -56,8 +56,8 @@ python deploy/pipeline/pipeline.py --config deploy/pipeline/config/examples/infe
- region_polygon：自定义非法停车多边形，至少为3个点。
**注意:**
- 违法停车的测试视频必须是静止摄像头拍摄的，镜头不能抖动或移动。
- 判断车辆是否在违停区域内是**以车辆的中心点**作为参考，车辆擦边而过等场景不算作违法停车。
- 违章停车的测试视频必须是静止摄像头拍摄的，镜头不能抖动或移动。
- 判断车辆是否在违停区域内是**以车辆的中心点**作为参考，车辆擦边而过等场景不算作违章停车。
- `--region_polygon`表示用户自定义区域的多边形的点坐标序列，每两个为一对点坐标(x,y)，**按顺时针顺序**连成一个**封闭区域**，至少需要3对点也即6个整数，默认值是`[]`，需要用户自行设置点坐标，如是四边形区域，坐标顺序是`左上、右上、右下、左下`。用户可以运行[此段代码](../../tools/get_video_info.py)获取所测视频的分辨率帧数，以及可以自定义画出自己想要的多边形区域的可视化并自己调整。
自定义多边形区域的可视化代码运行如下：
```python
......@@ -83,8 +83,8 @@ python deploy/pipeline/pipeline.py --config deploy/pipeline/config/examples/infe
## 方案说明
1. 目标检测/多目标跟踪获取图片/视频输入中的车辆检测框，模型方案为PP-YOLOE，详细文档参考[PP-YOLOE](../../../configs/ppyoloe/README_cn.md)
2. 基于跟踪算法获取每辆车的轨迹，如果车辆中心在违停区域内且在指定时间内未发生移动，则视为违法停车；
3. 使用车牌识别模型得到违法停车车牌并可视化。
2. 基于跟踪算法获取每辆车的轨迹，如果车辆中心在违停区域内且在指定时间内未发生移动，则视为违章停车；
3. 使用车牌识别模型得到违章停车车牌并可视化。
## 参考资料
......
deploy/pipeline/docs/tutorials/ppvehicle_mot_en.md 0 → 100644
浏览文件 @ 8a1efe25
English | [简体中文](ppvehicle_mot.md)
# PP-Vehicle Vehicle Tracking Module
【Application Introduction】
Vehicle detection and tracking are widely used in traffic monitoring and autonomous driving. The detection and tracking module is integrated in PP-Vehicle, providing a solid foundation for tasks including license plate detection and vehicle attribute recognition. We provide pre-trained models that can be directly used by developers.
【Model Download】
| Task | Algorithm | Accuracy | Inference speed(ms) | Download Link |
| -------------------------- | ---------- | ----------------------- | ------------------------------------ | ------------------------------------------------------------------------------------------ |
| Vehicle Detection/Tracking | PP-YOLOE-l | mAP: 63.9<br>MOTA: 50.1 | Detection: 25.1ms<br>Tracking：31.8ms | [Link](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_l_36e_ppvehicle.zip) |
| Vehicle Detection/Tracking | PP-YOLOE-s | mAP: 61.3<br>MOTA: 46.8 | Detection: 16.2ms<br>Tracking：21.0ms | [Link](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_s_36e_ppvehicle.zip) |
1. The detection/tracking model uses the PPVehicle dataset ( which integrates BDD100K-MOT and UA-DETRAC). The dataset merged car, truck, bus, van from BDD100K-MOT and car, bus, van from UA-DETRAC all into 1 class vehicle(1). The detection accuracy mAP was tested on the test set of PPVehicle, and the tracking accuracy MOTA was obtained on the test set of BDD100K-MOT (`car, truck, bus, van` were combined into 1 class `vehicle`). For more details about the training procedure, please refer to [ppvehicle](... /... /... /... /configs/ppvehicle).
2. Inference speed is obtained at T4 with TensorRT FP16 enabled, which includes data pre-processing, model inference and post-processing.
## How To Use
【Config】
The parameters associated with the attributes in the configuration file are as follows.
```
DET:
model_dir: output_inference/mot_ppyoloe_l_36e_ppvehicle/ # Vehicle detection model path
batch_size: 1 # Batch_size size for model inference
MOT:
model_dir: output_inference/mot_ppyoloe_l_36e_ppvehicle/ # Vehicle tracking model path
tracker_config: deploy/pipeline/config/tracker_config.yml
batch_size: 1 # Batch_size size for model inference, 1 only for tracking task.
skip_frame_num: -1 # Number of frames to skip, -1 means no skipping, the maximum skipped frames are recommended to be 3
enable: False # Whether or not to enable this function, please make sure it is set to True before tracking
```
【Usage】
1. Download the model from the link in the table above and unzip it to ``. /output_inference`` and change the model path in the configuration file. The default is to download the model automatically, no changes are needed.
2. The image input will start a pure detection task, and the start command is as follows
```python
python deploy/pipeline/pipeline.py --config deploy/pipeline/config/infer_cfg_ppvehicle.yml \
--image_file=test_image.jpg \
--device=gpu
```
3. Video input will start a tracking task. Please set `enable=True` for the MOT configuration in infer_cfg_ppvehicle.yml. If skip frames are needed for faster detection and tracking, it is recommended to set `skip_frame_num: 2` the maximum should not exceed 3.
```
MOT:
model_dir: https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_l_36e_ppvehicle.zip
tracker_config: deploy/pipeline/config/tracker_config.yml
batch_size: 1
skip_frame_num: 2
enable: True
```
```python
python deploy/pipeline/pipeline.py --config deploy/pipeline/config/infer_cfg_ppvehicle.yml \
--video_file=test_video.mp4 \
--device=gpu
```
4. There are two ways to modify the model path
- Config different model path in```./deploy/pipeline/config/infer_cfg_ppvehicle.yml```. The detection and tracking models correspond to the `DET` and `MOT` fields respectively. Modify the path under the corresponding field to the actual path.
- **[Recommand]** Add`-o MOT.model_dir=[YOUR_DETMODEL_PATH]` after the config in the command line to modify model path
```python
python deploy/pipeline/pipeline.py --config deploy/pipeline/config/infer_cfg_ppvehicle.yml \
--video_file=test_video.mp4 \
--device=gpu \
--region_type=horizontal \
--do_entrance_counting \
--draw_center_traj \
-o MOT.model_dir=ppyoloe/
```
**Note:**
- `--do_entrance_counting` : Whether to count entrance/exit traffic flows, the default is False
- `--draw_center_traj` : Whether to draw center trajectory, the default is False. Its input video is preferably taken from a still camera
- `--region_type` : The region for traffic counting. When setting `--do_entrance_counting`, there are two options: `horizontal` or `vertical`. The default is `horizontal`, which means the center horizontal line of the video picture is the entrance and exit. When the center point of the same object frame is on both sides of the centre horizontal line of the region in two adjacent seconds, i.e. the count adds 1.
5. Regional break-in and counting
Please set the MOT config: enable=True in `infer_cfg_ppvehicle.yml` before running the starting command:
```
python deploy/pipeline/pipeline.py --config deploy/pipeline/config/infer_cfg_ppvehicle.yml \
--video_file=test_video.mp4 \
--device=gpu \
--draw_center_traj \
--do_break_in_counting \
--region_type=custom \
--region_polygon 200 200 400 200 300 400 100 400
```
**Note:**
- Test video of area break-ins must be taken from a still camera, with no shaky or moving footage.
- `--do_break_in_counting`Indicates whether or not to count the entrance and exit of the area. The default is False.
- `--region_type` indicates the region for traffic counting, when setting `--do_break_in_counting` only `custom` can be selected, and the default is `custom`. It means that the customized region is used as the entry and exit. When the coordinates of the lower boundary midpoint of the same object frame goes to the inside of the region within two adjacent seconds, i.e. the count adds one.
- `--region_polygon` indicates a sequence of point coordinates for a polygon in a customized region. Every two are a pair of point coordinates (x,y). **In clockwise order** they are connected into a **closed region**, at least 3 pairs of points are needed (or 6 integers). The default value is `[]`. Developers need to set the point coordinates manually. If it is a quadrilateral region, the coordinate order is `top left, top right , bottom right, bottom left`. Developers can run [this code](... /... /tools/get_video_info.py) to get the resolution frames of the predicted video. It also supports customizing and adjusting the visualisation of the polygon area.
The code for the visualisation of the customized polygon area runs as follows.
```python
python get_video_info.py --video_file=demo.mp4 --region_polygon 200 200 400 200 300 400 100 400
```
A quick tip for drawing customized area: first take any point to get the picture, open it with the drawing tool, mouse over the area point and the coordinates will be displayed, record it and round it up, use it as a region_polygon parameter for this visualisation code and run the visualisation again, and fine-tune the point coordinates parameter.
【Showcase】
<div width="1000" align="center">
<img src="../images/mot_vehicle.gif"/>
</div>
## Solution
【Solution and feature】
- PP-YOLOE is adopted for vehicle detection frame of object detection, multi-object tracking in the picture/video input. For details, please refer to [PP-YOLOE](... /... /... /configs/ppyoloe/README_cn.md) and [PPVehicle](../../../../configs/ppvehicle)
- [OC-SORT](https://arxiv.org/pdf/2203.14360.pdf) is adopted as multi-object tracking model. PP-YOLOE replaced YOLOX as detector, and OCSORTTracker is the tracker. For more details, please refer to [OC-SORT](../../../../configs/mot/ocsort)
## Reference
```
@article{cao2022observation,
title={Observation-Centric SORT: Rethinking SORT for Robust Multi-Object Tracking},
author={Cao, Jinkun and Weng, Xinshuo and Khirodkar, Rawal and Pang, Jiangmiao and Kitani, Kris},
journal={arXiv preprint arXiv:2203.14360},
year={2022}
}
```
deploy/pipeline/docs/tutorials/ppvehicle_plate.md
浏览文件 @ 8a1efe25
[English](ppvehicle_plate_en.md) | 简体中文
# PP-Vehicle车牌识别模块
......
deploy/pipeline/docs/tutorials/ppvehicle_plate_en.md 0 → 100644
浏览文件 @ 8a1efe25
English | [简体中文](ppvehicle_plate.md)
# PP-Vehicle License Plate Recognition Modules
License plate recognition has a very wide range of applications with vehicle identification functions, such as automatic vehicle entrance/exit gates.
PP-Vehicle supports vehicle tracking and license plate recognition. Models are available for download:
| Task | Algorithm | Accuracy | Inference speed(ms) | Model Download |
|:-------------------------- |:---------------:|:------------:|:-------------------:|:------------------------------------------------------------------------------------------:|
| Vehicle Detection/Tracking | PP-YOLOE-l | mAP: 63.9 | - | [Link](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_l_36e_ppvehicle.zip) |
| Vehicle Detection Model | ch_PP-OCRv3_det | hmean: 0.979 | - | [Link](https://bj.bcebos.com/v1/paddledet/models/pipeline/ch_PP-OCRv3_det_infer.tar.gz) |
| Vehicle Detection Model | ch_PP-OCRv3_rec | acc: 0.773 | - | [Link](https://bj.bcebos.com/v1/paddledet/models/pipeline/ch_PP-OCRv3_rec_infer.tar.gz) |
1. The tracking model uses the PPVehicle dataset ( which integrates BDD100K-MOT and UA-DETRAC). The dataset merged car, truck, bus, van from BDD100K-MOT and car, bus, van from UA-DETRAC all into 1 class vehicle(1).
2. License plate detection and recognition models were obtained from fine-tuned PP-OCRv3 model on the CCPD2019 and CCPD2020 mixed license plate datasets.
## How to Use
1. Download models from the above table and unzip it to```PaddleDetection/output_inference```, and modify the model path in the configuration file. Models can also be downloaded automatically by default: Set enable: True of `VEHICLE_PLATE` in `deploy/pipeline/config/infer_cfg_ppvehicle.yml`
Config Description of `infer_cfg_ppvehicle.yml`：
```
VEHICLE_PLATE: #模块名称
det_model_dir: output_inference/ch_PP-OCRv3_det_infer/ #车牌检测模型路径
det_limit_side_len: 480 #检测模型单边输入尺寸
det_limit_type: "max" #检测模型输入尺寸长短边选择，"max"表示长边
rec_model_dir: output_inference/ch_PP-OCRv3_rec_infer/ #车牌识别模型路径
rec_image_shape: [3, 48, 320] #车牌识别模型输入尺寸
rec_batch_num: 6 #车牌识别batchsize
word_dict_path: deploy/pipeline/ppvehicle/rec_word_dict.txt #OCR模型查询字典
basemode: "idbased" #流程类型，'idbased'表示基于跟踪模型
enable: False #功能是否开启
```
2. For picture input, the start command is as follows (for more descriptions of the command parameters, please refer to [Quick Start - Parameter Description](. /PPVehicle_QUICK_STARTED.md#41-parameter description)).
```python
#Single image
python deploy/pipeline/pipeline.py --config deploy/pipeline/config/infer_cfg_ppvehicle.yml \
--image_file=test_image.jpg \
--device=gpu \
#Image folder
python deploy/pipeline/pipeline.py --config deploy/pipeline/config/infer_cfg_ppvehicle.yml \
--image_dir=images/ \
--device=gpu \
```
3. For video input, the start command is as follows
```bash
#Single video
python deploy/pipeline/pipeline.py --config deploy/pipeline/config/infer_cfg_ppvehicle.yml \
--video_file=test_video.mp4 \
--device=gpu \
#Video folder
python deploy/pipeline/pipeline.py --config deploy/pipeline/config/infer_cfg_ppvehicle.yml \
--video_dir=test_videos/ \
--device=gpu \
```
4. There are two ways to modify the model path
- Config different model path in ```./deploy/pipeline/config/infer_cfg_ppvehicle.yml```, and modify`VEHICLE_PLATE`field to config license plate recognition model modification
- **[Recommand]** Add`-o VEHICLE_PLATE.det_model_dir=[YOUR_DETMODEL_PATH] VEHICLE_PLATE.rec_model_dir=[YOUR_RECMODEL_PATH]` to config file in command line.
The test results are as follows:
<div width="1000" align="center">
<img src="../images/ppvehicleplate.jpg"/>
</div>
## Solutions
1. PP-YOLOE is adopted for vehicle detection frame of object detection, multi-object tracking in the picture/video input. For details, please refer to [PP-YOLOE](... /... /... /configs/ppyoloe/README_cn.md)
2. By using the coordinates of the vehicle detection frame, each vehicle's image is intercepted in the input image
3. Use the license plate detection model to identify the location of the license plate in each vehicle screenshot as well as the license plate area. The PP-OCRv3_det model is adopted as the solution, obtained from fine-tuned CCPD dataset in terms of number plate.
4. Use a character recognition model to identify characters in a number plate. The PP-OCRv3_det model is adopted as the solution, obtained from fine-tuned CCPD dataset in terms of number plate.
**Performance optimization measures：**
1. Use a frame skipping strategy to detect license plates every 10 frames to reduce the computing workload.
2. Use the license plate result stabilization strategy to avoid the volatility of single frame results; use all historical license plate recognition results of the same id to gain the most likely result for that id.
## Reference
1. PaddeDetection featured detection model PP-YOLOE](../../../../configs/ppyoloe)。
2. Paddle OCR Model Library [PaddleOCR](https://github.com/PaddlePaddle/PaddleOCR)。
Markdown is supported
0% 或 .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑！
想要评论请 注册 或