diff --git a/deploy/pipeline/README.md b/deploy/pipeline/README.md index 19534194700f4dcc270e63840302303d784e508b..b4fd9b858b83be0c8eee9f4d323a47d55a9dceeb 100644 --- a/deploy/pipeline/README.md +++ b/deploy/pipeline/README.md @@ -35,10 +35,10 @@ | ⭐ 功能 | 💟 方案优势 | 💡示例图 | | ---------- | ------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------- | -| **车牌识别** | 超强性能：针对车辆密集、车牌大小不一问题进行优化，实现【待补充】 | | -| **车辆属性分析** | 支持车型、颜色类别识别

兼容多种数据格式：支持图片、视频、在线视频流输入

高性能：融合开源数据集与企业真实数据进行训练，实现【待补充】

| | -| **违章检测** | 易用性高：一行命令即可实现违停检测

鲁棒性强：对光照、视角、背景环境无限制 | | -| **车流量计数** | 一键运行：单个参数即可开启车流量计数与轨迹记录功能 | | +| **车牌识别** | 支持传统车牌和新能源绿色车牌

车牌识别采用长间隔采样识别与多次结果统计投票方式，算力消耗少，识别精度高，结果稳定性好。 检测模型 hmean: 0.979; 识别模型 acc: 0.773 | | +| **车辆属性分析** | 支持多种车型、颜色类别识别

使用更强力的Backbone模型PP-HGNet、PP-LCNet,精度高、速度快。识别精度: 90.81 | | +| **违章检测** | 简单易用：一行命令即可实现违停检测，自定义设置区域

检测、跟踪效果好，可实现违停车辆车牌识别 | | +| **车流量计数** | 简单易用：一行命令即可开启功能，自定义出入位置

可提供目标跟踪轨迹显示，统计准确度高 | | ## 🗳 模型库 @@ -53,6 +53,7 @@ | 行人检测（轻量级） | 16.2ms | [多目标跟踪](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_s_36e_pipeline.zip) | 27M | | 行人跟踪（高精度） | 31.8ms | [多目标跟踪](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_l_36e_pipeline.zip) | 182M | | 行人跟踪（轻量级） | 21.0ms | [多目标跟踪](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_s_36e_pipeline.zip) | 27M | +| 跨镜跟踪(REID) | 单人1.5ms | [REID](https://bj.bcebos.com/v1/paddledet/models/pipeline/reid_model.zip) | REID：92M | | 属性识别（高精度） | 单人8.5ms | [目标检测](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_l_36e_pipeline.zip)
[属性识别](https://bj.bcebos.com/v1/paddledet/models/pipeline/strongbaseline_r50_30e_pa100k.zip) | 目标检测：182M
属性识别：86M | | 属性识别（轻量级） | 单人7.1ms | [目标检测](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_l_36e_pipeline.zip)
[属性识别](https://bj.bcebos.com/v1/paddledet/models/pipeline/strongbaseline_r50_30e_pa100k.zip) | 目标检测：182M
属性识别：86M | | 摔倒识别 | 单人10ms | [多目标跟踪](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_l_36e_pipeline.zip)
[关键点检测](https://bj.bcebos.com/v1/paddledet/models/pipeline/dark_hrnet_w32_256x192.zip)
[基于关键点行为识别](https://bj.bcebos.com/v1/paddledet/models/pipeline/STGCN.zip) | 多目标跟踪：182M
关键点检测：101M
基于关键点行为识别：21.8M | diff --git a/deploy/pipeline/README_en.md b/deploy/pipeline/README_en.md index 7008dd265d71c4cd1b44d4116a7e275bbe41cf67..eeb3a7eed752c1129631d9ea481d8d0179942d9a 100644 --- a/deploy/pipeline/README_en.md +++ b/deploy/pipeline/README_en.md @@ -1,27 +1,31 @@ [简体中文](README.md) | English -# Real Time Pedestrian Analysis Tool PP-Human + -**PP-Human is the industry's first open-sourced real-time pedestrian analysis tool based on PaddlePaddle deep learning framework. It has three major features: rich functions, wide application, and efficient deployment.** +**PaddleDetection has provide out-of-the-box tools in pedestrian and vehicle analysis, and it support multiple input format such as images/videos/multi-videos/online video streams. This make it popular in smart-city\smart transportation and so on. It can be deployed easily with GPU server and TensorRT, which achieves real-time performace.** +- 🚶‍♂️🚶‍♀️ **PP-Human has four major toolbox for pedestrian analysis: five example of behavior analysis、26 attributes recognition、in-out counting、multi-target-multi-camera tracking(REID).** +- 🚗🚙 **PP-Vehicle has four major toolbox for vehicle analysis: The license plate recognition、vechile attributes、in-out counting、illegal_parking recognition.** -![](https://user-images.githubusercontent.com/22989727/178965250-14be25c1-125d-4d90-8642-7a9b01fecbe2.gif) - - - -PP-Human supports various inputs such as images, single-camera, and multi-camera videos. It covers multi-object tracking, attributes recognition, behavior analysis, visitor traffic statistics, and trace records. PP-Human can be applied to fields including Smart Transportation, Smart Community, and industrial inspections. It can also be deployed on server sides and TensorRT accelerator. On the T4 server, it could achieve real-time analysis. +![](https://user-images.githubusercontent.com/48054808/184843170-c3ef7d29-913b-4c6e-b533-b83892a8b0e2.gif) ## 📣 Updates -- 🔥 **2022.7.13：PP-Human v2 launched with a full upgrade of four industrial features: behavior analysis, attributes recognition, visitor traffic statistics and ReID. It provides a strong core algorithm for pedestrian detection, tracking and attribute analysis with a simple and detailed release/2.5ment process and model optimization strategy.** +- 🔥🔥🔥 **2022.8.20：PP-Vehicle was first launched with four major toolbox for vehicle analysis，and it also provide detailed documentation for user to train with their own datas and model optimize.** +- 🔥 2022.7.13：PP-Human v2 launched with a full upgrade of four industrial features: behavior analysis, attributes recognition, visitor traffic statistics and ReID. It provides a strong core algorithm for pedestrian detection, tracking and attribute analysis with a simple and detailed development process and model optimization strategy. - 2022.4.18: Add PP-Human practical tutorials, including training, deployment, and action expansion. Details for AIStudio project please see [Link](https://aistudio.baidu.com/aistudio/projectdetail/3842982) - 2022.4.10: Add PP-Human examples; empower refined management of intelligent community management. A quick start for AIStudio [Link](https://aistudio.baidu.com/aistudio/projectdetail/3679564) - 2022.4.5: Launch the real-time pedestrian analysis tool PP-Human. It supports pedestrian tracking, visitor traffic statistics, attributes recognition, and falling detection. Due to its specific optimization of real-scene data, it can accurately recognize various falling gestures, and adapt to different environmental backgrounds, light and camera angles. +![](https://user-images.githubusercontent.com/48054808/184843170-c3ef7d29-913b-4c6e-b533-b83892a8b0e2.gif) + + ## 🔮 Features and demonstration +### PP-Human + | ⭐ Feature | 💟 Advantages | 💡Example | | -------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------- | | **ReID** | Extraordinary performance: special optimization for technical challenges such as target occlusion, uncompleted and blurry objects to achieve mAP 98.8, 1.5ms/person | | @@ -29,6 +33,16 @@ PP-Human supports various inputs such as images, single-camera, and multi-camera | **Behaviour detection** | Rich function: support five high-frequency anomaly behavior detection of falling, fighting, smoking, telephoning, and intrusion

Robust: unlimited by different environmental backgrounds, light, and camera angles.

High performance: Compared with video recognition technology, it takes significantly smaller computation resources; support localization and service-oriented rapid deployment

Fast training: only takes 15 minutes to produce high precision behavior detection models | | | **Visitor traffic statistics**
**Trace record** | Simple and easy to use: single parameter to initiate functions of visitor traffic statistics and trace record | | +### PP-Vehicle + +| ⭐ Feature | 💟 Advantages | 💡 Example | +| ---------- | ------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------- | +| **License Plate-Recognition** | Both support for traditional plate and new green plate

Sample the frame in a time windows to recognice the plate license, and vots the license in many samples, which lead less compute cost and better accuracy, and the result is much more stable.

hmean of text detector: 0.979; accuracy of recognition model: 0.773

| | +| **Vehicle Attributes** | Identify 10 vehicle colors and 9 models

More powerfull backbone: PP-HGNet/PP-LCNet, with higher accuracy and faster speed

accuracy of model: 90.81

| | +| **Illegal Parking** | Easy to use with one line command, and define the illegal area by yourself

Get the license of illegal car

| | +| **in-out counting** | Easy to use with one line command, and define the in-out line by yourself

Target route visualize with high tracking performance | | + + ## 🗳 Model Zoo
@@ -40,6 +54,7 @@ PP-Human supports various inputs such as images, single-camera, and multi-camera | Pedestrian detection (lightweight) | 16.2ms | [Multi-object tracking](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_s_36e_pipeline.zip) | 27M | | Pedestrian tracking (high precision) | 31.8ms | [Multi-object tracking](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_l_36e_pipeline.zip) | 182M | | Pedestrian tracking (lightweight) | 21.0ms | [Multi-object tracking](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_s_36e_pipeline.zip) | 27M | +| MTMCT(REID) | Single Person 1.5ms | [REID](https://bj.bcebos.com/v1/paddledet/models/pipeline/reid_model.zip) | REID：92M | | Attribute recognition (high precision) | Single person8.5ms | [Object detection](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_l_36e_pipeline.zip)
[Attribute recognition](https://bj.bcebos.com/v1/paddledet/models/pipeline/strongbaseline_r50_30e_pa100k.zip) | Object detection：182M
Attribute recognition：86M | | Attribute recognition (lightweight) | Single person 7.1ms | [Object detection](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_l_36e_pipeline.zip)
[Attribute recognition](https://bj.bcebos.com/v1/paddledet/models/pipeline/strongbaseline_r50_30e_pa100k.zip) | Object detection：182M
Attribute recognition：86M | | Falling detection | Single person 10ms | [Multi-object tracking](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_l_36e_pipeline.zip)
[Keypoint detection](https://bj.bcebos.com/v1/paddledet/models/pipeline/dark_hrnet_w32_256x192.zip)
[Behavior detection based on key points](https://bj.bcebos.com/v1/paddledet/models/pipeline/STGCN.zip) | Multi-object tracking：182M
Keypoint detection：101M
Behavior detection based on key points: 21.8M | @@ -69,40 +84,60 @@ Click to download the model, then unzip and save it in the `. /output_inference` ## 📚 Doc Tutorials -### [A Quick Start](docs/tutorials/PPHuman_QUICK_STARTED_en.md) +### 🚶‍♀️ PP-Human + +#### [A Quick Start](docs/tutorials/PPHuman_QUICK_STARTED_en.md) -### Pedestrian attribute/feature recognition +#### Pedestrian attribute/feature recognition * [A quick start](docs/tutorials/pphuman_attribute_en.md) + * [Customized development tutorials](../../docs/advanced_tutorials/customization/pphuman_attribute_en.md) - * Data Preparation - * Model Optimization - * New Attributes -### Behavior detection +#### Behavior detection * [A quick start](docs/tutorials/pphuman_action_en.md) - * Falling detection - * Fighting detection + * [Customized development tutorials](../../docs/advanced_tutorials/customization/action_recognotion/README_en.md) - * Solution Selection - * Data Preparation - * Model Optimization - * New Attributes -### ReID +#### ReID * [A quick start](docs/tutorials/pphuman_mtmct_en.md) + * [Customized development tutorials](../../docs/advanced_tutorials/customization/pphuman_mtmct_en.md) - * Data Preparation - * Model Optimization -### Pedestrian tracking, visitor traffic statistics, trace records +#### Pedestrian tracking, visitor traffic statistics, trace records * [A quick start](docs/tutorials/pphuman_mot_en.md) - * Pedestrian tracking, - * Visitor traffic statistics - * Regional intrusion diagnosis and counting + * [Customized development tutorials](../../docs/advanced_tutorials/customization/pphuman_mot_en.md) - * Data Preparation - * Model Optimization + + +### 🚘 PP-Vehicle + +#### [A Quick Start](docs/tutorials/PPVehicle_QUICK_STARTED.md) + +#### Vehicle Plate License + +- [A quick start](docs/tutorials/ppvehicle_plate_en.md) + +- [Customized development tutorials](../../docs/advanced_tutorials/customization/ppvehicle_plate.md) + +#### Vehicle Attributes + +- [A quick start](docs/tutorials/ppvehicle_attribute_en.md) + +- [Customized development tutorials](../../docs/advanced_tutorials/customization/ppvehicle_attribute_en.md) + +#### Illegal Parking + +- [A quick start](docs/tutorials/ppvehicle_illegal_parking_en.md) + +- [Customized development tutorials](../../docs/advanced_tutorials/customization/pphuman_mot_en.md) + +#### Vehicle Tracking/in-out counint/Route Visualize + +- [A quick start](docs/tutorials/ppvehicle_mot_en.md) + +- [Customized development tutorials](../../docs/advanced_tutorials/customization/pphuman_mot_en.md) + diff --git a/deploy/pipeline/cfg_utils.py b/deploy/pipeline/cfg_utils.py index 37c963299a9cac35cbdcc1010e4404935a8c0f33..1ad6ed999460a5df60cc6e61782747031488c1d4 100644 --- a/deploy/pipeline/cfg_utils.py +++ b/deploy/pipeline/cfg_utils.py @@ -85,6 +85,11 @@ def argsparser(): type=str, default="output", help="Directory of output visualization files.") + parser.add_argument( + "--pushurl", + type=str, + default="", + help="url of output visualization stream.") parser.add_argument( "--run_mode", type=str, diff --git a/deploy/pipeline/docs/tutorials/PPHuman_QUICK_STARTED.md b/deploy/pipeline/docs/tutorials/PPHuman_QUICK_STARTED.md index 80e32176467d24409f1046c6e3fd34f40e261910..5628029869f03ff395befa61f9ce072ff6c40c9c 100644 --- a/deploy/pipeline/docs/tutorials/PPHuman_QUICK_STARTED.md +++ b/deploy/pipeline/docs/tutorials/PPHuman_QUICK_STARTED.md @@ -51,6 +51,7 @@ PP-Human提供了目标检测、属性识别、行为识别、ReID预训练模 | 行人检测（轻量级） | 16.2ms | [多目标跟踪](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_s_36e_pipeline.zip) | 27M | | 行人跟踪（高精度） | 31.8ms | [多目标跟踪](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_l_36e_pipeline.zip) | 182M | | 行人跟踪（轻量级） | 21.0ms | [多目标跟踪](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_s_36e_pipeline.zip) | 27M | +| 跨镜跟踪(REID) | 单人1.5ms | [REID](https://bj.bcebos.com/v1/paddledet/models/pipeline/reid_model.zip) | REID：92M | | 属性识别（高精度） | 单人8.5ms | [目标检测](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_l_36e_pipeline.zip)
[属性识别](https://bj.bcebos.com/v1/paddledet/models/pipeline/PPHGNet_small_person_attribute_954_infer.zip) | 目标检测：182M
属性识别：86M | | 属性识别（轻量级） | 单人7.1ms | [目标检测](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_l_36e_pipeline.zip)
[属性识别](https://bj.bcebos.com/v1/paddledet/models/pipeline/PPLCNet_x1_0_person_attribute_945_infer.zip) | 目标检测：182M
属性识别：86M | | 摔倒识别 | 单人10ms | [多目标跟踪](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_l_36e_pipeline.zip)
[关键点检测](https://bj.bcebos.com/v1/paddledet/models/pipeline/dark_hrnet_w32_256x192.zip)
[基于关键点行为识别](https://bj.bcebos.com/v1/paddledet/models/pipeline/STGCN.zip) | 多目标跟踪：182M
关键点检测：101M
基于关键点行为识别：21.8M | @@ -125,7 +126,10 @@ python deploy/pipeline/pipeline.py --config deploy/pipeline/config/infer_cfg_pph python deploy/pipeline/pipeline.py --config deploy/pipeline/config/infer_cfg_pphuman.yml -o SKELETON_ACTION.enbale=True --video_file=test_video.mp4 --device=gpu ``` -3. 对rtsp流的支持，使用--rtsp RTSP [RTSP ...]参数指定一路或者多路rtsp视频流，如果是多路地址中间用空格隔开。(或者video_file后面的视频地址直接更换为rtsp流地址)，示例如下： +3. rtsp推拉流 +- rtsp拉流预测 + +对rtsp拉流的支持，使用--rtsp RTSP [RTSP ...]参数指定一路或者多路rtsp视频流，如果是多路地址中间用空格隔开。(或者video_file后面的视频地址直接更换为rtsp流地址)，示例如下： ``` # 例：行人属性识别，单路视频流 python deploy/pipeline/pipeline.py --config deploy/pipeline/config/examples/infer_cfg_human_attr.yml -o visual=False --rtsp rtsp://[YOUR_RTSP_SITE] --device=gpu @@ -134,6 +138,18 @@ python deploy/pipeline/pipeline.py --config deploy/pipeline/config/examples/infe python deploy/pipeline/pipeline.py --config deploy/pipeline/config/examples/infer_cfg_human_attr.yml -o visual=False --rtsp rtsp://[YOUR_RTSP_SITE1] rtsp://[YOUR_RTSP_SITE2] --device=gpu ``` +- 视频结果推流rtsp + +预测结果进行rtsp推流，使用--pushurl rtsp:[IP] 推流到IP地址端，PC端可以使用[VLC播放器](https://vlc.onl/)打开网络流进行播放，播放地址为 `rtsp:[IP]/videoname`。其中`videoname`是预测的视频文件名，如果视频来源是本地摄像头则`videoname`默认为`output`. +``` +# 例：行人属性识别，单路视频流，该示例播放地址为 rtsp://[YOUR_SERVER_IP]:8554/test_video +python deploy/pipeline/pipeline.py --config deploy/pipeline/config/examples/infer_cfg_human_attr.yml --video_file=test_video.mp4 --device=gpu --pushurl rtsp://[YOUR_SERVER_IP]:8554 +``` +注： +1. rtsp推流服务基于 [rtsp-simple-server](https://github.com/aler9/rtsp-simple-server), 如使用推流功能请先开启该服务. +2. rtsp推流如果模型处理速度跟不上会出现很明显的卡顿现象，建议跟踪模型使用ppyoloe_s版本，即修改配置中跟踪模型mot_ppyoloe_l_36e_pipeline.zip替换为mot_ppyoloe_s_36e_pipeline.zip。 + + ### Jetson部署说明 由于Jetson平台算力相比服务器有较大差距，有如下使用建议： @@ -158,6 +174,7 @@ python deploy/pipeline/pipeline.py --config deploy/pipeline/config/examples/infe | --rtsp | Option | rtsp视频流地址，支持一路或者多路同时输入 | | --camera_id | Option | 用来预测的摄像头ID，默认为-1(表示不使用摄像头预测，可设置为：0 - (摄像头数目-1) )，预测过程中在可视化界面按`q`退出输出预测结果到：output/output.mp4| | --device | Option | 运行时的设备，可选择`CPU/GPU/XPU`，默认为`CPU`| +| --pushurl | Option| 对预测结果视频进行推流的地址，以rtsp://开头，该选项优先级高于视频结果本地存储，打开时不再另外存储本地预测结果视频| | --output_dir | Option|可视化结果保存的根目录，默认为output/| | --run_mode | Option |使用GPU时，默认为paddle, 可选（paddle/trt_fp32/trt_fp16/trt_int8）| | --enable_mkldnn | Option | CPU预测中是否开启MKLDNN加速，默认为False | diff --git a/deploy/pipeline/docs/tutorials/PPHuman_QUICK_STARTED_en.md b/deploy/pipeline/docs/tutorials/PPHuman_QUICK_STARTED_en.md index 8e0ffa3805c2f6995391c8cadf2234cf489a47bb..8250f2341e09a6540eb1ebba6f1360b734db8c64 100644 --- a/deploy/pipeline/docs/tutorials/PPHuman_QUICK_STARTED_en.md +++ b/deploy/pipeline/docs/tutorials/PPHuman_QUICK_STARTED_en.md @@ -51,6 +51,7 @@ PP-Human provides object detection, attribute recognition, behaviour recognition | Pedestrian Detection (Lightweight) | 16.2ms | [Multi-Object Tracking](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_s_36e_pipeline.zip) | 27M | | Pedestrian Tracking (high precision) | 31.8ms | [Multi-Object Tracking](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_l_36e_pipeline.zip) | 182M | | Pedestrian Tracking (Lightweight) | 21.0ms | [Multi-Object Tracking](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_s_36e_pipeline.zip) | 27M | +| MTMCT(REID) | Single Person 1.5ms | [REID](https://bj.bcebos.com/v1/paddledet/models/pipeline/reid_model.zip) | REID：92M | | Attribute Recognition (high precision) | Single Person 8.5ms | [Object Detection](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_l_36e_pipeline.zip)
[Attribute Recognition](https://bj.bcebos.com/v1/paddledet/models/pipeline/PPHGNet_small_person_attribute_954_infer.zip) | Object Detection：182M
Attribute Recogniton：86M | | Attribute Recognition (Lightweight) | Single Person 7.1ms | [Object Detection](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_l_36e_pipeline.zip)
[Attribute Recogniton](https://bj.bcebos.com/v1/paddledet/models/pipeline/PPLCNet_x1_0_person_attribute_945_infer.zip) | Object Detection：182M
Attribute Recogniton：86M | | Falling Detection | Single Person 10ms | [Multi-Object Tracking](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_l_36e_pipeline.zip)
[Keypoint Detection](https://bj.bcebos.com/v1/paddledet/models/pipeline/dark_hrnet_w32_256x192.zip)
[Skeleton Based Action Recognition](https://bj.bcebos.com/v1/paddledet/models/pipeline/STGCN.zip) | Multi-Object Tracking：182M
Keypoint Detection：101M
Skeleton Based Action Recognition：21.8M | @@ -125,7 +126,10 @@ python deploy/pipeline/pipeline.py --config deploy/pipeline/config/infer_cfg_pph python deploy/pipeline/pipeline.py --config deploy/pipeline/config/infer_cfg_pphuman.yml -o SKELETON_ACTION.enbale=True --video_file=test_video.mp4 --device=gpu ``` -3. For rtsp stream, use --rtsp RTSP [RTSP ...] parameter to specify one or more rtsp streams. Separate the multiple addresses with a space, or replace the video address directly after the video_file with the rtsp stream address), examples as follows +3. rtsp push/pull stream +- rtsp pull stream + +For rtsp pull stream, use `--rtsp RTSP [RTSP ...]` parameter to specify one or more rtsp streams. Separate the multiple addresses with a space, or replace the video address directly after the video_file with the rtsp stream address), examples as follows ``` # Example: Single video stream for pedestrian attribute recognition @@ -134,6 +138,19 @@ python deploy/pipeline/pipeline.py --config deploy/pipeline/config/examples/infe python deploy/pipeline/pipeline.py --config deploy/pipeline/config/examples/infer_cfg_human_attr.yml -o visual=False --rtsp rtsp://[YOUR_RTSP_SITE1] rtsp://[YOUR_RTSP_SITE2] --device=gpu | ``` +- rtsp push stream + +For rtsp push stream, use `--pushurl rtsp:[IP]` parameter to push stream to a IP set, and you can visualize the output video by [VLC Player](https://vlc.onl/) with the `open network` funciton. the whole url path is `rtsp:[IP]/videoname`, the videoname here is the basename of the video file to infer, and the default of videoname is `output` when the video come from local camera and there is no video name. + +``` +# Example：Pedestrian attribute recognition，in this example the whole url path is: rtsp://[YOUR_SERVER_IP]:8554/test_video +python deploy/pipeline/pipeline.py --config deploy/pipeline/config/examples/infer_cfg_human_attr.yml --video_file=test_video.mp4 --device=gpu --pushurl rtsp://[YOUR_SERVER_IP]:8554 +``` +Note: +1. rtsp push stream is based on [rtsp-simple-server](https://github.com/aler9/rtsp-simple-server), please enable this serving first. +2. the output visualize will be frozen frequently if the model cost too much time, we suggest to use faster model like ppyoloe_s in tracking, this is simply replace mot_ppyoloe_l_36e_pipeline.zip with mot_ppyoloe_s_36e_pipeline.zip in model config yaml file. + + ### Jetson Deployment Due to the large gap in computing power of the Jetson platform compared to the server, we suggest: @@ -155,6 +172,8 @@ With this recommended configuration, it is possible to achieve higher speeds on | --rtsp | Option | rtsp video stream address, supports one or more simultaneous streams input | | --camera_id | Option | The camera ID for prediction, default is -1 ( for no camera prediction, can be set to 0 - (number of cameras - 1) ), press `q` in the visualization interface during the prediction process to output the prediction result to: output/output.mp4 | | --device | Option | Running device, options include `CPU/GPU/XPU`, and the default is `CPU`. | +| --pushurl | Option | push the output video to rtsp stream, normaly start with `rtsp://`; this has higher priority than local video save, while this is set, pipeline will not save local visualize video, the default is "", means this will not work now. + | | --output_dir | Option | The root directory for the visualization results, and the default is output/ | | --run_mode | Option | For GPU, the default is paddle, with (paddle/trt_fp32/trt_fp16/trt_int8) as optional | | --enable_mkldnn | Option | Whether to enable MKLDNN acceleration in CPU prediction, the default is False | diff --git a/deploy/pipeline/docs/tutorials/PPVehicle_QUICK_STARTED.md b/deploy/pipeline/docs/tutorials/PPVehicle_QUICK_STARTED.md index 14a43510a213b154039b35b240585ce4c0bf61bb..8b140d4729d902052d751d525417981157a48f38 100644 --- a/deploy/pipeline/docs/tutorials/PPVehicle_QUICK_STARTED.md +++ b/deploy/pipeline/docs/tutorials/PPVehicle_QUICK_STARTED.md @@ -19,7 +19,7 @@ ## 环境准备 -环境要求： PaddleDetection版本 >= release/2.4 或 release/2.5版本 +环境要求： PaddleDetection版本 >= release/2.5 或 develop版本 PaddlePaddle和PaddleDetection安装 @@ -75,7 +75,7 @@ PP-Vehicle相关配置位于```deploy/pipeline/config/infer_cfg_ppvehicle.yml``` |-------|-------|----------|-----| | 图片 | 属性识别 | 目标检测 属性识别 | DET ATTR | | 单镜头视频 | 属性识别 | 多目标跟踪 属性识别 | MOT ATTR | -| 单镜头视频 | 属性识别 | 多目标跟踪 属性识别 | MOT VEHICLEPLATE | +| 单镜头视频 | 车牌识别 | 多目标跟踪 车牌识别 | MOT VEHICLEPLATE | 例如基于视频输入的属性识别，任务类型包含多目标跟踪和属性识别，具体配置如下： @@ -131,15 +131,29 @@ python deploy/pipeline/pipeline.py --config deploy/pipeline/config/examples/infe ``` -3. 对rtsp流的支持，使用--rtsp RTSP [RTSP ...]参数指定一路或者多路rtsp视频流，如果是多路地址中间用空格隔开。(或者video_file后面的视频地址直接更换为rtsp流地址)，示例如下： +3. rtsp推拉流 +- rtsp拉流预测 + +对rtsp拉流的支持，使用--rtsp RTSP [RTSP ...]参数指定一路或者多路rtsp视频流，如果是多路地址中间用空格隔开。(或者video_file后面的视频地址直接更换为rtsp流地址)，示例如下： ``` -# 例：行人属性识别，单路视频流 +# 例：车辆属性识别，单路视频流 python deploy/pipeline/pipeline.py --config deploy/pipeline/config/examples/infer_cfg_vehicle_attr.yml -o visual=False --rtsp rtsp://[YOUR_RTSP_SITE] --device=gpu -# 例：行人属性识别，多路视频流 +# 例：车辆属性识别，多路视频流 python deploy/pipeline/pipeline.py --config deploy/pipeline/config/examples/infer_cfg_vehicle_attr.yml -o visual=False --rtsp rtsp://[YOUR_RTSP_SITE1] rtsp://[YOUR_RTSP_SITE2] --device=gpu ``` +- 视频结果推流rtsp + +预测结果进行rtsp推流，使用--pushurl rtsp:[IP] 推流到IP地址端，PC端可以使用[VLC播放器](https://vlc.onl/)打开网络流进行播放，播放地址为 `rtsp:[IP]/videoname`。其中`videoname`是预测的视频文件名，如果视频来源是本地摄像头则`videoname`默认为`output`. +``` +# 例：车辆属性识别，单路视频流，该示例播放地址为 rtsp://[YOUR_SERVER_IP]:8554/test_video +python deploy/pipeline/pipeline.py --config deploy/pipeline/config/examples/infer_cfg_vehicle_attr.yml -o visual=False --video_file=test_video.mp4 --device=gpu --pushurl rtsp://[YOUR_SERVER_IP]:8554 +``` +注： +1. rtsp推流服务基于 [rtsp-simple-server](https://github.com/aler9/rtsp-simple-server), 如使用推流功能请先开启该服务. +2. rtsp推流如果模型处理速度跟不上会出现很明显的卡顿现象，建议跟踪模型使用ppyoloe_s版本，即修改配置中跟踪模型mot_ppyoloe_l_36e_pipeline.zip替换为mot_ppyoloe_s_36e_pipeline.zip。 + ### Jetson部署说明 由于Jetson平台算力相比服务器有较大差距，有如下使用建议： @@ -164,6 +178,7 @@ python deploy/pipeline/pipeline.py --config deploy/pipeline/config/examples/infe | --rtsp | Option | rtsp视频流地址，支持一路或者多路同时输入 | | --camera_id | Option | 用来预测的摄像头ID，默认为-1(表示不使用摄像头预测，可设置为：0 - (摄像头数目-1) )，预测过程中在可视化界面按`q`退出输出预测结果到：output/output.mp4| | --device | Option | 运行时的设备，可选择`CPU/GPU/XPU`，默认为`CPU`| +| --pushurl | Option| 对预测结果视频进行推流的地址，以rtsp://开头，该选项优先级高于视频结果本地存储，打开时不再另外存储本地预测结果视频, 默认为空，表示没有开启| | --output_dir | Option|可视化结果保存的根目录，默认为output/| | --run_mode | Option |使用GPU时，默认为paddle, 可选（paddle/trt_fp32/trt_fp16/trt_int8）| | --enable_mkldnn | Option | CPU预测中是否开启MKLDNN加速，默认为False | diff --git a/deploy/pipeline/docs/tutorials/PPVehicle_QUICK_STARTED_en.md b/deploy/pipeline/docs/tutorials/PPVehicle_QUICK_STARTED_en.md index dc42b5056ffcda9e0a69dcfcc8c1bb54498e5c1e..ec720212b309dd2653c54ded7c98d6936d0cb95a 100644 --- a/deploy/pipeline/docs/tutorials/PPVehicle_QUICK_STARTED_en.md +++ b/deploy/pipeline/docs/tutorials/PPVehicle_QUICK_STARTED_en.md @@ -18,7 +18,7 @@ English | [简体中文](PPVehicle_QUICK_STARTED.md) ## Environment Preparation -Environment Preparation: PaddleDetection version >= release/2.4 or release/2.5 +Environment Preparation: PaddleDetection version >= release/2.5 or develop Installation of PaddlePaddle and PaddleDetection @@ -73,7 +73,7 @@ The features and corresponding task types are as follows. | ------------------- | --------------------- | ------------------------------------------- | ---------------- | | Image | Attribute Recognition | Object Detection Attribute Recognition | DET ATTR | | Single-camera video | Attribute Recognition | Multi-Object Tracking Attribute Recognition | MOT ATTR | -| Single-camera video | Attribute Recognition | Multi-Object Tracking Attribute Recognition | MOT VEHICLEPLATE | +| Single-camera video | License-plate Recognition | Multi-Object Tracking License-plate Recognition | MOT VEHICLEPLATE | Take attribute recognition based on video input as an example: Its task type includes multi-object tracking and attributes recognition. The specific configuration is as follows. @@ -129,7 +129,10 @@ python deploy/pipeline/pipeline.py --config deploy/pipeline/config/examples/infe --region_polygon 600 300 1300 300 1300 800 600 800 ``` -3. For rtsp stream, use --rtsp RTSP [RTSP ...] parameter to specify one or more rtsp streams. Separate the multiple addresses with a space, or replace the video address directly after the video_file with the rtsp stream address), examples as follows +3. rtsp push/pull stream +- rtsp pull stream + +For rtsp pull stream, use --rtsp RTSP [RTSP ...] parameter to specify one or more rtsp streams. Separate the multiple addresses with a space, or replace the video address directly after the video_file with the rtsp stream address), examples as follows ``` # Example: Single video stream for pedestrian attribute recognition @@ -138,6 +141,18 @@ python deploy/pipeline/pipeline.py --config deploy/pipeline/config/examples/infe python deploy/pipeline/pipeline.py --config deploy/pipeline/config/examples/infer_cfg_human_attr.yml -o visual=False --rtsp rtsp://[YOUR_RTSP_SITE1] rtsp://[YOUR_RTSP_SITE2] --device=gpu | ``` +- rtsp push stream + +For rtsp push stream, use --pushurl rtsp:[IP] parameter to push stream to a IP set, and you can visualize the output video by [VLC Player](https://vlc.onl/) with the `open network` funciton. the whole url path is `rtsp:[IP]/videoname`, the videoname here is the basename of the video file to infer, and the default of videoname is `output` when the video come from local camera and there is no video name. + +``` +# Example：license plate recognition，in this example the whole url path is: rtsp://[YOUR_SERVER_IP]:8554/test_video +python deploy/pipeline/pipeline.py --config deploy/pipeline/config/examples/infer_cfg_vehicle_plate.yml --video_file=test_video.mp4 --device=gpu --pushurl rtsp://[YOUR_SERVER_IP]:8554 +``` +Note: +1. rtsp push stream is based on [rtsp-simple-server](https://github.com/aler9/rtsp-simple-server), please enable this serving first. +2. the output visualize will be frozen frequently if the model cost too much time, we suggest to use faster model like ppyoloe_s in tracking, this is simply replace mot_ppyoloe_l_36e_pipeline.zip with mot_ppyoloe_s_36e_pipeline.zip in model config yaml file. + ### Jetson Deployment Due to the large gap in computing power of the Jetson platform compared to the server, we suggest: @@ -161,6 +176,8 @@ With this recommended configuration, it is possible to achieve higher speeds on | --rtsp | Option | rtsp video stream address, supports one or more simultaneous streams input | | --camera_id | Option | The camera ID for prediction, default is -1 ( for no camera prediction, can be set to 0 - (number of cameras - 1) ), press `q` in the visualization interface during the prediction process to output the prediction result to: output/output.mp4 | | --device | Option | Running device, options include `CPU/GPU/XPU`, and the default is `CPU`. | +| --pushurl | Option | push the output video to rtsp stream, normaly start with `rtsp://`; this has higher priority than local video save, while this is set, pipeline will not save local visualize video, the default is "", means this will not work now. + | | --output_dir | Option | The root directory for the visualization results, and the default is output/ | | --run_mode | Option | For GPU, the default is paddle, with (paddle/trt_fp32/trt_fp16/trt_int8) as optional | | --enable_mkldnn | Option | Whether to enable MKLDNN acceleration in CPU prediction, the default is False | diff --git a/deploy/pipeline/pipe_utils.py b/deploy/pipeline/pipe_utils.py index e17f8aeab784a673d09fda91bf8b3ffefab5c7ec..996a76a64e02b9e3623a304ac8c53947f365c08d 100644 --- a/deploy/pipeline/pipe_utils.py +++ b/deploy/pipeline/pipe_utils.py @@ -19,6 +19,7 @@ import glob import yaml import copy import numpy as np +import subprocess as sp from python.keypoint_preprocess import EvalAffine, TopDownEvalAffine, expand_crop @@ -122,6 +123,26 @@ class PipeTimer(Times): dic['img_num'] = self.img_num return dic +class PushStream(object): + def __init__(self, pushurl = "rtsp://127.0.0.1:8554/"): + self.command = "" + # 自行设置 + self.pushurl = pushurl + + def initcmd(self, fps, width, height): + self.command = ['ffmpeg', + '-y', + '-f', 'rawvideo', + '-vcodec','rawvideo', + '-pix_fmt', 'bgr24', + '-s', "{}x{}".format(width, height), + '-r', str(fps), + '-i', '-', + '-pix_fmt', 'yuv420p', + '-f', 'rtsp', + self.pushurl] + self.pipe = sp.Popen(self.command, stdin=sp.PIPE) + def get_test_images(infer_dir, infer_img): """ diff --git a/deploy/pipeline/pipeline.py b/deploy/pipeline/pipeline.py index 8865d5d754c269d0e263c3f88f33f8d07c05a203..e13afeb0a28aee77ca2ad190b17dee19dea0d478 100644 --- a/deploy/pipeline/pipeline.py +++ b/deploy/pipeline/pipeline.py @@ -31,6 +31,7 @@ sys.path.insert(0, parent_path) from cfg_utils import argsparser, print_arguments, merge_cfg from pipe_utils import PipeTimer from pipe_utils import get_test_images, crop_image_with_det, crop_image_with_mot, parse_mot_res, parse_mot_keypoint +from pipe_utils import PushStream from python.infer import Detector, DetectorPicoDet from python.keypoint_infer import KeyPointDetector @@ -340,6 +341,8 @@ class PipePredictor(object): self.file_name = None self.collector = DataCollector() + self.pushurl = args.pushurl + # auto download inference model get_model_dir(self.cfg) @@ -471,6 +474,8 @@ class PipePredictor(object): def set_file_name(self, path): if path is not None: self.file_name = os.path.split(path)[-1] + if "." in self.file_name: + self.file_name = self.file_name.split(".")[-2] else: # use camera id self.file_name = None @@ -571,10 +576,6 @@ class PipePredictor(object): # mot -> attr # mot -> pose -> action capture = cv2.VideoCapture(video_file) - video_out_name = 'output.mp4' if self.file_name is None else self.file_name - if "rtsp" in video_file: - video_out_name = video_out_name + "_t" + str(thread_idx).zfill( - 2) + "_rtsp.mp4" # Get Video info : resolution, fps, frame count width = int(capture.get(cv2.CAP_PROP_FRAME_WIDTH)) @@ -583,11 +584,23 @@ class PipePredictor(object): frame_count = int(capture.get(cv2.CAP_PROP_FRAME_COUNT)) print("video fps: %d, frame_count: %d" % (fps, frame_count)) - if not os.path.exists(self.output_dir): - os.makedirs(self.output_dir) - out_path = os.path.join(self.output_dir, video_out_name) - fourcc = cv2.VideoWriter_fourcc(* 'mp4v') - writer = cv2.VideoWriter(out_path, fourcc, fps, (width, height)) + if len(self.pushurl) > 0: + video_out_name = 'output' if self.file_name is None else self.file_name + pushurl = os.path.join(self.pushurl, video_out_name) + print("the result will push stream to url:{}".format(pushurl)) + pushstream = PushStream(pushurl) + pushstream.initcmd(fps, width, height) + elif self.cfg['visual']: + video_out_name = 'output' if self.file_name is None else self.file_name + if "rtsp" in video_file: + video_out_name = video_out_name + "_t" + str(thread_idx).zfill( + 2) + "_rtsp" + if not os.path.exists(self.output_dir): + os.makedirs(self.output_dir) + out_path = os.path.join(self.output_dir, video_out_name+".mp4") + fourcc = cv2.VideoWriter_fourcc(* 'mp4v') + writer = cv2.VideoWriter(out_path, fourcc, fps, (width, height)) + frame_id = 0 entrance, records, center_traj = None, None, None @@ -709,11 +722,14 @@ class PipePredictor(object): im = self.visualize_video(frame, mot_res, frame_id, fps, entrance, records, center_traj) # visualize - writer.write(im) - if self.file_name is None: # use camera_id - cv2.imshow('Paddle-Pipeline', im) - if cv2.waitKey(1) & 0xFF == ord('q'): - break + if len(self.pushurl)>0: + pushstream.pipe.stdin.write(im.tobytes()) + else: + writer.write(im) + if self.file_name is None: # use camera_id + cv2.imshow('Paddle-Pipeline', im) + if cv2.waitKey(1) & 0xFF == ord('q'): + break continue self.pipeline_res.update(mot_res, 'mot') @@ -882,14 +898,17 @@ class PipePredictor(object): entrance, records, center_traj, self.illegal_parking_time != -1, illegal_parking_dict) # visualize - writer.write(im) - if self.file_name is None: # use camera_id - cv2.imshow('Paddle-Pipeline', im) - if cv2.waitKey(1) & 0xFF == ord('q'): - break - - writer.release() - print('save result to {}'.format(out_path)) + if len(self.pushurl)>0: + pushstream.pipe.stdin.write(im.tobytes()) + else: + writer.write(im) + if self.file_name is None: # use camera_id + cv2.imshow('Paddle-Pipeline', im) + if cv2.waitKey(1) & 0xFF == ord('q'): + break + if self.cfg['visual'] and len(self.pushurl)==0: + writer.release() + print('save result to {}'.format(out_path)) def visualize_video(self, image, diff --git a/deploy/pipeline/pphuman/mtmct.py b/deploy/pipeline/pphuman/mtmct.py index 5792e15d239c4bac624336e713a27b399eb32b59..9c434903c6ff237cb8965c19df52e1eef5776965 100644 --- a/deploy/pipeline/pphuman/mtmct.py +++ b/deploy/pipeline/pphuman/mtmct.py @@ -347,7 +347,7 @@ def res2dict(multi_res): for tid, res in c_res.items(): key = "c" + str(cid) + "_t" + str(tid) if key not in cid_tid_dict: - if len(res["rects"]) < 10: + if len(res["features"])==0: continue cid_tid_dict[key] = res cid_tid_dict[key]['mean_feat'] = distill_idfeat(res)