未验证 提交 8b032fbe 编写于 作者: Z zhiboniu 提交者: GitHub

cherry-pick of recently work (#7078)

* update empty features check (#6985)

* rtsp push stream in pipeline (#7000)

* add push stream

* update push stream docs

* update class name

* update PP-Vehicle docs and en docs (#7042)

* update PP-Vehicle docs and en docs

* add reid model in v2.5 modelzoo

* update more
上级 ee563f01
...@@ -35,10 +35,10 @@ ...@@ -35,10 +35,10 @@
| ⭐ 功能 | 💟 方案优势 | 💡示例图 | | ⭐ 功能 | 💟 方案优势 | 💡示例图 |
| ---------- | ------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------- | | ---------- | ------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------- |
| **车牌识别** | 超强性能:针对车辆密集、车牌大小不一问题进行优化,实现【待补充】 | <img title="" src="https://user-images.githubusercontent.com/48054808/185027987-6144cafd-0286-4c32-8425-7ab9515d1ec3.png" alt="" width="191"> | | **车牌识别** | 支持传统车牌和新能源绿色车牌 <br/><br/> 车牌识别采用长间隔采样识别与多次结果统计投票方式,算力消耗少,识别精度高,结果稳定性好。 检测模型 hmean: 0.979; 识别模型 acc: 0.773 | <img title="" src="https://user-images.githubusercontent.com/48054808/185027987-6144cafd-0286-4c32-8425-7ab9515d1ec3.png" alt="" width="191"> |
| **车辆属性分析** | 支持车型、颜色类别识别<br/><br/>兼容多种数据格式:支持图片、视频、在线视频流输入<br/><br/>高性能:融合开源数据集与企业真实数据进行训练,实现【待补充】<br/><br/> | <img title="" src="https://user-images.githubusercontent.com/48054808/185044490-00edd930-1885-4e79-b3d4-3a39a77dea93.gif" alt="" width="207"> | | **车辆属性分析** | 支持多种车型、颜色类别识别 <br/><br/> 使用更强力的Backbone模型PP-HGNet、PP-LCNet,精度高、速度快。识别精度: 90.81 | <img title="" src="https://user-images.githubusercontent.com/48054808/185044490-00edd930-1885-4e79-b3d4-3a39a77dea93.gif" alt="" width="207"> |
| **违章检测** | 易用性高:一行命令即可实现违停检测<br/><br/>鲁棒性强:对光照、视角、背景环境无限制 | <img title="" src="https://user-images.githubusercontent.com/48054808/185028419-58ae0af8-a035-42e7-9583-25f5e4ce0169.png" alt="" width="209"> | | **违章检测** | 简单易用:一行命令即可实现违停检测,自定义设置区域 <br/><br/> 检测、跟踪效果好,可实现违停车辆车牌识别 | <img title="" src="https://user-images.githubusercontent.com/48054808/185028419-58ae0af8-a035-42e7-9583-25f5e4ce0169.png" alt="" width="209"> |
| **车流量计数** | 一键运行:单个参数即可开启车流量计数与轨迹记录功能 | <img title="" src="https://user-images.githubusercontent.com/48054808/185028798-9e07379f-7486-4266-9d27-3aec943593e0.gif" alt="" width="200"> | | **车流量计数** | 简单易用:一行命令即可开启功能,自定义出入位置 <br/><br/> 可提供目标跟踪轨迹显示,统计准确度高 | <img title="" src="https://user-images.githubusercontent.com/48054808/185028798-9e07379f-7486-4266-9d27-3aec943593e0.gif" alt="" width="200"> |
## 🗳 模型库 ## 🗳 模型库
...@@ -53,6 +53,7 @@ ...@@ -53,6 +53,7 @@
| 行人检测(轻量级) | 16.2ms | [多目标跟踪](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_s_36e_pipeline.zip) | 27M | | 行人检测(轻量级) | 16.2ms | [多目标跟踪](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_s_36e_pipeline.zip) | 27M |
| 行人跟踪(高精度) | 31.8ms | [多目标跟踪](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_l_36e_pipeline.zip) | 182M | | 行人跟踪(高精度) | 31.8ms | [多目标跟踪](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_l_36e_pipeline.zip) | 182M |
| 行人跟踪(轻量级) | 21.0ms | [多目标跟踪](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_s_36e_pipeline.zip) | 27M | | 行人跟踪(轻量级) | 21.0ms | [多目标跟踪](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_s_36e_pipeline.zip) | 27M |
| 跨镜跟踪(REID) | 单人1.5ms | [REID](https://bj.bcebos.com/v1/paddledet/models/pipeline/reid_model.zip) | REID:92M |
| 属性识别(高精度) | 单人8.5ms | [目标检测](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_l_36e_pipeline.zip)<br> [属性识别](https://bj.bcebos.com/v1/paddledet/models/pipeline/strongbaseline_r50_30e_pa100k.zip) | 目标检测:182M<br>属性识别:86M | | 属性识别(高精度) | 单人8.5ms | [目标检测](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_l_36e_pipeline.zip)<br> [属性识别](https://bj.bcebos.com/v1/paddledet/models/pipeline/strongbaseline_r50_30e_pa100k.zip) | 目标检测:182M<br>属性识别:86M |
| 属性识别(轻量级) | 单人7.1ms | [目标检测](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_l_36e_pipeline.zip)<br> [属性识别](https://bj.bcebos.com/v1/paddledet/models/pipeline/strongbaseline_r50_30e_pa100k.zip) | 目标检测:182M<br>属性识别:86M | | 属性识别(轻量级) | 单人7.1ms | [目标检测](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_l_36e_pipeline.zip)<br> [属性识别](https://bj.bcebos.com/v1/paddledet/models/pipeline/strongbaseline_r50_30e_pa100k.zip) | 目标检测:182M<br>属性识别:86M |
| 摔倒识别 | 单人10ms | [多目标跟踪](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_l_36e_pipeline.zip) <br> [关键点检测](https://bj.bcebos.com/v1/paddledet/models/pipeline/dark_hrnet_w32_256x192.zip) <br> [基于关键点行为识别](https://bj.bcebos.com/v1/paddledet/models/pipeline/STGCN.zip) | 多目标跟踪:182M<br>关键点检测:101M<br>基于关键点行为识别:21.8M | | 摔倒识别 | 单人10ms | [多目标跟踪](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_l_36e_pipeline.zip) <br> [关键点检测](https://bj.bcebos.com/v1/paddledet/models/pipeline/dark_hrnet_w32_256x192.zip) <br> [基于关键点行为识别](https://bj.bcebos.com/v1/paddledet/models/pipeline/STGCN.zip) | 多目标跟踪:182M<br>关键点检测:101M<br>基于关键点行为识别:21.8M |
......
[简体中文](README.md) | English [简体中文](README.md) | English
# Real Time Pedestrian Analysis Tool PP-Human <img src="https://user-images.githubusercontent.com/48054808/185032511-0c97b21c-8bab-4ab1-89ee-16e5e81c22cc.png" title="" alt="" data-align="center">
**PP-Human is the industry's first open-sourced real-time pedestrian analysis tool based on PaddlePaddle deep learning framework. It has three major features: rich functions, wide application, and efficient deployment.** **PaddleDetection has provide out-of-the-box tools in pedestrian and vehicle analysis, and it support multiple input format such as images/videos/multi-videos/online video streams. This make it popular in smart-city\smart transportation and so on. It can be deployed easily with GPU server and TensorRT, which achieves real-time performace.**
- 🚶‍♂️🚶‍♀️ **PP-Human has four major toolbox for pedestrian analysis: five example of behavior analysis、26 attributes recognition、in-out counting、multi-target-multi-camera tracking(REID).**
- 🚗🚙 **PP-Vehicle has four major toolbox for vehicle analysis: The license plate recognition、vechile attributes、in-out counting、illegal_parking recognition.**
![](https://user-images.githubusercontent.com/22989727/178965250-14be25c1-125d-4d90-8642-7a9b01fecbe2.gif) ![](https://user-images.githubusercontent.com/48054808/184843170-c3ef7d29-913b-4c6e-b533-b83892a8b0e2.gif)
PP-Human supports various inputs such as images, single-camera, and multi-camera videos. It covers multi-object tracking, attributes recognition, behavior analysis, visitor traffic statistics, and trace records. PP-Human can be applied to fields including Smart Transportation, Smart Community, and industrial inspections. It can also be deployed on server sides and TensorRT accelerator. On the T4 server, it could achieve real-time analysis.
## 📣 Updates ## 📣 Updates
- 🔥 **2022.7.13:PP-Human v2 launched with a full upgrade of four industrial features: behavior analysis, attributes recognition, visitor traffic statistics and ReID. It provides a strong core algorithm for pedestrian detection, tracking and attribute analysis with a simple and detailed release/2.5ment process and model optimization strategy.** - 🔥🔥🔥 **2022.8.20:PP-Vehicle was first launched with four major toolbox for vehicle analysis,and it also provide detailed documentation for user to train with their own datas and model optimize.**
- 🔥 2022.7.13:PP-Human v2 launched with a full upgrade of four industrial features: behavior analysis, attributes recognition, visitor traffic statistics and ReID. It provides a strong core algorithm for pedestrian detection, tracking and attribute analysis with a simple and detailed development process and model optimization strategy.
- 2022.4.18: Add PP-Human practical tutorials, including training, deployment, and action expansion. Details for AIStudio project please see [Link](https://aistudio.baidu.com/aistudio/projectdetail/3842982) - 2022.4.18: Add PP-Human practical tutorials, including training, deployment, and action expansion. Details for AIStudio project please see [Link](https://aistudio.baidu.com/aistudio/projectdetail/3842982)
- 2022.4.10: Add PP-Human examples; empower refined management of intelligent community management. A quick start for AIStudio [Link](https://aistudio.baidu.com/aistudio/projectdetail/3679564) - 2022.4.10: Add PP-Human examples; empower refined management of intelligent community management. A quick start for AIStudio [Link](https://aistudio.baidu.com/aistudio/projectdetail/3679564)
- 2022.4.5: Launch the real-time pedestrian analysis tool PP-Human. It supports pedestrian tracking, visitor traffic statistics, attributes recognition, and falling detection. Due to its specific optimization of real-scene data, it can accurately recognize various falling gestures, and adapt to different environmental backgrounds, light and camera angles. - 2022.4.5: Launch the real-time pedestrian analysis tool PP-Human. It supports pedestrian tracking, visitor traffic statistics, attributes recognition, and falling detection. Due to its specific optimization of real-scene data, it can accurately recognize various falling gestures, and adapt to different environmental backgrounds, light and camera angles.
![](https://user-images.githubusercontent.com/48054808/184843170-c3ef7d29-913b-4c6e-b533-b83892a8b0e2.gif)
## 🔮 Features and demonstration ## 🔮 Features and demonstration
### PP-Human
| ⭐ Feature | 💟 Advantages | 💡Example | | ⭐ Feature | 💟 Advantages | 💡Example |
| -------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------- | | -------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------- |
| **ReID** | Extraordinary performance: special optimization for technical challenges such as target occlusion, uncompleted and blurry objects to achieve mAP 98.8, 1.5ms/person | <img src="https://user-images.githubusercontent.com/48054808/173037607-0a5deadc-076e-4dcc-bd96-d54eea205f1f.png" title="" alt="" width="191"> | | **ReID** | Extraordinary performance: special optimization for technical challenges such as target occlusion, uncompleted and blurry objects to achieve mAP 98.8, 1.5ms/person | <img src="https://user-images.githubusercontent.com/48054808/173037607-0a5deadc-076e-4dcc-bd96-d54eea205f1f.png" title="" alt="" width="191"> |
...@@ -29,6 +33,16 @@ PP-Human supports various inputs such as images, single-camera, and multi-camera ...@@ -29,6 +33,16 @@ PP-Human supports various inputs such as images, single-camera, and multi-camera
| **Behaviour detection** | Rich function: support five high-frequency anomaly behavior detection of falling, fighting, smoking, telephoning, and intrusion<br/><br/>Robust: unlimited by different environmental backgrounds, light, and camera angles.<br/><br/>High performance: Compared with video recognition technology, it takes significantly smaller computation resources; support localization and service-oriented rapid deployment<br/><br/>Fast training: only takes 15 minutes to produce high precision behavior detection models | <img src="https://user-images.githubusercontent.com/48054808/173034825-623e4f78-22a5-4f14-9b83-dc47aa868478.gif" title="" alt="" width="209"> | | **Behaviour detection** | Rich function: support five high-frequency anomaly behavior detection of falling, fighting, smoking, telephoning, and intrusion<br/><br/>Robust: unlimited by different environmental backgrounds, light, and camera angles.<br/><br/>High performance: Compared with video recognition technology, it takes significantly smaller computation resources; support localization and service-oriented rapid deployment<br/><br/>Fast training: only takes 15 minutes to produce high precision behavior detection models | <img src="https://user-images.githubusercontent.com/48054808/173034825-623e4f78-22a5-4f14-9b83-dc47aa868478.gif" title="" alt="" width="209"> |
| **Visitor traffic statistics**<br>**Trace record** | Simple and easy to use: single parameter to initiate functions of visitor traffic statistics and trace record | <img src="https://user-images.githubusercontent.com/22989727/174736440-87cd5169-c939-48f8-90a1-0495a1fcb2b1.gif" title="" alt="" width="200"> | | **Visitor traffic statistics**<br>**Trace record** | Simple and easy to use: single parameter to initiate functions of visitor traffic statistics and trace record | <img src="https://user-images.githubusercontent.com/22989727/174736440-87cd5169-c939-48f8-90a1-0495a1fcb2b1.gif" title="" alt="" width="200"> |
### PP-Vehicle
| ⭐ Feature | 💟 Advantages | 💡 Example |
| ---------- | ------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------- |
| **License Plate-Recognition** | Both support for traditional plate and new green plate <br/><br/> Sample the frame in a time windows to recognice the plate license, and vots the license in many samples, which lead less compute cost and better accuracy, and the result is much more stable. <br/><br/> hmean of text detector: 0.979; accuracy of recognition model: 0.773 <br/><br/> | <img title="" src="https://user-images.githubusercontent.com/48054808/185027987-6144cafd-0286-4c32-8425-7ab9515d1ec3.png" alt="" width="191"> |
| **Vehicle Attributes** | Identify 10 vehicle colors and 9 models <br/><br/> More powerfull backbone: PP-HGNet/PP-LCNet, with higher accuracy and faster speed <br/><br/> accuracy of model: 90.81 <br/><br/> | <img title="" src="https://user-images.githubusercontent.com/48054808/185044490-00edd930-1885-4e79-b3d4-3a39a77dea93.gif" alt="" width="207"> |
| **Illegal Parking** | Easy to use with one line command, and define the illegal area by yourself <br/><br/> Get the license of illegal car <br/><br/> | <img title="" src="https://user-images.githubusercontent.com/48054808/185028419-58ae0af8-a035-42e7-9583-25f5e4ce0169.png" alt="" width="209"> |
| **in-out counting** | Easy to use with one line command, and define the in-out line by yourself <br/><br/> Target route visualize with high tracking performance | <img title="" src="https://user-images.githubusercontent.com/48054808/185028798-9e07379f-7486-4266-9d27-3aec943593e0.gif" alt="" width="200"> |
## 🗳 Model Zoo ## 🗳 Model Zoo
<details> <details>
...@@ -40,6 +54,7 @@ PP-Human supports various inputs such as images, single-camera, and multi-camera ...@@ -40,6 +54,7 @@ PP-Human supports various inputs such as images, single-camera, and multi-camera
| Pedestrian detection (lightweight) | 16.2ms | [Multi-object tracking](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_s_36e_pipeline.zip) | 27M | | Pedestrian detection (lightweight) | 16.2ms | [Multi-object tracking](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_s_36e_pipeline.zip) | 27M |
| Pedestrian tracking (high precision) | 31.8ms | [Multi-object tracking](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_l_36e_pipeline.zip) | 182M | | Pedestrian tracking (high precision) | 31.8ms | [Multi-object tracking](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_l_36e_pipeline.zip) | 182M |
| Pedestrian tracking (lightweight) | 21.0ms | [Multi-object tracking](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_s_36e_pipeline.zip) | 27M | | Pedestrian tracking (lightweight) | 21.0ms | [Multi-object tracking](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_s_36e_pipeline.zip) | 27M |
| MTMCT(REID) | Single Person 1.5ms | [REID](https://bj.bcebos.com/v1/paddledet/models/pipeline/reid_model.zip) | REID:92M |
| Attribute recognition (high precision) | Single person8.5ms | [Object detection](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_l_36e_pipeline.zip)<br> [Attribute recognition](https://bj.bcebos.com/v1/paddledet/models/pipeline/strongbaseline_r50_30e_pa100k.zip) | Object detection:182M<br>Attribute recognition:86M | | Attribute recognition (high precision) | Single person8.5ms | [Object detection](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_l_36e_pipeline.zip)<br> [Attribute recognition](https://bj.bcebos.com/v1/paddledet/models/pipeline/strongbaseline_r50_30e_pa100k.zip) | Object detection:182M<br>Attribute recognition:86M |
| Attribute recognition (lightweight) | Single person 7.1ms | [Object detection](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_l_36e_pipeline.zip)<br> [Attribute recognition](https://bj.bcebos.com/v1/paddledet/models/pipeline/strongbaseline_r50_30e_pa100k.zip) | Object detection:182M<br>Attribute recognition:86M | | Attribute recognition (lightweight) | Single person 7.1ms | [Object detection](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_l_36e_pipeline.zip)<br> [Attribute recognition](https://bj.bcebos.com/v1/paddledet/models/pipeline/strongbaseline_r50_30e_pa100k.zip) | Object detection:182M<br>Attribute recognition:86M |
| Falling detection | Single person 10ms | [Multi-object tracking](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_l_36e_pipeline.zip) <br> [Keypoint detection](https://bj.bcebos.com/v1/paddledet/models/pipeline/dark_hrnet_w32_256x192.zip) <br> [Behavior detection based on key points](https://bj.bcebos.com/v1/paddledet/models/pipeline/STGCN.zip) | Multi-object tracking:182M<br>Keypoint detection:101M<br>Behavior detection based on key points: 21.8M | | Falling detection | Single person 10ms | [Multi-object tracking](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_l_36e_pipeline.zip) <br> [Keypoint detection](https://bj.bcebos.com/v1/paddledet/models/pipeline/dark_hrnet_w32_256x192.zip) <br> [Behavior detection based on key points](https://bj.bcebos.com/v1/paddledet/models/pipeline/STGCN.zip) | Multi-object tracking:182M<br>Keypoint detection:101M<br>Behavior detection based on key points: 21.8M |
...@@ -69,40 +84,60 @@ Click to download the model, then unzip and save it in the `. /output_inference` ...@@ -69,40 +84,60 @@ Click to download the model, then unzip and save it in the `. /output_inference`
## 📚 Doc Tutorials ## 📚 Doc Tutorials
### [A Quick Start](docs/tutorials/PPHuman_QUICK_STARTED_en.md) ### 🚶‍♀️ PP-Human
#### [A Quick Start](docs/tutorials/PPHuman_QUICK_STARTED_en.md)
### Pedestrian attribute/feature recognition #### Pedestrian attribute/feature recognition
* [A quick start](docs/tutorials/pphuman_attribute_en.md) * [A quick start](docs/tutorials/pphuman_attribute_en.md)
* [Customized development tutorials](../../docs/advanced_tutorials/customization/pphuman_attribute_en.md) * [Customized development tutorials](../../docs/advanced_tutorials/customization/pphuman_attribute_en.md)
* Data Preparation
* Model Optimization
* New Attributes
### Behavior detection #### Behavior detection
* [A quick start](docs/tutorials/pphuman_action_en.md) * [A quick start](docs/tutorials/pphuman_action_en.md)
* Falling detection
* Fighting detection
* [Customized development tutorials](../../docs/advanced_tutorials/customization/action_recognotion/README_en.md) * [Customized development tutorials](../../docs/advanced_tutorials/customization/action_recognotion/README_en.md)
* Solution Selection
* Data Preparation
* Model Optimization
* New Attributes
### ReID #### ReID
* [A quick start](docs/tutorials/pphuman_mtmct_en.md) * [A quick start](docs/tutorials/pphuman_mtmct_en.md)
* [Customized development tutorials](../../docs/advanced_tutorials/customization/pphuman_mtmct_en.md) * [Customized development tutorials](../../docs/advanced_tutorials/customization/pphuman_mtmct_en.md)
* Data Preparation
* Model Optimization
### Pedestrian tracking, visitor traffic statistics, trace records #### Pedestrian tracking, visitor traffic statistics, trace records
* [A quick start](docs/tutorials/pphuman_mot_en.md) * [A quick start](docs/tutorials/pphuman_mot_en.md)
* Pedestrian tracking,
* Visitor traffic statistics
* Regional intrusion diagnosis and counting
* [Customized development tutorials](../../docs/advanced_tutorials/customization/pphuman_mot_en.md) * [Customized development tutorials](../../docs/advanced_tutorials/customization/pphuman_mot_en.md)
* Data Preparation
* Model Optimization
### 🚘 PP-Vehicle
#### [A Quick Start](docs/tutorials/PPVehicle_QUICK_STARTED.md)
#### Vehicle Plate License
- [A quick start](docs/tutorials/ppvehicle_plate_en.md)
- [Customized development tutorials](../../docs/advanced_tutorials/customization/ppvehicle_plate.md)
#### Vehicle Attributes
- [A quick start](docs/tutorials/ppvehicle_attribute_en.md)
- [Customized development tutorials](../../docs/advanced_tutorials/customization/ppvehicle_attribute_en.md)
#### Illegal Parking
- [A quick start](docs/tutorials/ppvehicle_illegal_parking_en.md)
- [Customized development tutorials](../../docs/advanced_tutorials/customization/pphuman_mot_en.md)
#### Vehicle Tracking/in-out counint/Route Visualize
- [A quick start](docs/tutorials/ppvehicle_mot_en.md)
- [Customized development tutorials](../../docs/advanced_tutorials/customization/pphuman_mot_en.md)
...@@ -85,6 +85,11 @@ def argsparser(): ...@@ -85,6 +85,11 @@ def argsparser():
type=str, type=str,
default="output", default="output",
help="Directory of output visualization files.") help="Directory of output visualization files.")
parser.add_argument(
"--pushurl",
type=str,
default="",
help="url of output visualization stream.")
parser.add_argument( parser.add_argument(
"--run_mode", "--run_mode",
type=str, type=str,
......
...@@ -51,6 +51,7 @@ PP-Human提供了目标检测、属性识别、行为识别、ReID预训练模 ...@@ -51,6 +51,7 @@ PP-Human提供了目标检测、属性识别、行为识别、ReID预训练模
| 行人检测(轻量级) | 16.2ms | [多目标跟踪](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_s_36e_pipeline.zip) | 27M | | 行人检测(轻量级) | 16.2ms | [多目标跟踪](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_s_36e_pipeline.zip) | 27M |
| 行人跟踪(高精度) | 31.8ms | [多目标跟踪](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_l_36e_pipeline.zip) | 182M | | 行人跟踪(高精度) | 31.8ms | [多目标跟踪](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_l_36e_pipeline.zip) | 182M |
| 行人跟踪(轻量级) | 21.0ms | [多目标跟踪](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_s_36e_pipeline.zip) | 27M | | 行人跟踪(轻量级) | 21.0ms | [多目标跟踪](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_s_36e_pipeline.zip) | 27M |
| 跨镜跟踪(REID) | 单人1.5ms | [REID](https://bj.bcebos.com/v1/paddledet/models/pipeline/reid_model.zip) | REID:92M |
| 属性识别(高精度) | 单人8.5ms | [目标检测](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_l_36e_pipeline.zip)<br> [属性识别](https://bj.bcebos.com/v1/paddledet/models/pipeline/PPHGNet_small_person_attribute_954_infer.zip) | 目标检测:182M<br>属性识别:86M | | 属性识别(高精度) | 单人8.5ms | [目标检测](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_l_36e_pipeline.zip)<br> [属性识别](https://bj.bcebos.com/v1/paddledet/models/pipeline/PPHGNet_small_person_attribute_954_infer.zip) | 目标检测:182M<br>属性识别:86M |
| 属性识别(轻量级) | 单人7.1ms | [目标检测](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_l_36e_pipeline.zip)<br> [属性识别](https://bj.bcebos.com/v1/paddledet/models/pipeline/PPLCNet_x1_0_person_attribute_945_infer.zip) | 目标检测:182M<br>属性识别:86M | | 属性识别(轻量级) | 单人7.1ms | [目标检测](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_l_36e_pipeline.zip)<br> [属性识别](https://bj.bcebos.com/v1/paddledet/models/pipeline/PPLCNet_x1_0_person_attribute_945_infer.zip) | 目标检测:182M<br>属性识别:86M |
| 摔倒识别 | 单人10ms | [多目标跟踪](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_l_36e_pipeline.zip) <br> [关键点检测](https://bj.bcebos.com/v1/paddledet/models/pipeline/dark_hrnet_w32_256x192.zip) <br> [基于关键点行为识别](https://bj.bcebos.com/v1/paddledet/models/pipeline/STGCN.zip) | 多目标跟踪:182M<br>关键点检测:101M<br>基于关键点行为识别:21.8M | | 摔倒识别 | 单人10ms | [多目标跟踪](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_l_36e_pipeline.zip) <br> [关键点检测](https://bj.bcebos.com/v1/paddledet/models/pipeline/dark_hrnet_w32_256x192.zip) <br> [基于关键点行为识别](https://bj.bcebos.com/v1/paddledet/models/pipeline/STGCN.zip) | 多目标跟踪:182M<br>关键点检测:101M<br>基于关键点行为识别:21.8M |
...@@ -125,7 +126,10 @@ python deploy/pipeline/pipeline.py --config deploy/pipeline/config/infer_cfg_pph ...@@ -125,7 +126,10 @@ python deploy/pipeline/pipeline.py --config deploy/pipeline/config/infer_cfg_pph
python deploy/pipeline/pipeline.py --config deploy/pipeline/config/infer_cfg_pphuman.yml -o SKELETON_ACTION.enbale=True --video_file=test_video.mp4 --device=gpu python deploy/pipeline/pipeline.py --config deploy/pipeline/config/infer_cfg_pphuman.yml -o SKELETON_ACTION.enbale=True --video_file=test_video.mp4 --device=gpu
``` ```
3. 对rtsp流的支持,使用--rtsp RTSP [RTSP ...]参数指定一路或者多路rtsp视频流,如果是多路地址中间用空格隔开。(或者video_file后面的视频地址直接更换为rtsp流地址),示例如下: 3. rtsp推拉流
- rtsp拉流预测
对rtsp拉流的支持,使用--rtsp RTSP [RTSP ...]参数指定一路或者多路rtsp视频流,如果是多路地址中间用空格隔开。(或者video_file后面的视频地址直接更换为rtsp流地址),示例如下:
``` ```
# 例:行人属性识别,单路视频流 # 例:行人属性识别,单路视频流
python deploy/pipeline/pipeline.py --config deploy/pipeline/config/examples/infer_cfg_human_attr.yml -o visual=False --rtsp rtsp://[YOUR_RTSP_SITE] --device=gpu python deploy/pipeline/pipeline.py --config deploy/pipeline/config/examples/infer_cfg_human_attr.yml -o visual=False --rtsp rtsp://[YOUR_RTSP_SITE] --device=gpu
...@@ -134,6 +138,18 @@ python deploy/pipeline/pipeline.py --config deploy/pipeline/config/examples/infe ...@@ -134,6 +138,18 @@ python deploy/pipeline/pipeline.py --config deploy/pipeline/config/examples/infe
python deploy/pipeline/pipeline.py --config deploy/pipeline/config/examples/infer_cfg_human_attr.yml -o visual=False --rtsp rtsp://[YOUR_RTSP_SITE1] rtsp://[YOUR_RTSP_SITE2] --device=gpu python deploy/pipeline/pipeline.py --config deploy/pipeline/config/examples/infer_cfg_human_attr.yml -o visual=False --rtsp rtsp://[YOUR_RTSP_SITE1] rtsp://[YOUR_RTSP_SITE2] --device=gpu
``` ```
- 视频结果推流rtsp
预测结果进行rtsp推流,使用--pushurl rtsp:[IP] 推流到IP地址端,PC端可以使用[VLC播放器](https://vlc.onl/)打开网络流进行播放,播放地址为 `rtsp:[IP]/videoname`。其中`videoname`是预测的视频文件名,如果视频来源是本地摄像头则`videoname`默认为`output`.
```
# 例:行人属性识别,单路视频流,该示例播放地址为 rtsp://[YOUR_SERVER_IP]:8554/test_video
python deploy/pipeline/pipeline.py --config deploy/pipeline/config/examples/infer_cfg_human_attr.yml --video_file=test_video.mp4 --device=gpu --pushurl rtsp://[YOUR_SERVER_IP]:8554
```
注:
1. rtsp推流服务基于 [rtsp-simple-server](https://github.com/aler9/rtsp-simple-server), 如使用推流功能请先开启该服务.
2. rtsp推流如果模型处理速度跟不上会出现很明显的卡顿现象,建议跟踪模型使用ppyoloe_s版本,即修改配置中跟踪模型mot_ppyoloe_l_36e_pipeline.zip替换为mot_ppyoloe_s_36e_pipeline.zip。
### Jetson部署说明 ### Jetson部署说明
由于Jetson平台算力相比服务器有较大差距,有如下使用建议: 由于Jetson平台算力相比服务器有较大差距,有如下使用建议:
...@@ -158,6 +174,7 @@ python deploy/pipeline/pipeline.py --config deploy/pipeline/config/examples/infe ...@@ -158,6 +174,7 @@ python deploy/pipeline/pipeline.py --config deploy/pipeline/config/examples/infe
| --rtsp | Option | rtsp视频流地址,支持一路或者多路同时输入 | | --rtsp | Option | rtsp视频流地址,支持一路或者多路同时输入 |
| --camera_id | Option | 用来预测的摄像头ID,默认为-1(表示不使用摄像头预测,可设置为:0 - (摄像头数目-1) ),预测过程中在可视化界面按`q`退出输出预测结果到:output/output.mp4| | --camera_id | Option | 用来预测的摄像头ID,默认为-1(表示不使用摄像头预测,可设置为:0 - (摄像头数目-1) ),预测过程中在可视化界面按`q`退出输出预测结果到:output/output.mp4|
| --device | Option | 运行时的设备,可选择`CPU/GPU/XPU`,默认为`CPU`| | --device | Option | 运行时的设备,可选择`CPU/GPU/XPU`,默认为`CPU`|
| --pushurl | Option| 对预测结果视频进行推流的地址,以rtsp://开头,该选项优先级高于视频结果本地存储,打开时不再另外存储本地预测结果视频|
| --output_dir | Option|可视化结果保存的根目录,默认为output/| | --output_dir | Option|可视化结果保存的根目录,默认为output/|
| --run_mode | Option |使用GPU时,默认为paddle, 可选(paddle/trt_fp32/trt_fp16/trt_int8)| | --run_mode | Option |使用GPU时,默认为paddle, 可选(paddle/trt_fp32/trt_fp16/trt_int8)|
| --enable_mkldnn | Option | CPU预测中是否开启MKLDNN加速,默认为False | | --enable_mkldnn | Option | CPU预测中是否开启MKLDNN加速,默认为False |
......
...@@ -51,6 +51,7 @@ PP-Human provides object detection, attribute recognition, behaviour recognition ...@@ -51,6 +51,7 @@ PP-Human provides object detection, attribute recognition, behaviour recognition
| Pedestrian Detection (Lightweight) | 16.2ms | [Multi-Object Tracking](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_s_36e_pipeline.zip) | 27M | | Pedestrian Detection (Lightweight) | 16.2ms | [Multi-Object Tracking](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_s_36e_pipeline.zip) | 27M |
| Pedestrian Tracking (high precision) | 31.8ms | [Multi-Object Tracking](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_l_36e_pipeline.zip) | 182M | | Pedestrian Tracking (high precision) | 31.8ms | [Multi-Object Tracking](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_l_36e_pipeline.zip) | 182M |
| Pedestrian Tracking (Lightweight) | 21.0ms | [Multi-Object Tracking](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_s_36e_pipeline.zip) | 27M | | Pedestrian Tracking (Lightweight) | 21.0ms | [Multi-Object Tracking](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_s_36e_pipeline.zip) | 27M |
| MTMCT(REID) | Single Person 1.5ms | [REID](https://bj.bcebos.com/v1/paddledet/models/pipeline/reid_model.zip) | REID:92M |
| Attribute Recognition (high precision) | Single Person 8.5ms | [Object Detection](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_l_36e_pipeline.zip)<br> [Attribute Recognition](https://bj.bcebos.com/v1/paddledet/models/pipeline/PPHGNet_small_person_attribute_954_infer.zip) | Object Detection:182M<br>Attribute Recogniton:86M | | Attribute Recognition (high precision) | Single Person 8.5ms | [Object Detection](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_l_36e_pipeline.zip)<br> [Attribute Recognition](https://bj.bcebos.com/v1/paddledet/models/pipeline/PPHGNet_small_person_attribute_954_infer.zip) | Object Detection:182M<br>Attribute Recogniton:86M |
| Attribute Recognition (Lightweight) | Single Person 7.1ms | [Object Detection](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_l_36e_pipeline.zip)<br> [Attribute Recogniton](https://bj.bcebos.com/v1/paddledet/models/pipeline/PPLCNet_x1_0_person_attribute_945_infer.zip) | Object Detection:182M<br>Attribute Recogniton:86M | | Attribute Recognition (Lightweight) | Single Person 7.1ms | [Object Detection](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_l_36e_pipeline.zip)<br> [Attribute Recogniton](https://bj.bcebos.com/v1/paddledet/models/pipeline/PPLCNet_x1_0_person_attribute_945_infer.zip) | Object Detection:182M<br>Attribute Recogniton:86M |
| Falling Detection | Single Person 10ms | [Multi-Object Tracking](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_l_36e_pipeline.zip) <br> [Keypoint Detection](https://bj.bcebos.com/v1/paddledet/models/pipeline/dark_hrnet_w32_256x192.zip) <br> [Skeleton Based Action Recognition](https://bj.bcebos.com/v1/paddledet/models/pipeline/STGCN.zip) | Multi-Object Tracking:182M<br>Keypoint Detection:101M<br>Skeleton Based Action Recognition:21.8M | | Falling Detection | Single Person 10ms | [Multi-Object Tracking](https://bj.bcebos.com/v1/paddledet/models/pipeline/mot_ppyoloe_l_36e_pipeline.zip) <br> [Keypoint Detection](https://bj.bcebos.com/v1/paddledet/models/pipeline/dark_hrnet_w32_256x192.zip) <br> [Skeleton Based Action Recognition](https://bj.bcebos.com/v1/paddledet/models/pipeline/STGCN.zip) | Multi-Object Tracking:182M<br>Keypoint Detection:101M<br>Skeleton Based Action Recognition:21.8M |
...@@ -125,7 +126,10 @@ python deploy/pipeline/pipeline.py --config deploy/pipeline/config/infer_cfg_pph ...@@ -125,7 +126,10 @@ python deploy/pipeline/pipeline.py --config deploy/pipeline/config/infer_cfg_pph
python deploy/pipeline/pipeline.py --config deploy/pipeline/config/infer_cfg_pphuman.yml -o SKELETON_ACTION.enbale=True --video_file=test_video.mp4 --device=gpu python deploy/pipeline/pipeline.py --config deploy/pipeline/config/infer_cfg_pphuman.yml -o SKELETON_ACTION.enbale=True --video_file=test_video.mp4 --device=gpu
``` ```
3. For rtsp stream, use --rtsp RTSP [RTSP ...] parameter to specify one or more rtsp streams. Separate the multiple addresses with a space, or replace the video address directly after the video_file with the rtsp stream address), examples as follows 3. rtsp push/pull stream
- rtsp pull stream
For rtsp pull stream, use `--rtsp RTSP [RTSP ...]` parameter to specify one or more rtsp streams. Separate the multiple addresses with a space, or replace the video address directly after the video_file with the rtsp stream address), examples as follows
``` ```
# Example: Single video stream for pedestrian attribute recognition # Example: Single video stream for pedestrian attribute recognition
...@@ -134,6 +138,19 @@ python deploy/pipeline/pipeline.py --config deploy/pipeline/config/examples/infe ...@@ -134,6 +138,19 @@ python deploy/pipeline/pipeline.py --config deploy/pipeline/config/examples/infe
python deploy/pipeline/pipeline.py --config deploy/pipeline/config/examples/infer_cfg_human_attr.yml -o visual=False --rtsp rtsp://[YOUR_RTSP_SITE1] rtsp://[YOUR_RTSP_SITE2] --device=gpu | python deploy/pipeline/pipeline.py --config deploy/pipeline/config/examples/infer_cfg_human_attr.yml -o visual=False --rtsp rtsp://[YOUR_RTSP_SITE1] rtsp://[YOUR_RTSP_SITE2] --device=gpu |
``` ```
- rtsp push stream
For rtsp push stream, use `--pushurl rtsp:[IP]` parameter to push stream to a IP set, and you can visualize the output video by [VLC Player](https://vlc.onl/) with the `open network` funciton. the whole url path is `rtsp:[IP]/videoname`, the videoname here is the basename of the video file to infer, and the default of videoname is `output` when the video come from local camera and there is no video name.
```
# Example:Pedestrian attribute recognition,in this example the whole url path is: rtsp://[YOUR_SERVER_IP]:8554/test_video
python deploy/pipeline/pipeline.py --config deploy/pipeline/config/examples/infer_cfg_human_attr.yml --video_file=test_video.mp4 --device=gpu --pushurl rtsp://[YOUR_SERVER_IP]:8554
```
Note:
1. rtsp push stream is based on [rtsp-simple-server](https://github.com/aler9/rtsp-simple-server), please enable this serving first.
2. the output visualize will be frozen frequently if the model cost too much time, we suggest to use faster model like ppyoloe_s in tracking, this is simply replace mot_ppyoloe_l_36e_pipeline.zip with mot_ppyoloe_s_36e_pipeline.zip in model config yaml file.
### Jetson Deployment ### Jetson Deployment
Due to the large gap in computing power of the Jetson platform compared to the server, we suggest: Due to the large gap in computing power of the Jetson platform compared to the server, we suggest:
...@@ -155,6 +172,8 @@ With this recommended configuration, it is possible to achieve higher speeds on ...@@ -155,6 +172,8 @@ With this recommended configuration, it is possible to achieve higher speeds on
| --rtsp | Option | rtsp video stream address, supports one or more simultaneous streams input | | --rtsp | Option | rtsp video stream address, supports one or more simultaneous streams input |
| --camera_id | Option | The camera ID for prediction, default is -1 ( for no camera prediction, can be set to 0 - (number of cameras - 1) ), press `q` in the visualization interface during the prediction process to output the prediction result to: output/output.mp4 | | --camera_id | Option | The camera ID for prediction, default is -1 ( for no camera prediction, can be set to 0 - (number of cameras - 1) ), press `q` in the visualization interface during the prediction process to output the prediction result to: output/output.mp4 |
| --device | Option | Running device, options include `CPU/GPU/XPU`, and the default is `CPU`. | | --device | Option | Running device, options include `CPU/GPU/XPU`, and the default is `CPU`. |
| --pushurl | Option | push the output video to rtsp stream, normaly start with `rtsp://`; this has higher priority than local video save, while this is set, pipeline will not save local visualize video, the default is "", means this will not work now.
|
| --output_dir | Option | The root directory for the visualization results, and the default is output/ | | --output_dir | Option | The root directory for the visualization results, and the default is output/ |
| --run_mode | Option | For GPU, the default is paddle, with (paddle/trt_fp32/trt_fp16/trt_int8) as optional | | --run_mode | Option | For GPU, the default is paddle, with (paddle/trt_fp32/trt_fp16/trt_int8) as optional |
| --enable_mkldnn | Option | Whether to enable MKLDNN acceleration in CPU prediction, the default is False | | --enable_mkldnn | Option | Whether to enable MKLDNN acceleration in CPU prediction, the default is False |
......
...@@ -19,7 +19,7 @@ ...@@ -19,7 +19,7 @@
## 环境准备 ## 环境准备
环境要求: PaddleDetection版本 >= release/2.4 或 release/2.5版本 环境要求: PaddleDetection版本 >= release/2.5 或 develop版本
PaddlePaddle和PaddleDetection安装 PaddlePaddle和PaddleDetection安装
...@@ -75,7 +75,7 @@ PP-Vehicle相关配置位于```deploy/pipeline/config/infer_cfg_ppvehicle.yml``` ...@@ -75,7 +75,7 @@ PP-Vehicle相关配置位于```deploy/pipeline/config/infer_cfg_ppvehicle.yml```
|-------|-------|----------|-----| |-------|-------|----------|-----|
| 图片 | 属性识别 | 目标检测 属性识别 | DET ATTR | | 图片 | 属性识别 | 目标检测 属性识别 | DET ATTR |
| 单镜头视频 | 属性识别 | 多目标跟踪 属性识别 | MOT ATTR | | 单镜头视频 | 属性识别 | 多目标跟踪 属性识别 | MOT ATTR |
| 单镜头视频 | 属性识别 | 多目标跟踪 属性识别 | MOT VEHICLEPLATE | | 单镜头视频 | 车牌识别 | 多目标跟踪 车牌识别 | MOT VEHICLEPLATE |
例如基于视频输入的属性识别,任务类型包含多目标跟踪和属性识别,具体配置如下: 例如基于视频输入的属性识别,任务类型包含多目标跟踪和属性识别,具体配置如下:
...@@ -131,15 +131,29 @@ python deploy/pipeline/pipeline.py --config deploy/pipeline/config/examples/infe ...@@ -131,15 +131,29 @@ python deploy/pipeline/pipeline.py --config deploy/pipeline/config/examples/infe
``` ```
3. 对rtsp流的支持,使用--rtsp RTSP [RTSP ...]参数指定一路或者多路rtsp视频流,如果是多路地址中间用空格隔开。(或者video_file后面的视频地址直接更换为rtsp流地址),示例如下: 3. rtsp推拉流
- rtsp拉流预测
对rtsp拉流的支持,使用--rtsp RTSP [RTSP ...]参数指定一路或者多路rtsp视频流,如果是多路地址中间用空格隔开。(或者video_file后面的视频地址直接更换为rtsp流地址),示例如下:
``` ```
# 例:行人属性识别,单路视频流 # 例:车辆属性识别,单路视频流
python deploy/pipeline/pipeline.py --config deploy/pipeline/config/examples/infer_cfg_vehicle_attr.yml -o visual=False --rtsp rtsp://[YOUR_RTSP_SITE] --device=gpu python deploy/pipeline/pipeline.py --config deploy/pipeline/config/examples/infer_cfg_vehicle_attr.yml -o visual=False --rtsp rtsp://[YOUR_RTSP_SITE] --device=gpu
# 例:行人属性识别,多路视频流 # 例:车辆属性识别,多路视频流
python deploy/pipeline/pipeline.py --config deploy/pipeline/config/examples/infer_cfg_vehicle_attr.yml -o visual=False --rtsp rtsp://[YOUR_RTSP_SITE1] rtsp://[YOUR_RTSP_SITE2] --device=gpu python deploy/pipeline/pipeline.py --config deploy/pipeline/config/examples/infer_cfg_vehicle_attr.yml -o visual=False --rtsp rtsp://[YOUR_RTSP_SITE1] rtsp://[YOUR_RTSP_SITE2] --device=gpu
``` ```
- 视频结果推流rtsp
预测结果进行rtsp推流,使用--pushurl rtsp:[IP] 推流到IP地址端,PC端可以使用[VLC播放器](https://vlc.onl/)打开网络流进行播放,播放地址为 `rtsp:[IP]/videoname`。其中`videoname`是预测的视频文件名,如果视频来源是本地摄像头则`videoname`默认为`output`.
```
# 例:车辆属性识别,单路视频流,该示例播放地址为 rtsp://[YOUR_SERVER_IP]:8554/test_video
python deploy/pipeline/pipeline.py --config deploy/pipeline/config/examples/infer_cfg_vehicle_attr.yml -o visual=False --video_file=test_video.mp4 --device=gpu --pushurl rtsp://[YOUR_SERVER_IP]:8554
```
注:
1. rtsp推流服务基于 [rtsp-simple-server](https://github.com/aler9/rtsp-simple-server), 如使用推流功能请先开启该服务.
2. rtsp推流如果模型处理速度跟不上会出现很明显的卡顿现象,建议跟踪模型使用ppyoloe_s版本,即修改配置中跟踪模型mot_ppyoloe_l_36e_pipeline.zip替换为mot_ppyoloe_s_36e_pipeline.zip。
### Jetson部署说明 ### Jetson部署说明
由于Jetson平台算力相比服务器有较大差距,有如下使用建议: 由于Jetson平台算力相比服务器有较大差距,有如下使用建议:
...@@ -164,6 +178,7 @@ python deploy/pipeline/pipeline.py --config deploy/pipeline/config/examples/infe ...@@ -164,6 +178,7 @@ python deploy/pipeline/pipeline.py --config deploy/pipeline/config/examples/infe
| --rtsp | Option | rtsp视频流地址,支持一路或者多路同时输入 | | --rtsp | Option | rtsp视频流地址,支持一路或者多路同时输入 |
| --camera_id | Option | 用来预测的摄像头ID,默认为-1(表示不使用摄像头预测,可设置为:0 - (摄像头数目-1) ),预测过程中在可视化界面按`q`退出输出预测结果到:output/output.mp4| | --camera_id | Option | 用来预测的摄像头ID,默认为-1(表示不使用摄像头预测,可设置为:0 - (摄像头数目-1) ),预测过程中在可视化界面按`q`退出输出预测结果到:output/output.mp4|
| --device | Option | 运行时的设备,可选择`CPU/GPU/XPU`,默认为`CPU`| | --device | Option | 运行时的设备,可选择`CPU/GPU/XPU`,默认为`CPU`|
| --pushurl | Option| 对预测结果视频进行推流的地址,以rtsp://开头,该选项优先级高于视频结果本地存储,打开时不再另外存储本地预测结果视频, 默认为空,表示没有开启|
| --output_dir | Option|可视化结果保存的根目录,默认为output/| | --output_dir | Option|可视化结果保存的根目录,默认为output/|
| --run_mode | Option |使用GPU时,默认为paddle, 可选(paddle/trt_fp32/trt_fp16/trt_int8)| | --run_mode | Option |使用GPU时,默认为paddle, 可选(paddle/trt_fp32/trt_fp16/trt_int8)|
| --enable_mkldnn | Option | CPU预测中是否开启MKLDNN加速,默认为False | | --enable_mkldnn | Option | CPU预测中是否开启MKLDNN加速,默认为False |
......
...@@ -18,7 +18,7 @@ English | [简体中文](PPVehicle_QUICK_STARTED.md) ...@@ -18,7 +18,7 @@ English | [简体中文](PPVehicle_QUICK_STARTED.md)
## Environment Preparation ## Environment Preparation
Environment Preparation: PaddleDetection version >= release/2.4 or release/2.5 Environment Preparation: PaddleDetection version >= release/2.5 or develop
Installation of PaddlePaddle and PaddleDetection Installation of PaddlePaddle and PaddleDetection
...@@ -73,7 +73,7 @@ The features and corresponding task types are as follows. ...@@ -73,7 +73,7 @@ The features and corresponding task types are as follows.
| ------------------- | --------------------- | ------------------------------------------- | ---------------- | | ------------------- | --------------------- | ------------------------------------------- | ---------------- |
| Image | Attribute Recognition | Object Detection Attribute Recognition | DET ATTR | | Image | Attribute Recognition | Object Detection Attribute Recognition | DET ATTR |
| Single-camera video | Attribute Recognition | Multi-Object Tracking Attribute Recognition | MOT ATTR | | Single-camera video | Attribute Recognition | Multi-Object Tracking Attribute Recognition | MOT ATTR |
| Single-camera video | Attribute Recognition | Multi-Object Tracking Attribute Recognition | MOT VEHICLEPLATE | | Single-camera video | License-plate Recognition | Multi-Object Tracking License-plate Recognition | MOT VEHICLEPLATE |
Take attribute recognition based on video input as an example: Its task type includes multi-object tracking and attributes recognition. The specific configuration is as follows. Take attribute recognition based on video input as an example: Its task type includes multi-object tracking and attributes recognition. The specific configuration is as follows.
...@@ -129,7 +129,10 @@ python deploy/pipeline/pipeline.py --config deploy/pipeline/config/examples/infe ...@@ -129,7 +129,10 @@ python deploy/pipeline/pipeline.py --config deploy/pipeline/config/examples/infe
--region_polygon 600 300 1300 300 1300 800 600 800 --region_polygon 600 300 1300 300 1300 800 600 800
``` ```
3. For rtsp stream, use --rtsp RTSP [RTSP ...] parameter to specify one or more rtsp streams. Separate the multiple addresses with a space, or replace the video address directly after the video_file with the rtsp stream address), examples as follows 3. rtsp push/pull stream
- rtsp pull stream
For rtsp pull stream, use --rtsp RTSP [RTSP ...] parameter to specify one or more rtsp streams. Separate the multiple addresses with a space, or replace the video address directly after the video_file with the rtsp stream address), examples as follows
``` ```
# Example: Single video stream for pedestrian attribute recognition # Example: Single video stream for pedestrian attribute recognition
...@@ -138,6 +141,18 @@ python deploy/pipeline/pipeline.py --config deploy/pipeline/config/examples/infe ...@@ -138,6 +141,18 @@ python deploy/pipeline/pipeline.py --config deploy/pipeline/config/examples/infe
python deploy/pipeline/pipeline.py --config deploy/pipeline/config/examples/infer_cfg_human_attr.yml -o visual=False --rtsp rtsp://[YOUR_RTSP_SITE1] rtsp://[YOUR_RTSP_SITE2] --device=gpu | python deploy/pipeline/pipeline.py --config deploy/pipeline/config/examples/infer_cfg_human_attr.yml -o visual=False --rtsp rtsp://[YOUR_RTSP_SITE1] rtsp://[YOUR_RTSP_SITE2] --device=gpu |
``` ```
- rtsp push stream
For rtsp push stream, use --pushurl rtsp:[IP] parameter to push stream to a IP set, and you can visualize the output video by [VLC Player](https://vlc.onl/) with the `open network` funciton. the whole url path is `rtsp:[IP]/videoname`, the videoname here is the basename of the video file to infer, and the default of videoname is `output` when the video come from local camera and there is no video name.
```
# Example:license plate recognition,in this example the whole url path is: rtsp://[YOUR_SERVER_IP]:8554/test_video
python deploy/pipeline/pipeline.py --config deploy/pipeline/config/examples/infer_cfg_vehicle_plate.yml --video_file=test_video.mp4 --device=gpu --pushurl rtsp://[YOUR_SERVER_IP]:8554
```
Note:
1. rtsp push stream is based on [rtsp-simple-server](https://github.com/aler9/rtsp-simple-server), please enable this serving first.
2. the output visualize will be frozen frequently if the model cost too much time, we suggest to use faster model like ppyoloe_s in tracking, this is simply replace mot_ppyoloe_l_36e_pipeline.zip with mot_ppyoloe_s_36e_pipeline.zip in model config yaml file.
### Jetson Deployment ### Jetson Deployment
Due to the large gap in computing power of the Jetson platform compared to the server, we suggest: Due to the large gap in computing power of the Jetson platform compared to the server, we suggest:
...@@ -161,6 +176,8 @@ With this recommended configuration, it is possible to achieve higher speeds on ...@@ -161,6 +176,8 @@ With this recommended configuration, it is possible to achieve higher speeds on
| --rtsp | Option | rtsp video stream address, supports one or more simultaneous streams input | | --rtsp | Option | rtsp video stream address, supports one or more simultaneous streams input |
| --camera_id | Option | The camera ID for prediction, default is -1 ( for no camera prediction, can be set to 0 - (number of cameras - 1) ), press `q` in the visualization interface during the prediction process to output the prediction result to: output/output.mp4 | | --camera_id | Option | The camera ID for prediction, default is -1 ( for no camera prediction, can be set to 0 - (number of cameras - 1) ), press `q` in the visualization interface during the prediction process to output the prediction result to: output/output.mp4 |
| --device | Option | Running device, options include `CPU/GPU/XPU`, and the default is `CPU`. | | --device | Option | Running device, options include `CPU/GPU/XPU`, and the default is `CPU`. |
| --pushurl | Option | push the output video to rtsp stream, normaly start with `rtsp://`; this has higher priority than local video save, while this is set, pipeline will not save local visualize video, the default is "", means this will not work now.
|
| --output_dir | Option | The root directory for the visualization results, and the default is output/ | | --output_dir | Option | The root directory for the visualization results, and the default is output/ |
| --run_mode | Option | For GPU, the default is paddle, with (paddle/trt_fp32/trt_fp16/trt_int8) as optional | | --run_mode | Option | For GPU, the default is paddle, with (paddle/trt_fp32/trt_fp16/trt_int8) as optional |
| --enable_mkldnn | Option | Whether to enable MKLDNN acceleration in CPU prediction, the default is False | | --enable_mkldnn | Option | Whether to enable MKLDNN acceleration in CPU prediction, the default is False |
......
...@@ -19,6 +19,7 @@ import glob ...@@ -19,6 +19,7 @@ import glob
import yaml import yaml
import copy import copy
import numpy as np import numpy as np
import subprocess as sp
from python.keypoint_preprocess import EvalAffine, TopDownEvalAffine, expand_crop from python.keypoint_preprocess import EvalAffine, TopDownEvalAffine, expand_crop
...@@ -122,6 +123,26 @@ class PipeTimer(Times): ...@@ -122,6 +123,26 @@ class PipeTimer(Times):
dic['img_num'] = self.img_num dic['img_num'] = self.img_num
return dic return dic
class PushStream(object):
def __init__(self, pushurl = "rtsp://127.0.0.1:8554/"):
self.command = ""
# 自行设置
self.pushurl = pushurl
def initcmd(self, fps, width, height):
self.command = ['ffmpeg',
'-y',
'-f', 'rawvideo',
'-vcodec','rawvideo',
'-pix_fmt', 'bgr24',
'-s', "{}x{}".format(width, height),
'-r', str(fps),
'-i', '-',
'-pix_fmt', 'yuv420p',
'-f', 'rtsp',
self.pushurl]
self.pipe = sp.Popen(self.command, stdin=sp.PIPE)
def get_test_images(infer_dir, infer_img): def get_test_images(infer_dir, infer_img):
""" """
......
...@@ -31,6 +31,7 @@ sys.path.insert(0, parent_path) ...@@ -31,6 +31,7 @@ sys.path.insert(0, parent_path)
from cfg_utils import argsparser, print_arguments, merge_cfg from cfg_utils import argsparser, print_arguments, merge_cfg
from pipe_utils import PipeTimer from pipe_utils import PipeTimer
from pipe_utils import get_test_images, crop_image_with_det, crop_image_with_mot, parse_mot_res, parse_mot_keypoint from pipe_utils import get_test_images, crop_image_with_det, crop_image_with_mot, parse_mot_res, parse_mot_keypoint
from pipe_utils import PushStream
from python.infer import Detector, DetectorPicoDet from python.infer import Detector, DetectorPicoDet
from python.keypoint_infer import KeyPointDetector from python.keypoint_infer import KeyPointDetector
...@@ -340,6 +341,8 @@ class PipePredictor(object): ...@@ -340,6 +341,8 @@ class PipePredictor(object):
self.file_name = None self.file_name = None
self.collector = DataCollector() self.collector = DataCollector()
self.pushurl = args.pushurl
# auto download inference model # auto download inference model
get_model_dir(self.cfg) get_model_dir(self.cfg)
...@@ -471,6 +474,8 @@ class PipePredictor(object): ...@@ -471,6 +474,8 @@ class PipePredictor(object):
def set_file_name(self, path): def set_file_name(self, path):
if path is not None: if path is not None:
self.file_name = os.path.split(path)[-1] self.file_name = os.path.split(path)[-1]
if "." in self.file_name:
self.file_name = self.file_name.split(".")[-2]
else: else:
# use camera id # use camera id
self.file_name = None self.file_name = None
...@@ -571,10 +576,6 @@ class PipePredictor(object): ...@@ -571,10 +576,6 @@ class PipePredictor(object):
# mot -> attr # mot -> attr
# mot -> pose -> action # mot -> pose -> action
capture = cv2.VideoCapture(video_file) capture = cv2.VideoCapture(video_file)
video_out_name = 'output.mp4' if self.file_name is None else self.file_name
if "rtsp" in video_file:
video_out_name = video_out_name + "_t" + str(thread_idx).zfill(
2) + "_rtsp.mp4"
# Get Video info : resolution, fps, frame count # Get Video info : resolution, fps, frame count
width = int(capture.get(cv2.CAP_PROP_FRAME_WIDTH)) width = int(capture.get(cv2.CAP_PROP_FRAME_WIDTH))
...@@ -583,11 +584,23 @@ class PipePredictor(object): ...@@ -583,11 +584,23 @@ class PipePredictor(object):
frame_count = int(capture.get(cv2.CAP_PROP_FRAME_COUNT)) frame_count = int(capture.get(cv2.CAP_PROP_FRAME_COUNT))
print("video fps: %d, frame_count: %d" % (fps, frame_count)) print("video fps: %d, frame_count: %d" % (fps, frame_count))
if len(self.pushurl) > 0:
video_out_name = 'output' if self.file_name is None else self.file_name
pushurl = os.path.join(self.pushurl, video_out_name)
print("the result will push stream to url:{}".format(pushurl))
pushstream = PushStream(pushurl)
pushstream.initcmd(fps, width, height)
elif self.cfg['visual']:
video_out_name = 'output' if self.file_name is None else self.file_name
if "rtsp" in video_file:
video_out_name = video_out_name + "_t" + str(thread_idx).zfill(
2) + "_rtsp"
if not os.path.exists(self.output_dir): if not os.path.exists(self.output_dir):
os.makedirs(self.output_dir) os.makedirs(self.output_dir)
out_path = os.path.join(self.output_dir, video_out_name) out_path = os.path.join(self.output_dir, video_out_name+".mp4")
fourcc = cv2.VideoWriter_fourcc(* 'mp4v') fourcc = cv2.VideoWriter_fourcc(* 'mp4v')
writer = cv2.VideoWriter(out_path, fourcc, fps, (width, height)) writer = cv2.VideoWriter(out_path, fourcc, fps, (width, height))
frame_id = 0 frame_id = 0
entrance, records, center_traj = None, None, None entrance, records, center_traj = None, None, None
...@@ -709,6 +722,9 @@ class PipePredictor(object): ...@@ -709,6 +722,9 @@ class PipePredictor(object):
im = self.visualize_video(frame, mot_res, frame_id, fps, im = self.visualize_video(frame, mot_res, frame_id, fps,
entrance, records, entrance, records,
center_traj) # visualize center_traj) # visualize
if len(self.pushurl)>0:
pushstream.pipe.stdin.write(im.tobytes())
else:
writer.write(im) writer.write(im)
if self.file_name is None: # use camera_id if self.file_name is None: # use camera_id
cv2.imshow('Paddle-Pipeline', im) cv2.imshow('Paddle-Pipeline', im)
...@@ -882,12 +898,15 @@ class PipePredictor(object): ...@@ -882,12 +898,15 @@ class PipePredictor(object):
entrance, records, center_traj, entrance, records, center_traj,
self.illegal_parking_time != -1, self.illegal_parking_time != -1,
illegal_parking_dict) # visualize illegal_parking_dict) # visualize
if len(self.pushurl)>0:
pushstream.pipe.stdin.write(im.tobytes())
else:
writer.write(im) writer.write(im)
if self.file_name is None: # use camera_id if self.file_name is None: # use camera_id
cv2.imshow('Paddle-Pipeline', im) cv2.imshow('Paddle-Pipeline', im)
if cv2.waitKey(1) & 0xFF == ord('q'): if cv2.waitKey(1) & 0xFF == ord('q'):
break break
if self.cfg['visual'] and len(self.pushurl)==0:
writer.release() writer.release()
print('save result to {}'.format(out_path)) print('save result to {}'.format(out_path))
......
...@@ -347,7 +347,7 @@ def res2dict(multi_res): ...@@ -347,7 +347,7 @@ def res2dict(multi_res):
for tid, res in c_res.items(): for tid, res in c_res.items():
key = "c" + str(cid) + "_t" + str(tid) key = "c" + str(cid) + "_t" + str(tid)
if key not in cid_tid_dict: if key not in cid_tid_dict:
if len(res["rects"]) < 10: if len(res["features"])==0:
continue continue
cid_tid_dict[key] = res cid_tid_dict[key] = res
cid_tid_dict[key]['mean_feat'] = distill_idfeat(res) cid_tid_dict[key]['mean_feat'] = distill_idfeat(res)
......
Markdown is supported
0% .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑!
想要评论请 注册