| Single-Camera Video | Attribute Recognition | Multi-Object Tracking Attribute Recognition | MOT ATTR |
| Single-Camera Video | Attribute Recognition | Multi-Object Tracking Attribute Recognition | MOT ATTR |
| Single-Camera Video | Behavior Recognition | Multi-Object Tracking Keypoint Detection Action Recognition | MOT KPT ACTION |
| Single-Camera Video | Behavior Recognition | Multi-Object Tracking Keypoint Detection Falling Recognition | MOT KPT FALLING |
For example, for the attribute recognition with the video input, its task types contain multi-object tracking and attribute recognition, and the config is:
For example, for the attribute recognition with the video input, its task types contain multi-object tracking and attribute recognition, and the config is:
...
@@ -89,7 +89,7 @@ ATTR:
...
@@ -89,7 +89,7 @@ ATTR:
**Note: **
**Note: **
- For different tasks, users could add `--enable_attr=True` or `--enable_action=True` in command line and do not need to set config file.
- For different tasks, users could add `--enable_attr=True` or `--enable_falling=True` in command line and do not need to set config file.
- if only need to change the model path, users could add `--model_dir det=ppyoloe/` in command line and do not need to set config file. For details info please refer to doc below.
- if only need to change the model path, users could add `--model_dir det=ppyoloe/` in command line and do not need to set config file. For details info please refer to doc below.
| --camera_id | Option | ID of the inference camera is -1 by default (means inference without cameras，and it can be set to 0 - (number of cameras-1)), and during the inference, click `q` on the visual interface to exit and output the inference result to output/output.mp4|
| --camera_id | Option | ID of the inference camera is -1 by default (means inference without cameras，and it can be set to 0 - (number of cameras-1)), and during the inference, click `q` on the visual interface to exit and output the inference result to output/output.mp4|
| --enable_attr| Option | Enable attribute recognition or not |
| --enable_attr| Option | Enable attribute recognition or not |
| --enable_action| Option | Enable action recognition or not |
| --enable_falling| Option | Enable action recognition or not |
| --device | Option | During the operation，available devices are `CPU/GPU/XPU`，and the default is `CPU`|
| --device | Option | During the operation，available devices are `CPU/GPU/XPU`，and the default is `CPU`|
| --output_dir | Option| The default root directory which stores the visualization result is output/|
| --output_dir | Option| The default root directory which stores the visualization result is output/|
| --run_mode | Option | When using GPU，the default one is paddle, and all these are available（paddle/trt_fp32/trt_fp16/trt_int8）.|
| --run_mode | Option | When using GPU，the default one is paddle, and all these are available（paddle/trt_fp32/trt_fp16/trt_int8）.|
...
@@ -169,8 +169,8 @@ The overall solution of PP-Human is as follows:
...
@@ -169,8 +169,8 @@ The overall solution of PP-Human is as follows:
- Use StrongBaseline（a multi-class model）to conduct attribute recognition, and the main attributes include age, gender, hats, eyes, clothing, and backpacks.
- Use StrongBaseline（a multi-class model）to conduct attribute recognition, and the main attributes include age, gender, hats, eyes, clothing, and backpacks.
- For details, please refer to [Attribute Recognition](docs/attribute_en.md)
- For details, please refer to [Attribute Recognition](docs/attribute_en.md)
### 5. Action Recognition
### 5. Falling Recognition
- Use PP-YOLOE + Bytetrack to track humans
- Use PP-YOLOE + Bytetrack to track humans
- Use HRNet for keypoint detection and get the information of the 17 key points in the human body
- Use HRNet for keypoint detection and get the information of the 17 key points in the human body
- According to the changes of the key points of the same person within 50 frames, judge whether the action made by the person within 50 frames is a fall with the help of ST-GCN
- According to the changes of the key points of the same person within 50 frames, judge whether the action made by the person within 50 frames is a fall with the help of ST-GCN
- For details, please refer to [Action Recognition](docs/action_en.md)
- For details, please refer to [Falling Recognition](docs/action_en.md)
Action Recognition is widely used in the intelligent community/smart city, and security monitoring. PP-Human provides the module of skeleton-based action recognition.
Falling Recognition is widely used in the intelligent community/smart city, and security monitoring. PP-Human provides the module of skeleton-based action recognition.
<divalign="center"><imgsrc="./images/action.gif"width='1000'/><center>Data source and copyright owner：Skyinfor
<divalign="center"><imgsrc="./images/action.gif"width='1000'/><center>Data source and copyright owner：Skyinfor
Technology. Thanks for the provision of actual scenario data, which are only
Technology. Thanks for the provision of actual scenario data, which are only
...
@@ -18,7 +18,7 @@ There are multiple available pretrained models including pedestrian detection/tr
...
@@ -18,7 +18,7 @@ There are multiple available pretrained models including pedestrian detection/tr
| Keypoint Detection | HRNet | AP: 87.1 | Single Person 2.9ms |[Link](https://bj.bcebos.com/v1/paddledet/models/pipeline/dark_hrnet_w32_256x192.pdparams) |[Link](https://bj.bcebos.com/v1/paddledet/models/pipeline/dark_hrnet_w32_256x192.zip) |
| Keypoint Detection | HRNet | AP: 87.1 | Single Person 2.9ms |[Link](https://bj.bcebos.com/v1/paddledet/models/pipeline/dark_hrnet_w32_256x192.pdparams) |[Link](https://bj.bcebos.com/v1/paddledet/models/pipeline/dark_hrnet_w32_256x192.zip) |
| Action Recognition | ST-GCN | Precision Rate: 96.43 | Single Person 2.7ms | - |[Link](https://bj.bcebos.com/v1/paddledet/models/pipeline/STGCN.zip) |
| Falling Recognition | ST-GCN | Precision Rate: 96.43 | Single Person 2.7ms | - |[Link](https://bj.bcebos.com/v1/paddledet/models/pipeline/STGCN.zip) |
Note:
Note:
...
@@ -35,7 +35,7 @@ Note:
...
@@ -35,7 +35,7 @@ Note:
Parameters related to action recognition in the [config file](../config/infer_cfg.yml) are as follow:
Parameters related to action recognition in the [config file](../config/infer_cfg.yml) are as follow:
```
```
ACTION:
FALLING:
model_dir: output_inference/STGCN # Path of the model
model_dir: output_inference/STGCN # Path of the model
batch_size: 1 # The size of the inference batch. The only avilable size for inference is 1.
batch_size: 1 # The size of the inference batch. The only avilable size for inference is 1.
max_frames: 50 # The number of frames of action segments. When frames of time-ordered skeleton keypoints of each pedestrian ID achieve the max value,the action type will be judged by the action recognition model. If the setting is the same as the training, there will be an ideal inference result.
max_frames: 50 # The number of frames of action segments. When frames of time-ordered skeleton keypoints of each pedestrian ID achieve the max value,the action type will be judged by the action recognition model. If the setting is the same as the training, there will be an ideal inference result.
- In ```./deploy/pphuman/config/infer_cfg.yml```, you can configurate different model paths，which is proper only if you match keypoint models and action recognition models with the fields of `KPT` and `ACTION`respectively, and modify the corresponding path of each field into the expected path.
- In ```./deploy/pphuman/config/infer_cfg.yml```, you can configurate different model paths，which is proper only if you match keypoint models and action recognition models with the fields of `KPT` and `FALLING`respectively, and modify the corresponding path of each field into the expected path.
- Add `--model_dir` in the command line to revise the model path：
- Add `--model_dir` in the command line to revise the model path：
4. The action recognition model uses [ST-GCN](https://arxiv.org/abs/1801.07455), and employ the [PaddleVideo](https://github.com/PaddlePaddle/PaddleVideo/blob/develop/docs/zh-CN/model_zoo/recognition/stgcn.md) toolkit to complete model training.
4. The action recognition model uses [ST-GCN](https://arxiv.org/abs/1801.07455), and employ the [PaddleVideo](https://github.com/PaddlePaddle/PaddleVideo/blob/develop/docs/zh-CN/model_zoo/recognition/stgcn.md) toolkit to complete model training.
## Custom Action Training
## Custom Falling Training
The pretrained models are provided and can be used directly, including pedestrian detection/ tracking, keypoint detection and fall recognition. If users need to train custom action or optimize the model performance, please refer the link below.
The pretrained models are provided and can be used directly, including pedestrian detection/ tracking, keypoint detection and fall recognition. If users need to train custom action or optimize the model performance, please refer the link below.
...
@@ -108,7 +108,7 @@ The pretrained models are provided and can be used directly, including pedestria
...
@@ -108,7 +108,7 @@ The pretrained models are provided and can be used directly, including pedestria
```
```
@inproceedings{stgcn2018aaai,
@inproceedings{stgcn2018aaai,
title = {Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition},
title = {Spatial Temporal Graph Convolutional Networks for Skeleton-Based Falling Recognition},
author = {Sijie Yan and Yuanjun Xiong and Dahua Lin},
author = {Sijie Yan and Yuanjun Xiong and Dahua Lin},
