提交 05588760 编写于 作者: Z zhiboniu 提交者: zhiboniu

update docs

上级 d5ae73f0
......@@ -67,7 +67,7 @@ PP-Human相关配置位于```deploy/pphuman/config/infer_cfg.yml```中,存放
|-------|-------|----------|-----|
| 图片 | 属性识别 | 目标检测 属性识别 | DET ATTR |
| 单镜头视频 | 属性识别 | 多目标跟踪 属性识别 | MOT ATTR |
| 单镜头视频 | 行为识别 | 多目标跟踪 关键点检测 行为识别 | MOT KPT FALLING |
| 单镜头视频 | 行为识别 | 多目标跟踪 关键点检测 摔倒识别 | MOT KPT SKELETON_ACTION |
例如基于视频输入的属性识别,任务类型包含多目标跟踪和属性识别,具体配置如下:
......@@ -80,15 +80,18 @@ MOT:
model_dir: output_inference/mot_ppyoloe_l_36e_pipeline/
tracker_config: deploy/pphuman/config/tracker_config.yml
batch_size: 1
basemode: "idbased"
ATTR:
model_dir: output_inference/strongbaseline_r50_30e_pa100k/
batch_size: 8
basemode: "idbased"
enable: False
```
**注意:**
- 如果用户仅需要实现不同任务,可以在命令行中加入 `--enable_attr=True``--enable_falling=True`即可,无需修改配置文件
- 如果用户需要实现不同任务,可以在配置文件对应enable选项设置为True, 其basemode类型会在代码中开启依赖的基础能力模型,比如跟踪模型。
- 如果用户仅需要修改模型文件路径,可以在命令行中加入 `--model_dir det=ppyoloe/` 即可,无需修改配置文件,详细说明参考下方参数说明文档
......@@ -105,14 +108,14 @@ python deploy/pphuman/pipeline.py --config deploy/pphuman/config/infer_cfg.yml -
# 命令行中指定的模型路径优先级高于配置文件
python deploy/pphuman/pipeline.py --config deploy/pphuman/config/infer_cfg.yml --video_file=test_video.mp4 --device=gpu --model_dir det=ppyoloe/ [--run_mode trt_fp16]
# 行人属性识别,指定配置文件路径和测试视频
python deploy/pphuman/pipeline.py --config deploy/pphuman/config/infer_cfg.yml --video_file=test_video.mp4 --device=gpu --enable_attr=True [--run_mode trt_fp16]
# 行人属性识别,指定配置文件路径和测试视频,在配置文件中ATTR部分开启enable选项。
python deploy/pphuman/pipeline.py --config deploy/pphuman/config/infer_cfg.yml --video_file=test_video.mp4 --device=gpu [--run_mode trt_fp16]
# 行为识别,指定配置文件路径和测试视频
python deploy/pphuman/pipeline.py --config deploy/pphuman/config/infer_cfg.yml --video_file=test_video.mp4 --device=gpu --enable_falling=True [--run_mode trt_fp16]
# 行为识别,指定配置文件路径和测试视频,在配置文件中对应行为识别功能开启enable选项。
python deploy/pphuman/pipeline.py --config deploy/pphuman/config/infer_cfg.yml --video_file=test_video.mp4 --device=gpu [--run_mode trt_fp16]
# 行人跨境跟踪,指定配置文件路径和测试视频列表文件夹
python deploy/pphuman/pipeline.py --config deploy/pphuman/config/infer_cfg.yml --video_dir=mtmct_dir/ --device=gpu --enable_mtmct=True [--run_mode trt_fp16]
# 行人跨境跟踪,指定配置文件路径和测试视频列表文件夹,在配置文件中REID部分开启enable选项。
python deploy/pphuman/pipeline.py --config deploy/pphuman/config/infer_cfg.yml --video_dir=mtmct_dir/ --device=gpu [--run_mode trt_fp16]
```
其他用法请参考[子任务文档](./docs)
......@@ -127,9 +130,6 @@ python deploy/pphuman/pipeline.py --config deploy/pphuman/config/infer_cfg.yml -
| --image_dir | Option | 要预测的图片文件夹路径 |
| --video_file | Option | 需要预测的视频 |
| --camera_id | Option | 用来预测的摄像头ID,默认为-1(表示不使用摄像头预测,可设置为:0 - (摄像头数目-1) ),预测过程中在可视化界面按`q`退出输出预测结果到:output/output.mp4|
| --enable_attr| Option | 是否进行属性识别, 默认为False,即不开启属性识别 |
| --enable_falling| Option | 是否进行行为识别,默认为False,即不开启行为识别 |
| --enable_mtmct| Option | 是否进行跨境头跟踪,默认为False,即不开启跨境头跟踪 |
| --device | Option | 运行时的设备,可选择`CPU/GPU/XPU`,默认为`CPU`|
| --output_dir | Option|可视化结果保存的根目录,默认为output/|
| --run_mode | Option |使用GPU时,默认为paddle, 可选(paddle/trt_fp32/trt_fp16/trt_int8)|
......
......@@ -68,7 +68,7 @@ Their correspondence is as follows:
|-------|-------|----------|-----|
| Image | Attribute Recognition | Object Detection Attribute Recognition | DET ATTR |
| Single-Camera Video | Attribute Recognition | Multi-Object Tracking Attribute Recognition | MOT ATTR |
| Single-Camera Video | Behavior Recognition | Multi-Object Tracking Keypoint Detection Falling Recognition | MOT KPT FALLING |
| Single-Camera Video | Behavior Recognition | Multi-Object Tracking Keypoint Detection Falling Recognition | MOT KPT SKELETON_ACTION |
For example, for the attribute recognition with the video input, its task types contain multi-object tracking and attribute recognition, and the config is:
......@@ -89,7 +89,7 @@ ATTR:
**Note: **
- For different tasks, users could add `--enable_attr=True` or `--enable_falling=True` in command line and do not need to set config file.
- For different tasks, users should set the "enable" to "True" in coresponding configs in the infer_cfg.yml file.
- if only need to change the model path, users could add `--model_dir det=ppyoloe/` in command line and do not need to set config file. For details info please refer to doc below.
......@@ -106,14 +106,14 @@ python deploy/pphuman/pipeline.py --config deploy/pphuman/config/infer_cfg.yml -
# The model path specified on the command line prioritizes over the config file
python deploy/pphuman/pipeline.py --config deploy/pphuman/config/infer_cfg.yml --video_file=test_video.mp4 --device=gpu --model_dir det=ppyoloe/ [--run_mode trt_fp16]
# Attribute recognition. Specify the config file path and test videos
python deploy/pphuman/pipeline.py --config deploy/pphuman/config/infer_cfg.yml --video_file=test_video.mp4 --device=gpu --enable_attr=True [--run_mode trt_fp16]
# Attribute recognition. Specify the config file path and test videos, and set the "enable" to "True" in ATTR of infer_cfg.yml
python deploy/pphuman/pipeline.py --config deploy/pphuman/config/infer_cfg.yml --video_file=test_video.mp4 --device=gpu [--run_mode trt_fp16]
# Falling Recognition. Specify the config file path and test videos
python deploy/pphuman/pipeline.py --config deploy/pphuman/config/infer_cfg.yml --video_file=test_video.mp4 --device=gpu --enable_falling=True [--run_mode trt_fp16]
# Action Recognition. Specify the config file path and test videos, and set the "enable" to "True" in corresponding action configs of infer_cfg.yml
python deploy/pphuman/pipeline.py --config deploy/pphuman/config/infer_cfg.yml --video_file=test_video.mp4 --device=gpu [--run_mode trt_fp16]
# Pedestrian Multi-Target Multi-Camera tracking. Specify the config file path and the directory of test videos
python deploy/pphuman/pipeline.py --config deploy/pphuman/config/infer_cfg.yml --video_dir=mtmct_dir/ --device=gpu --enable_mtmct=True [--run_mode trt_fp16]
# Pedestrian Multi-Target Multi-Camera tracking. Specify the config file path and the directory of test videos, and set the "enable" to "True" in REID in infer_cfg.yml
python deploy/pphuman/pipeline.py --config deploy/pphuman/config/infer_cfg.yml --video_dir=mtmct_dir/ --device=gpu [--run_mode trt_fp16]
```
......@@ -129,8 +129,6 @@ Other usage please refer to [sub-task docs](./docs)
| --image_dir | Option | The path of folders of to-be-predicted images |
| --video_file | Option | Videos to-be-predicted |
| --camera_id | Option | ID of the inference camera is -1 by default (means inference without cameras,and it can be set to 0 - (number of cameras-1)), and during the inference, click `q` on the visual interface to exit and output the inference result to output/output.mp4|
| --enable_attr| Option | Enable attribute recognition or not |
| --enable_falling| Option | Enable action recognition or not |
| --device | Option | During the operation,available devices are `CPU/GPU/XPU`,and the default is `CPU`|
| --output_dir | Option| The default root directory which stores the visualization result is output/|
| --run_mode | Option | When using GPU,the default one is paddle, and all these are available(paddle/trt_fp32/trt_fp16/trt_int8).|
......
......@@ -41,11 +41,13 @@ SKELETON_ACTION:
ID_BASED_DETACTION:
model_dir: output_inference/detector
batch_size: 1
basemode: "idbased"
enable: False
ID_BASED_CLSACTION:
model_dir: output_inference/classification
batch_size: 1
basemode: "idbased"
enable: False
REID:
......
......@@ -28,32 +28,32 @@
## 配置说明
[配置文件](../config/infer_cfg.yml)中与行为识别相关的参数如下:
```
FALLING:
SKELETON_ACTION:
model_dir: output_inference/STGCN # 模型所在路径
batch_size: 1 # 预测批大小。 当前仅支持为1进行推理
max_frames: 50 # 动作片段对应的帧数。在行人ID对应时序骨骼点结果时达到该帧数后,会通过行为识别模型判断该段序列的动作类型。与训练设置一致时效果最佳。
display_frames: 80 # 显示帧数。当预测结果为摔倒时,在对应人物ID中显示状态的持续时间。
coord_size: [384, 512] # 坐标统一缩放到的尺度大小。与训练设置一致时效果最佳。
basemode: "skeletonbased" #模型基于的路线分支,是否需要skeleton作为输入
enable: False #是否开启该功能
```
## 使用方法
1. 从上表链接中下载模型并解压到```./output_inference```路径下。
2. 目前行为识别模块仅支持视频输入,启动命令如下:
2. 目前行为识别模块仅支持视频输入,设置infer_cfg.yml中`SKELETON_ACTION`的enable: True, 然后启动命令如下:
```python
python deploy/pphuman/pipeline.py --config deploy/pphuman/config/infer_cfg.yml \
--video_file=test_video.mp4 \
--device=gpu \
--enable_falling=True
```
3. 若修改模型路径,有以下两种方式:
- ```./deploy/pphuman/config/infer_cfg.yml```下可以配置不同模型路径,关键点模型和行为识别模型分别对应`KPT`和`FALLING`字段,修改对应字段下的路径为实际期望的路径即可。
- ```./deploy/pphuman/config/infer_cfg.yml```下可以配置不同模型路径,关键点模型和摔倒行为识别模型分别对应`KPT`和`SKELETON_ACTION`字段,修改对应字段下的路径为实际期望的路径即可。
- 命令行中增加`--model_dir`修改模型路径:
```python
python deploy/pphuman/pipeline.py --config deploy/pphuman/config/infer_cfg.yml \
--video_file=test_video.mp4 \
--device=gpu \
--enable_falling=True \
--model_dir kpt=./dark_hrnet_w32_256x192 action=./STGCN
```
......@@ -78,7 +78,7 @@ python deploy/pphuman/pipeline.py --config deploy/pphuman/config/infer_cfg.yml \
## 参考文献
```
@inproceedings{stgcn2018aaai,
title = {Spatial Temporal Graph Convolutional Networks for Skeleton-Based Falling Recognition},
title = {Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition},
author = {Sijie Yan and Yuanjun Xiong and Dahua Lin},
booktitle = {AAAI},
year = {2018},
......
......@@ -35,12 +35,14 @@ Note:
Parameters related to action recognition in the [config file](../config/infer_cfg.yml) are as follow:
```
FALLING:
SKELETON_ACTION:
model_dir: output_inference/STGCN # Path of the model
batch_size: 1 # The size of the inference batch. The only avilable size for inference is 1.
max_frames: 50 # The number of frames of action segments. When frames of time-ordered skeleton keypoints of each pedestrian ID achieve the max value,the action type will be judged by the action recognition model. If the setting is the same as the training, there will be an ideal inference result.
display_frames: 80 # The number of display frames. When the inferred action type is falling down, the time length of the act will be displayed in the ID.
coord_size: [384, 512] # The unified size of the coordinate, which is the best when it is the same as the training setting.
basemode: "skeletonbased" #the models which is based on,whether we need the skeleton model.
enable: False #whether to enable this function
```
......@@ -50,18 +52,17 @@ FALLING:
- Download models from the links of the above table and unzip them to ```./output_inference```.
- Now the only available input is the video input in the action recognition module. The start command is:
- Now the only available input is the video input in the action recognition module. set the "enable: True" in SKELETON_ACTION of infer_cfg.yml. And then run the command:
```python
python deploy/pphuman/pipeline.py --config deploy/pphuman/config/infer_cfg.yml \
--video_file=test_video.mp4 \
--device=gpu \
--enable_falling=True
--device=gpu
```
- There are two ways to modify the model path:
- In ```./deploy/pphuman/config/infer_cfg.yml```, you can configurate different model paths,which is proper only if you match keypoint models and action recognition models with the fields of `KPT` and `FALLING`respectively, and modify the corresponding path of each field into the expected path.
- In ```./deploy/pphuman/config/infer_cfg.yml```, you can configurate different model paths,which is proper only if you match keypoint models and action recognition models with the fields of `KPT` and `SKELETON_ACTION` respectively, and modify the corresponding path of each field into the expected path.
- Add `--model_dir` in the command line to revise the model path:
......@@ -69,7 +70,6 @@ FALLING:
python deploy/pphuman/pipeline.py --config deploy/pphuman/config/infer_cfg.yml \
--video_file=test_video.mp4 \
--device=gpu \
--enable_falling=True \
--model_dir kpt=./dark_hrnet_w32_256x192 action=./STGCN
```
......@@ -108,7 +108,7 @@ The pretrained models are provided and can be used directly, including pedestria
```
@inproceedings{stgcn2018aaai,
title = {Spatial Temporal Graph Convolutional Networks for Skeleton-Based Falling Recognition},
title = {Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition},
author = {Sijie Yan and Yuanjun Xiong and Dahua Lin},
booktitle = {AAAI},
year = {2018},
......
......@@ -9,9 +9,9 @@ PP-Human跨镜头跟踪模块主要目的在于提供一套简洁、高效的跨
1. 下载模型 [REID模型](https://bj.bcebos.com/v1/paddledet/models/pipeline/reid_model.zip) 并解压到```./output_inference```路径下, MOT模型请参考[mot说明](./mot.md)文件下载。
2. 跨镜头跟踪模式下,要求输入的多个视频放在同一目录下,命令如下:
2. 跨镜头跟踪模式下,要求输入的多个视频放在同一目录下,同时开启infer_cfg.yml 中的REID选择中的enable=True, 命令如下:
```python
python3 deploy/pphuman/pipeline.py --config deploy/pphuman/config/infer_cfg.yml --video_dir=[your_video_file_directory] --device=gpu --enable_mtmct=True
python3 deploy/pphuman/pipeline.py --config deploy/pphuman/config/infer_cfg.yml --video_dir=[your_video_file_directory] --device=gpu
```
3. 相关配置在`./deploy/pphuman/config/infer_cfg.yml`文件中修改:
......
......@@ -9,7 +9,7 @@ The MTMCT module of PP-Human aims to provide a multi-target multi-camera pipleli
1. Download [REID model](https://bj.bcebos.com/v1/paddledet/models/pipeline/reid_model.zip) and unzip it to ```./output_inference```. For the MOT model, please refer to [mot description](./mot.md).
2. In the MTMCT mode, input videos are required to be put in the same directory. The command line is:
2. In the MTMCT mode, input videos are required to be put in the same directory. set the REID "enable: True" in the infer_cfg.yml. The command line is:
```python
python3 deploy/pphuman/pipeline.py --config deploy/pphuman/config/infer_cfg.yml --video_dir=[your_video_file_directory] --device=gpu
```
......
......@@ -361,6 +361,18 @@ class PipePredictor(object):
model_dir, device, run_mode, batch_size, trt_min_shape,
trt_max_shape, trt_opt_shape, trt_calib_mode, cpu_threads,
enable_mkldnn)
if self.with_idbased_detaction:
idbased_detaction_cfg = self.cfg['SKELETON_ACTION']
idbased_detaction_model_dir = idbased_detaction_cfg['model_dir']
idbased_detaction_batch_size = idbased_detaction_cfg[
'batch_size']
# IDBasedDetActionRecognizer = IDBasedDetActionRecognizer()
if self.with_idbased_clsaction:
idbased_clsaction_cfg = self.cfg['SKELETON_ACTION']
idbased_clsaction_model_dir = idbased_clsaction_cfg['model_dir']
idbased_clsaction_batch_size = idbased_clsaction_cfg[
'batch_size']
# IDBasedDetActionRecognizer = IDBasedClsActionRecognizer()
if self.with_skeleton_action:
skeleton_action_cfg = self.cfg['SKELETON_ACTION']
skeleton_action_model_dir = skeleton_action_cfg['model_dir']
......@@ -588,25 +600,23 @@ class PipePredictor(object):
pass
if self.with_skeleton_action:
if self.modebase["skeletonbased"]:
if frame_id > self.warmup_frame:
self.pipe_timer.module_time['kpt'].start()
kpt_pred = self.kpt_predictor.predict_image(
crop_input, visual=False)
keypoint_vector, score_vector = translate_to_ori_images(
kpt_pred, np.array(new_bboxes))
kpt_res = {}
kpt_res['keypoint'] = [
keypoint_vector.tolist(), score_vector.tolist()
] if len(keypoint_vector) > 0 else [[], []]
kpt_res['bbox'] = ori_bboxes
if frame_id > self.warmup_frame:
self.pipe_timer.module_time['kpt'].end()
if frame_id > self.warmup_frame:
self.pipe_timer.module_time['kpt'].start()
kpt_pred = self.kpt_predictor.predict_image(
crop_input, visual=False)
keypoint_vector, score_vector = translate_to_ori_images(
kpt_pred, np.array(new_bboxes))
kpt_res = {}
kpt_res['keypoint'] = [
keypoint_vector.tolist(), score_vector.tolist()
] if len(keypoint_vector) > 0 else [[], []]
kpt_res['bbox'] = ori_bboxes
if frame_id > self.warmup_frame:
self.pipe_timer.module_time['kpt'].end()
self.pipeline_res.update(kpt_res, 'kpt')
self.pipeline_res.update(kpt_res, 'kpt')
self.kpt_buff.update(kpt_res,
mot_res) # collect kpt output
self.kpt_buff.update(kpt_res, mot_res) # collect kpt output
state = self.kpt_buff.get_state(
) # whether frame num is enough or lost tracker
......
......@@ -33,7 +33,7 @@ from benchmark_utils import PaddleInferBenchmark
from infer import Detector, print_arguments
class FallingRecognizer(Detector):
class SkeletonActionRecognizer(Detector):
"""
Args:
model_dir (str): root path of model.pdiparams, model.pdmodel and infer_cfg.yml
......@@ -67,8 +67,8 @@ class FallingRecognizer(Detector):
threshold=0.5,
window_size=100,
random_pad=False):
assert batch_size == 1, "FallingRecognizer only support batch_size=1 now."
super(FallingRecognizer, self).__init__(
assert batch_size == 1, "SkeletonActionRecognizer only support batch_size=1 now."
super(SkeletonActionRecognizer, self).__init__(
model_dir=model_dir,
device=device,
run_mode=run_mode,
......@@ -264,7 +264,7 @@ def get_test_skeletons(input_file):
def main():
detector = FallingRecognizer(
detector = SkeletonActionRecognizer(
FLAGS.model_dir,
device=FLAGS.device,
run_mode=FLAGS.run_mode,
......@@ -305,7 +305,7 @@ def main():
}
det_log = PaddleInferBenchmark(detector.config, model_info, data_info,
perf_info, mems)
det_log('Falling')
det_log('SkeletonAction')
if __name__ == '__main__':
......
......@@ -68,7 +68,7 @@ class KeyPointBuff(object):
def get_collected_keypoint(self):
"""
Output (List): List of keypoint results for Falling Recognition task, where
Output (List): List of keypoint results for Skeletonbased Recognition task, where
the format of each element is [tracker_id, KeyPointSequence of tracker_id]
"""
output = []
......@@ -80,7 +80,7 @@ class KeyPointBuff(object):
return output
class FallingVisualHelper(object):
class SkeletonActionVisualHelper(object):
def __init__(self, frame_life=20):
self.frame_life = frame_life
self.action_history = {}
......
Markdown is supported
0% .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑!
想要评论请 注册