未验证 提交 b21d1294 编写于 作者: G Guanghua Yu 提交者: GitHub

Update PicoDet docs (#4407)

上级 972ee207
# PicoDet
# PP-PicoDet
![](../../docs/images/picedet_demo.jpeg)
## Introduction
We developed a series of lightweight models, which named `PicoDet`. Because of its excellent performance, it is very suitable for deployment on mobile or CPU.
We developed a series of lightweight models, which named `PP-PicoDet`. Because of its excellent performance, it is very suitable for deployment on mobile or CPU.
- 🌟 Higher mAP: the **first** object detectors that surpass mAP(0.5:0.95) **30+** within 1M parameters when the input size is 416.
- 🚀 Faster latency: 129FPS on mobile ARM CPU.
- 🚀 Faster latency: 150FPS on mobile ARM CPU.
- 😊 Deploy friendly: support PaddleLite/MNN/NCNN/OpenVINO and provide C++/Python/Android implementation.
- 😍 Advanced algorithm: use the most advanced algorithms and innovate, such as ESNet, CSP-PAN, SimOTA with VFL, etc.
......@@ -17,40 +17,35 @@ We developed a series of lightweight models, which named `PicoDet`. Because of i
## Requirements
- PaddlePaddle >= 2.1.2
- PaddleSlim >= 2.1.1
## Benchmark
| Model | Input size | mAP<sup>val<br>0.5:0.95 | mAP<sup>val<br>0.5 | Params<br><sup>(M) | FLOPS<br><sup>(G) | Latency<sup>*<sup><br><sup>(ms) | Latency<sup>#<sup><br><sup>(ms) | download | config |
| Model | Input size | mAP<sup>val<br>0.5:0.95 | mAP<sup>val<br>0.5 | Params<br><sup>(M) | FLOPS<br><sup>(G) | Latency<sup><small>[NCNN](#latency)</small><sup><br><sup>(ms) | Latency<sup><small>[Lite](#latency)</small><sup><br><sup>(ms) | download | config |
| :-------- | :--------: | :---------------------: | :----------------: | :----------------: | :---------------: | :-----------------------------: | :-----------------------------: | :---------------------------------------------------------------------------------------------------------------------------------------------------------: | :------------------------------------------------------------------------------------------------------------ |
| PicoDet-S | 320*320 | 27.1 | 41.4 | 0.99 | 0.73 | 8.13 | **6.65** | [model](https://paddledet.bj.bcebos.com/models/picodet_s_320_coco.pdparams) &#124; [log](https://paddledet.bj.bcebos.com/logs/train_picodet_s_320_coco.log) | [config](https://github.com/PaddlePaddle/PaddleDetection/tree/develop/configs/picodet/picodet_s_320_coco.yml) |
| PicoDet-S | 416*416 | 30.6 | 45.5 | 0.99 | 1.24 | 12.37 | **9.82** | [model](https://paddledet.bj.bcebos.com/models/picodet_s_416_coco.pdparams) &#124; [log](https://paddledet.bj.bcebos.com/logs/train_picodet_s_416_coco.log) | [config](https://github.com/PaddlePaddle/PaddleDetection/tree/develop/configs/picodet/picodet_s_416_coco.yml) |
| PicoDet-M | 320*320 | 30.9 | 45.7 | 2.15 | 1.48 | 11.27 | **9.61** | [model](https://paddledet.bj.bcebos.com/models/picodet_m_320_coco.pdparams) &#124; [log](https://paddledet.bj.bcebos.com/logs/train_picodet_m_320_coco.log) | [config](https://github.com/PaddlePaddle/PaddleDetection/tree/develop/configs/picodet/picodet_m_320_coco.yml) |
| PicoDet-M | 416*416 | 34.3 | 49.8 | 2.15 | 2.50 | 17.39 | **15.88** | [model](https://paddledet.bj.bcebos.com/models/picodet_m_416_coco.pdparams) &#124; [log](https://paddledet.bj.bcebos.com/logs/train_picodet_m_416_coco.log) | [config](https://github.com/PaddlePaddle/PaddleDetection/tree/develop/configs/picodet/picodet_m_416_coco.yml) |
| PicoDet-L | 320*320 | 32.6 | 47.9 | 3.24 | 2.18 | 15.26 | **13.42** | [model](https://paddledet.bj.bcebos.com/models/picodet_l_320_coco.pdparams) &#124; [log](https://paddledet.bj.bcebos.com/logs/train_picodet_l_320_coco.log) | [config](https://github.com/PaddlePaddle/PaddleDetection/tree/develop/configs/picodet/picodet_l_320_coco.yml) |
| PicoDet-L | 416*416 | 35.9 | 51.7 | 3.24 | 3.69 | 23.36 | **21.85** | [model](https://paddledet.bj.bcebos.com/models/picodet_l_416_coco.pdparams) &#124; [log](https://paddledet.bj.bcebos.com/logs/train_picodet_l_416_coco.log) | [config](https://github.com/PaddlePaddle/PaddleDetection/tree/develop/configs/picodet/picodet_l_416_coco.yml) |
| PicoDet-L | 640*640 | 40.3 | 57.1 | 3.24 | 8.74 | 54.11 | **50.55** | [model](https://paddledet.bj.bcebos.com/models/picodet_l_640_coco.pdparams) &#124; [log](https://paddledet.bj.bcebos.com/logs/train_picodet_l_640_coco.log) | [config](https://github.com/PaddlePaddle/PaddleDetection/tree/develop/configs/picodet/picodet_l_640_coco.yml) |
| PicoDet-L | 320*320 | 32.9 | 48.2 | 3.30 | 2.23 | 15.26 | **13.42** | [model](https://paddledet.bj.bcebos.com/models/picodet_l_320_coco.pdparams) &#124; [log](https://paddledet.bj.bcebos.com/logs/train_picodet_l_320_coco.log) | [config](https://github.com/PaddlePaddle/PaddleDetection/tree/develop/configs/picodet/picodet_l_320_coco.yml) |
| PicoDet-L | 416*416 | 36.6 | 52.5 | 3.30 | 3.76 | 23.36 | **21.85** | [model](https://paddledet.bj.bcebos.com/models/picodet_l_416_coco.pdparams) &#124; [log](https://paddledet.bj.bcebos.com/logs/train_picodet_l_416_coco.log) | [config](https://github.com/PaddlePaddle/PaddleDetection/tree/develop/configs/picodet/picodet_l_416_coco.yml) |
| PicoDet-L | 640*640 | 40.9 | 57.6 | 3.30 | 8.91 | 54.11 | **50.55** | [model](https://paddledet.bj.bcebos.com/models/picodet_l_640_coco.pdparams) &#124; [log](https://paddledet.bj.bcebos.com/logs/train_picodet_l_640_coco.log) | [config](https://github.com/PaddlePaddle/PaddleDetection/tree/develop/configs/picodet/picodet_l_640_coco.yml) |
**Attetnion:** * represents NCNN inference speed, # represents Paddle-Lite inference speed.
#### More config
<details>
<summary>Table Notes (click to expand)</summary>
- PicoDet inference speed is tested on SD 888(1*X1+3*A78+4*A55) with 4 threads by arm8 and with FP16.
- PicoDet is trained on COCO train2017 dataset and evaluated on COCO val2017.
- PicoDet used 4 or 8 GPUs for training and all checkpoints are trained to 300 epochs with default settings and hyperparameters.
</details>
## More config
| Model | Input size | mAP<sup>val<br>0.5:0.95 | mAP<sup>val<br>0.5 | Params<br><sup>(M) | FLOPS<br><sup>(G) | Latency<sup>*<sup><br><sup>(ms) | Latency<sup>#<sup><br><sup>(ms) | download | config |
| Model | Input size | mAP<sup>val<br>0.5:0.95 | mAP<sup>val<br>0.5 | Params<br><sup>(M) | FLOPS<br><sup>(G) | Latency<sup><small>[NCNN](#latency)</small><sup><br><sup>(ms) | Latency<sup><small>[Lite](#latency)</small><sup><br><sup>(ms) | download | config |
| :--------------------------- | :--------: | :---------------------: | :----------------: | :----------------: | :---------------: | :-----------------------------: | :-----------------------------: | :-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------: | :------------------------------------------------------------------------------------------------------------------------------------------- |
| PicoDet-Shufflenetv2 1x | 416*416 | 30.0 | 44.6 | 1.17 | 1.53 | 15.06 | **10.63** | [model](https://paddledet.bj.bcebos.com/models/picodet_shufflenetv2_1x_416_coco.pdparams) &#124; [log](https://paddledet.bj.bcebos.com/logs/train_picodet_shufflenetv2_1x_416_coco.log) | [config](https://github.com/PaddlePaddle/PaddleDetection/tree/develop/configs/picodet/more_config/picodet_shufflenetv2_1x_416_coco.yml) |
| PicoDet-MobileNetv3-large 1x | 416*416 | 35.6 | 52.0 | 3.55 | 2.80 | 20.71 | **17.88** | [model](https://paddledet.bj.bcebos.com/models/picodet_mobilenetv3_large_1x_416_coco.pdparams) &#124; [log](https://paddledet.bj.bcebos.com/logs/train_picodet_mobilenetv3_large_1x_416_coco.log) | [config](https://github.com/PaddlePaddle/PaddleDetection/tree/develop/configs/picodet/more_config/picodet_mobilenetv3_large_1x_416_coco.yml) |
| PicoDet-LCNet 1.5x | 416*416 | 36.3 | 52.2 | 3.10 | 3.85 | 21.29 | **20.8** | [model](https://paddledet.bj.bcebos.com/models/picodet_lcnet_1_5x_416_coco.pdparams) &#124; [log](https://paddledet.bj.bcebos.com/logs/train_picodet_lcnet_1_5x_416_coco.log) | [config](https://github.com/PaddlePaddle/PaddleDetection/tree/develop/configs/picodet/more_config/picodet_lcnet_1_5x_416_coco.yml) |
**Attetnion:** * represents NCNN inference speed, # represents Paddle-Lite inference speed.
<details open>
<summary><b>Table Notes:</b></summary>
- <a name="latency">Latency:</a> All our models test on `Qualcomm Snapdragon 865(4\*A77+4\*A55)` with 4 threads by arm8 and with FP16. In the above table, test latency on [NCNN](https://github.com/Tencent/ncnn) and `Lite`->[Paddle-Lite](https://github.com/PaddlePaddle/Paddle-Lite). And testing latency with code:[MobileDetBenchmark](https://github.com/JiweiMaster/MobileDetBenchmark).
- PicoDet is trained on COCO train2017 dataset and evaluated on COCO val2017.
- PicoDet used 4 or 8 GPUs for training and all checkpoints are trained with default settings and hyperparameters.
</details>
## Deployment
......@@ -134,6 +129,20 @@ paddle2onnx --model_dir output_inference/picodet_s_320_coco/ \
### quantization
<details open>
<summary>Requirements:</summary>
- PaddlePaddle >= 2.2.0rc0
- PaddleSlim >= 2.2.0rc0
**Install:**
```shell
pip install paddleslim==2.2.0rc0
```
</details>
<details>
<summary>Quant aware (click to expand)</summary>
......@@ -156,6 +165,8 @@ python tools/post_quant.py -c configs/picodet/picodet_s_320_coco.yml \
--slim_config configs/slim/post_quant/picodet_s_ptq.yml
```
- Notes: Now the accuracy of post quant is abnormal and it is being debugged.
</details>
## Cite PiocDet
......
......@@ -20,16 +20,19 @@ ESNet:
act: hard_swish
channel_ratio: [0.875, 0.5, 1.0, 0.625, 0.5, 0.75, 0.625, 0.625, 0.5, 0.625, 1.0, 0.625, 0.75]
CSPPAN:
out_channels: 160
PicoHead:
conv_feat:
name: PicoFeat
feat_in: 128
feat_out: 128
feat_in: 160
feat_out: 160
num_convs: 4
num_fpn_stride: 4
norm_type: bn
share_cls_reg: False
feat_in_chan: 128
share_cls_reg: True
feat_in_chan: 160
TrainReader:
batch_size: 56
......
......@@ -20,16 +20,19 @@ ESNet:
act: hard_swish
channel_ratio: [0.875, 0.5, 1.0, 0.625, 0.5, 0.75, 0.625, 0.625, 0.5, 0.625, 1.0, 0.625, 0.75]
CSPPAN:
out_channels: 160
PicoHead:
conv_feat:
name: PicoFeat
feat_in: 128
feat_out: 128
feat_in: 160
feat_out: 160
num_convs: 4
num_fpn_stride: 4
norm_type: bn
share_cls_reg: False
feat_in_chan: 128
share_cls_reg: True
feat_in_chan: 160
TrainReader:
batch_size: 48
......
......@@ -20,16 +20,19 @@ ESNet:
act: hard_swish
channel_ratio: [0.875, 0.5, 1.0, 0.625, 0.5, 0.75, 0.625, 0.625, 0.5, 0.625, 1.0, 0.625, 0.75]
CSPPAN:
out_channels: 160
PicoHead:
conv_feat:
name: PicoFeat
feat_in: 128
feat_out: 128
feat_in: 160
feat_out: 160
num_convs: 4
num_fpn_stride: 4
norm_type: bn
share_cls_reg: False
feat_in_chan: 128
share_cls_reg: True
feat_in_chan: 160
TrainReader:
batch_size: 32
......
......@@ -24,7 +24,7 @@ Paddle Lite是飞桨轻量化推理引擎,为手机、IOT端提供高效推理
1. [**建议**]直接下载,预测库下载链接如下:
|平台|预测库下载链接|
|-|-|
|Android|[arm7](https://github.com/PaddlePaddle/Paddle-Lite/releases/download/v2.9.1/inference_lite_lib.android.armv7.clang.c++_static.with_extra.with_cv.tar.gz) / [arm8](https://github.com/PaddlePaddle/Paddle-Lite/releases/download/v2.9.1/inference_lite_lib.android.armv8.clang.c++_static.with_extra.with_cv.tar.gz)|
|Android|[arm7](https://github.com/PaddlePaddle/Paddle-Lite/releases/download/v2.10-rc/inference_lite_lib.android.armv7.clang.c++_static.with_extra.with_cv.tar.gz) / [arm8](https://github.com/PaddlePaddle/Paddle-Lite/releases/download/v2.10-rc/inference_lite_lib.android.armv8.clang.c++_static.with_extra.with_cv.tar.gz)|
**注意**:1. 如果是从 Paddle-Lite [官方文档](https://paddle-lite.readthedocs.io/zh/latest/quick_start/release_lib.html#android-toolchain-gcc)下载的预测库,注意选择`with_extra=ON,with_cv=ON`的下载链接。2. 目前只提供Android端demo,IOS端demo可以参考[Paddle-Lite IOS demo](https://github.com/PaddlePaddle/Paddle-Lite-Demo/tree/master/PaddleLite-ios-demo)
......@@ -77,13 +77,13 @@ Paddle-Lite 提供了多种策略来自动优化原始的模型,其中包括
**注意**:如果已经准备好了 `.nb` 结尾的模型文件,可以跳过此步骤。
#### 2.1.1 安装paddle_lite_opt工具
安装paddle_lite_opt工具有如下两种方法:
安装`paddle_lite_opt`工具有如下两种方法:
1. [**建议**]pip安装paddlelite并进行转换
```shell
pip install paddlelite
pip install paddlelite==2.10rc
```
2. 源码编译Paddle-Lite生成opt工具
2. 源码编译Paddle-Lite生成`paddle_lite_opt`工具
模型优化需要Paddle-Lite的`opt`可执行文件,可以通过编译Paddle-Lite源码获得,编译步骤如下:
```shell
......@@ -120,23 +120,24 @@ Paddle-Lite 提供了多种策略来自动优化原始的模型,其中包括
#### 2.1.3 转换示例
下面以PaddleDetection中的 `ppyolo` 模型为例,介绍使用`paddle_lite_opt`完成预训练模型到inference模型,再到Paddle-Lite优化模型的转换。
下面以PaddleDetection中的 `PicoDet` 模型为例,介绍使用`paddle_lite_opt`完成预训练模型到inference模型,再到Paddle-Lite优化模型的转换。
```shell
# 进入PaddleDetection根目录
cd PaddleDetection_root_path
# 将预训练模型导出为inference模型
python tools/export_model.py -c configs/ppyolo/ppyolo_tiny_650e_coco.yml -o weights=https://paddledet.bj.bcebos.com/models/ppyolo_tiny_650e_coco.pdparams
python tools/export_model.py -c configs/picodet/picodet_s_320_coco.yml \
-o weights=https://paddledet.bj.bcebos.com/models/picodet_s_320_coco.pdparams --output_dir=output_inference
# 将inference模型转化为Paddle-Lite优化模型
paddle_lite_opt --valid_targets=arm --model_file=output_inference/ppyolo_tiny_650e_coco/model.pdmodel --param_file=output_inference/ppyolo_tiny_650e_coco/model.pdiparams --optimize_out=output_inference/ppyolo_tiny_650e_coco/model
paddle_lite_opt --valid_targets=arm --model_file=output_inference/picodet_s_320_coco/model.pdmodel --param_file=output_inference/picodet_s_320_coco/model.pdiparams --optimize_out=output_inference/picodet_s_320_coco/model
# 将inference模型配置转化为json格式
python deploy/lite/convert_yml_to_json.py output_inference/ppyolo_tiny_650e_coco/infer_cfg.yml
python deploy/lite/convert_yml_to_json.py output_inference/picodet_s_320_coco/infer_cfg.yml
```
最终在output_inference/ppyolo_tiny_650e_coco/文件夹下生成`ppyolo_tiny.nb``infer_cfg.json`的文件。
最终在output_inference/picodet_s_320_coco/文件夹下生成`model.nb``infer_cfg.json`的文件。
**注意**`--optimize_out` 参数为优化后模型的保存路径,无需加后缀`.nb``--model_file` 参数为模型结构信息文件的路径,`--param_file` 参数为模型权重信息文件的路径,请注意文件名。
......@@ -185,7 +186,7 @@ cp -r Makefile src/ include/ *runtime_config.json $inference_lite_path/demo/cxx/
cd $inference_lite_path/demo/cxx/lite
# 执行编译,等待完成后得到可执行文件main
make ARM_ABI = arm8
make ARM_ABI=arm8
#如果是arm7,则执行 make ARM_ABI = arm7 (或者在Makefile中修改该项)
```
......@@ -200,10 +201,10 @@ mkdir model_det
mkdir model_keypoint
# 将优化后的模型、预测库文件、测试图像放置在预测库中的demo/cxx/detection文件夹下
cp {PadddleDetection_Root}/output_inference/ppyolo_tiny_650e_coco/model.nb ./model_det/
cp {PadddleDetection_Root}/output_inference/ppyolo_tiny_650e_coco/infer_cfg.json ./model_det/
cp {PadddleDetection_Root}/output_inference/picodet_s_320_coco/model.nb ./model_det/
cp {PadddleDetection_Root}/output_inference/picodet_s_320_coco/infer_cfg.json ./model_det/
# 如果需要关键点模型,则只需一下操作
# 如果需要关键点模型,则只需操作:
cp {PadddleDetection_Root}/output_inference/hrnet_w32_256x192/model.nb ./model_keypoint/
cp {PadddleDetection_Root}/output_inference/hrnet_w32_256x192/infer_cfg.json ./model_keypoint/
......@@ -219,10 +220,10 @@ cp ../../../cxx/lib/libpaddle_light_api_shared.so ./
```
deploy/
|-- model_det/
| |--mdoel.nb 优化后的检测模型文件
| |--model.nb 优化后的检测模型文件
| |--infer_cfg.json 检测器模型配置文件
|-- model_keypoint/
| |--mdoel.nb 优化后的关键点模型文件
| |--model.nb 优化后的关键点模型文件
| |--infer_cfg.json 关键点模型配置文件
|-- main 生成的移动端执行文件
|-- det_runtime_config.json 目标检测执行时参数配置文件
......@@ -240,7 +241,7 @@ deploy/
"threshold_det": 0.5, #检测器输出阈值
"image_file": "demo.jpg", #测试图片
"image_dir": "", #测试图片文件夹
"run_benchmark": false, #性能测试开关
"run_benchmark": true, #性能测试开关
"cpu_threads": 4 #线程数
}
```
......@@ -253,7 +254,7 @@ deploy/
"threshold_keypoint": 0.5, #关键点输出阈值
"image_file": "demo.jpg", #测试图片
"image_dir": "", #测试图片文件夹
"run_benchmark": false, #性能测试开关
"run_benchmark": true, #性能测试开关
"cpu_threads": 4 #线程数
}
```
......
......@@ -129,15 +129,11 @@ class SimOTAAssigner(object):
valid_mask[valid_mask.copy()] = fg_mask_inboxes
matched_gt_inds = matching_matrix[fg_mask_inboxes, :].argmax(1)
matched_pred_ious = (matching_matrix *
pairwise_ious.numpy()).sum(1)[fg_mask_inboxes]
matched_pred_ious = paddle.to_tensor(
matched_pred_ious, place=pairwise_ious.place)
matched_gt_inds = paddle.to_tensor(
matched_gt_inds, place=pairwise_ious.place)
return matched_pred_ious, matched_gt_inds, valid_mask
return matched_gt_inds, valid_mask
def get_sample(self, assign_gt_inds, gt_bboxes):
pos_inds = np.unique(np.nonzero(assign_gt_inds > 0)[0])
......@@ -231,7 +227,7 @@ class SimOTAAssigner(object):
paddle.logical_not(is_in_boxes_and_center).cast('float32') * INF
)
matched_pred_ious, matched_gt_inds, valid_mask = \
matched_gt_inds, valid_mask = \
self.dynamic_k_matching(
cost_matrix, pairwise_ious, num_gt, valid_mask.numpy())
......
Markdown is supported
0% .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑!
想要评论请 注册