Skip to content

P
PaddleDetection

PaddlePaddle / PaddleDetection
大约 2 年 前同步成功

通知 708
Star 11112
Fork 2696
P
PaddleDetection
“98057a361cfd4bbbfcf82e0580c0755fce5be8ed”上不存在“develop/doc/dev/use_eigen_en.html”
未验证 提交 fb6d68da 编写于 作者: F Feng Ni 提交者： GitHub
浏览文件
操作

[cherry-pick] Fix distill codes and docs (#7742) 

* fix distill codes and docs

* fix slim distill docs

* fix docs

* add details of docs, test=document_fix
上级 6ceed758
隐藏空白更改
内联 并排
Showing with 432 addition and 391 deletion
configs/ppyoloe/distill/README.md
浏览文件 @ fb6d68da
# PPYOLOE+ Distillation(PPYOLOE+ 蒸馏) # PPYOLOE+ Distillation(PPYOLOE+ 蒸馏)
PaddleDetection提供了对PPYOLOE+ 进行模型蒸馏的方案，结合了logits蒸馏和feature蒸馏。 PaddleDetection提供了对PPYOLOE+ 进行模型蒸馏的方案，结合了logits蒸馏和feature蒸馏。更多蒸馏方案可以查看[slim/distill](../../slim/distill/)。
## 模型库 ## 模型库
......
configs/slim/distill/README.md
浏览文件 @ fb6d68da
# Distillation(蒸馏) # Distillation(蒸馏)
## 内容
- [YOLOv3模型蒸馏](#YOLOv3模型蒸馏)
- [FGD模型蒸馏](#FGD模型蒸馏)
- [CWD模型蒸馏](#CWD模型蒸馏)
- [LD模型蒸馏](#LD模型蒸馏)
- [PPYOLOE模型蒸馏](#PPYOLOE模型蒸馏)
- [引用](#引用)
## YOLOv3模型蒸馏 ## YOLOv3模型蒸馏
以YOLOv3-MobileNetV1为例，使用YOLOv3-ResNet34作为蒸馏训练的teacher网络, 对YOLOv3-MobileNetV1结构的student网络进行蒸馏。 以YOLOv3-MobileNetV1为例，使用YOLOv3-ResNet34作为蒸馏训练的teacher网络, 对YOLOv3-MobileNetV1结构的student网络进行蒸馏。
...@@ -12,6 +20,25 @@ COCO数据集作为目标检测任务的训练目标难度更大，意味着teac ...@@ -12,6 +20,25 @@ COCO数据集作为目标检测任务的训练目标难度更大，意味着teac
| YOLOv3-MobileNetV1 | student | 608 | 270e | 29.4 | [config](../../yolov3/yolov3_mobilenet_v1_270e_coco.yml) | [download](https://paddledet.bj.bcebos.com/models/yolov3_mobilenet_v1_270e_coco.pdparams) | | YOLOv3-MobileNetV1 | student | 608 | 270e | 29.4 | [config](../../yolov3/yolov3_mobilenet_v1_270e_coco.yml) | [download](https://paddledet.bj.bcebos.com/models/yolov3_mobilenet_v1_270e_coco.pdparams) |
| YOLOv3-MobileNetV1 | distill | 608 | 270e | 31.0(+1.6) | [config](../../yolov3/yolov3_mobilenet_v1_270e_coco.yml),[slim_config](./yolov3_mobilenet_v1_coco_distill.yml) | [download](https://paddledet.bj.bcebos.com/models/slim/yolov3_mobilenet_v1_coco_distill.pdparams) | | YOLOv3-MobileNetV1 | distill | 608 | 270e | 31.0(+1.6) | [config](../../yolov3/yolov3_mobilenet_v1_270e_coco.yml),[slim_config](./yolov3_mobilenet_v1_coco_distill.yml) | [download](https://paddledet.bj.bcebos.com/models/slim/yolov3_mobilenet_v1_coco_distill.pdparams) |
<details>
<summary> 快速开始 </summary>
```shell
# 单卡训练(不推荐)
python tools/train.py -c configs/yolov3/yolov3_mobilenet_v1_270e_coco.yml --slim_config configs/slim/distill/yolov3_mobilenet_v1_coco_distill.yml
# 多卡训练
python -m paddle.distributed.launch --log_dir=logs/ --gpus 0,1,2,3,4,5,6,7 tools/train.py -c configs/yolov3/yolov3_mobilenet_v1_270e_coco.yml --slim_config configs/slim/distill/yolov3_mobilenet_v1_coco_distill.yml
# 评估
python tools/eval.py -c configs/yolov3/yolov3_mobilenet_v1_270e_coco.yml -o weights=https://paddledet.bj.bcebos.com/models/slim/yolov3_mobilenet_v1_coco_distill.pdparams
# 预测
python tools/infer.py -c configs/yolov3/yolov3_mobilenet_v1_270e_coco.yml -o weights=https://paddledet.bj.bcebos.com/models/slim/yolov3_mobilenet_v1_coco_distill.pdparams --infer_img=demo/000000014439_640x640.jpg
```
- `-c`: 指定模型配置文件，也是student配置文件。
- `--slim_config`: 指定压缩策略配置文件，也是teacher配置文件。
</details>
## FGD模型蒸馏 ## FGD模型蒸馏
...@@ -24,16 +51,24 @@ FGD全称为[Focal and Global Knowledge Distillation for Detectors](https://arxi ...@@ -24,16 +51,24 @@ FGD全称为[Focal and Global Knowledge Distillation for Detectors](https://arxi
| RetinaNet-ResNet50 | student | 1333x800 | 2x | 39.1 | [config](../../retinanet/retinanet_r50_fpn_2x_coco.yml) | [download](https://paddledet.bj.bcebos.com/models/retinanet_r50_fpn_2x_coco.pdparams) | | RetinaNet-ResNet50 | student | 1333x800 | 2x | 39.1 | [config](../../retinanet/retinanet_r50_fpn_2x_coco.yml) | [download](https://paddledet.bj.bcebos.com/models/retinanet_r50_fpn_2x_coco.pdparams) |
| RetinaNet-ResNet50 | FGD | 1333x800 | 2x | 40.8(+1.7) | [config](../../retinanet/retinanet_r50_fpn_2x_coco.yml),[slim_config](./retinanet_resnet101_coco_distill.yml) | [download](https://paddledet.bj.bcebos.com/models/retinanet_r101_distill_r50_2x_coco.pdparams) | | RetinaNet-ResNet50 | FGD | 1333x800 | 2x | 40.8(+1.7) | [config](../../retinanet/retinanet_r50_fpn_2x_coco.yml),[slim_config](./retinanet_resnet101_coco_distill.yml) | [download](https://paddledet.bj.bcebos.com/models/retinanet_r101_distill_r50_2x_coco.pdparams) |
<details>
<summary> 快速开始 </summary>
## LD模型蒸馏 ```shell
# 单卡训练(不推荐)
python tools/train.py -c configs/retinanet/retinanet_r50_fpn_2x_coco.yml --slim_config configs/slim/distill/retinanet_resnet101_coco_distill.yml
# 多卡训练
python -m paddle.distributed.launch --log_dir=logs/ --gpus 0,1,2,3,4,5,6,7 tools/train.py -c configs/retinanet/retinanet_r50_fpn_2x_coco.yml --slim_config configs/slim/distill/retinanet_resnet101_coco_distill.yml
# 评估
python tools/eval.py -c configs/retinanet/retinanet_r50_fpn_2x_coco.yml -o weights=https://paddledet.bj.bcebos.com/models/retinanet_r101_distill_r50_2x_coco.pdparams
# 预测
python tools/infer.py -c configs/retinanet/retinanet_r50_fpn_2x_coco.yml -o weights=https://paddledet.bj.bcebos.com/models/retinanet_r101_distill_r50_2x_coco.pdparams --infer_img=demo/000000014439_640x640.jpg
```
LD全称为[Localization Distillation for Dense Object Detection](https://arxiv.org/abs/2102.12252)，将回归框表示为概率分布，把分类任务的KD用在定位任务上，并且使用因地制宜、分而治之的策略，在不同的区域分别学习分类知识与定位知识。在PaddleDetection中，我们实现了LD算法，并基于GFL模型进行验证，实验结果如下： - `-c`: 指定模型配置文件，也是student配置文件。
- `--slim_config`: 指定压缩策略配置文件，也是teacher配置文件。
| 模型 | 方案 | 输入尺寸 | epochs | Box mAP | 配置文件 | 下载链接 | </details>
| ----------------- | ----------- | ------ | :----: | :-----------: | :--------------: | :------------: |
| GFL_ResNet101-vd| teacher | 1333x800 | 2x | 46.8 | [config](../../gfl/gfl_r101vd_fpn_mstrain_2x_coco.yml) | [download](https://paddledet.bj.bcebos.com/models/gfl_r101vd_fpn_mstrain_2x_coco.pdparams) |
| GFL_ResNet18-vd | student | 1333x800 | 1x | 36.6 | [config](../../gfl/gfl_r18vd_1x_coco.yml) | [download](https://paddledet.bj.bcebos.com/models/gfl_r18vd_1x_coco.pdparams) |
| GFL_ResNet18-vd | LD | 1333x800 | 1x | 38.2(+1.6) | [config](../../gfl/gfl_slim_ld_r18vd_1x_coco.yml),[slim_config](./gfl_ld_distill.yml) | [download](https://bj.bcebos.com/v1/paddledet/models/gfl_slim_ld_r18vd_1x_coco.pdparams) |
## CWD模型蒸馏 ## CWD模型蒸馏
...@@ -44,60 +79,104 @@ CWD全称为[Channel-wise Knowledge Distillation for Dense Prediction*](https:// ...@@ -44,60 +79,104 @@ CWD全称为[Channel-wise Knowledge Distillation for Dense Prediction*](https://
| ----------------- | ----------- | ------ | :----: | :-----------: | :--------------: | :------------: | | ----------------- | ----------- | ------ | :----: | :-----------: | :--------------: | :------------: |
| RetinaNet-ResNet101| teacher | 1333x800 | 2x | 40.6 | [config](../../retinanet/retinanet_r101_fpn_2x_coco.yml) | [download](https://paddledet.bj.bcebos.com/models/retinanet_r101_fpn_2x_coco.pdparams) | | RetinaNet-ResNet101| teacher | 1333x800 | 2x | 40.6 | [config](../../retinanet/retinanet_r101_fpn_2x_coco.yml) | [download](https://paddledet.bj.bcebos.com/models/retinanet_r101_fpn_2x_coco.pdparams) |
| RetinaNet-ResNet50 | student | 1333x800 | 2x | 39.1 | [config](../../retinanet/retinanet_r50_fpn_2x_coco.yml) | [download](https://paddledet.bj.bcebos.com/models/retinanet_r50_fpn_2x_coco.pdparams) | | RetinaNet-ResNet50 | student | 1333x800 | 2x | 39.1 | [config](../../retinanet/retinanet_r50_fpn_2x_coco.yml) | [download](https://paddledet.bj.bcebos.com/models/retinanet_r50_fpn_2x_coco.pdparams) |
| RetinaNet-ResNet50 | CWD | 1333x800 | 2x | 40.5(+1.4) | [config](../../retinanet/retinanet_r50_fpn_2x_coco_cwd.yml),[slim_config](./retinanet_resnet101_coco_distill_cwd.yml) | [download](https://paddledet.bj.bcebos.com/models/retinanet_r50_fpn_2x_coco_cwd.pdparams) | | RetinaNet-ResNet50 | CWD | 1333x800 | 2x | 40.5(+1.4) | [config](../../retinanet/retinanet_r50_fpn_2x_coco.yml),[slim_config](./retinanet_resnet101_coco_distill_cwd.yml) | [download](https://paddledet.bj.bcebos.com/models/retinanet_r50_fpn_2x_coco_cwd.pdparams) |
| GFL_ResNet101-vd| teacher | 1333x800 | 2x | 46.8 | [config](../../gfl/gfl_r101vd_fpn_mstrain_2x_coco.yml) | [download](https://paddledet.bj.bcebos.com/models/gfl_r101vd_fpn_mstrain_2x_coco.pdparams) | | GFL_ResNet101-vd| teacher | 1333x800 | 2x | 46.8 | [config](../../gfl/gfl_r101vd_fpn_mstrain_2x_coco.yml) | [download](https://paddledet.bj.bcebos.com/models/gfl_r101vd_fpn_mstrain_2x_coco.pdparams) |
| GFL_ResNet50 | student | 1333x800 | 1x | 41.0 | [config](../../gfl/gfl_r50_fpn_1x_coco.yml) | [download](https://paddledet.bj.bcebos.com/models/gfl_r50_fpn_1x_coco.pdparams) | | GFL_ResNet50 | student | 1333x800 | 1x | 41.0 | [config](../../gfl/gfl_r50_fpn_1x_coco.yml) | [download](https://paddledet.bj.bcebos.com/models/gfl_r50_fpn_1x_coco.pdparams) |
| GFL_ResNet50 | LD | 1333x800 | 2x | 44.0(+3.0) | [config](../../gfl/gfl_r50_fpn_2x_coco_cwd.yml),[slim_config](./gfl_r101vd_fpn_coco_distill_cwd.yml) | [download](https://bj.bcebos.com/v1/paddledet/models/gfl_r50_fpn_2x_coco_cwd.pdparams) | | GFL_ResNet50 | CWD | 1333x800 | 2x | 44.0(+3.0) | [config](../../gfl/gfl_r50_fpn_1x_coco.yml),[slim_config](./gfl_r101vd_fpn_coco_distill_cwd.yml) | [download](https://bj.bcebos.com/v1/paddledet/models/gfl_r50_fpn_2x_coco_cwd.pdparams) |
<details>
<summary> 快速开始 </summary>
```shell
# 单卡训练(不推荐)
python tools/train.py -c configs/retinanet/retinanet_r50_fpn_2x_coco.yml --slim_config configs/slim/distill/retinanet_resnet101_coco_distill_cwd.yml
# 多卡训练
python -m paddle.distributed.launch --log_dir=logs/ --gpus 0,1,2,3,4,5,6,7 tools/train.py -c configs/retinanet/retinanet_r50_fpn_2x_coco.yml --slim_config configs/slim/distill/retinanet_resnet101_coco_distill_cwd.yml
# 评估
python tools/eval.py -c configs/retinanet/retinanet_r50_fpn_2x_coco.yml -o weights=https://paddledet.bj.bcebos.com/models/retinanet_r50_fpn_2x_coco_cwd.pdparams
# 预测
python tools/infer.py -c configs/retinanet/retinanet_r50_fpn_2x_coco.yml -o weights=https://paddledet.bj.bcebos.com/models/retinanet_r50_fpn_2x_coco_cwd.pdparams --infer_img=demo/000000014439_640x640.jpg
# 单卡训练(不推荐)
python tools/train.py -c configs/gfl/gfl_r50_fpn_1x_coco.yml --slim_config configs/slim/distill/gfl_r101vd_fpn_coco_distill_cwd.yml
# 多卡训练
python -m paddle.distributed.launch --log_dir=logs/ --gpus 0,1,2,3,4,5,6,7 tools/train.py -c configs/gfl/gfl_r50_fpn_1x_coco.yml --slim_config configs/slim/distill/gfl_r101vd_fpn_coco_distill_cwd.yml
# 评估
python tools/eval.py -c configs/gfl/gfl_r50_fpn_1x_coco.yml -o weights=https://paddledet.bj.bcebos.com/models/gfl_r50_fpn_2x_coco_cwd.pdparams
# 预测
python tools/infer.py -c configs/gfl/gfl_r50_fpn_1x_coco.yml -o weights=https://paddledet.bj.bcebos.com/models/gfl_r50_fpn_2x_coco_cwd.pdparams --infer_img=demo/000000014439_640x640.jpg
```
- `-c`: 指定模型配置文件，也是student配置文件。
- `--slim_config`: 指定压缩策略配置文件，也是teacher配置文件。
## PPYOLOE+ 模型蒸馏 </details>
PaddleDetection提供了对PPYOLOE+ 进行模型蒸馏的方案，结合了logits蒸馏和feature蒸馏。
## LD模型蒸馏
LD全称为[Localization Distillation for Dense Object Detection](https://arxiv.org/abs/2102.12252)，将回归框表示为概率分布，把分类任务的KD用在定位任务上，并且使用因地制宜、分而治之的策略，在不同的区域分别学习分类知识与定位知识。在PaddleDetection中，我们实现了LD算法，并基于GFL模型进行验证，实验结果如下：
| 模型 | 方案 | 输入尺寸 | epochs | Box mAP | 配置文件 | 下载链接 | | 模型 | 方案 | 输入尺寸 | epochs | Box mAP | 配置文件 | 下载链接 |
| ----------------- | ----------- | ------ | :----: | :-----------: | :--------------: | :------------: | | ----------------- | ----------- | ------ | :----: | :-----------: | :--------------: | :------------: |
| PP-YOLOE+_x | teacher | 640 | 80e | 54.7 | [config](../../ppyoloe/ppyoloe_plus_crn_x_80e_coco.yml) | [model](https://bj.bcebos.com/v1/paddledet/models/ppyoloe_plus_crn_x_80e_coco.pdparams) | | GFL_ResNet101-vd| teacher | 1333x800 | 2x | 46.8 | [config](../../gfl/gfl_r101vd_fpn_mstrain_2x_coco.yml) | [download](https://paddledet.bj.bcebos.com/models/gfl_r101vd_fpn_mstrain_2x_coco.pdparams) |
| PP-YOLOE+_l | student | 640 | 80e | 52.9 | [config](../../ppyoloe/ppyoloe_plus_crn_l_80e_coco.yml) | [model](https://bj.bcebos.com/v1/paddledet/models/ppyoloe_plus_crn_l_80e_coco.pdparams) | | GFL_ResNet18-vd | student | 1333x800 | 1x | 36.6 | [config](../../gfl/gfl_r18vd_1x_coco.yml) | [download](https://paddledet.bj.bcebos.com/models/gfl_r18vd_1x_coco.pdparams) |
| PP-YOLOE+_l | distill | 640 | 80e | 53.9(+1.0) | [config](../../ppyoloe/distill/ppyoloe_plus_crn_l_80e_coco_distill.yml),[slim_config](./ppyoloe_plus_distill_x_distill_l.yml) | [model](https://bj.bcebos.com/v1/paddledet/models/ppyoloe_plus_crn_l_80e_coco_distill.pdparams) | | GFL_ResNet18-vd | LD | 1333x800 | 1x | 38.2(+1.6) | [config](../../gfl/gfl_slim_ld_r18vd_1x_coco.yml),[slim_config](./gfl_ld_distill.yml) | [download](https://bj.bcebos.com/v1/paddledet/models/gfl_slim_ld_r18vd_1x_coco.pdparams) |
| PP-YOLOE+_l | teacher | 640 | 80e | 52.9 | [config](../../ppyoloe/ppyoloe_plus_crn_l_80e_coco.yml) | [model](https://bj.bcebos.com/v1/paddledet/models/ppyoloe_plus_crn_l_80e_coco.pdparams) |
| PP-YOLOE+_m | student | 640 | 80e | 49.8 | [config](../../ppyoloe/ppyoloe_plus_crn_m_80e_coco.yml) | [model](https://bj.bcebos.com/v1/paddledet/models/ppyoloe_plus_crn_m_80e_coco.pdparams) |
| PP-YOLOE+_m | distill | 640 | 80e | 50.7(+0.9) | [config](../../ppyoloe/distill/ppyoloe_plus_crn_m_80e_coco_distill.yml),[slim_config](./ppyoloe_plus_distill_l_distill_m.yml) | [model](https://bj.bcebos.com/v1/paddledet/models/ppyoloe_plus_crn_m_80e_coco_distill.pdparams) |
## 快速开始 <details>
<summary> 快速开始 </summary>
### 训练
```shell ```shell
# 单卡 # 单卡训练(不推荐)
python tools/train.py -c configs/ppyoloe/distill/ppyoloe_plus_crn_l_80e_coco_distill.yml --slim_config configs/slim/distill/ppyoloe_plus_distill_x_distill_l.yml python tools/train.py -c configs/gfl/gfl_slim_ld_r18vd_1x_coco.yml --slim_config configs/slim/distill/gfl_ld_distill.yml
# 多卡 # 多卡训练
python -m paddle.distributed.launch --log_dir=ppyoloe_plus_distill_x_distill_l/ --gpus 0,1,2,3,4,5,6,7 tools/train.py -c configs/ppyoloe/distill/ppyoloe_plus_crn_l_80e_coco_distill.yml --slim_config configs/slim/distill/ppyoloe_plus_distill_x_distill_l.yml python -m paddle.distributed.launch --log_dir=logs/ --gpus 0,1,2,3,4,5,6,7 tools/train.py -c configs/gfl/gfl_slim_ld_r18vd_1x_coco.yml --slim_config configs/slim/distill/gfl_ld_distill.yml
# 评估
python tools/eval.py -c configs/gfl/gfl_slim_ld_r18vd_1x_coco.yml -o weights=https://paddledet.bj.bcebos.com/models/gfl_slim_ld_r18vd_1x_coco.pdparams
# 预测
python tools/infer.py -c configs/gfl/gfl_slim_ld_r18vd_1x_coco.yml -o weights=https://paddledet.bj.bcebos.com/models/gfl_slim_ld_r18vd_1x_coco.pdparams --infer_img=demo/000000014439_640x640.jpg
``` ```
- `-c`: 指定模型配置文件，也是student配置文件。 - `-c`: 指定模型配置文件，也是student配置文件。
- `--slim_config`: 指定压缩策略配置文件，也是teacher配置文件。 - `--slim_config`: 指定压缩策略配置文件，也是teacher配置文件。
### 评估 </details>
## PPYOLOE模型蒸馏
PaddleDetection提供了对PPYOLOE+ 进行模型蒸馏的方案，结合了logits蒸馏和feature蒸馏。
| 模型 | 方案 | 输入尺寸 | epochs | Box mAP | 配置文件 | 下载链接 |
| ----------------- | ----------- | ------ | :----: | :-----------: | :--------------: | :------------: |
| PP-YOLOE+_x | teacher | 640 | 80e | 54.7 | [config](../../ppyoloe/ppyoloe_plus_crn_x_80e_coco.yml) | [model](https://bj.bcebos.com/v1/paddledet/models/ppyoloe_plus_crn_x_80e_coco.pdparams) |
| PP-YOLOE+_l | student | 640 | 80e | 52.9 | [config](../../ppyoloe/ppyoloe_plus_crn_l_80e_coco.yml) | [model](https://bj.bcebos.com/v1/paddledet/models/ppyoloe_plus_crn_l_80e_coco.pdparams) |
| PP-YOLOE+_l | distill | 640 | 80e | **54.0(+1.1)** | [config](../../ppyoloe/distill/ppyoloe_plus_crn_l_80e_coco_distill.yml),[slim_config](./ppyoloe_plus_distill_x_distill_l.yml) | [model](https://bj.bcebos.com/v1/paddledet/models/ppyoloe_plus_crn_l_80e_coco_distill.pdparams) |
| PP-YOLOE+_l | teacher | 640 | 80e | 52.9 | [config](../../ppyoloe/ppyoloe_plus_crn_l_80e_coco.yml) | [model](https://bj.bcebos.com/v1/paddledet/models/ppyoloe_plus_crn_l_80e_coco.pdparams) |
| PP-YOLOE+_m | student | 640 | 80e | 49.8 | [config](../../ppyoloe/ppyoloe_plus_crn_m_80e_coco.yml) | [model](https://bj.bcebos.com/v1/paddledet/models/ppyoloe_plus_crn_m_80e_coco.pdparams) |
| PP-YOLOE+_m | distill | 640 | 80e | **51.0(+1.2)** | [config](../../ppyoloe/distill/ppyoloe_plus_crn_m_80e_coco_distill.yml),[slim_config](./ppyoloe_plus_distill_l_distill_m.yml) | [model](https://bj.bcebos.com/v1/paddledet/models/ppyoloe_plus_crn_m_80e_coco_distill.pdparams) |
<details>
<summary> 快速开始 </summary>
```shell ```shell
python tools/eval.py -c configs/ppyoloe/distill/ppyoloe_plus_crn_l_80e_coco_distill.yml -o weights=output/ppyoloe_plus_crn_l_80e_coco_distill/model_final.pdparams # 单卡训练(不推荐)
python tools/train.py -c configs/ppyoloe/distill/ppyoloe_plus_crn_l_80e_coco_distill.yml --slim_config configs/slim/distill/ppyoloe_plus_distill_x_distill_l.yml
# 多卡训练
python -m paddle.distributed.launch --log_dir=logs/ --gpus 0,1,2,3,4,5,6,7 tools/train.py -c configs/ppyoloe/distill/ppyoloe_plus_crn_l_80e_coco_distill.yml --slim_config configs/slim/distill/ppyoloe_plus_distill_x_distill_l.yml
# 评估
python tools/eval.py -c configs/ppyoloe/distill/ppyoloe_plus_crn_l_80e_coco_distill.yml -o weights=https://paddledet.bj.bcebos.com/models/ppyoloe_plus_crn_l_80e_coco_distill.pdparams
# 预测
python tools/infer.py -c configs/ppyoloe/distill/ppyoloe_plus_crn_l_80e_coco_distill.yml -o weights=https://paddledet.bj.bcebos.com/models/ppyoloe_plus_crn_l_80e_coco_distill.pdparams --infer_img=demo/000000014439_640x640.jpg
``` ```
- `-c`: 指定模型配置文件，也是student配置文件。 - `-c`: 指定模型配置文件，也是student配置文件。
- `--slim_config`: 指定压缩策略配置文件，也是teacher配置文件。 - `--slim_config`: 指定压缩策略配置文件，也是teacher配置文件。
- `-o weights`: 指定压缩算法训好的模型路径。
### 测试
```shell
python tools/infer.py -c configs/ppyoloe/distill/ppyoloe_plus_crn_l_80e_coco_distill.yml -o weights=output/ppyoloe_plus_crn_l_80e_coco_distill/model_final.pdparams --infer_img=demo/000000014439_640x640.jpg
```
- `-c`: 指定模型配置文件。 </details>
- `--slim_config`: 指定压缩策略配置文件。
- `-o weights`: 指定压缩算法训好的模型路径。
- `--infer_img`: 指定测试图像路径。
## Citations ## 引用
``` ```
@article{mehta2018object, @article{mehta2018object,
title={Object detection at 200 Frames Per Second}, title={Object detection at 200 Frames Per Second},
......
ppdet/modeling/heads/__init__.py
浏览文件 @ fb6d68da
...@@ -35,7 +35,6 @@ from . import retina_head ...@@ -35,7 +35,6 @@ from . import retina_head
from . import ppyoloe_head from . import ppyoloe_head
from . import fcosr_head from . import fcosr_head
from . import ppyoloe_r_head from . import ppyoloe_r_head
from . import ld_gfl_head
from . import yolof_head from . import yolof_head
from . import ppyoloe_contrast_head from . import ppyoloe_contrast_head
from . import centertrack_head from . import centertrack_head
...@@ -63,7 +62,6 @@ from .tood_head import * ...@@ -63,7 +62,6 @@ from .tood_head import *
from .retina_head import * from .retina_head import *
from .ppyoloe_head import * from .ppyoloe_head import *
from .fcosr_head import * from .fcosr_head import *
from .ld_gfl_head import *
from .ppyoloe_r_head import * from .ppyoloe_r_head import *
from .yolof_head import * from .yolof_head import *
from .ppyoloe_contrast_head import * from .ppyoloe_contrast_head import *
......
ppdet/modeling/heads/gfl_head.py
浏览文件 @ fb6d68da
...@@ -28,10 +28,11 @@ from paddle import ParamAttr ...@@ -28,10 +28,11 @@ from paddle import ParamAttr
from paddle.nn.initializer import Normal, Constant from paddle.nn.initializer import Normal, Constant
from ppdet.core.workspace import register from ppdet.core.workspace import register
from ppdet.modeling.layers import ConvNormLayer
from ppdet.modeling.bbox_utils import distance2bbox, bbox2distance, batch_distance2bbox from ppdet.modeling.bbox_utils import distance2bbox, bbox2distance, batch_distance2bbox
from ppdet.data.transform.atss_assigner import bbox_overlaps from ppdet.data.transform.atss_assigner import bbox_overlaps
__all__ = ['GFLHead', 'LDGFLHead']
class ScaleReg(nn.Layer): class ScaleReg(nn.Layer):
""" """
...@@ -437,3 +438,299 @@ class GFLHead(nn.Layer): ...@@ -437,3 +438,299 @@ class GFLHead(nn.Layer):
mlvl_scores = mlvl_scores.transpose([0, 2, 1]) mlvl_scores = mlvl_scores.transpose([0, 2, 1])
bbox_pred, bbox_num, _ = self.nms(bboxes, mlvl_scores) bbox_pred, bbox_num, _ = self.nms(bboxes, mlvl_scores)
return bbox_pred, bbox_num return bbox_pred, bbox_num
@register
class LDGFLHead(GFLHead):
"""
GFLHead for LD distill
Args:
conv_feat (object): Instance of 'FCOSFeat'
num_classes (int): Number of classes
fpn_stride (list): The stride of each FPN Layer
prior_prob (float): Used to set the bias init for the class prediction layer
loss_class (object): Instance of QualityFocalLoss.
loss_dfl (object): Instance of DistributionFocalLoss.
loss_bbox (object): Instance of bbox loss.
reg_max: Max value of integral set :math: `{0, ..., reg_max}`
n QFL setting. Default: 16.
"""
__inject__ = [
'conv_feat', 'dgqp_module', 'loss_class', 'loss_dfl', 'loss_bbox',
'loss_ld', 'loss_ld_vlr', 'loss_kd', 'nms'
]
__shared__ = ['num_classes']
def __init__(self,
conv_feat='FCOSFeat',
dgqp_module=None,
num_classes=80,
fpn_stride=[8, 16, 32, 64, 128],
prior_prob=0.01,
loss_class='QualityFocalLoss',
loss_dfl='DistributionFocalLoss',
loss_bbox='GIoULoss',
loss_ld='KnowledgeDistillationKLDivLoss',
loss_ld_vlr='KnowledgeDistillationKLDivLoss',
loss_kd='KnowledgeDistillationKLDivLoss',
reg_max=16,
feat_in_chan=256,
nms=None,
nms_pre=1000,
cell_offset=0):
super(LDGFLHead, self).__init__(
conv_feat=conv_feat,
dgqp_module=dgqp_module,
num_classes=num_classes,
fpn_stride=fpn_stride,
prior_prob=prior_prob,
loss_class=loss_class,
loss_dfl=loss_dfl,
loss_bbox=loss_bbox,
reg_max=reg_max,
feat_in_chan=feat_in_chan,
nms=nms,
nms_pre=nms_pre,
cell_offset=cell_offset)
self.loss_ld = loss_ld
self.loss_kd = loss_kd
self.loss_ld_vlr = loss_ld_vlr
def forward(self, fpn_feats):
assert len(fpn_feats) == len(
self.fpn_stride
), "The size of fpn_feats is not equal to size of fpn_stride"
cls_logits_list = []
bboxes_reg_list = []
for stride, scale_reg, fpn_feat in zip(self.fpn_stride,
self.scales_regs, fpn_feats):
conv_cls_feat, conv_reg_feat = self.conv_feat(fpn_feat)
cls_score = self.gfl_head_cls(conv_cls_feat)
bbox_pred = scale_reg(self.gfl_head_reg(conv_reg_feat))
if self.dgqp_module:
quality_score = self.dgqp_module(bbox_pred)
cls_score = F.sigmoid(cls_score) * quality_score
if not self.training:
cls_score = F.sigmoid(cls_score.transpose([0, 2, 3, 1]))
bbox_pred = bbox_pred.transpose([0, 2, 3, 1])
b, cell_h, cell_w, _ = paddle.shape(cls_score)
y, x = self.get_single_level_center_point(
[cell_h, cell_w], stride, cell_offset=self.cell_offset)
center_points = paddle.stack([x, y], axis=-1)
cls_score = cls_score.reshape([b, -1, self.cls_out_channels])
bbox_pred = self.distribution_project(bbox_pred) * stride
bbox_pred = bbox_pred.reshape([b, cell_h * cell_w, 4])
# NOTE: If keep_ratio=False and image shape value that
# multiples of 32, distance2bbox not set max_shapes parameter
# to speed up model prediction. If need to set max_shapes,
# please use inputs['im_shape'].
bbox_pred = batch_distance2bbox(
center_points, bbox_pred, max_shapes=None)
cls_logits_list.append(cls_score)
bboxes_reg_list.append(bbox_pred)
return (cls_logits_list, bboxes_reg_list)
def get_loss(self, gfl_head_outs, gt_meta, soft_label_list,
soft_targets_list):
cls_logits, bboxes_reg = gfl_head_outs
num_level_anchors = [
featmap.shape[-2] * featmap.shape[-1] for featmap in cls_logits
]
grid_cells_list = self._images_to_levels(gt_meta['grid_cells'],
num_level_anchors)
labels_list = self._images_to_levels(gt_meta['labels'],
num_level_anchors)
label_weights_list = self._images_to_levels(gt_meta['label_weights'],
num_level_anchors)
bbox_targets_list = self._images_to_levels(gt_meta['bbox_targets'],
num_level_anchors)
# vlr regions
vlr_regions_list = self._images_to_levels(gt_meta['vlr_regions'],
num_level_anchors)
num_total_pos = sum(gt_meta['pos_num'])
try:
paddle.distributed.all_reduce(num_total_pos)
num_total_pos = paddle.clip(
num_total_pos / paddle.distributed.get_world_size(), min=1.)
except:
num_total_pos = max(num_total_pos, 1)
loss_bbox_list, loss_dfl_list, loss_qfl_list, loss_ld_list, avg_factor = [], [], [], [], []
loss_ld_vlr_list, loss_kd_list = [], []
for cls_score, bbox_pred, grid_cells, labels, label_weights, bbox_targets, stride, soft_targets,\
soft_label, vlr_region in zip(
cls_logits, bboxes_reg, grid_cells_list, labels_list,
label_weights_list, bbox_targets_list, self.fpn_stride, soft_targets_list,
soft_label_list, vlr_regions_list):
grid_cells = grid_cells.reshape([-1, 4])
cls_score = cls_score.transpose([0, 2, 3, 1]).reshape(
[-1, self.cls_out_channels])
bbox_pred = bbox_pred.transpose([0, 2, 3, 1]).reshape(
[-1, 4 * (self.reg_max + 1)])
soft_targets = soft_targets.transpose([0, 2, 3, 1]).reshape(
[-1, 4 * (self.reg_max + 1)])
soft_label = soft_label.transpose([0, 2, 3, 1]).reshape(
[-1, self.cls_out_channels])
# feture im
# teacher_x = teacher_x.transpose([0, 2, 3, 1]).reshape([-1, 256])
# x = x.transpose([0, 2, 3, 1]).reshape([-1, 256])
bbox_targets = bbox_targets.reshape([-1, 4])
labels = labels.reshape([-1])
label_weights = label_weights.reshape([-1])
vlr_region = vlr_region.reshape([-1])
bg_class_ind = self.num_classes
pos_inds = paddle.nonzero(
paddle.logical_and((labels >= 0), (labels < bg_class_ind)),
as_tuple=False).squeeze(1)
score = np.zeros(labels.shape)
remain_inds = (vlr_region > 0).nonzero()
if len(pos_inds) > 0:
pos_bbox_targets = paddle.gather(bbox_targets, pos_inds, axis=0)
pos_bbox_pred = paddle.gather(bbox_pred, pos_inds, axis=0)
pos_grid_cells = paddle.gather(grid_cells, pos_inds, axis=0)
pos_grid_cell_centers = self._grid_cells_to_center(
pos_grid_cells) / stride
weight_targets = F.sigmoid(cls_score.detach())
weight_targets = paddle.gather(
weight_targets.max(axis=1, keepdim=True), pos_inds, axis=0)
pos_bbox_pred_corners = self.distribution_project(pos_bbox_pred)
pos_decode_bbox_pred = distance2bbox(pos_grid_cell_centers,
pos_bbox_pred_corners)
pos_decode_bbox_targets = pos_bbox_targets / stride
bbox_iou = bbox_overlaps(
pos_decode_bbox_pred.detach().numpy(),
pos_decode_bbox_targets.detach().numpy(),
is_aligned=True)
score[pos_inds.numpy()] = bbox_iou
pred_corners = pos_bbox_pred.reshape([-1, self.reg_max + 1])
pos_soft_targets = paddle.gather(soft_targets, pos_inds, axis=0)
soft_corners = pos_soft_targets.reshape([-1, self.reg_max + 1])
target_corners = bbox2distance(pos_grid_cell_centers,
pos_decode_bbox_targets,
self.reg_max).reshape([-1])
# regression loss
loss_bbox = paddle.sum(
self.loss_bbox(pos_decode_bbox_pred,
pos_decode_bbox_targets) * weight_targets)
# dfl loss
loss_dfl = self.loss_dfl(
pred_corners,
target_corners,
weight=weight_targets.expand([-1, 4]).reshape([-1]),
avg_factor=4.0)
# ld loss
loss_ld = self.loss_ld(
pred_corners,
soft_corners,
weight=weight_targets.expand([-1, 4]).reshape([-1]),
avg_factor=4.0)
loss_kd = self.loss_kd(
paddle.gather(
cls_score, pos_inds, axis=0),
paddle.gather(
soft_label, pos_inds, axis=0),
weight=paddle.gather(
label_weights, pos_inds, axis=0),
avg_factor=pos_inds.shape[0])
else:
loss_bbox = bbox_pred.sum() * 0
loss_dfl = bbox_pred.sum() * 0
loss_ld = bbox_pred.sum() * 0
loss_kd = bbox_pred.sum() * 0
weight_targets = paddle.to_tensor([0], dtype='float32')
if len(remain_inds) > 0:
neg_pred_corners = bbox_pred[remain_inds].reshape(
[-1, self.reg_max + 1])
neg_soft_corners = soft_targets[remain_inds].reshape(
[-1, self.reg_max + 1])
remain_targets = vlr_region[remain_inds]
loss_ld_vlr = self.loss_ld_vlr(
neg_pred_corners,
neg_soft_corners,
weight=remain_targets.expand([-1, 4]).reshape([-1]),
avg_factor=16.0)
else:
loss_ld_vlr = bbox_pred.sum() * 0
# qfl loss
score = paddle.to_tensor(score)
loss_qfl = self.loss_qfl(
cls_score, (labels, score),
weight=label_weights,
avg_factor=num_total_pos)
loss_bbox_list.append(loss_bbox)
loss_dfl_list.append(loss_dfl)
loss_qfl_list.append(loss_qfl)
loss_ld_list.append(loss_ld)
loss_ld_vlr_list.append(loss_ld_vlr)
loss_kd_list.append(loss_kd)
avg_factor.append(weight_targets.sum())
avg_factor = sum(avg_factor) # + 1e-6
try:
paddle.distributed.all_reduce(avg_factor)
avg_factor = paddle.clip(
avg_factor / paddle.distributed.get_world_size(), min=1)
except:
avg_factor = max(avg_factor.item(), 1)
if avg_factor <= 0:
loss_qfl = paddle.to_tensor(0, dtype='float32', stop_gradient=False)
loss_bbox = paddle.to_tensor(
0, dtype='float32', stop_gradient=False)
loss_dfl = paddle.to_tensor(0, dtype='float32', stop_gradient=False)
loss_ld = paddle.to_tensor(0, dtype='float32', stop_gradient=False)
loss_ld_vlr = paddle.to_tensor(
0, dtype='float32', stop_gradient=False)
loss_kd = paddle.to_tensor(0, dtype='float32', stop_gradient=False)
else:
losses_bbox = list(map(lambda x: x / avg_factor, loss_bbox_list))
losses_dfl = list(map(lambda x: x / avg_factor, loss_dfl_list))
loss_qfl = sum(loss_qfl_list)
loss_bbox = sum(losses_bbox)
loss_dfl = sum(losses_dfl)
loss_ld = sum(loss_ld_list)
loss_ld_vlr = sum(loss_ld_vlr_list)
loss_kd = sum(loss_kd_list)
loss_states = dict(
loss_qfl=loss_qfl,
loss_bbox=loss_bbox,
loss_dfl=loss_dfl,
loss_ld=loss_ld,
loss_ld_vlr=loss_ld_vlr,
loss_kd=loss_kd)
return loss_states
ppdet/modeling/heads/ld_gfl_head.py 已删除 100644 → 0
浏览文件 @ 6ceed758
# Copyright (c) 2021 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# The code is based on:
# https://github.com/open-mmlab/mmdetection/blob/master/mmdet/models/dense_heads/ld_head.py
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
import math
import numpy as np
import paddle
import paddle.nn as nn
import paddle.nn.functional as F
from paddle import ParamAttr
from paddle.nn.initializer import Normal, Constant
from ppdet.core.workspace import register, serializable
from ppdet.modeling.layers import ConvNormLayer
from ppdet.modeling.bbox_utils import distance2bbox, bbox2distance, batch_distance2bbox
from ppdet.data.transform.atss_assigner import bbox_overlaps
from .gfl_head import GFLHead
@register
class LDGFLHead(GFLHead):
"""
GFLHead for LD distill
Args:
conv_feat (object): Instance of 'FCOSFeat'
num_classes (int): Number of classes
fpn_stride (list): The stride of each FPN Layer
prior_prob (float): Used to set the bias init for the class prediction layer
loss_class (object): Instance of QualityFocalLoss.
loss_dfl (object): Instance of DistributionFocalLoss.
loss_bbox (object): Instance of bbox loss.
reg_max: Max value of integral set :math: `{0, ..., reg_max}`
n QFL setting. Default: 16.
"""
__inject__ = [
'conv_feat', 'dgqp_module', 'loss_class', 'loss_dfl', 'loss_bbox',
'loss_ld', 'loss_ld_vlr', 'loss_kd', 'nms'
]
__shared__ = ['num_classes']
def __init__(self,
conv_feat='FCOSFeat',
dgqp_module=None,
num_classes=80,
fpn_stride=[8, 16, 32, 64, 128],
prior_prob=0.01,
loss_class='QualityFocalLoss',
loss_dfl='DistributionFocalLoss',
loss_bbox='GIoULoss',
loss_ld='KnowledgeDistillationKLDivLoss',
loss_ld_vlr='KnowledgeDistillationKLDivLoss',
loss_kd='KnowledgeDistillationKLDivLoss',
reg_max=16,
feat_in_chan=256,
nms=None,
nms_pre=1000,
cell_offset=0):
super(LDGFLHead, self).__init__(
conv_feat=conv_feat,
dgqp_module=dgqp_module,
num_classes=num_classes,
fpn_stride=fpn_stride,
prior_prob=prior_prob,
loss_class=loss_class,
loss_dfl=loss_dfl,
loss_bbox=loss_bbox,
reg_max=reg_max,
feat_in_chan=feat_in_chan,
nms=nms,
nms_pre=nms_pre,
cell_offset=cell_offset)
self.loss_ld = loss_ld
self.loss_kd = loss_kd
self.loss_ld_vlr = loss_ld_vlr
def forward(self, fpn_feats):
assert len(fpn_feats) == len(
self.fpn_stride
), "The size of fpn_feats is not equal to size of fpn_stride"
cls_logits_list = []
bboxes_reg_list = []
for stride, scale_reg, fpn_feat in zip(self.fpn_stride,
self.scales_regs, fpn_feats):
conv_cls_feat, conv_reg_feat = self.conv_feat(fpn_feat)
cls_score = self.gfl_head_cls(conv_cls_feat)
bbox_pred = scale_reg(self.gfl_head_reg(conv_reg_feat))
if self.dgqp_module:
quality_score = self.dgqp_module(bbox_pred)
cls_score = F.sigmoid(cls_score) * quality_score
if not self.training:
cls_score = F.sigmoid(cls_score.transpose([0, 2, 3, 1]))
bbox_pred = bbox_pred.transpose([0, 2, 3, 1])
b, cell_h, cell_w, _ = paddle.shape(cls_score)
y, x = self.get_single_level_center_point(
[cell_h, cell_w], stride, cell_offset=self.cell_offset)
center_points = paddle.stack([x, y], axis=-1)
cls_score = cls_score.reshape([b, -1, self.cls_out_channels])
bbox_pred = self.distribution_project(bbox_pred) * stride
bbox_pred = bbox_pred.reshape([b, cell_h * cell_w, 4])
# NOTE: If keep_ratio=False and image shape value that
# multiples of 32, distance2bbox not set max_shapes parameter
# to speed up model prediction. If need to set max_shapes,
# please use inputs['im_shape'].
bbox_pred = batch_distance2bbox(
center_points, bbox_pred, max_shapes=None)
cls_logits_list.append(cls_score)
bboxes_reg_list.append(bbox_pred)
return (cls_logits_list, bboxes_reg_list)
def get_loss(self, gfl_head_outs, gt_meta, soft_label_list,
soft_targets_list):
cls_logits, bboxes_reg = gfl_head_outs
num_level_anchors = [
featmap.shape[-2] * featmap.shape[-1] for featmap in cls_logits
]
grid_cells_list = self._images_to_levels(gt_meta['grid_cells'],
num_level_anchors)
labels_list = self._images_to_levels(gt_meta['labels'],
num_level_anchors)
label_weights_list = self._images_to_levels(gt_meta['label_weights'],
num_level_anchors)
bbox_targets_list = self._images_to_levels(gt_meta['bbox_targets'],
num_level_anchors)
# vlr regions
vlr_regions_list = self._images_to_levels(gt_meta['vlr_regions'],
num_level_anchors)
num_total_pos = sum(gt_meta['pos_num'])
try:
paddle.distributed.all_reduce(num_total_pos)
num_total_pos = paddle.clip(
num_total_pos / paddle.distributed.get_world_size(), min=1.)
except:
num_total_pos = max(num_total_pos, 1)
loss_bbox_list, loss_dfl_list, loss_qfl_list, loss_ld_list, avg_factor = [], [], [], [], []
loss_ld_vlr_list, loss_kd_list = [], []
for cls_score, bbox_pred, grid_cells, labels, label_weights, bbox_targets, stride, soft_targets,\
soft_label, vlr_region in zip(
cls_logits, bboxes_reg, grid_cells_list, labels_list,
label_weights_list, bbox_targets_list, self.fpn_stride, soft_targets_list,
soft_label_list, vlr_regions_list):
grid_cells = grid_cells.reshape([-1, 4])
cls_score = cls_score.transpose([0, 2, 3, 1]).reshape(
[-1, self.cls_out_channels])
bbox_pred = bbox_pred.transpose([0, 2, 3, 1]).reshape(
[-1, 4 * (self.reg_max + 1)])
soft_targets = soft_targets.transpose([0, 2, 3, 1]).reshape(
[-1, 4 * (self.reg_max + 1)])
soft_label = soft_label.transpose([0, 2, 3, 1]).reshape(
[-1, self.cls_out_channels])
# feture im
# teacher_x = teacher_x.transpose([0, 2, 3, 1]).reshape([-1, 256])
# x = x.transpose([0, 2, 3, 1]).reshape([-1, 256])
bbox_targets = bbox_targets.reshape([-1, 4])
labels = labels.reshape([-1])
label_weights = label_weights.reshape([-1])
vlr_region = vlr_region.reshape([-1])
bg_class_ind = self.num_classes
pos_inds = paddle.nonzero(
paddle.logical_and((labels >= 0), (labels < bg_class_ind)),
as_tuple=False).squeeze(1)
score = np.zeros(labels.shape)
remain_inds = (vlr_region > 0).nonzero()
if len(pos_inds) > 0:
pos_bbox_targets = paddle.gather(bbox_targets, pos_inds, axis=0)
pos_bbox_pred = paddle.gather(bbox_pred, pos_inds, axis=0)
pos_grid_cells = paddle.gather(grid_cells, pos_inds, axis=0)
pos_grid_cell_centers = self._grid_cells_to_center(
pos_grid_cells) / stride
weight_targets = F.sigmoid(cls_score.detach())
weight_targets = paddle.gather(
weight_targets.max(axis=1, keepdim=True), pos_inds, axis=0)
pos_bbox_pred_corners = self.distribution_project(pos_bbox_pred)
pos_decode_bbox_pred = distance2bbox(pos_grid_cell_centers,
pos_bbox_pred_corners)
pos_decode_bbox_targets = pos_bbox_targets / stride
bbox_iou = bbox_overlaps(
pos_decode_bbox_pred.detach().numpy(),
pos_decode_bbox_targets.detach().numpy(),
is_aligned=True)
score[pos_inds.numpy()] = bbox_iou
pred_corners = pos_bbox_pred.reshape([-1, self.reg_max + 1])
pos_soft_targets = paddle.gather(soft_targets, pos_inds, axis=0)
soft_corners = pos_soft_targets.reshape([-1, self.reg_max + 1])
target_corners = bbox2distance(pos_grid_cell_centers,
pos_decode_bbox_targets,
self.reg_max).reshape([-1])
# regression loss
loss_bbox = paddle.sum(
self.loss_bbox(pos_decode_bbox_pred,
pos_decode_bbox_targets) * weight_targets)
# dfl loss
loss_dfl = self.loss_dfl(
pred_corners,
target_corners,
weight=weight_targets.expand([-1, 4]).reshape([-1]),
avg_factor=4.0)
# ld loss
loss_ld = self.loss_ld(
pred_corners,
soft_corners,
weight=weight_targets.expand([-1, 4]).reshape([-1]),
avg_factor=4.0)
loss_kd = self.loss_kd(
paddle.gather(
cls_score, pos_inds, axis=0),
paddle.gather(
soft_label, pos_inds, axis=0),
weight=paddle.gather(
label_weights, pos_inds, axis=0),
avg_factor=pos_inds.shape[0])
else:
loss_bbox = bbox_pred.sum() * 0
loss_dfl = bbox_pred.sum() * 0
loss_ld = bbox_pred.sum() * 0
loss_kd = bbox_pred.sum() * 0
weight_targets = paddle.to_tensor([0], dtype='float32')
if len(remain_inds) > 0:
neg_pred_corners = bbox_pred[remain_inds].reshape(
[-1, self.reg_max + 1])
neg_soft_corners = soft_targets[remain_inds].reshape(
[-1, self.reg_max + 1])
remain_targets = vlr_region[remain_inds]
loss_ld_vlr = self.loss_ld_vlr(
neg_pred_corners,
neg_soft_corners,
weight=remain_targets.expand([-1, 4]).reshape([-1]),
avg_factor=16.0)
else:
loss_ld_vlr = bbox_pred.sum() * 0
# qfl loss
score = paddle.to_tensor(score)
loss_qfl = self.loss_qfl(
cls_score, (labels, score),
weight=label_weights,
avg_factor=num_total_pos)
loss_bbox_list.append(loss_bbox)
loss_dfl_list.append(loss_dfl)
loss_qfl_list.append(loss_qfl)
loss_ld_list.append(loss_ld)
loss_ld_vlr_list.append(loss_ld_vlr)
loss_kd_list.append(loss_kd)
avg_factor.append(weight_targets.sum())
avg_factor = sum(avg_factor) # + 1e-6
try:
paddle.distributed.all_reduce(avg_factor)
avg_factor = paddle.clip(
avg_factor / paddle.distributed.get_world_size(), min=1)
except:
avg_factor = max(avg_factor.item(), 1)
if avg_factor <= 0:
loss_qfl = paddle.to_tensor(0, dtype='float32', stop_gradient=False)
loss_bbox = paddle.to_tensor(
0, dtype='float32', stop_gradient=False)
loss_dfl = paddle.to_tensor(0, dtype='float32', stop_gradient=False)
loss_ld = paddle.to_tensor(0, dtype='float32', stop_gradient=False)
loss_ld_vlr = paddle.to_tensor(
0, dtype='float32', stop_gradient=False)
loss_kd = paddle.to_tensor(0, dtype='float32', stop_gradient=False)
else:
losses_bbox = list(map(lambda x: x / avg_factor, loss_bbox_list))
losses_dfl = list(map(lambda x: x / avg_factor, loss_dfl_list))
loss_qfl = sum(loss_qfl_list)
loss_bbox = sum(losses_bbox)
loss_dfl = sum(losses_dfl)
loss_ld = sum(loss_ld_list)
loss_ld_vlr = sum(loss_ld_vlr_list)
loss_kd = sum(loss_kd_list)
loss_states = dict(
loss_qfl=loss_qfl,
loss_bbox=loss_bbox,
loss_dfl=loss_dfl,
loss_ld=loss_ld,
loss_ld_vlr=loss_ld_vlr,
loss_kd=loss_kd)
return loss_states
ppdet/slim/distill_loss.py
浏览文件 @ fb6d68da
...@@ -17,14 +17,12 @@ from __future__ import division ...@@ -17,14 +17,12 @@ from __future__ import division
from __future__ import print_function from __future__ import print_function
import math import math
import numpy as np
import paddle import paddle
import paddle.nn as nn import paddle.nn as nn
import paddle.nn.functional as F import paddle.nn.functional as F
from paddle import ParamAttr from paddle import ParamAttr
from ppdet.core.workspace import register, create from ppdet.core.workspace import register
from ppdet.modeling import ops from ppdet.modeling import ops
from ppdet.modeling.losses.iou_loss import GIoULoss from ppdet.modeling.losses.iou_loss import GIoULoss
from ppdet.utils.logger import setup_logger from ppdet.utils.logger import setup_logger
...@@ -456,7 +454,7 @@ class CWDFeatureLoss(nn.Layer): ...@@ -456,7 +454,7 @@ class CWDFeatureLoss(nn.Layer):
x /= tau x /= tau
return F.softmax(x, axis=1) return F.softmax(x, axis=1)
def forward(self, preds_s, preds_t, inputs): def forward(self, preds_s, preds_t, inputs=None):
assert preds_s.shape[-2:] == preds_t.shape[-2:] assert preds_s.shape[-2:] == preds_t.shape[-2:]
N, C, H, W = preds_s.shape N, C, H, W = preds_s.shape
eps = 1e-5 eps = 1e-5
...@@ -676,7 +674,7 @@ class FGDFeatureLoss(nn.Layer): ...@@ -676,7 +674,7 @@ class FGDFeatureLoss(nn.Layer):
wmin, wmax, hmin, hmax = [], [], [], [] wmin, wmax, hmin, hmax = [], [], [], []
if gt_bboxes.shape[1] == 0: if len(gt_bboxes) == 0:
loss = self.relation_loss(stu_feature, tea_feature) loss = self.relation_loss(stu_feature, tea_feature)
return self.lambda_fgd * loss return self.lambda_fgd * loss
...@@ -750,7 +748,7 @@ class PKDFeatureLoss(nn.Layer): ...@@ -750,7 +748,7 @@ class PKDFeatureLoss(nn.Layer):
self.loss_weight = loss_weight self.loss_weight = loss_weight
self.resize_stu = resize_stu self.resize_stu = resize_stu
def forward(self, stu_feature, tea_feature, inputs): def forward(self, stu_feature, tea_feature, inputs=None):
size_s, size_t = stu_feature.shape[2:], tea_feature.shape[2:] size_s, size_t = stu_feature.shape[2:], tea_feature.shape[2:]
if size_s[0] != size_t[0]: if size_s[0] != size_t[0]:
if self.resize_stu: if self.resize_stu:
...@@ -791,7 +789,7 @@ class MimicFeatureLoss(nn.Layer): ...@@ -791,7 +789,7 @@ class MimicFeatureLoss(nn.Layer):
else: else:
self.align = None self.align = None
def forward(self, stu_feature, tea_feature, inputs): def forward(self, stu_feature, tea_feature, inputs=None):
if self.align is not None: if self.align is not None:
stu_feature = self.align(stu_feature) stu_feature = self.align(stu_feature)
...@@ -839,7 +837,7 @@ class MGDFeatureLoss(nn.Layer): ...@@ -839,7 +837,7 @@ class MGDFeatureLoss(nn.Layer):
nn.Conv2D( nn.Conv2D(
teacher_channels, teacher_channels, kernel_size=3, padding=1)) teacher_channels, teacher_channels, kernel_size=3, padding=1))
def forward(self, stu_feature, tea_feature, inputs): def forward(self, stu_feature, tea_feature, inputs=None):
N = stu_feature.shape[0] N = stu_feature.shape[0]
if self.align is not None: if self.align is not None:
stu_feature = self.align(stu_feature) stu_feature = self.align(stu_feature)
......
ppdet/slim/distill_model.py
浏览文件 @ fb6d68da
...@@ -18,8 +18,6 @@ from __future__ import print_function ...@@ -18,8 +18,6 @@ from __future__ import print_function
import paddle import paddle
import paddle.nn as nn import paddle.nn as nn
import paddle.nn.functional as F
from paddle import ParamAttr
from ppdet.core.workspace import register, create, load_config from ppdet.core.workspace import register, create, load_config
from ppdet.utils.checkpoint import load_pretrain_weight from ppdet.utils.checkpoint import load_pretrain_weight
...@@ -206,13 +204,13 @@ class CWDDistillModel(DistillModel): ...@@ -206,13 +204,13 @@ class CWDDistillModel(DistillModel):
def get_loss_retinanet(self, stu_fea_list, tea_fea_list, inputs): def get_loss_retinanet(self, stu_fea_list, tea_fea_list, inputs):
loss = self.student_model.head(stu_fea_list, inputs) loss = self.student_model.head(stu_fea_list, inputs)
distill_loss = {} loss_dict = {}
for idx, k in enumerate(self.loss_dic): for idx, k in enumerate(self.distill_loss):
distill_loss[k] = self.loss_dic[k](stu_fea_list[idx], loss_dict[k] = self.distill_loss[k](stu_fea_list[idx],
tea_fea_list[idx]) tea_fea_list[idx])
loss['loss'] += distill_loss[k] loss['loss'] += loss_dict[k]
loss[k] = distill_loss[k] loss[k] = loss_dict[k]
return loss return loss
def get_loss_gfl(self, stu_fea_list, tea_fea_list, inputs): def get_loss_gfl(self, stu_fea_list, tea_fea_list, inputs):
...@@ -234,10 +232,11 @@ class CWDDistillModel(DistillModel): ...@@ -234,10 +232,11 @@ class CWDDistillModel(DistillModel):
s_cls_feat.append(cls_score) s_cls_feat.append(cls_score)
t_cls_feat.append(t_cls_score) t_cls_feat.append(t_cls_score)
for idx, k in enumerate(self.loss_dic): for idx, k in enumerate(self.distill_loss):
loss_dict[k] = self.loss_dic[k](s_cls_feat[idx], t_cls_feat[idx]) loss_dict[k] = self.distill_loss[k](s_cls_feat[idx],
feat_loss[f"neck_f_{idx}"] = self.loss_dic[k](stu_fea_list[idx], t_cls_feat[idx])
tea_fea_list[idx]) feat_loss[f"neck_f_{idx}"] = self.distill_loss[k](stu_fea_list[idx],
tea_fea_list[idx])
for k in feat_loss: for k in feat_loss:
loss['loss'] += feat_loss[k] loss['loss'] += feat_loss[k]
......
Markdown is supported
0% 或 .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑！
想要评论请 注册 或