diff --git a/README.md b/README.md
index bab001a2ed01734555271d3159ba68fb08c615b6..7d35d8d9194b3fb32b0086c3769a21119aafb70c 100644
--- a/README.md
+++ b/README.md
@@ -29,17 +29,17 @@ PaddleDetection的目的是为工业界和学术界提供丰富、易用的目
支持的模型结构:
-| | ResNet | ResNet-vd [1](#vd) | ResNeXt-vd | SENet | MobileNet | DarkNet | VGG |
-|--------------------|:------:|------------------------------:|:----------:|:-----:|:---------:|:-------:|:---:|
-| Faster R-CNN | ✓ | ✓ | x | ✓ | ✗ | ✗ | ✗ |
-| Faster R-CNN + FPN | ✓ | ✓ | ✓ | ✓ | ✗ | ✗ | ✗ |
-| Mask R-CNN | ✓ | ✓ | x | ✓ | ✗ | ✗ | ✗ |
-| Mask R-CNN + FPN | ✓ | ✓ | ✓ | ✓ | ✗ | ✗ | ✗ |
-| Cascade Faster-CNN | ✓ | ✓ | ✓ | ✗ | ✗ | ✗ | ✗ |
-| Cascade Mask-CNN | ✓ | ✗ | ✗ | ✓ | ✗ | ✗ | ✗ |
-| RetinaNet | ✓ | ✗ | ✓ | ✗ | ✗ | ✗ | ✗ |
-| YOLOv3 | ✓ | ✗ | ✗ | ✗ | ✓ | ✓ | ✗ |
-| SSD | ✗ | ✗ | ✗ | ✗ | ✓ | ✗ | ✓ |
+| | ResNet | ResNet-vd [1](#vd) | ResNeXt-vd | SENet | MobileNet | DarkNet | VGG | HRNet | Res2Net |
+|--------------------|:------:|------------------------------:|:----------:|:-----:|:---------:|:-------:|:---:|:-----:| :--: |
+| Faster R-CNN | ✓ | ✓ | x | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ |
+| Faster R-CNN + FPN | ✓ | ✓ | ✓ | ✓ | ✗ | ✗ | ✗ | ✓ | ✓ |
+| Mask R-CNN | ✓ | ✓ | x | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ |
+| Mask R-CNN + FPN | ✓ | ✓ | ✓ | ✓ | ✗ | ✗ | ✗ | ✗ | ✓ |
+| Cascade Faster-RCNN | ✓ | ✓ | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ |
+| Cascade Mask-RCNN | ✓ | ✗ | ✗ | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ |
+| RetinaNet | ✓ | ✗ | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ |
+| YOLOv3 | ✓ | ✗ | ✗ | ✗ | ✓ | ✓ | ✗ | ✗ | ✗ |
+| SSD | ✗ | ✗ | ✗ | ✗ | ✓ | ✗ | ✓ | ✗ | ✗ |
[1] [ResNet-vd](https://arxiv.org/pdf/1812.01187) 模型提供了较大的精度提高和较少的性能损失。
@@ -91,6 +91,11 @@ PaddleDetection的目的是为工业界和学术界提供丰富、易用的目
## 版本更新
+### 12/2019
+- 增加Res2Net模型。
+- 增加HRNet模型。
+
+
### 21/11/2019
- 增加CascadeClsAware RCNN模型。
- 增加CBNet,ResNet200和Non-local模型。
@@ -131,3 +136,4 @@ PaddleDetection的目的是为工业界和学术界提供丰富、易用的目
## 如何贡献代码
我们非常欢迎你可以为PaddleDetection提供代码,也十分感谢你的反馈。
+ ✗ |
diff --git a/README_en.md b/README_en.md
index e083193dd21536a315843220a854bf09baac64a3..431ba164a56c93486f5971fce1d25017e92c587e 100644
--- a/README_en.md
+++ b/README_en.md
@@ -39,17 +39,17 @@ multi-GPU training.
Supported Architectures:
-| | ResNet | ResNet-vd [1](#vd) | ResNeXt-vd | SENet | MobileNet | DarkNet | VGG |
-| ------------------- | :----: | ----------------------------: | :--------: | :---: | :-------: | :-----: | :--: |
-| Faster R-CNN | ✓ | ✓ | x | ✓ | ✗ | ✗ | ✗ |
-| Faster R-CNN + FPN | ✓ | ✓ | ✓ | ✓ | ✗ | ✗ | ✗ |
-| Mask R-CNN | ✓ | ✓ | x | ✓ | ✗ | ✗ | ✗ |
-| Mask R-CNN + FPN | ✓ | ✓ | ✓ | ✓ | ✗ | ✗ | ✗ |
-| Cascade Faster-RCNN | ✓ | ✓ | ✓ | ✗ | ✗ | ✗ | ✗ |
-| Cascade Mask-RCNN | ✓ | ✗ | ✗ | ✓ | ✗ | ✗ | ✗ |
-| RetinaNet | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ |
-| YOLOv3 | ✓ | ✗ | ✗ | ✗ | ✓ | ✓ | ✗ |
-| SSD | ✗ | ✗ | ✗ | ✗ | ✓ | ✗ | ✓ |
+| | ResNet | ResNet-vd [1](#vd) | ResNeXt-vd | SENet | MobileNet | DarkNet | VGG | HRNet | Res2Net |
+| ------------------- | :----: | ----------------------------: | :--------: | :---: | :-------: | :-----: | :--: | :--: | :--: |
+| Faster R-CNN | ✓ | ✓ | x | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ |
+| Faster R-CNN + FPN | ✓ | ✓ | ✓ | ✓ | ✗ | ✗ | ✗ | ✓ | ✓ |
+| Mask R-CNN | ✓ | ✓ | x | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ |
+| Mask R-CNN + FPN | ✓ | ✓ | ✓ | ✓ | ✗ | ✗ | ✗ | ✗ | ✓ |
+| Cascade Faster-RCNN | ✓ | ✓ | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ |
+| Cascade Mask-RCNN | ✓ | ✗ | ✗ | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ |
+| RetinaNet | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ |
+| YOLOv3 | ✓ | ✗ | ✗ | ✗ | ✓ | ✓ | ✗ | ✗ | ✗ |
+| SSD | ✗ | ✗ | ✗ | ✗ | ✓ | ✗ | ✓ | ✗ | ✗ |
[1] [ResNet-vd](https://arxiv.org/pdf/1812.01187) models offer much improved accuracy with negligible performance cost.
@@ -77,7 +77,7 @@ Advanced Features:
- Pretrained models are available in the [PaddleDetection model zoo](docs/MODEL_ZOO.md).
- [Face detection models](configs/face_detection/README.md)
-- [Pretrained models for pedestrian and vehicle detection](contrib/README.md) Models for object detection in specific scenarios.
+- [Pretrained models for pedestrian and vehicle detection](contrib/README.md) Models for object detection in specific scenarios.
- [YOLOv3 enhanced model](docs/YOLOv3_ENHANCEMENT.md) Compared to MAP of 33.0% in paper, enhanced YOLOv3 reaches the MAP of 41.4% and inference speed is improved as well
- [Objects365 2019 Challenge champion model](docs/CACascadeRCNN.md) One of the best single models in Objects365 Full Track of which MAP reaches 31.7%.
- [Open Images Dataset V5 and Objects365 Dataset models](docs/OIDV5_BASELINE_MODEL.md)
@@ -99,6 +99,10 @@ Advanced Features:
## Updates
+#### 12/2019
+- Add Res2Net model.
+- Add HRNet model.
+
#### 21/11/2019
- Add CascadeClsAware RCNN model.
- Add CBNet, ResNet200 and Non-local model.
diff --git a/configs/cascade_rcnn_cls_aware_r101_vd_fpn_1x_softnms.yml b/configs/cascade_rcnn_cls_aware_r101_vd_fpn_1x_softnms.yml
index 4daebad5b9299a239a66eed47902dc36f63ec199..a2e9b39a5576182c48b0f60801840e3bc89f75a0 100644
--- a/configs/cascade_rcnn_cls_aware_r101_vd_fpn_1x_softnms.yml
+++ b/configs/cascade_rcnn_cls_aware_r101_vd_fpn_1x_softnms.yml
@@ -93,8 +93,8 @@ LearningRate:
gamma: 0.1
milestones: [60000, 80000]
- !LinearWarmup
- start_factor: 0.1
- steps: 1000
+ start_factor: 0.0
+ steps: 2000
OptimizerBuilder:
optimizer:
diff --git a/configs/hrnet/README.md b/configs/hrnet/README.md
new file mode 100644
index 0000000000000000000000000000000000000000..da4e1dc6f09f9a118c8905eb6d8820eb0bc141b3
--- /dev/null
+++ b/configs/hrnet/README.md
@@ -0,0 +1,34 @@
+# High-resolution networks (HRNets) for object detection
+
+## Introduction
+
+- Deep High-Resolution Representation Learning for Human Pose Estimation: [https://arxiv.org/abs/1902.09212](https://arxiv.org/abs/1902.09212)
+
+```
+@inproceedings{SunXLW19,
+ title={Deep High-Resolution Representation Learning for Human Pose Estimation},
+ author={Ke Sun and Bin Xiao and Dong Liu and Jingdong Wang},
+ booktitle={CVPR},
+ year={2019}
+}
+```
+
+- High-Resolution Representations for Labeling Pixels and Regions: [https://arxiv.org/abs/1904.04514](https://arxiv.org/abs/1904.04514)
+
+```
+@article{SunZJCXLMWLW19,
+ title={High-Resolution Representations for Labeling Pixels and Regions},
+ author={Ke Sun and Yang Zhao and Borui Jiang and Tianheng Cheng and Bin Xiao
+ and Dong Liu and Yadong Mu and Xinggang Wang and Wenyu Liu and Jingdong Wang},
+ journal = {CoRR},
+ volume = {abs/1904.04514},
+ year={2019}
+}
+```
+
+## Model Zoo
+
+| Backbone | Type | deformable Conv | Image/gpu | Lr schd | Inf time (fps) | Box AP | Mask AP | Download |
+| :---------------------- | :------------- | :---: | :-------: | :-----: | :------------: | :----: | :-----: | :----------------------------------------------------------: |
+| HRNetV2p_W18 | Faster | False | 2 | 1x | 17.509 | 36.0 | - | [model](https://paddlemodels.bj.bcebos.com/object_detection/faster_rcnn_hrnetv2p_w18_1x.tar) |
+| HRNetV2p_W18 | Faster | False | 2 | 2x | 17.509 | 38.0 | - | [model](https://paddlemodels.bj.bcebos.com/object_detection/faster_rcnn_hrnetv2p_w18_2x.tar) |
diff --git a/configs/hrnet/faster_rcnn_hrnetv2p_w18_1x.yml b/configs/hrnet/faster_rcnn_hrnetv2p_w18_1x.yml
new file mode 100644
index 0000000000000000000000000000000000000000..e8bfde6972a449aaa9745dae0ed711d7086304ea
--- /dev/null
+++ b/configs/hrnet/faster_rcnn_hrnetv2p_w18_1x.yml
@@ -0,0 +1,103 @@
+architecture: FasterRCNN
+max_iters: 90000
+snapshot_iter: 10000
+use_gpu: true
+log_smooth_window: 20
+save_dir: output
+pretrain_weights: https://paddle-imagenet-models-name.bj.bcebos.com/HRNet_W18_C_pretrained.tar
+weights: output/faster_rcnn_hrnetv2p_w18_1x/model_final
+metric: COCO
+num_classes: 81
+
+FasterRCNN:
+ backbone: HRNet
+ fpn: HRFPN
+ rpn_head: FPNRPNHead
+ roi_extractor: FPNRoIAlign
+ bbox_head: BBoxHead
+ bbox_assigner: BBoxAssigner
+
+HRNet:
+ feature_maps: [2, 3, 4, 5]
+ width: 18
+ freeze_at: 0
+ norm_type: bn
+
+HRFPN:
+ num_chan: 256
+ share_conv: false
+
+FPNRPNHead:
+ anchor_generator:
+ anchor_sizes: [32, 64, 128, 256, 512]
+ aspect_ratios: [0.5, 1.0, 2.0]
+ stride: [16.0, 16.0]
+ variance: [1.0, 1.0, 1.0, 1.0]
+ anchor_start_size: 32
+ max_level: 6
+ min_level: 2
+ num_chan: 256
+ rpn_target_assign:
+ rpn_batch_size_per_im: 256
+ rpn_fg_fraction: 0.5
+ rpn_negative_overlap: 0.3
+ rpn_positive_overlap: 0.7
+ rpn_straddle_thresh: 0.0
+ train_proposal:
+ min_size: 0.0
+ nms_thresh: 0.7
+ post_nms_top_n: 2000
+ pre_nms_top_n: 2000
+ test_proposal:
+ min_size: 0.0
+ nms_thresh: 0.7
+ post_nms_top_n: 1000
+ pre_nms_top_n: 1000
+
+FPNRoIAlign:
+ canconical_level: 4
+ canonical_size: 224
+ max_level: 5
+ min_level: 2
+ box_resolution: 7
+ sampling_ratio: 2
+
+BBoxAssigner:
+ batch_size_per_im: 512
+ bbox_reg_weights: [0.1, 0.1, 0.2, 0.2]
+ bg_thresh_hi: 0.5
+ bg_thresh_lo: 0.0
+ fg_fraction: 0.25
+ fg_thresh: 0.5
+
+BBoxHead:
+ head: TwoFCHead
+ nms:
+ keep_top_k: 100
+ nms_threshold: 0.5
+ score_threshold: 0.05
+
+TwoFCHead:
+ mlp_dim: 1024
+
+LearningRate:
+ base_lr: 0.02
+ schedulers:
+ - !PiecewiseDecay
+ gamma: 0.1
+ milestones: [60000, 80000]
+ - !LinearWarmup
+ start_factor: 0.1
+ steps: 1000
+
+OptimizerBuilder:
+ optimizer:
+ momentum: 0.9
+ type: Momentum
+ regularizer:
+ factor: 0.0001
+ type: L2
+
+_READER_: '../faster_fpn_reader.yml'
+TrainReader:
+ batch_size: 2
diff --git a/configs/hrnet/faster_rcnn_hrnetv2p_w18_2x.yml b/configs/hrnet/faster_rcnn_hrnetv2p_w18_2x.yml
new file mode 100644
index 0000000000000000000000000000000000000000..ee19ae58576c481eaa27e30e6d44ff4cf04fd740
--- /dev/null
+++ b/configs/hrnet/faster_rcnn_hrnetv2p_w18_2x.yml
@@ -0,0 +1,103 @@
+architecture: FasterRCNN
+max_iters: 180000
+snapshot_iter: 10000
+use_gpu: true
+log_smooth_window: 20
+save_dir: output
+pretrain_weights: https://paddle-imagenet-models-name.bj.bcebos.com/HRNet_W18_C_pretrained.tar
+weights: output/faster_rcnn_hrnetv2p_w18_2x/model_final
+metric: COCO
+num_classes: 81
+
+FasterRCNN:
+ backbone: HRNet
+ fpn: HRFPN
+ rpn_head: FPNRPNHead
+ roi_extractor: FPNRoIAlign
+ bbox_head: BBoxHead
+ bbox_assigner: BBoxAssigner
+
+HRNet:
+ feature_maps: [2, 3, 4, 5]
+ width: 18
+ freeze_at: 0
+ norm_type: bn
+
+HRFPN:
+ num_chan: 256
+ share_conv: false
+
+FPNRPNHead:
+ anchor_generator:
+ anchor_sizes: [32, 64, 128, 256, 512]
+ aspect_ratios: [0.5, 1.0, 2.0]
+ stride: [16.0, 16.0]
+ variance: [1.0, 1.0, 1.0, 1.0]
+ anchor_start_size: 32
+ max_level: 6
+ min_level: 2
+ num_chan: 256
+ rpn_target_assign:
+ rpn_batch_size_per_im: 256
+ rpn_fg_fraction: 0.5
+ rpn_negative_overlap: 0.3
+ rpn_positive_overlap: 0.7
+ rpn_straddle_thresh: 0.0
+ train_proposal:
+ min_size: 0.0
+ nms_thresh: 0.7
+ post_nms_top_n: 2000
+ pre_nms_top_n: 2000
+ test_proposal:
+ min_size: 0.0
+ nms_thresh: 0.7
+ post_nms_top_n: 1000
+ pre_nms_top_n: 1000
+
+FPNRoIAlign:
+ canconical_level: 4
+ canonical_size: 224
+ max_level: 5
+ min_level: 2
+ box_resolution: 7
+ sampling_ratio: 2
+
+BBoxAssigner:
+ batch_size_per_im: 512
+ bbox_reg_weights: [0.1, 0.1, 0.2, 0.2]
+ bg_thresh_hi: 0.5
+ bg_thresh_lo: 0.0
+ fg_fraction: 0.25
+ fg_thresh: 0.5
+
+BBoxHead:
+ head: TwoFCHead
+ nms:
+ keep_top_k: 100
+ nms_threshold: 0.5
+ score_threshold: 0.05
+
+TwoFCHead:
+ mlp_dim: 1024
+
+LearningRate:
+ base_lr: 0.02
+ schedulers:
+ - !PiecewiseDecay
+ gamma: 0.1
+ milestones: [120000, 160000]
+ - !LinearWarmup
+ start_factor: 0.1
+ steps: 1000
+
+OptimizerBuilder:
+ optimizer:
+ momentum: 0.9
+ type: Momentum
+ regularizer:
+ factor: 0.0001
+ type: L2
+
+_READER_: '../faster_fpn_reader.yml'
+TrainReader:
+ batch_size: 2
diff --git a/configs/res2net/README.md b/configs/res2net/README.md
new file mode 100644
index 0000000000000000000000000000000000000000..003839032c42839def56f764692b2aa43c63d429
--- /dev/null
+++ b/configs/res2net/README.md
@@ -0,0 +1,36 @@
+# Res2Net
+
+## Introduction
+
+- Res2Net: A New Multi-scale Backbone Architecture: [https://arxiv.org/abs/1904.01169](https://arxiv.org/abs/1904.01169)
+
+```
+@article{DBLP:journals/corr/abs-1904-01169,
+ author = {Shanghua Gao and
+ Ming{-}Ming Cheng and
+ Kai Zhao and
+ Xinyu Zhang and
+ Ming{-}Hsuan Yang and
+ Philip H. S. Torr},
+ title = {Res2Net: {A} New Multi-scale Backbone Architecture},
+ journal = {CoRR},
+ volume = {abs/1904.01169},
+ year = {2019},
+ url = {http://arxiv.org/abs/1904.01169},
+ archivePrefix = {arXiv},
+ eprint = {1904.01169},
+ timestamp = {Thu, 25 Apr 2019 10:24:54 +0200},
+ biburl = {https://dblp.org/rec/bib/journals/corr/abs-1904-01169},
+ bibsource = {dblp computer science bibliography, https://dblp.org}
+}
+```
+
+
+## Model Zoo
+
+| Backbone | Type | deformable Conv | Image/gpu | Lr schd | Inf time (fps) | Box AP | Mask AP | Download |
+| :---------------------- | :------------- | :---: | :-------: | :-----: | :------------: | :----: | :-----: | :----------------------------------------------------------: |
+| Res2Net50-FPN | Faster | False | 2 | 1x | 20.320 | 39.5 | - | [model](https://paddlemodels.bj.bcebos.com/object_detection/faster_rcnn_res2net50_vb_26w_4s_fpn_1x.tar) |
+| Res2Net50-FPN | Mask | False | 2 | 2x | 16.069 | 40.7 | 36.2 | [model](https://paddlemodels.bj.bcebos.com/object_detection/mask_rcnn_res2net50_vb_26w_4s_fpn_2x.tar) |
+| Res2Net50-vd-FPN | Mask | False | 2 | 2x | 15.816 | 40.9 | 36.2 | [model](https://paddlemodels.bj.bcebos.com/object_detection/mask_rcnn_res2net50_vd_26w_4s_fpn_2x.tar) |
+| Res2Net50-vd-FPN | Mask | True | 2 | 2x | 14.478 | 43.5 | 38.4 | [model](https://paddlemodels.bj.bcebos.com/object_detection/mask_rcnn_res2net50_vd_26w_4s_fpn_dcnv2_1x.tar) |
diff --git a/configs/res2net/faster_rcnn_res2net50_vb_26w_4s_fpn_1x.yml b/configs/res2net/faster_rcnn_res2net50_vb_26w_4s_fpn_1x.yml
new file mode 100644
index 0000000000000000000000000000000000000000..61039b4d58df404e297a874645818d1477f0b642
--- /dev/null
+++ b/configs/res2net/faster_rcnn_res2net50_vb_26w_4s_fpn_1x.yml
@@ -0,0 +1,108 @@
+architecture: FasterRCNN
+max_iters: 90000
+snapshot_iter: 10000
+use_gpu: true
+log_smooth_window: 20
+save_dir: output
+pretrain_weights: https://paddle-imagenet-models-name.bj.bcebos.com/Res2Net50_26w_4s_pretrained.tar
+weights: output/faster_rcnn_res2net50_vb_26w_4s_fpn_1x/model_final
+metric: COCO
+num_classes: 81
+
+FasterRCNN:
+ backbone: Res2Net
+ fpn: FPN
+ rpn_head: FPNRPNHead
+ roi_extractor: FPNRoIAlign
+ bbox_head: BBoxHead
+ bbox_assigner: BBoxAssigner
+
+Res2Net:
+ depth: 50
+ width: 26
+ scales: 4
+ feature_maps: [2, 3, 4, 5]
+ freeze_at: 2
+ norm_type: bn
+ variant: b
+
+FPN:
+ max_level: 6
+ min_level: 2
+ num_chan: 256
+ spatial_scale: [0.03125, 0.0625, 0.125, 0.25]
+
+FPNRPNHead:
+ anchor_generator:
+ anchor_sizes: [32, 64, 128, 256, 512]
+ aspect_ratios: [0.5, 1.0, 2.0]
+ stride: [16.0, 16.0]
+ variance: [1.0, 1.0, 1.0, 1.0]
+ anchor_start_size: 32
+ max_level: 6
+ min_level: 2
+ num_chan: 256
+ rpn_target_assign:
+ rpn_batch_size_per_im: 256
+ rpn_fg_fraction: 0.5
+ rpn_negative_overlap: 0.3
+ rpn_positive_overlap: 0.7
+ rpn_straddle_thresh: 0.0
+ train_proposal:
+ min_size: 0.0
+ nms_thresh: 0.7
+ post_nms_top_n: 2000
+ pre_nms_top_n: 2000
+ test_proposal:
+ min_size: 0.0
+ nms_thresh: 0.7
+ post_nms_top_n: 1000
+ pre_nms_top_n: 1000
+
+FPNRoIAlign:
+ canconical_level: 4
+ canonical_size: 224
+ max_level: 5
+ min_level: 2
+ box_resolution: 7
+ sampling_ratio: 2
+
+BBoxAssigner:
+ batch_size_per_im: 512
+ bbox_reg_weights: [0.1, 0.1, 0.2, 0.2]
+ bg_thresh_hi: 0.5
+ bg_thresh_lo: 0.0
+ fg_fraction: 0.25
+ fg_thresh: 0.5
+
+BBoxHead:
+ head: TwoFCHead
+ nms:
+ keep_top_k: 100
+ nms_threshold: 0.5
+ score_threshold: 0.05
+
+TwoFCHead:
+ mlp_dim: 1024
+
+LearningRate:
+ base_lr: 0.02
+ schedulers:
+ - !PiecewiseDecay
+ gamma: 0.1
+ milestones: [60000, 80000]
+ - !LinearWarmup
+ start_factor: 0.1
+ steps: 1000
+
+OptimizerBuilder:
+ optimizer:
+ momentum: 0.9
+ type: Momentum
+ regularizer:
+ factor: 0.0001
+ type: L2
+
+_READER_: '../faster_fpn_reader.yml'
+TrainReader:
+ batch_size: 2
diff --git a/configs/res2net/mask_rcnn_res2net50_vb_26w_4s_fpn_2x.yml b/configs/res2net/mask_rcnn_res2net50_vb_26w_4s_fpn_2x.yml
new file mode 100644
index 0000000000000000000000000000000000000000..5703e164af98c4655b19581518ddd24e44a25d69
--- /dev/null
+++ b/configs/res2net/mask_rcnn_res2net50_vb_26w_4s_fpn_2x.yml
@@ -0,0 +1,117 @@
+architecture: MaskRCNN
+trarchitecture: MaskRCNN
+use_gpu: true
+max_iters: 180000
+snapshot_iter: 10000
+log_smooth_window: 20
+save_dir: output
+pretrain_weights: https://paddle-imagenet-models-name.bj.bcebos.com/Res2Net50_26w_4s_pretrained.tar
+metric: COCO
+weights: output/mask_rcnn_res2net50_vb_26w_4s_fpn_2x/model_final/
+num_classes: 81
+
+MaskRCNN:
+ backbone: Res2Net
+ fpn: FPN
+ rpn_head: FPNRPNHead
+ roi_extractor: FPNRoIAlign
+ bbox_head: BBoxHead
+ bbox_assigner: BBoxAssigner
+
+Res2Net:
+ depth: 50
+ width: 26
+ scales: 4
+ feature_maps: [2, 3, 4, 5]
+ freeze_at: 2
+ norm_type: bn
+ variant: b
+
+FPN:
+ max_level: 6
+ min_level: 2
+ num_chan: 256
+ spatial_scale: [0.03125, 0.0625, 0.125, 0.25]
+
+FPNRPNHead:
+ anchor_generator:
+ aspect_ratios: [0.5, 1.0, 2.0]
+ variance: [1.0, 1.0, 1.0, 1.0]
+ anchor_start_size: 32
+ max_level: 6
+ min_level: 2
+ num_chan: 256
+ rpn_target_assign:
+ rpn_batch_size_per_im: 256
+ rpn_fg_fraction: 0.5
+ rpn_negative_overlap: 0.3
+ rpn_positive_overlap: 0.7
+ rpn_straddle_thresh: 0.0
+ train_proposal:
+ min_size: 0.0
+ nms_thresh: 0.7
+ pre_nms_top_n: 2000
+ post_nms_top_n: 2000
+ test_proposal:
+ min_size: 0.0
+ nms_thresh: 0.7
+ pre_nms_top_n: 1000
+ post_nms_top_n: 1000
+
+FPNRoIAlign:
+ canconical_level: 4
+ canonical_size: 224
+ max_level: 5
+ min_level: 2
+ sampling_ratio: 2
+ box_resolution: 7
+ mask_resolution: 14
+
+MaskHead:
+ dilation: 1
+ conv_dim: 256
+ num_convs: 4
+ resolution: 28
+
+BBoxAssigner:
+ batch_size_per_im: 512
+ bbox_reg_weights: [0.1, 0.1, 0.2, 0.2]
+ bg_thresh_hi: 0.5
+ bg_thresh_lo: 0.0
+ fg_fraction: 0.25
+ fg_thresh: 0.5
+
+MaskAssigner:
+ resolution: 28
+
+BBoxHead:
+ head: TwoFCHead
+ nms:
+ keep_top_k: 100
+ nms_threshold: 0.5
+ score_threshold: 0.05
+
+TwoFCHead:
+ mlp_dim: 1024
+
+LearningRate:
+ base_lr: 0.02
+ schedulers:
+ - !PiecewiseDecay
+ gamma: 0.1
+ milestones: [120000, 160000]
+ - !LinearWarmup
+ start_factor: 0.1
+ steps: 1000
+
+OptimizerBuilder:
+ optimizer:
+ momentum: 0.9
+ type: Momentum
+ regularizer:
+ factor: 0.0001
+ type: L2
+
+_READER_: '../mask_fpn_reader.yml'
+TrainReader:
+ batch_size: 2
diff --git a/configs/res2net/mask_rcnn_res2net50_vd_26w_4s_fpn_2x.yml b/configs/res2net/mask_rcnn_res2net50_vd_26w_4s_fpn_2x.yml
new file mode 100644
index 0000000000000000000000000000000000000000..c2cdf1de89e795b555a8763104a6b3b99729ff6e
--- /dev/null
+++ b/configs/res2net/mask_rcnn_res2net50_vd_26w_4s_fpn_2x.yml
@@ -0,0 +1,117 @@
+architecture: MaskRCNN
+trarchitecture: MaskRCNN
+use_gpu: true
+max_iters: 180000
+snapshot_iter: 10000
+log_smooth_window: 20
+save_dir: output
+pretrain_weights: https://paddle-imagenet-models-name.bj.bcebos.com/Res2Net50_vd_26w_4s_pretrained.tar
+metric: COCO
+weights: output/mask_rcnn_res2net50_vd_26w_4s_fpn_2x/model_final/
+num_classes: 81
+
+MaskRCNN:
+ backbone: Res2Net
+ fpn: FPN
+ rpn_head: FPNRPNHead
+ roi_extractor: FPNRoIAlign
+ bbox_head: BBoxHead
+ bbox_assigner: BBoxAssigner
+
+Res2Net:
+ depth: 50
+ width: 26
+ scales: 4
+ feature_maps: [2, 3, 4, 5]
+ freeze_at: 2
+ norm_type: bn
+ variant: d
+
+FPN:
+ max_level: 6
+ min_level: 2
+ num_chan: 256
+ spatial_scale: [0.03125, 0.0625, 0.125, 0.25]
+
+FPNRPNHead:
+ anchor_generator:
+ aspect_ratios: [0.5, 1.0, 2.0]
+ variance: [1.0, 1.0, 1.0, 1.0]
+ anchor_start_size: 32
+ max_level: 6
+ min_level: 2
+ num_chan: 256
+ rpn_target_assign:
+ rpn_batch_size_per_im: 256
+ rpn_fg_fraction: 0.5
+ rpn_negative_overlap: 0.3
+ rpn_positive_overlap: 0.7
+ rpn_straddle_thresh: 0.0
+ train_proposal:
+ min_size: 0.0
+ nms_thresh: 0.7
+ pre_nms_top_n: 2000
+ post_nms_top_n: 2000
+ test_proposal:
+ min_size: 0.0
+ nms_thresh: 0.7
+ pre_nms_top_n: 1000
+ post_nms_top_n: 1000
+
+FPNRoIAlign:
+ canconical_level: 4
+ canonical_size: 224
+ max_level: 5
+ min_level: 2
+ sampling_ratio: 2
+ box_resolution: 7
+ mask_resolution: 14
+
+MaskHead:
+ dilation: 1
+ conv_dim: 256
+ num_convs: 4
+ resolution: 28
+
+BBoxAssigner:
+ batch_size_per_im: 512
+ bbox_reg_weights: [0.1, 0.1, 0.2, 0.2]
+ bg_thresh_hi: 0.5
+ bg_thresh_lo: 0.0
+ fg_fraction: 0.25
+ fg_thresh: 0.5
+
+MaskAssigner:
+ resolution: 28
+
+BBoxHead:
+ head: TwoFCHead
+ nms:
+ keep_top_k: 100
+ nms_threshold: 0.5
+ score_threshold: 0.05
+
+TwoFCHead:
+ mlp_dim: 1024
+
+LearningRate:
+ base_lr: 0.02
+ schedulers:
+ - !PiecewiseDecay
+ gamma: 0.1
+ milestones: [120000, 160000]
+ - !LinearWarmup
+ start_factor: 0.1
+ steps: 1000
+
+OptimizerBuilder:
+ optimizer:
+ momentum: 0.9
+ type: Momentum
+ regularizer:
+ factor: 0.0001
+ type: L2
+
+_READER_: '../mask_fpn_reader.yml'
+TrainReader:
+ batch_size: 2
diff --git a/configs/res2net/mask_rcnn_res2net50_vd_26w_4s_fpn_dcnv2_1x.yml b/configs/res2net/mask_rcnn_res2net50_vd_26w_4s_fpn_dcnv2_1x.yml
new file mode 100644
index 0000000000000000000000000000000000000000..f637055b64fc4d1dcea204e3809ccf5e7ddc45da
--- /dev/null
+++ b/configs/res2net/mask_rcnn_res2net50_vd_26w_4s_fpn_dcnv2_1x.yml
@@ -0,0 +1,118 @@
+architecture: MaskRCNN
+trarchitecture: MaskRCNN
+use_gpu: true
+max_iters: 90000
+snapshot_iter: 10000
+log_smooth_window: 20
+save_dir: output
+pretrain_weights: https://paddle-imagenet-models-name.bj.bcebos.com/Res2Net50_vd_26w_4s_pretrained.tar
+metric: COCO
+weights: output/mask_rcnn_res2net50_vd_26w_4s_fpn_dcnv2_1x/model_final/
+num_classes: 81
+
+MaskRCNN:
+ backbone: Res2Net
+ fpn: FPN
+ rpn_head: FPNRPNHead
+ roi_extractor: FPNRoIAlign
+ bbox_head: BBoxHead
+ bbox_assigner: BBoxAssigner
+
+Res2Net:
+ depth: 50
+ width: 26
+ scales: 4
+ feature_maps: [2, 3, 4, 5]
+ freeze_at: 2
+ norm_type: bn
+ variant: d
+ dcn_v2_stages: [3, 4, 5]
+
+FPN:
+ max_level: 6
+ min_level: 2
+ num_chan: 256
+ spatial_scale: [0.03125, 0.0625, 0.125, 0.25]
+
+FPNRPNHead:
+ anchor_generator:
+ aspect_ratios: [0.5, 1.0, 2.0]
+ variance: [1.0, 1.0, 1.0, 1.0]
+ anchor_start_size: 32
+ max_level: 6
+ min_level: 2
+ num_chan: 256
+ rpn_target_assign:
+ rpn_batch_size_per_im: 256
+ rpn_fg_fraction: 0.5
+ rpn_negative_overlap: 0.3
+ rpn_positive_overlap: 0.7
+ rpn_straddle_thresh: 0.0
+ train_proposal:
+ min_size: 0.0
+ nms_thresh: 0.7
+ pre_nms_top_n: 2000
+ post_nms_top_n: 2000
+ test_proposal:
+ min_size: 0.0
+ nms_thresh: 0.7
+ pre_nms_top_n: 1000
+ post_nms_top_n: 1000
+
+FPNRoIAlign:
+ canconical_level: 4
+ canonical_size: 224
+ max_level: 5
+ min_level: 2
+ sampling_ratio: 2
+ box_resolution: 7
+ mask_resolution: 14
+
+MaskHead:
+ dilation: 1
+ conv_dim: 256
+ num_convs: 4
+ resolution: 28
+
+BBoxAssigner:
+ batch_size_per_im: 512
+ bbox_reg_weights: [0.1, 0.1, 0.2, 0.2]
+ bg_thresh_hi: 0.5
+ bg_thresh_lo: 0.0
+ fg_fraction: 0.25
+ fg_thresh: 0.5
+
+MaskAssigner:
+ resolution: 28
+
+BBoxHead:
+ head: TwoFCHead
+ nms:
+ keep_top_k: 100
+ nms_threshold: 0.5
+ score_threshold: 0.05
+
+TwoFCHead:
+ mlp_dim: 1024
+
+LearningRate:
+ base_lr: 0.02
+ schedulers:
+ - !PiecewiseDecay
+ gamma: 0.1
+ milestones: [60000, 80000]
+ - !LinearWarmup
+ start_factor: 0.1
+ steps: 1000
+
+OptimizerBuilder:
+ optimizer:
+ momentum: 0.9
+ type: Momentum
+ regularizer:
+ factor: 0.0001
+ type: L2
+
+_READER_: '../mask_fpn_reader.yml'
+TrainReader:
+ batch_size: 2
diff --git a/docs/MODEL_ZOO.md b/docs/MODEL_ZOO.md
index 842a25a24d7cfdc17f4f3ba874b96cf337a44d6c..ac3b52f67afa92a5f350a690a73949261447eb8d 100644
--- a/docs/MODEL_ZOO.md
+++ b/docs/MODEL_ZOO.md
@@ -86,12 +86,20 @@ The backbone models pretrained on ImageNet are available. All backbone models ar
| CBResNet200-vd-FPN-Nonlocal | Cascade Faster | c3-c5 | 1 | 2.5x | - | 53.3%(softnms) | - | [model](https://paddlemodels.bj.bcebos.com/object_detection/cascade_rcnn_cbr200_vd_fpn_dcnv2_nonlocal_softnms.tar) |
-
#### Notes:
- Deformable ConvNets v2(dcn_v2) reference from [Deformable ConvNets v2](https://arxiv.org/abs/1811.11168).
- `c3-c5` means adding `dcn` in resnet stage 3 to 5.
- Detailed configuration file in [configs/dcn](https://github.com/PaddlePaddle/PaddleDetection/tree/master/configs/dcn)
+
+### HRNet
+* See more details in [HRNet model zoo](../configs/hrnet/README.md)。
+
+
+### Res2Net
+* See more details in [Res2Net model zoo](../configs/res2net/README.md)。
+
+
### Group Normalization
| Backbone | Type | Image/gpu | Lr schd | Box AP | Mask AP | Download |
| :------------------- | :------------- | :-----: | :-----: | :----: | :-----: | :----------------------------------------------------------: |
diff --git a/docs/MODEL_ZOO_cn.md b/docs/MODEL_ZOO_cn.md
index 6a7ec48260738533296a63a0f11065c77950ba2f..457e14c7c1115e56bc6082dad5ad441ecda6de44 100644
--- a/docs/MODEL_ZOO_cn.md
+++ b/docs/MODEL_ZOO_cn.md
@@ -82,13 +82,20 @@ Paddle提供基于ImageNet的骨架网络预训练模型。所有预训练模型
| ResNet200-vd-FPN-Nonlocal | CascadeClsAware Faster | c3-c5 | 1 | 2.5x | - | 51.7%(softnms) | - | [下载链接](https://paddlemodels.bj.bcebos.com/object_detection/cascade_rcnn_cls_aware_r200_vd_fpn_dcnv2_nonlocal_softnms.tar) |
| CBResNet200-vd-FPN-Nonlocal | Cascade Faster | c3-c5 | 1 | 2.5x | - | 53.3%(softnms) | - | [下载链接](https://paddlemodels.bj.bcebos.com/object_detection/cascade_rcnn_cbr200_vd_fpn_dcnv2_nonlocal_softnms.tar) |
-
-
#### 注意事项:
- Deformable卷积网络v2(dcn_v2)参考自论文[Deformable ConvNets v2](https://arxiv.org/abs/1811.11168).
- `c3-c5`意思是在resnet模块的3到5阶段增加`dcn`.
- 详细的配置文件在[configs/dcn](https://github.com/PaddlePaddle/PaddleDetection/tree/master/configs/dcn)
+
+### HRNet
+* 详情见[HRNet模型库](../configs/hrnet/README.md)。
+
+
+### Res2Net
+* 详情见[Res2Net模型库](../configs/res2net/README.md)。
+
+
### Group Normalization
| 骨架网络 | 网络类型 | 每张GPU图片个数 | 学习率策略 | Box AP | Mask AP | 下载 |
| :------------------- | :------------- |:--------: | :-----: | :----: | :-----: | :----------------------------------------------------------: |
diff --git a/ppdet/modeling/backbones/__init__.py b/ppdet/modeling/backbones/__init__.py
index 9daaf9a4b5ff34e3ff9ac67447f84c9189c78149..22f78004b4066d7e5c82333d06b2733ed711ad6a 100644
--- a/ppdet/modeling/backbones/__init__.py
+++ b/ppdet/modeling/backbones/__init__.py
@@ -24,6 +24,9 @@ from . import vgg
from . import blazenet
from . import faceboxnet
from . import cb_resnet
+from . import res2net
+from . import hrnet
+from . import hrfpn
from .resnet import *
from .resnext import *
@@ -35,3 +38,6 @@ from .vgg import *
from .blazenet import *
from .faceboxnet import *
from .cb_resnet import *
+from .res2net import *
+from .hrnet import *
+from .hrfpn import *
\ No newline at end of file
diff --git a/ppdet/modeling/backbones/hrfpn.py b/ppdet/modeling/backbones/hrfpn.py
new file mode 100644
index 0000000000000000000000000000000000000000..1f1d65433e7cab908ecb548f61edb2e1f47bad74
--- /dev/null
+++ b/ppdet/modeling/backbones/hrfpn.py
@@ -0,0 +1,126 @@
+# Copyright (c) 2019 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from __future__ import absolute_import
+from __future__ import division
+from __future__ import print_function
+
+from collections import OrderedDict
+
+from paddle import fluid
+from paddle.fluid.param_attr import ParamAttr
+from paddle.fluid.initializer import Xavier
+from paddle.fluid.regularizer import L2Decay
+
+from ppdet.core.workspace import register
+
+__all__ = ['HRFPN']
+
+
+@register
+class HRFPN(object):
+ """
+ HRNet, see https://arxiv.org/abs/1908.07919
+
+ Args:
+ num_chan (int): number of feature channels
+ pooling_type (str): pooling type of downsampling
+ share_conv (bool): whethet to share conv for different layers' reduction
+ spatial_scale (list): feature map scaling factor
+ """
+
+ def __init__(self,
+ num_chan=256,
+ pooling_type="avg",
+ share_conv=False,
+ spatial_scale=[1./64, 1./32, 1./16, 1./8, 1./4],
+ ):
+ self.num_chan = num_chan
+ self.pooling_type = pooling_type
+ self.share_conv = share_conv
+ self.spatial_scale = spatial_scale
+ return
+
+ def get_output(self, body_dict):
+ num_out = len(self.spatial_scale)
+ body_name_list = list(body_dict.keys())
+
+ num_backbone_stages = len(body_name_list)
+
+ outs = []
+ outs.append(body_dict[body_name_list[0]])
+
+ # resize
+ for i in range(1, len(body_dict)):
+ resized = self.resize_input_tensor(body_dict[body_name_list[i]], outs[0], 2**i)
+ outs.append( resized )
+
+ # concat
+ out = fluid.layers.concat( outs, axis=1 )
+
+ # reduction
+ out = fluid.layers.conv2d(
+ input=out,
+ num_filters=self.num_chan,
+ filter_size=1,
+ stride=1,
+ padding=0,
+ param_attr=ParamAttr(name='hrfpn_reduction_weights'),
+ bias_attr=False)
+
+ # conv
+ outs = [out]
+ for i in range(1, num_out):
+ outs.append(self.pooling(out, size=2**i, stride=2**i, pooling_type=self.pooling_type))
+ outputs = []
+
+ for i in range(num_out):
+ conv_name = "shared_fpn_conv" if self.share_conv else "shared_fpn_conv_"+str(i)
+ conv = fluid.layers.conv2d(
+ input=outs[i],
+ num_filters=self.num_chan,
+ filter_size=3,
+ stride=1,
+ padding=1,
+ param_attr=ParamAttr(name=conv_name+"_weights"),
+ bias_attr=False)
+ outputs.append( conv )
+
+ for idx in range(0, num_out-len(body_name_list)):
+ body_name_list.append("fpn_res5_sum_subsampled_{}x".format( 2**(idx+1) ))
+
+ outputs = outputs[::-1]
+ body_name_list = body_name_list[::-1]
+
+ res_dict = OrderedDict([(body_name_list[k], outputs[k]) for k in range(len(body_name_list))])
+ return res_dict, self.spatial_scale
+
+ def resize_input_tensor(self, body_input, ref_output, scale):
+ shape = fluid.layers.shape(ref_output)
+ shape_hw = fluid.layers.slice(shape, axes=[0], starts=[2], ends=[4])
+ out_shape_ = shape_hw
+ out_shape = fluid.layers.cast(out_shape_, dtype='int32')
+ out_shape.stop_gradient = True
+ body_output = fluid.layers.resize_bilinear(
+ body_input, scale=scale, actual_shape=out_shape)
+ return body_output
+
+ def pooling(self, input, size, stride, pooling_type):
+ pool = fluid.layers.pool2d(input=input,
+ pool_size=size,
+ pool_stride=stride,
+ pool_type=pooling_type)
+ return pool
+
+
\ No newline at end of file
diff --git a/ppdet/modeling/backbones/hrnet.py b/ppdet/modeling/backbones/hrnet.py
new file mode 100644
index 0000000000000000000000000000000000000000..323e1dcc2719f445fe79c2b6d1221066b3cff6ee
--- /dev/null
+++ b/ppdet/modeling/backbones/hrnet.py
@@ -0,0 +1,377 @@
+# Copyright (c) 2019 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from __future__ import absolute_import
+from __future__ import division
+from __future__ import print_function
+
+from collections import OrderedDict
+
+from paddle import fluid
+from paddle.fluid.param_attr import ParamAttr
+from paddle.fluid.framework import Variable
+from paddle.fluid.regularizer import L2Decay
+
+from ppdet.core.workspace import register, serializable
+from numbers import Integral
+from paddle.fluid.initializer import MSRA
+import math
+
+from .name_adapter import NameAdapter
+
+__all__ = ['HRNet']
+
+
+@register
+@serializable
+class HRNet(object):
+ """
+ HRNet, see https://arxiv.org/abs/1908.07919
+ Args:
+ depth (int): ResNet depth, should be 18, 34, 50, 101, 152.
+ freeze_at (int): freeze the backbone at which stage
+ norm_type (str): normalization type, 'bn'/'sync_bn'/'affine_channel'
+ freeze_norm (bool): freeze normalization layers
+ norm_decay (float): weight decay for normalization layer weights
+ variant (str): ResNet variant, supports 'a', 'b', 'c', 'd' currently
+ feature_maps (list): index of stages whose feature maps are returned
+ """
+
+ def __init__(self,
+ width=40,
+ has_se=False,
+ freeze_at=2,
+ norm_type='bn',
+ freeze_norm=True,
+ norm_decay=0.,
+ feature_maps=[2, 3, 4, 5]):
+ super(HRNet, self).__init__()
+
+ if isinstance(feature_maps, Integral):
+ feature_maps = [feature_maps]
+
+ assert 0 <= freeze_at <= 4, "freeze_at should be 0, 1, 2, 3 or 4"
+ assert len(feature_maps) > 0, "need one or more feature maps"
+ assert norm_type in ['bn', 'sync_bn']
+
+ self.width = width
+ self.has_se = has_se
+ self.channels = {
+ 18: [[18, 36], [18, 36, 72], [18, 36, 72, 144]],
+ 30: [[30, 60], [30, 60, 120], [30, 60, 120, 240]],
+ 32: [[32, 64], [32, 64, 128], [32, 64, 128, 256]],
+ 40: [[40, 80], [40, 80, 160], [40, 80, 160, 320]],
+ 44: [[44, 88], [44, 88, 176], [44, 88, 176, 352]],
+ 48: [[48, 96], [48, 96, 192], [48, 96, 192, 384]],
+ 60: [[60, 120], [60, 120, 240], [60, 120, 240, 480]],
+ 64: [[64, 128], [64, 128, 256], [64, 128, 256, 512]],
+ }
+
+ self.freeze_at = freeze_at
+ self.norm_type = norm_type
+ self.norm_decay = norm_decay
+ self.freeze_norm = freeze_norm
+ self._model_type = 'HRNet'
+ self.feature_maps = feature_maps
+ self.end_points = []
+ return
+
+ def net(self, input, class_dim=1000):
+ width = self.width
+ channels_2, channels_3, channels_4 = self.channels[width]
+ num_modules_2, num_modules_3, num_modules_4 = 1, 4, 3
+
+ x = self.conv_bn_layer(input=input,
+ filter_size=3,
+ num_filters=64,
+ stride=2,
+ if_act=True,
+ name='layer1_1')
+ x = self.conv_bn_layer(input=x,
+ filter_size=3,
+ num_filters=64,
+ stride=2,
+ if_act=True,
+ name='layer1_2')
+
+ la1 = self.layer1(x, name='layer2')
+ tr1 = self.transition_layer([la1], [256], channels_2, name='tr1')
+ st2 = self.stage(tr1, num_modules_2, channels_2, name='st2')
+ tr2 = self.transition_layer(st2, channels_2, channels_3, name='tr2')
+ st3 = self.stage(tr2, num_modules_3, channels_3, name='st3')
+ tr3 = self.transition_layer(st3, channels_3, channels_4, name='tr3')
+ st4 = self.stage(tr3, num_modules_4, channels_4, name='st4')
+
+ self.end_points = st4
+ return st4[-1]
+
+ def layer1(self, input, name=None):
+ conv = input
+ for i in range(4):
+ conv = self.bottleneck_block(conv,
+ num_filters=64,
+ downsample=True if i == 0 else False,
+ name=name+'_'+str(i+1))
+ return conv
+
+ def transition_layer(self, x, in_channels, out_channels, name=None):
+ num_in = len(in_channels)
+ num_out = len(out_channels)
+ out = []
+ for i in range(num_out):
+ if i < num_in:
+ if in_channels[i] != out_channels[i]:
+ residual = self.conv_bn_layer(x[i],
+ filter_size=3,
+ num_filters=out_channels[i],
+ name=name+'_layer_'+str(i+1))
+ out.append(residual)
+ else:
+ out.append(x[i])
+ else:
+ residual = self.conv_bn_layer(x[-1],
+ filter_size=3,
+ num_filters=out_channels[i],
+ stride=2,
+ name=name+'_layer_'+str(i+1))
+ out.append(residual)
+ return out
+
+ def branches(self, x, block_num, channels, name=None):
+ out = []
+ for i in range(len(channels)):
+ residual = x[i]
+ for j in range(block_num):
+ residual = self.basic_block(residual,
+ channels[i],
+ name=name+'_branch_layer_'+str(i+1)+'_'+str(j+1))
+ out.append(residual)
+ return out
+
+ def fuse_layers(self, x, channels, multi_scale_output=True, name=None):
+ out = []
+ for i in range(len(channels) if multi_scale_output else 1):
+ residual = x[i]
+ for j in range(len(channels)):
+ if j > i:
+ y = self.conv_bn_layer(x[j],
+ filter_size=1,
+ num_filters=channels[i],
+ if_act=False,
+ name=name+'_layer_'+str(i+1)+'_'+str(j+1))
+ y = fluid.layers.resize_nearest(input=y, scale=2 ** (j - i))
+ residual = fluid.layers.elementwise_add(
+ x=residual, y=y, act=None)
+ elif j < i:
+ y = x[j]
+ for k in range(i - j):
+ if k == i - j - 1:
+ y = self.conv_bn_layer(y,
+ filter_size=3,
+ num_filters=channels[i],
+ stride=2,if_act=False,
+ name=name+'_layer_'+str(i+1)+'_'+str(j+1)+'_'+str(k+1))
+ else:
+ y = self.conv_bn_layer(y,
+ filter_size=3,
+ num_filters=channels[j],
+ stride=2,
+ name=name+'_layer_'+str(i+1)+'_'+str(j+1)+'_'+str(k+1))
+ residual = fluid.layers.elementwise_add(
+ x=residual, y=y, act=None)
+
+ residual = fluid.layers.relu(residual)
+ out.append(residual)
+ return out
+
+ def high_resolution_module(self, x, channels, multi_scale_output=True, name=None):
+ residual = self.branches(x, 4, channels, name=name)
+ out = self.fuse_layers(residual, channels, multi_scale_output=multi_scale_output, name=name)
+ return out
+
+ def stage(self, x, num_modules, channels, multi_scale_output=True, name=None):
+ out = x
+ for i in range(num_modules):
+ if i == num_modules - 1 and multi_scale_output == False:
+ out = self.high_resolution_module(out,
+ channels,
+ multi_scale_output=False,
+ name=name+'_'+str(i+1))
+ else:
+ out = self.high_resolution_module(out,
+ channels,
+ name=name+'_'+str(i+1))
+
+ return out
+
+ def last_cls_out(self, x, name=None):
+ out = []
+ num_filters_list = [128, 256, 512, 1024]
+ for i in range(len(x)):
+ out.append(self.conv_bn_layer(input=x[i],
+ filter_size=1,
+ num_filters=num_filters_list[i],
+ name=name+'conv_'+str(i+1)))
+ return out
+
+
+ def basic_block(self, input, num_filters, stride=1, downsample=False, name=None):
+ residual = input
+ conv = self.conv_bn_layer(input=input,
+ filter_size=3,
+ num_filters=num_filters,
+ stride=stride,
+ name=name+'_conv1')
+ conv = self.conv_bn_layer(input=conv,
+ filter_size=3,
+ num_filters=num_filters,
+ if_act=False,
+ name=name+'_conv2')
+ if downsample:
+ residual = self.conv_bn_layer(input=input,
+ filter_size=1,
+ num_filters=num_filters,
+ if_act=False,
+ name=name+'_downsample')
+ if self.has_se:
+ conv = self.squeeze_excitation(
+ input=conv,
+ num_channels=num_filters,
+ reduction_ratio=16,
+ name='fc'+name)
+ return fluid.layers.elementwise_add(x=residual, y=conv, act='relu')
+
+
+ def bottleneck_block(self, input, num_filters, stride=1, downsample=False, name=None):
+ residual = input
+ conv = self.conv_bn_layer(input=input,
+ filter_size=1,
+ num_filters=num_filters,
+ name=name+'_conv1')
+ conv = self.conv_bn_layer(input=conv,
+ filter_size=3,
+ num_filters=num_filters,
+ stride=stride,
+ name=name+'_conv2')
+ conv = self.conv_bn_layer(input=conv,
+ filter_size=1,
+ num_filters=num_filters*4,
+ if_act=False,
+ name=name+'_conv3')
+ if downsample:
+ residual = self.conv_bn_layer(input=input,
+ filter_size=1,
+ num_filters=num_filters*4,
+ if_act=False,
+ name=name+'_downsample')
+ if self.has_se:
+ conv = self.squeeze_excitation(
+ input=conv,
+ num_channels=num_filters * 4,
+ reduction_ratio=16,
+ name='fc'+name)
+ return fluid.layers.elementwise_add(x=residual, y=conv, act='relu')
+
+ def squeeze_excitation(self, input, num_channels, reduction_ratio, name=None):
+ pool = fluid.layers.pool2d(
+ input=input, pool_size=0, pool_type='avg', global_pooling=True)
+ stdv = 1.0 / math.sqrt(pool.shape[1] * 1.0)
+ squeeze = fluid.layers.fc(input=pool,
+ size=num_channels / reduction_ratio,
+ act='relu',
+ param_attr=fluid.param_attr.ParamAttr(
+ initializer=fluid.initializer.Uniform(
+ -stdv, stdv),name=name+'_sqz_weights'),
+ bias_attr=ParamAttr(name=name+'_sqz_offset'))
+ stdv = 1.0 / math.sqrt(squeeze.shape[1] * 1.0)
+ excitation = fluid.layers.fc(input=squeeze,
+ size=num_channels,
+ act='sigmoid',
+ param_attr=fluid.param_attr.ParamAttr(
+ initializer=fluid.initializer.Uniform(
+ -stdv, stdv),name=name+'_exc_weights'),
+ bias_attr=ParamAttr(name=name+'_exc_offset'))
+ scale = fluid.layers.elementwise_mul(x=input, y=excitation, axis=0)
+ return scale
+
+ def conv_bn_layer(self,input, filter_size, num_filters, stride=1, padding=1, num_groups=1, if_act=True, name=None):
+ conv = fluid.layers.conv2d(
+ input=input,
+ num_filters=num_filters,
+ filter_size=filter_size,
+ stride=stride,
+ padding=(filter_size-1)//2,
+ groups=num_groups,
+ act=None,
+ param_attr=ParamAttr(initializer=MSRA(), name=name+'_weights'),
+ bias_attr=False)
+ bn_name = name + '_bn'
+ bn = self._bn( input=conv, bn_name=bn_name )
+ if if_act:
+ bn = fluid.layers.relu(bn)
+ return bn
+
+ def _bn(self,
+ input,
+ act=None,
+ bn_name=None):
+ norm_lr = 0. if self.freeze_norm else 1.
+ norm_decay = self.norm_decay
+ pattr = ParamAttr(
+ name=bn_name + '_scale',
+ learning_rate=norm_lr,
+ regularizer=L2Decay(norm_decay))
+ battr = ParamAttr(
+ name=bn_name + '_offset',
+ learning_rate=norm_lr,
+ regularizer=L2Decay(norm_decay))
+
+ global_stats = True if self.freeze_norm else False
+ out = fluid.layers.batch_norm(
+ input=input,
+ act=act,
+ name=bn_name + '.output.1',
+ param_attr=pattr,
+ bias_attr=battr,
+ moving_mean_name=bn_name + '_mean',
+ moving_variance_name=bn_name + '_variance',
+ use_global_stats=global_stats)
+ scale = fluid.framework._get_var(pattr.name)
+ bias = fluid.framework._get_var(battr.name)
+ if self.freeze_norm:
+ scale.stop_gradient = True
+ bias.stop_gradient = True
+ return out
+
+ def __call__(self, input):
+ assert isinstance(input, Variable)
+ assert not (set(self.feature_maps) - set([2, 3, 4, 5])), \
+ "feature maps {} not in [2, 3, 4, 5]".format(self.feature_maps)
+
+ res_endpoints = []
+
+ res = input
+ feature_maps = self.feature_maps
+ self.net( input )
+
+ for i in feature_maps:
+ res = self.end_points[i-2]
+ if i in self.feature_maps:
+ res_endpoints.append(res)
+ if self.freeze_at >= i:
+ res.stop_gradient = True
+
+ return OrderedDict([('res{}_sum'.format(self.feature_maps[idx]), feat)
+ for idx, feat in enumerate(res_endpoints)])
+
diff --git a/ppdet/modeling/backbones/res2net.py b/ppdet/modeling/backbones/res2net.py
new file mode 100644
index 0000000000000000000000000000000000000000..ffac0353d613b0e4835f30bf0b57d2fda35cf56d
--- /dev/null
+++ b/ppdet/modeling/backbones/res2net.py
@@ -0,0 +1,223 @@
+# Copyright (c) 2019 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from __future__ import absolute_import
+from __future__ import division
+from __future__ import print_function
+
+from collections import OrderedDict
+
+from paddle import fluid
+from paddle.fluid.param_attr import ParamAttr
+from paddle.fluid.framework import Variable
+from paddle.fluid.regularizer import L2Decay
+from paddle.fluid.initializer import Constant
+
+from ppdet.core.workspace import register, serializable
+from numbers import Integral
+
+from .nonlocal_helper import add_space_nonlocal
+from .name_adapter import NameAdapter
+from .resnet import ResNet, ResNetC5
+
+__all__ = ['Res2Net', 'Res2NetC5']
+
+
+@register
+@serializable
+class Res2Net(ResNet):
+ """
+ Res2Net, see https://arxiv.org/abs/1904.01169
+ Args:
+ depth (int): Res2Net depth, should be 50, 101, 152, 200.
+ width (int): Res2Net width
+ scales (int): Res2Net scale
+ freeze_at (int): freeze the backbone at which stage
+ norm_type (str): normalization type, 'bn'/'sync_bn'/'affine_channel'
+ freeze_norm (bool): freeze normalization layers
+ norm_decay (float): weight decay for normalization layer weights
+ variant (str): Res2Net variant, supports 'a', 'b', 'c', 'd' currently
+ feature_maps (list): index of stages whose feature maps are returned
+ dcn_v2_stages (list): index of stages who select deformable conv v2
+ nonlocal_stages (list): index of stages who select nonlocal networks
+ """
+ __shared__ = ['norm_type', 'freeze_norm', 'weight_prefix_name']
+
+ def __init__(self,
+ depth=50,
+ width=26,
+ scales=4,
+ freeze_at=2,
+ norm_type='bn',
+ freeze_norm=True,
+ norm_decay=0.,
+ variant='b',
+ feature_maps=[2, 3, 4, 5],
+ dcn_v2_stages=[],
+ weight_prefix_name='',
+ nonlocal_stages=[],):
+ super(Res2Net, self).__init__(depth=depth,
+ freeze_at=freeze_at,
+ norm_type=norm_type,
+ freeze_norm=freeze_norm,
+ norm_decay=norm_decay,
+ variant=variant,
+ feature_maps=feature_maps,
+ dcn_v2_stages=dcn_v2_stages,
+ weight_prefix_name=weight_prefix_name,
+ nonlocal_stages=nonlocal_stages)
+
+ assert depth >= 50, "just support depth>=50 in res2net, but got depth=".format(depth)
+ # res2net config
+ self.scales = scales
+ self.width = width
+ basic_width = self.width * self.scales
+ self.num_filters1 = [basic_width * t for t in [1, 2, 4, 8]]
+ self.num_filters2 = [256 * t for t in [1, 2, 4, 8]]
+ self.num_filters = [64, 128, 384, 768]
+
+ def bottleneck(self,
+ input,
+ num_filters1,
+ num_filters2,
+ stride,
+ is_first,
+ name,
+ dcn_v2=False):
+ conv0 = self._conv_norm(
+ input=input,
+ num_filters=num_filters1,
+ filter_size=1,
+ stride=1,
+ act='relu',
+ name=name + '_branch2a')
+
+ xs = fluid.layers.split(conv0, self.scales, 1)
+ ys = []
+ for s in range(self.scales - 1):
+ if s == 0 or stride == 2:
+ ys.append(self._conv_norm(input=xs[s],
+ num_filters=num_filters1//self.scales,
+ stride=stride,
+ filter_size=3,
+ act='relu',
+ name=name+ '_branch2b_' + str(s+1),
+ dcn_v2=dcn_v2))
+ else:
+ ys.append(self._conv_norm(input=xs[s]+ys[-1],
+ num_filters=num_filters1//self.scales,
+ stride=stride,
+ filter_size=3,
+ act='relu',
+ name=name+ '_branch2b_' + str(s+1),
+ dcn_v2=dcn_v2))
+
+ if stride == 1:
+ ys.append(xs[-1])
+ else:
+ ys.append(fluid.layers.pool2d(input=xs[-1],
+ pool_size=3,
+ pool_stride=stride,
+ pool_padding=1,
+ pool_type='avg'))
+
+ conv1 = fluid.layers.concat(ys, axis=1)
+ conv2 = self._conv_norm(
+ input=conv1,
+ num_filters=num_filters2,
+ filter_size=1,
+ act=None,
+ name=name+"_branch2c")
+
+ short = self._shortcut(input,
+ num_filters2,
+ stride,
+ is_first,
+ name=name + "_branch1")
+
+ return fluid.layers.elementwise_add(
+ x=short, y=conv2, act='relu', name=name + ".add.output.5")
+
+
+ def layer_warp(self, input, stage_num):
+ """
+ Args:
+ input (Variable): input variable.
+ stage_num (int): the stage number, should be 2, 3, 4, 5
+
+ Returns:
+ The last variable in endpoint-th stage.
+ """
+ assert stage_num in [2, 3, 4, 5]
+
+ stages, block_func = self.depth_cfg[self.depth]
+ count = stages[stage_num - 2]
+
+ ch_out = self.stage_filters[stage_num - 2]
+ is_first = False if stage_num != 2 else True
+ dcn_v2 = True if stage_num in self.dcn_v2_stages else False
+
+ num_filters1 = self.num_filters1[stage_num-2]
+ num_filters2 = self.num_filters2[stage_num-2]
+
+ nonlocal_mod = 1000
+ if stage_num in self.nonlocal_stages:
+ nonlocal_mod = self.nonlocal_mod_cfg[self.depth] if stage_num==4 else 2
+
+ # Make the layer name and parameter name consistent
+ # with ImageNet pre-trained model
+ conv = input
+ for i in range(count):
+ conv_name = self.na.fix_layer_warp_name(stage_num, count, i)
+ if self.depth < 50:
+ is_first = True if i == 0 and stage_num == 2 else False
+ conv = block_func(
+ input=conv,
+ num_filters1=num_filters1,
+ num_filters2=num_filters2,
+ stride=2 if i == 0 and stage_num != 2 else 1,
+ is_first=is_first,
+ name=conv_name,
+ dcn_v2=dcn_v2)
+
+ # add non local model
+ dim_in = conv.shape[1]
+ nonlocal_name = "nonlocal_conv{}".format( stage_num )
+ if i % nonlocal_mod == nonlocal_mod - 1:
+ conv = add_space_nonlocal(
+ conv, dim_in, dim_in,
+ nonlocal_name + '_{}'.format(i), int(dim_in / 2) )
+ return conv
+
+
+@register
+@serializable
+class Res2NetC5(Res2Net):
+ __doc__ = Res2Net.__doc__
+
+ def __init__(self,
+ depth=50,
+ width=26,
+ scales=4,
+ freeze_at=2,
+ norm_type='bn',
+ freeze_norm=True,
+ norm_decay=0.,
+ variant='b',
+ feature_maps=[5],
+ weight_prefix_name=''):
+ super(Res2NetC5, self).__init__(depth, width, scales,
+ freeze_at, norm_type, freeze_norm,
+ norm_decay, variant, feature_maps)
+ self.severed_head = True
diff --git a/ppdet/modeling/ops.py b/ppdet/modeling/ops.py
index 08c967de468501693dd452e3a64f3c3c32c00f6e..08657f5cac146acdb4b9e35b97650819899da0f0 100644
--- a/ppdet/modeling/ops.py
+++ b/ppdet/modeling/ops.py
@@ -207,31 +207,29 @@ class MultiClassNMS(object):
self.nms_eta = nms_eta
self.background_label = background_label
+
@register
@serializable
class MultiClassSoftNMS(object):
- def __init__(self,
- score_threshold=0.01,
- keep_top_k=300,
- softnms_sigma=0.5,
- normalized=False,
- background_label=0,
- ):
+ def __init__(
+ self,
+ score_threshold=0.01,
+ keep_top_k=300,
+ softnms_sigma=0.5,
+ normalized=False,
+ background_label=0, ):
super(MultiClassSoftNMS, self).__init__()
self.score_threshold = score_threshold
self.keep_top_k = keep_top_k
self.softnms_sigma = softnms_sigma
self.normalized = normalized
self.background_label = background_label
-
- def __call__( self, bboxes, scores ):
-
+
+ def __call__(self, bboxes, scores):
def create_tmp_var(program, name, dtype, shape, lod_leval):
- return program.current_block().create_var(name=name,
- dtype=dtype,
- shape=shape,
- lod_leval=lod_leval)
-
+ return program.current_block().create_var(
+ name=name, dtype=dtype, shape=shape, lod_leval=lod_leval)
+
def _soft_nms_for_cls(dets, sigma, thres):
"""soft_nms_for_cls"""
dets_final = []
@@ -240,6 +238,8 @@ class MultiClassSoftNMS(object):
dets_final.append(dets[maxpos].copy())
ts, tx1, ty1, tx2, ty2 = dets[maxpos]
scores = dets[:, 0]
+ # force remove bbox at maxpos
+ scores[maxpos] = -1
x1 = dets[:, 1]
y1 = dets[:, 2]
x2 = dets[:, 3]
@@ -253,65 +253,69 @@ class MultiClassSoftNMS(object):
w = np.maximum(0.0, xx2 - xx1 + eta)
h = np.maximum(0.0, yy2 - yy1 + eta)
inter = w * h
- ovr = inter / (areas + areas[maxpos] - inter)
+ ovr = inter / (areas + areas[maxpos] - inter)
weight = np.exp(-(ovr * ovr) / sigma)
scores = scores * weight
idx_keep = np.where(scores >= thres)
dets[:, 0] = scores
dets = dets[idx_keep]
dets_final = np.array(dets_final).reshape(-1, 5)
- return dets_final
+ return dets_final
def _soft_nms(bboxes, scores):
bboxes = np.array(bboxes)
scores = np.array(scores)
class_nums = scores.shape[-1]
-
+
softnms_thres = self.score_threshold
softnms_sigma = self.softnms_sigma
keep_top_k = self.keep_top_k
-
+
cls_boxes = [[] for _ in range(class_nums)]
cls_ids = [[] for _ in range(class_nums)]
-
+
start_idx = 1 if self.background_label == 0 else 0
for j in range(start_idx, class_nums):
inds = np.where(scores[:, j] >= softnms_thres)[0]
scores_j = scores[inds, j]
rois_j = bboxes[inds, j, :]
- dets_j = np.hstack((scores_j[:, np.newaxis], rois_j)).astype(np.float32, copy=False)
+ dets_j = np.hstack((scores_j[:, np.newaxis], rois_j)).astype(
+ np.float32, copy=False)
cls_rank = np.argsort(-dets_j[:, 0])
dets_j = dets_j[cls_rank]
- cls_boxes[j] = _soft_nms_for_cls( dets_j, sigma=softnms_sigma, thres=softnms_thres )
- cls_ids[j] = np.array( [j]*cls_boxes[j].shape[0] ).reshape(-1,1)
-
+ cls_boxes[j] = _soft_nms_for_cls(
+ dets_j, sigma=softnms_sigma, thres=softnms_thres)
+ cls_ids[j] = np.array([j] * cls_boxes[j].shape[0]).reshape(-1,
+ 1)
+
cls_boxes = np.vstack(cls_boxes[start_idx:])
cls_ids = np.vstack(cls_ids[start_idx:])
- pred_result = np.hstack( [cls_ids, cls_boxes] )
+ pred_result = np.hstack([cls_ids, cls_boxes])
# Limit to max_per_image detections **over all classes**
- image_scores = cls_boxes[:,0]
+ image_scores = cls_boxes[:, 0]
if len(image_scores) > keep_top_k:
image_thresh = np.sort(image_scores)[-keep_top_k]
keep = np.where(cls_boxes[:, 0] >= image_thresh)[0]
pred_result = pred_result[keep, :]
-
+
res = fluid.LoDTensor()
res.set_lod([[0, pred_result.shape[0]]])
if pred_result.shape[0] == 0:
- pred_result = np.array( [[1]], dtype=np.float32 )
+ pred_result = np.array([[1]], dtype=np.float32)
res.set(pred_result, fluid.CPUPlace())
-
+
return res
-
- pred_result = create_tmp_var(fluid.default_main_program(),
- name='softnms_pred_result',
- dtype='float32',
- shape=[6],
- lod_leval=1)
- fluid.layers.py_func(func=_soft_nms,
- x=[bboxes, scores], out=pred_result)
+
+ pred_result = create_tmp_var(
+ fluid.default_main_program(),
+ name='softnms_pred_result',
+ dtype='float32',
+ shape=[6],
+ lod_leval=1)
+ fluid.layers.py_func(
+ func=_soft_nms, x=[bboxes, scores], out=pred_result)
return pred_result