[PaddleSlim]Add yolov3_distillation demo to release/1.6 (#3564)

e9b5c69d · Bai Yifan · whs · 203b6e19 · e9b5c69d · e9b5c69d
5 changed file
--- a/PaddleCV/PaddleDetection/slim/distillation/README.md
+++ b/PaddleCV/PaddleDetection/slim/distillation/README.md
+>运行该示例前请安装Paddle1.6或更高版本
+# 检测模型蒸馏示例
+## 概述
+该示例使用PaddleSlim提供的[蒸馏策略](https://github.com/PaddlePaddle/models/blob/develop/PaddleSlim/docs/tutorial.md#3-蒸馏)对检测库中的模型进行蒸馏训练。
+在阅读该示例前，建议您先了解以下内容：
+- [检测库的常规训练方法](https://github.com/PaddlePaddle/models/tree/develop/PaddleCV/PaddleDetection)
+- [PaddleSlim使用文档](https://github.com/PaddlePaddle/models/blob/develop/PaddleSlim/docs/usage.md)
+## 配置文件说明
+关于配置文件如何编写您可以参考：
+- [PaddleSlim配置文件编写说明](https://github.com/PaddlePaddle/models/blob/develop/PaddleSlim/docs/usage.md#122-%E9%85%8D%E7%BD%AE%E6%96%87%E4%BB%B6%E7%9A%84%E4%BD%BF%E7%94%A8)
+- [蒸馏策略配置文件编写说明](https://github.com/PaddlePaddle/models/blob/develop/PaddleSlim/docs/usage.md#23-蒸馏)
+这里以ResNet34-YoloV3蒸馏MobileNetV1-YoloV3模型为例，首先，为了对`student model`和`teacher model`有个总体的认识，从而进一步确认蒸馏的对象，我们通过以下命令分别观察两个网络变量（Variable）的名称和形状：
+```python
+# 观察student model的Variable
+for v in fluid.default_main_program().list_vars():
+    if "py_reader" not in v.name and "double_buffer" not in v.name and "generated_var" not in v.name:
+        print(v.name, v.shape)
+# 观察teacher model的Variable
+for v in teacher_program.list_vars():
+    print(v.name, v.shape)
+```
+经过对比可以发现，`student model`和`teacher model`的部分中间结果分别为：
+```bash
+# student model
+conv2d_15.tmp_0
+# teacher model
+teacher_teacher_conv2d_1.tmp_0
+```
+所以，我们用`l2_distiller`对这两个特征图做蒸馏。在配置文件中进行如下配置：
+```yaml
+distillers:
+    l2_distiller:
+        class: 'L2Distiller'
+        teacher_feature_map: 'teacher_teacher_conv2d_1.tmp_0'
+        student_feature_map: 'conv2d_15.tmp_0'
+        distillation_loss_weight: 1
+strategies:
+    distillation_strategy:
+        class: 'DistillationStrategy'
+        distillers: ['l2_distiller']
+        start_epoch: 0
+        end_epoch: 270
+```
+我们也可以根据上述操作为蒸馏策略选择其他loss，PaddleSlim支持的有`FSP_loss`, `L2_loss`和`softmax_with_cross_entropy_loss` 。
+## 训练
+根据[PaddleDetection/tools/train.py](https://github.com/PaddlePaddle/models/blob/develop/PaddleCV/PaddleDetection/tools/train.py)编写压缩脚本compress.py。
+在该脚本中定义了Compressor对象，用于执行压缩任务。
+您可以通过运行脚本`run.sh`运行该示例。
+### 保存断点（checkpoint）
+如果在配置文件中设置了`checkpoint_path`, 则在蒸馏任务执行过程中会自动保存断点，当任务异常中断时，
+重启任务会自动从`checkpoint_path`路径下按数字顺序加载最新的checkpoint文件。如果不想让重启的任务从断点恢复，
+需要修改配置文件中的`checkpoint_path`，或者将`checkpoint_path`路径下文件清空。
+>注意：配置文件中的信息不会保存在断点中，重启前对配置文件的修改将会生效。
+## 评估
+如果在配置文件中设置了`checkpoint_path`，则每个epoch会保存一个压缩后的用于评估的模型，
+该模型会保存在`${checkpoint_path}/${epoch_id}/eval_model/`路径下，包含`__model__`和`__params__`两个文件。
+其中，`__model__`用于保存模型结构信息，`__params__`用于保存参数（parameters）信息。
+如果不需要保存评估模型，可以在定义Compressor对象时，将`save_eval_model`选项设置为False（默认为True）。
+## 预测
+如果在配置文件中设置了`checkpoint_path`，并且在定义Compressor对象时指定了`prune_infer_model`选项，则每个epoch都会
+保存一个`inference model`。该模型是通过删除eval_program中多余的operators而得到的。
+该模型会保存在`${checkpoint_path}/${epoch_id}/eval_model/`路径下，包含`__model__.infer`和`__params__`两个文件。
+其中，`__model__.infer`用于保存模型结构信息，`__params__`用于保存参数（parameters）信息。
+更多关于`prune_infer_model`选项的介绍，请参考：[Compressor介绍](https://github.com/PaddlePaddle/models/blob/develop/PaddleSlim/docs/usage.md#121-%E5%A6%82%E4%BD%95%E6%94%B9%E5%86%99%E6%99%AE%E9%80%9A%E8%AE%AD%E7%BB%83%E8%84%9A%E6%9C%AC)
+### python预测
+在脚本<a href="../infer.py">slim/infer.py</a>中展示了如何使用fluid python API加载使用预测模型进行预测。
+### PaddleLite
+该示例中产出的预测（inference）模型可以直接用PaddleLite进行加载使用。
+关于PaddleLite如何使用，请参考：[PaddleLite使用文档](https://github.com/PaddlePaddle/Paddle-Lite/wiki#%E4%BD%BF%E7%94%A8)
+## 示例结果
+### MobileNetV1-YOLO-V3
+| FLOPS |Box AP|
+|---|---|
+|baseline|76.2     |
+|蒸馏后|- |
+## FAQ
--- a/PaddleCV/PaddleDetection/slim/distillation/compress.py
+++ b/PaddleCV/PaddleDetection/slim/distillation/compress.py
+# Copyright (c) 2019 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+from __future__ import absolute_import
+from __future__ import division
+from __future__ import print_function
+import os
+import time
+import multiprocessing
+import numpy as np
+from collections import deque, OrderedDict
+from paddle.fluid.contrib.slim.core import Compressor
+from paddle.fluid.framework import IrGraph
+def set_paddle_flags(**kwargs):
+    for key, value in kwargs.items():
+        if os.environ.get(key, None) is None:
+            os.environ[key] = str(value)
+# NOTE(paddle-dev): All of these flags should be set before 
+# `import paddle`. Otherwise, it would not take any effect.
+set_paddle_flags(
+    FLAGS_eager_delete_tensor_gb=0,  # enable GC to save memory
+)
+from paddle import fluid
+import sys
+sys.path.append("../../")
+from ppdet.core.workspace import load_config, merge_config, create
+from ppdet.data.data_feed import create_reader
+from ppdet.utils.eval_utils import parse_fetches, eval_results
+from ppdet.utils.stats import TrainingStats
+from ppdet.utils.cli import ArgsParser
+from ppdet.utils.check import check_gpu
+import ppdet.utils.checkpoint as checkpoint
+from ppdet.modeling.model_input import create_feed
+import logging
+FORMAT = '%(asctime)s-%(levelname)s: %(message)s'
+logging.basicConfig(level=logging.INFO, format=FORMAT)
+logger = logging.getLogger(__name__)
+def eval_run(exe, compile_program, reader, keys, values, cls, test_feed):
+    """
+    Run evaluation program, return program outputs.
+    """
+    iter_id = 0
+    results = []
+    if len(cls) != 0:
+        values = []
+        for i in range(len(cls)):
+            _, accum_map = cls[i].get_map_var()
+            cls[i].reset(exe)
+            values.append(accum_map)
+    images_num = 0
+    start_time = time.time()
+    has_bbox = 'bbox' in keys
+    for data in reader():
+        data = test_feed.feed(data)
+        feed_data = {'image': data['image'], 'im_size': data['im_size']}
+        outs = exe.run(compile_program,
+                       feed=feed_data,
+                       fetch_list=[values[0]],
+                       return_numpy=False)
+        outs.append(data['gt_box'])
+        outs.append(data['gt_label'])
+        outs.append(data['is_difficult'])
+        res = {
+            k: (np.array(v), v.recursive_sequence_lengths())
+            for k, v in zip(keys, outs)
+        }
+        results.append(res)
+        if iter_id % 100 == 0:
+            logger.info('Test iter {}'.format(iter_id))
+        iter_id += 1
+        images_num += len(res['bbox'][1][0]) if has_bbox else 1
+    logger.info('Test finish iter {}'.format(iter_id))
+    end_time = time.time()
+    fps = images_num / (end_time - start_time)
+    if has_bbox:
+        logger.info('Total number of images: {}, inference time: {} fps.'.
+                    format(images_num, fps))
+    else:
+        logger.info('Total iteration: {}, inference time: {} batch/s.'.format(
+            images_num, fps))
+    return results
+def main():
+    cfg = load_config(FLAGS.config)
+    if 'architecture' in cfg:
+        main_arch = cfg.architecture
+    else:
+        raise ValueError("'architecture' not specified in config file.")
+    merge_config(FLAGS.opt)
+    if 'log_iter' not in cfg:
+        cfg.log_iter = 20
+    # check if set use_gpu=True in paddlepaddle cpu version
+    check_gpu(cfg.use_gpu)
+    if cfg.use_gpu:
+        devices_num = fluid.core.get_cuda_device_count()
+    else:
+        devices_num = int(
+            os.environ.get('CPU_NUM', multiprocessing.cpu_count()))
+    if 'train_feed' not in cfg:
+        train_feed = create(main_arch + 'TrainFeed')
+    else:
+        train_feed = create(cfg.train_feed)
+    if 'eval_feed' not in cfg:
+        eval_feed = create(main_arch + 'EvalFeed')
+    else:
+        eval_feed = create(cfg.eval_feed)
+    place = fluid.CUDAPlace(0) if cfg.use_gpu else fluid.CPUPlace()
+    exe = fluid.Executor(place)
+    lr_builder = create('LearningRate')
+    optim_builder = create('OptimizerBuilder')
+    # build program
+    model = create(main_arch)
+    train_loader, train_feed_vars = create_feed(train_feed, iterable=True)
+    train_fetches = model.train(train_feed_vars)
+    loss = train_fetches['loss']
+    lr = lr_builder()
+    opt = optim_builder(lr)
+    opt.minimize(loss)
+    #for v in fluid.default_main_program().list_vars():
+    #    if "py_reader" not in v.name and "double_buffer" not in v.name and "generated_var" not in v.name:
+    #        print(v.name, v.shape)
+    cfg.max_iters = 258
+    train_reader = create_reader(train_feed, cfg.max_iters, FLAGS.dataset_dir)
+    train_loader.set_sample_list_generator(train_reader, place)
+    exe.run(fluid.default_startup_program())
+    # parse train fetches
+    train_keys, train_values, _ = parse_fetches(train_fetches)
+    train_keys.append('lr')
+    train_values.append(lr.name)
+    train_fetch_list = []
+    for k, v in zip(train_keys, train_values):
+        train_fetch_list.append((k, v))
+    print("train_fetch_list: {}".format(train_fetch_list))
+    eval_prog = fluid.Program()
+    startup_prog = fluid.Program()
+    with fluid.program_guard(eval_prog, startup_prog):
+        with fluid.unique_name.guard():
+            model = create(main_arch)
+            _, test_feed_vars = create_feed(eval_feed, iterable=True)
+            fetches = model.eval(test_feed_vars)
+    eval_prog = eval_prog.clone(True)
+    eval_reader = create_reader(eval_feed, args_path=FLAGS.dataset_dir)
+    test_data_feed = fluid.DataFeeder(test_feed_vars.values(), place)
+    # parse eval fetches
+    extra_keys = []
+    if cfg.metric == 'COCO':
+        extra_keys = ['im_info', 'im_id', 'im_shape']
+    if cfg.metric == 'VOC':
+        extra_keys = ['gt_box', 'gt_label', 'is_difficult']
+    eval_keys, eval_values, eval_cls = parse_fetches(fetches, eval_prog,
+                                                     extra_keys)
+    eval_fetch_list = []
+    for k, v in zip(eval_keys, eval_values):
+        eval_fetch_list.append((k, v))
+    print("eval_fetch_list: {}".format(eval_fetch_list))
+    exe.run(startup_prog)
+    checkpoint.load_params(exe,
+                           fluid.default_main_program(), cfg.pretrain_weights)
+    best_box_ap_list = []
+    def eval_func(program, scope):
+        results = eval_run(exe, program, eval_reader, eval_keys, eval_values,
+                           eval_cls, test_data_feed)
+        resolution = None
+        is_bbox_normalized = False
+        if 'mask' in results[0]:
+            resolution = model.mask_head.resolution
+        box_ap_stats = eval_results(results, eval_feed, cfg.metric,
+                                    cfg.num_classes, resolution,
+                                    is_bbox_normalized, FLAGS.output_eval)
+        if len(best_box_ap_list) == 0:
+            best_box_ap_list.append(box_ap_stats[0])
+        elif box_ap_stats[0] > best_box_ap_list[0]:
+            best_box_ap_list[0] = box_ap_stats[0]
+        logger.info("Best test box ap: {}".format(best_box_ap_list[0]))
+        return best_box_ap_list[0]
+    test_feed = [('image', test_feed_vars['image'].name),
+                 ('im_size', test_feed_vars['im_size'].name)]
+    teacher_cfg = load_config(FLAGS.teacher_config)
+    teacher_arch = teacher_cfg.architecture
+    teacher_programs = []
+    teacher_program = fluid.Program()
+    teacher_startup_program = fluid.Program()
+    with fluid.program_guard(teacher_program, teacher_startup_program):
+        with fluid.unique_name.guard('teacher_'):
+            teacher_feed_vars = OrderedDict()
+            for name, var in train_feed_vars.items():
+                teacher_feed_vars[name] = teacher_program.global_block(
+                )._clone_variable(
+                    var, force_persistable=False)
+            model = create(teacher_arch)
+            train_fetches = model.train(teacher_feed_vars)
+    #print("="*50+"teacher_model_params"+"="*50)
+    #for v in teacher_program.list_vars():
+    #    print(v.name, v.shape)
+    #return
+    exe.run(teacher_startup_program)
+    assert FLAGS.teacher_pretrained and os.path.exists(
+        FLAGS.teacher_pretrained
+    ), "teacher_pretrained should be set when teacher_model is not None."
+    def if_exist(var):
+        return os.path.exists(os.path.join(FLAGS.teacher_pretrained, var.name))
+    fluid.io.load_vars(
+        exe,
+        FLAGS.teacher_pretrained,
+        main_program=teacher_program,
+        predicate=if_exist)
+    teacher_programs.append(teacher_program.clone(for_test=True))
+    com = Compressor(
+        place,
+        fluid.global_scope(),
+        fluid.default_main_program(),
+        train_reader=train_reader,
+        train_feed_list=[(key, value.name)
+                         for key, value in train_feed_vars.items()],
+        train_fetch_list=train_fetch_list,
+        eval_program=eval_prog,
+        eval_reader=eval_reader,
+        eval_feed_list=test_feed,
+        eval_func={'map': eval_func},
+        eval_fetch_list=eval_fetch_list[0:1],
+        save_eval_model=True,
+        prune_infer_model=[["image", "im_size"], ["multiclass_nms_0.tmp_0"]],
+        teacher_programs=teacher_programs,
+        train_optimizer=None,
+        distiller_optimizer=opt,
+        log_period=20)
+    com.config(FLAGS.slim_file)
+    com.run()
+if __name__ == '__main__':
+    parser = ArgsParser()
+    parser.add_argument(
+        "-t",
+        "--teacher_config",
+        default=None,
+        type=str,
+        help="Config file of teacher architecture.")
+    parser.add_argument(
+        "-s",
+        "--slim_file",
+        default=None,
+        type=str,
+        help="Config file of PaddleSlim.")
+    parser.add_argument(
+        "-r",
+        "--resume_checkpoint",
+        default=None,
+        type=str,
+        help="Checkpoint path for resuming training.")
+    parser.add_argument(
+        "--eval",
+        action='store_true',
+        default=False,
+        help="Whether to perform evaluation in train")
+    parser.add_argument(
+        "--teacher_pretrained",
+        default=None,
+        type=str,
+        help="Whether to use pretrained model.")
+    parser.add_argument(
+        "--output_eval",
+        default=None,
+        type=str,
+        help="Evaluation directory, default is current directory.")
+    parser.add_argument(
+        "-d",
+        "--dataset_dir",
+        default=None,
+        type=str,
+        help="Dataset path, same as DataFeed.dataset.dataset_dir")
+    FLAGS = parser.parse_args()
+    main()
--- a/PaddleCV/PaddleDetection/slim/distillation/run.sh
+++ b/PaddleCV/PaddleDetection/slim/distillation/run.sh
+#!/usr/bin/env bash
+# download pretrain model
+root_url="https://paddlemodels.bj.bcebos.com/object_detection"
+yolov3_r34_voc="yolov3_r34_voc.tar"
+pretrain_dir='./pretrain'
+if [ ! -d ${pretrain_dir} ]; then
+  mkdir ${pretrain_dir}
+fi
+cd ${pretrain_dir}
+if [ ! -f ${yolov3_r34_voc} ]; then
+    wget ${root_url}/${yolov3_r34_voc}
+    tar xf ${yolov3_r34_voc}
+fi
+cd -
+# enable GC strategy
+export FLAGS_fast_eager_deletion_mode=1
+export FLAGS_eager_delete_tensor_gb=0.0
+# for distillation
+#-----------------
+export CUDA_VISIBLE_DEVICES=0,1,2,3
+# Fixing name conflicts in distillation
+cd ${pretrain_dir}/yolov3_r34_voc
+for files in $(ls teacher_*)
+    do mv $files ${files#*_}
+done
+for files in $(ls *)
+    do mv $files "teacher_"$files
+done
+cd -
+python -u compress.py \
+-c ../../configs/yolov3_mobilenet_v1_voc.yml \
+-t yolov3_resnet34.yml \
+-s yolov3_mobilenet_v1_yolov3_resnet34_distillation.yml \
+-o YoloTrainFeed.batch_size=64 \
+-d ../../dataset/voc \
+--teacher_pretrained ./pretrain/yolov3_r34_voc \
+> yolov3_distallation.log 2>&1 &
+tailf yolov3_distallation.log
--- a/PaddleCV/PaddleDetection/slim/distillation/yolov3_mobilenet_v1_yolov3_resnet34_distillation.yml
+++ b/PaddleCV/PaddleDetection/slim/distillation/yolov3_mobilenet_v1_yolov3_resnet34_distillation.yml
+version: 1.0
+distillers:
+    l2_distiller:
+        class: 'L2Distiller'
+        teacher_feature_map: 'teacher_teacher_conv2d_1.tmp_0' 
+        student_feature_map: 'conv2d_15.tmp_0' 
+        distillation_loss_weight: 1
+strategies:
+    distillation_strategy:
+        class: 'DistillationStrategy'
+        distillers: ['l2_distiller']
+        start_epoch: 0
+        end_epoch: 270
+compressor:
+    epoch: 271
+    checkpoint_path: './checkpoints/'
+    strategies:
+        - distillation_strategy
--- a/PaddleCV/PaddleDetection/slim/distillation/yolov3_resnet34.yml
+++ b/PaddleCV/PaddleDetection/slim/distillation/yolov3_resnet34.yml
+architecture: YOLOv3
+log_smooth_window: 20
+metric: VOC
+map_type: 11point
+num_classes: 20
+weight_prefix_name: teacher_
+YOLOv3:
+  backbone: ResNet
+  yolo_head: YOLOv3Head
+ResNet:
+  norm_type: sync_bn
+  freeze_at: 0
+  freeze_norm: false
+  norm_decay: 0.
+  depth: 34
+  feature_maps: [3, 4, 5]
+YOLOv3Head:
+  anchor_masks: [[6, 7, 8], [3, 4, 5], [0, 1, 2]]
+  anchors: [[10, 13], [16, 30], [33, 23],
+            [30, 61], [62, 45], [59, 119],
+            [116, 90], [156, 198], [373, 326]]
+  norm_decay: 0.
+  ignore_thresh: 0.7
+  label_smooth: false
+  nms:
+    background_label: -1
+    keep_top_k: 100
+    nms_threshold: 0.45
+    nms_top_k: 1000
+    normalized: false
+    score_threshold: 0.01