Merge branch 'develop' of https://github.com/PaddlePaddle/models into add_dist_benchmark_data

.gitignore

-paddle/operators/check_t.save
-paddle/operators/check_tensor.ls
-paddle/operators/tensor.save
-python/paddle/v2/fluid/tests/book/image_classification_resnet.inference.model/
-python/paddle/v2/fluid/tests/book/image_classification_vgg.inference.model/
-python/paddle/v2/fluid/tests/book/label_semantic_roles.inference.model/
 *.DS_Store
 *.vs
-build/
-build_doc/
 *.user
-
-.vscode
-.idea
-.project
-.cproject
-.pydevproject
-.settings/
-
 *.pyc
-CMakeSettings.json
-Makefile
-.test_env/
-third_party/
-
 *~
-bazel-*
-third_party/
-
-build_*
-# clion workspace.
-cmake-build-*
-model_test
\ No newline at end of file

README.md

@@ -26,7 +26,8 @@ PaddlePaddle 提供了丰富的计算单元，使得用户可以采用模块化
 [SE-ResNeXt](https://github.com/PaddlePaddle/models/tree/develop/fluid/PaddleCV/image_classification/models)|图像分类模型|ResNeXt中加入了SE block，提高了模型准确率|[Squeeze-and-excitation networks](https://arxiv.org/abs/1709.01507)
 [SSD](https://github.com/PaddlePaddle/models/blob/develop/fluid/PaddleCV/object_detection/README_cn.md)|单阶段目标检测器|在不同尺度的特征图上检测对应尺度的目标,可以方便地插入到任何一种标准卷积网络中|[SSD: Single Shot MultiBox Detector](https://arxiv.org/abs/1512.02325)
 [Face Detector: PyramidBox](https://github.com/PaddlePaddle/models/tree/develop/fluid/PaddleCV/face_detection/README_cn.md)|基于SSD的单阶段人脸检测器|利用上下文信息解决困难人脸的检测问题，网络表达能力高，鲁棒性强|[PyramidBox: A Context-assisted Single Shot Face Detector](https://arxiv.org/pdf/1803.07737.pdf)
-[Faster RCNN](https://github.com/PaddlePaddle/models/tree/develop/fluid/PaddleCV/faster_rcnn/README_cn.md)|典型的两阶段目标检测器|创造性地采用卷积网络自行产生建议框，并且和目标检测网络共享卷积网络，建议框数目减少，质量提高|[Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://arxiv.org/abs/1506.01497)
+[Faster RCNN](https://github.com/PaddlePaddle/models/tree/develop/fluid/PaddleCV/rcnn/README_cn.md)|典型的两阶段目标检测器|创造性地采用卷积网络自行产生建议框，并且和目标检测网络共享卷积网络，建议框数目减少，质量提高|[Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://arxiv.org/abs/1506.01497)
+[Mask RCNN](https://github.com/PaddlePaddle/models/tree/develop/fluid/PaddleCV/rcnn/README_cn.md)|基于Faster RCNN模型的经典实例分割模型|在原有Faster RCNN模型基础上添加分割分支，得到掩码结果，实现了掩码和类别预测关系的解藕。|[Mask R-CNN](https://arxiv.org/abs/1703.06870)
 [ICNet](https://github.com/PaddlePaddle/models/tree/develop/fluid/PaddleCV/icnet)|图像实时语义分割模型|即考虑了速度，也考虑了准确性，在高分辨率图像的准确性和低复杂度网络的效率之间获得平衡|[ICNet for Real-Time Semantic Segmentation on High-Resolution Images](https://arxiv.org/abs/1704.08545)
 [DCGAN](https://github.com/PaddlePaddle/models/tree/develop/fluid/PaddleCV/gan/c_gan)|图像生成模型|深度卷积生成对抗网络，将GAN和卷积网络结合起来，以解决GAN训练不稳定的问题|[Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks](https://arxiv.org/pdf/1511.06434.pdf)
 [ConditionalGAN](https://github.com/PaddlePaddle/models/tree/develop/fluid/PaddleCV/gan/c_gan)|图像生成模型|条件生成对抗网络，一种带条件约束的GAN，使用额外信息对模型增加条件，可以指导数据生成过程|[Conditional Generative Adversarial Nets](https://arxiv.org/abs/1411.1784)

fluid/AutoDL/LRC/README.md

+# LRC Local Rademachar Complexity Regularization
+Regularization of Deep Neural Networks(DNNs) for the sake of improving their generalization capability is important and chllenging. This directory contains image classification model based on a novel regularizer rooted in Local Rademacher Complexity (LRC). We appreciate the contribution by [DARTS](https://arxiv.org/abs/1806.09055) for our research. The regularization by LRC and DARTS are combined in this model on CIFAR-10 dataset. Code accompanying the paper
+> [An Empirical Study on Regularization of Deep Neural Networks by Local Rademacher Complexity](https://arxiv.org/abs/1902.00873)\
+> Yingzhen Yang, Xingjian Li, Jun Huan.\
+> _arXiv:1902.00873_.
+
+---
+# Table of Contents
+
+- [Installation](#installation)
+- [Data preparation](#data-preparation)
+- [Training](#training)
+
+## Installation
+
+Running sample code in this directory requires PaddelPaddle Fluid v.1.2.0 and later. If the PaddlePaddle on your device is lower than this version, please follow the instructions in [installation document](http://www.paddlepaddle.org/documentation/docs/zh/1.2/beginners_guide/install/index_cn.html#paddlepaddle) and make an update.
+
+## Data preparation
+
+When you want to use the cifar-10 dataset for the first time, you can download the dataset as:
+
+    sh ./dataset/download.sh
+
+Please make sure your environment has an internet connection.
+
+The dataset will be downloaded to `dataset/cifar/cifar-10-batches-py` in the same directory as the `train.py`. If automatic download fails, you can download cifar-10-python.tar.gz from https://www.cs.toronto.edu/~kriz/cifar.html and decompress it to the location mentioned above.
+
+
+## Training
+
+After data preparation, one can start the training step by:
+
+    python -u train_mixup.py \
+        --batch_size=80 \
+        --auxiliary \
+        --weight_decay=0.0003 \
+        --learning_rate=0.025 \
+        --lrc_loss_lambda=0.7 \
+        --cutout
+- Set ```export CUDA_VISIBLE_DEVICES=0``` to specifiy one GPU to train.
+- For more help on arguments:
+
+    python train_mixup.py --help
+
+**data reader introduction:**
+
+* Data reader is defined in `reader.py`.
+* Reshape the images to 32 * 32.
+* In training stage, images are padding to 40 * 40 and cropped randomly to the original size.
+* In training stage, images are horizontally random flipped.
+* Images are standardized to (0, 1).
+* In training stage, cutout images randomly.
+* Shuffle the order of the input images during training.
+
+**model configuration:**
+
+* Use auxiliary loss and auxiliary\_weight=0.4.
+* Use dropout and drop\_path\_prob=0.2.
+* Set lrc\_loss\_lambda=0.7.
+
+**training strategy:**
+
+*  Use momentum optimizer with momentum=0.9.
+*  Weight decay is 0.0003.
+*  Use cosine decay with init\_lr=0.025.
+*  Total epoch is 600.
+*  Use Xaiver initalizer to weight in conv2d, Constant initalizer to weight in batch norm and Normal initalizer to weight in fc.
+*  Initalize bias in batch norm and fc to zero constant and do not add bias to conv2d.
+
+
+## Reference
+
+  - DARTS: Differentiable Architecture Search [`paper`](https://arxiv.org/abs/1806.09055)
+  - Differentiable architecture search in PyTorch [`code`](https://github.com/quark0/darts)

fluid/AutoDL/LRC/README_cn.md

+# LRC 局部Rademachar复杂度正则化
+为了在深度神经网络中提升泛化能力，正则化的选择十分重要也具有挑战性。本目录包括了一种基于局部rademacher复杂度的新型正则（LRC）的图像分类模型。十分感谢[DARTS](https://arxiv.org/abs/1806.09055)模型对本研究提供的帮助。该模型将LRC正则和DARTS网络相结合，在CIFAR-10数据集中得到了很出色的效果。代码和文章一同发布
+> [An Empirical Study on Regularization of Deep Neural Networks by Local Rademacher Complexity](https://arxiv.org/abs/1902.00873)\
+> Yingzhen Yang, Xingjian Li, Jun Huan.\
+> _arXiv:1902.00873_.
+
+---
+# 内容
+
+- [安装](#安装)
+- [数据准备](#数据准备)
+- [模型训练](#模型训练)
+
+## 安装
+
+在当前目录下运行样例代码需要PadddlePaddle Fluid的v.1.2.0或以上的版本。如果你的运行环境中的PaddlePaddle低于此版本，请根据[安装文档](http://www.paddlepaddle.org/documentation/docs/zh/1.2/beginners_guide/install/index_cn.html#paddlepaddle)中的说明来更新PaddlePaddle。
+
+## 数据准备
+
+第一次使用CIFAR-10数据集时，您可以通过如果命令下载：
+
+    sh ./dataset/download.sh
+
+请确保您的环境有互联网连接。数据会下载到`train.py`同目录下的`dataset/cifar/cifar-10-batches-py`。如果下载失败，您可以自行从https://www.cs.toronto.edu/~kriz/cifar.html上下载cifar-10-python.tar.gz并解压到上述位置。
+
+## 模型训练
+
+数据准备好后，可以通过如下命令开始训练：
+
+    python -u train_mixup.py \
+        --batch_size=80 \
+        --auxiliary \
+        --weight_decay=0.0003 \
+        --learning_rate=0.025 \
+        --lrc_loss_lambda=0.7 \
+        --cutout
+- 通过设置 ```export CUDA_VISIBLE_DEVICES=0```指定单张GPU训练。
+- 可选参数见：
+
+    python train_mixup.py --help
+
+**数据读取器说明：**
+
+* 数据读取器定义在`reader.py`中
+* 输入图像尺寸统一变换为32 * 32
+* 训练时将图像填充为40 * 40然后随机剪裁为原输入图像大小
+* 训练时图像随机水平翻转
+* 对图像每个像素做归一化处理
+* 训练时对图像做随机遮挡
+* 训练时对输入图像做随机洗牌
+
+**模型配置：**
+
+* 使用辅助损失，辅助损失权重为0.4
+* 使用dropout，随机丢弃率为0.2
+* 设置lrc\_loss\_lambda为0.7
+
+**训练策略：**
+
+* 采用momentum优化算法训练，momentum=0.9
+* 权重衰减系数为0.0001
+* 采用正弦学习率衰减，初始学习率为0.025
+* 总共训练600轮
+* 对卷积权重采用Xaiver初始化，对batch norm权重采用固定初始化，对全连接层权重采用高斯初始化
+* 对batch norm和全连接层偏差采用固定初始化，不对卷积设置偏差
+
+
+## 引用
+
+  - DARTS: Differentiable Architecture Search [`论文`](https://arxiv.org/abs/1806.09055)
+  - Differentiable Architecture Search in PyTorch [`代码`](https://github.com/quark0/darts)

fluid/AutoDL/LRC/dataset/download.sh

+DIR="$( cd "$(dirname "$0")" ; pwd -P )"
+cd "$DIR"
+mkdir cifar
+cd cifar
+# Download the data.
+echo "Downloading..."
+wget https://www.cs.toronto.edu/~kriz/cifar-10-python.tar.gz
+# Extract the data.
+echo "Extracting..."
+tar zvxf cifar-10-python.tar.gz

fluid/AutoDL/LRC/genotypes.py

+# Copyright (c) 2019 PaddlePaddle Authors. All Rights Reserved
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+# Based on:
+# --------------------------------------------------------
+# DARTS
+# Copyright (c) 2018, Hanxiao Liu.
+# Licensed under the Apache License, Version 2.0;
+# --------------------------------------------------------
+
+from collections import namedtuple
+
+Genotype = namedtuple('Genotype', 'normal normal_concat reduce reduce_concat')
+
+PRIMITIVES = [
+    'none', 'max_pool_3x3', 'avg_pool_3x3', 'skip_connect', 'sep_conv_3x3',
+    'sep_conv_5x5', 'dil_conv_3x3', 'dil_conv_5x5'
+]
+
+NASNet = Genotype(
+    normal=[
+        ('sep_conv_5x5', 1),
+        ('sep_conv_3x3', 0),
+        ('sep_conv_5x5', 0),
+        ('sep_conv_3x3', 0),
+        ('avg_pool_3x3', 1),
+        ('skip_connect', 0),
+        ('avg_pool_3x3', 0),
+        ('avg_pool_3x3', 0),
+        ('sep_conv_3x3', 1),
+        ('skip_connect', 1),
+    ],
+    normal_concat=[2, 3, 4, 5, 6],
+    reduce=[
+        ('sep_conv_5x5', 1),
+        ('sep_conv_7x7', 0),
+        ('max_pool_3x3', 1),
+        ('sep_conv_7x7', 0),
+        ('avg_pool_3x3', 1),
+        ('sep_conv_5x5', 0),
+        ('skip_connect', 3),
+        ('avg_pool_3x3', 2),
+        ('sep_conv_3x3', 2),
+        ('max_pool_3x3', 1),
+    ],
+    reduce_concat=[4, 5, 6], )
+
+AmoebaNet = Genotype(
+    normal=[
+        ('avg_pool_3x3', 0),
+        ('max_pool_3x3', 1),
+        ('sep_conv_3x3', 0),
+        ('sep_conv_5x5', 2),
+        ('sep_conv_3x3', 0),
+        ('avg_pool_3x3', 3),
+        ('sep_conv_3x3', 1),
+        ('skip_connect', 1),
+        ('skip_connect', 0),
+        ('avg_pool_3x3', 1),
+    ],
+    normal_concat=[4, 5, 6],
+    reduce=[
+        ('avg_pool_3x3', 0),
+        ('sep_conv_3x3', 1),
+        ('max_pool_3x3', 0),
+        ('sep_conv_7x7', 2),
+        ('sep_conv_7x7', 0),
+        ('avg_pool_3x3', 1),
+        ('max_pool_3x3', 0),
+        ('max_pool_3x3', 1),
+        ('conv_7x1_1x7', 0),
+        ('sep_conv_3x3', 5),
+    ],
+    reduce_concat=[3, 4, 6])
+
+DARTS_V1 = Genotype(
+    normal=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('skip_connect', 0),
+            ('sep_conv_3x3', 1), ('skip_connect', 0), ('sep_conv_3x3', 1),
+            ('sep_conv_3x3', 0), ('skip_connect', 2)],
+    normal_concat=[2, 3, 4, 5],
+    reduce=[('max_pool_3x3', 0), ('max_pool_3x3', 1), ('skip_connect', 2),
+            ('max_pool_3x3', 0), ('max_pool_3x3', 0), ('skip_connect', 2),
+            ('skip_connect', 2), ('avg_pool_3x3', 0)],
+    reduce_concat=[2, 3, 4, 5])
+DARTS_V2 = Genotype(
+    normal=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0),
+            ('sep_conv_3x3', 1), ('sep_conv_3x3', 1), ('skip_connect', 0),
+            ('skip_connect', 0), ('dil_conv_3x3', 2)],
+    normal_concat=[2, 3, 4, 5],
+    reduce=[('max_pool_3x3', 0), ('max_pool_3x3', 1), ('skip_connect', 2),
+            ('max_pool_3x3', 1), ('max_pool_3x3', 0), ('skip_connect', 2),
+            ('skip_connect', 2), ('max_pool_3x3', 1)],
+    reduce_concat=[2, 3, 4, 5])
+
+MY_DARTS = Genotype(
+    normal=[('sep_conv_3x3', 0), ('skip_connect', 1), ('skip_connect', 0),
+            ('dil_conv_5x5', 1), ('skip_connect', 0), ('sep_conv_3x3', 1),
+            ('skip_connect', 0), ('sep_conv_3x3', 1)],
+    normal_concat=range(2, 6),
+    reduce=[('max_pool_3x3', 0), ('max_pool_3x3', 1), ('max_pool_3x3', 0),
+            ('skip_connect', 2), ('max_pool_3x3', 0), ('skip_connect', 2),
+            ('skip_connect', 2), ('skip_connect', 3)],
+    reduce_concat=range(2, 6))
+
+DARTS = MY_DARTS

fluid/AutoDL/LRC/learning_rate.py

+# Copyright (c) 2019 PaddlePaddle Authors. All Rights Reserved
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+# Based on:
+# --------------------------------------------------------
+# DARTS
+# Copyright (c) 2018, Hanxiao Liu.
+# Licensed under the Apache License, Version 2.0;
+# --------------------------------------------------------
+
+from __future__ import absolute_import
+from __future__ import division
+from __future__ import print_function
+import paddle
+import paddle.fluid as fluid
+import paddle.fluid.layers.ops as ops
+from paddle.fluid.layers.learning_rate_scheduler import _decay_step_counter
+import math
+from paddle.fluid.initializer import init_on_cpu
+
+
+def cosine_decay(learning_rate, num_epoch, steps_one_epoch):
+    """Applies cosine decay to the learning rate.
+    lr = 0.5 * (math.cos(epoch * (math.pi / 120)) + 1)
+    """
+    global_step = _decay_step_counter()
+
+    with init_on_cpu():
+        decayed_lr = learning_rate * \
+                 (ops.cos((global_step / steps_one_epoch) \
+                 * math.pi / num_epoch) + 1)/2
+    return decayed_lr

fluid/AutoDL/LRC/model.py

+#  Copyright (c) 2019 PaddlePaddle Authors. All Rights Reserve.
+#
+#Licensed under the Apache License, Version 2.0 (the "License");
+#you may not use this file except in compliance with the License.
+#You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+#Unless required by applicable law or agreed to in writing, software
+#distributed under the License is distributed on an "AS IS" BASIS,
+#WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#See the License for the specific language governing permissions and
+#limitations under the License.
+#
+# Based on:
+# --------------------------------------------------------
+# DARTS
+# Copyright (c) 2018, Hanxiao Liu.
+# Licensed under the Apache License, Version 2.0;
+# --------------------------------------------------------
+
+from __future__ import absolute_import
+from __future__ import division
+from __future__ import print_function
+import os
+import sys
+import numpy as np
+import time
+import functools
+import paddle
+import paddle.fluid as fluid
+from operations import *
+
+
+class Cell():
+    def __init__(self, genotype, C_prev_prev, C_prev, C, reduction,
+                 reduction_prev):
+        print(C_prev_prev, C_prev, C)
+
+        if reduction_prev:
+            self.preprocess0 = functools.partial(FactorizedReduce, C_out=C)
+        else:
+            self.preprocess0 = functools.partial(
+                ReLUConvBN, C_out=C, kernel_size=1, stride=1, padding=0)
+        self.preprocess1 = functools.partial(
+            ReLUConvBN, C_out=C, kernel_size=1, stride=1, padding=0)
+        if reduction:
+            op_names, indices = zip(*genotype.reduce)
+            concat = genotype.reduce_concat
+        else:
+            op_names, indices = zip(*genotype.normal)
+            concat = genotype.normal_concat
+        print(op_names, indices, concat, reduction)
+        self._compile(C, op_names, indices, concat, reduction)
+
+    def _compile(self, C, op_names, indices, concat, reduction):
+        assert len(op_names) == len(indices)
+        self._steps = len(op_names) // 2
+        self._concat = concat
+        self.multiplier = len(concat)
+
+        self._ops = []
+        for name, index in zip(op_names, indices):
+            stride = 2 if reduction and index < 2 else 1
+            op = functools.partial(OPS[name], C=C, stride=stride, affine=True)
+            self._ops += [op]
+        self._indices = indices
+
+    def forward(self, s0, s1, drop_prob, is_train, name):
+        self.training = is_train
+        preprocess0_name = name + 'preprocess0.'
+        preprocess1_name = name + 'preprocess1.'
+        s0 = self.preprocess0(s0, name=preprocess0_name)
+        s1 = self.preprocess1(s1, name=preprocess1_name)
+        out = [s0, s1]
+        for i in range(self._steps):
+            h1 = out[self._indices[2 * i]]
+            h2 = out[self._indices[2 * i + 1]]
+            op1 = self._ops[2 * i]
+            op2 = self._ops[2 * i + 1]
+            h3 = op1(h1, name=name + '_ops.' + str(2 * i) + '.')
+            h4 = op2(h2, name=name + '_ops.' + str(2 * i + 1) + '.')
+            if self.training and drop_prob > 0.:
+                if h3 != h1:
+                    h3 = fluid.layers.dropout(
+                        h3,
+                        drop_prob,
+                        dropout_implementation='upscale_in_train')
+                if h4 != h2:
+                    h4 = fluid.layers.dropout(
+                        h4,
+                        drop_prob,
+                        dropout_implementation='upscale_in_train')
+            s = h3 + h4
+            out += [s]
+        return fluid.layers.concat([out[i] for i in self._concat], axis=1)
+
+
+def AuxiliaryHeadCIFAR(input, num_classes, aux_name='auxiliary_head'):
+    relu_a = fluid.layers.relu(input)
+    pool_a = fluid.layers.pool2d(relu_a, 5, 'avg', 3)
+    conv2d_a = fluid.layers.conv2d(
+        pool_a,
+        128,
+        1,
+        name=aux_name + '.features.2',
+        param_attr=ParamAttr(
+            initializer=Xavier(
+                uniform=False, fan_in=0),
+            name=aux_name + '.features.2.weight'),
+        bias_attr=False)
+    bn_a_name = aux_name + '.features.3'
+    bn_a = fluid.layers.batch_norm(
+        conv2d_a,
+        act='relu',
+        name=bn_a_name,
+        param_attr=ParamAttr(
+            initializer=Constant(1.), name=bn_a_name + '.weight'),
+        bias_attr=ParamAttr(
+            initializer=Constant(0.), name=bn_a_name + '.bias'),
+        moving_mean_name=bn_a_name + '.running_mean',
+        moving_variance_name=bn_a_name + '.running_var')
+    conv2d_b = fluid.layers.conv2d(
+        bn_a,
+        768,
+        2,
+        name=aux_name + '.features.5',
+        param_attr=ParamAttr(
+            initializer=Xavier(
+                uniform=False, fan_in=0),
+            name=aux_name + '.features.5.weight'),
+        bias_attr=False)
+    bn_b_name = aux_name + '.features.6'
+    bn_b = fluid.layers.batch_norm(
+        conv2d_b,
+        act='relu',
+        name=bn_b_name,
+        param_attr=ParamAttr(
+            initializer=Constant(1.), name=bn_b_name + '.weight'),
+        bias_attr=ParamAttr(
+            initializer=Constant(0.), name=bn_b_name + '.bias'),
+        moving_mean_name=bn_b_name + '.running_mean',
+        moving_variance_name=bn_b_name + '.running_var')
+    fc_name = aux_name + '.classifier'
+    fc = fluid.layers.fc(bn_b,
+                         num_classes,
+                         name=fc_name,
+                         param_attr=ParamAttr(
+                             initializer=Normal(scale=1e-3),
+                             name=fc_name + '.weight'),
+                         bias_attr=ParamAttr(
+                             initializer=Constant(0.), name=fc_name + '.bias'))
+    return fc
+
+
+def StemConv(input, C_out, kernel_size, padding):
+    conv_a = fluid.layers.conv2d(
+        input,
+        C_out,
+        kernel_size,
+        padding=padding,
+        param_attr=ParamAttr(
+            initializer=Xavier(
+                uniform=False, fan_in=0), name='stem.0.weight'),
+        bias_attr=False)
+    bn_a = fluid.layers.batch_norm(
+        conv_a,
+        param_attr=ParamAttr(
+            initializer=Constant(1.), name='stem.1.weight'),
+        bias_attr=ParamAttr(
+            initializer=Constant(0.), name='stem.1.bias'),
+        moving_mean_name='stem.1.running_mean',
+        moving_variance_name='stem.1.running_var')
+    return bn_a
+
+
+class NetworkCIFAR(object):
+    def __init__(self, C, class_num, layers, auxiliary, genotype):
+        self.class_num = class_num
+        self._layers = layers
+        self._auxiliary = auxiliary
+
+        stem_multiplier = 3
+        self.drop_path_prob = 0
+        C_curr = stem_multiplier * C
+
+        C_prev_prev, C_prev, C_curr = C_curr, C_curr, C
+        self.cells = []
+        reduction_prev = False
+        for i in range(layers):
+            if i in [layers // 3, 2 * layers // 3]:
+                C_curr *= 2
+                reduction = True
+            else:
+                reduction = False
+            cell = Cell(genotype, C_prev_prev, C_prev, C_curr, reduction,
+                        reduction_prev)
+            reduction_prev = reduction
+            self.cells += [cell]
+            C_prev_prev, C_prev = C_prev, cell.multiplier * C_curr
+            if i == 2 * layers // 3:
+                C_to_auxiliary = C_prev
+
+    def forward(self, init_channel, is_train):
+        self.training = is_train
+        self.logits_aux = None
+        num_channel = init_channel * 3
+        s0 = StemConv(self.image, num_channel, kernel_size=3, padding=1)
+        s1 = s0
+        for i, cell in enumerate(self.cells):
+            name = 'cells.' + str(i) + '.'
+            s0, s1 = s1, cell.forward(s0, s1, self.drop_path_prob, is_train,
+                                      name)
+            if i == int(2 * self._layers // 3):
+                if self._auxiliary and self.training:
+                    self.logits_aux = AuxiliaryHeadCIFAR(s1, self.class_num)
+        out = fluid.layers.adaptive_pool2d(s1, (1, 1), "avg")
+        self.logits = fluid.layers.fc(out,
+                                      size=self.class_num,
+                                      param_attr=ParamAttr(
+                                          initializer=Normal(scale=1e-3),
+                                          name='classifier.weight'),
+                                      bias_attr=ParamAttr(
+                                          initializer=Constant(0.),
+                                          name='classifier.bias'))
+        return self.logits, self.logits_aux
+
+    def build_input(self, image_shape, batch_size, is_train):
+        if is_train:
+            py_reader = fluid.layers.py_reader(
+                capacity=64,
+                shapes=[[-1] + image_shape, [-1, 1], [-1, 1], [-1, 1], [-1, 1],
+                        [-1, 1], [-1, batch_size, self.class_num - 1]],
+                lod_levels=[0, 0, 0, 0, 0, 0, 0],
+                dtypes=[
+                    "float32", "int64", "int64", "float32", "int32", "int32",
+                    "float32"
+                ],
+                use_double_buffer=True,
+                name='train_reader')
+        else:
+            py_reader = fluid.layers.py_reader(
+                capacity=64,
+                shapes=[[-1] + image_shape, [-1, 1]],
+                lod_levels=[0, 0],
+                dtypes=["float32", "int64"],
+                use_double_buffer=True,
+                name='test_reader')
+        return py_reader
+
+    def train_model(self, py_reader, init_channels, aux, aux_w, batch_size,
+                    loss_lambda):
+        self.image, self.ya, self.yb, self.lam, self.label_reshape,\
+           self.non_label_reshape, self.rad_var = fluid.layers.read_file(py_reader)
+        self.logits, self.logits_aux = self.forward(init_channels, True)
+        self.mixup_loss = self.mixup_loss(aux, aux_w)
+        self.lrc_loss = self.lrc_loss(batch_size)
+        return self.mixup_loss + loss_lambda * self.lrc_loss
+
+    def test_model(self, py_reader, init_channels):
+        self.image, self.ya = fluid.layers.read_file(py_reader)
+        self.logits, _ = self.forward(init_channels, False)
+        prob = fluid.layers.softmax(self.logits, use_cudnn=False)
+        loss = fluid.layers.cross_entropy(prob, self.ya)
+        acc_1 = fluid.layers.accuracy(self.logits, self.ya, k=1)
+        acc_5 = fluid.layers.accuracy(self.logits, self.ya, k=5)
+        return loss, acc_1, acc_5
+
+    def mixup_loss(self, auxiliary, auxiliary_weight):
+        prob = fluid.layers.softmax(self.logits, use_cudnn=False)
+        loss_a = fluid.layers.cross_entropy(prob, self.ya)
+        loss_b = fluid.layers.cross_entropy(prob, self.yb)
+        loss_a_mean = fluid.layers.reduce_mean(loss_a)
+        loss_b_mean = fluid.layers.reduce_mean(loss_b)
+        loss = self.lam * loss_a_mean + (1 - self.lam) * loss_b_mean
+        if auxiliary:
+            prob_aux = fluid.layers.softmax(self.logits_aux, use_cudnn=False)
+            loss_a_aux = fluid.layers.cross_entropy(prob_aux, self.ya)
+            loss_b_aux = fluid.layers.cross_entropy(prob_aux, self.yb)
+            loss_a_aux_mean = fluid.layers.reduce_mean(loss_a_aux)
+            loss_b_aux_mean = fluid.layers.reduce_mean(loss_b_aux)
+            loss_aux = self.lam * loss_a_aux_mean + (1 - self.lam
+                                                     ) * loss_b_aux_mean
+        return loss + auxiliary_weight * loss_aux
+
+    def lrc_loss(self, batch_size):
+        y_diff_reshape = fluid.layers.reshape(self.logits, shape=(-1, 1))
+        label_reshape = fluid.layers.squeeze(self.label_reshape, axes=[1])
+        non_label_reshape = fluid.layers.squeeze(
+            self.non_label_reshape, axes=[1])
+        label_reshape.stop_gradient = True
+        non_label_reshape.stop_graident = True
+
+        y_diff_label_reshape = fluid.layers.gather(y_diff_reshape,
+                                                   label_reshape)
+        y_diff_non_label_reshape = fluid.layers.gather(y_diff_reshape,
+                                                       non_label_reshape)
+        y_diff_label = fluid.layers.reshape(
+            y_diff_label_reshape, shape=(-1, batch_size, 1))
+        y_diff_non_label = fluid.layers.reshape(
+            y_diff_non_label_reshape,
+            shape=(-1, batch_size, self.class_num - 1))
+        y_diff_ = y_diff_non_label - y_diff_label
+
+        y_diff_ = fluid.layers.transpose(y_diff_, perm=[1, 2, 0])
+        rad_var_trans = fluid.layers.transpose(self.rad_var, perm=[1, 2, 0])
+        rad_y_diff_trans = rad_var_trans * y_diff_
+        lrc_loss_sum = fluid.layers.reduce_sum(rad_y_diff_trans, dim=[0, 1])
+        lrc_loss_ = fluid.layers.abs(lrc_loss_sum) / (batch_size *
+                                                      (self.class_num - 1))
+        lrc_loss_mean = fluid.layers.reduce_mean(lrc_loss_)
+
+        return lrc_loss_mean

fluid/AutoDL/LRC/operations.py

+#  Copyright (c) 2019 PaddlePaddle Authors. All Rights Reserve.
+#
+#Licensed under the Apache License, Version 2.0 (the "License");
+#you may not use this file except in compliance with the License.
+#You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+#Unless required by applicable law or agreed to in writing, software
+#distributed under the License is distributed on an "AS IS" BASIS,
+#WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#See the License for the specific language governing permissions and
+#limitations under the License.
+#
+# Based on:
+# --------------------------------------------------------
+# DARTS
+# Copyright (c) 2018, Hanxiao Liu.
+# Licensed under the Apache License, Version 2.0;
+# --------------------------------------------------------
+from __future__ import absolute_import
+from __future__ import division
+from __future__ import print_function
+import os
+import sys
+import numpy as np
+import time
+import paddle
+import paddle.fluid as fluid
+from paddle.fluid.param_attr import ParamAttr
+from paddle.fluid.initializer import Xavier
+from paddle.fluid.initializer import Normal
+from paddle.fluid.initializer import Constant
+
+OPS = {
+    'none' : lambda input, C, stride, name, affine: Zero(input, stride, name),
+    'avg_pool_3x3' : lambda input, C, stride, name, affine: fluid.layers.pool2d(input, 3, 'avg', pool_stride=stride, pool_padding=1, name=name),
+    'max_pool_3x3' : lambda input, C, stride, name, affine: fluid.layers.pool2d(input, 3, 'max', pool_stride=stride, pool_padding=1, name=name),
+    'skip_connect' : lambda input,C, stride, name, affine: Identity(input, name) if stride == 1 else FactorizedReduce(input, C, name=name, affine=affine),
+    'sep_conv_3x3' : lambda input,C, stride, name, affine: SepConv(input, C, C, 3, stride, 1, name=name, affine=affine),
+    'sep_conv_5x5' : lambda input,C, stride, name, affine: SepConv(input, C, C, 5, stride, 2, name=name, affine=affine),
+    'sep_conv_7x7' : lambda input,C, stride, name, affine: SepConv(input, C, C, 7, stride, 3, name=name, affine=affine),
+    'dil_conv_3x3' : lambda input,C, stride, name, affine: DilConv(input, C, C, 3, stride, 2, 2, name=name, affine=affine),
+    'dil_conv_5x5' : lambda input,C, stride, name, affine: DilConv(input, C, C, 5, stride, 4, 2, name=name, affine=affine),
+    'conv_7x1_1x7' : lambda input,C, stride, name, affine: SevenConv(input, C, name=name, affine=affine)
+}
+
+
+def ReLUConvBN(input, C_out, kernel_size, stride, padding, name='',
+               affine=True):
+    relu_a = fluid.layers.relu(input)
+    conv2d_a = fluid.layers.conv2d(
+        relu_a,
+        C_out,
+        kernel_size,
+        stride,
+        padding,
+        param_attr=ParamAttr(
+            initializer=Xavier(
+                uniform=False, fan_in=0),
+            name=name + 'op.1.weight'),
+        bias_attr=False)
+    if affine:
+        reluconvbn_out = fluid.layers.batch_norm(
+            conv2d_a,
+            param_attr=ParamAttr(
+                initializer=Constant(1.), name=name + 'op.2.weight'),
+            bias_attr=ParamAttr(
+                initializer=Constant(0.), name=name + 'op.2.bias'),
+            moving_mean_name=name + 'op.2.running_mean',
+            moving_variance_name=name + 'op.2.running_var')
+    else:
+        reluconvbn_out = fluid.layers.batch_norm(
+            conv2d_a,
+            param_attr=ParamAttr(
+                initializer=Constant(1.),
+                learning_rate=0.,
+                name=name + 'op.2.weight'),
+            bias_attr=ParamAttr(
+                initializer=Constant(0.),
+                learning_rate=0.,
+                name=name + 'op.2.bias'),
+            moving_mean_name=name + 'op.2.running_mean',
+            moving_variance_name=name + 'op.2.running_var')
+    return reluconvbn_out
+
+
+def DilConv(input,
+            C_in,
+            C_out,
+            kernel_size,
+            stride,
+            padding,
+            dilation,
+            name='',
+            affine=True):
+    relu_a = fluid.layers.relu(input)
+    conv2d_a = fluid.layers.conv2d(
+        relu_a,
+        C_in,
+        kernel_size,
+        stride,
+        padding,
+        dilation,
+        groups=C_in,
+        param_attr=ParamAttr(
+            initializer=Xavier(
+                uniform=False, fan_in=0),
+            name=name + 'op.1.weight'),
+        bias_attr=False,
+        use_cudnn=False)
+    conv2d_b = fluid.layers.conv2d(
+        conv2d_a,
+        C_out,
+        1,
+        param_attr=ParamAttr(
+            initializer=Xavier(
+                uniform=False, fan_in=0),
+            name=name + 'op.2.weight'),
+        bias_attr=False)
+    if affine:
+        dilconv_out = fluid.layers.batch_norm(
+            conv2d_b,
+            param_attr=ParamAttr(
+                initializer=Constant(1.), name=name + 'op.3.weight'),
+            bias_attr=ParamAttr(
+                initializer=Constant(0.), name=name + 'op.3.bias'),
+            moving_mean_name=name + 'op.3.running_mean',
+            moving_variance_name=name + 'op.3.running_var')
+    else:
+        dilconv_out = fluid.layers.batch_norm(
+            conv2d_b,
+            param_attr=ParamAttr(
+                initializer=Constant(1.),
+                learning_rate=0.,
+                name=name + 'op.3.weight'),
+            bias_attr=ParamAttr(
+                initializer=Constant(0.),
+                learning_rate=0.,
+                name=name + 'op.3.bias'),
+            moving_mean_name=name + 'op.3.running_mean',
+            moving_variance_name=name + 'op.3.running_var')
+    return dilconv_out
+
+
+def SepConv(input,
+            C_in,
+            C_out,
+            kernel_size,
+            stride,
+            padding,
+            name='',
+            affine=True):
+    relu_a = fluid.layers.relu(input)
+    conv2d_a = fluid.layers.conv2d(
+        relu_a,
+        C_in,
+        kernel_size,
+        stride,
+        padding,
+        groups=C_in,
+        param_attr=ParamAttr(
+            initializer=Xavier(
+                uniform=False, fan_in=0),
+            name=name + 'op.1.weight'),
+        bias_attr=False,
+        use_cudnn=False)
+    conv2d_b = fluid.layers.conv2d(
+        conv2d_a,
+        C_in,
+        1,
+        param_attr=ParamAttr(
+            initializer=Xavier(
+                uniform=False, fan_in=0),
+            name=name + 'op.2.weight'),
+        bias_attr=False)
+    if affine:
+        bn_a = fluid.layers.batch_norm(
+            conv2d_b,
+            param_attr=ParamAttr(
+                initializer=Constant(1.), name=name + 'op.3.weight'),
+            bias_attr=ParamAttr(
+                initializer=Constant(0.), name=name + 'op.3.bias'),
+            moving_mean_name=name + 'op.3.running_mean',
+            moving_variance_name=name + 'op.3.running_var')
+    else:
+        bn_a = fluid.layers.batch_norm(
+            conv2d_b,
+            param_attr=ParamAttr(
+                initializer=Constant(1.),
+                learning_rate=0.,
+                name=name + 'op.3.weight'),
+            bias_attr=ParamAttr(
+                initializer=Constant(0.),
+                learning_rate=0.,
+                name=name + 'op.3.bias'),
+            moving_mean_name=name + 'op.3.running_mean',
+            moving_variance_name=name + 'op.3.running_var')
+
+    relu_b = fluid.layers.relu(bn_a)
+    conv2d_d = fluid.layers.conv2d(
+        relu_b,
+        C_in,
+        kernel_size,
+        1,
+        padding,
+        groups=C_in,
+        param_attr=ParamAttr(
+            initializer=Xavier(
+                uniform=False, fan_in=0),
+            name=name + 'op.5.weight'),
+        bias_attr=False,
+        use_cudnn=False)
+    conv2d_e = fluid.layers.conv2d(
+        conv2d_d,
+        C_out,
+        1,
+        param_attr=ParamAttr(
+            initializer=Xavier(
+                uniform=False, fan_in=0),
+            name=name + 'op.6.weight'),
+        bias_attr=False)
+    if affine:
+        sepconv_out = fluid.layers.batch_norm(
+            conv2d_e,
+            param_attr=ParamAttr(
+                initializer=Constant(1.), name=name + 'op.7.weight'),
+            bias_attr=ParamAttr(
+                initializer=Constant(0.), name=name + 'op.7.bias'),
+            moving_mean_name=name + 'op.7.running_mean',
+            moving_variance_name=name + 'op.7.running_var')
+    else:
+        sepconv_out = fluid.layers.batch_norm(
+            conv2d_e,
+            param_attr=ParamAttr(
+                initializer=Constant(1.),
+                learning_rate=0.,
+                name=name + 'op.7.weight'),
+            bias_attr=ParamAttr(
+                initializer=Constant(0.),
+                learning_rate=0.,
+                name=name + 'op.7.bias'),
+            moving_mean_name=name + 'op.7.running_mean',
+            moving_variance_name=name + 'op.7.running_var')
+    return sepconv_out
+
+
+def SevenConv(input, C_out, stride, name='', affine=True):
+    relu_a = fluid.layers.relu(input)
+    conv2d_a = fluid.layers.conv2d(
+        relu_a,
+        C_out, (1, 7), (1, stride), (0, 3),
+        param_attr=ParamAttr(
+            initializer=Xavier(
+                uniform=False, fan_in=0),
+            name=name + 'op.1.weight'),
+        bias_attr=False)
+    conv2d_b = fluid.layers.conv2d(
+        conv2d_a,
+        C_out, (7, 1), (stride, 1), (3, 0),
+        param_attr=ParamAttr(
+            initializer=Xavier(
+                uniform=False, fan_in=0),
+            name=name + 'op.2.weight'),
+        bias_attr=False)
+    if affine:
+        out = fluid.layers.batch_norm(
+            conv2d_b,
+            param_attr=ParamAttr(
+                initializer=Constant(1.), name=name + 'op.3.weight'),
+            bias_attr=ParamAttr(
+                initializer=Constant(0.), name=name + 'op.3.bias'),
+            moving_mean_name=name + 'op.3.running_mean',
+            moving_variance_name=name + 'op.3.running_var')
+    else:
+        out = fluid.layers.batch_norm(
+            conv2d_b,
+            param_attr=ParamAttr(
+                initializer=Constant(1.),
+                learning_rate=0.,
+                name=name + 'op.3.weight'),
+            bias_attr=ParamAttr(
+                initializer=Constant(0.),
+                learning_rate=0.,
+                name=name + 'op.3.bias'),
+            moving_mean_name=name + 'op.3.running_mean',
+            moving_variance_name=name + 'op.3.running_var')
+
+
+def Identity(input, name=''):
+    return input
+
+
+def Zero(input, stride, name=''):
+    ones = np.ones(input.shape[-2:])
+    ones[::stride, ::stride] = 0
+    ones = fluid.layers.assign(ones)
+    return input * ones
+
+
+def FactorizedReduce(input, C_out, name='', affine=True):
+    relu_a = fluid.layers.relu(input)
+    conv2d_a = fluid.layers.conv2d(
+        relu_a,
+        C_out // 2,
+        1,
+        2,
+        param_attr=ParamAttr(
+            initializer=Xavier(
+                uniform=False, fan_in=0),
+            name=name + 'conv_1.weight'),
+        bias_attr=False)
+    h_end = relu_a.shape[2]
+    w_end = relu_a.shape[3]
+    slice_a = fluid.layers.slice(relu_a, [2, 3], [1, 1], [h_end, w_end])
+    conv2d_b = fluid.layers.conv2d(
+        slice_a,
+        C_out // 2,
+        1,
+        2,
+        param_attr=ParamAttr(
+            initializer=Xavier(
+                uniform=False, fan_in=0),
+            name=name + 'conv_2.weight'),
+        bias_attr=False)
+    out = fluid.layers.concat([conv2d_a, conv2d_b], axis=1)
+    if affine:
+        out = fluid.layers.batch_norm(
+            out,
+            param_attr=ParamAttr(
+                initializer=Constant(1.), name=name + 'bn.weight'),
+            bias_attr=ParamAttr(
+                initializer=Constant(0.), name=name + 'bn.bias'),
+            moving_mean_name=name + 'bn.running_mean',
+            moving_variance_name=name + 'bn.running_var')
+    else:
+        out = fluid.layers.batch_norm(
+            out,
+            param_attr=ParamAttr(
+                initializer=Constant(1.),
+                learning_rate=0.,
+                name=name + 'bn.weight'),
+            bias_attr=ParamAttr(
+                initializer=Constant(0.),
+                learning_rate=0.,
+                name=name + 'bn.bias'),
+            moving_mean_name=name + 'bn.running_mean',
+            moving_variance_name=name + 'bn.running_var')
+    return out

fluid/AutoDL/LRC/reader.py

+# Copyright (c) 2019 PaddlePaddle Authors. All Rig hts Reserved
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+# Based on:
+# --------------------------------------------------------
+# DARTS
+# Copyright (c) 2018, Hanxiao Liu.
+# Licensed under the Apache License, Version 2.0;
+# --------------------------------------------------------
+"""
+CIFAR-10 dataset.
+This module will download dataset from
+https://www.cs.toronto.edu/~kriz/cifar.html and parse train/test set into
+paddle reader creators.
+The CIFAR-10 dataset consists of 60000 32x32 colour images in 10 classes,
+with 6000 images per class. There are 50000 training images and 10000 test images.
+"""
+
+from PIL import Image
+from PIL import ImageOps
+import numpy as np
+
+import cPickle
+import random
+import utils
+import paddle.fluid as fluid
+import time
+import os
+import functools
+import paddle.reader
+
+__all__ = ['train10', 'test10']
+
+image_size = 32
+image_depth = 3
+half_length = 8
+
+CIFAR_MEAN = [0.4914, 0.4822, 0.4465]
+CIFAR_STD = [0.24703233, 0.24348505, 0.26158768]
+
+
+def generate_reshape_label(label, batch_size, CIFAR_CLASSES=10):
+    reshape_label = np.zeros((batch_size, 1), dtype='int32')
+    reshape_non_label = np.zeros(
+        (batch_size * (CIFAR_CLASSES - 1), 1), dtype='int32')
+    num = 0
+    for i in range(batch_size):
+        label_i = label[i]
+        reshape_label[i] = label_i + i * CIFAR_CLASSES
+        for j in range(CIFAR_CLASSES):
+            if label_i != j:
+                reshape_non_label[num] = \
+                          j + i * CIFAR_CLASSES
+                num += 1
+    return reshape_label, reshape_non_label
+
+
+def generate_bernoulli_number(batch_size, CIFAR_CLASSES=10):
+    rcc_iters = 50
+    rad_var = np.zeros((rcc_iters, batch_size, CIFAR_CLASSES - 1))
+    for i in range(rcc_iters):
+        bernoulli_num = np.random.binomial(size=batch_size, n=1, p=0.5)
+        bernoulli_map = np.array([])
+        ones = np.ones((CIFAR_CLASSES - 1, 1))
+        for batch_id in range(batch_size):
+            num = bernoulli_num[batch_id]
+            var_id = 2 * ones * num - 1
+            bernoulli_map = np.append(bernoulli_map, var_id)
+        rad_var[i] = bernoulli_map.reshape((batch_size, CIFAR_CLASSES - 1))
+    return rad_var.astype('float32')
+
+
+def preprocess(sample, is_training, args):
+    image_array = sample.reshape(3, image_size, image_size)
+    rgb_array = np.transpose(image_array, (1, 2, 0))
+    img = Image.fromarray(rgb_array, 'RGB')
+
+    if is_training:
+        # pad and ramdom crop
+        img = ImageOps.expand(img, (4, 4, 4, 4), fill=0)  # pad to 40 * 40 * 3
+        left_top = np.random.randint(9, size=2)  # rand 0 - 8
+        img = img.crop((left_top[0], left_top[1], left_top[0] + image_size,
+                        left_top[1] + image_size))
+        if np.random.randint(2):
+            img = img.transpose(Image.FLIP_LEFT_RIGHT)
+
+    img = np.array(img).astype(np.float32)
+
+    # per_image_standardization
+    img_float = img / 255.0
+    img = (img_float - CIFAR_MEAN) / CIFAR_STD
+
+    if is_training and args.cutout:
+        center = np.random.randint(image_size, size=2)
+        offset_width = max(0, center[0] - half_length)
+        offset_height = max(0, center[1] - half_length)
+        target_width = min(center[0] + half_length, image_size)
+        target_height = min(center[1] + half_length, image_size)
+
+        for i in range(offset_height, target_height):
+            for j in range(offset_width, target_width):
+                img[i][j][:] = 0.0
+
+    img = np.transpose(img, (2, 0, 1))
+    return img
+
+
+def reader_creator_filepath(filename, sub_name, is_training, args):
+    files = os.listdir(filename)
+    names = [each_item for each_item in files if sub_name in each_item]
+    names.sort()
+    datasets = []
+    for name in names:
+        print("Reading file " + name)
+        batch = cPickle.load(open(filename + name, 'rb'))
+        data = batch['data']
+        labels = batch.get('labels', batch.get('fine_labels', None))
+        assert labels is not None
+        dataset = zip(data, labels)
+        datasets.extend(dataset)
+    random.shuffle(datasets)
+
+    def read_batch(datasets, args):
+        for sample, label in datasets:
+            im = preprocess(sample, is_training, args)
+            yield im, [int(label)]
+
+    def reader():
+        batch_data = []
+        batch_label = []
+        for data, label in read_batch(datasets, args):
+            batch_data.append(data)
+            batch_label.append(label)
+            if len(batch_data) == args.batch_size:
+                batch_data = np.array(batch_data, dtype='float32')
+                batch_label = np.array(batch_label, dtype='int64')
+                if is_training:
+                    flatten_label, flatten_non_label = \
+                      generate_reshape_label(batch_label, args.batch_size)
+                    rad_var = generate_bernoulli_number(args.batch_size)
+                    mixed_x, y_a, y_b, lam = utils.mixup_data(
+                        batch_data, batch_label, args.batch_size,
+                        args.mix_alpha)
+                    batch_out = [[mixed_x, y_a, y_b, lam, flatten_label, \
+                                flatten_non_label, rad_var]]
+                    yield batch_out
+                else:
+                    batch_out = [[batch_data, batch_label]]
+                    yield batch_out
+                batch_data = []
+                batch_label = []
+
+    return reader
+
+
+def train10(args):
+    """
+    CIFAR-10 training set creator.
+    It returns a reader creator, each sample in the reader is image pixels in
+    [0, 1] and label in [0, 9].
+    :return: Training reader creator
+    :rtype: callable
+    """
+
+    return reader_creator_filepath(args.data, 'data_batch', True, args)
+
+
+def test10(args):
+    """
+    CIFAR-10 test set creator.
+    It returns a reader creator, each sample in the reader is image pixels in
+    [0, 1] and label in [0, 9].
+    :return: Test reader creator.
+    :rtype: callable
+    """
+    return reader_creator_filepath(args.data, 'test_batch', False, args)

fluid/AutoDL/LRC/run.sh

+CUDA_VISIBLE_DEVICES=0 python -u train_mixup.py \
+--batch_size=80 \
+--auxiliary \
+--weight_decay=0.0003 \
+--learning_rate=0.025 \
+--lrc_loss_lambda=0.7 \
+--cutout
+

fluid/AutoDL/LRC/train_mixup.py

+#  Copyright (c) 2019 PaddlePaddle Authors. All Rights Reserve.
+#
+#Licensed under the Apache License, Version 2.0 (the "License");
+#you may not use this file except in compliance with the License.
+#You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+#Unless required by applicable law or agreed to in writing, software
+#distributed under the License is distributed on an "AS IS" BASIS,
+#WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#See the License for the specific language governing permissions and
+#limitations under the License.
+#
+# Based on:
+# --------------------------------------------------------
+# DARTS
+# Copyright (c) 2018, Hanxiao Liu.
+# Licensed under the Apache License, Version 2.0;
+# --------------------------------------------------------
+
+from __future__ import absolute_import
+from __future__ import division
+from __future__ import print_function
+from learning_rate import cosine_decay
+import numpy as np
+import argparse
+from model import NetworkCIFAR as Network
+import reader
+import sys
+import os
+import time
+import logging
+import genotypes
+import paddle.fluid as fluid
+import shutil
+import utils
+import cPickle as cp
+
+parser = argparse.ArgumentParser("cifar")
+parser.add_argument(
+    '--data',
+    type=str,
+    default='./dataset/cifar/cifar-10-batches-py/',
+    help='location of the data corpus')
+parser.add_argument('--batch_size', type=int, default=96, help='batch size')
+parser.add_argument(
+    '--learning_rate', type=float, default=0.025, help='init learning rate')
+parser.add_argument('--momentum', type=float, default=0.9, help='momentum')
+parser.add_argument(
+    '--weight_decay', type=float, default=3e-4, help='weight decay')
+parser.add_argument(
+    '--report_freq', type=float, default=50, help='report frequency')
+parser.add_argument(
+    '--epochs', type=int, default=600, help='num of training epochs')
+parser.add_argument(
+    '--init_channels', type=int, default=36, help='num of init channels')
+parser.add_argument(
+    '--layers', type=int, default=20, help='total number of layers')
+parser.add_argument(
+    '--model_path',
+    type=str,
+    default='saved_models',
+    help='path to save the model')
+parser.add_argument(
+    '--auxiliary',
+    action='store_true',
+    default=False,
+    help='use auxiliary tower')
+parser.add_argument(
+    '--auxiliary_weight',
+    type=float,
+    default=0.4,
+    help='weight for auxiliary loss')
+parser.add_argument(
+    '--cutout', action='store_true', default=False, help='use cutout')
+parser.add_argument(
+    '--cutout_length', type=int, default=16, help='cutout length')
+parser.add_argument(
+    '--drop_path_prob', type=float, default=0.2, help='drop path probability')
+parser.add_argument('--save', type=str, default='EXP', help='experiment name')
+parser.add_argument(
+    '--arch', type=str, default='DARTS', help='which architecture to use')
+parser.add_argument(
+    '--grad_clip', type=float, default=5, help='gradient clipping')
+parser.add_argument(
+    '--lr_exp_decay',
+    action='store_true',
+    default=False,
+    help='use exponential_decay learning_rate')
+parser.add_argument('--mix_alpha', type=float, default=0.5, help='mixup alpha')
+parser.add_argument(
+    '--lrc_loss_lambda', default=0, type=float, help='lrc_loss_lambda')
+parser.add_argument(
+    '--loss_type',
+    default=1,
+    type=float,
+    help='loss_type 0: cross entropy 1: multi margin loss 2: max margin loss')
+
+args = parser.parse_args()
+
+CIFAR_CLASSES = 10
+dataset_train_size = 50000
+image_size = 32
+
+
+def main():
+    image_shape = [3, image_size, image_size]
+    devices = os.getenv("CUDA_VISIBLE_DEVICES") or ""
+    devices_num = len(devices.split(","))
+    logging.info("args = %s", args)
+    genotype = eval("genotypes.%s" % args.arch)
+    model = Network(args.init_channels, CIFAR_CLASSES, args.layers,
+                    args.auxiliary, genotype)
+    steps_one_epoch = dataset_train_size / (devices_num * args.batch_size)
+    train(model, args, image_shape, steps_one_epoch)
+
+
+def build_program(main_prog, startup_prog, args, is_train, model, im_shape,
+                  steps_one_epoch):
+    out = []
+    with fluid.program_guard(main_prog, startup_prog):
+        py_reader = model.build_input(im_shape, args.batch_size, is_train)
+        if is_train:
+            with fluid.unique_name.guard():
+                loss = model.train_model(py_reader, args.init_channels,
+                                         args.auxiliary, args.auxiliary_weight,
+                                         args.batch_size, args.lrc_loss_lambda)
+                optimizer = fluid.optimizer.Momentum(
+                        learning_rate=cosine_decay(args.learning_rate, \
+                            args.epochs, steps_one_epoch),
+                        regularization=fluid.regularizer.L2Decay(\
+                            args.weight_decay),
+                        momentum=args.momentum)
+                optimizer.minimize(loss)
+                out = [py_reader, loss]
+        else:
+            with fluid.unique_name.guard():
+                loss, acc_1, acc_5 = model.test_model(py_reader,
+                                                      args.init_channels)
+                out = [py_reader, loss, acc_1, acc_5]
+    return out
+
+
+def train(model, args, im_shape, steps_one_epoch):
+    train_startup_prog = fluid.Program()
+    test_startup_prog = fluid.Program()
+    train_prog = fluid.Program()
+    test_prog = fluid.Program()
+
+    train_py_reader, loss_train = build_program(train_prog, train_startup_prog,
+                                                args, True, model, im_shape,
+                                                steps_one_epoch)
+
+    test_py_reader, loss_test, acc_1, acc_5 = build_program(
+        test_prog, test_startup_prog, args, False, model, im_shape,
+        steps_one_epoch)
+
+    test_prog = test_prog.clone(for_test=True)
+
+    place = fluid.CUDAPlace(0)
+    exe = fluid.Executor(place)
+    exe.run(train_startup_prog)
+    exe.run(test_startup_prog)
+
+    exec_strategy = fluid.ExecutionStrategy()
+    exec_strategy.num_threads = 1
+    train_exe = fluid.ParallelExecutor(
+        main_program=train_prog,
+        use_cuda=True,
+        loss_name=loss_train.name,
+        exec_strategy=exec_strategy)
+    train_reader = reader.train10(args)
+    test_reader = reader.test10(args)
+    train_py_reader.decorate_paddle_reader(train_reader)
+    test_py_reader.decorate_paddle_reader(test_reader)
+
+    fluid.clip.set_gradient_clip(fluid.clip.GradientClipByNorm(args.grad_clip))
+    fluid.memory_optimize(fluid.default_main_program())
+
+    def save_model(postfix, main_prog):
+        model_path = os.path.join(args.model_path, postfix)
+        if os.path.isdir(model_path):
+            shutil.rmtree(model_path)
+        fluid.io.save_persistables(exe, model_path, main_program=main_prog)
+
+    def test(epoch_id):
+        test_fetch_list = [loss_test, acc_1, acc_5]
+        objs = utils.AvgrageMeter()
+        top1 = utils.AvgrageMeter()
+        top5 = utils.AvgrageMeter()
+        test_py_reader.start()
+        test_start_time = time.time()
+        step_id = 0
+        try:
+            while True:
+                prev_test_start_time = test_start_time
+                test_start_time = time.time()
+                loss_test_v, acc_1_v, acc_5_v = exe.run(
+                    test_prog, fetch_list=test_fetch_list)
+                objs.update(np.array(loss_test_v), args.batch_size)
+                top1.update(np.array(acc_1_v), args.batch_size)
+                top5.update(np.array(acc_5_v), args.batch_size)
+                if step_id % args.report_freq == 0:
+                    print("Epoch {}, Step {}, acc_1 {}, acc_5 {}, time {}".
+                          format(epoch_id, step_id,
+                                 np.array(acc_1_v),
+                                 np.array(acc_5_v), test_start_time -
+                                 prev_test_start_time))
+                step_id += 1
+        except fluid.core.EOFException:
+            test_py_reader.reset()
+        print("Epoch {0}, top1 {1}, top5 {2}".format(epoch_id, top1.avg,
+                                                     top5.avg))
+
+    train_fetch_list = [loss_train]
+    epoch_start_time = time.time()
+    for epoch_id in range(args.epochs):
+        model.drop_path_prob = args.drop_path_prob * epoch_id / args.epochs
+        train_py_reader.start()
+        epoch_end_time = time.time()
+        if epoch_id > 0:
+            print("Epoch {}, total time {}".format(epoch_id - 1, epoch_end_time
+                                                   - epoch_start_time))
+        epoch_start_time = epoch_end_time
+        epoch_end_time
+        start_time = time.time()
+        step_id = 0
+        try:
+            while True:
+                prev_start_time = start_time
+                start_time = time.time()
+                loss_v, = train_exe.run(
+                    fetch_list=[v.name for v in train_fetch_list])
+                print("Epoch {}, Step {}, loss {}, time {}".format(epoch_id, step_id, \
+                        np.array(loss_v).mean(), start_time-prev_start_time))
+                step_id += 1
+                sys.stdout.flush()
+        except fluid.core.EOFException:
+            train_py_reader.reset()
+        if epoch_id % 50 == 0 or epoch_id == args.epochs - 1:
+            save_model(str(epoch_id), train_prog)
+        test(epoch_id)
+
+
+if __name__ == '__main__':
+    main()

fluid/AutoDL/LRC/utils.py

+# Copyright (c) 2019 PaddlePaddle Authors. All Rights Reserved
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+# Based on:
+# --------------------------------------------------------
+# DARTS
+# Copyright (c) 2018, Hanxiao Liu.
+# Licensed under the Apache License, Version 2.0;
+# --------------------------------------------------------
+
+import os
+import sys
+import time
+import math
+import numpy as np
+
+
+def mixup_data(x, y, batch_size, alpha=1.0):
+    '''Compute the mixup data. Return mixed inputs, pairs of targets, and lambda'''
+    if alpha > 0.:
+        lam = np.random.beta(alpha, alpha)
+    else:
+        lam = 1.
+    index = np.random.permutation(batch_size)
+
+    mixed_x = lam * x + (1 - lam) * x[index, :]
+    y_a, y_b = y, y[index]
+    return mixed_x.astype('float32'), y_a.astype('int64'),\
+           y_b.astype('int64'), np.array(lam, dtype='float32')
+
+
+class AvgrageMeter(object):
+    def __init__(self):
+        self.reset()
+
+    def reset(self):
+        self.avg = 0
+        self.sum = 0
+        self.cnt = 0
+
+    def update(self, val, n=1):
+        self.sum += val * n
+        self.cnt += n
+        self.avg = self.sum / self.cnt

fluid/DeepQNetwork/DQN_agent.py

 #-*- coding: utf-8 -*-
 
+import math
+import numpy as np
 import paddle.fluid as fluid
 from paddle.fluid.param_attr import ParamAttr
-import numpy as np
-import math
 from tqdm import tqdm
-from utils import fluid_flatten
 
 
 class DQNModel(object):
@@ -39,34 +38,51 @@ class DQNModel(object):
                    name='isOver', shape=[], dtype='bool')
 
     def _build_net(self):
-        state, action, reward, next_s, isOver = self._get_inputs()
-        self.pred_value = self.get_DQN_prediction(state)
-        self.predict_program = fluid.default_main_program().clone()
+        self.predict_program = fluid.Program()
+        self.train_program = fluid.Program()
+        self._sync_program = fluid.Program()
 
-        reward = fluid.layers.clip(reward, min=-1.0, max=1.0)
+        with fluid.program_guard(self.predict_program):
+            state, action, reward, next_s, isOver = self._get_inputs()
+            self.pred_value = self.get_DQN_prediction(state)
 
-        action_onehot = fluid.layers.one_hot(action, self.action_dim)
-        action_onehot = fluid.layers.cast(action_onehot, dtype='float32')
+        with fluid.program_guard(self.train_program):
+            state, action, reward, next_s, isOver = self._get_inputs()
+            pred_value = self.get_DQN_prediction(state)
 
-        pred_action_value = fluid.layers.reduce_sum(
-            fluid.layers.elementwise_mul(action_onehot, self.pred_value), dim=1)
+            reward = fluid.layers.clip(reward, min=-1.0, max=1.0)
 
-        targetQ_predict_value = self.get_DQN_prediction(next_s, target=True)
-        best_v = fluid.layers.reduce_max(targetQ_predict_value, dim=1)
-        best_v.stop_gradient = True
+            action_onehot = fluid.layers.one_hot(action, self.action_dim)
+            action_onehot = fluid.layers.cast(action_onehot, dtype='float32')
 
-        target = reward + (1.0 - fluid.layers.cast(
-            isOver, dtype='float32')) * self.gamma * best_v
-        cost = fluid.layers.square_error_cost(pred_action_value, target)
-        cost = fluid.layers.reduce_mean(cost)
+            pred_action_value = fluid.layers.reduce_sum(
+                fluid.layers.elementwise_mul(action_onehot, pred_value), dim=1)
 
-        self._sync_program = self._build_sync_target_network()
+            targetQ_predict_value = self.get_DQN_prediction(next_s, target=True)
+            best_v = fluid.layers.reduce_max(targetQ_predict_value, dim=1)
+            best_v.stop_gradient = True
 
-        optimizer = fluid.optimizer.Adam(1e-3 * 0.5, epsilon=1e-3)
-        optimizer.minimize(cost)
+            target = reward + (1.0 - fluid.layers.cast(
+                isOver, dtype='float32')) * self.gamma * best_v
+            cost = fluid.layers.square_error_cost(pred_action_value, target)
+            cost = fluid.layers.reduce_mean(cost)
 
-        # define program
-        self.train_program = fluid.default_main_program()
+            optimizer = fluid.optimizer.Adam(1e-3 * 0.5, epsilon=1e-3)
+            optimizer.minimize(cost)
+
+        vars = list(self.train_program.list_vars())
+        policy_vars = list(filter(
+            lambda x: 'GRAD' not in x.name and 'policy' in x.name, vars))
+        target_vars = list(filter(
+            lambda x: 'GRAD' not in x.name and 'target' in x.name, vars))
+        policy_vars.sort(key=lambda x: x.name)
+        target_vars.sort(key=lambda x: x.name)
+        
+        with fluid.program_guard(self._sync_program):
+            sync_ops = []
+            for i, var in enumerate(policy_vars):
+                sync_op = fluid.layers.assign(policy_vars[i], target_vars[i])
+                sync_ops.append(sync_op)
 
         # fluid exe
         place = fluid.CUDAPlace(0) if self.use_cuda else fluid.CPUPlace()
@@ -81,50 +97,50 @@ class DQNModel(object):
         conv1 = fluid.layers.conv2d(
             input=image,
             num_filters=32,
-            filter_size=[5, 5],
-            stride=[1, 1],
-            padding=[2, 2],
+            filter_size=5,
+            stride=1,
+            padding=2,
             act='relu',
             param_attr=ParamAttr(name='{}_conv1'.format(variable_field)),
             bias_attr=ParamAttr(name='{}_conv1_b'.format(variable_field)))
         max_pool1 = fluid.layers.pool2d(
-            input=conv1, pool_size=[2, 2], pool_stride=[2, 2], pool_type='max')
+            input=conv1, pool_size=2, pool_stride=2, pool_type='max')
 
         conv2 = fluid.layers.conv2d(
             input=max_pool1,
             num_filters=32,
-            filter_size=[5, 5],
-            stride=[1, 1],
-            padding=[2, 2],
+            filter_size=5,
+            stride=1,
+            padding=2,
             act='relu',
             param_attr=ParamAttr(name='{}_conv2'.format(variable_field)),
             bias_attr=ParamAttr(name='{}_conv2_b'.format(variable_field)))
         max_pool2 = fluid.layers.pool2d(
-            input=conv2, pool_size=[2, 2], pool_stride=[2, 2], pool_type='max')
+            input=conv2, pool_size=2, pool_stride=2, pool_type='max')
 
         conv3 = fluid.layers.conv2d(
             input=max_pool2,
             num_filters=64,
-            filter_size=[4, 4],
-            stride=[1, 1],
-            padding=[1, 1],
+            filter_size=4,
+            stride=1,
+            padding=1,
             act='relu',
             param_attr=ParamAttr(name='{}_conv3'.format(variable_field)),
             bias_attr=ParamAttr(name='{}_conv3_b'.format(variable_field)))
         max_pool3 = fluid.layers.pool2d(
-            input=conv3, pool_size=[2, 2], pool_stride=[2, 2], pool_type='max')
+            input=conv3, pool_size=2, pool_stride=2, pool_type='max')
 
         conv4 = fluid.layers.conv2d(
             input=max_pool3,
             num_filters=64,
-            filter_size=[3, 3],
-            stride=[1, 1],
-            padding=[1, 1],
+            filter_size=3,
+            stride=1,
+            padding=1,
             act='relu',
             param_attr=ParamAttr(name='{}_conv4'.format(variable_field)),
             bias_attr=ParamAttr(name='{}_conv4_b'.format(variable_field)))
 
-        flatten = fluid_flatten(conv4)
+        flatten = fluid.layers.flatten(conv4, axis=1)
 
         out = fluid.layers.fc(
             input=flatten,
@@ -133,23 +149,6 @@ class DQNModel(object):
             bias_attr=ParamAttr(name='{}_fc1_b'.format(variable_field)))
         return out
 
-    def _build_sync_target_network(self):
-        vars = list(fluid.default_main_program().list_vars())
-        policy_vars = list(filter(
-            lambda x: 'GRAD' not in x.name and 'policy' in x.name, vars))
-        target_vars = list(filter(
-            lambda x: 'GRAD' not in x.name and 'target' in x.name, vars))
-        policy_vars.sort(key=lambda x: x.name)
-        target_vars.sort(key=lambda x: x.name)
-
-        sync_program = fluid.default_main_program().clone()
-        with fluid.program_guard(sync_program):
-            sync_ops = []
-            for i, var in enumerate(policy_vars):
-                sync_op = fluid.layers.assign(policy_vars[i], target_vars[i])
-                sync_ops.append(sync_op)
-        sync_program = sync_program.prune(sync_ops)
-        return sync_program
 
     def act(self, state, train_or_test):
         sample = np.random.random()

fluid/DeepQNetwork/DoubleDQN_agent.py

 #-*- coding: utf-8 -*-
 
+import math
+import numpy as np
 import paddle.fluid as fluid
 from paddle.fluid.param_attr import ParamAttr
-import numpy as np
 from tqdm import tqdm
-import math
-from utils import fluid_argmax, fluid_flatten
 
 
 class DoubleDQNModel(object):
@@ -39,41 +38,59 @@ class DoubleDQNModel(object):
                    name='isOver', shape=[], dtype='bool')
 
     def _build_net(self):
-        state, action, reward, next_s, isOver = self._get_inputs()
-        self.pred_value = self.get_DQN_prediction(state)
-        self.predict_program = fluid.default_main_program().clone()
+        self.predict_program = fluid.Program()
+        self.train_program = fluid.Program()
+        self._sync_program = fluid.Program()
 
-        reward = fluid.layers.clip(reward, min=-1.0, max=1.0)
+        with fluid.program_guard(self.predict_program):
+            state, action, reward, next_s, isOver = self._get_inputs()
+            self.pred_value = self.get_DQN_prediction(state)
 
-        action_onehot = fluid.layers.one_hot(action, self.action_dim)
-        action_onehot = fluid.layers.cast(action_onehot, dtype='float32')
+        with fluid.program_guard(self.train_program):
+            state, action, reward, next_s, isOver = self._get_inputs()
+            pred_value = self.get_DQN_prediction(state)
 
-        pred_action_value = fluid.layers.reduce_sum(
-            fluid.layers.elementwise_mul(action_onehot, self.pred_value), dim=1)
+            reward = fluid.layers.clip(reward, min=-1.0, max=1.0)
 
-        targetQ_predict_value = self.get_DQN_prediction(next_s, target=True)
+            action_onehot = fluid.layers.one_hot(action, self.action_dim)
+            action_onehot = fluid.layers.cast(action_onehot, dtype='float32')
 
-        next_s_predcit_value = self.get_DQN_prediction(next_s)
-        greedy_action = fluid_argmax(next_s_predcit_value)
+            pred_action_value = fluid.layers.reduce_sum(
+                fluid.layers.elementwise_mul(action_onehot, pred_value), dim=1)
 
-        predict_onehot = fluid.layers.one_hot(greedy_action, self.action_dim)
-        best_v = fluid.layers.reduce_sum(
-            fluid.layers.elementwise_mul(predict_onehot, targetQ_predict_value),
-            dim=1)
-        best_v.stop_gradient = True
+            targetQ_predict_value = self.get_DQN_prediction(next_s, target=True)
 
-        target = reward + (1.0 - fluid.layers.cast(
-            isOver, dtype='float32')) * self.gamma * best_v
-        cost = fluid.layers.square_error_cost(pred_action_value, target)
-        cost = fluid.layers.reduce_mean(cost)
+            next_s_predcit_value = self.get_DQN_prediction(next_s)
+            greedy_action = fluid.layers.argmax(next_s_predcit_value, axis=1)
+            greedy_action = fluid.layers.unsqueeze(greedy_action, axes=[1])
 
-        self._sync_program = self._build_sync_target_network()
+            predict_onehot = fluid.layers.one_hot(greedy_action, self.action_dim)
+            best_v = fluid.layers.reduce_sum(
+                fluid.layers.elementwise_mul(predict_onehot, targetQ_predict_value),
+                dim=1)
+            best_v.stop_gradient = True
 
-        optimizer = fluid.optimizer.Adam(1e-3 * 0.5, epsilon=1e-3)
-        optimizer.minimize(cost)
+            target = reward + (1.0 - fluid.layers.cast(
+                isOver, dtype='float32')) * self.gamma * best_v
+            cost = fluid.layers.square_error_cost(pred_action_value, target)
+            cost = fluid.layers.reduce_mean(cost)
 
-        # define program
-        self.train_program = fluid.default_main_program()
+            optimizer = fluid.optimizer.Adam(1e-3 * 0.5, epsilon=1e-3)
+            optimizer.minimize(cost)
+
+        vars = list(self.train_program.list_vars())
+        policy_vars = list(filter(
+            lambda x: 'GRAD' not in x.name and 'policy' in x.name, vars))
+        target_vars = list(filter(
+            lambda x: 'GRAD' not in x.name and 'target' in x.name, vars))
+        policy_vars.sort(key=lambda x: x.name)
+        target_vars.sort(key=lambda x: x.name)
+        
+        with fluid.program_guard(self._sync_program):
+            sync_ops = []
+            for i, var in enumerate(policy_vars):
+                sync_op = fluid.layers.assign(policy_vars[i], target_vars[i])
+                sync_ops.append(sync_op)
 
         # fluid exe
         place = fluid.CUDAPlace(0) if self.use_cuda else fluid.CPUPlace()
@@ -88,50 +105,50 @@ class DoubleDQNModel(object):
         conv1 = fluid.layers.conv2d(
             input=image,
             num_filters=32,
-            filter_size=[5, 5],
-            stride=[1, 1],
-            padding=[2, 2],
+            filter_size=5,
+            stride=1,
+            padding=2,
             act='relu',
             param_attr=ParamAttr(name='{}_conv1'.format(variable_field)),
             bias_attr=ParamAttr(name='{}_conv1_b'.format(variable_field)))
         max_pool1 = fluid.layers.pool2d(
-            input=conv1, pool_size=[2, 2], pool_stride=[2, 2], pool_type='max')
+            input=conv1, pool_size=2, pool_stride=2, pool_type='max')
 
         conv2 = fluid.layers.conv2d(
             input=max_pool1,
             num_filters=32,
-            filter_size=[5, 5],
-            stride=[1, 1],
-            padding=[2, 2],
+            filter_size=5,
+            stride=1,
+            padding=2,
             act='relu',
             param_attr=ParamAttr(name='{}_conv2'.format(variable_field)),
             bias_attr=ParamAttr(name='{}_conv2_b'.format(variable_field)))
         max_pool2 = fluid.layers.pool2d(
-            input=conv2, pool_size=[2, 2], pool_stride=[2, 2], pool_type='max')
+            input=conv2, pool_size=2, pool_stride=2, pool_type='max')
 
         conv3 = fluid.layers.conv2d(
             input=max_pool2,
             num_filters=64,
-            filter_size=[4, 4],
-            stride=[1, 1],
-            padding=[1, 1],
+            filter_size=4,
+            stride=1,
+            padding=1,
             act='relu',
             param_attr=ParamAttr(name='{}_conv3'.format(variable_field)),
             bias_attr=ParamAttr(name='{}_conv3_b'.format(variable_field)))
         max_pool3 = fluid.layers.pool2d(
-            input=conv3, pool_size=[2, 2], pool_stride=[2, 2], pool_type='max')
+            input=conv3, pool_size=2, pool_stride=2, pool_type='max')
 
         conv4 = fluid.layers.conv2d(
             input=max_pool3,
             num_filters=64,
-            filter_size=[3, 3],
-            stride=[1, 1],
-            padding=[1, 1],
+            filter_size=3,
+            stride=1,
+            padding=1,
             act='relu',
             param_attr=ParamAttr(name='{}_conv4'.format(variable_field)),
             bias_attr=ParamAttr(name='{}_conv4_b'.format(variable_field)))
 
-        flatten = fluid_flatten(conv4)
+        flatten = fluid.layers.flatten(conv4, axis=1)
 
         out = fluid.layers.fc(
             input=flatten,
@@ -140,23 +157,6 @@ class DoubleDQNModel(object):
             bias_attr=ParamAttr(name='{}_fc1_b'.format(variable_field)))
         return out
 
-    def _build_sync_target_network(self):
-        vars = list(fluid.default_main_program().list_vars())
-        policy_vars = list(filter(
-            lambda x: 'GRAD' not in x.name and 'policy' in x.name, vars))
-        target_vars = list(filter(
-            lambda x: 'GRAD' not in x.name and 'target' in x.name, vars))
-        policy_vars.sort(key=lambda x: x.name)
-        target_vars.sort(key=lambda x: x.name)
-
-        sync_program = fluid.default_main_program().clone()
-        with fluid.program_guard(sync_program):
-            sync_ops = []
-            for i, var in enumerate(policy_vars):
-                sync_op = fluid.layers.assign(policy_vars[i], target_vars[i])
-                sync_ops.append(sync_op)
-        sync_program = sync_program.prune(sync_ops)
-        return sync_program
 
     def act(self, state, train_or_test):
         sample = np.random.random()

fluid/DeepQNetwork/DuelingDQN_agent.py

 #-*- coding: utf-8 -*-
 
+import math
+import numpy as np
 import paddle.fluid as fluid
 from paddle.fluid.param_attr import ParamAttr
-import numpy as np
 from tqdm import tqdm
-import math
-from utils import fluid_flatten
 
 
 class DuelingDQNModel(object):
@@ -39,34 +38,51 @@ class DuelingDQNModel(object):
                    name='isOver', shape=[], dtype='bool')
 
     def _build_net(self):
-        state, action, reward, next_s, isOver = self._get_inputs()
-        self.pred_value = self.get_DQN_prediction(state)
-        self.predict_program = fluid.default_main_program().clone()
+        self.predict_program = fluid.Program()
+        self.train_program = fluid.Program()
+        self._sync_program = fluid.Program()
 
-        reward = fluid.layers.clip(reward, min=-1.0, max=1.0)
+        with fluid.program_guard(self.predict_program):
+            state, action, reward, next_s, isOver = self._get_inputs()
+            self.pred_value = self.get_DQN_prediction(state)
 
-        action_onehot = fluid.layers.one_hot(action, self.action_dim)
-        action_onehot = fluid.layers.cast(action_onehot, dtype='float32')
+        with fluid.program_guard(self.train_program):
+            state, action, reward, next_s, isOver = self._get_inputs()
+            pred_value = self.get_DQN_prediction(state)
 
-        pred_action_value = fluid.layers.reduce_sum(
-            fluid.layers.elementwise_mul(action_onehot, self.pred_value), dim=1)
+            reward = fluid.layers.clip(reward, min=-1.0, max=1.0)
 
-        targetQ_predict_value = self.get_DQN_prediction(next_s, target=True)
-        best_v = fluid.layers.reduce_max(targetQ_predict_value, dim=1)
-        best_v.stop_gradient = True
+            action_onehot = fluid.layers.one_hot(action, self.action_dim)
+            action_onehot = fluid.layers.cast(action_onehot, dtype='float32')
 
-        target = reward + (1.0 - fluid.layers.cast(
-            isOver, dtype='float32')) * self.gamma * best_v
-        cost = fluid.layers.square_error_cost(pred_action_value, target)
-        cost = fluid.layers.reduce_mean(cost)
+            pred_action_value = fluid.layers.reduce_sum(
+                fluid.layers.elementwise_mul(action_onehot, pred_value), dim=1)
 
-        self._sync_program = self._build_sync_target_network()
+            targetQ_predict_value = self.get_DQN_prediction(next_s, target=True)
+            best_v = fluid.layers.reduce_max(targetQ_predict_value, dim=1)
+            best_v.stop_gradient = True
 
-        optimizer = fluid.optimizer.Adam(1e-3 * 0.5, epsilon=1e-3)
-        optimizer.minimize(cost)
+            target = reward + (1.0 - fluid.layers.cast(
+                isOver, dtype='float32')) * self.gamma * best_v
+            cost = fluid.layers.square_error_cost(pred_action_value, target)
+            cost = fluid.layers.reduce_mean(cost)
 
-        # define program
-        self.train_program = fluid.default_main_program()
+            optimizer = fluid.optimizer.Adam(1e-3 * 0.5, epsilon=1e-3)
+            optimizer.minimize(cost)
+
+        vars = list(self.train_program.list_vars())
+        policy_vars = list(filter(
+            lambda x: 'GRAD' not in x.name and 'policy' in x.name, vars))
+        target_vars = list(filter(
+            lambda x: 'GRAD' not in x.name and 'target' in x.name, vars))
+        policy_vars.sort(key=lambda x: x.name)
+        target_vars.sort(key=lambda x: x.name)
+        
+        with fluid.program_guard(self._sync_program):
+            sync_ops = []
+            for i, var in enumerate(policy_vars):
+                sync_op = fluid.layers.assign(policy_vars[i], target_vars[i])
+                sync_ops.append(sync_op)
 
         # fluid exe
         place = fluid.CUDAPlace(0) if self.use_cuda else fluid.CPUPlace()
@@ -81,50 +97,50 @@ class DuelingDQNModel(object):
         conv1 = fluid.layers.conv2d(
             input=image,
             num_filters=32,
-            filter_size=[5, 5],
-            stride=[1, 1],
-            padding=[2, 2],
+            filter_size=5,
+            stride=1,
+            padding=2,
             act='relu',
             param_attr=ParamAttr(name='{}_conv1'.format(variable_field)),
             bias_attr=ParamAttr(name='{}_conv1_b'.format(variable_field)))
         max_pool1 = fluid.layers.pool2d(
-            input=conv1, pool_size=[2, 2], pool_stride=[2, 2], pool_type='max')
+            input=conv1, pool_size=2, pool_stride=2, pool_type='max')
 
         conv2 = fluid.layers.conv2d(
             input=max_pool1,
             num_filters=32,
-            filter_size=[5, 5],
-            stride=[1, 1],
-            padding=[2, 2],
+            filter_size=5,
+            stride=1,
+            padding=2,
             act='relu',
             param_attr=ParamAttr(name='{}_conv2'.format(variable_field)),
             bias_attr=ParamAttr(name='{}_conv2_b'.format(variable_field)))
         max_pool2 = fluid.layers.pool2d(
-            input=conv2, pool_size=[2, 2], pool_stride=[2, 2], pool_type='max')
+            input=conv2, pool_size=2, pool_stride=2, pool_type='max')
 
         conv3 = fluid.layers.conv2d(
             input=max_pool2,
             num_filters=64,
-            filter_size=[4, 4],
-            stride=[1, 1],
-            padding=[1, 1],
+            filter_size=4,
+            stride=1,
+            padding=1,
             act='relu',
             param_attr=ParamAttr(name='{}_conv3'.format(variable_field)),
             bias_attr=ParamAttr(name='{}_conv3_b'.format(variable_field)))
         max_pool3 = fluid.layers.pool2d(
-            input=conv3, pool_size=[2, 2], pool_stride=[2, 2], pool_type='max')
+            input=conv3, pool_size=2, pool_stride=2, pool_type='max')
 
         conv4 = fluid.layers.conv2d(
             input=max_pool3,
             num_filters=64,
-            filter_size=[3, 3],
-            stride=[1, 1],
-            padding=[1, 1],
+            filter_size=3,
+            stride=1,
+            padding=1,
             act='relu',
             param_attr=ParamAttr(name='{}_conv4'.format(variable_field)),
             bias_attr=ParamAttr(name='{}_conv4_b'.format(variable_field)))
 
-        flatten = fluid_flatten(conv4)
+        flatten = fluid.layers.flatten(conv4, axis=1)
 
         value = fluid.layers.fc(
             input=flatten,
@@ -143,24 +159,6 @@ class DuelingDQNModel(object):
             advantage, dim=1, keep_dim=True))
         return Q
 
-    def _build_sync_target_network(self):
-        vars = list(fluid.default_main_program().list_vars())
-        policy_vars = list(filter(
-            lambda x: 'GRAD' not in x.name and 'policy' in x.name, vars))
-        target_vars = list(filter(
-            lambda x: 'GRAD' not in x.name and 'target' in x.name, vars))
-        policy_vars.sort(key=lambda x: x.name)
-        target_vars.sort(key=lambda x: x.name)
-
-        sync_program = fluid.default_main_program().clone()
-        with fluid.program_guard(sync_program):
-            sync_ops = []
-            for i, var in enumerate(policy_vars):
-                sync_op = fluid.layers.assign(policy_vars[i], target_vars[i])
-                sync_ops.append(sync_op)
-        # The prune API is deprecated, please don't use it any more.
-        sync_program = sync_program._prune(sync_ops)
-        return sync_program
 
     def act(self, state, train_or_test):
         sample = np.random.random()
@@ -186,12 +184,14 @@ class DuelingDQNModel(object):
         self.global_step += 1
 
         action = np.expand_dims(action, -1)
-        self.exe.run(self.train_program, \
-                  feed={'state': state.astype('float32'), \
-                        'action': action.astype('int32'), \
-                        'reward': reward, \
-                        'next_s': next_state.astype('float32'), \
-                        'isOver': isOver})
+        self.exe.run(self.train_program,
+                     feed={
+                         'state': state.astype('float32'),
+                         'action': action.astype('int32'),
+                         'reward': reward,
+                         'next_s': next_state.astype('float32'),
+                         'isOver': isOver
+                     })
 
     def sync_target_network(self):
         self.exe.run(self._sync_program)

fluid/DeepQNetwork/README.md

@@ -29,7 +29,7 @@ The average game rewards that can be obtained for the three models as the number
 + gym
 + tqdm
 + opencv-python
-+ paddlepaddle-gpu>=0.12.0
++ paddlepaddle-gpu>=1.0.0
 + ale_python_interface
 
 ### Install Dependencies:

fluid/DeepQNetwork/README_cn.md

@@ -28,7 +28,7 @@
 + gym
 + tqdm
 + opencv-python
-+ paddlepaddle-gpu>=0.12.0
++ paddlepaddle-gpu>=1.0.0
 + ale_python_interface
 
 ### 下载依赖：

fluid/DeepQNetwork/utils.py

-#-*- coding: utf-8 -*-
-#File: utils.py
-
-import paddle.fluid as fluid
-import numpy as np
-
-
-def fluid_argmax(x):
-    """
-    Get index of max value for the last dimension
-    """
-    _, max_index = fluid.layers.topk(x, k=1)
-    return max_index
-
-
-def fluid_flatten(x):
-    """
-    Flatten fluid variable along the first dimension
-    """
-    return fluid.layers.reshape(x, shape=[-1, np.prod(x.shape[1:])])

fluid/PaddleCV/HiNAS_models/nn_paddle.py

@@ -21,6 +21,7 @@ import math
 import numpy as np
 import paddle
 import paddle.fluid as fluid
+from paddle.fluid.contrib.trainer import *
 from paddle.fluid.layers.learning_rate_scheduler import _decay_step_counter
 import reader
 
@@ -104,7 +105,7 @@ class Model(object):
         accs = []
 
         def event_handler(event):
-            if isinstance(event, fluid.EndStepEvent):
+            if isinstance(event, EndStepEvent):
                 costs.append(event.metrics[0])
                 accs.append(event.metrics[1])
                 if event.step % 20 == 0:
@@ -113,7 +114,7 @@ class Model(object):
                     del costs[:]
                     del accs[:]
 
-            if isinstance(event, fluid.EndEpochEvent):
+            if isinstance(event, EndEpochEvent):
                 if event.epoch % 3 == 0 or event.epoch == FLAGS.num_epochs - 1:
                     avg_cost, accuracy = trainer.test(
                         reader=test_reader, feed_order=['pixel', 'label'])
@@ -126,7 +127,7 @@ class Model(object):
 
         event_handler.best_acc = 0.0
         place = fluid.CUDAPlace(0)
-        trainer = fluid.Trainer(
+        trainer = Trainer(
             train_func=self.train_network,
             optimizer_func=self.optimizer_program,
             place=place)

fluid/PaddleCV/face_detection/data_util.py

-"""
-This code is based on https://github.com/fchollet/keras/blob/master/keras/utils/data_utils.py
-"""
-from __future__ import absolute_import
-from __future__ import division
-from __future__ import print_function
-
-import time
-import numpy as np
-import threading
-import multiprocessing
-import traceback
-try:
-    import queue
-except ImportError:
-    import Queue as queue
-
-
-class GeneratorEnqueuer(object):
-    """
-    Builds a queue out of a data generator.
-
-    Args:
-        generator: a generator function which endlessly yields data
-        use_multiprocessing (bool): use multiprocessing if True,
-            otherwise use threading.
-        wait_time (float): time to sleep in-between calls to `put()`.
-        random_seed (int): Initial seed for workers,
-            will be incremented by one for each workers.
-    """
-
-    def __init__(self,
-                 generator,
-                 use_multiprocessing=False,
-                 wait_time=0.05,
-                 random_seed=None):
-        self.wait_time = wait_time
-        self._generator = generator
-        self._use_multiprocessing = use_multiprocessing
-        self._threads = []
-        self._stop_event = None
-        self.queue = None
-        self._manager = None
-        self.seed = random_seed
-
-    def start(self, workers=1, max_queue_size=10):
-        """
-        Start worker threads which add data from the generator into the queue.
-
-        Args:
-            workers (int): number of worker threads
-            max_queue_size (int): queue size
-                (when full, threads could block on `put()`)
-        """
-
-        def data_generator_task():
-            """
-            Data generator task.
-            """
-
-            def task():
-                if (self.queue is not None and
-                        self.queue.qsize() < max_queue_size):
-                    generator_output = next(self._generator)
-                    self.queue.put((generator_output))
-                else:
-                    time.sleep(self.wait_time)
-
-            if not self._use_multiprocessing:
-                while not self._stop_event.is_set():
-                    with self.genlock:
-                        try:
-                            task()
-                        except Exception:
-                            traceback.print_exc()
-                            self._stop_event.set()
-                            break
-            else:
-                while not self._stop_event.is_set():
-                    try:
-                        task()
-                    except Exception:
-                        traceback.print_exc()
-                        self._stop_event.set()
-                        break
-
-        try:
-            if self._use_multiprocessing:
-                self._manager = multiprocessing.Manager()
-                self.queue = self._manager.Queue(maxsize=max_queue_size)
-                self._stop_event = multiprocessing.Event()
-            else:
-                self.genlock = threading.Lock()
-                self.queue = queue.Queue()
-                self._stop_event = threading.Event()
-            for _ in range(workers):
-                if self._use_multiprocessing:
-                    # Reset random seed else all children processes
-                    # share the same seed
-                    np.random.seed(self.seed)
-                    thread = multiprocessing.Process(target=data_generator_task)
-                    thread.daemon = True
-                    if self.seed is not None:
-                        self.seed += 1
-                else:
-                    thread = threading.Thread(target=data_generator_task)
-                self._threads.append(thread)
-                thread.start()
-        except:
-            self.stop()
-            raise
-
-    def is_running(self):
-        """
-        Returns:
-            bool: Whether the worker theads are running.
-        """
-        return self._stop_event is not None and not self._stop_event.is_set()
-
-    def stop(self, timeout=None):
-        """
-        Stops running threads and wait for them to exit, if necessary.
-        Should be called by the same thread which called `start()`.
-
-        Args:
-            timeout(int|None): maximum time to wait on `thread.join()`.
-        """
-        if self.is_running():
-            self._stop_event.set()
-        for thread in self._threads:
-            if self._use_multiprocessing:
-                if thread.is_alive():
-                    thread.terminate()
-            else:
-                thread.join(timeout)
-        if self._manager:
-            self._manager.shutdown()
-
-        self._threads = []
-        self._stop_event = None
-        self.queue = None
-
-    def get(self):
-        """
-        Creates a generator to extract data from the queue.
-        Skip the data if it is `None`.
-
-        # Yields
-            tuple of data in the queue.
-        """
-        while self.is_running():
-            if not self.queue.empty():
-                inputs = self.queue.get()
-                if inputs is not None:
-                    yield inputs
-            else:
-                time.sleep(self.wait_time)

fluid/PaddleCV/face_detection/reader.py

@@ -16,8 +16,6 @@ from __future__ import absolute_import
 from __future__ import division
 from __future__ import print_function
 
-import image_util
-from paddle.utils.image_util import *
 from PIL import Image
 from PIL import ImageDraw
 import numpy as np
@@ -28,7 +26,10 @@ import copy
 import random
 import cv2
 import six
-from data_util import GeneratorEnqueuer
+import math
+from itertools import islice
+import paddle
+import image_util
 
 
 class Settings(object):
@@ -199,7 +200,7 @@ def load_file_list(input_txt):
             else:
                 file_dict[num_class].append(line_txt)
 
-    return file_dict
+    return list(file_dict.values())
 
 
 def expand_bboxes(bboxes,
@@ -227,13 +228,12 @@ def expand_bboxes(bboxes,
 
 
 def train_generator(settings, file_list, batch_size, shuffle=True):
-    file_dict = load_file_list(file_list)
-    while True:
+    def reader():
         if shuffle:
-            np.random.shuffle(file_dict)
+            np.random.shuffle(file_list)
         batch_out = []
-        for index_image in file_dict.keys():
-            image_name = file_dict[index_image][0]
+        for item in file_list:
+            image_name = item[0]
             image_path = os.path.join(settings.data_dir, image_name)
             im = Image.open(image_path)
             if im.mode == 'L':
@@ -242,10 +242,10 @@ def train_generator(settings, file_list, batch_size, shuffle=True):
 
             # layout: label | xmin | ymin | xmax | ymax
             bbox_labels = []
-            for index_box in range(len(file_dict[index_image])):
+            for index_box in range(len(item)):
                 if index_box >= 2:
                     bbox_sample = []
-                    temp_info_box = file_dict[index_image][index_box].split(' ')
+                    temp_info_box = item[index_box].split(' ')
                     xmin = float(temp_info_box[0])
                     ymin = float(temp_info_box[1])
                     w = float(temp_info_box[2])
@@ -277,43 +277,25 @@ def train_generator(settings, file_list, batch_size, shuffle=True):
                 yield batch_out
                 batch_out = []
 
+    return reader
 
-def train(settings,
-          file_list,
-          batch_size,
-          shuffle=True,
-          use_multiprocessing=True,
-          num_workers=8,
-          max_queue=24):
-    def reader():
-        try:
-            enqueuer = GeneratorEnqueuer(
-                train_generator(settings, file_list, batch_size, shuffle),
-                use_multiprocessing=use_multiprocessing)
-            enqueuer.start(max_queue_size=max_queue, workers=num_workers)
-            generator_output = None
-            while True:
-                while enqueuer.is_running():
-                    if not enqueuer.queue.empty():
-                        generator_output = enqueuer.queue.get()
-                        break
-                    else:
-                        time.sleep(0.01)
-                yield generator_output
-                generator_output = None
-        finally:
-            if enqueuer is not None:
-                enqueuer.stop()
 
-    return reader
+def train(settings, file_list, batch_size, shuffle=True, num_workers=8):
+    file_lists = load_file_list(file_list)
+    n = int(math.ceil(len(file_lists) // num_workers))
+    split_lists = [file_lists[i:i + n] for i in range(0, len(file_lists), n)]
+    readers = []
+    for iterm in split_lists:
+        readers.append(train_generator(settings, iterm, batch_size, shuffle))
+    return paddle.reader.multiprocess_reader(readers, False)
 
 
 def test(settings, file_list):
-    file_dict = load_file_list(file_list)
+    file_lists = load_file_list(file_list)
 
     def reader():
-        for index_image in file_dict.keys():
-            image_name = file_dict[index_image][0]
+        for image in file_lists:
+            image_name = image[0]
             image_path = os.path.join(settings.data_dir, image_name)
             im = Image.open(image_path)
             if im.mode == 'L':

fluid/PaddleCV/face_detection/train.py

@@ -163,9 +163,7 @@ def train(args, config, train_params, train_file_list):
                                 train_file_list,
                                 batch_size_per_device,
                                 shuffle = is_shuffle,
-                                use_multiprocessing=True,
-                                num_workers = num_workers,
-                                max_queue=24)
+                                num_workers = num_workers)
     train_py_reader.decorate_paddle_reader(train_reader)
 
     if args.parallel:
@@ -182,61 +180,59 @@ def train(args, config, train_params, train_file_list):
         print('save models to %s' % (model_path))
         fluid.io.save_persistables(exe, model_path, main_program=program)
 
-    train_py_reader.start()
-    try:
-        total_time = 0.0
-        epoch_idx = 0
-        face_loss = 0
-        head_loss = 0
-        for pass_id in range(start_epoc, epoc_num):
-            epoch_idx += 1
-            start_time = time.time()
-            prev_start_time = start_time
-            end_time = 0
-            batch_id = 0
-            for batch_id in range(iters_per_epoc):
+    total_time = 0.0
+    epoch_idx = 0
+    face_loss = 0
+    head_loss = 0
+    for pass_id in range(start_epoc, epoc_num):
+        epoch_idx += 1
+        start_time = time.time()
+        prev_start_time = start_time
+        end_time = 0
+        batch_id = 0
+        train_py_reader.start()
+        while True:
+            try:
                 prev_start_time = start_time
                 start_time = time.time()
                 if args.parallel:
                     fetch_vars = train_exe.run(fetch_list=
                         [v.name for v in fetches])
                 else:
-                    fetch_vars = exe.run(train_prog,
-                                         fetch_list=fetches)
+                    fetch_vars = exe.run(train_prog, fetch_list=fetches)
                 end_time = time.time()
                 fetch_vars = [np.mean(np.array(v)) for v in fetch_vars]
+                face_loss = fetch_vars[0]
+                head_loss = fetch_vars[1]
                 if batch_id % 10 == 0:
                     if not args.use_pyramidbox:
                         print("Pass {:d}, batch {:d}, loss {:.6f}, time {:.5f}".format(
-                            pass_id, batch_id, fetch_vars[0],
+                            pass_id, batch_id, face_loss,
                             start_time - prev_start_time))
                     else:
                         print("Pass {:d}, batch {:d}, face loss {:.6f}, " \
                               "head loss {:.6f}, " \
                               "time {:.5f}".format(pass_id,
-                               batch_id, fetch_vars[0], fetch_vars[1],
+                               batch_id, face_loss, head_loss,
                                start_time - prev_start_time))
-                face_loss = fetch_vars[0]
-                head_loss = fetch_vars[1]
-            epoch_end_time = time.time()
-            total_time += epoch_end_time - start_time
-            if pass_id % 1 == 0 or pass_id == epoc_num - 1:
-                save_model(str(pass_id), train_prog)
-        # only for ce
-        if args.enable_ce:
-            gpu_num = get_cards(args)
-            print("kpis\teach_pass_duration_card%s\t%s" %
-                    (gpu_num, total_time / epoch_idx))
-            print("kpis\ttrain_face_loss_card%s\t%s" %
-                    (gpu_num, face_loss))
-            print("kpis\ttrain_head_loss_card%s\t%s" %
-                    (gpu_num, head_loss))
-
-    except fluid.core.EOFException:
-        train_py_reader.reset()
-    except StopIteration:
-        train_py_reader.reset()
-    train_py_reader.reset()
+                batch_id += 1
+            except (fluid.core.EOFException, StopIteration):
+                train_py_reader.reset()
+                break
+        epoch_end_time = time.time()
+        total_time += epoch_end_time - start_time
+        save_model(str(pass_id), train_prog)
+
+    # only for ce
+    if args.enable_ce:
+        gpu_num = get_cards(args)
+        print("kpis\teach_pass_duration_card%s\t%s" %
+                (gpu_num, total_time / epoch_idx))
+        print("kpis\ttrain_face_loss_card%s\t%s" %
+                (gpu_num, face_loss))
+        print("kpis\ttrain_head_loss_card%s\t%s" %
+                (gpu_num, head_loss))
+
 
 
 def get_cards(args):

fluid/PaddleCV/face_detection/widerface_eval.py

@@ -121,7 +121,7 @@ def detect_face(image, shrink):
                          return_numpy=False)
     detection = np.array(detection)
     # layout: xmin, ymin, xmax. ymax, score
-    if detection.shape == (1, ):
+    if np.prod(detection.shape) == 1:
         print("No face detected")
         return np.array([[0, 0, 0, 0, 0]])
     det_conf = detection[:, 1]

fluid/PaddleCV/icnet/README.md

@@ -103,7 +103,7 @@ python infer.py \
 ## 其他信息
 |数据集 | pretrained model |
 |---|---|
-|CityScape | [Model]()[md: ] |
+|CityScape | [pretrained_model](https://paddle-icnet-models.bj.bcebos.com/model_1000.tar.gz) |
 
 ## 参考
 

fluid/PaddleCV/image_classification/README.md

@@ -209,6 +209,7 @@ Models are trained by starting with learning rate ```0.1``` and decaying it by `
 |[VGG16](https://paddle-imagenet-models-name.bj.bcebos.com/VGG16_pretrained.zip) | 72.08%/90.63% | 71.65%/90.57% |
 |[VGG19](https://paddle-imagenet-models-name.bj.bcebos.com/VGG19_pretrained.zip) | 72.56%/90.83% | 72.32%/90.98% |
 |[MobileNetV1](http://paddle-imagenet-models-name.bj.bcebos.com/MobileNetV1_pretrained.zip) | 70.91%/89.54% | 70.51%/89.35% |
+|[MobileNetV2](https://paddle-imagenet-models-name.bj.bcebos.com/MobileNetV2_pretrained.zip) | 71.90%/90.55% | 71.53%/90.41% |
 |[ResNet50](http://paddle-imagenet-models-name.bj.bcebos.com/ResNet50_pretrained.zip) | 76.35%/92.80% | 76.22%/92.92% |
 |[ResNet101](http://paddle-imagenet-models-name.bj.bcebos.com/ResNet101_pretrained.zip) | 77.49%/93.57% | 77.56%/93.64% |
 |[ResNet152](https://paddle-imagenet-models-name.bj.bcebos.com/ResNet152_pretrained.zip) | 78.12%/93.93% | 77.92%/93.87% |

fluid/PaddleCV/image_classification/README_cn.md

@@ -204,6 +204,7 @@ Models包括两种模型：带有参数名字的模型，和不带有参数名
 |[VGG16](https://paddle-imagenet-models-name.bj.bcebos.com/VGG16_pretrained.zip) | 72.08%/90.63% | 71.65%/90.57% |
 |[VGG19](https://paddle-imagenet-models-name.bj.bcebos.com/VGG19_pretrained.zip) | 72.56%/90.83% | 72.32%/90.98% |
 |[MobileNetV1](http://paddle-imagenet-models-name.bj.bcebos.com/MobileNetV1_pretrained.zip) | 70.91%/89.54% | 70.51%/89.35% |
+|[MobileNetV2](https://paddle-imagenet-models-name.bj.bcebos.com/MobileNetV2_pretrained.zip) | 71.90%/90.55% | 71.53%/90.41% |
 |[ResNet50](http://paddle-imagenet-models-name.bj.bcebos.com/ResNet50_pretrained.zip) | 76.35%/92.80% | 76.22%/92.92% |
 |[ResNet101](http://paddle-imagenet-models-name.bj.bcebos.com/ResNet101_pretrained.zip) | 77.49%/93.57% | 77.56%/93.64% |
 |[ResNet152](https://paddle-imagenet-models-name.bj.bcebos.com/ResNet152_pretrained.zip) | 78.12%/93.93% | 77.92%/93.87% |

fluid/PaddleCV/image_classification/dist_train/dist_train.py

@@ -80,9 +80,9 @@ def get_device_num():
         device_num = subprocess.check_output(['nvidia-smi', '-L']).decode().count('\n')
     return device_num
 
-def prepare_reader(is_train, pyreader, args):
+def prepare_reader(is_train, pyreader, args, pass_id=0):
     if is_train:
-        reader = train(data_dir=args.data_dir)
+        reader = train(data_dir=args.data_dir, pass_id_as_seed=pass_id)
     else:
         reader = val(data_dir=args.data_dir)
     if is_train:
@@ -262,6 +262,8 @@ def train_parallel(args):
         num_samples = 0
         start_time = time.time()
         batch_id = 1
+        # use pass_id+1 as per pass global shuffle for distributed training
+        prepare_reader(True, train_pyreader, args, pass_id + 1)
         train_pyreader.start()
         while True:
             try:

fluid/PaddleCV/image_classification/reader.py

@@ -130,11 +130,14 @@ def _reader_creator(file_list,
                     shuffle=False,
                     color_jitter=False,
                     rotate=False,
-                    data_dir=DATA_DIR):
+                    data_dir=DATA_DIR,
+                    pass_id_as_seed=0):
     def reader():
         with open(file_list) as flist:
             full_lines = [line.strip() for line in flist]
             if shuffle:
+                if pass_id_as_seed:
+                    np.random.seed(pass_id_as_seed)
                 np.random.shuffle(full_lines)
             if mode == 'train' and os.getenv('PADDLE_TRAINING_ROLE'):
                 # distributed mode if the env var `PADDLE_TRAINING_ROLE` exits
@@ -166,7 +169,7 @@ def _reader_creator(file_list,
     return paddle.reader.xmap_readers(mapper, reader, THREAD, BUF_SIZE)
 
 
-def train(data_dir=DATA_DIR):
+def train(data_dir=DATA_DIR, pass_id_as_seed=0):
     file_list = os.path.join(data_dir, 'train_list.txt')
     return _reader_creator(
         file_list,
@@ -174,7 +177,8 @@ def train(data_dir=DATA_DIR):
         shuffle=True,
         color_jitter=False,
         rotate=False,
-        data_dir=data_dir)
+        data_dir=data_dir,
+        pass_id_as_seed=pass_id_as_seed)
 
 
 def val(data_dir=DATA_DIR):

fluid/PaddleCV/image_classification/run.sh

@@ -6,7 +6,7 @@ python train.py \
        --class_dim=1000 \
        --image_shape=3,224,224 \
        --model_save_dir=output/ \
-       --with_mem_opt=False \
+       --with_mem_opt=True \
        --lr_strategy=piecewise_decay \
        --lr=0.1
 #      >log_SE_ResNeXt50_32x4d.txt 2>&1 &
@@ -19,7 +19,7 @@ python train.py \
 #       --class_dim=1000 \
 #       --image_shape=3,224,224 \
 #       --model_save_dir=output/ \
-#       --with_mem_opt=False \
+#       --with_mem_opt=True \
 #       --lr_strategy=piecewise_decay \
 #	--num_epochs=120 \
 #       --lr=0.01
@@ -32,7 +32,7 @@ python train.py \
 #       --class_dim=1000 \
 #       --image_shape=3,224,224 \
 #       --model_save_dir=output/ \
-#       --with_mem_opt=False \
+#       --with_mem_opt=True \
 #       --lr_strategy=piecewise_decay \
 #       --num_epochs=120 \
 #       --lr=0.1
@@ -46,12 +46,22 @@ python train.py \
 #       --class_dim=1000 \
 #       --image_shape=3,224,224 \
 #       --model_save_dir=output/ \
-#       --with_mem_opt=False \
+#       --with_mem_opt=True \
 #       --lr_strategy=piecewise_decay \
 #	--num_epochs=120 \
 #       --lr=0.1
 
-
+#python train.py \
+#	--model=MobileNetV2 \
+#	--batch_size=500 \
+#	--total_images=1281167 \
+#	--class_dim=1000 \
+#	--image_shape=3,224,224 \
+#	--model_save_dir=output/ \
+#	--with_mem_opt=True \
+#	--lr_strategy=cosine_decay \
+#	--num_epochs=200 \
+#	--lr=0.1
 #ResNet50:
 #python train.py \
 #       --model=ResNet50 \
@@ -60,7 +70,7 @@ python train.py \
 #       --class_dim=1000 \
 #       --image_shape=3,224,224 \
 #       --model_save_dir=output/ \
-#       --with_mem_opt=False \
+#       --with_mem_opt=True \
 #       --lr_strategy=piecewise_decay \
 #	--num_epochs=120 \
 #       --lr=0.1
@@ -87,7 +97,7 @@ python train.py \
 #       --lr_strategy=piecewise_decay \
 #       --lr=0.1 \
 #       --num_epochs=120 \
-#       --l2_decay=1e-4 \(TODO)
+#       --l2_decay=1e-4
 
 
 #SE_ResNeXt50:
@@ -99,7 +109,7 @@ python train.py \
 #       --lr_strategy=cosine_decay \
 #       --lr=0.1 \
 #       --num_epochs=200 \
-#       --l2_decay=12e-5 \(TODO)
+#       --l2_decay=12e-5
 
 #SE_ResNeXt101:
 #python train.py \
@@ -110,7 +120,7 @@ python train.py \
 #        --lr_strategy=cosine_decay \
 #        --lr=0.1 \
 #        --num_epochs=200 \
-#        --l2_decay=15e-5 \(TODO)
+#        --l2_decay=15e-5
 
 #VGG11:
 #python train.py \
@@ -121,7 +131,7 @@ python train.py \
 #        --lr_strategy=cosine_decay \
 #        --lr=0.1 \
 #        --num_epochs=90 \
-#        --l2_decay=2e-4 \(TODO)
+#        --l2_decay=2e-4
 
 #VGG13:
 #python train.py
@@ -132,4 +142,4 @@ python train.py \
 #        --lr_strategy=cosine_decay \
 #        --lr=0.01 \
 #        --num_epochs=90 \
-#        --l2_decay=3e-4 \(TODO)
+#        --l2_decay=3e-4

fluid/PaddleCV/image_classification/train.py

@@ -10,7 +10,6 @@ import math
 import paddle
 import paddle.fluid as fluid
 import paddle.dataset.flowers as flowers
-import models
 import reader
 import argparse
 import functools
@@ -19,8 +18,8 @@ import utils
 from utils.learning_rate import cosine_decay
 from utils.fp16_utils import create_master_params_grads, master_param_to_train_param
 from utility import add_arguments, print_arguments
-import models
-import models_name
+
+IMAGENET1000 = 1281167
 
 parser = argparse.ArgumentParser(description=__doc__)
 add_arg = functools.partial(add_arguments, argparser=parser)
@@ -40,25 +39,32 @@ add_arg('lr_strategy',      str,   "piecewise_decay",    "Set the learning rate
 add_arg('model',            str,   "SE_ResNeXt50_32x4d", "Set the network to use.")
 add_arg('enable_ce',        bool,  False,                "If set True, enable continuous evaluation job.")
 add_arg('data_dir',         str,   "./data/ILSVRC2012",  "The ImageNet dataset root dir.")
-add_arg('model_category',   str,   "models",             "Whether to use models_name or not, valid value:'models','models_name'" )
+add_arg('model_category',   str,   "models",        "Whether to use models_name or not, valid value:'models','models_name'." )
 add_arg('fp16',             bool,  False,                "Enable half precision training with fp16." )
 add_arg('scale_loss',       float, 1.0,                  "Scale loss for fp16." )
+add_arg('l2_decay',         float, 1e-4,                 "L2_decay parameter.")
+add_arg('momentum_rate',    float, 0.9,                  "momentum_rate.")
 # yapf: enable
 
 
-def set_models(model):
+def set_models(model_category):
     global models
-    if model == "models":
-        models = models
+    assert model_category in ["models", "models_name"
+                              ], "{} is not in lists: {}".format(
+                                  model_category, ["models", "models_name"])
+    if model_category == "models_name":
+        import models_name as models
     else:
-        models = models_name
+        import models as models
 
 
 def optimizer_setting(params):
     ls = params["learning_strategy"]
+    l2_decay = params["l2_decay"]
+    momentum_rate = params["momentum_rate"]
     if ls["name"] == "piecewise_decay":
         if "total_images" not in params:
-            total_images = 1281167
+            total_images = IMAGENET1000
         else:
             total_images = params["total_images"]
         batch_size = ls["batch_size"]
@@ -71,16 +77,17 @@ def optimizer_setting(params):
         optimizer = fluid.optimizer.Momentum(
             learning_rate=fluid.layers.piecewise_decay(
                 boundaries=bd, values=lr),
-            momentum=0.9,
-            regularization=fluid.regularizer.L2Decay(1e-4))
+            momentum=momentum_rate,
+            regularization=fluid.regularizer.L2Decay(l2_decay))
 
     elif ls["name"] == "cosine_decay":
         if "total_images" not in params:
-            total_images = 1281167
+            total_images = IMAGENET1000
         else:
             total_images = params["total_images"]
-
         batch_size = ls["batch_size"]
+        l2_decay = params["l2_decay"]
+        momentum_rate = params["momentum_rate"]
         step = int(total_images / batch_size + 1)
 
         lr = params["lr"]
@@ -89,43 +96,42 @@ def optimizer_setting(params):
         optimizer = fluid.optimizer.Momentum(
             learning_rate=cosine_decay(
                 learning_rate=lr, step_each_epoch=step, epochs=num_epochs),
-            momentum=0.9,
-            regularization=fluid.regularizer.L2Decay(4e-5))
-    elif ls["name"] == "exponential_decay":
+            momentum=momentum_rate,
+            regularization=fluid.regularizer.L2Decay(l2_decay))
+    elif ls["name"] == "linear_decay":
         if "total_images" not in params:
-            total_images = 1281167
+            total_images = IMAGENET1000
         else:
             total_images = params["total_images"]
         batch_size = ls["batch_size"]
-        step = int(total_images / batch_size +1)
-        lr = params["lr"]
         num_epochs = params["num_epochs"]
-        learning_decay_rate_factor=ls["learning_decay_rate_factor"]
-        num_epochs_per_decay = ls["num_epochs_per_decay"]
-        NUM_GPUS = 1
-
+        start_lr = params["lr"]
+        l2_decay = params["l2_decay"]
+        momentum_rate = params["momentum_rate"]
+        end_lr = 0
+        total_step = int((total_images / batch_size) * num_epochs)
+        lr = fluid.layers.polynomial_decay(
+            start_lr, total_step, end_lr, power=1)
         optimizer = fluid.optimizer.Momentum(
-            learning_rate=fluid.layers.exponential_decay(
-                learning_rate = lr * NUM_GPUS,
-                decay_steps = step * num_epochs_per_decay / NUM_GPUS,
-                decay_rate = learning_decay_rate_factor),
-            momentum=0.9,
-
-            regularization = fluid.regularizer.L2Decay(4e-5))
-
+            learning_rate=lr,
+            momentum=momentum_rate,
+            regularization=fluid.regularizer.L2Decay(l2_decay))
     else:
         lr = params["lr"]
+        l2_decay = params["l2_decay"]
+        momentum_rate = params["momentum_rate"]
         optimizer = fluid.optimizer.Momentum(
             learning_rate=lr,
-            momentum=0.9,
-            regularization=fluid.regularizer.L2Decay(1e-4))
+            momentum=momentum_rate,
+            regularization=fluid.regularizer.L2Decay(l2_decay))
 
     return optimizer
 
+
 def net_config(image, label, model, args):
     model_list = [m for m in dir(models) if "__" not in m]
-    assert args.model in model_list,"{} is not lists: {}".format(
-        args.model, model_list)
+    assert args.model in model_list, "{} is not lists: {}".format(args.model,
+                                                                  model_list)
 
     class_dim = args.class_dim
     model_name = args.model
@@ -148,8 +154,9 @@ def net_config(image, label, model, args):
         acc_top1 = fluid.layers.accuracy(input=out0, label=label, k=1)
         acc_top5 = fluid.layers.accuracy(input=out0, label=label, k=5)
     else:
-        out = model.net(input=image, class_dim=class_dim)    
-        cost, pred = fluid.layers.softmax_with_cross_entropy(out, label, return_softmax=True) 
+        out = model.net(input=image, class_dim=class_dim)
+        cost, pred = fluid.layers.softmax_with_cross_entropy(
+            out, label, return_softmax=True)
         if args.scale_loss > 1:
             avg_cost = fluid.layers.mean(x=cost) * float(args.scale_loss)
         else:
@@ -190,19 +197,25 @@ def build_program(is_train, main_prog, startup_prog, args):
                 params["num_epochs"] = args.num_epochs
                 params["learning_strategy"]["batch_size"] = args.batch_size
                 params["learning_strategy"]["name"] = args.lr_strategy
+                params["l2_decay"] = args.l2_decay
+                params["momentum_rate"] = args.momentum_rate
 
                 optimizer = optimizer_setting(params)
-
                 if args.fp16:
                     params_grads = optimizer.backward(avg_cost)
                     master_params_grads = create_master_params_grads(
                         params_grads, main_prog, startup_prog, args.scale_loss)
                     optimizer.apply_gradients(master_params_grads)
-                    master_param_to_train_param(master_params_grads, params_grads, main_prog)
+                    master_param_to_train_param(master_params_grads,
+                                                params_grads, main_prog)
                 else:
                     optimizer.minimize(avg_cost)
+                global_lr = optimizer._global_learning_rate()
 
-    return py_reader, avg_cost, acc_top1, acc_top5
+    if is_train:
+        return py_reader, avg_cost, acc_top1, acc_top5, global_lr
+    else:
+        return py_reader, avg_cost, acc_top1, acc_top5
 
 
 def train(args):
@@ -220,7 +233,7 @@ def train(args):
         startup_prog.random_seed = 1000
         train_prog.random_seed = 1000
 
-    train_py_reader, train_cost, train_acc1, train_acc5 = build_program(
+    train_py_reader, train_cost, train_acc1, train_acc5, global_lr = build_program(
         is_train=True,
         main_prog=train_prog,
         startup_prog=startup_prog,
@@ -255,7 +268,8 @@ def train(args):
     if visible_device:
         device_num = len(visible_device.split(','))
     else:
-        device_num = subprocess.check_output(['nvidia-smi', '-L']).decode().count('\n')
+        device_num = subprocess.check_output(
+            ['nvidia-smi', '-L']).decode().count('\n')
 
     train_batch_size = args.batch_size / device_num
     test_batch_size = 16
@@ -283,11 +297,12 @@ def train(args):
         use_cuda=bool(args.use_gpu),
         loss_name=train_cost.name)
 
-    train_fetch_list = [train_cost.name, train_acc1.name, train_acc5.name]
+    train_fetch_list = [
+        train_cost.name, train_acc1.name, train_acc5.name, global_lr.name
+    ]
     test_fetch_list = [test_cost.name, test_acc1.name, test_acc5.name]
 
     params = models.__dict__[args.model]().params
-
     for pass_id in range(params["num_epochs"]):
 
         train_py_reader.start()
@@ -299,7 +314,9 @@ def train(args):
         try:
             while True:
                 t1 = time.time()
-                loss, acc1, acc5 = train_exe.run(fetch_list=train_fetch_list)
+                loss, acc1, acc5, lr = train_exe.run(
+                    fetch_list=train_fetch_list)
+
                 t2 = time.time()
                 period = t2 - t1
                 loss = np.mean(np.array(loss))
@@ -308,12 +325,14 @@ def train(args):
                 train_info[0].append(loss)
                 train_info[1].append(acc1)
                 train_info[2].append(acc5)
+                lr = np.mean(np.array(lr))
                 train_time.append(period)
+
                 if batch_id % 10 == 0:
                     print("Pass {0}, trainbatch {1}, loss {2}, \
-                        acc1 {3}, acc5 {4} time {5}"
-                          .format(pass_id, batch_id, loss, acc1, acc5,
-                                  "%2.2f sec" % period))
+                        acc1 {3}, acc5 {4}, lr{5}, time {6}"
+                          .format(pass_id, batch_id, loss, acc1, acc5, "%.5f" %
+                                  lr, "%2.2f sec" % period))
                     sys.stdout.flush()
                 batch_id += 1
         except fluid.core.EOFException:
@@ -322,7 +341,8 @@ def train(args):
         train_loss = np.array(train_info[0]).mean()
         train_acc1 = np.array(train_info[1]).mean()
         train_acc5 = np.array(train_info[2]).mean()
-        train_speed = np.array(train_time).mean() / (train_batch_size * device_num)
+        train_speed = np.array(train_time).mean() / (train_batch_size *
+                                                     device_num)
 
         test_py_reader.start()
 
@@ -394,10 +414,7 @@ def train(args):
 
 def main():
     args = parser.parse_args()
-    models_now = args.model_category
-    assert models_now in ["models", "models_name"], "{} is not in lists: {}".format(
-            models_now, ["models", "models_name"])
-    set_models(models_now)
+    set_models(args.model_category)
     print_arguments(args)
     train(args)
 

fluid/PaddleCV/metric_learning/README.md

 # Deep Metric Learning
-Metric learning is a kind of methods to learn discriminative features for each sample, with the purpose that intra-class samples have smaller distances while inter-class samples have larger distances in the learned space. With the develop of deep learning technique, metric learning methods are combined with deep neural networks to boost the performance of traditional tasks, such as face recognition/verification, human re-identification, image retrieval and so on. In this page, we introduce the way to implement deep metric learning using PaddlePaddle Fluid, including [data preparation](#data-preparation), [training](#training-a-model), [finetuning](#finetuning), [evaluation](#evaluation) and [inference](#inference).
+Metric learning is a kind of methods to learn discriminative features for each sample, with the purpose that intra-class samples have smaller distances while inter-class samples have larger distances in the learned space. With the develop of deep learning technique, metric learning methods are combined with deep neural networks to boost the performance of traditional tasks, such as face recognition/verification, human re-identification, image retrieval and so on. In this page, we introduce the way to implement deep metric learning using PaddlePaddle Fluid, including [data preparation](#data-preparation), [training](#training-metric-learning-models), [finetuning](#finetuning), [evaluation](#evaluation), [inference](#inference) and [Performances](#performances).
 
 ---
 ## Table of Contents
 - [Installation](#installation)
 - [Data preparation](#data-preparation)
-- [Training metric learning models](#training-a-model)
+- [Training metric learning models](#training-metric-learning-models)
 - [Finetuning](#finetuning)
 - [Evaluation](#evaluation)
 - [Inference](#inference)
-- [Performances](#supported-models)
+- [Performances](#performances)
 
 ## Installation
 
@@ -17,7 +17,7 @@ Running sample code in this directory requires PaddelPaddle Fluid v0.14.0 and la
 
 ## Data preparation
 
-Stanford Online Product(SOP) dataset contains 120,053 images of 22,634 products downloaded from eBay.com. We use it to conduct the metric learning experiments. For training, 59,5511 out of 11,318 classes are used, and 11,316 classes(60,502 images) are held out for testing. First of all, preparation of SOP data can be done as:
+Stanford Online Product(SOP) dataset contains 120,053 images of 22,634 products downloaded from eBay.com. We use it to conduct the metric learning experiments. For training, 59,551 out of 11,318 classes are used, and 11,316 classes(60,502 images) are held out for testing. First of all, preparation of SOP data can be done as:
 ```
 cd data/
 sh download_sop.sh
@@ -25,7 +25,7 @@ sh download_sop.sh
 
 ## Training metric learning models
 
-To train a metric learning model, one need to set the neural network as backbone and the metric loss function to optimize. We train meiric learning model using softmax or [arcmargin](https://arxiv.org/abs/1801.07698) loss firstly, and then fine-turned the model using other metric learning loss, such as triplet, [quadruplet](https://arxiv.org/abs/1710.00478) and [eml](https://arxiv.org/abs/1212.6094) loss. One example of training using arcmargin loss is shown below:
+To train a metric learning model, one need to set the neural network as backbone and the metric loss function to optimize. We train meiric learning model using softmax or arcmargin loss firstly, and then fine-turned the model using other metric learning loss, such as triplet, quadruplet and eml loss. One example of training using arcmargin loss is shown below:
 
 
 ```
@@ -52,7 +52,7 @@ python train_elem.py  \
 * **use_gpu**: whether to use GPU or not. Default: True.
 * **pretrained_model**: model path for pretraining. Default: None.
 * **model_save_dir**: the directory to save trained model. Default: "output".
-* **loss_name**: loss fortraining model. Default: "softmax".
+* **loss_name**: loss for training model. Default: "softmax".
 * **arc_scale**: parameter of arcmargin loss. Default: 80.0.
 * **arc_margin**: parameter of arcmargin loss. Default: 0.15.
 * **arc_easy_margin**: parameter of arcmargin loss. Default: False.
@@ -103,3 +103,9 @@ For comparation, many metric learning models with different neural networks and
 |fine-tuned with triplet | 78.37% | 79.21%
 |fine-tuned with quadruplet | 78.10% | 79.59%
 |fine-tuned with eml | 79.32% | 80.11%
+
+## Reference
+
+- ArcFace: Additive Angular Margin Loss for Deep Face Recognition [link](https://arxiv.org/abs/1801.07698)
+- Margin Sample Mining Loss: A Deep Learning Based Method for Person Re-identification [link](https://arxiv.org/abs/1710.00478)
+- Large Scale Strongly Supervised Ensemble Metric Learning, with Applications to Face Verification and Retrieval [link](https://arxiv.org/abs/1212.6094)

fluid/PaddleCV/metric_learning/README_cn.md

+# 深度度量学习
+度量学习是一种为样本对学习具有区分性特征的方法，目的是在特征空间中，让同一个类别的样本具有较小的特征距离，不同类的样本具有较大的特征距离。随着深度学习技术的发展，基于深度神经网络的度量学习方法已经在许多视觉任务上提升了很大的性能，例如：人脸识别、人脸校验、行人重识别和图像检索等等。在本章节，介绍在PaddlePaddle Fluid里实现的几种度量学习方法和使用方法，具体包括[数据准备](#数据准备)，[模型训练](#模型训练)，[模型微调](#模型微调)，[模型评估](#模型评估)，[模型预测](#模型预测)。
+
+---
+## 简介
+- [安装](#安装)
+- [数据准备](#数据准备)
+- [模型训练](#模型训练)
+- [模型微调](#模型微调)
+- [模型评估](#模型评估)
+- [模型预测](#模型预测)
+- [模型性能](#模型性能)
+
+## 安装
+
+运行本章节代码需要在PaddlePaddle Fluid v0.14.0 或更高的版本环境。如果你的设备上的PaddlePaddle版本低于v0.14.0，请按照此[安装文档](http://www.paddlepaddle.org/docs/develop/documentation/zh/build_and_install/pip_install_cn.html)进行安装和跟新。
+
+## 数据准备
+
+Stanford Online Product(SOP) 数据集下载自eBay，包含120053张商品图片，有22634个类别。我们使用该数据集进行实验。训练时，使用59551张图片，11318个类别的数据；测试时，使用60502张图片，11316个类别。首先，SOP数据集可以使用以下脚本下载：
+```
+cd data/
+sh download_sop.sh
+```
+
+## 模型训练 
+
+为了训练度量学习模型，我们需要一个神经网络模型作为骨架模型（如ResNet50）和度量学习代价函数来进行优化。我们首先使用 softmax 或者 arcmargin 来进行训练，然后使用其它的代价函数来进行微调，例如：triplet，quadruplet和eml。下面是一个使用arcmargin训练的例子：
+
+
+```
+python train_elem.py  \
+        --model=ResNet50 \
+        --train_batch_size=256 \
+        --test_batch_size=50 \
+        --lr=0.01 \
+        --total_iter_num=30000 \
+        --use_gpu=True \
+        --pretrained_model=${path_to_pretrain_imagenet_model} \
+        --model_save_dir=${output_model_path} \
+        --loss_name=arcmargin \
+        --arc_scale=80.0 \ 
+        --arc_margin=0.15 \
+        --arc_easy_margin=False
+```
+**参数介绍:**
+* **model**: 使用的模型名字. 默认: "ResNet50".
+* **train_batch_size**: 训练的 mini-batch大小. 默认: 256.
+* **test_batch_size**: 测试的 mini-batch大小. 默认: 50.
+* **lr**: 初始学习率. 默认: 0.01.
+* **total_iter_num**: 总的训练迭代轮数. 默认: 30000.
+* **use_gpu**: 是否使用GPU. 默认: True.
+* **pretrained_model**: 预训练模型的路径. 默认: None.
+* **model_save_dir**: 保存模型的路径. 默认: "output".
+* **loss_name**: 优化的代价函数. 默认: "softmax".
+* **arc_scale**: arcmargin的参数. 默认: 80.0.
+* **arc_margin**: arcmargin的参数. 默认: 0.15.
+* **arc_easy_margin**: arcmargin的参数. 默认: False.
+
+## 模型微调
+
+网络微调是在指定的任务上加载已有的模型来微调网络。在用softmax和arcmargin训完网络后，可以继续使用triplet，quadruplet或eml来微调网络。下面是一个使用eml来微调网络的例子：
+
+```
+python train_pair.py  \
+        --model=ResNet50 \
+        --train_batch_size=160 \
+        --test_batch_size=50 \
+        --lr=0.0001 \
+        --total_iter_num=100000 \
+        --use_gpu=True \
+        --pretrained_model=${path_to_pretrain_arcmargin_model} \
+        --model_save_dir=${output_model_path} \
+        --loss_name=eml \
+        --samples_each_class=2
+```
+
+## 模型评估
+模型评估主要是评估模型的检索性能。这里需要设置```path_to_pretrain_model```。可以使用下面命令来计算Recall@Rank-1。
+```
+python eval.py \
+       --model=ResNet50 \
+       --batch_size=50 \
+       --pretrained_model=${path_to_pretrain_model} \
+```
+
+## 模型预测
+模型预测主要是基于训练好的网络来获取图像数据的特征，下面是模型预测的例子：
+```
+python infer.py \
+       --model=ResNet50 \
+       --batch_size=1 \         
+       --pretrained_model=${path_to_pretrain_model}
+```
+
+## 模型性能
+
+下面列举了几种度量学习的代价函数在SOP数据集上的检索效果，这里使用Recall@Rank-1来进行评估。
+
+|预训练模型 | softmax | arcmargin
+|- | - | -:
+|未微调 | 77.42% | 78.11%
+|使用triplet微调 | 78.37% | 79.21%
+|使用quadruplet微调 | 78.10% | 79.59%
+|使用eml微调 | 79.32% | 80.11%
+
+## 引用
+
+- ArcFace: Additive Angular Margin Loss for Deep Face Recognition [链接](https://arxiv.org/abs/1801.07698)
+- Margin Sample Mining Loss: A Deep Learning Based Method for Person Re-identification [链接](https://arxiv.org/abs/1710.00478)
+- Large Scale Strongly Supervised Ensemble Metric Learning, with Applications to Face Verification and Retrieval [链接](https://arxiv.org/abs/1212.6094)

fluid/PaddleCV/metric_learning/reader.py

@@ -63,6 +63,7 @@ def common_iterator(data, settings):
     assert (batch_size % samples_each_class == 0)
     class_num = batch_size // samples_each_class 
     def train_iterator():
+        count = 0
         labs = list(data.keys())
         lab_num = len(labs)
         ind = list(range(0, lab_num))
@@ -79,6 +80,9 @@ def common_iterator(data, settings):
                 for anchor_ind_i in anchor_ind:
                     anchor_path = DATA_DIR + data_list[anchor_ind_i]
                     yield anchor_path, lab
+            count += 1
+            if count >= settings.total_iter_num + 1:
+                return
 
     return train_iterator
 
@@ -86,6 +90,8 @@ def triplet_iterator(data, settings):
     batch_size = settings.train_batch_size
     assert (batch_size % 3 == 0)
     def train_iterator():
+        total_count = settings.train_batch_size * (settings.total_iter_num + 1)
+        count = 0
         labs = list(data.keys())
         lab_num = len(labs)
         ind = list(range(0, lab_num))
@@ -108,16 +114,24 @@ def triplet_iterator(data, settings):
             yield pos_path, lab_pos
             neg_path = DATA_DIR + neg_data_list[neg_ind]
             yield neg_path, lab_neg
+            count += 3
+            if count >= total_count:
+                return
 
     return train_iterator
 
 def arcmargin_iterator(data, settings):
     def train_iterator():
+        total_count = settings.train_batch_size * (settings.total_iter_num + 1)
+        count = 0
         while True:
             for items in data:
                 path, label = items
                 path = DATA_DIR + path
                 yield path, label
+                count += 1
+                if count >= total_count:
+                    return
     return train_iterator
 
 def image_iterator(data, mode):

fluid/PaddleCV/object_detection/README.md

@@ -21,9 +21,7 @@ SSD is readily pluggable into a wide variant standard convolutional network, suc
 
 ### Data Preparation
 
-You can use [PASCAL VOC dataset](http://host.robots.ox.ac.uk/pascal/VOC/) or [MS-COCO dataset](http://cocodataset.org/#download).
-
-If you want to train a model on PASCAL VOC dataset, please download dataset at first, skip this step if you already have one.
+Please download [PASCAL VOC dataset](http://host.robots.ox.ac.uk/pascal/VOC/) at first, skip this step if you already have one.
 
 ```bash
 cd data/pascalvoc
@@ -32,30 +30,18 @@ cd data/pascalvoc
 
 The command `download.sh` also will create training and testing file lists.
 
-If you want to train a model on MS-COCO dataset, please download dataset at first, skip this step if you already have one.
-
-```
-cd data/coco
-./download.sh
-```
-
 ### Train
 
 #### Download the Pre-trained Model.
 
-We provide two pre-trained models. The one is MobileNet-v1 SSD trained on COCO dataset, but removed the convolutional predictors for COCO dataset. This model can be used to initialize the models when training other datasets, like PASCAL VOC. The other pre-trained model is MobileNet-v1 trained on ImageNet 2012 dataset but removed the last weights and bias in the Fully-Connected layer.
-
-Declaration: the MobileNet-v1 SSD model is converted by [TensorFlow model](https://github.com/tensorflow/models/blob/f87a58cd96d45de73c9a8330a06b2ab56749a7fa/research/object_detection/g3doc/detection_model_zoo.md). The MobileNet-v1 model is converted from [Caffe](https://github.com/shicai/MobileNet-Caffe).
-We will release the pre-trained models by ourself in the upcoming soon.
+We provide two pre-trained models. The one is MobileNet-v1 SSD trained on COCO dataset, but removed the convolutional predictors for COCO dataset. This model can be used to initialize the models when training other datasets, like PASCAL VOC. The other pre-trained model is MobileNet-v1 trained on ImageNet 2012 dataset but removed the last weights and bias in the Fully-Connected layer. Download MobileNet-v1 SSD:
 
-  - Download MobileNet-v1 SSD:
     ```bash
     ./pretrained/download_coco.sh
     ```
-  - Download MobileNet-v1:
-    ```bash
-    ./pretrained/download_imagenet.sh
-    ```
+
+Declaration: the MobileNet-v1 SSD model is converted by [TensorFlow model](https://github.com/tensorflow/models/blob/f87a58cd96d45de73c9a8330a06b2ab56749a7fa/research/object_detection/g3doc/detection_model_zoo.md).
+
 
 #### Train on PASCAL VOC
 
@@ -64,7 +50,6 @@ We will release the pre-trained models by ourself in the upcoming soon.
   python -u train.py --batch_size=64 --dataset='pascalvoc' --pretrained_model='pretrained/ssd_mobilenet_v1_coco/'
   ```
    - Set ```export CUDA_VISIBLE_DEVICES=0,1``` to specifiy the number of GPU you want to use.
-   - Set ```--dataset='coco2014'``` or ```--dataset='coco2017'``` to train model on MS COCO dataset.
    - For more help on arguments:
 
   ```bash
@@ -88,19 +73,6 @@ You can evaluate your trained model in different metrics like 11point, integral
 python eval.py --dataset='pascalvoc' --model_dir='train_pascal_model/best_model' --data_dir='data/pascalvoc' --test_list='test.txt' --ap_version='11point' --nms_threshold=0.45
 ```
 
-You can set ```--dataset``` to ```coco2014``` or ```coco2017``` to evaluate COCO dataset. Moreover, we provide `eval_coco_map.py` which uses a COCO-specific mAP metric defined by [COCO committee](http://cocodataset.org/#detections-eval). To use this eval_coco_map.py, [cocoapi](https://github.com/cocodataset/cocoapi) is needed.
-Install the cocoapi:
-```
-# COCOAPI=/path/to/clone/cocoapi
-git clone https://github.com/cocodataset/cocoapi.git $COCOAPI
-cd $COCOAPI/PythonAPI
-# Install into global site-packages
-make install
-# Alternatively, if you do not have permissions or prefer
-# not to install the COCO API into global site-packages
-python2 setup.py install --user
-```
-
 ### Infer and Visualize
 `infer.py` is the main caller of the inferring module. Examples of usage are shown below.
 ```bash

fluid/PaddleCV/object_detection/README_cn.md

@@ -21,9 +21,8 @@ SSD 可以方便地插入到任何一种标准卷积网络中，比如 VGG、Res
 
 ### 数据准备
 
-你可以使用 [PASCAL VOC 数据集](http://host.robots.ox.ac.uk/pascal/VOC/) 或者 [MS-COCO 数据集](http://cocodataset.org/#download)。
 
-如果你想在 PASCAL VOC 数据集上进行训练，请先使用下面的命令下载数据集。
+请先使用下面的命令下载 [PASCAL VOC 数据集](http://host.robots.ox.ac.uk/pascal/VOC/)：
 
 ```bash
 cd data/pascalvoc
@@ -32,29 +31,19 @@ cd data/pascalvoc
 
 `download.sh` 命令会自动创建训练和测试用的列表文件。
 
-如果你想在 MS-COCO 数据集上进行训练，请先使用下面的命令下载数据集。
-
-```
-cd data/coco
-./download.sh
-```
 
 ### 模型训练
 
 #### 下载预训练模型
 
-我们提供了两个预训练模型。第一个模型是在 COCO 数据集上预训练的 MobileNet-v1 SSD，我们将它的预测头移除了以便在 COCO 以外的数据集上进行训练。第二个模型是在 ImageNet 2012 数据集上预训练的 MobileNet-v1，我们也将最后的全连接层移除以便进行目标检测训练。
-
-声明：MobileNet-v1 SSD 模型转换自[TensorFlow model](https://github.com/tensorflow/models/blob/f87a58cd96d45de73c9a8330a06b2ab56749a7fa/research/object_detection/g3doc/detection_model_zoo.md)。MobileNet-v1 模型转换自[Caffe](https://github.com/shicai/MobileNet-Caffe)。我们不久也会发布我们自己预训练的模型。
+我们提供了两个预训练模型。第一个模型是在 COCO 数据集上预训练的 MobileNet-v1 SSD，我们将它的预测头移除了以便在 COCO 以外的数据集上进行训练。第二个模型是在 ImageNet 2012 数据集上预训练的 MobileNet-v1，我们也将最后的全连接层移除以便进行目标检测训练。下载 MobileNet-v1 SSD:
 
-  - 下载 MobileNet-v1 SSD:
     ```bash
     ./pretrained/download_coco.sh
     ```
-  - 下载 MobileNet-v1:
-    ```bash
-    ./pretrained/download_imagenet.sh
-    ```
+
+声明：MobileNet-v1 SSD 模型转换自[TensorFlow model](https://github.com/tensorflow/models/blob/f87a58cd96d45de73c9a8330a06b2ab56749a7fa/research/object_detection/g3doc/detection_model_zoo.md)。MobileNet-v1 模型转换自[Caffe](https://github.com/shicai/MobileNet-Caffe)。
+
 
 #### 训练
 
@@ -63,7 +52,6 @@ cd data/coco
   python -u train.py --batch_size=64 --dataset='pascalvoc' --pretrained_model='pretrained/ssd_mobilenet_v1_coco/'
   ```
    - 可以通过设置 ```export CUDA_VISIBLE_DEVICES=0,1``` 指定想要使用的GPU数量。
-   - 可以通过设置 ```--dataset='coco2014'``` 或 ```--dataset='coco2017'``` 指定训练 MS-COCO数据集。
    - 更多的可选参数见:
 
   ```bash
@@ -80,25 +68,13 @@ cd data/coco
 
 ### 模型评估
 
-你可以使用11point、integral等指标在PASCAL VOC 和 COCO 数据集上评估训练好的模型。不失一般性，我们采用相应数据集的测试列表作为样例代码的默认列表，你也可以通过设置```--test_list```来指定自己的测试样本列表。
+你可以使用11point、integral等指标在PASCAL VOC 数据集上评估训练好的模型。不失一般性，我们采用相应数据集的测试列表作为样例代码的默认列表，你也可以通过设置```--test_list```来指定自己的测试样本列表。
 
 `eval.py`是评估模块的主要执行程序，调用示例如下：
 ```bash
 python eval.py --dataset='pascalvoc' --model_dir='train_pascal_model/best_model' --data_dir='data/pascalvoc' --test_list='test.txt' --ap_version='11point' --nms_threshold=0.45
 ```
 
-你可以设置```--dataset``` 为 ```coco2014``` 或 ```coco2017```来评估 COCO 数据集。我们也提供了`eval_coco_map.py`以进行[COCO官方评估](http://cocodataset.org/#detections-eval)。若要使用 eval_coco_map.py, 需要首先下载[cocoapi](https://github.com/cocodataset/cocoapi)：
-```
-# COCOAPI=/path/to/clone/cocoapi
-git clone https://github.com/cocodataset/cocoapi.git $COCOAPI
-cd $COCOAPI/PythonAPI
-# Install into global site-packages
-make install
-# Alternatively, if you do not have permissions or prefer
-# not to install the COCO API into global site-packages
-python2 setup.py install --user
-```
-
 ### 模型预测以及可视化
 
 `infer.py`是预测及可视化模块的主要执行程序，调用示例如下：

fluid/PaddleCV/object_detection/README_quant.md

@@ -2,7 +2,7 @@
 
 ### Introduction
 
-The quantization-aware training used in this experiments is introduced in [fixed-point quantization desigin](https://gthub.com/PaddlePaddle/FluidDoc/blob/develop/doc/fluid/design/quantization/fixed_point_quantization.md). Since quantization-aware training is still an active area of research and experimentation,
+The quantization-aware training used in this experiments is introduced in [fixed-point quantization desigin](https://github.com/PaddlePaddle/FluidDoc/blob/develop/doc/fluid/design/quantization/fixed_point_quantization.md). Since quantization-aware training is still an active area of research and experimentation,
 here, we just give an simple quantization training usage in Fluid based on MobileNet-SSD model, and more other exeperiments are still needed, like how to quantization traning by considering fusing batch normalization and convolution/fully-connected layers, channel-wise quantization of weights and so on.
 
 
@@ -130,6 +130,9 @@ A Python transpiler is used to rewrite Fluid training program or evaluation prog
   ```
   See 002271.jpg for the visualized image with bbouding boxes.
 
+
+  **Note**, if you want to convert model to 8-bit, you should call `fluid.contrib.QuantizeTranspiler.convert_to_int8` to do this. But, now Paddle can't load 8-bit model to do inference.
+
 ### Results
 
 Results of MobileNet-v1-SSD 300x300 model on PascalVOC dataset.

fluid/PaddleCV/object_detection/_ce.py

@@ -9,10 +9,10 @@ from kpi import CostKpi, DurationKpi, AccKpi
 
 train_cost_kpi = CostKpi('train_cost', 0.02, 0, actived=True)
 test_acc_kpi = AccKpi('test_acc', 0.01, 0, actived=False)
-train_speed_kpi = AccKpi('train_speed', 0.1, 0, actived=True)
+train_speed_kpi = DurationKpi('train_speed', 0.1, 0, actived=True, unit_repr="s/epoch")
 train_cost_card4_kpi = CostKpi('train_cost_card4', 0.02, 0, actived=True)
 test_acc_card4_kpi = AccKpi('test_acc_card4', 0.01, 0, actived=False)
-train_speed_card4_kpi = AccKpi('train_speed_card4', 0.1, 0, actived=True)
+train_speed_card4_kpi = DurationKpi('train_speed_card4', 0.1, 0, actived=True, unit_repr="s/epoch")
 
 tracking_kpis = [
     train_cost_kpi,

fluid/PaddleCV/object_detection/data_util.py

-"""
-This code is based on https://github.com/fchollet/keras/blob/master/keras/utils/data_utils.py
-"""
-
-import time
-import numpy as np
-import threading
-import multiprocessing
-try:
-    import queue
-except ImportError:
-    import Queue as queue
-
-
-class GeneratorEnqueuer(object):
-    """
-    Builds a queue out of a data generator.
-
-    Args:
-        generator: a generator function which endlessly yields data
-        use_multiprocessing (bool): use multiprocessing if True,
-            otherwise use threading.
-        wait_time (float): time to sleep in-between calls to `put()`.
-        random_seed (int): Initial seed for workers,
-            will be incremented by one for each workers.
-    """
-
-    def __init__(self,
-                 generator,
-                 use_multiprocessing=False,
-                 wait_time=0.05,
-                 random_seed=None):
-        self.wait_time = wait_time
-        self._generator = generator
-        self._use_multiprocessing = use_multiprocessing
-        self._threads = []
-        self._stop_event = None
-        self.queue = None
-        self._manager = None
-        self.seed = random_seed
-
-    def start(self, workers=1, max_queue_size=10):
-        """
-        Start worker threads which add data from the generator into the queue.
-
-        Args:
-            workers (int): number of worker threads
-            max_queue_size (int): queue size
-                (when full, threads could block on `put()`)
-        """
-
-        def data_generator_task():
-            """
-            Data generator task.
-            """
-
-            def task():
-                if (self.queue is not None and
-                        self.queue.qsize() < max_queue_size):
-                    generator_output = next(self._generator)
-                    self.queue.put((generator_output))
-                else:
-                    time.sleep(self.wait_time)
-
-            if not self._use_multiprocessing:
-                while not self._stop_event.is_set():
-                    with self.genlock:
-                        try:
-                            task()
-                        except Exception:
-                            traceback.print_exc()
-                            self._stop_event.set()
-                            break
-            else:
-                while not self._stop_event.is_set():
-                    try:
-                        task()
-                    except Exception:
-                        traceback.print_exc()
-                        self._stop_event.set()
-                        break
-
-        try:
-            if self._use_multiprocessing:
-                self._manager = multiprocessing.Manager()
-                self.queue = self._manager.Queue(maxsize=max_queue_size)
-                self._stop_event = multiprocessing.Event()
-            else:
-                self.genlock = threading.Lock()
-                self.queue = queue.Queue()
-                self._stop_event = threading.Event()
-            for _ in range(workers):
-                if self._use_multiprocessing:
-                    # Reset random seed else all children processes
-                    # share the same seed
-                    np.random.seed(self.seed)
-                    thread = multiprocessing.Process(target=data_generator_task)
-                    thread.daemon = True
-                    if self.seed is not None:
-                        self.seed += 1
-                else:
-                    thread = threading.Thread(target=data_generator_task)
-                self._threads.append(thread)
-                thread.start()
-        except:
-            self.stop()
-            raise
-
-    def is_running(self):
-        """
-        Returns:
-            bool: Whether the worker theads are running.
-        """
-        return self._stop_event is not None and not self._stop_event.is_set()
-
-    def stop(self, timeout=None):
-        """
-        Stops running threads and wait for them to exit, if necessary.
-        Should be called by the same thread which called `start()`.
-
-        Args:
-            timeout(int|None): maximum time to wait on `thread.join()`.
-        """
-        if self.is_running():
-            self._stop_event.set()
-        for thread in self._threads:
-            if self._use_multiprocessing:
-                if thread.is_alive():
-                    thread.terminate()
-            else:
-                thread.join(timeout)
-        if self._manager:
-            self._manager.shutdown()
-
-        self._threads = []
-        self._stop_event = None
-        self.queue = None
-
-    def get(self):
-        """
-        Creates a generator to extract data from the queue.
-        Skip the data if it is `None`.
-
-        # Yields
-            tuple of data in the queue.
-        """
-        while self.is_running():
-            if not self.queue.empty():
-                inputs = self.queue.get()
-                if inputs is not None:
-                    yield inputs
-            else:
-                time.sleep(self.wait_time)

fluid/PaddleCV/object_detection/eval.py

@@ -52,7 +52,7 @@ def build_program(main_prog, startup_prog, args, data_args):
             nmsed_out = fluid.layers.detection_output(
                 locs, confs, box, box_var, nms_threshold=args.nms_threshold)
             with fluid.program_guard(main_prog):
-                map = fluid.evaluator.DetectionMAP(
+                map = fluid.metrics.DetectionMAP(
                     nmsed_out,
                     gt_label,
                     gt_box,

fluid/PaddleCV/object_detection/eval_coco_map.py

@@ -47,7 +47,7 @@ def eval(args, data_args, test_list, batch_size, model_dir=None):
     gt_iscrowd = fluid.layers.data(
         name='gt_iscrowd', shape=[1], dtype='int32', lod_level=1)
     gt_image_info = fluid.layers.data(
-        name='gt_image_id', shape=[3], dtype='int32', lod_level=1)
+        name='gt_image_id', shape=[3], dtype='int32')
 
     locs, confs, box, box_var = mobile_net(num_classes, image, image_shape)
     nmsed_out = fluid.layers.detection_output(
@@ -57,14 +57,14 @@ def eval(args, data_args, test_list, batch_size, model_dir=None):
 
     place = fluid.CUDAPlace(0) if args.use_gpu else fluid.CPUPlace()
     exe = fluid.Executor(place)
+    exe.run(fluid.default_startup_program())
     # yapf: disable
     if model_dir:
         def if_exist(var):
             return os.path.exists(os.path.join(model_dir, var.name))
         fluid.io.load_vars(exe, model_dir, predicate=if_exist)
     # yapf: enable
-    test_reader = paddle.batch(
-        reader.test(data_args, test_list), batch_size=batch_size)
+    test_reader = reader.test(data_args, test_list, batch_size)
     feeder = fluid.DataFeeder(
         place=place,
         feed_list=[image, gt_box, gt_label, gt_iscrowd, gt_image_info])
@@ -146,8 +146,7 @@ if __name__ == '__main__':
         mean_value=[args.mean_value_B, args.mean_value_G, args.mean_value_R],
         apply_distort=False,
         apply_expand=False,
-        ap_version=args.ap_version,
-        toy=0)
+        ap_version=args.ap_version)
     eval(
         args,
         data_args=data_args,

fluid/PaddleCV/object_detection/main_quant.py

@@ -85,7 +85,6 @@ def train(args,
 
     batch_size = train_params['batch_size']
     batch_size_per_device = batch_size // devices_num
-    iters_per_epoc = train_params["train_images"] // batch_size
     num_workers = 4
 
     startup_prog = fluid.Program()
@@ -134,22 +133,22 @@ def train(args,
                                 train_file_list,
                                 batch_size_per_device,
                                 shuffle=is_shuffle,
-                                use_multiprocessing=True,
-                                num_workers=num_workers,
-                                max_queue=24)
+                                num_workers=num_workers)
     test_reader = reader.test(data_args, val_file_list, batch_size)
     train_py_reader.decorate_paddle_reader(train_reader)
     test_py_reader.decorate_paddle_reader(test_reader)
 
     train_py_reader.start()
     best_map = 0.
-    try:
-        for epoc in range(epoc_num):
-            if epoc == 0:
-                # test quantized model without quantization-aware training.
-                test_map = test(exe, test_prog, map_eval, test_py_reader)
-            # train
-            for batch in range(iters_per_epoc):
+    for epoc in range(epoc_num):
+        if epoc == 0:
+            # test quantized model without quantization-aware training.
+            test_map = test(exe, test_prog, map_eval, test_py_reader)
+        batch = 0
+        train_py_reader.start()
+        while True:
+            try:
+                # train
                 start_time = time.time()
                 if parallel:
                     outs = train_exe.run(fetch_list=[loss.name])
@@ -157,18 +156,19 @@ def train(args,
                     outs = exe.run(train_prog, fetch_list=[loss])
                 end_time = time.time()
                 avg_loss = np.mean(np.array(outs[0]))
-                if batch % 20 == 0:
+                if batch % 10 == 0:
                     print("Epoc {:d}, batch {:d}, loss {:.6f}, time {:.5f}".format(
                         epoc , batch, avg_loss, end_time - start_time))
-            end_time = time.time()
-            test_map = test(exe, test_prog, map_eval, test_py_reader)
-            save_model(exe, train_prog, model_save_dir, str(epoc))
-            if test_map > best_map:
-                best_map = test_map
-                save_model(exe, train_prog, model_save_dir, 'best_map')
-            print("Best test map {0}".format(best_map))
-    except (fluid.core.EOFException, StopIteration):
-        train_py_reader.reset()
+            except (fluid.core.EOFException, StopIteration):
+                train_reader().close()
+                train_py_reader.reset()
+                break
+        test_map = test(exe, test_prog, map_eval, test_py_reader)
+        save_model(exe, train_prog, model_save_dir, str(epoc))
+        if test_map > best_map:
+            best_map = test_map
+            save_model(exe, train_prog, model_save_dir, 'best_map')
+        print("Best test map {0}".format(best_map))
 
 
 def eval(args, data_args, configs, val_file_list):
@@ -212,6 +212,9 @@ def eval(args, data_args, configs, val_file_list):
 
     test_map = test(exe, test_prog, map_eval, test_py_reader)
     print("Test model {0}, map {1}".format(init_model, test_map))
+    # convert model to 8-bit before saving, but now Paddle can't load
+    # the 8-bit model to do inference.
+    # transpiler.convert_to_int8(test_prog, place)
     fluid.io.save_inference_model(model_save_dir, [image.name],
                                   [nmsed_out], exe, test_prog)
 

fluid/PaddleCV/object_detection/reader.py

@@ -12,17 +12,17 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-import image_util
-from paddle.utils.image_util import *
-from PIL import Image
-from PIL import ImageDraw
-import numpy as np
 import xml.etree.ElementTree
 import os
 import time
 import copy
 import six
-from data_util import GeneratorEnqueuer
+import math
+import numpy as np
+from PIL import Image
+from PIL import ImageDraw
+import image_util
+import paddle
 
 
 class Settings(object):
@@ -162,26 +162,14 @@ def preprocess(img, bbox_labels, mode, settings):
     return img, sampled_labels
 
 
-def coco(settings, file_list, mode, batch_size, shuffle):
-    # cocoapi
+def coco(settings, coco_api, file_list, mode, batch_size, shuffle, data_dir):
     from pycocotools.coco import COCO
-    from pycocotools.cocoeval import COCOeval
-
-    coco = COCO(file_list)
-    image_ids = coco.getImgIds()
-    images = coco.loadImgs(image_ids)
-    print("{} on {} with {} images".format(mode, settings.dataset, len(images)))
 
     def reader():
         if mode == 'train' and shuffle:
-            np.random.shuffle(images)
+            np.random.shuffle(file_list)
         batch_out = []
-        if '2014' in file_list:
-            sub_dir = "train2014" if model == "train" else "val2014"
-        elif '2017' in file_list:
-            sub_dir = "train2017" if mode == "train" else "val2017"
-        data_dir = os.path.join(settings.data_dir, sub_dir)
-        for image in images:
+        for image in file_list:
             image_name = image['file_name']
             image_path = os.path.join(data_dir, image_name)
             if not os.path.exists(image_path):
@@ -195,8 +183,8 @@ def coco(settings, file_list, mode, batch_size, shuffle):
 
             # layout: category_id | xmin | ymin | xmax | ymax | iscrowd
             bbox_labels = []
-            annIds = coco.getAnnIds(imgIds=image['id'])
-            anns = coco.loadAnns(annIds)
+            annIds = coco_api.getAnnIds(imgIds=image['id'])
+            anns = coco_api.loadAnns(annIds)
             for ann in anns:
                 bbox_sample = []
                 # start from 1, leave 0 to background
@@ -236,16 +224,12 @@ def coco(settings, file_list, mode, batch_size, shuffle):
 
 
 def pascalvoc(settings, file_list, mode, batch_size, shuffle):
-    flist = open(file_list)
-    images = [line.strip() for line in flist]
-    print("{} on {} with {} images".format(mode, settings.dataset, len(images)))
-
     def reader():
         if mode == 'train' and shuffle:
-            np.random.shuffle(images)
+            np.random.shuffle(file_list)
         batch_out = []
         cnt = 0
-        for image in images:
+        for image in file_list:
             image_path, label_path = image.split()
             image_path = os.path.join(settings.data_dir, image_path)
             label_path = os.path.join(settings.data_dir, label_path)
@@ -299,52 +283,55 @@ def train(settings,
           file_list,
           batch_size,
           shuffle=True,
-          use_multiprocessing=True,
           num_workers=8,
-          max_queue=24,
           enable_ce=False):
-    file_list = os.path.join(settings.data_dir, file_list)
+    file_path = os.path.join(settings.data_dir, file_list)
+    readers = []
     if 'coco' in settings.dataset:
-        generator = coco(settings, file_list, "train", batch_size, shuffle)
-    else:
-        generator = pascalvoc(settings, file_list, "train", batch_size, shuffle)
+        # cocoapi
+        from pycocotools.coco import COCO
+        coco_api = COCO(file_path)
+        image_ids = coco_api.getImgIds()
+        images = coco_api.loadImgs(image_ids)
+        n = int(math.ceil(len(images) // num_workers))
+        image_lists = [images[i:i + n] for i in range(0, len(images), n)]
 
-    def infinite_reader():
-        while True:
-            for data in generator():
-                yield data
-
-    def reader():
-        try:
-            enqueuer = GeneratorEnqueuer(
-                infinite_reader(), use_multiprocessing=use_multiprocessing)
-            enqueuer.start(max_queue_size=max_queue, workers=num_workers)
-            generator_output = None
-            while True:
-                while enqueuer.is_running():
-                    if not enqueuer.queue.empty():
-                        generator_output = enqueuer.queue.get()
-                        break
-                    else:
-                        time.sleep(0.02)
-                yield generator_output
-                generator_output = None
-        finally:
-            if enqueuer is not None:
-                enqueuer.stop()
-
-    if enable_ce:
-        return infinite_reader
+        if '2014' in file_list:
+            sub_dir = "train2014"
+        elif '2017' in file_list:
+            sub_dir = "train2017"
+        data_dir = os.path.join(settings.data_dir, sub_dir)
+        for l in image_lists:
+            readers.append(
+                coco(settings, coco_api, l, 'train', batch_size, shuffle,
+                     data_dir))
     else:
-        return reader
+        images = [line.strip() for line in open(file_path)]
+        n = int(math.ceil(len(images) // num_workers))
+        image_lists = [images[i:i + n] for i in range(0, len(images), n)]
+        for l in image_lists:
+            readers.append(pascalvoc(settings, l, 'train', batch_size, shuffle))
+
+    return paddle.reader.multiprocess_reader(readers, False)
 
 
 def test(settings, file_list, batch_size):
     file_list = os.path.join(settings.data_dir, file_list)
     if 'coco' in settings.dataset:
-        return coco(settings, file_list, 'test', batch_size, False)
+        from pycocotools.coco import COCO
+        coco_api = COCO(file_list)
+        image_ids = coco_api.getImgIds()
+        images = coco_api.loadImgs(image_ids)
+        if '2014' in file_list:
+            sub_dir = "val2014"
+        elif '2017' in file_list:
+            sub_dir = "val2017"
+        data_dir = os.path.join(settings.data_dir, sub_dir)
+        return coco(settings, coco_api, images, 'test', batch_size, False,
+                    data_dir)
     else:
-        return pascalvoc(settings, file_list, 'test', batch_size, False)
+        image_list = [line.strip() for line in open(file_list)]
+        return pascalvoc(settings, image_list, 'test', batch_size, False)
 
 
 def infer(settings, image_path):

fluid/PaddleCV/object_detection/train.py

@@ -105,7 +105,7 @@ def build_program(main_prog, startup_prog, train_params, is_train):
                 with fluid.unique_name.guard("inference"):
                     nmsed_out = fluid.layers.detection_output(
                         locs, confs, box, box_var, nms_threshold=0.45)
-                    map_eval = fluid.evaluator.DetectionMAP(
+                    map_eval = fluid.metrics.DetectionMAP(
                         nmsed_out,
                         gt_label,
                         gt_box,
@@ -141,7 +141,6 @@ def train(args,
     batch_size = train_params['batch_size']
     epoc_num = train_params['epoc_num']
     batch_size_per_device = batch_size // devices_num
-    iters_per_epoc = train_params["train_images"] // batch_size
     num_workers = 8
 
     startup_prog = fluid.Program()
@@ -186,9 +185,7 @@ def train(args,
                                 train_file_list,
                                 batch_size_per_device,
                                 shuffle=is_shuffle,
-                                use_multiprocessing=True,
                                 num_workers=num_workers,
-                                max_queue=24,
                                 enable_ce=enable_ce)
     test_reader = reader.test(data_args, val_file_list, batch_size)
     train_py_reader.decorate_paddle_reader(train_reader)
@@ -205,7 +202,7 @@ def train(args,
     def test(epoc_id, best_map):
         _, accum_map = map_eval.get_map_var()
         map_eval.reset(exe)
-        every_epoc_map=[]
+        every_epoc_map=[] # for CE
         test_py_reader.start()
         try:
             batch_id = 0
@@ -218,22 +215,23 @@ def train(args,
         except fluid.core.EOFException:
             test_py_reader.reset()
         mean_map = np.mean(every_epoc_map)
-        print("Epoc {0}, test map {1}".format(epoc_id, test_map))
+        print("Epoc {0}, test map {1}".format(epoc_id, test_map[0]))
         if test_map[0] > best_map:
             best_map = test_map[0]
             save_model('best_model', test_prog)
         return best_map, mean_map
 
 
-    train_py_reader.start()
     total_time = 0.0
-    try:
-        for epoc_id in range(epoc_num):
-            epoch_idx = epoc_id + 1
-            start_time = time.time()
-            prev_start_time = start_time
-            every_epoc_loss = []
-            for batch_id in range(iters_per_epoc):
+    for epoc_id in range(epoc_num):
+        epoch_idx = epoc_id + 1
+        start_time = time.time()
+        prev_start_time = start_time
+        every_epoc_loss = []
+        batch_id = 0
+        train_py_reader.start()
+        while True:
+            try:
                 prev_start_time = start_time
                 start_time = time.time()
                 if parallel:
@@ -242,34 +240,35 @@ def train(args,
                     loss_v, = exe.run(train_prog, fetch_list=[loss])
                 loss_v = np.mean(np.array(loss_v))
                 every_epoc_loss.append(loss_v)
-                if batch_id % 20 == 0:
+                if batch_id % 10 == 0:
                     print("Epoc {:d}, batch {:d}, loss {:.6f}, time {:.5f}".format(
                         epoc_id, batch_id, loss_v, start_time - prev_start_time))
-            end_time = time.time()
-            total_time += end_time - start_time
-
-            best_map, mean_map = test(epoc_id, best_map)
-            print("Best test map {0}".format(best_map))
-            if epoc_id % 10 == 0 or epoc_id == epoc_num - 1:
-                save_model(str(epoc_id), train_prog)
-
-            if enable_ce and epoc_id == epoc_num - 1:
-                train_avg_loss = np.mean(every_epoc_loss)
-                if devices_num == 1:
-                    print("kpis	train_cost	%s" % train_avg_loss)
-                    print("kpis	test_acc	%s" % mean_map)
-                    print("kpis	train_speed	%s" % (total_time / epoch_idx))
-                else:
-                    print("kpis	train_cost_card%s	%s" %
-                           (devices_num, train_avg_loss))
-                    print("kpis	test_acc_card%s	%s" %
-                           (devices_num, mean_map))
-                    print("kpis	train_speed_card%s	%f" %
-                           (devices_num, total_time / epoch_idx))
-
-    except (fluid.core.EOFException, StopIteration):
-        train_reader().close()
-        train_py_reader.reset()
+                batch_id += 1
+            except (fluid.core.EOFException, StopIteration):
+                train_reader().close()
+                train_py_reader.reset()
+                break
+
+        end_time = time.time()
+        total_time += end_time - start_time
+        best_map, mean_map = test(epoc_id, best_map)
+        print("Best test map {0}".format(best_map))
+        if epoc_id % 10 == 0 or epoc_id == epoc_num - 1:
+            save_model(str(epoc_id), train_prog)
+
+    if enable_ce:
+        train_avg_loss = np.mean(every_epoc_loss)
+        if devices_num == 1:
+            print("kpis	train_cost	%s" % train_avg_loss)
+            print("kpis	test_acc	%s" % mean_map)
+            print("kpis	train_speed	%s" % (total_time / epoch_idx))
+        else:
+            print("kpis	train_cost_card%s	%s" %
+                   (devices_num, train_avg_loss))
+            print("kpis	test_acc_card%s	%s" %
+                   (devices_num, mean_map))
+            print("kpis	train_speed_card%s	%f" %
+                   (devices_num, total_time / epoch_idx))
 
 
 if __name__ == '__main__':

fluid/PaddleCV/ocr_recognition/README.md

@@ -202,5 +202,5 @@ env CUDA_VISIBLE_DEVICE=0 python infer.py \
 
 |模型| 错误率|
 |- |:-: |
-|[ocr_ctc_params](https://drive.google.com/open?id=1gsg2ODO2_F2pswXwW5MXpf8RY8-BMRyZ) | 22.3% |
-|[ocr_attention_params](https://drive.google.com/open?id=1Bx7-94mngyTaMA5kVjzYHDPAdXxOYbRm) | 15.8%|
+|[ocr_ctc_params](https://paddle-ocr-models.bj.bcebos.com/ocr_ctc.zip) | 22.3% |
+|[ocr_attention_params](https://paddle-ocr-models.bj.bcebos.com/ocr_attention.zip) | 15.8%|

fluid/PaddleCV/faster_rcnn/README.md → fluid/PaddleCV/rcnn/README.md

-# Faster RCNN Objective Detection
+# RCNN Objective Detection
 
 ---
 ## Table of Contents
@@ -9,7 +9,6 @@
 - [Training](#training)
 - [Evaluation](#evaluation)
 - [Inference and Visualization](#inference-and-visualization)
-- [Appendix](#appendix)
 
 ## Installation
 
@@ -17,17 +16,20 @@ Running sample code in this directory requires PaddelPaddle Fluid v.1.0.0 and la
 
 ## Introduction
 
-[Faster Rcnn](https://arxiv.org/abs/1506.01497) is a typical two stage detector. The total framework of network can be divided into four parts, as shown below:
-<p align="center">
-<img src="image/Faster_RCNN.jpg" height=400 width=400 hspace='10'/> <br />
-Faster RCNN model
-</p>
+Region Convolutional Neural Network (RCNN) models are two stages detector. According to proposals and feature extraction, obtain class and more precise proposals.
+Now RCNN model contains two typical models: Faster RCNN and Mask RCNN.
+
+[Faster RCNN](https://arxiv.org/abs/1506.01497), The total framework of network can be divided into four parts:
 
 1. Base conv layer. As a CNN objective dection, Faster RCNN extract feature maps using a basic convolutional network. The feature maps then can be shared by RPN and fc layers. This sampel uses [ResNet-50](https://arxiv.org/abs/1512.03385) as base conv layer.
 2. Region Proposal Network (RPN). RPN generates proposals for detection。This block generates anchors by a set of size and ratio and classifies anchors into fore-ground and back-ground by softmax. Then refine anchors to obtain more precise proposals using box regression.
 3. RoI Align. This layer takes feature maps and proposals as input. The proposals are mapped to feature maps and pooled to the same size. The output are sent to fc layers for classification and regression. RoIPool and RoIAlign are used separately to this layer and it can be set in roi\_func in config.py.
 4. Detection layer. Using the output of roi pooling to compute the class and locatoin of each proposal in two fc layers.
 
+[Mask RCNN](https://arxiv.org/abs/1703.06870) is a classical instance segmentation model and an extension of Faster RCNN
+
+Mask RCNN is a two stage model as well. At the first stage, it generates proposals from input images. At the second stage, it obtains class result, bbox and mask which is the result from segmentation branch on original Faster RCNN model. It decouples the relation between mask and classification.  
+
 ## Data preparation
 
 Train the model on [MS-COCO dataset](http://cocodataset.org/#download), download dataset as below:
@@ -62,12 +64,24 @@ To train the model, [cocoapi](https://github.com/cocodataset/cocoapi) is needed.
 
 After data preparation, one can start the training step by:
 
+- Faster RCNN
+
     python train.py \
        --model_save_dir=output/ \
-       --pretrained_model=${path_to_pretrain_model}
-       --data_dir=${path_to_data}
+       --pretrained_model=${path_to_pretrain_model} \
+       --data_dir=${path_to_data} \
+       --MASK_ON=False
+
+- Mask RCNN
+
+    python train.py \
+       --model_save_dir=output/ \
+       --pretrained_model=${path_to_pretrain_model} \
+       --data_dir=${path_to_data} \
+       --MASK_ON=True
 
 - Set ```export CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7``` to specifiy 8 GPU to train.
+- Set ```MASK_ON``` to choose Faster RCNN or Mask RCNN model.
 - For more help on arguments:
 
     python train.py --help
@@ -93,7 +107,6 @@ After data preparation, one can start the training step by:
 *  In first 500 iteration, the learning rate increases linearly from 0.00333 to 0.01. Then lr is decayed at 120000, 160000 iteration with multiplier 0.1, 0.01. The maximum iteration is 180000. Also, we released a 2x model which has 360000 iterations and lr is decayed at 240000, 320000. These configuration can be set by max_iter and lr_steps in config.py.
 *  Set the learning rate of bias to two times as global lr in non basic convolutional layers.
 *  In basic convolutional layers, parameters of affine layers and res body do not update.
-*  Use Nvidia Tesla V100 8GPU, total time for training is about 40 hours.
 
 ## Evaluation
 
@@ -101,14 +114,27 @@ Evaluation is to evaluate the performance of a trained model. This sample provid
 
 `eval_coco_map.py` is the main executor for evalution, one can start evalution step by:
 
+- Faster RCNN
+
     python eval_coco_map.py \
         --dataset=coco2017 \
         --pretrained_model=${path_to_pretrain_model} \
+        --MASK_ON=False
+
+- Mask RCNN
+
+    python eval_coco_map.py \
+        --dataset=coco2017 \
+        --pretrained_model=${path_to_pretrain_model} \
+        --MASK_ON=True
 
 - Set ```export CUDA_VISIBLE_DEVICES=0``` to specifiy one GPU to eval.
+- Set ```MASK_ON``` to choose Faster RCNN or Mask RCNN model.
 
 Evalutaion result is shown as below:
 
+Faster RCNN:
+
 | Model              | RoI function    | Batch size     | Max iteration    | mAP  |
 | :--------------- | :--------: | :------------:    | :------------------:    |------: |
 | [Fluid RoIPool minibatch padding](http://paddlemodels.bj.bcebos.com/faster_rcnn/model_pool_minibatch_padding.tar.gz) | RoIPool | 8   |    180000        | 0.316 |
@@ -121,6 +147,14 @@ Evalutaion result is shown as below:
 * Fluid RoIAlign no padding: Images without padding.
 * Fluid RoIAlign no padding 2x: Images without padding, train for 360000 iterations, learning rate is decayed at 240000, 320000.
 
+Mask RCNN:
+
+| Model              | Batch size     | Max iteration | box mAP | mask mAP |
+| :--------------- | :--------: | :------------:    | :--------:    |------: |
+| [Fluid mask no padding](https://paddlemodels.bj.bcebos.com/faster_rcnn/Fluid_mask_no_padding.tar.gz) | 8 | 180000 | 0.359 | 0.314 |
+
+* Fluid mask no padding: Use RoIAlign. Images without padding.
+
 ## Inference and Visualization
 
 Inference is used to get prediction score or image features based on trained models. `infer.py`  is the main executor for inference, one can start infer step by:
@@ -135,8 +169,12 @@ Inference is used to get prediction score or image features based on trained mod
 Visualization of infer result is shown as below:
 <p align="center">
 <img src="image/000000000139.jpg" height=300 width=400 hspace='10'/>
-<img src="image/000000127517.jpg" height=300 width=400 hspace='10'/>
-<img src="image/000000203864.jpg" height=300 width=400 hspace='10'/>
-<img src="image/000000515077.jpg" height=300 width=400 hspace='10'/> <br />
+<img src="image/000000127517.jpg" height=300 width=400 hspace='10'/> <br />
 Faster RCNN Visualization Examples
 </p>
+
+<p align="center">
+<img src="image/000000000139_mask.jpg" height=300 width=400 hspace='10'/>
+<img src="image/000000127517_mask.jpg" height=300 width=400 hspace='10'/> <br />
+Mask RCNN Visualization Examples
+</p>

fluid/PaddleCV/faster_rcnn/README_cn.md → fluid/PaddleCV/rcnn/README_cn.md

-# Faster RCNN 目标检测
+# RCNN 系列目标检测
 
 ---
 ## 内容
@@ -9,25 +9,27 @@
 - [模型训练](#模型训练)
 - [模型评估](#模型评估)
 - [模型推断及可视化](#模型推断及可视化)
-- [附录](#附录)
 
 ## 安装
 
 在当前目录下运行样例代码需要PadddlePaddle Fluid的v.1.0.0或以上的版本。如果你的运行环境中的PaddlePaddle低于此版本，请根据[安装文档](http://www.paddlepaddle.org/documentation/docs/zh/0.15.0/beginners_guide/install/install_doc.html#paddlepaddle)中的说明来更新PaddlePaddle。
 
 ## 简介
+区域卷积神经网络（RCNN）系列模型为两阶段目标检测器。通过对图像生成候选区域，提取特征，判别特征类别并修正候选框位置。
+RCNN系列目前包含两个代表模型：Faster RCNN，Mask RCNN
 
-[Faster Rcnn](https://arxiv.org/abs/1506.01497) 是典型的两阶段目标检测器。如下图所示，整体网络可以分为4个主要内容：
-<p align="center">
-<img src="image/Faster_RCNN.jpg" height=400 width=400 hspace='10'/> <br />
-Faster RCNN 目标检测模型
-</p>
+[Faster RCNN](https://arxiv.org/abs/1506.01497) 整体网络可以分为4个主要内容：
 
 1. 基础卷积层。作为一种卷积神经网络目标检测方法，Faster RCNN首先使用一组基础的卷积网络提取图像的特征图。特征图被后续RPN层和全连接层共享。本示例采用[ResNet-50](https://arxiv.org/abs/1512.03385)作为基础卷积层。
 2. 区域生成网络(RPN)。RPN网络用于生成候选区域(proposals)。该层通过一组固定的尺寸和比例得到一组锚点(anchors), 通过softmax判断锚点属于前景或者背景，再利用区域回归修正锚点从而获得精确的候选区域。
 3. RoI Align。该层收集输入的特征图和候选区域，将候选区域映射到特征图中并池化为统一大小的区域特征图，送入全连接层判定目标类别, 该层可选用RoIPool和RoIAlign两种方式，在config.py中设置roi\_func。
 4. 检测层。利用区域特征图计算候选区域的类别，同时再次通过区域回归获得检测框最终的精确位置。
 
+[Mask RCNN](https://arxiv.org/abs/1703.06870) 扩展自Faster RCNN，是经典的实例分割模型。
+
+Mask RCNN同样为两阶段框架，第一阶段扫描图像生成候选框；第二阶段根据候选框得到分类结果，边界框，同时在原有Faster RCNN模型基础上添加分割分支，得到掩码结果，实现了掩码和类别预测关系的解藕。
+
+
 ## 数据准备
 
 在[MS-COCO数据集](http://cocodataset.org/#download)上进行训练，通过如下方式下载数据集。
@@ -61,12 +63,24 @@ Faster RCNN 目标检测模型
 
 数据准备完毕后，可以通过如下的方式启动训练：
 
+- Faster RCNN
+
     python train.py \
        --model_save_dir=output/ \
-       --pretrained_model=${path_to_pretrain_model}
-       --data_dir=${path_to_data}
+       --pretrained_model=${path_to_pretrain_model} \
+       --data_dir=${path_to_data} \
+       --MASK_ON=False
+
+- Mask RCNN
+
+    python train.py \
+       --model_save_dir=output/ \
+       --pretrained_model=${path_to_pretrain_model} \
+       --data_dir=${path_to_data} \
+       --MASK_ON=True
 
 - 通过设置export CUDA\_VISIBLE\_DEVICES=0,1,2,3,4,5,6,7指定8卡GPU训练。
+- 通过设置```MASK_ON```选择Faster RCNN和Mask RCNN模型。
 - 可选参数见：
 
     python train.py --help
@@ -83,11 +97,10 @@ Faster RCNN 目标检测模型
 
 **训练策略：**
 
-*  采用momentum优化算法训练Faster RCNN，momentum=0.9。
+*  采用momentum优化算法训练，momentum=0.9。
 *  权重衰减系数为0.0001，前500轮学习率从0.00333线性增加至0.01。在120000，160000轮时使用0.1,0.01乘子进行学习率衰减，最大训练180000轮。同时我们也提供了2x模型，该模型采用更多的迭代轮数进行训练，训练360000轮，学习率在240000，320000轮衰减，其他参数不变，训练最大轮数和学习率策略可以在config.py中对max_iter和lr_steps进行设置。
 *  非基础卷积层卷积bias学习率为整体学习率2倍。
 *  基础卷积层中，affine_layers参数不更新，res2层参数不更新。
-*  使用Nvidia Tesla V100 8卡并行，总共训练时长大约40小时。
 
 ## 模型评估
 
@@ -95,14 +108,27 @@ Faster RCNN 目标检测模型
 
 `eval_coco_map.py`是评估模块的主要执行程序，调用示例如下：
 
+- Faster RCNN
+
     python eval_coco_map.py \
         --dataset=coco2017 \
         --pretrained_model=${path_to_pretrain_model} \
+        --MASK_ON=False
+
+- Mask RCNN
+
+    python eval_coco_map.py \
+        --dataset=coco2017 \
+        --pretrained_model=${path_to_pretrain_model} \
+        --MASK_ON=True
 
 - 通过设置export CUDA\_VISIBLE\_DEVICES=0指定单卡GPU评估。
+- 通过设置```MASK_ON```选择Faster RCNN和Mask RCNN模型。
 
 下表为模型评估结果：
 
+Faster RCNN
+
 | 模型                   |   RoI处理方式  | 批量大小   | 迭代次数   | mAP  |
 | :--------------- | :--------: | :------------:    | :------------------:    |------: |
 | [Fluid RoIPool minibatch padding](http://paddlemodels.bj.bcebos.com/faster_rcnn/model_pool_minibatch_padding.tar.gz) | RoIPool | 8   |    180000        | 0.316 |
@@ -117,6 +143,14 @@ Faster RCNN 目标检测模型
 * Fluid RoIAlign no padding: 使用RoIAlign，不对图像做填充处理。
 * Fluid RoIAlign no padding 2x: 使用RoIAlign，不对图像做填充处理。训练360000轮，学习率在240000，320000轮衰减。
 
+Mask RCNN:
+
+| 模型                   | 批量大小   | 迭代次数   | box mAP  | mask mAP |
+| :--------------- | :--------: | :------------:    | :--------:    |------: |
+| [Fluid mask no padding](https://paddlemodels.bj.bcebos.com/faster_rcnn/Fluid_mask_no_padding.tar.gz) | 8 | 180000 | 0.359 | 0.314 |  
+
+* Fluid mask no padding: 使用RoIAlign，不对图像做填充处理
+
 ## 模型推断及可视化
 
 模型推断可以获取图像中的物体及其对应的类别，`infer.py`是主要执行程序，调用示例如下：
@@ -131,8 +165,12 @@ Faster RCNN 目标检测模型
 下图为模型可视化预测结果：
 <p align="center">
 <img src="image/000000000139.jpg" height=300 width=400 hspace='10'/>
-<img src="image/000000127517.jpg" height=300 width=400 hspace='10'/>
-<img src="image/000000203864.jpg" height=300 width=400 hspace='10'/>
-<img src="image/000000515077.jpg" height=300 width=400 hspace='10'/> <br />
+<img src="image/000000127517.jpg" height=300 width=400 hspace='10'/> <br />
 Faster RCNN 预测可视化
 </p>
+
+<p align="center">
+<img src="image/000000000139_mask.jpg" height=300 width=400 hspace='10'/>
+<img src="image/000000127517_mask.jpg" height=300 width=400 hspace='10'/> <br />
+Mask RCNN 预测可视化
+</p>

fluid/PaddleCV/faster_rcnn/_ce.py → fluid/PaddleCV/rcnn/_ce.py

@@ -6,18 +6,19 @@ sys.path.append(os.environ['ceroot'])
 from kpi import CostKpi
 from kpi import DurationKpi
 
-
-each_pass_duration_card1_kpi = DurationKpi('each_pass_duration_card1', 0.08, 0, actived=True)
+each_pass_duration_card1_kpi = DurationKpi(
+    'each_pass_duration_card1', 0.08, 0, actived=True)
 train_loss_card1_kpi = CostKpi('train_loss_card1', 0.08, 0)
-each_pass_duration_card4_kpi = DurationKpi('each_pass_duration_card4', 0.08, 0, actived=True)
+each_pass_duration_card4_kpi = DurationKpi(
+    'each_pass_duration_card4', 0.08, 0, actived=True)
 train_loss_card4_kpi = CostKpi('train_loss_card4', 0.08, 0)
 
 tracking_kpis = [
-        each_pass_duration_card1_kpi,
-        train_loss_card1_kpi,
-        each_pass_duration_card4_kpi,
-        train_loss_card4_kpi,
-        ]
+    each_pass_duration_card1_kpi,
+    train_loss_card1_kpi,
+    each_pass_duration_card4_kpi,
+    train_loss_card4_kpi,
+]
 
 
 def parse_log(log):

fluid/PaddleCV/faster_rcnn/box_utils.py → fluid/PaddleCV/rcnn/box_utils.py

@@ -69,6 +69,7 @@ def clip_xyxy_to_image(x1, y1, x2, y2, height, width):
     y2 = np.minimum(height - 1., np.maximum(0., y2))
     return x1, y1, x2, y2
 
+
 def nms(dets, thresh):
     """Apply classic DPM-style greedy NMS."""
     if dets.shape[0] == 0:
@@ -123,3 +124,21 @@ def nms(dets, thresh):
 
     return np.where(suppressed == 0)[0]
 
+
+def expand_boxes(boxes, scale):
+    """Expand an array of boxes by a given scale."""
+    w_half = (boxes[:, 2] - boxes[:, 0]) * .5
+    h_half = (boxes[:, 3] - boxes[:, 1]) * .5
+    x_c = (boxes[:, 2] + boxes[:, 0]) * .5
+    y_c = (boxes[:, 3] + boxes[:, 1]) * .5
+
+    w_half *= scale
+    h_half *= scale
+
+    boxes_exp = np.zeros(boxes.shape)
+    boxes_exp[:, 0] = x_c - w_half
+    boxes_exp[:, 2] = x_c + w_half
+    boxes_exp[:, 1] = y_c - h_half
+    boxes_exp[:, 3] = y_c + h_half
+
+    return boxes_exp

fluid/PaddleCV/rcnn/colormap.py

+#  Copyright (c) 2018 PaddlePaddle Authors. All Rights Reserve.
+#
+#Licensed under the Apache License, Version 2.0 (the "License");
+#you may not use this file except in compliance with the License.
+#You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+#Unless required by applicable law or agreed to in writing, software
+#distributed under the License is distributed on an "AS IS" BASIS,
+#WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#See the License for the specific language governing permissions and
+#limitations under the License.
+#
+# Based on:
+# --------------------------------------------------------
+# Detectron
+# Copyright (c) 2017-present, Facebook, Inc.
+# Licensed under the Apache License, Version 2.0;
+# Written by Ross Girshick
+# --------------------------------------------------------
+
+from __future__ import absolute_import
+from __future__ import division
+from __future__ import print_function
+from __future__ import unicode_literals
+
+import numpy as np
+
+
+def colormap(rgb=False):
+    color_list = np.array([
+        0.000, 0.447, 0.741, 0.850, 0.325, 0.098, 0.929, 0.694, 0.125, 0.494,
+        0.184, 0.556, 0.466, 0.674, 0.188, 0.301, 0.745, 0.933, 0.635, 0.078,
+        0.184, 0.300, 0.300, 0.300, 0.600, 0.600, 0.600, 1.000, 0.000, 0.000,
+        1.000, 0.500, 0.000, 0.749, 0.749, 0.000, 0.000, 1.000, 0.000, 0.000,
+        0.000, 1.000, 0.667, 0.000, 1.000, 0.333, 0.333, 0.000, 0.333, 0.667,
+        0.000, 0.333, 1.000, 0.000, 0.667, 0.333, 0.000, 0.667, 0.667, 0.000,
+        0.667, 1.000, 0.000, 1.000, 0.333, 0.000, 1.000, 0.667, 0.000, 1.000,
+        1.000, 0.000, 0.000, 0.333, 0.500, 0.000, 0.667, 0.500, 0.000, 1.000,
+        0.500, 0.333, 0.000, 0.500, 0.333, 0.333, 0.500, 0.333, 0.667, 0.500,
+        0.333, 1.000, 0.500, 0.667, 0.000, 0.500, 0.667, 0.333, 0.500, 0.667,
+        0.667, 0.500, 0.667, 1.000, 0.500, 1.000, 0.000, 0.500, 1.000, 0.333,
+        0.500, 1.000, 0.667, 0.500, 1.000, 1.000, 0.500, 0.000, 0.333, 1.000,
+        0.000, 0.667, 1.000, 0.000, 1.000, 1.000, 0.333, 0.000, 1.000, 0.333,
+        0.333, 1.000, 0.333, 0.667, 1.000, 0.333, 1.000, 1.000, 0.667, 0.000,
+        1.000, 0.667, 0.333, 1.000, 0.667, 0.667, 1.000, 0.667, 1.000, 1.000,
+        1.000, 0.000, 1.000, 1.000, 0.333, 1.000, 1.000, 0.667, 1.000, 0.167,
+        0.000, 0.000, 0.333, 0.000, 0.000, 0.500, 0.000, 0.000, 0.667, 0.000,
+        0.000, 0.833, 0.000, 0.000, 1.000, 0.000, 0.000, 0.000, 0.167, 0.000,
+        0.000, 0.333, 0.000, 0.000, 0.500, 0.000, 0.000, 0.667, 0.000, 0.000,
+        0.833, 0.000, 0.000, 1.000, 0.000, 0.000, 0.000, 0.167, 0.000, 0.000,
+        0.333, 0.000, 0.000, 0.500, 0.000, 0.000, 0.667, 0.000, 0.000, 0.833,
+        0.000, 0.000, 1.000, 0.000, 0.000, 0.000, 0.143, 0.143, 0.143, 0.286,
+        0.286, 0.286, 0.429, 0.429, 0.429, 0.571, 0.571, 0.571, 0.714, 0.714,
+        0.714, 0.857, 0.857, 0.857, 1.000, 1.000, 1.000
+    ]).astype(np.float32)
+    color_list = color_list.reshape((-1, 3)) * 255
+    if not rgb:
+        color_list = color_list[:, ::-1]
+    return color_list

fluid/PaddleCV/faster_rcnn/config.py → fluid/PaddleCV/rcnn/config.py

@@ -90,6 +90,9 @@ _C.TRAIN.freeze_at = 2
 # min area of ground truth box
 _C.TRAIN.gt_min_area = -1
 
+# Use horizontally-flipped images during training?
+_C.TRAIN.use_flipped = True
+
 #
 # Inference options
 #
@@ -120,7 +123,7 @@ _C.TEST.rpn_post_nms_top_n = 1000
 _C.TEST.rpn_min_size = 0.0
 
 # max number of detections
-_C.TEST.detectiions_per_im = 100
+_C.TEST.detections_per_im = 100
 
 # NMS threshold used on RPN proposals
 _C.TEST.rpn_nms_thresh = 0.7
@@ -129,6 +132,9 @@ _C.TEST.rpn_nms_thresh = 0.7
 # Model options
 #
 
+# Whether use mask rcnn head
+_C.MASK_ON = True
+
 # weight for bbox regression targets
 _C.bbox_reg_weights = [0.1, 0.1, 0.2, 0.2]
 
@@ -156,6 +162,15 @@ _C.roi_resolution = 14
 # spatial scale 
 _C.spatial_scale = 1. / 16.
 
+# resolution to represent mask labels
+_C.resolution = 14
+
+# Number of channels in the mask head
+_C.dim_reduced = 256
+
+# Threshold for converting soft masks to hard masks
+_C.mrcnn_thresh_binarize = 0.5
+
 #
 # SOLVER options
 #
@@ -204,12 +219,6 @@ _C.pixel_means = [102.9801, 115.9465, 122.7717]
 # clip box to prevent overflowing
 _C.bbox_clip = np.log(1000. / 16.)
 
-# dataset path
-_C.train_file_list = 'annotations/instances_train2017.json'
-_C.train_data_dir = 'train2017'
-_C.val_file_list = 'annotations/instances_val2017.json'
-_C.val_data_dir = 'val2017'
-
 
 def merge_cfg_from_args(args, mode):
     """Merge config keys, values in args into the global config."""

fluid/PaddleCV/faster_rcnn/eval_coco_map.py → fluid/PaddleCV/rcnn/eval_coco_map.py

@@ -18,8 +18,7 @@ from __future__ import print_function
 import os
 import time
 import numpy as np
-from eval_helper import get_nmsed_box
-from eval_helper import get_dt_res
+from eval_helper import *
 import paddle
 import paddle.fluid as fluid
 import reader
@@ -30,21 +29,21 @@ import json
 from pycocotools.coco import COCO
 from pycocotools.cocoeval import COCOeval, Params
 from config import cfg
+from roidbs import DatasetPath
 
 
 def eval():
-    if '2014' in cfg.dataset:
-        test_list = 'annotations/instances_val2014.json'
-    elif '2017' in cfg.dataset:
-        test_list = 'annotations/instances_val2017.json'
+
+    data_path = DatasetPath('val')
+    test_list = data_path.get_file_list()
 
     image_shape = [3, cfg.TEST.max_size, cfg.TEST.max_size]
     class_nums = cfg.class_num
     devices = os.getenv("CUDA_VISIBLE_DEVICES") or ""
     devices_num = len(devices.split(","))
     total_batch_size = devices_num * cfg.TRAIN.im_per_batch
-    cocoGt = COCO(os.path.join(cfg.data_dir, test_list))
-    numId_to_catId_map = {i + 1: v for i, v in enumerate(cocoGt.getCatIds())}
+    cocoGt = COCO(test_list)
+    num_id_to_cat_id_map = {i + 1: v for i, v in enumerate(cocoGt.getCatIds())}
     category_ids = cocoGt.getCatIds()
     label_list = {
         item['id']: item['name']
@@ -52,51 +51,82 @@ def eval():
     }
     label_list[0] = ['background']
 
-    model = model_builder.FasterRCNN(
+    model = model_builder.RCNN(
         add_conv_body_func=resnet.add_ResNet50_conv4_body,
         add_roi_box_head_func=resnet.add_ResNet_roi_conv5_head,
         use_pyreader=False,
         is_train=False)
     model.build_model(image_shape)
-    rpn_rois, confs, locs = model.eval_out()
+    pred_boxes = model.eval_bbox_out()
+    if cfg.MASK_ON:
+        masks = model.eval_mask_out()
     place = fluid.CUDAPlace(0) if cfg.use_gpu else fluid.CPUPlace()
     exe = fluid.Executor(place)
+    exe.run(fluid.default_startup_program())
     # yapf: disable
     if cfg.pretrained_model:
         def if_exist(var):
             return os.path.exists(os.path.join(cfg.pretrained_model, var.name))
         fluid.io.load_vars(exe, cfg.pretrained_model, predicate=if_exist)
+
     # yapf: enable
     test_reader = reader.test(total_batch_size)
     feeder = fluid.DataFeeder(place=place, feed_list=model.feeds())
 
     dts_res = []
-    fetch_list = [rpn_rois, confs, locs]
+    segms_res = []
+    if cfg.MASK_ON:
+        fetch_list = [pred_boxes, masks]
+    else:
+        fetch_list = [pred_boxes]
+    eval_start = time.time()
     for batch_id, batch_data in enumerate(test_reader()):
         start = time.time()
         im_info = []
         for data in batch_data:
             im_info.append(data[1])
-        rpn_rois_v, confs_v, locs_v = exe.run(
-            fetch_list=[v.name for v in fetch_list],
-            feed=feeder.feed(batch_data),
-            return_numpy=False)
-        new_lod, nmsed_out = get_nmsed_box(rpn_rois_v, confs_v, locs_v,
-                                           class_nums, im_info,
-                                           numId_to_catId_map)
+        result = exe.run(fetch_list=[v.name for v in fetch_list],
+                         feed=feeder.feed(batch_data),
+                         return_numpy=False)
+
+        pred_boxes_v = result[0]
+        if cfg.MASK_ON:
+            masks_v = result[1]
 
-        dts_res += get_dt_res(total_batch_size, new_lod, nmsed_out, batch_data)
+        new_lod = pred_boxes_v.lod()
+        nmsed_out = pred_boxes_v
+
+        dts_res += get_dt_res(total_batch_size, new_lod[0], nmsed_out,
+                              batch_data, num_id_to_cat_id_map)
+
+        if cfg.MASK_ON and np.array(masks_v).shape != (1, 1):
+            segms_out = segm_results(nmsed_out, masks_v, im_info)
+            segms_res += get_segms_res(total_batch_size, new_lod[0], segms_out,
+                                       batch_data, num_id_to_cat_id_map)
         end = time.time()
         print('batch id: {}, time: {}'.format(batch_id, end - start))
-    with open("detection_result.json", 'w') as outfile:
+    eval_end = time.time()
+    total_time = eval_end - eval_start
+    print('average time of eval is: {}'.format(total_time / (batch_id + 1)))
+    with open("detection_bbox_result.json", 'w') as outfile:
         json.dump(dts_res, outfile)
-    print("start evaluate using coco api")
-    cocoDt = cocoGt.loadRes("detection_result.json")
+    print("start evaluate bbox using coco api")
+    cocoDt = cocoGt.loadRes("detection_bbox_result.json")
     cocoEval = COCOeval(cocoGt, cocoDt, 'bbox')
     cocoEval.evaluate()
     cocoEval.accumulate()
     cocoEval.summarize()
 
+    if cfg.MASK_ON:
+        with open("detection_segms_result.json", 'w') as outfile:
+            json.dump(segms_res, outfile)
+        print("start evaluate mask using coco api")
+        cocoDt = cocoGt.loadRes("detection_segms_result.json")
+        cocoEval = COCOeval(cocoGt, cocoDt, 'segm')
+        cocoEval.evaluate()
+        cocoEval.accumulate()
+        cocoEval.summarize()
+
 
 if __name__ == '__main__':
     args = parse_args()

fluid/PaddleCV/faster_rcnn/eval_helper.py → fluid/PaddleCV/rcnn/eval_helper.py

@@ -21,6 +21,10 @@ from PIL import Image
 from PIL import ImageDraw
 from PIL import ImageFont
 from config import cfg
+import pycocotools.mask as mask_util
+import six
+from colormap import colormap
+import cv2
 
 
 def box_decoder(deltas, boxes, weights):
@@ -80,8 +84,7 @@ def clip_tiled_boxes(boxes, im_shape):
     return boxes
 
 
-def get_nmsed_box(rpn_rois, confs, locs, class_nums, im_info,
-                  numId_to_catId_map):
+def get_nmsed_box(rpn_rois, confs, locs, class_nums, im_info):
     lod = rpn_rois.lod()[0]
     rpn_rois_v = np.array(rpn_rois)
     variance_v = np.array(cfg.bbox_reg_weights)
@@ -106,38 +109,41 @@ def get_nmsed_box(rpn_rois, confs, locs, class_nums, im_info,
             inds = np.where(scores_n[:, j] > cfg.TEST.score_thresh)[0]
             scores_j = scores_n[inds, j]
             rois_j = rois_n[inds, j * 4:(j + 1) * 4]
-            dets_j = np.hstack((rois_j, scores_j[:, np.newaxis])).astype(
+            dets_j = np.hstack((scores_j[:, np.newaxis], rois_j)).astype(
                 np.float32, copy=False)
             keep = box_utils.nms(dets_j, cfg.TEST.nms_thresh)
             nms_dets = dets_j[keep, :]
             #add labels
-            cat_id = numId_to_catId_map[j]
-            label = np.array([cat_id for _ in range(len(keep))])
+            label = np.array([j for _ in range(len(keep))])
             nms_dets = np.hstack((nms_dets, label[:, np.newaxis])).astype(
                 np.float32, copy=False)
             cls_boxes[j] = nms_dets
     # Limit to max_per_image detections **over all classes**
         image_scores = np.hstack(
-            [cls_boxes[j][:, -2] for j in range(1, class_nums)])
-        if len(image_scores) > cfg.TEST.detectiions_per_im:
-            image_thresh = np.sort(image_scores)[-cfg.TEST.detectiions_per_im]
+            [cls_boxes[j][:, 1] for j in range(1, class_nums)])
+        if len(image_scores) > cfg.TEST.detections_per_im:
+            image_thresh = np.sort(image_scores)[-cfg.TEST.detections_per_im]
             for j in range(1, class_nums):
-                keep = np.where(cls_boxes[j][:, -2] >= image_thresh)[0]
+                keep = np.where(cls_boxes[j][:, 1] >= image_thresh)[0]
                 cls_boxes[j] = cls_boxes[j][keep, :]
 
         im_results_n = np.vstack([cls_boxes[j] for j in range(1, class_nums)])
         im_results[i] = im_results_n
         new_lod.append(len(im_results_n) + new_lod[-1])
-        boxes = im_results_n[:, :-2]
-        scores = im_results_n[:, -2]
-        labels = im_results_n[:, -1]
+        boxes = im_results_n[:, 2:]
+        scores = im_results_n[:, 1]
+        labels = im_results_n[:, 0]
     im_results = np.vstack([im_results[k] for k in range(len(lod) - 1)])
     return new_lod, im_results
 
 
-def get_dt_res(batch_size, lod, nmsed_out, data):
+def get_dt_res(batch_size, lod, nmsed_out, data, num_id_to_cat_id_map):
     dts_res = []
     nmsed_out_v = np.array(nmsed_out)
+    if nmsed_out_v.shape == (
+            1,
+            1, ):
+        return dts_res
     assert (len(lod) == batch_size + 1), \
       "Error Lod Tensor offset dimension. Lod({}) vs. batch_size({})"\
                     .format(len(lod), batch_size)
@@ -150,7 +156,8 @@ def get_dt_res(batch_size, lod, nmsed_out, data):
         for j in range(dt_num_this_img):
             dt = nmsed_out_v[k]
             k = k + 1
-            xmin, ymin, xmax, ymax, score, category_id = dt.tolist()
+            num_id, score, xmin, ymin, xmax, ymax = dt.tolist()
+            category_id = num_id_to_cat_id_map[num_id]
             w = xmax - xmin + 1
             h = ymax - ymin + 1
             bbox = [xmin, ymin, w, h]
@@ -164,24 +171,131 @@ def get_dt_res(batch_size, lod, nmsed_out, data):
     return dts_res
 
 
-def draw_bounding_box_on_image(image_path, nms_out, draw_threshold, label_list):
-    image = Image.open(image_path)
+def get_segms_res(batch_size, lod, segms_out, data, num_id_to_cat_id_map):
+    segms_res = []
+    segms_out_v = np.array(segms_out)
+    k = 0
+    for i in range(batch_size):
+        dt_num_this_img = lod[i + 1] - lod[i]
+        image_id = int(data[i][-1])
+        for j in range(dt_num_this_img):
+            dt = segms_out_v[k]
+            k = k + 1
+            segm, num_id, score = dt.tolist()
+            cat_id = num_id_to_cat_id_map[num_id]
+            if six.PY3:
+                if 'counts' in segm:
+                    segm['counts'] = segm['counts'].decode("utf8")
+            segm_res = {
+                'image_id': image_id,
+                'category_id': cat_id,
+                'segmentation': segm,
+                'score': score
+            }
+            segms_res.append(segm_res)
+    return segms_res
+
+
+def draw_bounding_box_on_image(image_path,
+                               nms_out,
+                               draw_threshold,
+                               label_list,
+                               num_id_to_cat_id_map,
+                               image=None):
+    if image is None:
+        image = Image.open(image_path)
     draw = ImageDraw.Draw(image)
     im_width, im_height = image.size
 
-    for dt in nms_out:
-        xmin, ymin, xmax, ymax, score, category_id = dt.tolist()
+    for dt in np.array(nms_out):
+        num_id, score, xmin, ymin, xmax, ymax = dt.tolist()
+        category_id = num_id_to_cat_id_map[num_id]
         if score < draw_threshold:
             continue
-        bbox = dt[:4]
-        xmin, ymin, xmax, ymax = bbox
         draw.line(
             [(xmin, ymin), (xmin, ymax), (xmax, ymax), (xmax, ymin),
              (xmin, ymin)],
-            width=4,
+            width=2,
             fill='red')
         if image.mode == 'RGB':
             draw.text((xmin, ymin), label_list[int(category_id)], (255, 255, 0))
     image_name = image_path.split('/')[-1]
     print("image with bbox drawed saved as {}".format(image_name))
     image.save(image_name)
+
+
+def draw_mask_on_image(image_path, segms_out, draw_threshold, alpha=0.7):
+    image = Image.open(image_path)
+    draw = ImageDraw.Draw(image)
+    im_width, im_height = image.size
+    mask_color_id = 0
+    w_ratio = .4
+    image = np.array(image).astype('float32')
+    for dt in np.array(segms_out):
+        segm, num_id, score = dt.tolist()
+        if score < draw_threshold:
+            continue
+        mask = mask_util.decode(segm) * 255
+        color_list = colormap(rgb=True)
+        color_mask = color_list[mask_color_id % len(color_list), 0:3]
+        mask_color_id += 1
+        for c in range(3):
+            color_mask[c] = color_mask[c] * (1 - w_ratio) + w_ratio * 255
+        idx = np.nonzero(mask)
+        image[idx[0], idx[1], :] *= 1.0 - alpha
+        image[idx[0], idx[1], :] += alpha * color_mask
+    image = Image.fromarray(image.astype('uint8'))
+    return image
+
+
+def segm_results(im_results, masks, im_info):
+    im_results = np.array(im_results)
+    class_num = cfg.class_num
+    M = cfg.resolution
+    scale = (M + 2.0) / M
+    lod = masks.lod()[0]
+    masks_v = np.array(masks)
+    boxes = im_results[:, 2:]
+    labels = im_results[:, 0]
+    segms_results = [[] for _ in range(len(lod) - 1)]
+    sum = 0
+    for i in range(len(lod) - 1):
+        im_results_n = im_results[lod[i]:lod[i + 1]]
+        cls_segms = []
+        masks_n = masks_v[lod[i]:lod[i + 1]]
+        boxes_n = boxes[lod[i]:lod[i + 1]]
+        labels_n = labels[lod[i]:lod[i + 1]]
+        im_h = int(round(im_info[i][0] / im_info[i][2]))
+        im_w = int(round(im_info[i][1] / im_info[i][2]))
+        boxes_n = box_utils.expand_boxes(boxes_n, scale)
+        boxes_n = boxes_n.astype(np.int32)
+        padded_mask = np.zeros((M + 2, M + 2), dtype=np.float32)
+        for j in range(len(im_results_n)):
+            class_id = int(labels_n[j])
+            padded_mask[1:-1, 1:-1] = masks_n[j, class_id, :, :]
+
+            ref_box = boxes_n[j, :]
+            w = ref_box[2] - ref_box[0] + 1
+            h = ref_box[3] - ref_box[1] + 1
+            w = np.maximum(w, 1)
+            h = np.maximum(h, 1)
+
+            mask = cv2.resize(padded_mask, (w, h))
+            mask = np.array(mask > cfg.mrcnn_thresh_binarize, dtype=np.uint8)
+            im_mask = np.zeros((im_h, im_w), dtype=np.uint8)
+
+            x_0 = max(ref_box[0], 0)
+            x_1 = min(ref_box[2] + 1, im_w)
+            y_0 = max(ref_box[1], 0)
+            y_1 = min(ref_box[3] + 1, im_h)
+            im_mask[y_0:y_1, x_0:x_1] = mask[(y_0 - ref_box[1]):(y_1 - ref_box[
+                1]), (x_0 - ref_box[0]):(x_1 - ref_box[0])]
+            sum += im_mask.sum()
+            rle = mask_util.encode(
+                np.array(
+                    im_mask[:, :, np.newaxis], order='F'))[0]
+            cls_segms.append(rle)
+        segms_results[i] = np.array(cls_segms)[:, np.newaxis]
+    segms_results = np.vstack([segms_results[k] for k in range(len(lod) - 1)])
+    im_results = np.hstack([segms_results, im_results])
+    return im_results[:, :3]

fluid/PaddleCV/faster_rcnn/infer.py → fluid/PaddleCV/rcnn/infer.py

 import os
 import time
 import numpy as np
-from eval_helper import get_nmsed_box
-from eval_helper import get_dt_res
-from eval_helper import draw_bounding_box_on_image
+from eval_helper import *
 import paddle
 import paddle.fluid as fluid
 import reader
@@ -14,17 +12,16 @@ import json
 from pycocotools.coco import COCO
 from pycocotools.cocoeval import COCOeval, Params
 from config import cfg
+from roidbs import DatasetPath
 
 
 def infer():
 
-    if '2014' in cfg.dataset:
-        test_list = 'annotations/instances_val2014.json'
-    elif '2017' in cfg.dataset:
-        test_list = 'annotations/instances_val2017.json'
+    data_path = DatasetPath('val')
+    test_list = data_path.get_file_list()
 
-    cocoGt = COCO(os.path.join(cfg.data_dir, test_list))
-    numId_to_catId_map = {i + 1: v for i, v in enumerate(cocoGt.getCatIds())}
+    cocoGt = COCO(test_list)
+    num_id_to_cat_id_map = {i + 1: v for i, v in enumerate(cocoGt.getCatIds())}
     category_ids = cocoGt.getCatIds()
     label_list = {
         item['id']: item['name']
@@ -34,13 +31,15 @@ def infer():
     image_shape = [3, cfg.TEST.max_size, cfg.TEST.max_size]
     class_nums = cfg.class_num
 
-    model = model_builder.FasterRCNN(
+    model = model_builder.RCNN(
         add_conv_body_func=resnet.add_ResNet50_conv4_body,
         add_roi_box_head_func=resnet.add_ResNet_roi_conv5_head,
         use_pyreader=False,
         is_train=False)
     model.build_model(image_shape)
-    rpn_rois, confs, locs = model.eval_out()
+    pred_boxes = model.eval_bbox_out()
+    if cfg.MASK_ON:
+        masks = model.eval_mask_out()
     place = fluid.CUDAPlace(0) if cfg.use_gpu else fluid.CPUPlace()
     exe = fluid.Executor(place)
     # yapf: disable
@@ -53,17 +52,29 @@ def infer():
     feeder = fluid.DataFeeder(place=place, feed_list=model.feeds())
 
     dts_res = []
-    fetch_list = [rpn_rois, confs, locs]
+    segms_res = []
+    if cfg.MASK_ON:
+        fetch_list = [pred_boxes, masks]
+    else:
+        fetch_list = [pred_boxes]
     data = next(infer_reader())
     im_info = [data[0][1]]
-    rpn_rois_v, confs_v, locs_v = exe.run(
-        fetch_list=[v.name for v in fetch_list],
-        feed=feeder.feed(data),
-        return_numpy=False)
-    new_lod, nmsed_out = get_nmsed_box(rpn_rois_v, confs_v, locs_v, class_nums,
-                                       im_info, numId_to_catId_map)
+    result = exe.run(fetch_list=[v.name for v in fetch_list],
+                     feed=feeder.feed(data),
+                     return_numpy=False)
+    pred_boxes_v = result[0]
+    if cfg.MASK_ON:
+        masks_v = result[1]
+    new_lod = pred_boxes_v.lod()
+    nmsed_out = pred_boxes_v
     path = os.path.join(cfg.image_path, cfg.image_name)
-    draw_bounding_box_on_image(path, nmsed_out, cfg.draw_threshold, label_list)
+    image = None
+    if cfg.MASK_ON:
+        segms_out = segm_results(nmsed_out, masks_v, im_info)
+        image = draw_mask_on_image(path, segms_out, cfg.draw_threshold)
+
+    draw_bounding_box_on_image(path, nmsed_out, cfg.draw_threshold, label_list,
+                               num_id_to_cat_id_map, image)
 
 
 if __name__ == '__main__':

fluid/PaddleCV/faster_rcnn/models/resnet.py → fluid/PaddleCV/rcnn/models/resnet.py

@@ -160,8 +160,22 @@ def add_ResNet50_conv4_body(body_input):
     return res4
 
 
-def add_ResNet_roi_conv5_head(head_input):
-    res5 = layer_warp(bottleneck, head_input, 512, 3, 2, name="res5")
-    res5_pool = fluid.layers.pool2d(
-        res5, pool_type='avg', pool_size=7, name='res5_pool')
-    return res5_pool
+def add_ResNet_roi_conv5_head(head_input, rois):
+    if cfg.roi_func == 'RoIPool':
+        pool = fluid.layers.roi_pool(
+            input=head_input,
+            rois=rois,
+            pooled_height=cfg.roi_resolution,
+            pooled_width=cfg.roi_resolution,
+            spatial_scale=cfg.spatial_scale)
+    elif cfg.roi_func == 'RoIAlign':
+        pool = fluid.layers.roi_align(
+            input=head_input,
+            rois=rois,
+            pooled_height=cfg.roi_resolution,
+            pooled_width=cfg.roi_resolution,
+            spatial_scale=cfg.spatial_scale,
+            sampling_ratio=cfg.sampling_ratio)
+
+    res5 = layer_warp(bottleneck, pool, 512, 3, 2, name="res5")
+    return res5

fluid/PaddleCV/faster_rcnn/profile.py → fluid/PaddleCV/rcnn/profile.py

@@ -37,18 +37,15 @@ def train():
     devices = os.getenv("CUDA_VISIBLE_DEVICES") or ""
     devices_num = len(devices.split(","))
     total_batch_size = devices_num * cfg.TRAIN.im_per_batch
-    model = model_builder.FasterRCNN(
+    model = model_builder.RCNN(
         add_conv_body_func=resnet.add_ResNet50_conv4_body,
         add_roi_box_head_func=resnet.add_ResNet_roi_conv5_head,
         use_pyreader=cfg.use_pyreader,
         use_random=False)
     model.build_model(image_shape)
-    loss_cls, loss_bbox, rpn_cls_loss, rpn_reg_loss = model.loss()
-    loss_cls.persistable = True
-    loss_bbox.persistable = True
-    rpn_cls_loss.persistable = True
-    rpn_reg_loss.persistable = True
-    loss = loss_cls + loss_bbox + rpn_cls_loss + rpn_reg_loss
+    losses, keys = model.loss()
+    loss = losses[0]
+    fetch_list = [loss]
 
     boundaries = cfg.lr_steps
     gamma = cfg.lr_gamma
@@ -95,8 +92,6 @@ def train():
         train_reader = reader.train(batch_size=total_batch_size, shuffle=False)
         feeder = fluid.DataFeeder(place=place, feed_list=model.feeds())
 
-    fetch_list = [loss, loss_cls, loss_bbox, rpn_cls_loss, rpn_reg_loss]
-
     def run(iterations):
         reader_time = []
         run_time = []
@@ -109,20 +104,16 @@ def train():
             reader_time.append(end_time - start_time)
             start_time = time.time()
             if cfg.parallel:
-                losses = train_exe.run(fetch_list=[v.name for v in fetch_list],
-                                       feed=feeder.feed(data))
+                outs = train_exe.run(fetch_list=[v.name for v in fetch_list],
+                                     feed=feeder.feed(data))
             else:
-                losses = exe.run(fluid.default_main_program(),
-                                 fetch_list=[v.name for v in fetch_list],
-                                 feed=feeder.feed(data))
+                outs = exe.run(fluid.default_main_program(),
+                               fetch_list=[v.name for v in fetch_list],
+                               feed=feeder.feed(data))
             end_time = time.time()
             run_time.append(end_time - start_time)
             total_images += len(data)
-
-            lr = np.array(fluid.global_scope().find_var('learning_rate')
-                          .get_tensor())
-            print("Batch {:d}, lr {:.6f}, loss {:.6f} ".format(batch_id, lr[0],
-                                                               losses[0][0]))
+            print("Batch {:d}, loss {:.6f} ".format(batch_id, np.mean(outs[0])))
         return reader_time, run_time, total_images
 
     def run_pyreader(iterations):
@@ -135,18 +126,16 @@ def train():
             for batch_id in range(iterations):
                 start_time = time.time()
                 if cfg.parallel:
-                    losses = train_exe.run(
+                    outs = train_exe.run(
                         fetch_list=[v.name for v in fetch_list])
                 else:
-                    losses = exe.run(fluid.default_main_program(),
-                                     fetch_list=[v.name for v in fetch_list])
+                    outs = exe.run(fluid.default_main_program(),
+                                   fetch_list=[v.name for v in fetch_list])
                 end_time = time.time()
                 run_time.append(end_time - start_time)
                 total_images += devices_num
-                lr = np.array(fluid.global_scope().find_var('learning_rate')
-                              .get_tensor())
-                print("Batch {:d}, lr {:.6f}, loss {:.6f} ".format(batch_id, lr[
-                    0], losses[0][0]))
+                print("Batch {:d}, loss {:.6f} ".format(batch_id,
+                                                        np.mean(outs[0])))
         except fluid.core.EOFException:
             py_reader.reset()
 

fluid/PaddleCV/faster_rcnn/roidbs.py → fluid/PaddleCV/rcnn/roidbs.py

@@ -36,24 +36,39 @@ import matplotlib
 matplotlib.use('Agg')
 from pycocotools.coco import COCO
 import box_utils
+import segm_utils
 from config import cfg
 
 logger = logging.getLogger(__name__)
 
 
+class DatasetPath(object):
+    def __init__(self, mode):
+        self.mode = mode
+        mode_name = 'train' if mode == 'train' else 'val'
+        if cfg.dataset != 'coco2014' and cfg.dataset != 'coco2017':
+            raise NotImplementedError('Dataset {} not supported'.format(
+                cfg.dataset))
+        self.sub_name = mode_name + cfg.dataset[-4:]
+
+    def get_data_dir(self):
+        return os.path.join(cfg.data_dir, self.sub_name)
+
+    def get_file_list(self):
+        sfile_list = 'annotations/instances_' + self.sub_name + '.json'
+        return os.path.join(cfg.data_dir, sfile_list)
+
+
 class JsonDataset(object):
     """A class representing a COCO json dataset."""
 
-    def __init__(self, train=False):
+    def __init__(self, mode):
         print('Creating: {}'.format(cfg.dataset))
         self.name = cfg.dataset
-        self.is_train = train
-        if self.is_train:
-            data_dir = cfg.train_data_dir
-            file_list = cfg.train_file_list
-        else:
-            data_dir = cfg.val_data_dir
-            file_list = cfg.val_file_list
+        self.is_train = mode == 'train'
+        data_path = DatasetPath(mode)
+        data_dir = data_path.get_data_dir()
+        file_list = data_path.get_file_list()
         self.image_directory = data_dir
         self.COCO = COCO(file_list)
         # Set up dataset classes
@@ -91,8 +106,9 @@ class JsonDataset(object):
             end_time = time.time()
             print('_add_gt_annotations took {:.3f}s'.format(end_time -
                                                             start_time))
-            print('Appending horizontally-flipped training examples...')
-            self._extend_with_flipped_entries(roidb)
+            if cfg.TRAIN.use_flipped:
+                print('Appending horizontally-flipped training examples...')
+                self._extend_with_flipped_entries(roidb)
         print('Loaded dataset: {:s}'.format(self.name))
         print('{:d} roidb entries'.format(len(roidb)))
         if self.is_train:
@@ -111,6 +127,7 @@ class JsonDataset(object):
         entry['gt_classes'] = np.empty((0), dtype=np.int32)
         entry['gt_id'] = np.empty((0), dtype=np.int32)
         entry['is_crowd'] = np.empty((0), dtype=np.bool)
+        entry['segms'] = []
         # Remove unwanted fields that come from the json file (if they exist)
         for k in ['date_captured', 'url', 'license', 'file_name']:
             if k in entry:
@@ -126,9 +143,15 @@ class JsonDataset(object):
         objs = self.COCO.loadAnns(ann_ids)
         # Sanitize bboxes -- some are invalid
         valid_objs = []
+        valid_segms = []
         width = entry['width']
         height = entry['height']
         for obj in objs:
+            if isinstance(obj['segmentation'], list):
+                # Valid polygons have >= 3 points, so require >= 6 coordinates
+                obj['segmentation'] = [
+                    p for p in obj['segmentation'] if len(p) >= 6
+                ]
             if obj['area'] < cfg.TRAIN.gt_min_area:
                 continue
             if 'ignore' in obj and obj['ignore'] == 1:
@@ -141,6 +164,8 @@ class JsonDataset(object):
             if obj['area'] > 0 and x2 > x1 and y2 > y1:
                 obj['clean_bbox'] = [x1, y1, x2, y2]
                 valid_objs.append(obj)
+                valid_segms.append(obj['segmentation'])
+
         num_valid_objs = len(valid_objs)
 
         gt_boxes = np.zeros((num_valid_objs, 4), dtype=entry['gt_boxes'].dtype)
@@ -158,6 +183,7 @@ class JsonDataset(object):
         entry['gt_classes'] = np.append(entry['gt_classes'], gt_classes)
         entry['gt_id'] = np.append(entry['gt_id'], gt_id)
         entry['is_crowd'] = np.append(entry['is_crowd'], is_crowd)
+        entry['segms'].extend(valid_segms)
 
     def _extend_with_flipped_entries(self, roidb):
         """Flip each entry in the given roidb and return a new roidb that is the
@@ -175,11 +201,13 @@ class JsonDataset(object):
             gt_boxes[:, 2] = width - oldx1 - 1
             assert (gt_boxes[:, 2] >= gt_boxes[:, 0]).all()
             flipped_entry = {}
-            dont_copy = ('gt_boxes', 'flipped')
+            dont_copy = ('gt_boxes', 'flipped', 'segms')
             for k, v in entry.items():
                 if k not in dont_copy:
                     flipped_entry[k] = v
             flipped_entry['gt_boxes'] = gt_boxes
+            flipped_entry['segms'] = segm_utils.flip_segms(
+                entry['segms'], entry['height'], entry['width'])
             flipped_entry['flipped'] = True
             flipped_roidb.append(flipped_entry)
         roidb.extend(flipped_roidb)

fluid/PaddleCV/rcnn/scripts/eval.sh

+#!/bin/bash
+export CUDA_VISIBLE_DEVICES=0
+
+model=$1 # faster_rcnn, mask_rcnn
+if [ "$model" = "faster_rcnn" ]; then
+  mask_on="--MASK_ON False"
+elif [ "$model" = "mask_rcnn" ]; then
+  mask_on="--MASK_ON True"
+else
+  echo "Invalid model provided. Please use one of {faster_rcnn, mask_rcnn}"
+  exit 1
+fi
+
+python -u ../eval_coco_map.py \
+    $mask_on \
+    --pretrained_model=../output/model_iter179999 \
+    --data_dir=../dataset/coco/ \

fluid/PaddleCV/video/dataset/youtube8m/README.md

+1. Tensorflow is required to process tfrecords
+2. python tf2pkl.py $Source_dir $Target_dir