Merge remote-tracking branch 'paddle/develop' into prior_box

op_list.md

 # X2Paddle支持OP列表
-> 目前X2Paddle支持50+的TensorFlow OP，30+的Caffe Layer，覆盖了大部分CV分类模型常用的操作。我们在如下列表中给出了目前X2Paddle支持的全部OP。
+> 目前X2Paddle支持70+的TensorFlow OP，30+的Caffe Layer，覆盖了大部分CV分类模型常用的操作。我们在如下列表中给出了目前X2Paddle支持的全部OP。
 
 **注：** 目前，部分OP暂未支持，如您在转换过程中出现OP不支持的情况，可自行添加或反馈给我们。欢迎通过[ISSUE反馈](https://github.com/PaddlePaddle/X2Paddle/issues/new)的方式告知我们(模型名，代码实现或模型获取方式)，我们会及时跟进：）
 
@@ -7,20 +7,24 @@
 
 | 序号 | OP | 序号 | OP | 序号 | OP | 序号 | OP |
 |------|------|------|------|------|------|------|------|
-| 1  | Relu        | 2  | Relu6     | 3  | Shape     | 4  | Abs                   |
-| 5  | Sigmoid     | 6  | Exp       | 7  | Rsqrt     | 8  | swish_f32             |
-| 9  | Tanh        | 10 | LeakyRelu | 11 | Add       | 12 | RealDiv               |
-| 13 | Sub         | 14 | Maximum   | 15 | Mul       | 16 | FloorDiv              |
-| 17 | Placeholder | 18 | Const     | 19 | Transpose | 20 | FusedBatchNorm        |
-| 21 | Conv2D      | 22 | BiasAdd   | 23 | MaxPool   | 24 | DepthwiseConv2dNative |
-| 25 | Reshape     | 26 | AvgPool   | 27 | SplitV    | 28 | SquaredDifference     |
-| 29 | Tile        | 30 | Pack      | 31 | Pad       | 32 | ResizeBilinear        |
-| 33 | Mean        | 34 | MatMul    | 35 | ArgMax    | 36 | StridedSlice          |
-| 37 | Slice       | 38 | Sum       | 39 | Max       | 40 | Conv2DBackpropInput   |
-| 41 | Cast        | 42 | Split     | 43 | Squeeze   | 44 | ResizeNearestNeighbor |
-| 45 | Softmax     | 46 | Range     | 47 | ConcatV2  | 48 | MirrorPad             |
-| 49 | Identity    | 50 | GreaterEqual  | 51 | StopGradient | 52 | Minimum |
-| 53 | RadnomUniform | 54 | Fill | 55 | Floor | 56 | DepthToSpace |
+| 1  | Relu             | 2  | Relu6          | 3  | Shape          | 4  | Abs                   |
+| 5  | Sigmoid          | 6  | Exp            | 7  | Rsqrt          | 8  | swish_f32             |
+| 9  | Tanh             | 10 | LeakyRelu      | 11 | Add            | 12 | RealDiv               |
+| 13 | Sub              | 14 | Maximum        | 15 | Mul            | 16 | FloorDiv              |
+| 17 | Placeholder      | 18 | Const          | 19 | Transpose      | 20 | FusedBatchNorm        |
+| 21 | Conv2D           | 22 | BiasAdd        | 23 | MaxPool        | 24 | DepthwiseConv2dNative |
+| 25 | Reshape          | 26 | AvgPool        | 27 | SplitV         | 28 | SquaredDifference     |
+| 29 | Tile             | 30 | Pack           | 31 | Pad            | 32 | ResizeBilinear        |
+| 33 | Mean             | 34 | MatMul         | 35 | ArgMax         | 36 | StridedSlice          |
+| 37 | Slice            | 38 | Sum            | 39 | Max            | 40 | Conv2DBackpropInput   |
+| 41 | Cast             | 42 | Split          | 43 | Squeeze        | 44 | ResizeNearestNeighbor |
+| 45 | Softmax          | 46 | Range          | 47 | ConcatV2       | 48 | MirrorPad             |
+| 49 | Identity         | 50 | GreaterEqual   | 51 | StopGradient   | 52 | Minimum               |
+| 53 | RadnomUniform    | 54 | Fill           | 55 | Floor          | 56 | DepthToSpace          |
+| 57 | Sqrt             | 58 | Softplus       | 59 | Erf            | 60 | AddV2                 |
+| 61 | LessEqual        | 62 | BatchMatMul    | 63 | BatchMatMulV2  | 64 | ExpandDims            |
+| 65 | BatchToSpaceND   | 66 | SpaceToBatchND | 67 | OneHot         | 68 | Pow                   |
+| 69 | All              | 70 | GatherV2       | 71 | IteratorV2     |    |                       |
 
 ## Caffe
 

x2paddle/decoder/onnx_shape_inference.py

@@ -267,9 +267,8 @@ class SymbolicShapeInference:
 
         if pending_nodes and self.verbose_ > 0:
             print('SymbolicShapeInference: orphaned nodes discarded: ')
-            print(
-                * [n.op_type + ': ' + n.output[0] for n in pending_nodes],
-                sep='\n')
+            print('\n'.join(
+                [n.op_type + ': ' + n.output[0] for n in pending_nodes]))
         if input_shapes is not None:
             for input_name, shape in input_shapes.items():
                 for idx in range(len(self.out_mp_.graph.input)):

x2paddle/op_mapper/caffe_custom_layer/normalize.py

@@ -17,7 +17,7 @@ def normalize_layer(inputs,
     scale_param = fluid.layers.create_parameter(
         shape=[1] if channel_shared else [1, 1, 1, input_shape[0][1]],
         dtype=input.dtype,
-        attr=name + '_scale')
+        attr=fluid.ParamAttr(name=name + '_scale'))
     scale_param = fluid.layers.reshape(x=scale_param, \
                   shape=[1] if channel_shared else [input_shape[0][1]])
     out = fluid.layers.elementwise_mul(

x2paddle/op_mapper/paddle2onnx/opset9/opset.py

@@ -875,6 +875,14 @@ class OpSet9(object):
             axes=op.attr('axes'))
         return node
 
+    def cast(self, op, block):
+        node = helper.make_node(
+            'Cast',
+            inputs=op.input('X'),
+            outputs=op.output('Out'),
+            to=self.paddle_onnx_dtype_map[op.attr('out_dtype')])
+        return node
+
     def arg_max(self, op, block):
         node = helper.make_node(
             'ArgMax',

x2paddle/op_mapper/tf_op_mapper_nhwc.py

@@ -299,6 +299,10 @@ class TFOpMapperNHWC(OpMapper):
         data_format = node.get_attr("data_format").decode()
         pad_mode = node.get_attr("padding").decode()
         channel_first = data_format == "NCHW"
+        if data_format == "NHWC":
+            n, h, w, c = input.out_shapes[0]
+        else:
+            n, c, h, w = input.out_shapes[0]
 
         if kernel.layer_type == 'Const':
             kernel_value = kernel.value
@@ -329,10 +333,15 @@ class TFOpMapperNHWC(OpMapper):
             "dilation": dilations[2:4],
             "padding": string(pad_mode)
         }
-
         if hasattr(node, 'dilation') and attr['dilation'] == [1, 1]:
             if len(node.dilation) == 1:
                 attr['dilation'] = [1, node.dilation[0]]
+
+        if c == -1:
+            reshape_attr = {"shape": [0, k_size[2], 0, 0]}
+            node.fluid_code.add_layer(
+                "reshape", inputs=input, output=input, param_attr=reshape_attr)
+
         node.fluid_code.add_layer(
             "conv2d", inputs=input, output=node, param_attr=attr)
         if not channel_first:
@@ -748,11 +757,12 @@ class TFOpMapperNHWC(OpMapper):
             self.add_omit_nodes(begin.layer_name, node.layer_name)
             begin = begin.value.tolist()
         else:
-            begin = begin
-            shape = begin.out_shapes[0]
-            attr = {"shape": shape}
-            node.fluid_code.add_layer(
-                "reshape", inputs=begin, output=begin, param_attr=attr)
+            begin = self.decoder.infer_tensor(begin).tolist()
+
+#             shape = begin.out_shapes[0]
+#             attr = {"shape": shape}
+#             node.fluid_code.add_layer(
+#                 "reshape", inputs=begin, output=begin, param_attr=attr)
         if size.layer_type == "Const":
             self.add_omit_nodes(size.layer_name, node.layer_name)
             size = size.value.tolist()

x2paddle/optimizer/tf_optimizer.py

@@ -863,6 +863,9 @@ class TFOptimizer(object):
                     weight = numpy.expand_dims(weight, 2)
                     weight = numpy.expand_dims(weight, 3)
                     self.op_mapper.weights[in_nodes3[0].layer_name] = weight
+                    # fix bug in Paddle1.8.3 and may change in next version.
+                    self.op_mapper.weights[in_nodes3[0].layer_name +
+                                           '_1'] = weight.reshape(1, -1)
                     in_nodes3[0].fluid_code.layers[0].param_attr["shape"] = [
                         1, in_shape[-1], 1, 1
                     ]
@@ -885,7 +888,7 @@ class TFOptimizer(object):
                     node.fluid_code.clear()
                     attr = {
                         "mode": string(mode),
-                        "param_attr": string(in_nodes3[0].layer_name)
+                        "param_attr": string(in_nodes3[0].layer_name + "_1")
                     }
 
                     node.fluid_code.add_layer(

x2paddle_model_zoo.md

 # X2Paddle模型测试库
-> 目前X2Paddle支持50+的TensorFlow OP，40+的Caffe Layer，覆盖了大部分CV分类模型常用的操作。我们在如下模型列表中测试了X2Paddle的转换。
+> 目前X2Paddle支持70+的TensorFlow OP，40+的Caffe Layer，覆盖了大部分CV分类模型常用的操作。我们在如下模型列表中测试了X2Paddle的转换。
 
 **注：** 受限于不同框架的差异，部分模型可能会存在目前无法转换的情况，如TensorFlow中包含控制流的模型，NLP模型等。对于CV常见的模型，如若您发现无法转换或转换失败，存在较大diff等问题，欢迎通过[ISSUE反馈](https://github.com/PaddlePaddle/X2Paddle/issues/new)的方式告知我们(模型名，代码实现或模型获取方式)，我们会及时跟进：）
 
@@ -20,10 +20,13 @@
 | ResNet_V1_101 | [code](https://github.com/tensorflow/models/tree/master/research/slim/nets) |-|
 | ResNet_V2_101 | [code](https://github.com/tensorflow/models/tree/master/research/slim/nets) |-|
 | UNet | [code1](https://github.com/jakeret/tf_unet )/[code2](https://github.com/lyatdawn/Unet-Tensorflow) |-|
-|MTCNN | [code](https://github.com/AITTSMD/MTCNN-Tensorflow) |-|
-|YOLO-V3| [code](https://github.com/YunYang1994/tensorflow-yolov3) | 转换需要关闭NHWC->NCHW的优化，见[文档Q2](FAQ.md) |
-| FALSR | [code](https://github.com/xiaomi-automl/FALSR) | - |
-| DCSCN | [code](https://modelzoo.co/model/dcscn-super-resolution) | - |
+| MTCNN | [code](https://github.com/AITTSMD/MTCNN-Tensorflow) |-|
+| YOLO-V3| [code](https://github.com/YunYang1994/tensorflow-yolov3) | 转换需要关闭NHWC->NCHW的优化，见[文档Q2](FAQ.md) |
+| FALSR | [code](https://github.com/xiaomi-automl/FALSR) | 需使用参数without_data_format_optimization |
+| DCSCN | [code](https://modelzoo.co/model/dcscn-super-resolution) | 需使用参数without_data_format_optimization |
+| Bert（albert） | [code](https://github.com/google-research/albert#pre-trained-models) | 需使用参数without_data_format_optimization |
+| Bert（chinese_L-12_H-768_A-12） | [code](https://github.com/google-research/bert#pre-trained-models) | 需使用参数without_data_format_optimization |
+| Bert（multi_cased_L-12_H-768_A-12） | [code](https://github.com/google-research/bert#pre-trained-models) | 需使用参数without_data_format_optimization |
 
 ## Caffe