[onnx]support paddle1.8, remove onnxruntime

.pre-commit-config.yaml

--   repo: local
+-   repo: https://github.com/PaddlePaddle/mirrors-yapf.git
+    sha: 0d79c0c469bab64f7229c9aca2b1186ef47f0e37
     hooks:
     -   id: yapf
-        name: yapf
-        entry: yapf
-        language: system
-        args: [-i, --style .style.yapf]
         files: \.py$
-
 -   repo: https://github.com/pre-commit/pre-commit-hooks
     sha: a11d9314b22d8f8c7556443875b731ef05965464
     hooks:
@@ -18,6 +14,7 @@
     -   id: check-symlinks
     -   id: check-added-large-files
 -   repo: local
+
     hooks:
     -   id: copyright_checker
         name: copyright_checker

.travis.yml

 language: python
 
 python:
-  - '2.7'
   - '3.5'
-  - '3.6'
 
 script:
   - if [[ $TRAVIS_PYTHON_VERSION != 2.7 ]]; then /bin/bash ./tools/check_code_style.sh; fi

FAQ.md

@@ -11,3 +11,6 @@ x2paddle -f tensorflow -m tf.pb -s pd-model --without_data_format_optimization -
 ```
 
 > 1. 目前Tensorflow的CV模型大部分均为`NHWC`的输入格式，而Paddle的默认输入格式为`NCHW`，因此X2Paddle在转换过程中，会对如`axis`， `shape`等参数进行转换，适应Paddle的NCHW格式。但在这种情况下，可能会由于TensorFlow模型太复杂，导致出错。  指定`--without_data_format_optimization`后，会停止对`axis`，`shape`等参数的优化（这可能会带来一定数量的transpose操作）
+
+**Q3. ONNX模型转换过程中，提示『Unknown shape for input tensor[tensor name: "input"] -> shape: ['batch', 'sequence']， Please define shape of input here』**  
+A：该提示信息表示从ONNX的模型中获取到输入tensor(tensor名为"input:)的shape是语义象征性的['batch', 'sequence']，而不是dim为int类型的shape，从而可能会因为部分node的shape无法推理，导致转换失败。所以用户可以尝试手动在提示后输入详细的shape信息，如:-1,3,224,224  其中-1表示Batch

README.md

@@ -10,12 +10,12 @@ X2Paddle在多个主流的CV模型上，测试过TensorFlow/Caffe/ONNX模型的
 ## 环境依赖
 
 python == 2.7 | python >= 3.5  
-paddlepaddle >= 1.6.0  
+paddlepaddle >= 1.8.0  
 
 **按需安装以下依赖**  
 tensorflow ： tensorflow == 1.14.0  
 caffe ： 无  
-onnx ： onnx == 1.6.0  onnxruntime == 1.0.0
+onnx ： onnx == 1.6.0
 
 ## 安装
 ### 安装方式一（推荐）
@@ -44,10 +44,15 @@ x2paddle --framework=caffe --prototxt=deploy.prototxt --weight=deploy.caffemodel
 ```
 x2paddle --framework=onnx --model=onnx_model.onnx --save_dir=pd_model
 ```
+### Paddle2ONNX
+```
+# 注意：paddle_infer_model_dir下需包含__model__和__params__两个文件
+x2paddle --framework=paddle2onnx --model=paddle_infer_model_dir --save_dir=onnx_model
+```
 ### 参数选项
 | 参数 | |
 |----------|--------------|
-|--framework | 源模型类型 (tensorflow、caffe、onnx) |
+|--framework | 源模型类型 (tensorflow、caffe、onnx、paddle2onnx) |
 |--prototxt | 当framework为caffe时，该参数指定caffe模型的proto文件路径 |
 |--weight | 当framework为caffe时，该参数指定caffe模型的参数文件路径 |
 |--save_dir | 指定转换后的模型保存目录路径 |
@@ -58,6 +63,7 @@ x2paddle --framework=onnx --model=onnx_model.onnx --save_dir=pd_model
 |--params_merge | **[可选]** 当指定该参数时，转换完成后，inference_model中的所有模型参数将合并保存为一个文件__params__ |
 
 
+
 ## 使用转换后的模型
 转换后的模型包括`model_with_code`和`inference_model`两个目录。  
 `model_with_code`中保存了模型参数，和转换后的python模型代码  

setup.py

@@ -11,8 +11,7 @@ setuptools.setup(
     version=x2paddle.__version__,
     author="dltp-sz",
     author_email="dltp-sz@baidu.com",
-    description=
-    "a toolkit for converting trained model to PaddlePaddle from other deep learning frameworks.",
+    description="a toolkit for converting trained model to PaddlePaddle from other deep learning frameworks.",
     long_description=long_description,
     long_description_content_type="text/plain",
     url="https://github.com/PaddlePaddle/x2paddle",
@@ -23,6 +22,4 @@ setuptools.setup(
         "Operating System :: OS Independent",
     ],
     license='Apache 2.0',
-    entry_points={'console_scripts': [
-        'x2paddle=x2paddle.convert:main',
-    ]})
+    entry_points={'console_scripts': ['x2paddle=x2paddle.convert:main', ]})

tools/merge_params.py

@@ -5,12 +5,14 @@ model_dir = sys.argv[1]
 new_model_dir = sys.argv[2]
 exe = fluid.Executor(fluid.CPUPlace())
 [inference_program, feed_target_names,
- fetch_targets] = fluid.io.load_inference_model(dirname=model_dir, executor=exe)
+ fetch_targets] = fluid.io.load_inference_model(
+     dirname=model_dir, executor=exe)
 
 print(feed_target_names)
-fluid.io.save_inference_model(dirname=new_model_dir,
-                              feeded_var_names=feed_target_names,
-                              target_vars=fetch_targets,
-                              executor=exe,
-                              main_program=inference_program,
-                              params_filename="__params__")
+fluid.io.save_inference_model(
+    dirname=new_model_dir,
+    feeded_var_names=feed_target_names,
+    target_vars=fetch_targets,
+    executor=exe,
+    main_program=inference_program,
+    params_filename="__params__")

x2paddle/__init__.py

-__version__ = "0.7.1"
+__version__ = "0.7.4"

x2paddle/convert.py

-#   Copyright (c) 2019  PaddlePaddle Authors. All Rights Reserved.
+# Copyright (c) 2020  PaddlePaddle Authors. All Rights Reserved.
 #
 # Licensed under the Apache License, Version 2.0 (the "License"
 # you may not use this file except in compliance with the License.
@@ -19,32 +19,37 @@ import sys
 
 def arg_parser():
     parser = argparse.ArgumentParser()
-    parser.add_argument("--model",
-                        "-m",
-                        type=_text_type,
-                        default=None,
-                        help="define model file path for tensorflow or onnx")
-    parser.add_argument("--prototxt",
-                        "-p",
-                        type=_text_type,
-                        default=None,
-                        help="prototxt file of caffe model")
-    parser.add_argument("--weight",
-                        "-w",
-                        type=_text_type,
-                        default=None,
-                        help="weight file of caffe model")
-    parser.add_argument("--save_dir",
-                        "-s",
-                        type=_text_type,
-                        default=None,
-                        help="path to save translated model")
+    parser.add_argument(
+        "--model",
+        "-m",
+        type=_text_type,
+        default=None,
+        help="define model file path for tensorflow or onnx")
+    parser.add_argument(
+        "--prototxt",
+        "-p",
+        type=_text_type,
+        default=None,
+        help="prototxt file of caffe model")
+    parser.add_argument(
+        "--weight",
+        "-w",
+        type=_text_type,
+        default=None,
+        help="weight file of caffe model")
+    parser.add_argument(
+        "--save_dir",
+        "-s",
+        type=_text_type,
+        default=None,
+        help="path to save translated model")
     parser.add_argument(
         "--framework",
         "-f",
         type=_text_type,
         default=None,
-        help="define which deeplearning framework(tensorflow/caffe/onnx)")
+        help="define which deeplearning framework(tensorflow/caffe/onnx/paddle2onnx)"
+    )
     parser.add_argument(
         "--caffe_proto",
         "-c",
@@ -52,27 +57,30 @@ def arg_parser():
         default=None,
         help="optional: the .py file compiled by caffe proto file of caffe model"
     )
-    parser.add_argument("--version",
-                        "-v",
-                        action="store_true",
-                        default=False,
-                        help="get version of x2paddle")
+    parser.add_argument(
+        "--version",
+        "-v",
+        action="store_true",
+        default=False,
+        help="get version of x2paddle")
     parser.add_argument(
         "--without_data_format_optimization",
         "-wo",
         action="store_true",
         default=False,
         help="tf model conversion without data format optimization")
-    parser.add_argument("--define_input_shape",
-                        "-d",
-                        action="store_true",
-                        default=False,
-                        help="define input shape for tf model")
-    parser.add_argument("--params_merge",
-                        "-pm",
-                        action="store_true",
-                        default=False,
-                        help="define whether merge the params")
+    parser.add_argument(
+        "--define_input_shape",
+        "-d",
+        action="store_true",
+        default=False,
+        help="define input shape for tf model")
+    parser.add_argument(
+        "--params_merge",
+        "-pm",
+        action="store_true",
+        default=False,
+        help="define whether merge the params")
 
     return parser
 
@@ -117,7 +125,6 @@ def tf2paddle(model_path,
         optimizer.merge_bias()
         optimizer.optimize_sub_graph()
 
-
 #        optimizer.merge_batch_norm()
 #        optimizer.merge_prelu()
     else:
@@ -165,16 +172,35 @@ def onnx2paddle(model_path, save_dir, params_merge=False):
         return
     print("Now translating model from onnx to paddle.")
 
-    from x2paddle.op_mapper.onnx_op_mapper import ONNXOpMapper
+    from x2paddle.op_mapper.onnx.onnx_helper import ONNXOpMapperFactory
     from x2paddle.decoder.onnx_decoder import ONNXDecoder
     from x2paddle.optimizer.onnx_optimizer import ONNXOptimizer
-    import onnxruntime
     model = ONNXDecoder(model_path)
-    mapper = ONNXOpMapper(model, save_dir)
+    factory = ONNXOpMapperFactory()
+    mapper = factory.create_onnx_op_mapper(model)
+    print("Model optimizing ...")
     optimizer = ONNXOptimizer(mapper)
+    print("Model optimized.")
 
-    optimizer.delete_redundance_code()
+    print("Paddle model and code generating ...")
     mapper.save_inference_model(save_dir, params_merge)
+    print("Paddle model and code generated.")
+
+
+def paddle2onnx(model_path, save_dir):
+    from x2paddle.decoder.paddle_decoder import PaddleDecoder
+    from x2paddle.op_mapper.paddle_op_mapper import PaddleOpMapper
+    model = PaddleDecoder(model_path, '__model__', '__params__')
+    mapper = PaddleOpMapper()
+    mapper.convert(model.program, save_dir)
+
+
+def paddle2onnx(model_path, save_dir):
+    from x2paddle.decoder.paddle_decoder import PaddleDecoder
+    from x2paddle.op_mapper.paddle_op_mapper import PaddleOpMapper
+    model = PaddleDecoder(model_path, '__model__', '__params__')
+    mapper = PaddleOpMapper()
+    mapper.convert(model.program, save_dir)
 
 
 def main():
@@ -196,22 +222,13 @@ def main():
     assert args.framework is not None, "--framework is not defined(support tensorflow/caffe/onnx)"
     assert args.save_dir is not None, "--save_dir is not defined"
 
-    if args.framework == "onnx":
-        try:
-            import onnxruntime as rt
-            version = rt.__version__
-            if version != '1.0.0':
-                print("[ERROR] onnxruntime==1.0.0 is required")
-                return
-        except:
-            print(
-                "[ERROR] onnxruntime is not installed, use \"pip install onnxruntime==1.0.0\"."
-            )
-
     try:
         import paddle
         v0, v1, v2 = paddle.__version__.split('.')
-        if int(v0) != 1 or int(v1) < 6:
+        print("paddle.__version__ = {}".format(paddle.__version__))
+        if v0 == '0' and v1 == '0' and v2 == '0':
+            print("[WARNING] You are use develop version of paddlepaddle")
+        elif int(v0) != 1 or int(v1) < 6:
             print("[ERROR] paddlepaddle>=1.6.0 is required")
             return
     except:
@@ -243,11 +260,18 @@ def main():
     elif args.framework == "onnx":
         assert args.model is not None, "--model should be defined while translating onnx model"
         params_merge = False
+
         if args.params_merge:
             params_merge = True
         onnx2paddle(args.model, args.save_dir, params_merge)
+
+    elif args.framework == "paddle2onnx":
+        assert args.model is not None, "--model should be defined while translating paddle model to onnx"
+        paddle2onnx(args.model, args.save_dir)
+
     else:
-        raise Exception("--framework only support tensorflow/caffe/onnx now")
+        raise Exception(
+            "--framework only support tensorflow/caffe/onnx/paddle2onnx now")
 
 
 if __name__ == "__main__":

x2paddle/core/fluid_code.py

@@ -46,8 +46,9 @@ class Layer(object):
             for input in self.inputs:
                 if isinstance(input, GraphNode):
                     if hasattr(input, "index"):
-                        in_list += (input.layer_name +
-                                    "[{}]".format(input.index) + ", ")
+                        in_list += (
+                            input.layer_name + "[{}]".format(input.index) + ", "
+                        )
                     else:
                         in_list += (input.layer_name + ", ")
                 elif isinstance(input, six.string_types):
@@ -71,8 +72,8 @@ class Layer(object):
                     layer_code = layer_code + key + "={}, ".format(input)
         elif isinstance(self.inputs, GraphNode):
             if hasattr(self.inputs, "index"):
-                layer_code += (self.inputs.layer_name +
-                               "[{}]".format(self.inputs.index))
+                layer_code += (
+                    self.inputs.layer_name + "[{}]".format(self.inputs.index))
             else:
                 layer_code += (self.inputs.layer_name)
             if self.op != "=":
@@ -88,6 +89,8 @@ class Layer(object):
         for key, value in param_attr.items():
             if '\n' in str(value):
                 value = string(str(value).replace('\n', ','))
+            if str(key) == 'attr':
+                value = 'ParamAttr(' + str(value) + ')'
             layer_code = layer_code + key + "={}, ".format(value)
         layer_code = layer_code.strip(", ")
 

x2paddle/core/op_mapper.py

@@ -29,11 +29,14 @@ def export_paddle_param(param, param_name, dir):
         "bool": [framework_pb2.VarType.BOOL, None]
     }
     shape = param.shape
+    if str(param.dtype) in ['uint8', 'uint_8', 'bool']:
+        param = param.astype('int64')
     if len(shape) == 0:
         assert param.size == 1, "Unexpected situation happend!"
         shape = [1]
-    assert str(param.dtype) in dtype_map, "Unknown dtype of params."
-
+    assert str(
+        param.dtype) in dtype_map, "Unknown dtype {} of params: {}.".format(
+            str(param.dtype), param_name)
     fp = open(os.path.join(dir, param_name), 'wb')
     numpy.array([0], dtype='int32').tofile(fp)
     numpy.array([0], dtype='int64').tofile(fp)
@@ -64,10 +67,8 @@ def run_net(param_dir="./"):
         b = os.path.exists(os.path.join(param_dir, var.name))
         return b
 
-    fluid.io.load_vars(exe,
-                       param_dir,
-                       fluid.default_main_program(),
-                       predicate=if_exist)
+    fluid.io.load_vars(
+        exe, param_dir, fluid.default_main_program(), predicate=if_exist)
 
 
 class OpMapper(object):
@@ -98,8 +99,8 @@ class OpMapper(object):
     def add_codes(self, codes, indent=0):
         if isinstance(codes, list):
             for code in codes:
-                self.paddle_codes += (self.tab * indent + code.strip('\n') +
-                                      '\n')
+                self.paddle_codes += (
+                    self.tab * indent + code.strip('\n') + '\n')
         elif isinstance(codes, str):
             self.paddle_codes += (self.tab * indent + codes.strip('\n') + '\n')
         else:
@@ -135,24 +136,25 @@ class OpMapper(object):
                     os.path.join(os.path.join(py_code_dir, var.name)))
                 return b
 
-            fluid.io.load_vars(exe,
-                               py_code_dir,
-                               fluid.default_main_program(),
-                               predicate=if_exist)
+            fluid.io.load_vars(
+                exe,
+                py_code_dir,
+                fluid.default_main_program(),
+                predicate=if_exist)
             if params_merge:
-                fluid.io.save_inference_model(dirname=os.path.join(
-                    save_dir, "inference_model"),
-                                              feeded_var_names=input_names,
-                                              target_vars=outputs,
-                                              executor=exe,
-                                              params_filename="__params__")
+                fluid.io.save_inference_model(
+                    dirname=os.path.join(save_dir, "inference_model"),
+                    feeded_var_names=input_names,
+                    target_vars=outputs,
+                    executor=exe,
+                    params_filename="__params__")
             else:
-                fluid.io.save_inference_model(dirname=os.path.join(
-                    save_dir, "inference_model"),
-                                              feeded_var_names=input_names,
-                                              target_vars=outputs,
-                                              executor=exe,
-                                              params_filename=None)
+                fluid.io.save_inference_model(
+                    dirname=os.path.join(save_dir, "inference_model"),
+                    feeded_var_names=input_names,
+                    target_vars=outputs,
+                    executor=exe,
+                    params_filename=None)
         except:
             raise Exception(
                 "Paddle code was saved in {}/model.py, but seems there's wrong exist, please check model.py manually."

x2paddle/decoder/caffe_decoder.py

@@ -49,13 +49,11 @@ class CaffeResolver(object):
 class CaffeGraphNode(GraphNode):
     def __init__(self, layer, type_str, layer_name=None):
         if layer_name is None:
-            super(CaffeGraphNode,
-                  self).__init__(layer,
-                                 layer.name.replace('/', '_').replace('-', '_'))
+            super(CaffeGraphNode, self).__init__(
+                layer, layer.name.replace('/', '_').replace('-', '_'))
         else:
-            super(CaffeGraphNode,
-                  self).__init__(layer,
-                                 layer_name.replace('/', '_').replace('-', '_'))
+            super(CaffeGraphNode, self).__init__(
+                layer, layer_name.replace('/', '_').replace('-', '_'))
         self.layer_type = type_str
         self.fluid_code = FluidCode()
         self.data = None
@@ -171,6 +169,14 @@ class CaffeGraph(Graph):
         self.input2layers(input_layers)
         self.transform_input_layers(layers, input_layers)
         layers = input_layers + layers
+        for layer in layers:
+            if hasattr(layer, 'name'):
+                name = getattr(layer, 'name')
+                setattr(layer, 'name', name.replace('/', '_').replace('-', '_'))
+            for i, name in enumerate(layer.bottom):
+                layer.bottom[i] = name.replace('/', '_').replace('-', '_')
+            for i, name in enumerate(layer.top):
+                layer.top[i] = name.replace('/', '_').replace('-', '_')
 
         top_layer = {}
         for layer in layers:
@@ -232,10 +238,12 @@ class CaffeDecoder(object):
 
     def load_using_pb(self):
         data = self.resolver.NetParameter()
-
         data.MergeFromString(open(self.model_path, 'rb').read())
-        pair = lambda layer: (layer.name, self.normalize_pb_data(layer))
         layers = data.layers or data.layer
+        for layer in layers:
+            setattr(layer, 'name',
+                    layer.name.replace('/', '_').replace('-', '_'))
+        pair = lambda layer: (layer.name, self.normalize_pb_data(layer))
         self.params = [pair(layer) for layer in layers if layer.blobs]
 
     def normalize_pb_data(self, layer):
@@ -246,21 +254,20 @@ class CaffeDecoder(object):
                 if layer.type == 'PReLU':
                     c_o, c_i, h, w = map(int, [1] + \
                         list(dims) + [1]* (3 - len(dims)))
-                elif layer.type == 'Normalize':
+                elif layer.type == 'Normalize' and len(dims) == 4:
                     data = np.asarray(list(blob.data), dtype=np.float32)
                     transformed.append(data)
                     continue
                 else:
-                    c_o, c_i, h, w = map(int, [1] * (4 - len(dims)) \
-                        + list(dims))
-
+                    c_o, c_i, h, w = map(int,
+                                         [1] * (4 - len(dims)) + list(dims))
             else:
                 c_o = blob.num
                 c_i = blob.channels
                 h = blob.height
                 w = blob.width
-            data = np.asarray(list(blob.data),
-                              dtype=np.float32).reshape(c_o, c_i, h, w)
+            data = np.asarray(
+                list(blob.data), dtype=np.float32).reshape(c_o, c_i, h, w)
 
             transformed.append(data)
         return transformed

x2paddle/decoder/onnx_decoder.py

@@ -14,6 +14,7 @@
 
 from x2paddle.core.graph import GraphNode, Graph
 from x2paddle.core.fluid_code import FluidCode
+from x2paddle.decoder.onnx_shape_inference import SymbolicShapeInference
 from onnx.checker import ValidationError
 from onnx.checker import check_model
 from onnx.utils import polish_model
@@ -53,7 +54,7 @@ class ONNXGraphNode(GraphNode):
         convert ONNX node attributes to dict
         """
         return {
-            attr.name: self.get_attribute_value2(attr)
+            attr.name: self.get_attribute_value(attr)
             for attr in self.layer.attribute
         }
 
@@ -64,16 +65,15 @@ class ONNXGraphNode(GraphNode):
             return None
         return self.attr_map['value']
 
-    def get_attribute_value2(self, attr):
+    def get_attribute_value(self, attr):
         """
         get_attribute_value enhanced
         """
         if attr.type == onnx.AttributeProto.TENSOR:
             dtype = np.dtype(TENSOR_TYPE_TO_NP_TYPE[attr.t.data_type])
             data = attr.t.raw_data
-            value = np.frombuffer(data,
-                                  dtype=dtype,
-                                  count=(len(data) // dtype.itemsize))
+            value = np.frombuffer(
+                data, dtype=dtype, count=(len(data) // dtype.itemsize))
         elif attr.type == onnx.AttributeProto.STRING:
             value = attr.s
             value = value.decode() if isinstance(value, bytes) else value
@@ -131,43 +131,90 @@ class ONNXGraphDataNode(GraphNode):
 
 class ONNXGraph(Graph):
     def __init__(self, onnx_model):
-        super(ONNXGraph, self).__init__(onnx_model.graph)
-        self.onnx_model = onnx_model
+        super(ONNXGraph, self).__init__(onnx_model)
+        self.fixed_input_shape = {}
         self.initializer = {}
         self.place_holder_nodes = list()
+        self.value_infos = {}
+        self.graph = onnx_model.graph
         self.get_place_holder_nodes()
-        self.value_infos = self.inferred_model_value_info(self.model)
-        self.results_of_inference = dict()
+        print("shape inferencing ...")
+        self.graph = SymbolicShapeInference.infer_shapes(
+            onnx_model, fixed_input_shape=self.fixed_input_shape)
+        print("shape inferenced.")
+        self.build()
+        self.collect_value_infos()
+        self.allocate_shapes()
 
     def get_inner_nodes(self):
         """
         generate inner node of ONNX model
         """
         inner_nodes = []
-        if not isinstance(self.model, onnx.GraphProto):
+        if not isinstance(self.graph, onnx.GraphProto):
             logger.error('graph is not a GraphProto instance')
             return
-        for initializer in self.model.initializer:
+        for initializer in self.graph.initializer:
             name = initializer.name
             inner_nodes.append(name)
         return inner_nodes
 
+    def get_symbolic_shape(self, dims):
+        shape = []
+        for dim in dims:
+            if dim.HasField('dim_param'):
+                shape.append(dim.dim_param)
+            else:
+                shape.append(dim.dim_value)
+        return shape
+
+    def check_input_shape(self, vi):
+        if vi.type.HasField('tensor_type'):
+            for dim in vi.type.tensor_type.shape.dim:
+                if dim.HasField(
+                        'dim_param') and vi.name not in self.fixed_input_shape:
+                    shape = self.get_symbolic_shape(
+                        vi.type.tensor_type.shape.dim)
+                    print(
+                        "Unknown shape for input tensor[tensor name: '{}'] -> shape: {}, Please define shape of input here,\nNote:you can use visualization tools like Netron to check input shape."
+                        .format(vi.name, shape))
+                    right_shape_been_input = False
+                    while not right_shape_been_input:
+                        try:
+                            shape = raw_input(
+                                "Shape of Input(e.g. -1,3,224,224), enter 'N' to skip: "
+                            )
+                        except:
+                            shape = input(
+                                "Shape of Input(e.g. -1,3,224,224), enter 'N' to skip: "
+                            )
+                        if shape.count("-1") > 1:
+                            print("Only 1 dimension can be -1, type again:)")
+                        else:
+                            right_shape_been_input = True
+                    if shape == 'N':
+                        break
+                    shape = [int(dim) for dim in shape.strip().split(',')]
+                    assert shape.count(-1) <= 1, "Only one dimension can be -1"
+                    self.fixed_input_shape[vi.name] = shape
+                    break
+
     def get_place_holder_nodes(self):
         """
         generate place_holder node of ONNX model
         """
         inner_nodes = self.get_inner_nodes()
-        input_nodes = [value.name for value in self.model.input]
-        for ipt_data in input_nodes:
-            if ipt_data not in inner_nodes:
-                self.place_holder_nodes.append(ipt_data)
+        for ipt_vi in self.graph.input:
+            if ipt_vi.name not in inner_nodes:
+                self.check_input_shape(ipt_vi)
+                self.place_holder_nodes.append(ipt_vi.name)
 
     def get_output_nodes(self):
         """
         generate output_nodes node of ONNX model
         """
         inner_nodes = self.get_inner_nodes()
-        output_nodes = [value.name for value in self.model.output]
+        output_nodes = [value.name for value in self.graph.output]
         for opt_data in output_nodes:
             if opt_data not in inner_nodes:
                 self.output_nodes.append(opt_data)
@@ -184,11 +231,11 @@ class ONNXGraph(Graph):
         """
         build topo_sort of ONNX model
         """
-        for layer in self.model.node:
+        for layer in self.graph.node:
             node = ONNXGraphNode(layer)
             self.node_map[layer.name] = node
 
-        for layer in self.model.input:
+        for layer in self.graph.input:
             if layer.name not in self.node_map:
                 is_place_holder = self.is_place_holder_nodes(layer.name)
                 self.node_map[layer.name] = ONNXGraphDataNode(
@@ -197,7 +244,7 @@ class ONNXGraph(Graph):
                     is_global_input=is_place_holder)
 
         #set data node's weight
-        for initializer in self.model.initializer:
+        for initializer in self.graph.initializer:
             name = initializer.name
             weight = to_array(initializer)
             if name in self.node_map:
@@ -205,9 +252,8 @@ class ONNXGraph(Graph):
                     self.node_map[name].weight = weight
                     self.node_map[name].embeded_as = []
             else:
-                self.node_map[name] = ONNXGraphDataNode(initializer,
-                                                        layer_name=name,
-                                                        is_global_input=False)
+                self.node_map[name] = ONNXGraphDataNode(
+                    initializer, layer_name=name, is_global_input=False)
                 self.node_map[name].weight = weight
                 self.node_map[name].embeded_as = []
 
@@ -230,7 +276,7 @@ class ONNXGraph(Graph):
                 continue
             if in_node not in self.node_map:
                 flag = 0
-                for nd in self.model.node:
+                for nd in self.graph.node:
                     for idx, opt in enumerate(nd.output):
                         if opt == in_node:
                             self.connect(nd.name, layer_name)
@@ -258,81 +304,86 @@ class ONNXGraph(Graph):
                 ipt_node.index = node.which_child[ipt_node.layer_name]
             return ipt_node
 
-    def graph_weights(self, graph):
+    def graph_weights(self):
         """
         generator for weights
         """
 
-        if not isinstance(graph, onnx.GraphProto):
+        if not isinstance(self.graph, onnx.GraphProto):
             logger.error('graph is not a GraphProto instance')
             return
 
-        for initializer in graph.initializer:
+        for initializer in self.graph.initializer:
             name = initializer.name
             weight = to_array(initializer)
             yield name, weight
 
-    def inferred_model_value_info(self, graph):
+    def collect_value_infos(self):
         """
         collect value/type info for an ONNX model
         """
-        assert isinstance(graph,
+        assert isinstance(self.graph,
                           onnx.GraphProto), 'model is not a ModelProto instance'
 
-        value_info = Dict()
-        for item in graph.value_info:
-            value_info[item.name] = {
+        for item in self.graph.value_info:
+            self.value_infos[item.name] = {
                 'dtype':
                 TENSOR_TYPE_TO_NP_TYPE[item.type.tensor_type.elem_type],
                 'shape':
                 [dim.dim_value for dim in item.type.tensor_type.shape.dim],
                 'external': False
             }
-        for item in graph.input:
-            assert item.name not in value_info
-            value_info[item.name] = {
-                'dtype':
-                TENSOR_TYPE_TO_NP_TYPE[item.type.tensor_type.elem_type],
-                'shape':
-                [dim.dim_value for dim in item.type.tensor_type.shape.dim],
-                'external': True
-            }
-        for item in graph.output:
-            assert item.name not in value_info
-            value_info[item.name] = {
-                'dtype':
-                TENSOR_TYPE_TO_NP_TYPE[item.type.tensor_type.elem_type],
-                'shape':
-                [dim.dim_value for dim in item.type.tensor_type.shape.dim],
-                'external': True
-            }
-        return value_info
+
+    # for item in self.graph.input:
+    #     self.value_infos[item.name] = {
+    #         'dtype':
+    #         TENSOR_TYPE_TO_NP_TYPE[item.type.tensor_type.elem_type],
+    #         'shape':
+    #         [dim.dim_value for dim in item.type.tensor_type.shape.dim],
+    #         'external': True
+    #     }
+    # for item in self.graph.output:
+    #     #assert item.name not in value_info
+    #     self.value_infos[item.name] = {
+    #         'dtype':
+    #         TENSOR_TYPE_TO_NP_TYPE[item.type.tensor_type.elem_type],
+    #         'shape':
+    #         [dim.dim_value for dim in item.type.tensor_type.shape.dim],
+    #         'external': True
+    #     }
+
+    def allocate_shapes(self):
+        """
+        run shape inference
+        """
+        for layer in self.graph.node:
+            node = self.node_map[layer.name]
+            for opt in layer.output:
+                if opt in self.value_infos:
+                    value_info = self.value_infos[opt]
+                    #if len(value_info['shape']) == 0 or value_info[
+                    #        'dtype'] is None or 0 in value_info['shape']:
+                    #    #TODO add node shape inference
+                    node.dtype = value_info['dtype']
+                    node.out_shapes.append(value_info['shape'])
+                else:
+                    node.out_shapes.append([])
 
 
 class ONNXDecoder(object):
     def __init__(self, onnx_model):
-        model = onnx.load(onnx_model)
+        onnx_model = onnx.load(onnx_model)
         print('model ir_version: {}, op version: {}'.format(
-            model.ir_version, model.opset_import[0].version))
-        if model.opset_import[0].version < 9:
-            _logger.warning(
-                'Now, onnx2paddle support convert onnx model opset_verison == 9,'
-                'opset_verison of your onnx model is %d < 9,'
-                'some operator maybe unsuccessful in convertion.',
-                model.opset_import[0].version)
-
-        check_model(model)
-        self.check_model_running_state(onnx_model)
-
-        model = onnx.shape_inference.infer_shapes(model)
-        model = self.optimize_model_skip_op_for_inference(model)
-        model = self.optimize_model_strip_initializer(model)
-        self.standardize_variable_name(model.graph)
-
-        self.model = model
-        graph = model.graph
-        self.onnx_graph = ONNXGraph(model)
-        self.onnx_graph.build()
+            onnx_model.ir_version, onnx_model.opset_import[0].version))
+        self.op_set = onnx_model.opset_import[0].version
+
+        check_model(onnx_model)
+
+        onnx_model = self.optimize_model_skip_op(onnx_model)
+        onnx_model = self.optimize_model_strip_initializer(onnx_model)
+        onnx_model = self.optimize_node_name(onnx_model)
+        self.graph = ONNXGraph(onnx_model)
+        #self.onnx_model = onnx_model
 
     def build_value_refs(self, nodes):
         """
@@ -375,14 +426,13 @@ class ONNXDecoder(object):
                     processed += 1
         return processed
 
-    def optimize_model_skip_op_for_inference(self, model, op_list=None):
+    def optimize_model_skip_op(self, model, op_list=None):
         """
         skip ops can be bypassed for inference
         """
+        nodes = model.graph.node
         if op_list is None:
             op_list = ['Dropout']
-
-        nodes = model.graph.node
         input_refs, output_refs = self.build_value_refs(nodes)
         ret = type(model)()
         ret.CopyFrom(model)
@@ -475,38 +525,11 @@ class ONNXDecoder(object):
             name = name.replace(s, '_')
         return 'x2paddle_' + name
 
-    def check_model_running_state(self, model_path):
-        import onnxruntime as rt
-        model = onnx.load(model_path)
-        model = onnx.shape_inference.infer_shapes(model)
-        if len(model.graph.value_info) < len(model.graph.node) - 1:
-            _logger.warning(
-                'During conversion of your  model, some operators will be assignd node.out_shape==None, '
-                'refer to https://github.com/onnx/onnx/blob/master/docs/ShapeInference.md'
-            )
-        try:
-            datatype_map = {
-                'tensor(int64)': 'int',
-                'tensor(float)': 'float32',
-                'tensor(int32)': 'int32'
-            }
-            input_dict = {}
-            sess = rt.InferenceSession(model_path)
-            for ipt in sess.get_inputs():
-                datatype = datatype_map[ipt.type]
-                input_dict[ipt.name] = np.random.random(
-                    ipt.shape).astype(datatype)
-
-            res = sess.run(None, input_feed=input_dict)
-        except:
-            raise Exception(
-                "onnxruntime inference onnx model failed, Please confirm the correctness of onnx model by onnxruntime, if onnx model is correct, please submit issue in github."
-            )
-
-    def standardize_variable_name(self, graph):
+    def optimize_node_name(self, model):
         """
         standardize variable name for paddle's code
         """
+        graph = model.graph
         for initializer in graph.initializer:
             initializer.name = self.make_variable_name(initializer.name)
         for ipt in graph.input:
@@ -525,3 +548,4 @@ class ONNXDecoder(object):
                     node.input[i] = self.make_variable_name(node.input[i])
             for i in range(len(node.output)):
                 node.output[i] = self.make_variable_name(node.output[i])
+        return model

x2paddle/decoder/onnx_shape_inference.py

+# Copyright (c) 2019  PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License"
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# Reference Code from https://github.com/microsoft/onnxruntime,  Licensed under the MIT License.
+
+# -*- coding: UTF-8 -*-
+import argparse
+import numpy as np
+import onnx
+import sys
+from onnx import helper, numpy_helper, shape_inference
+import sympy
+
+from packaging import version
+assert version.parse(onnx.__version__) >= version.parse("1.5.0")
+
+
+def get_attribute(node, attr_name, default_value=None):
+    found = [attr for attr in node.attribute if attr.name == attr_name]
+    if found:
+        return helper.get_attribute_value(found[0])
+    return default_value
+
+
+def get_dim_from_type_proto(dim):
+    return getattr(dim, dim.WhichOneof('value')) if type(
+        dim.WhichOneof('value')) == str else None
+
+
+def get_shape_from_type_proto(type_proto):
+    return [
+        get_dim_from_type_proto(d) for d in type_proto.tensor_type.shape.dim
+    ]
+
+
+def get_shape_from_sympy_shape(sympy_shape):
+    sympy_shape = [
+        None if i is None else (int(i) if is_literal(i) else str(i))
+        for i in sympy_shape
+    ]
+    return sympy_shape
+
+
+def is_literal(dim):
+    return type(dim) in [int, np.int64, np.int32, sympy.Integer] or (
+        hasattr(dim, 'is_number') and
+        dim.is_number)  # or (hasattr(dim, 'is_integer') and dim.is_integer)
+
+
+def handle_negative_axis(axis, rank):
+    assert axis < rank and axis >= -rank
+    return axis if axis >= 0 else rank + axis
+
+
+def get_opset(mp, domain=['', 'onnx', 'ai.onnx']):
+    if type(domain) != list:
+        domain = [domain]
+    for opset in mp.opset_import:
+        if opset.domain in domain:
+            return opset.version
+    return None
+
+
+def as_scalar(x):
+    if type(x) == list:
+        assert len(x) == 1
+        return x[0]
+    elif type(x) == np.ndarray:
+        return np.asscalar(x)
+    else:
+        return x
+
+
+def as_list(x, keep_none):
+    if type(x) == list:
+        return x
+    elif type(x) == np.ndarray:
+        return list(x)
+    elif keep_none and x is None:
+        return None
+    else:
+        return [x]
+
+
+def sympy_reduce_product(x):
+    if type(x) == list:
+        value = sympy.Integer(1)
+        for v in x:
+            value = value * v
+    else:
+        value = x
+    return value
+
+
+class SymbolicShapeInference:
+    def __init__(self, int_max, auto_merge, guess_output_rank, verbose):
+        self.dispatcher_ = {
+            'Add': self._infer_symbolic_compute_ops,
+            'ArrayFeatureExtractor': self._infer_ArrayFeatureExtractor,
+            'AveragePool': self._infer_Pool,
+            'Cast': self._infer_Cast,
+            'CategoryMapper': self._infer_CategoryMapper,
+            'Compress': self._infer_Compress,
+            'Concat': self._infer_Concat,
+            'ConstantOfShape': self._infer_ConstantOfShape,
+            'Conv': self._infer_Conv,
+            'CumSum': self._pass_on_shape_and_type,
+            'Div': self._infer_symbolic_compute_ops,
+            'Expand': self._infer_Expand,
+            'Equal': self._infer_symbolic_compute_ops,
+            'Gather': self._infer_Gather,
+            'GatherElements': self._infer_GatherElements,
+            'GatherND': self._infer_GatherND,
+            'If': self._infer_If,
+            'Loop': self._infer_Loop,
+            'MatMul': self._infer_MatMul,
+            'MatMulInteger16': self._infer_MatMulInteger,
+            'MaxPool': self._infer_Pool,
+            'Max': self._infer_symbolic_compute_ops,
+            'Min': self._infer_symbolic_compute_ops,
+            'Mul': self._infer_symbolic_compute_ops,
+            'NonMaxSuppression': self._infer_NonMaxSuppression,
+            'NonZero': self._infer_NonZero,
+            'OneHot': self._infer_OneHot,
+            'Pad': self._infer_Pad,
+            'Range': self._infer_Range,
+            'ReduceProd': self._infer_ReduceProd,
+            'Reshape': self._infer_Reshape,
+            'Resize': self._infer_Resize,
+            'Round': self._pass_on_shape_and_type,
+            'Scan': self._infer_Scan,
+            'ScatterElements': self._infer_ScatterElements,
+            'Shape': self._infer_Shape,
+            'Size': self._infer_Size,
+            'Slice': self._infer_Slice,
+            'Split': self._infer_Split,
+            'Squeeze': self._infer_Squeeze,
+            'Sub': self._infer_symbolic_compute_ops,
+            'Tile': self._infer_Tile,
+            'TopK': self._infer_TopK,
+            'Unsqueeze': self._infer_Unsqueeze,
+            'Where': self._infer_symbolic_compute_ops,
+            'Transpose': self._infer_Transpose,
+            'ZipMap': self._infer_ZipMap
+        }
+        self.run_ = True
+        self.suggested_merge_ = {}
+        self.symbolic_dims_ = {}
+        self.input_symbols_ = {}
+        self.auto_merge_ = auto_merge
+        self.guess_output_rank_ = guess_output_rank
+        self.verbose_ = verbose
+        self.int_max_ = int_max
+
+    def _add_suggested_merge(self, symbols, apply=False):
+        assert all([(type(s) == str and s in self.symbolic_dims_) or
+                    is_literal(s) for s in symbols])
+        symbols = set(symbols)
+        for k, v in self.suggested_merge_.items():
+            if k in symbols:
+                symbols.remove(k)
+                symbols.add(v)
+        map_to = None
+        # if there is literal, map to it first
+        for s in symbols:
+            if is_literal(s):
+                map_to = s
+                break
+        # when no literals, map to input symbolic dims, then existing symbolic dims
+        if map_to is None:
+            for s in symbols:
+                if s in self.input_symbols_:
+                    map_to = s
+                    break
+        if map_to is None:
+            for s in symbols:
+                if type(self.symbolic_dims_[s]) == sympy.Symbol:
+                    map_to = s
+                    break
+        # when nothing to map to, use the shorter one
+        if map_to is None:
+            if self.verbose_ > 0:
+                print(
+                    'Potential unsafe merge between symbolic expressions: ({})'.
+                    format(','.join(symbols)))
+            symbols_list = list(symbols)
+            lens = [len(s) for s in symbols_list]
+            map_to = symbols_list[lens.index(min(lens))]
+            symbols.remove(map_to)
+
+        for s in symbols:
+            if s == map_to:
+                continue
+            if is_literal(map_to) and is_literal(s):
+                assert int(map_to) == int(s)
+            self.suggested_merge_[s] = int(map_to) if is_literal(
+                map_to) else map_to
+            for k, v in self.suggested_merge_.items():
+                if v == s:
+                    self.suggested_merge_[k] = map_to
+        if apply and self.auto_merge_:
+            self._apply_suggested_merge()
+
+    def _apply_suggested_merge(self, graph_input_only=False):
+        if not self.suggested_merge_:
+            return
+        for i in list(self.out_mp_.graph.input) + (
+            [] if graph_input_only else list(self.out_mp_.graph.value_info)):
+            for d in i.type.tensor_type.shape.dim:
+                if d.dim_param in self.suggested_merge_:
+                    v = self.suggested_merge_[d.dim_param]
+                    if is_literal(v):
+                        d.dim_value = int(v)
+                    else:
+                        d.dim_param = v
+
+    def _preprocess(self, in_mp, input_shapes=None):
+        out_mp = onnx.ModelProto()
+        out_mp.CopyFrom(in_mp)
+        out_mp.graph.ClearField('node')
+        self.out_mp_ = out_mp
+
+        defined = set([
+            i.name
+            for i in list(in_mp.graph.input) + list(in_mp.graph.initializer)
+        ])
+        pending_nodes = []
+
+        # returns True if no more ready nodes
+        def _insert_ready_nodes():
+            ready_nodes = [
+                pn for pn in pending_nodes
+                if all([i in defined for i in pn.input if i])
+            ]
+            for rn in ready_nodes:
+                self.out_mp_.graph.node.add().CopyFrom(rn)
+                for o in rn.output:
+                    defined.add(o)
+                pending_nodes.remove(rn)
+            return not ready_nodes
+
+        # constant op -> initializer, topological sort
+        for in_n in in_mp.graph.node:
+            if in_n.op_type == 'Constant':
+                t = get_attribute(in_n, 'value')
+                t.name = in_n.output[0]
+                self.out_mp_.graph.initializer.add().CopyFrom(t)
+                defined.add(t.name)
+            else:
+                pending_nodes.append(in_n)
+            _insert_ready_nodes()
+
+        while pending_nodes:
+            if _insert_ready_nodes():
+                break
+
+        if pending_nodes and self.verbose_ > 0:
+            print('SymbolicShapeInference: orphaned nodes discarded: ')
+            print(
+                * [n.op_type + ': ' + n.output[0] for n in pending_nodes],
+                sep='\n')
+        if input_shapes is not None:
+            for input_name, shape in input_shapes.items():
+                for idx in range(len(self.out_mp_.graph.input)):
+                    if self.out_mp_.graph.input[idx].name == input_name:
+                        value_info = self.out_mp_.graph.input[idx]
+                        del self.out_mp_.graph.input[idx]
+                        self.out_mp_.graph.input.append(
+                            helper.make_tensor_value_info(
+                                value_info.name,
+                                value_info.type.tensor_type.elem_type, shape))
+        self.initializers_ = dict(
+            [(i.name, i) for i in self.out_mp_.graph.initializer])
+        self.known_vi_ = dict(
+            [(i.name, i) for i in list(self.out_mp_.graph.input)])
+        self.known_vi_.update(
+            dict([(i.name, helper.make_tensor_value_info(i.name, i.data_type,
+                                                         list(i.dims)))
+                  for i in self.out_mp_.graph.initializer]))
+
+    def _merge_symbols(self, dims):
+        if not all([type(d) == str for d in dims]):
+            if self.auto_merge_:
+                assert len(
+                    dims
+                ) == 2  # only allow symbol->int merge in binary ops for now
+                is_int = [is_literal(d) for d in dims]
+                if sum(is_int) == 1:
+                    int_dim = is_int.index(1)
+                    if self.verbose_ > 0:
+                        print('dim {} has been merged with value {}'.format(
+                            dims[1 - int_dim], dims[int_dim]))
+                    self._check_merged_dims(dims, allow_broadcast=False)
+                    return dims[int_dim]
+                else:
+                    if self.verbose_ > 0:
+                        print('dim {} has been mergd with dim {}'.format(dims[
+                            0], dims[1]))
+                    return dims[0]
+            else:
+                return None
+        if all([d == dims[0] for d in dims]):
+            return dims[0]
+        merged = [
+            self.suggested_merge_[d] if d in self.suggested_merge_ else d
+            for d in dims
+        ]
+        if all([d == merged[0] for d in merged]):
+            assert merged[0] in self.symbolic_dims_
+            return merged[0]
+        else:
+            return None
+
+    # broadcast from right to left, and merge symbolic dims if needed
+    def _broadcast_shapes(self, shape1, shape2):
+        new_shape = []
+        rank1 = len(shape1)
+        rank2 = len(shape2)
+        new_rank = max(rank1, rank2)
+        for i in range(new_rank):
+            dim1 = shape1[rank1 - 1 - i] if i < rank1 else 1
+            dim2 = shape2[rank2 - 1 - i] if i < rank2 else 1
+            if dim1 == 1 or dim1 == dim2:
+                new_dim = dim2
+            elif dim2 == 1:
+                new_dim = dim1
+            else:
+                new_dim = self._merge_symbols([dim1, dim2])
+                if not new_dim:
+                    # warning about unsupported broadcast when not auto merge
+                    # note that auto merge has the risk of incorrectly merge symbols while one of them being 1
+                    # for example, 'a' = 1, 'b' = 5 at runtime is valid broadcasting, but with auto merge 'a' == 'b'
+                    if self.auto_merge_:
+                        self._add_suggested_merge([dim1, dim2], apply=True)
+                    else:
+                        print('unsupported broadcast between ' + str(dim1) + ' '
+                              + str(dim2))
+            new_shape = [new_dim] + new_shape
+        return new_shape
+
+    def _get_shape(self, node, idx):
+        name = node.input[idx]
+        shape = []
+        if name in self.known_vi_:
+            shape = get_shape_from_type_proto(self.known_vi_[name].type)
+        elif name in self.initializers_:
+            assert name in self.initializers_
+            shape = list(self.initializers_[name].dims)
+        return shape
+
+    def _get_initializer_value(self, node, idx):
+        name = node.input[idx]
+        if name in self.initializers_:
+            value = numpy_helper.to_array(self.initializers_[name])
+            return value
+        else:
+            return False
+
+    def _get_shape_rank(self, node, idx):
+        return len(self._get_shape(node, idx))
+
+    def _get_sympy_shape(self, node, idx):
+        sympy_shape = []
+        for d in self._get_shape(node, idx):
+            if type(d) is str:
+                sympy_shape.append(self.symbolic_dims_[d] if d in
+                                   self.symbolic_dims_ else sympy.Symbol(
+                                       d, integer=True))
+            else:
+                assert None != d
+                sympy_shape.append(d)
+        return sympy_shape
+
+    def _get_value(self, node, idx):
+        name = node.input[idx]
+        assert name in self.sympy_data_ or name in self.initializers_
+        return self.sympy_data_[
+            name] if name in self.sympy_data_ else numpy_helper.to_array(
+                self.initializers_[name])
+
+    def _try_get_value(self, node, idx):
+        if idx >= len(node.input):
+            return None
+        name = node.input[idx]
+        if name in self.sympy_data_ or name in self.initializers_:
+            return self._get_value(node, idx)
+        return None
+
+    def _update_computed_dims(self, new_sympy_shape):
+        for i, new_dim in enumerate(new_sympy_shape):
+            if not is_literal(new_dim) and not type(new_dim) == str:
+                str_dim = str(new_dim)
+                if str_dim in self.suggested_merge_:
+                    new_sympy_shape[i] = self.symbolic_dims_[
+                        self.suggested_merge_[str_dim]]
+                else:
+                    # add new_dim if it's a computational expression
+                    if not str(new_dim) in self.symbolic_dims_:
+                        self.symbolic_dims_[str(new_dim)] = new_dim
+
+    def _onnx_infer_single_node(self, node):
+        # skip onnx shape inference for Scan/Loop
+        skip_infer = node.op_type in ['Scan', 'Loop']
+        if not skip_infer:
+            # run single node inference with self.known_vi_ shapes
+            # note that inference rely on initializer values is not handled
+            # as we don't copy initializer weights to tmp_graph for inference speed purpose
+            tmp_graph = helper.make_graph(
+                [node], 'tmp', [self.known_vi_[i] for i in node.input if i], [
+                    helper.make_tensor_value_info(i, onnx.TensorProto.UNDEFINED,
+                                                  None) for i in node.output
+                ])
+            self.tmp_mp_.graph.CopyFrom(tmp_graph)
+            self.tmp_mp_ = shape_inference.infer_shapes(self.tmp_mp_)
+        for i_o in range(len(node.output)):
+            o = node.output[i_o]
+            vi = self.out_mp_.graph.value_info.add()
+            if not skip_infer:
+                vi.CopyFrom(self.tmp_mp_.graph.output[i_o])
+            self.known_vi_[o] = vi
+
+    def _onnx_infer_subgraph(self, node, subgraph, use_node_input=True):
+        if self.verbose_ > 2:
+            print('Inferencing subgraph of node {} with output({}...): {}'.
+                  format(node.name, node.output[0], node.op_type))
+        # node inputs are not passed directly to the subgraph
+        # it's up to the node dispatcher to prepare subgraph input
+        # for example, with Scan/Loop, subgraph input shape would be trimmed from node input shape
+        # besides, inputs in subgraph could shadow implicit inputs
+        subgraph_inputs = set([
+            i.name for i in list(subgraph.initializer) + list(subgraph.input)
+        ])
+        subgraph_implicit_input = set([
+            name for name in self.known_vi_.keys()
+            if not name in subgraph_inputs
+        ])
+        tmp_graph = helper.make_graph(
+            list(subgraph.node), 'tmp',
+            list(subgraph.input) +
+            [self.known_vi_[i] for i in subgraph_implicit_input], [
+                helper.make_tensor_value_info(i.name,
+                                              onnx.TensorProto.UNDEFINED, None)
+                for i in subgraph.output
+            ])
+        tmp_graph.initializer.extend([
+            i for i in self.out_mp_.graph.initializer
+            if i.name in subgraph_implicit_input
+        ])
+        tmp_graph.initializer.extend(subgraph.initializer)
+        self.tmp_mp_.graph.CopyFrom(tmp_graph)
+
+        symbolic_shape_inference = SymbolicShapeInference(
+            self.int_max_, self.auto_merge_, self.guess_output_rank_,
+            self.verbose_)
+        all_shapes_inferred = False
+        symbolic_shape_inference._preprocess(self.tmp_mp_)
+        symbolic_shape_inference.suggested_merge_ = self.suggested_merge_.copy()
+        while symbolic_shape_inference.run_:
+            all_shapes_inferred = symbolic_shape_inference._infer_impl(
+                self.tmp_mp_, self.sympy_data_.copy())
+        symbolic_shape_inference._update_output_from_vi()
+        if use_node_input:
+            # if subgraph uses node input, it needs to update to merged dims
+            subgraph.ClearField('input')
+            subgraph.input.extend(
+                symbolic_shape_inference.out_mp_.graph.input[:len(node.input)])
+        subgraph.ClearField('output')
+        subgraph.output.extend(symbolic_shape_inference.out_mp_.graph.output)
+        subgraph.ClearField('value_info')
+        subgraph.value_info.extend(
+            symbolic_shape_inference.out_mp_.graph.value_info)
+        subgraph.ClearField('node')
+        subgraph.node.extend(symbolic_shape_inference.out_mp_.graph.node)
+        # for new symbolic dims from subgraph output, add to main graph symbolic dims
+        subgraph_shapes = [
+            get_shape_from_type_proto(o.type)
+            for o in symbolic_shape_inference.out_mp_.graph.output
+        ]
+        subgraph_new_symbolic_dims = set([
+            d for s in subgraph_shapes
+            if s for d in s if type(d) == str and not d in self.symbolic_dims_
+        ])
+        new_dims = {}
+        for d in subgraph_new_symbolic_dims:
+            assert d in symbolic_shape_inference.symbolic_dims_
+            new_dims[d] = symbolic_shape_inference.symbolic_dims_[d]
+        self.symbolic_dims_.update(new_dims)
+        return symbolic_shape_inference
+
+    def _get_int_values(self, node, broadcast=False):
+        values = [self._try_get_value(node, i) for i in range(len(node.input))]
+        if all([v is not None for v in values]):
+            # some shape compute is in floating point, cast to int for sympy
+            for i, v in enumerate(values):
+                if type(v) != np.ndarray:
+                    continue
+                if len(v.shape) > 1:
+                    new_v = None  # ignore value for rank > 1
+                elif len(v.shape) == 0:
+                    new_v = int(np.asscalar(v))
+                else:
+                    assert len(v.shape) == 1
+                    new_v = [int(vv) for vv in v]
+                values[i] = new_v
+        values_len = [len(v) if type(v) == list else 0 for v in values]
+        max_len = max(values_len)
+        if max_len >= 1 and broadcast:
+            # broadcast
+            for i, v in enumerate(values):
+                if v is None:
+                    continue  # don't broadcast if value is unknown
+                if type(v) == list:
+                    if len(v) < max_len:
+                        values[i] = v * max_len
+                    else:
+                        assert len(v) == max_len
+                else:
+                    values[i] = [v] * max_len
+        return values
+
+    def _compute_on_sympy_data(self, node, op_func):
+        assert len(node.output) == 1
+        values = self._get_int_values(node, broadcast=True)
+        if all([v is not None for v in values]):
+            new_shape = []
+            is_list = [type(v) == list for v in values]
+            as_list = any(is_list)
+            if as_list:
+                data = [op_func(vs) for vs in zip(*values)]
+                self.sympy_data_[node.output[0]] = data
+                new_shape = np.array(data).shape
+            else:
+                data = op_func(values)
+                self.sympy_data_[node.output[0]] = data
+                new_shape = np.array(data).shape
+            vi = self.known_vi_[node.output[0]]
+            #print(node.output[0])
+            #print(new_shape)
+            #vi.CopyFrom(helper.make_tensor_value_info(node.output[0], self.known_vi_[node.input[0]].type.tensor_type.elem_type, list(new_shape)))
+
+    def _pass_on_sympy_data(self, node):
+        assert len(node.input) == 1 or node.op_type == 'Reshape'
+        self._compute_on_sympy_data(node, lambda x: x[0])
+
+    def _pass_on_shape_and_type(self, node):
+        vi = self.known_vi_[node.output[0]]
+        vi.CopyFrom(
+            helper.make_tensor_value_info(node.output[0], self.known_vi_[
+                node.input[0]].type.tensor_type.elem_type,
+                                          self._get_shape(node, 0)))
+
+    def _new_symbolic_dim(self, prefix, dim):
+        new_dim = '{}_d{}'.format(prefix, dim)
+        if new_dim in self.suggested_merge_:
+            v = self.suggested_merge_[new_dim]
+            new_dim = sympy.Integer(int(v)) if is_literal(v) else v
+        else:
+            self.symbolic_dims_[new_dim] = sympy.Symbol(new_dim, integer=True)
+        return new_dim
+
+    def _new_symbolic_dim_from_output(self, node, out_idx=0, dim=0):
+        return self._new_symbolic_dim('{}{}_o{}_'.format(
+            node.op_type, list(self.out_mp_.graph.node).index(node), out_idx),
+                                      dim)
+
+    def _new_symbolic_shape(self, rank, node, out_idx=0):
+        return [
+            self._new_symbolic_dim_from_output(node, out_idx, i)
+            for i in range(rank)
+        ]
+
+    def _compute_conv_pool_shape(self, node):
+        sympy_shape = self._get_sympy_shape(node, 0)
+        if len(node.input) > 1:
+            W_shape = self._get_sympy_shape(node, 1)
+            rank = len(W_shape) - 2  # number of spatial axes
+            kernel_shape = W_shape[-rank:]
+            sympy_shape[1] = W_shape[0]
+        else:
+            W_shape = None
+            kernel_shape = get_attribute(node, 'kernel_shape')
+            rank = len(kernel_shape)
+
+        assert len(sympy_shape) == rank + 2
+
+        # only need to symbolic shape inference if input has symbolic dims in spatial axes
+        is_symbolic_dims = [not is_literal(i) for i in sympy_shape[-rank:]]
+
+        if not any(is_symbolic_dims):
+            shape = get_shape_from_type_proto(self.known_vi_[node.output[0]]
+                                              .type)
+            if len(shape) > 0:
+                assert len(sympy_shape) == len(shape)
+                sympy_shape[-rank:] = [sympy.Integer(d) for d in shape[-rank:]]
+                return sympy_shape
+
+        dilations = get_attribute(node, 'dilations', [1] * rank)
+        strides = get_attribute(node, 'strides', [1] * rank)
+        effective_kernel_shape = [(k - 1) * d + 1
+                                  for k, d in zip(kernel_shape, dilations)]
+        pads = get_attribute(node, 'pads')
+        if pads is None:
+            pads = [0] * (2 * rank)
+            auto_pad = get_attribute(node, 'auto_pad',
+                                     b'NOTSET').decode('utf-8')
+            if auto_pad != 'VALID' and auto_pad != 'NOTSET':
+                try:
+                    residual = [
+                        sympy.Mod(d, s)
+                        for d, s in zip(sympy_shape[-rank:], strides)
+                    ]
+                    total_pads = [
+                        max(0, (k - s) if r == 0 else (k - r))
+                        for k, s, r in zip(effective_kernel_shape, strides,
+                                           residual)
+                    ]
+                except TypeError:  # sympy may throw TypeError: cannot determine truth value of Relational
+                    total_pads = [
+                        max(0, (k - s))
+                        for k, s in zip(effective_kernel_shape, strides)
+                    ]  # assuming no residual if sympy throws error
+            elif auto_pad == 'VALID':
+                total_pads = []
+            else:
+                total_pads = [0] * rank
+        else:
+            assert len(pads) == 2 * rank
+            total_pads = [p1 + p2 for p1, p2 in zip(pads[:rank], pads[rank:])]
+
+        ceil_mode = get_attribute(node, 'ceil_mode', 0)
+        for i in range(rank):
+            effective_input_size = sympy_shape[-rank + i]
+            if len(total_pads) > 0:
+                effective_input_size = effective_input_size + total_pads[i]
+            if ceil_mode:
+                strided_kernel_positions = sympy.ceiling(
+                    (effective_input_size - effective_kernel_shape[i]) /
+                    strides[i])
+            else:
+                strided_kernel_positions = (
+                    effective_input_size - effective_kernel_shape[i]
+                ) // strides[i]
+            sympy_shape[-rank + i] = strided_kernel_positions + 1
+        return sympy_shape
+
+    def _check_merged_dims(self, dims, allow_broadcast=True):
+        if allow_broadcast:
+            dims = [d for d in dims if not (is_literal(d) and int(d) <= 1)]
+        if not all([d == dims[0] for d in dims]):
+            self._add_suggested_merge(dims, apply=True)
+
+    def _compute_matmul_shape(self, node, output_dtype=None):
+        lhs_shape = self._get_shape(node, 0)
+        rhs_shape = self._get_shape(node, 1)
+        lhs_rank = len(lhs_shape)
+        rhs_rank = len(rhs_shape)
+        lhs_reduce_dim = 0
+        rhs_reduce_dim = 0
+        assert lhs_rank > 0 and rhs_rank > 0
+        if lhs_rank == 1 and rhs_rank == 1:
+            new_shape = []
+        elif lhs_rank == 1:
+            rhs_reduce_dim = -2
+            new_shape = rhs_shape[:rhs_reduce_dim] + [rhs_shape[-1]]
+        elif rhs_rank == 1:
+            lhs_reduce_dim = -1
+            new_shape = lhs_shape[:lhs_reduce_dim]
+        else:
+            lhs_reduce_dim = -1
+            rhs_reduce_dim = -2
+            new_shape = self._broadcast_shapes(
+                lhs_shape[:-2], rhs_shape[:-2]) + [lhs_shape[-2]
+                                                   ] + [rhs_shape[-1]]
+        # merge reduce dim
+        self._check_merged_dims(
+            [lhs_shape[lhs_reduce_dim], rhs_shape[rhs_reduce_dim]],
+            allow_broadcast=False)
+        if output_dtype is None:
+            # infer output_dtype from input type when not specified
+            output_dtype = self.known_vi_[node.input[
+                0]].type.tensor_type.elem_type
+        vi = self.known_vi_[node.output[0]]
+        vi.CopyFrom(
+            helper.make_tensor_value_info(node.output[0], output_dtype,
+                                          new_shape))
+
+    def _infer_ArrayFeatureExtractor(self, node):
+        data_shape = self._get_shape(node, 0)
+        indices_shape = self._get_shape(node, 1)
+        vi = self.known_vi_[node.output[0]]
+        vi.CopyFrom(
+            helper.make_tensor_value_info(node.output[0], self.known_vi_[
+                node.input[0]].type.tensor_type.elem_type, data_shape[:-1] +
+                                          indices_shape))
+
+    def _infer_symbolic_compute_ops(self, node):
+        funcs = {
+            'Add': lambda l: l[0] + l[1],
+            'Div': lambda l: l[0] // l[1],  # integer div in sympy
+            'Equal': lambda l: l[0] == l[1],
+            'Max':
+            lambda l: l[1] if is_literal(l[0]) and int(l[0]) < -self.int_max_ else (l[0] if is_literal(l[1]) and int(l[1]) < -self.int_max_ else sympy.Max(l[0], l[1])),
+            'Min':
+            lambda l: l[1] if is_literal(l[0]) and int(l[0]) > self.int_max_ else (l[0] if is_literal(l[1]) and int(l[1]) > self.int_max_ else sympy.Min(l[0], l[1])),
+            'Mul': lambda l: l[0] * l[1],
+            'Sub': lambda l: l[0] - l[1],
+            'Where': lambda l: l[1] if l[0] else l[2]
+        }
+        assert node.op_type in funcs
+        self._compute_on_sympy_data(node, funcs[node.op_type])
+
+    def _infer_Cast(self, node):
+        self._pass_on_sympy_data(node)
+
+    def _infer_CategoryMapper(self, node):
+        input_type = self.known_vi_[node.input[0]].type.tensor_type.elem_type
+        if input_type == onnx.TensorProto.STRING:
+            output_type = onnx.TensorProto.INT64
+        else:
+            output_type = onnx.TensorProto.STRING
+        vi = self.known_vi_[node.output[0]]
+        vi.CopyFrom(
+            helper.make_tensor_value_info(node.output[0], output_type,
+                                          self._get_shape(node, 0)))
+
+    def _infer_Transpose(self, node):
+        input_shape = self._get_shape(node, 0)
+        perm = get_attribute(node, 'perm')
+        output_shape = np.array(input_shape)[perm].tolist()
+        vi = self.known_vi_[node.output[0]]
+        vi.CopyFrom(
+            helper.make_tensor_value_info(node.output[0], self.known_vi_[
+                node.input[0]].type.tensor_type.elem_type, output_shape))
+
+    def _infer_Compress(self, node):
+        input_shape = self._get_shape(node, 0)
+        # create a new symbolic dimension for Compress output
+        compress_len = self._new_symbolic_dim_from_output(node)
+        axis = get_attribute(node, 'axis')
+        if axis == None:
+            # when axis is not specified, input is flattened before compress so output is 1D
+            output_shape = [compress_len]
+        else:
+            output_shape = input_shape
+            output_shape[handle_negative_axis(axis, len(
+                input_shape))] = compress_len
+        vi = self.known_vi_[node.output[0]]
+        vi.CopyFrom(
+            helper.make_tensor_value_info(node.output[0], self.known_vi_[
+                node.input[0]].type.tensor_type.elem_type, output_shape))
+
+    def _infer_Concat(self, node):
+        if any([i in self.sympy_data_ for i in node.input]):
+            values = self._get_int_values(node)
+            if all([v is not None for v in values]):
+                assert 0 == get_attribute(node, 'axis')
+                self.sympy_data_[node.output[0]] = []
+                for i in range(len(node.input)):
+                    value = values[i]
+                    if type(value) == list:
+                        self.sympy_data_[node.output[0]].extend(value)
+                    else:
+                        self.sympy_data_[node.output[0]].append(value)
+
+        sympy_shape = self._get_sympy_shape(node, 0)
+        axis = handle_negative_axis(
+            get_attribute(node, 'axis'), len(sympy_shape))
+        for i_idx in range(1, len(node.input)):
+            input_shape = self._get_sympy_shape(node, i_idx)
+            if input_shape:
+                sympy_shape[axis] = sympy_shape[axis] + input_shape[axis]
+        self._update_computed_dims(sympy_shape)
+        # merge symbolic dims for non-concat axes
+        for d in range(len(sympy_shape)):
+            if d == axis:
+                continue
+            dims = [
+                self._get_shape(node, i_idx)[d]
+                for i_idx in range(len(node.input))
+                if self._get_shape(node, i_idx)
+            ]
+            if all([d == dims[0] for d in dims]):
+                continue
+            merged = self._merge_symbols(dims)
+            if type(merged) == str:
+                sympy_shape[d] = self.symbolic_dims_[merged] if merged else None
+            else:
+                sympy_shape[d] = merged
+        vi = self.known_vi_[node.output[0]]
+        vi.CopyFrom(
+            helper.make_tensor_value_info(
+                node.output[0], self.known_vi_[node.input[0]].type.tensor_type.
+                elem_type, get_shape_from_sympy_shape(sympy_shape)))
+
+    def _infer_Conv(self, node):
+        sympy_shape = self._compute_conv_pool_shape(node)
+        self._update_computed_dims(sympy_shape)
+        vi = self.known_vi_[node.output[0]]
+        vi.CopyFrom(
+            helper.make_tensor_value_info(
+                node.output[0], vi.type.tensor_type.elem_type,
+                get_shape_from_sympy_shape(sympy_shape)))
+
+    def _infer_ConstantOfShape(self, node):
+        sympy_shape = self._get_int_values(node)[0]
+        vi = self.known_vi_[node.output[0]]
+        if sympy_shape is not None:
+            if type(sympy_shape) != list:
+                sympy_shape = [sympy_shape]
+            self._update_computed_dims(sympy_shape)
+            # update sympy data if output type is int, and shape is known
+            if vi.type.tensor_type.elem_type == onnx.TensorProto.INT64 and all(
+                [is_literal(x) for x in sympy_shape]):
+                self.sympy_data_[node.output[0]] = np.ones(
+                    [int(x) for x in sympy_shape],
+                    dtype=np.int64) * numpy_helper.to_array(
+                        get_attribute(node, 'value', 0))
+        else:
+            # create new dynamic shape
+            sympy_shape = self._new_symbolic_shape(
+                self._get_shape_rank(node, 0), node)
+
+        vi.CopyFrom(
+            helper.make_tensor_value_info(
+                node.output[0], vi.type.tensor_type.elem_type,
+                get_shape_from_sympy_shape(sympy_shape)))
+
+    def _infer_Expand(self, node):
+        expand_to_shape = self._try_get_value(node, 1)
+        if expand_to_shape is not None:
+            # new_shape's dim can come from shape value
+            self._update_computed_dims(expand_to_shape)
+            shape = self._get_shape(node, 0)
+            new_shape = self._broadcast_shapes(
+                shape, get_shape_from_sympy_shape(expand_to_shape))
+            vi = self.known_vi_[node.output[0]]
+            vi.CopyFrom(
+                helper.make_tensor_value_info(node.output[0], self.known_vi_[
+                    node.input[0]].type.tensor_type.elem_type, new_shape))
+
+    def _infer_Gather(self, node):
+        data_shape = self._get_shape(node, 0)
+        axis = handle_negative_axis(
+            get_attribute(node, 'axis', 0), len(data_shape))
+        indices_shape = self._get_shape(node, 1)
+        #if indices_shape == []:
+        #    value = self._get_initializer_value(node, 1)
+        #    if isinstance(value.tolist(), int):
+        #        indices_shape = [1]
+        new_shape = data_shape[:axis] + indices_shape + data_shape[axis + 1:]
+        #print(new_shape)
+        vi = self.known_vi_[node.output[0]]
+        vi.CopyFrom(
+            helper.make_tensor_value_info(node.output[
+                0], vi.type.tensor_type.elem_type, new_shape))
+        if node.input[0] in self.sympy_data_:
+            assert 0 == get_attribute(node, 'axis',
+                                      0)  # only handle 1D sympy compute
+            idx = self._get_value(node, 1)
+            data = self.sympy_data_[node.input[0]]
+            if type(data) == list:
+                if type(idx) == np.ndarray and len(idx.shape) == 1:
+                    self.sympy_data_[node.output[0]] = [
+                        data[int(i)] for i in idx
+                    ]
+                else:
+                    self.sympy_data_[node.output[0]] = data[int(idx)]
+            else:
+                assert idx == 0
+                self.sympy_data_[node.output[0]] = data
+
+    def _infer_GatherElements(self, node):
+        indices_shape = self._get_shape(node, 1)
+        vi = self.known_vi_[node.output[0]]
+        vi.CopyFrom(
+            helper.make_tensor_value_info(node.output[0], self.known_vi_[
+                node.input[0]].type.tensor_type.elem_type, indices_shape))
+
+    def _infer_GatherND(self, node):
+        data_shape = self._get_shape(node, 0)
+        data_rank = len(data_shape)
+        indices_shape = self._get_shape(node, 1)
+        indices_rank = len(indices_shape)
+        last_index_dimension = indices_shape[-1]
+        assert is_literal(
+            last_index_dimension) and last_index_dimension <= data_rank
+        new_shape = indices_shape[:-1] + data_shape[last_index_dimension:]
+        vi = self.known_vi_[node.output[0]]
+        vi.CopyFrom(
+            helper.make_tensor_value_info(node.output[0], self.known_vi_[
+                node.input[0]].type.tensor_type.elem_type, new_shape))
+
+    def _infer_If(self, node):
+        # special case for constant condition, in case there are mismatching shape from the non-executed branch
+        subgraphs = [
+            get_attribute(node, 'then_branch'),
+            get_attribute(node, 'else_branch')
+        ]
+        cond = self._try_get_value(node, 0)
+        if cond is not None:
+            if cond > 0:
+                subgraphs[1].CopyFrom(subgraphs[0])
+            else:
+                subgraphs[0].CopyFrom(subgraphs[1])
+
+        for i_sub, subgraph in enumerate(subgraphs):
+            subgraph_infer = self._onnx_infer_subgraph(
+                node, subgraph, use_node_input=False)
+            for i_out in range(len(node.output)):
+                vi = self.known_vi_[node.output[i_out]]
+                if i_sub == 0:
+                    vi.CopyFrom(subgraph.output[i_out])
+                    vi.name = node.output[i_out]
+                else:
+                    assert all([
+                        d1 == d2
+                        for d1, d2 in zip(vi.type.tensor_type.shape.dim,
+                                          subgraph.output[
+                                              i_out].type.tensor_type.shape.dim)
+                    ])
+                # pass on sympy data from subgraph, if cond is constant
+                if cond is not None and i_sub == (0 if cond > 0 else 1):
+                    if subgraph.output[
+                            i_out].name in subgraph_infer.sympy_data_:
+                        self.sympy_data_[vi.name] = subgraph_infer.sympy_data_[
+                            subgraph.output[i_out].name]
+
+    def _infer_Loop(self, node):
+        subgraph = get_attribute(node, 'body')
+        assert len(subgraph.input) == len(node.input)
+        for i, si in enumerate(subgraph.input):
+            subgraph_name = si.name
+            si.CopyFrom(self.known_vi_[node.input[i]])
+            si.name = subgraph_name
+        self._onnx_infer_subgraph(node, subgraph)
+        # create a new symbolic dimension for iteration dependent dimension
+        loop_iter_dim = self._new_symbolic_dim_from_output(node)
+        num_loop_carried = len(node.input) - 2
+        for i in range(len(node.output)):
+            vi = self.known_vi_[node.output[i]]
+            vi.CopyFrom(
+                subgraph.output[i + 1]
+            )  # first subgraph output is condition, not in node output
+            if i >= num_loop_carried:
+                subgraph_vi_dim = subgraph.output[i +
+                                                  1].type.tensor_type.shape.dim
+                vi.type.tensor_type.shape.ClearField('dim')
+                vi_dim = vi.type.tensor_type.shape.dim
+                vi_dim.add().dim_param = loop_iter_dim
+                vi_dim.extend(list(subgraph_vi_dim))
+            vi.name = node.output[i]
+
+    def _infer_MatMul(self, node):
+        self._compute_matmul_shape(node)
+
+    def _infer_MatMulInteger(self, node):
+        self._compute_matmul_shape(node, onnx.TensorProto.INT32)
+
+    def _infer_NonMaxSuppression(self, node):
+        selected = self._new_symbolic_dim_from_output(node)
+        vi = self.known_vi_[node.output[0]]
+        vi.CopyFrom(
+            helper.make_tensor_value_info(node.output[
+                0], onnx.TensorProto.INT64, [selected, 3]))
+
+    def _infer_NonZero(self, node):
+        input_rank = self._get_shape_rank(node, 0)
+        # create a new symbolic dimension for NonZero output
+        nz_len = self._new_symbolic_dim_from_output(node, 0, 1)
+        vi = self.known_vi_[node.output[0]]
+        vi.CopyFrom(
+            helper.make_tensor_value_info(node.output[
+                0], vi.type.tensor_type.elem_type, [input_rank, nz_len]))
+
+    def _infer_OneHot(self, node):
+        shape = self._get_shape(node, 0)
+        axis = get_attribute(node, 'axis', -1)
+        axis = handle_negative_axis(axis, len(shape) + 1)
+        new_shape = shape[:axis] + [self._new_symbolic_dim_from_output(node)
+                                    ] + shape[axis:]
+        vi = self.known_vi_[node.output[0]]
+        vi.CopyFrom(
+            helper.make_tensor_value_info(node.output[0], self.known_vi_[
+                node.input[2]].type.tensor_type.elem_type, new_shape))
+
+    def _infer_Pad(self, node):
+        if get_opset(self.out_mp_) <= 10:
+            pads = get_attribute(node, 'pads')
+        else:
+            pads = self._try_get_value(node, 1)
+
+        vi = self.known_vi_[node.output[0]]
+        output_shape = get_shape_from_type_proto(vi.type)
+        if len(output_shape) == 0 or None in output_shape:
+            sympy_shape = self._get_sympy_shape(node, 0)
+            rank = len(sympy_shape)
+            if pads is not None:
+                assert len(pads) == 2 * rank
+                new_sympy_shape = [
+                    d + pad_up + pad_down
+                    for d, pad_up, pad_down in zip(sympy_shape, pads[:rank],
+                                                   pads[rank:])
+                ]
+                self._update_computed_dims(new_sympy_shape)
+            else:
+                # dynamic pads, create new symbolic dimensions
+                new_sympy_shape = self._new_symbolic_shape(rank, node)
+            output_tp = self.known_vi_[node.input[0]].type.tensor_type.elem_type
+            vi.CopyFrom(
+                helper.make_tensor_value_info(node.output[
+                    0], output_tp, get_shape_from_sympy_shape(new_sympy_shape)))
+
+    def _infer_Pool(self, node):
+        sympy_shape = self._compute_conv_pool_shape(node)
+        self._update_computed_dims(sympy_shape)
+        for o in node.output:
+            if not o:
+                continue
+            vi = self.known_vi_[o]
+            vi.CopyFrom(
+                helper.make_tensor_value_info(o, vi.type.tensor_type.elem_type,
+                                              get_shape_from_sympy_shape(
+                                                  sympy_shape)))
+
+    def _infer_Range(self, node):
+        vi = self.known_vi_[node.output[0]]
+        input_data = self._get_int_values(node)
+        if all([i is not None for i in input_data]):
+            start = as_scalar(input_data[0])
+            limit = as_scalar(input_data[1])
+            delta = as_scalar(input_data[2])
+            new_sympy_shape = [
+                sympy.Max(sympy.ceiling((limit - start) / delta), 0)
+            ]
+        else:
+            new_dim = self._new_symbolic_dim_from_output(node)
+            new_sympy_shape = [self.symbolic_dims_[new_dim]]
+        self._update_computed_dims(new_sympy_shape)
+        vi.CopyFrom(
+            helper.make_tensor_value_info(
+                node.output[0], self.known_vi_[node.input[0]].type.tensor_type.
+                elem_type, get_shape_from_sympy_shape(new_sympy_shape)))
+
+    def _infer_ReduceProd(self, node):
+        axes = get_attribute(node, 'axes')
+        keep_dims = get_attribute(node, 'keepdims')
+        if keep_dims == 0 and axes == [0]:
+            data = self._get_int_values(node)[0]
+            if data is not None:
+                self.sympy_data_[node.output[0]] = sympy_reduce_product(data)
+
+    def _infer_Reshape(self, node):
+        shape_value = self._try_get_value(node, 1)
+        vi = self.known_vi_[node.output[0]]
+        if shape_value is None:
+            shape_shape = self._get_shape(node, 1)
+            assert len(shape_shape) == 1
+            shape_rank = shape_shape[0]
+            assert is_literal(shape_rank)
+            vi.CopyFrom(
+                helper.make_tensor_value_info(
+                    node.output[0], vi.type.tensor_type.elem_type,
+                    get_shape_from_sympy_shape(
+                        self._new_symbolic_shape(shape_rank, node))))
+        else:
+            input_shape = self._get_shape(node, 0)
+            input_sympy_shape = self._get_sympy_shape(node, 0)
+            total = int(1)
+            for d in input_sympy_shape:
+                total = total * d
+            new_sympy_shape = []
+            deferred_dim_idx = -1
+            non_deferred_size = int(1)
+            for i, d in enumerate(shape_value):
+                if type(d) == sympy.Symbol:
+                    new_sympy_shape.append(d)
+                elif d == 0:
+                    new_sympy_shape.append(input_sympy_shape[i])
+                    non_deferred_size = non_deferred_size * input_sympy_shape[i]
+                else:
+                    new_sympy_shape.append(d)
+                if d == -1:
+                    deferred_dim_idx = i
+                elif d != 0:
+                    non_deferred_size = non_deferred_size * d
+
+            assert new_sympy_shape.count(-1) < 2
+            if -1 in new_sympy_shape:
+                new_dim = total // non_deferred_size
+                new_sympy_shape[deferred_dim_idx] = new_dim
+                self._update_computed_dims(new_sympy_shape)
+
+            vi.CopyFrom(
+                helper.make_tensor_value_info(
+                    node.output[0], vi.type.tensor_type.elem_type,
+                    get_shape_from_sympy_shape(new_sympy_shape)))
+
+        self._pass_on_sympy_data(node)
+
+    def _infer_Resize(self, node):
+        vi = self.known_vi_[node.output[0]]
+        input_sympy_shape = self._get_sympy_shape(node, 0)
+        if get_opset(self.out_mp_) <= 10:
+            scales = self._try_get_value(node, 1)
+            if scales is not None:
+                new_sympy_shape = [
+                    sympy.simplify(sympy.floor(d * s))
+                    for d, s in zip(input_sympy_shape, scales)
+                ]
+                self._update_computed_dims(new_sympy_shape)
+                vi.CopyFrom(
+                    helper.make_tensor_value_info(
+                        node.output[0], self.known_vi_[node.input[
+                            0]].type.tensor_type.elem_type,
+                        get_shape_from_sympy_shape(new_sympy_shape)))
+        else:
+            roi = self._try_get_value(node, 1)
+            scales = self._try_get_value(node, 2)
+            sizes = self._try_get_value(node, 3)
+            if sizes is not None:
+                new_sympy_shape = [
+                    sympy.simplify(sympy.floor(s)) for s in sizes
+                ]
+                self._update_computed_dims(new_sympy_shape)
+            elif roi is not None and scales is not None:
+                rank = len(scales)
+                assert len(roi) == 2 * rank
+                roi_start = list(roi)[:rank]
+                roi_end = list(roi)[rank:]
+                scales = list(scales)
+                new_sympy_shape = [
+                    sympy.simplify(sympy.floor(d * (end - start) * scale))
+                    for d, start, end, scale in zip(input_sympy_shape,
+                                                    roi_start, roi_end, scales)
+                ]
+                self._update_computed_dims(new_sympy_shape)
+            else:
+                new_sympy_shape = self._new_symbolic_shape(
+                    self._get_shape_rank(node, 0), node)
+
+            vi.CopyFrom(
+                helper.make_tensor_value_info(node.output[0], self.known_vi_[
+                    node.input[0]].type.tensor_type.elem_type,
+                                              get_shape_from_sympy_shape(
+                                                  new_sympy_shape)))
+
+    def _infer_Scan(self, node):
+        subgraph = get_attribute(node, 'body')
+        num_scan_inputs = get_attribute(node, 'num_scan_inputs')
+        scan_input_axes = get_attribute(node, 'scan_input_axes',
+                                        [0] * num_scan_inputs)
+        num_scan_states = len(node.input) - num_scan_inputs
+        scan_input_axes = [
+            handle_negative_axis(
+                ax, self._get_shape_rank(node, i + num_scan_states))
+            for i, ax in enumerate(scan_input_axes)
+        ]
+        # We may have cases where the subgraph has optionial inputs that appear in both subgraph's input and initializer,
+        # but not in the node's input. In such cases, the input model might be invalid, but let's skip those optional inputs.
+        assert len(subgraph.input) >= len(node.input)
+        subgraph_inputs = subgraph.input[:len(node.input)]
+        for i, si in enumerate(subgraph_inputs):
+            subgraph_name = si.name
+            si.CopyFrom(self.known_vi_[node.input[i]])
+            if i >= num_scan_states:
+                scan_input_dim = si.type.tensor_type.shape.dim
+                scan_input_dim.remove(scan_input_dim[scan_input_axes[
+                    i - num_scan_states]])
+            si.name = subgraph_name
+        self._onnx_infer_subgraph(node, subgraph)
+        num_scan_outputs = len(node.output) - num_scan_states
+        scan_output_axes = get_attribute(node, 'scan_output_axes',
+                                         [0] * num_scan_outputs)
+        scan_input_dim = get_shape_from_type_proto(self.known_vi_[node.input[
+            -1]].type)[scan_input_axes[-1]]
+        for i, o in enumerate(node.output):
+            vi = self.known_vi_[o]
+            if i >= num_scan_states:
+                shape = get_shape_from_type_proto(subgraph.output[i].type)
+                new_dim = handle_negative_axis(
+                    scan_output_axes[i - num_scan_states], len(shape) + 1)
+                shape = shape[:new_dim] + [scan_input_dim] + shape[new_dim:]
+                vi.CopyFrom(
+                    helper.make_tensor_value_info(o, subgraph.output[
+                        i].type.tensor_type.elem_type, shape))
+            else:
+                vi.CopyFrom(subgraph.output[i])
+            vi.name = o
+
+    def _infer_ScatterElements(self, node):
+        data_shape = self._get_shape(node, 0)
+        vi = self.known_vi_[node.output[0]]
+        vi.CopyFrom(
+            helper.make_tensor_value_info(node.output[0], self.known_vi_[
+                node.input[0]].type.tensor_type.elem_type, data_shape))
+
+    def _infer_Shape(self, node):
+        self.sympy_data_[node.output[0]] = self._get_sympy_shape(node, 0)
+
+    def _infer_Size(self, node):
+        sympy_shape = self._get_sympy_shape(node, 0)
+        self.sympy_data_[node.output[0]] = sympy_reduce_product(sympy_shape)
+        self.known_vi_[node.output[0]].CopyFrom(
+            helper.make_tensor_value_info(node.output[0],
+                                          onnx.TensorProto.INT64, []))
+
+    def _infer_Slice(self, node):
+        if get_opset(self.out_mp_) <= 9:
+            axes = get_attribute(node, 'axes')
+            starts = get_attribute(node, 'starts')
+            ends = get_attribute(node, 'ends')
+            steps = [1] * len(axes)
+        else:
+            starts = as_list(self._try_get_value(node, 1), keep_none=True)
+            ends = as_list(self._try_get_value(node, 2), keep_none=True)
+            axes = self._try_get_value(node, 3)
+            steps = self._try_get_value(node, 4)
+            if axes is None and not (starts is None and ends is None):
+                axes = list(
+                    range(0, len(starts if starts is not None else ends)))
+            if steps is None and not (starts is None and ends is None):
+                steps = [1] * len(starts if starts is not None else ends)
+            axes = as_list(axes, keep_none=True)
+            steps = as_list(steps, keep_none=True)
+
+        new_sympy_shape = self._get_sympy_shape(node, 0)
+        if starts is None or ends is None:
+            if axes is None:
+                for i in range(len(new_sympy_shape)):
+                    new_sympy_shape[i] = self._new_symbolic_dim_from_output(
+                        node, 0, i)
+            else:
+                new_sympy_shape = get_shape_from_sympy_shape(new_sympy_shape)
+                for i in axes:
+                    new_sympy_shape[i] = self._new_symbolic_dim_from_output(
+                        node, 0, i)
+        else:
+            for i, s, e, t in zip(axes, starts, ends, steps):
+                idx = handle_negative_axis(i, len(new_sympy_shape))
+                if is_literal(e):
+                    if e >= self.int_max_:
+                        e = new_sympy_shape[i]
+                    elif e <= -self.int_max_:
+                        e = 0 if s > 0 else -1
+                    elif is_literal(new_sympy_shape[i]):
+                        if e < 0:
+                            e = e + new_sympy_shape[i]
+                        e = min(e, new_sympy_shape[i])
+                    else:
+                        if e > 0:
+                            e = sympy.Min(
+                                e, new_sympy_shape[i]
+                            ) if e > 1 else e  #special case for slicing first to make computation easier
+                        else:
+                            e = new_sympy_shape[i] + e
+                else:
+                    if is_literal(new_sympy_shape[i]):
+                        e = sympy.Min(e, new_sympy_shape[i])
+                    else:
+                        try:
+                            if e >= new_sympy_shape[i]:
+                                e = new_sympy_shape[i]
+                        except Exception:
+                            print(
+                                'Unable to determine if {} <= {}, treat as equal'
+                                .format(e, new_sympy_shape[i]))
+                            e = new_sympy_shape[i]
+
+                if is_literal(s) and int(s) < 0:
+                    s = new_sympy_shape[i] + s
+
+                new_sympy_shape[idx] = (e - s + t + (-1 if t > 0 else 1)) // t
+
+            self._update_computed_dims(new_sympy_shape)
+
+        vi = self.known_vi_[node.output[0]]
+        vi.CopyFrom(
+            helper.make_tensor_value_info(
+                node.output[0], vi.type.tensor_type.elem_type,
+                get_shape_from_sympy_shape(new_sympy_shape)))
+
+        # handle sympy_data if needed, for slice in shape computation
+        if node.input[0] in self.sympy_data_:
+            assert [0] == axes
+            assert len(starts) == 1
+            assert len(ends) == 1
+            self.sympy_data_[node.output[0]] = self.sympy_data_[node.input[0]][
+                starts[0]:ends[0]]
+
+    def _infer_Split(self, node):
+        input_sympy_shape = self._get_sympy_shape(node, 0)
+        axis = handle_negative_axis(
+            get_attribute(node, 'axis', 0), len(input_sympy_shape))
+        split = get_attribute(node, 'split')
+        if not split:
+            num_outputs = len(node.output)
+            split = [input_sympy_shape[axis] /
+                     sympy.Integer(num_outputs)] * num_outputs
+            self._update_computed_dims(split)
+        else:
+            split = [sympy.Integer(s) for s in split]
+
+        for i_o in range(len(split)):
+            vi = self.known_vi_[node.output[i_o]]
+            vi.CopyFrom(
+                helper.make_tensor_value_info(
+                    node.output[i_o], self.known_vi_[node.input[
+                        0]].type.tensor_type.elem_type,
+                    get_shape_from_sympy_shape(input_sympy_shape[:axis] + [
+                        split[i_o]
+                    ] + input_sympy_shape[axis + 1:])))
+            self.known_vi_[vi.name] = vi
+
+    def _infer_Squeeze(self, node):
+        self._pass_on_sympy_data(node)
+
+    def _infer_Tile(self, node):
+        repeats_value = self._get_value(node, 1)
+        input_sympy_shape = self._get_sympy_shape(node, 0)
+        new_sympy_shape = []
+        for i, d in enumerate(input_sympy_shape):
+            new_dim = d * repeats_value[i]
+            new_sympy_shape.append(new_dim)
+        self._update_computed_dims(new_sympy_shape)
+        vi = self.known_vi_[node.output[0]]
+        vi.CopyFrom(
+            helper.make_tensor_value_info(
+                node.output[0], vi.type.tensor_type.elem_type,
+                get_shape_from_sympy_shape(new_sympy_shape)))
+
+    def _infer_TopK(self, node):
+        rank = self._get_shape_rank(node, 0)
+        axis = handle_negative_axis(get_attribute(node, 'axis', -1), rank)
+        new_shape = self._get_shape(node, 0)
+
+        if get_opset(self.out_mp_) <= 9:
+            k = get_attribute(node, 'k')
+        else:
+            k = self._get_int_values(node)[1]
+
+        if k == None:
+            k = self._new_symbolic_dim_from_output(node)
+        else:
+            k = as_scalar(k)
+
+        if type(k) in [int, str]:
+            new_shape[axis] = k
+        else:
+            new_sympy_shape = self._get_sympy_shape(node, 0)
+            new_sympy_shape[axis] = k
+            self._update_computed_dims(
+                new_sympy_shape
+            )  # note that TopK dim could be computed in sympy_data, so need to update computed_dims when it enters shape
+            new_shape = get_shape_from_sympy_shape(new_sympy_shape)
+
+        for i_o in range(len(node.output)):
+            vi = self.known_vi_[node.output[i_o]]
+            vi.CopyFrom(
+                helper.make_tensor_value_info(node.output[
+                    i_o], vi.type.tensor_type.elem_type, new_shape))
+
+    def _infer_Unsqueeze(self, node):
+        self._pass_on_sympy_data(node)
+
+    def _infer_ZipMap(self, node):
+        map_key_type = None
+        if get_attribute(node, 'classlabels_int64s') is not None:
+            map_key_type = onnx.TensorProto.INT64
+        elif get_attribute(node, 'classlabels_strings') is not None:
+            map_key_type = onnx.TensorProto.STRING
+
+        assert map_key_type is not None
+        new_vi = onnx.ValueInfoProto()
+        new_vi.name = node.output[0]
+        new_vi.type.sequence_type.elem_type.map_type.value_type.tensor_type.elem_type = onnx.TensorProto.FLOAT
+        new_vi.type.sequence_type.elem_type.map_type.key_type = map_key_type
+        vi = self.known_vi_[node.output[0]]
+        vi.CopyFrom(new_vi)
+
+    def _infer_impl(self, in_mp, start_sympy_data={}):
+        self.sympy_data_ = start_sympy_data
+        self.out_mp_.graph.ClearField('value_info')
+        self._apply_suggested_merge(graph_input_only=True)
+        self.input_symbols_ = set()
+        for i in self.out_mp_.graph.input:
+            input_dims = i.type.tensor_type.shape.dim
+            for i_dim in range(len(input_dims)):
+                if get_dim_from_type_proto(input_dims[i_dim]) is None:
+                    # some models use None for symbolic dim in input, replace it with a string
+                    input_dims[i_dim].dim_param = self._new_symbolic_dim(i.name,
+                                                                         i_dim)
+            self.input_symbols_.update([
+                d for d in get_shape_from_type_proto(i.type) if type(d) == str
+            ])
+
+        for s in self.input_symbols_:
+            if s in self.suggested_merge_:
+                s_merge = self.suggested_merge_[s]
+                assert s_merge in self.symbolic_dims_
+                self.symbolic_dims_[s] = self.symbolic_dims_[s_merge]
+            else:
+                self.symbolic_dims_[s] = sympy.Symbol(s, integer=True)
+
+        # create a temporary ModelProto for single node inference
+        # note that we remove initializer to have faster inference
+        # for tensor ops like Reshape/Tile/Expand that read initializer, we need to do sympy computation based inference anyways
+        self.tmp_mp_ = onnx.ModelProto()
+        self.tmp_mp_.CopyFrom(self.out_mp_)
+        self.tmp_mp_.graph.ClearField('initializer')
+
+        for node in self.out_mp_.graph.node:
+            assert all([i in self.known_vi_ for i in node.input if i])
+            self._onnx_infer_single_node(node)
+            if node.op_type in self.dispatcher_:
+                self.dispatcher_[node.op_type](node)
+            if self.verbose_ > 2:
+                print(node.op_type + ': ' + node.name)
+                for i, name in enumerate(node.input):
+                    print('  Input {}: {} {}€5€5€5€5€5'.format(
+                        i, name, 'initializer'
+                        if name in self.initializers_ else ''))
+
+            # onnx automatically merge dims with value, i.e. Mul(['aaa', 'bbb'], [1000, 1]) -> [1000, 'bbb']
+            # symbolic shape inference needs to apply merge of 'aaa' -> 1000 in this case
+            if node.op_type in [
+                    'Add', 'Sub', 'Mul', 'Div', 'MatMul', 'MatMulInteger',
+                    'MatMulInteger16', 'Where', 'Sum'
+            ]:
+                vi = self.known_vi_[node.output[0]]
+                out_rank = len(get_shape_from_type_proto(vi.type))
+                in_shapes = [
+                    self._get_shape(node, i) for i in range(len(node.input))
+                ]
+                for d in range(out_rank - (2 if node.op_type in [
+                        'MatMul', 'MatMulInteger', 'MatMulInteger16'
+                ] else 0)):
+                    in_dims = [
+                        s[len(s) - out_rank + d] for s in in_shapes
+                        if len(s) + d >= out_rank
+                    ]
+                    if len(in_dims) > 1:
+                        self._check_merged_dims(in_dims, allow_broadcast=True)
+            for i_o in range(len(node.output)):
+                vi = self.known_vi_[node.output[i_o]]
+                out_type = vi.type
+                out_type_kind = out_type.WhichOneof('value')
+                # only TensorProto and SparseTensorProto have shape
+                if out_type_kind != 'tensor_type' and out_type_kind != 'sparse_tensor_type':
+                    continue
+                out_shape = get_shape_from_type_proto(vi.type)
+                out_type_undefined = out_type.tensor_type.elem_type == onnx.TensorProto.UNDEFINED
+                if self.verbose_ > 2:
+                    print('  {}: {} {}'.format(node.output[
+                        i_o], str(out_shape), vi.type.tensor_type.elem_type))
+                    if node.output[i_o] in self.sympy_data_:
+                        print('  Sympy Data: ' + str(self.sympy_data_[
+                            node.output[i_o]]))
+
+                if None in out_shape or out_type_undefined:
+                    if self.auto_merge_:
+                        if node.op_type in [
+                                'Add', 'Sub', 'Mul', 'Div', 'MatMul',
+                                'MatMulInteger', 'MatMulInteger16', 'Concat',
+                                'Where', 'Sum'
+                        ]:
+                            shapes = [
+                                self._get_shape(node, i)
+                                for i in range(len(node.input))
+                            ]
+                            if node.op_type in [
+                                    'MatMul', 'MatMulInteger', 'MatMulInteger16'
+                            ]:
+                                # only support auto merge for MatMul for dim < rank-2 when rank > 2
+                                assert len(shapes[0]) > 2 and dim_idx[0] < len(
+                                    shapes[0]) - 2
+                                assert len(shapes[1]) > 2 and dim_idx[1] < len(
+                                    shapes[1]) - 2
+                        elif node.op_type == 'Expand':
+                            # auto merge for cases like Expand([min(batch, 1), min(seq, 512)], [batch, seq])
+                            shapes = [
+                                self._get_shape(node, 0),
+                                self._get_value(node, 1)
+                            ]
+                        else:
+                            shapes = []
+
+                        if shapes:
+                            for idx in range(len(out_shape)):
+                                if out_shape[idx] is not None:
+                                    continue
+                                dim_idx = [
+                                    len(s) - len(out_shape) + idx
+                                    for s in shapes
+                                ]
+                                assert all([d >= 0 for d in dim_idx])
+                                self._add_suggested_merge([
+                                    s[i] if is_literal(s[i]) else str(s[i])
+                                    for s, i in zip(shapes, dim_idx)
+                                ])
+                            self.run_ = True
+                        else:
+                            self.run_ = False
+                    else:
+                        self.run_ = False
+
+                    # create new dynamic dims for ops not handled by symbolic shape inference
+                    if self.run_ == False and not node.op_type in self.dispatcher_:
+                        is_unknown_op = (out_type_undefined and
+                                         len(out_shape) == 0)
+                        if is_unknown_op:
+                            # unknown op to ONNX, maybe from higher opset or other domain
+                            # only guess the output rank from input 0 when using guess_output_rank option
+                            out_rank = self._get_shape_rank(
+                                node, 0) if self.guess_output_rank_ else -1
+                        else:
+                            # valid ONNX op, but not handled by symbolic shape inference, just assign dynamic shape
+                            out_rank = len(out_shape)
+
+                        if out_rank >= 0:
+                            new_shape = self._new_symbolic_shape(out_rank, node,
+                                                                 i_o)
+                            vi.CopyFrom(
+                                helper.make_tensor_value_info(
+                                    vi.name, self.known_vi_[node.input[
+                                        0]].type.tensor_type.elem_type,
+                                    get_shape_from_sympy_shape(new_shape)))
+
+                            if self.verbose_ > 0:
+                                if is_unknown_op:
+                                    print(
+                                        "Possible unknown op: {} node: {}, guessing {} shape"
+                                        .format(node.op_type, node.name,
+                                                vi.name))
+                                if self.verbose_ > 2:
+                                    print('  {}: {} {}'.format(
+                                        node.output[i_o],
+                                        str(new_shape),
+                                        vi.type.tensor_type.elem_type))
+
+                            self.run_ = True
+                            continue  # continue the inference after guess, no need to stop as no merge is needed
+
+                    if self.verbose_ > 0 or not self.auto_merge_ or out_type_undefined:
+                        print('Stopping at incomplete shape inference at ' +
+                              node.op_type + ': ' + node.name)
+                        print('node inputs:')
+                        for i in node.input:
+                            print(self.known_vi_[i])
+                        print('node outputs:')
+                        for o in node.output:
+                            print(self.known_vi_[o])
+                        if self.auto_merge_ and not out_type_undefined:
+                            print('Merging: ' + str(self.suggested_merge_))
+                    return False
+        self.run_ = False
+        return True
+
+    def _update_output_from_vi(self):
+        for output in self.out_mp_.graph.output:
+            if output.name in self.known_vi_:
+                tmp_output = self.known_vi_[output.name]
+                output.CopyFrom(tmp_output)
+
+    @staticmethod
+    def infer_shapes(in_mp,
+                     int_max=2**31 - 1,
+                     fixed_input_shape=None,
+                     auto_merge=True,
+                     guess_output_rank=False,
+                     verbose=0):
+        if get_opset(in_mp) < 7:
+            print('Only support shape inferencing models of opset 7 and above.')
+            return
+        symbolic_shape_inference = SymbolicShapeInference(
+            int_max, auto_merge, guess_output_rank, verbose)
+        all_shapes_inferred = False
+        symbolic_shape_inference._preprocess(
+            in_mp, input_shapes=fixed_input_shape)
+        try:
+            while symbolic_shape_inference.run_:
+                all_shapes_inferred = symbolic_shape_inference._infer_impl(
+                    in_mp)
+            symbolic_shape_inference._update_output_from_vi()
+            if not all_shapes_inferred:
+                print('!' * 10)
+                symbolic_shape_inference.out_mp_ = shape_inference.infer_shapes(
+                    symbolic_shape_inference.out_mp_)
+            #onnx.save(symbolic_shape_inference.out_mp_, 'tmp.onnx')
+        except:
+            print('Stopping at incomplete shape inference')
+            symbolic_shape_inference.out_mp_ = shape_inference.infer_shapes(
+                symbolic_shape_inference.out_mp_)
+        return symbolic_shape_inference.out_mp_.graph

x2paddle/decoder/paddle_decoder.py

+#   Copyright (c) 2019  PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License"
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import paddle.fluid as fluid
+
+
+class PaddleDecoder(object):
+    def __init__(self,
+                 model_dir,
+                 model_filename='__model__',
+                 params_filename=None):
+        exe = fluid.Executor(fluid.CPUPlace())
+        [self.program, feed, fetchs] = fluid.io.load_inference_model(
+            model_dir,
+            exe,
+            model_filename=model_filename,
+            params_filename=params_filename)

x2paddle/decoder/tf_decoder.py

@@ -48,7 +48,10 @@ class TFGraphNode(GraphNode):
 
     @property
     def out_shapes(self):
-        values = self.layer.attr["_output_shapes"].list.shape
+        if self.layer_type == "OneShotIterator":
+            values = self.layer.attr["output_shapes"].list.shape
+        else:
+            values = self.layer.attr["_output_shapes"].list.shape
         out_shapes = list()
         for value in values:
             shape = [dim.size for dim in value.dim]
@@ -62,6 +65,8 @@ class TFGraphNode(GraphNode):
             dtype = self.layer.attr[k].type
             if dtype > 0:
                 break
+        if dtype == 0:
+            dtype = self.layer.attr['output_types'].list.type[0]
         if dtype not in self.dtype_map:
             raise Exception("Dtype[{}] not in dtype_map".format(dtype))
         return self.dtype_map[dtype]
@@ -115,13 +120,13 @@ class TFGraph(Graph):
     def build(self):
         for layer in self.model.node:
             self.node_map[layer.name.replace('/', '_').replace(
-                '-', '_')] = TFGraphNode(layer, data_format=self.tf_data_format)
+                '-', '_')] = TFGraphNode(
+                    layer, data_format=self.tf_data_format)
 
         for layer_name, node in self.node_map.items():
             for in_node in node.layer.input:
-                in_node = in_node.replace('/',
-                                          '_').replace('-',
-                                                       '_').replace('^', '')
+                in_node = in_node.replace('/', '_').replace('-', '_').replace(
+                    '^', '')
                 if in_node not in self.node_map:
                     if in_node.strip().split(':')[0] in self.node_map:
                         self.connect(in_node.strip().split(':')[0], layer_name)
@@ -226,7 +231,7 @@ class TFGraph(Graph):
     def _remove_identity_node(self):
         identity_ops = [
             'Identity', 'StopGradient', 'Switch', 'Merge',
-            'PlaceholderWithDefault'
+            'PlaceholderWithDefault', 'IteratorGetNext'
         ]
         identity_node = list()
         for node_name, node in self.node_map.items():
@@ -317,7 +322,7 @@ class TFDecoder(object):
         graph_def = cp.deepcopy(graph_def)
         input_map = dict()
         for layer in graph_def.node:
-            if layer.op != "Placeholder":
+            if layer.op != "Placeholder" and layer.op != "OneShotIterator":
                 continue
             graph_node = TFGraphNode(layer)
             dtype = graph_node.layer.attr['dtype'].type
@@ -335,6 +340,14 @@ class TFDecoder(object):
                 if shape.count(-1) > 1:
                     need_define_shape = 2
 
+            if need_define_shape == 1:
+                try:
+                    shape = graph_node.out_shapes[0]
+                    if len(shape) > 0 and shape.count(-1) < 2:
+                        need_define_shape = 0
+                except:
+                    pass
+
             if need_define_shape > 0:
                 shape = None
                 if graph_node.get_attr("shape"):
@@ -377,10 +390,10 @@ class TFDecoder(object):
                         shape=shape,
                         name="x2paddle_{}".format(layer.name))
                 except:
-                    x2paddle_input = tf.placeholder(dtype=dtype,
-                                                    shape=shape,
-                                                    name="x2paddle_{}".format(
-                                                        layer.name))
+                    x2paddle_input = tf.placeholder(
+                        dtype=dtype,
+                        shape=shape,
+                        name="x2paddle_{}".format(layer.name))
 
                 input_map["{}:0".format(layer.name)] = x2paddle_input
                 if shape.count(None) > 0:

x2paddle/op_mapper/caffe_custom_layer/convolutiondepthwise.py

@@ -122,16 +122,17 @@ def convolutiondepthwise_layer(inputs,
     c_out = num_output if num_output is not None else input_shape[0][1]
     group = int(c_in / (c_in / c_out)) if c_in > c_out else int(c_in /
                                                                 (c_out / c_in))
-    out = fluid.layers.conv2d(input,
-                              dilation=[dila_h, dila_w],
-                              filter_size=[k_h, k_w],
-                              stride=[s_h, s_w],
-                              padding=[p_h, p_w],
-                              groups=group,
-                              num_filters=c_out,
-                              param_attr=name + '_weights',
-                              bias_attr=name + '_bias',
-                              name=name)
+    out = fluid.layers.conv2d(
+        input,
+        dilation=[dila_h, dila_w],
+        filter_size=[k_h, k_w],
+        stride=[s_h, s_w],
+        padding=[p_h, p_w],
+        groups=group,
+        num_filters=c_out,
+        param_attr=name + '_weights',
+        bias_attr=name + '_bias',
+        name=name)
     return out
 
 
@@ -142,7 +143,8 @@ def convolutiondepthwise_weights(name, data=None):
     return weights_name
 
 
-register(kind='ConvolutionDepthwise',
-         shape=convolutiondepthwise_shape,
-         layer=convolutiondepthwise_layer,
-         weights=convolutiondepthwise_weights)
+register(
+    kind='ConvolutionDepthwise',
+    shape=convolutiondepthwise_shape,
+    layer=convolutiondepthwise_layer,
+    weights=convolutiondepthwise_weights)

x2paddle/op_mapper/caffe_custom_layer/detectionoutput.py

@@ -12,7 +12,6 @@ def detectionoutput_layer(inputs,
                           share_location=True,
                           keep_top_k=100,
                           confidence_threshold=0.1,
-                          num_classes=2,
                           input_shape=None,
                           name=None):
     nms_param_str = nms_param
@@ -37,9 +36,9 @@ def detectionoutput_layer(inputs,
     pb = fluid.layers.reshape(x=pb, shape=[-1, 4])
     pbv = fluid.layers.reshape(x=pbv, shape=[-1, 4])
     mbox_loc = inputs[0]
-    mbox_loc = fluid.layers.reshape(x=mbox_loc, shape=[0, -1, 4])
-    mbox_conf_flatten = fluid.layers.reshape(x=mbox_conf_flatten,
-                                             shape=[0, -1, num_classes])
+    mbox_loc = fluid.layers.reshape(x=mbox_loc, shape=[-1, pb.shape[0], 4])
+    mbox_conf_flatten = fluid.layers.reshape(
+        x=mbox_conf_flatten, shape=[0, pb.shape[0], -1])
 
     default = {"nms_threshold": 0.3, "top_k": 10, "eta": 1.0}
     fields = ['eta', 'top_k', 'nms_threshold']
@@ -65,7 +64,8 @@ def detectionoutput_weights(name, data=None):
     return weights_name
 
 
-register(kind='DetectionOutput',
-         shape=detectionoutput_shape,
-         layer=detectionoutput_layer,
-         weights=detectionoutput_weights)
+register(
+    kind='DetectionOutput',
+    shape=detectionoutput_shape,
+    layer=detectionoutput_layer,
+    weights=detectionoutput_weights)

x2paddle/op_mapper/caffe_custom_layer/normalize.py

@@ -20,9 +20,8 @@ def normalize_layer(inputs,
         attr=name + '_scale')
     scale_param = fluid.layers.reshape(x=scale_param, \
                   shape=[1] if channel_shared else [input_shape[0][1]])
-    out = fluid.layers.elementwise_mul(x=l2_norm,
-                                       y=scale_param,
-                                       axis=-1 if channel_shared else 1)
+    out = fluid.layers.elementwise_mul(
+        x=l2_norm, y=scale_param, axis=-1 if channel_shared else 1)
     return out
 
 
@@ -31,7 +30,8 @@ def normalize_weights(name, data=None):
     return weights_name
 
 
-register(kind='Normalize',
-         shape=normalize_shape,
-         layer=normalize_layer,
-         weights=normalize_weights)
+register(
+    kind='Normalize',
+    shape=normalize_shape,
+    layer=normalize_layer,
+    weights=normalize_weights)

x2paddle/op_mapper/caffe_custom_layer/permute.py

@@ -23,7 +23,8 @@ def permute_weights(name, data=None):
     return weights_name
 
 
-register(kind='Permute',
-         shape=permute_shape,
-         layer=permute_layer,
-         weights=permute_weights)
+register(
+    kind='Permute',
+    shape=permute_shape,
+    layer=permute_layer,
+    weights=permute_weights)

x2paddle/op_mapper/caffe_custom_layer/priorbox.py

@@ -30,18 +30,19 @@ def priorbox_layer(inputs,
     steps = tuple(step) if type(step) is list or type(step) is tuple else (step,
                                                                            step)
 
-    box, variance_ = fluid.layers.prior_box(input,
-                                            image,
-                                            min_sizes=min_size,
-                                            max_sizes=max_size,
-                                            aspect_ratios=aspect_ratio,
-                                            variance=variance,
-                                            flip=flip,
-                                            clip=clip,
-                                            steps=steps,
-                                            offset=offset,
-                                            name=name,
-                                            min_max_aspect_ratios_order=True)
+    box, variance_ = fluid.layers.prior_box(
+        input,
+        image,
+        min_sizes=min_size,
+        max_sizes=max_size,
+        aspect_ratios=aspect_ratio,
+        variance=variance,
+        flip=flip,
+        clip=clip,
+        steps=steps,
+        offset=offset,
+        name=name,
+        min_max_aspect_ratios_order=True)
     box = fluid.layers.reshape(box, [1, 1, -1])
     variance_ = fluid.layers.reshape(variance_, [1, 1, -1])
     out = fluid.layers.concat([box, variance_], axis=1)
@@ -53,7 +54,8 @@ def priorbox_weights(name, data=None):
     return weights_name
 
 
-register(kind='PriorBox',
-         shape=priorbox_shape,
-         layer=priorbox_layer,
-         weights=priorbox_weights)
+register(
+    kind='PriorBox',
+    shape=priorbox_shape,
+    layer=priorbox_layer,
+    weights=priorbox_weights)

x2paddle/op_mapper/caffe_custom_layer/register.py

@@ -23,8 +23,7 @@ def register(kind, shape, layer, weights):
         kind = [kind]
     else:
         assert type(
-            kind
-        ) is list, 'invalid param "kind" for register, not a list or str'
+            kind) is list, 'invalid param "kind" for register, not a list or str'
 
     for k in kind:
         assert type(

x2paddle/op_mapper/caffe_custom_layer/roipooling.py

@@ -21,11 +21,12 @@ def roipooling_layer(inputs,
     input = inputs[0]
     roi = inputs[1]
     roi = fluid.layers.slice(roi, axes=[1], starts=[1], ends=[5])
-    out = fluid.layers.roi_pool(input,
-                                roi,
-                                pooled_height=pooled_h,
-                                pooled_width=pooled_w,
-                                spatial_scale=spatial_scale)
+    out = fluid.layers.roi_pool(
+        input,
+        roi,
+        pooled_height=pooled_h,
+        pooled_width=pooled_w,
+        spatial_scale=spatial_scale)
     return out
 
 
@@ -34,7 +35,8 @@ def roipooling_weights(name, data=None):
     return weights_name
 
 
-register(kind='ROIPooling',
-         shape=roipooling_shape,
-         layer=roipooling_layer,
-         weights=roipooling_weights)
+register(
+    kind='ROIPooling',
+    shape=roipooling_shape,
+    layer=roipooling_layer,
+    weights=roipooling_weights)

x2paddle/op_mapper/caffe_custom_layer/select.py

@@ -30,11 +30,12 @@ def select_layer(inputs,
     out = []
     for i in range(len(slice_point)):
         out.append(
-            fluid.layers.slice(input,
-                               axes=[axis],
-                               starts=[slice_point[i]],
-                               ends=[slice_point[i + 1]],
-                               name=name + '_' + str(i)))
+            fluid.layers.slice(
+                input,
+                axes=[axis],
+                starts=[slice_point[i]],
+                ends=[slice_point[i + 1]],
+                name=name + '_' + str(i)))
         if i == len(slice_point) - 2:
             break
     return out
@@ -45,7 +46,8 @@ def select_weights(name, data=None):
     return weights_name
 
 
-register(kind='Select',
-         shape=select_shape,
-         layer=select_layer,
-         weights=select_weights)
+register(
+    kind='Select',
+    shape=select_shape,
+    layer=select_layer,
+    weights=select_weights)

x2paddle/op_mapper/caffe_custom_layer/shufflechannel.py

@@ -17,7 +17,8 @@ def shufflechannel_weights(name, data=None):
     return weights_name
 
 
-register(kind='ShuffleChannel',
-         shape=shufflechannel_shape,
-         layer=shufflechannel_layer,
-         weights=shufflechannel_weights)
+register(
+    kind='ShuffleChannel',
+    shape=shufflechannel_shape,
+    layer=shufflechannel_layer,
+    weights=shufflechannel_weights)

x2paddle/op_mapper/caffe_op_mapper.py

@@ -144,8 +144,8 @@ class CaffeOpMapper(OpMapper):
             [s_h, s_w] = [params.stride] * 2
         elif len(params.stride) > 0:
             s_h = params.stride_h if params.stride_h > 0 else params.stride[0]
-            s_w = params.stride_w if params.stride_w > 0 else params.stride[
-                len(params.stride) - 1]
+            s_w = params.stride_w if params.stride_w > 0 else params.stride[len(
+                params.stride) - 1]
         elif params.stride_h > 0 or params.stride_w > 0:
             s_h = params.stride_h
             s_w = params.stride_w
@@ -154,8 +154,8 @@ class CaffeOpMapper(OpMapper):
             [p_h, p_w] = [params.pad] * 2
         elif len(params.pad) > 0:
             p_h = params.pad_h if params.pad_h > 0 else params.pad[0]
-            p_w = params.pad_w if params.pad_w > 0 else params.pad[
-                len(params.pad) - 1]
+            p_w = params.pad_w if params.pad_w > 0 else params.pad[len(
+                params.pad) - 1]
         elif params.pad_h > 0 or params.pad_w > 0:
             p_h = params.pad_h
             p_w = params.pad_w
@@ -195,10 +195,8 @@ class CaffeOpMapper(OpMapper):
             'shape': shape,
             'name': string(node.layer_name)
         }
-        node.fluid_code.add_layer("data",
-                                  inputs=None,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            "data", inputs=None, output=node, param_attr=attr)
 
     def MemoryData(self, node):
         # TODO(syf): Paddlepaddle can't fully support
@@ -209,10 +207,8 @@ class CaffeOpMapper(OpMapper):
             'shape': shape,
             'name': string(node.layer_name)
         }
-        node.fluid_code.add_layer("data",
-                                  inputs=None,
-                                  output=node.layer_name + '0',
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            "data", inputs=None, output=node.layer_name + '0', param_attr=attr)
         node.fluid_code.add_note('{} = [{}]'.format(node.layer_name,
                                                     node.layer_name + '0'))
 
@@ -229,11 +225,9 @@ class CaffeOpMapper(OpMapper):
             input_c = node.input_shape[0][1]
             output_c = channel
             data.append(
-                np.zeros([output_c, input_c, kernel[0],
-                          kernel[1]]).astype('float32'))
-            data.append(np.zeros([
-                output_c,
-            ])).astype('float32')
+                np.zeros([output_c, input_c, kernel[0], kernel[1]]).astype(
+                    'float32'))
+            data.append(np.zeros([output_c, ])).astype('float32')
         else:
             data = self.adjust_parameters(node)
         self.weights[node.layer_name + '_weights'] = data[0]
@@ -244,29 +238,19 @@ class CaffeOpMapper(OpMapper):
         input = self.graph.get_bottom_node(node, idx=0, copy=True)
 
         attr = {
-            'filter_size':
-            kernel,
-            'num_filters':
-            channel,
-            'stride':
-            stride,
-            'padding':
-            pad,
-            'dilation':
-            dilation,
-            'groups':
-            group,
-            'name':
-            string(node.layer_name),
-            'param_attr':
-            string(node.layer_name + '_weights'),
-            'bias_attr':
-            False if len(data) == 1 else string(node.layer_name + '_bias'),
+            'filter_size': kernel,
+            'num_filters': channel,
+            'stride': stride,
+            'padding': pad,
+            'dilation': dilation,
+            'groups': group,
+            'name': string(node.layer_name),
+            'param_attr': string(node.layer_name + '_weights'),
+            'bias_attr': False
+            if len(data) == 1 else string(node.layer_name + '_bias'),
         }
-        node.fluid_code.add_layer("conv2d",
-                                  inputs=input,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            "conv2d", inputs=input, output=node, param_attr=attr)
 
     def Deconvolution(self, node):
         data = node.data
@@ -281,11 +265,9 @@ class CaffeOpMapper(OpMapper):
             input_c = node.input_shape[0][1]
             output_c = channel
             data.append(
-                np.zeros([output_c, input_c, kernel[0],
-                          kernel[1]]).astype('float32'))
-            data.append(np.zeros([
-                output_c,
-            ]).astype('float32'))
+                np.zeros([output_c, input_c, kernel[0], kernel[1]]).astype(
+                    'float32'))
+            data.append(np.zeros([output_c, ]).astype('float32'))
         else:
             data = self.adjust_parameters(node)
         self.weights[node.layer_name + '_weights'] = data[0]
@@ -295,31 +277,20 @@ class CaffeOpMapper(OpMapper):
                    ) == 1, 'The count of Deconvolution node\'s input is not 1.'
         input = self.graph.get_bottom_node(node, idx=0, copy=True)
         attr = {
-            'output_size':
-            None,
-            'filter_size':
-            kernel,
-            'num_filters':
-            channel,
-            'stride':
-            stride,
-            'padding':
-            pad,
-            'dilation':
-            dilation,
-            'groups':
-            group,
-            'name':
-            string(node.layer_name),
-            'param_attr':
-            string(node.layer_name + '_weights'),
-            'bias_attr':
-            False if len(data) == 1 else string(node.layer_name + '_bias')
+            'output_size': None,
+            'filter_size': kernel,
+            'num_filters': channel,
+            'stride': stride,
+            'padding': pad,
+            'dilation': dilation,
+            'groups': group,
+            'name': string(node.layer_name),
+            'param_attr': string(node.layer_name + '_weights'),
+            'bias_attr': False
+            if len(data) == 1 else string(node.layer_name + '_bias')
         }
-        node.fluid_code.add_layer("conv2d_transpose",
-                                  inputs=input,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            "conv2d_transpose", inputs=input, output=node, param_attr=attr)
 
     def Pooling(self, node):
         params = node.layer.pooling_param
@@ -345,10 +316,8 @@ class CaffeOpMapper(OpMapper):
             'global_pooling': global_pool,
             'name': string(node.layer_name)
         }
-        node.fluid_code.add_layer("pool2d",
-                                  inputs=input,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            "pool2d", inputs=input, output=node, param_attr=attr)
 
     def LRN(self, node):
         assert len(node.inputs) == 1, 'The count of LRN node\'s input is not 1.'
@@ -368,10 +337,8 @@ class CaffeOpMapper(OpMapper):
             'beta': params.beta,
             'name': string(node.layer_name)
         }
-        node.fluid_code.add_layer("lrn",
-                                  inputs=input,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            "lrn", inputs=input, output=node, param_attr=attr)
 
     def InnerProduct(self, node):
         data = node.data
@@ -384,8 +351,8 @@ class CaffeOpMapper(OpMapper):
             output_c = params.num_output
             data = []
             data.append(
-                np.zeros([input_c,
-                          output_c]).astype('float32').astype('float32'))
+                np.zeros([input_c, output_c]).astype('float32').astype(
+                    'float32'))
             data.append(
                 np.zeros([output_c]).astype('float32').astype('float32'))
         else:
@@ -409,21 +376,15 @@ class CaffeOpMapper(OpMapper):
         assert params.bias_term == True
         input = self.graph.get_bottom_node(node, idx=0, copy=True)
         attr = {
-            'size':
-            params.num_output,
-            'name':
-            string(node.layer_name),
-            'act':
-            None,
-            'param_attr':
-            string(node.layer_name + '_weights'),
-            'bias_attr':
-            False if len(data) == 1 else string(node.layer_name + '_bias')
+            'size': params.num_output,
+            'name': string(node.layer_name),
+            'act': None,
+            'param_attr': string(node.layer_name + '_weights'),
+            'bias_attr': False
+            if len(data) == 1 else string(node.layer_name + '_bias')
         }
-        node.fluid_code.add_layer("fc",
-                                  inputs=input,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            "fc", inputs=input, output=node, param_attr=attr)
 
     def Softmax(self, node):
         assert len(
@@ -435,10 +396,8 @@ class CaffeOpMapper(OpMapper):
         dims = len(shape)
         axis = axis + dims if axis < 0 else axis
         attr = {'axis': axis, 'name': string(node.layer_name + '_softmax')}
-        node.fluid_code.add_layer("softmax",
-                                  inputs=input,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            "softmax", inputs=input, output=node, param_attr=attr)
 
     def Slice(self, node):
         assert len(
@@ -459,15 +418,13 @@ class CaffeOpMapper(OpMapper):
             'dim': axis,
             'name': string(node.layer_name)
         }
-        node.fluid_code.add_layer("split",
-                                  inputs=input,
-                                  output=node.layer_name,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            "split", inputs=input, output=node.layer_name, param_attr=attr)
 
     def Concat(self, node):
         assert len(
             node.inputs
-        ) > 1, 'The count of Concat node\'s input is not more than 1.'
+        ) >= 1, 'The count of Concat node\'s input is not more than 1.'
         inputs = []
         for i in range(len(node.inputs)):
             input = self.graph.get_bottom_node(node, idx=i, copy=True)
@@ -475,10 +432,8 @@ class CaffeOpMapper(OpMapper):
         params = node.layer.concat_param
         axis = params.axis
         attr = {'axis': axis, 'name': string(node.layer_name)}
-        node.fluid_code.add_layer("concat",
-                                  inputs=inputs,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            "concat", inputs=inputs, output=node, param_attr=attr)
 
     def PReLU(self, node):
         assert len(
@@ -499,10 +454,8 @@ class CaffeOpMapper(OpMapper):
             'param_attr': string(node.layer_name + '_weights'),
             'name': string(node.layer_name)
         }
-        node.fluid_code.add_layer("prelu",
-                                  inputs=input,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            "prelu", inputs=input, output=node, param_attr=attr)
 
     def Accuracy(self, node):
         assert len(
@@ -526,10 +479,8 @@ class CaffeOpMapper(OpMapper):
         assert axis == 1, 'PaddlePaddle can not support the situation when the axis is not 1.'
         assert not ignore_label >= 0, 'PaddlePaddle can not support the situation when the model has ignore label.'
         attr = {'k': top_k}
-        node.fluid_code.add_layer("accuracy",
-                                  inputs=inputs,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            "accuracy", inputs=inputs, output=node, param_attr=attr)
 
     def Eltwise(self, node):
         assert len(
@@ -546,10 +497,11 @@ class CaffeOpMapper(OpMapper):
             inputs_dict['x'] = inputs[0]
             inputs_dict['y'] = inputs[1]
             attr = {'act': None, 'name': string(node.layer_name)}
-            node.fluid_code.add_layer("elementwise_mul",
-                                      inputs=inputs_dict,
-                                      output=node,
-                                      param_attr=attr)
+            node.fluid_code.add_layer(
+                "elementwise_mul",
+                inputs=inputs_dict,
+                output=node,
+                param_attr=attr)
         elif mode == 1:
             if hasattr(params, 'coeff') and len(params.coeff) == 2:
                 coeff = params.coeff
@@ -559,57 +511,62 @@ class CaffeOpMapper(OpMapper):
                     'value': coeff[0],
                     'dtype': '{}.dtype'.format(input1_name)
                 }
-                node.fluid_code.add_layer("fill_constant",
-                                          inputs=None,
-                                          output=node.layer_name + '_const1',
-                                          param_attr=attr)
+                node.fluid_code.add_layer(
+                    "fill_constant",
+                    inputs=None,
+                    output=node.layer_name + '_const1',
+                    param_attr=attr)
                 attr = {'act': None, 'name': string(node.layer_name + '_mul1')}
-                node.fluid_code.add_layer("elementwise_mul",
-                                          inputs=input1_name + ', ' +
-                                          node.layer_name + '_const1',
-                                          output=node.layer_name + '_mul1',
-                                          param_attr=attr)
+                node.fluid_code.add_layer(
+                    "elementwise_mul",
+                    inputs=input1_name + ', ' + node.layer_name + '_const1',
+                    output=node.layer_name + '_mul1',
+                    param_attr=attr)
                 input2_name = self.get_input_name(inputs[1])
                 attr = {
                     'shape': [1],
                     'value': coeff[1],
                     'dtype': '{}.dtype'.format(input2_name)
                 }
-                node.fluid_code.add_layer("fill_constant",
-                                          inputs=None,
-                                          output=node.layer_name + '_const2',
-                                          param_attr=attr)
+                node.fluid_code.add_layer(
+                    "fill_constant",
+                    inputs=None,
+                    output=node.layer_name + '_const2',
+                    param_attr=attr)
                 attr = {'act': None, 'name': string(node.layer_name + '_mul2')}
-                node.fluid_code.add_layer("elementwise_mul",
-                                          inputs=input2_name + ', ' +
-                                          node.layer_name + '_const2',
-                                          output=node.layer_name + '_mul2',
-                                          param_attr=attr)
+                node.fluid_code.add_layer(
+                    "elementwise_mul",
+                    inputs=input2_name + ', ' + node.layer_name + '_const2',
+                    output=node.layer_name + '_mul2',
+                    param_attr=attr)
 
                 attr = {'act': None, 'name': string(node.layer_name)}
-                node.fluid_code.add_layer("elementwise_add",
-                                          inputs='{}_mul1, {}_mul2'.format(
-                                              node.layer_name, node.layer_name),
-                                          output=node,
-                                          param_attr=attr)
+                node.fluid_code.add_layer(
+                    "elementwise_add",
+                    inputs='{}_mul1, {}_mul2'.format(node.layer_name,
+                                                     node.layer_name),
+                    output=node,
+                    param_attr=attr)
             else:
                 inputs_dict = {}
                 inputs_dict['x'] = inputs[0]
                 inputs_dict['y'] = inputs[1]
                 attr = {'act': None, 'name': string(node.layer_name)}
-                node.fluid_code.add_layer("elementwise_add",
-                                          inputs=inputs_dict,
-                                          output=node,
-                                          param_attr=attr)
+                node.fluid_code.add_layer(
+                    "elementwise_add",
+                    inputs=inputs_dict,
+                    output=node,
+                    param_attr=attr)
         else:
             inputs_dict = {}
             inputs_dict['x'] = inputs[0]
             inputs_dict['y'] = inputs[1]
             attr = {'act': None, 'name': string(node.layer_name)}
-            node.fluid_code.add_layer("elementwise_max",
-                                      inputs=inputs_dict,
-                                      output=node,
-                                      param_attr=attr)
+            node.fluid_code.add_layer(
+                "elementwise_max",
+                inputs=inputs_dict,
+                output=node,
+                param_attr=attr)
 
     def BatchNorm(self, node):
         assert len(
@@ -625,12 +582,8 @@ class CaffeOpMapper(OpMapper):
                 'The parameter of {} (type is {}) is not set. So we set the parameters as 0'
                 .format(node.layer_name, node.layer_type))
             input_c = node.input_shape[0][1]
-            mean = np.zeros([
-                input_c,
-            ]).astype('float32')
-            variance = np.zeros([
-                input_c,
-            ]).astype('float32')
+            mean = np.zeros([input_c, ]).astype('float32')
+            variance = np.zeros([input_c, ]).astype('float32')
             scale = 0
         else:
 
@@ -651,10 +604,8 @@ class CaffeOpMapper(OpMapper):
             'epsilon': eps,
             'name': string(node.layer_name)
         }
-        node.fluid_code.add_layer("batch_norm",
-                                  inputs=input,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            "batch_norm", inputs=input, output=node, param_attr=attr)
 
     def Scale(self, node):
         if node.data is None:
@@ -669,10 +620,10 @@ class CaffeOpMapper(OpMapper):
                 input_c,
             ]).astype('float32')
         else:
-            self.weights[node.layer_name + '_scale'] = np.squeeze(
-                node.data[0]).astype('float32')
-            self.weights[node.layer_name + '_offset'] = np.squeeze(
-                node.data[1]).astype('float32')
+            self.weights[node.layer_name + '_scale'] = np.squeeze(node.data[
+                0]).astype('float32')
+            self.weights[node.layer_name + '_offset'] = np.squeeze(node.data[
+                1]).astype('float32')
         params = node.layer.scale_param
         axis = params.axis
         num_axes = params.num_axes
@@ -687,10 +638,11 @@ class CaffeOpMapper(OpMapper):
             inputs_dict['x'] = input0
             inputs_dict['y'] = input1
             attr = {'axis': axis, 'name': string(node.layer_name + '_mul')}
-            node.fluid_code.add_layer("elementwise_mul",
-                                      inputs=inputs_dict,
-                                      output=node.layer_name + '_mul',
-                                      param_attr=attr)
+            node.fluid_code.add_layer(
+                "elementwise_mul",
+                inputs=inputs_dict,
+                output=node.layer_name + '_mul',
+                param_attr=attr)
         else:
             bias_shape = node.input_shape[0][axis:axis + num_axes]
             input0 = self.graph.get_bottom_node(node, idx=0, copy=True)
@@ -703,18 +655,17 @@ class CaffeOpMapper(OpMapper):
                 'is_bias': True,
                 'default_initializer': 'Constant(value=1.0)'
             }
-            node.fluid_code.add_layer("create_parameter",
-                                      inputs=None,
-                                      output=node,
-                                      param_attr=attr)
+            node.fluid_code.add_layer(
+                "create_parameter", inputs=None, output=node, param_attr=attr)
             inputs_dict = {}
             inputs_dict['x'] = input0
             inputs_dict['y'] = node
             attr = {'axis': axis, 'name': string(node.layer_name + '_mul')}
-            node.fluid_code.add_layer("elementwise_mul",
-                                      inputs=inputs_dict,
-                                      output=node.layer_name + '_mul',
-                                      param_attr=attr)
+            node.fluid_code.add_layer(
+                "elementwise_mul",
+                inputs=inputs_dict,
+                output=node.layer_name + '_mul',
+                param_attr=attr)
         scale_shape = bias_shape
         input0_name = self.get_input_name(input0)
         attr = {
@@ -725,16 +676,18 @@ class CaffeOpMapper(OpMapper):
             'is_bias': True,
             'default_initializer': 'Constant(value=1.0)'
         }
-        node.fluid_code.add_layer("create_parameter",
-                                  inputs=None,
-                                  output=node.layer_name + '_offset_param',
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            "create_parameter",
+            inputs=None,
+            output=node.layer_name + '_offset_param',
+            param_attr=attr)
         attr = {'axis': axis, 'name': string(node.layer_name + '_add')}
-        node.fluid_code.add_layer("elementwise_add",
-                                  inputs='{}_mul, {}_offset_param'.format(
-                                      node.layer_name, node.layer_name),
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            "elementwise_add",
+            inputs='{}_mul, {}_offset_param'.format(node.layer_name,
+                                                    node.layer_name),
+            output=node,
+            param_attr=attr)
 
     def Reshape(self, node):
         input = self.graph.get_bottom_node(node, idx=0, copy=True)
@@ -747,10 +700,8 @@ class CaffeOpMapper(OpMapper):
             'act': None,
             'name': string(node.layer_name)
         }
-        node.fluid_code.add_layer("reshape",
-                                  inputs=input,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            "reshape", inputs=input, output=node, param_attr=attr)
 
     def ArgMax(self, node):
         assert len(node.inputs) == 1 and len(
@@ -767,11 +718,12 @@ class CaffeOpMapper(OpMapper):
             axis += len(input_shape)
         if out_max_val is True:
             attr = {'k': top_k, 'name': string(node.layer_name + '_topk')}
-            node.fluid_code.add_layer("topk",
-                                      inputs=input,
-                                      output='{}_topk_var, {}_index_var'.format(
-                                          node.layer_name, node.layer_name),
-                                      param_attr=attr)
+            node.fluid_code.add_layer(
+                "topk",
+                inputs=input,
+                output='{}_topk_var, {}_index_var'.format(node.layer_name,
+                                                          node.layer_name),
+                param_attr=attr)
             attr = {'dtype': '{}_topk_var.dtype'.format(node.layer_name)}
             node.fluid_code.add_layer(
                 "cast",
@@ -779,17 +731,19 @@ class CaffeOpMapper(OpMapper):
                 output='{}_index_var'.format(node.layer_name),
                 param_attr=attr)
             attr = {'axis': axis, 'name': string(node.layer_name)}
-            node.fluid_code.add_layer("concat",
-                                      inputs='{}_topk_var, {}_index_var'.format(
-                                          node.layer_name, node.layer_name),
-                                      output=node,
-                                      param_attr=attr)
+            node.fluid_code.add_layer(
+                "concat",
+                inputs='{}_topk_var, {}_index_var'.format(node.layer_name,
+                                                          node.layer_name),
+                output=node,
+                param_attr=attr)
         else:
             attr = {'k': top_k, 'name': string(node.layer_name)}
-            node.fluid_code.add_layer("topk",
-                                      inputs=input,
-                                      output='_, {}'.format(node.layer_name),
-                                      param_attr=attr)
+            node.fluid_code.add_layer(
+                "topk",
+                inputs=input,
+                output='_, {}'.format(node.layer_name),
+                param_attr=attr)
 
     def Crop(self, node):
         assert len(
@@ -797,36 +751,34 @@ class CaffeOpMapper(OpMapper):
         input = self.graph.get_bottom_node(node, idx=0, copy=True)
         example = self.graph.get_bottom_node(node, idx=1, copy=True)
         params = node.layer.crop_param
-        axis = parmas.axis
+        axis = params.axis
         input_shape = node.input_shape[0]
         if axis < 0:
             axis += len(input_shape)
         offset_real = [0] * len(input_shape)
-        if hasattr(params, offset):
+        if hasattr(params, "offset") and len(params.offset) > 0:
             offset = list(params.offset)
-            assert (len(input_shape) - axis) == len(
-                offset), "invalid offset[%s] in crop layer" % (str(offset))
+            assert (len(input_shape) - axis
+                    ) == len(offset), "invalid offset[%s] in crop layer" % (
+                        str(offset))
             offset_real = [0] * axis + offset
-        attr = {'offsets': offset_real, 'name': string(node.layer_name)}
-        node.fluid_code.add_layer("crop",
-                                  inputs={
-                                      'x': input,
-                                      'y': example
-                                  },
-                                  output=node,
-                                  param_attr=attr)
+        attr = {'offsets': list(offset_real), 'name': string(node.layer_name)}
+        node.fluid_code.add_layer(
+            "crop",
+            inputs={'x': input,
+                    'shape': node.input_shape[1]},
+            output=node,
+            param_attr=attr)
 
     def Flatten(self, node):
         assert len(
-            node.inputs
-        ) == 1, 'The count of DetectionOutput node\'s input is not 1.'
+            node.
+            inputs) == 1, 'The count of DetectionOutput node\'s input is not 1.'
         input = self.graph.get_bottom_node(node, idx=0, copy=True)
         shape = node.output_shape[0]
         attr = {'shape': shape, 'name': string(node.layer_name)}
-        node.fluid_code.add_layer("reshape",
-                                  inputs=input,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            "reshape", inputs=input, output=node, param_attr=attr)
 
     def Power(self, node):
         assert len(
@@ -842,15 +794,11 @@ class CaffeOpMapper(OpMapper):
             'bias_after_scale': True,
             'name': string(node.layer_name + '_scale')
         }
-        node.fluid_code.add_layer("scale",
-                                  inputs=input,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            "scale", inputs=input, output=node, param_attr=attr)
         attr = {'factor': power, 'name': string(node.layer_name)}
-        node.fluid_code.add_layer("pow",
-                                  inputs=node,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            "pow", inputs=node, output=node, param_attr=attr)
 
     def Reduction(self, node):
         assert len(
@@ -872,55 +820,41 @@ class CaffeOpMapper(OpMapper):
                 'keep_dim': False,
                 'name': string(node.layer_name)
             }
-            node.fluid_code.add_layer("reduce_sum",
-                                      inputs=input,
-                                      output=node,
-                                      param_attr=attr)
+            node.fluid_code.add_layer(
+                "reduce_sum", inputs=input, output=node, param_attr=attr)
         elif operation == 2:  ## operation = ASUM
             attr = {'name': string(node.layer_name + '_abs')}
-            node.fluid_code.add_layer("abs",
-                                      inputs=input,
-                                      output=node,
-                                      param_attr=attr)
+            node.fluid_code.add_layer(
+                "abs", inputs=input, output=node, param_attr=attr)
             attr = {
                 'dim': dim[axis:],
                 'keep_dim': False,
                 'name': string(node.layer_name)
             }
-            node.fluid_code.add_layer("reduce_sum",
-                                      inputs=node,
-                                      output=node,
-                                      param_attr=attr)
+            node.fluid_code.add_layer(
+                "reduce_sum", inputs=node, output=node, param_attr=attr)
         elif operation == 3:  ## operation = SUMSQ
             attr = {'factor': 2.0, 'name': string(node.layer_name + '_pow')}
-            node.fluid_code.add_layer("pow",
-                                      inputs=input,
-                                      output=node,
-                                      param_attr=attr)
+            node.fluid_code.add_layer(
+                "pow", inputs=input, output=node, param_attr=attr)
             attr = {
                 'dim': dim[axis:],
                 'keep_dim': False,
                 'name': string(node.layer_name)
             }
-            node.fluid_code.add_layer("reduce_sum",
-                                      inputs=node,
-                                      output=node,
-                                      param_attr=attr)
+            node.fluid_code.add_layer(
+                "reduce_sum", inputs=node, output=node, param_attr=attr)
         else:  ## operation = MEAN
             attr = {
                 'dim': dim[axis:],
                 'keep_dim': False,
                 'name': string(node.layer_name)
             }
-            node.fluid_code.add_layer("reduce_mean",
-                                      inputs=node,
-                                      output=node,
-                                      param_attr=attr)
+            node.fluid_code.add_layer(
+                "reduce_mean", inputs=node, output=node, param_attr=attr)
         attr = {'scale': coeff}
-        node.fluid_code.add_layer("scale",
-                                  inputs=node,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            "scale", inputs=node, output=node, param_attr=attr)
 
     def deal_custom_layer(self, node):
         op = node.layer_type
@@ -947,11 +881,12 @@ class CaffeOpMapper(OpMapper):
                 assert input is not None, 'This kind of DetectionOutput is not supported!'
                 input = self.graph.get_bottom_node(input, idx=0, copy=True)
             inputs_node.append(input)
-        node.fluid_code.add_layer(func.__code__.co_name,
-                                  inputs=inputs_node,
-                                  output=node,
-                                  param_attr=kwargs,
-                                  is_custom_layer=True)
+        node.fluid_code.add_layer(
+            func.__code__.co_name,
+            inputs=inputs_node,
+            output=node,
+            param_attr=kwargs,
+            is_custom_layer=True)
         if op not in self.used_custom_layers:
             self.used_custom_layers[op] = custom_code
 
@@ -960,7 +895,5 @@ class CaffeOpMapper(OpMapper):
         op_info = self.directly_map_ops[node.layer_type]
         input = self.graph.get_bottom_node(node, idx=0, copy=True)
         attr = {'name': string(node.layer_name)}
-        node.fluid_code.add_layer(op_info,
-                                  inputs=input,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            op_info, inputs=input, output=node, param_attr=attr)

x2paddle/op_mapper/caffe_shape.py

@@ -33,8 +33,8 @@ def get_kernel_parameters(params):
         [s_h, s_w] = [params.stride] * 2
     elif len(params.stride) > 0:
         s_h = params.stride_h if params.stride_h > 0 else params.stride[0]
-        s_w = params.stride_w if params.stride_w > 0 else params.stride[
-            len(params.stride) - 1]
+        s_w = params.stride_w if params.stride_w > 0 else params.stride[len(
+            params.stride) - 1]
     elif params.stride_h > 0 or params.stride_w > 0:
         s_h = params.stride_h
         s_w = params.stride_w
@@ -293,12 +293,15 @@ def shape_reshape(layer, input_shape):
             explicit_count *= count(l)
         for i in range(len(copy_axes)):
             explicit_count *= outshape[start_axis + copy_axes[i]]
-        outshape[start_axis + inferred_axis] = -1
-        outshape[0] = 0
-    else:
-        outshape[0] = -1
+        assert input_count % explicit_count == 0, "[Reshape]botom count[%d] "\
+                "must be divisible by product of the specified dimensions[%d] "\
+                % (input_count, explicit_count)
+        outshape[start_axis + inferred_axis] = int(input_count / explicit_count)
 
     output_count = count(outshape)
+    assert output_count == input_count, "[Reshape]output count[%d] must match input count[%d]" % (
+        output_count, input_count)
+    outshape[0] = -1
     return [outshape]
 
 
@@ -342,10 +345,9 @@ def shape_flatten(layer, input_shape):
     output_shape = inshape[0:start_axis]
     if len(inshape[start_axis:end_axis]) != 0:
         flat_sz = reduce(lambda a, b: a * b, inshape[start_axis:end_axis])
-        flat_sz = -1
-        output_shape[0] = 0
         output_shape += [flat_sz]
     output_shape += inshape[end_axis:len(inshape)]
+    output_shape[0] = -1
     return [output_shape]
 
 

x2paddle/op_mapper/onnx_custom_layer/InstanceNormalization.py → x2paddle/op_mapper/onnx/custom_layer/InstanceNormalization.py

@@ -24,21 +24,18 @@ def InstanceNormalization_layer(inputs, name=None):
     epsilon = 1e-5
     input_ = inputs[0]
     mean = fluid.layers.reduce_mean(input_, dim=[2, 3], keep_dim=True)
-    var = fluid.layers.reduce_mean(fluid.layers.square(input_ - mean),
-                                   dim=[2, 3],
-                                   keep_dim=True)
+    var = fluid.layers.reduce_mean(
+        fluid.layers.square(input_ - mean), dim=[2, 3], keep_dim=True)
     if name is not None:
         scale_name = name + "_scale"
         offset_name = name + "_offset"
 
     scale_param = inputs[1]
     offset_param = inputs[2]
-    scale = fluid.layers.create_parameter(name=scale_param.name,
-                                          shape=input_.shape[1:2],
-                                          dtype="float32")
-    offset = fluid.layers.create_parameter(name=offset_param.name,
-                                           shape=input_.shape[1:2],
-                                           dtype="float32")
+    scale = fluid.layers.create_parameter(
+        name=scale_param.name, shape=input_.shape[1:2], dtype="float32")
+    offset = fluid.layers.create_parameter(
+        name=offset_param.name, shape=input_.shape[1:2], dtype="float32")
 
     tmp = fluid.layers.elementwise_mul(x=(input_ - mean), y=scale, axis=1)
     tmp = tmp / fluid.layers.sqrt(var + epsilon)
@@ -51,8 +48,9 @@ def InstanceNormalization_weights(name, data=None):
     return weights_name
 
 
-register(kind='InstanceNormalization',
-         shape=InstanceNormalization_shape,
-         layer=InstanceNormalization_layer,
-         child_func=None,
-         weights=InstanceNormalization_weights)
+register(
+    kind='InstanceNormalization',
+    shape=InstanceNormalization_shape,
+    layer=InstanceNormalization_layer,
+    child_func=None,
+    weights=InstanceNormalization_weights)

x2paddle/op_mapper/onnx_custom_layer/register.py → x2paddle/op_mapper/onnx/custom_layer/register.py

@@ -36,8 +36,7 @@ def register(kind, shape, layer, child_func, weights):
         kind = [kind]
     else:
         assert type(
-            kind
-        ) is list, 'invalid param "kind" for register, not a list or str'
+            kind) is list, 'invalid param "kind" for register, not a list or str'
 
     for k in kind:
         assert type(

x2paddle/op_mapper/onnx/onnx_helper.py

+# Copyright (c) 2019  PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License"
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from x2paddle.op_mapper.onnx.opset9 import ONNXOpMapperOpSet9
+
+
+class ONNXOpMapperFactory:
+    def __init__(self):
+        self.support_op_sets = [9, ]
+        self.default_op_set = 9
+
+    def create_onnx_op_mapper(self, decoder):
+        run_op_set = self.default_op_set
+        OpMapper = ''
+        if decoder.op_set in self.support_op_sets:
+            OpMapper = 'ONNXOpMapperOpSet' + str(decoder.op_set)
+        elif decoder.op_set < self.default_op_set:
+            OpMapper = 'ONNXOpMapperOpSet' + str(self.default_op_set)
+        else:
+            for op_set in self.support_op_sets:
+                if decoder.op_set > op_set:
+                    run_op_set = op_set
+                else:
+                    break
+            OpMapper = 'ONNXOpMapperOpSet' + str(run_op_set)
+        print(
+            'Now, onnx2paddle support convert onnx model opset_verison {},'
+            'opset_verison of your onnx model is {}, automatically treated as op_set: {}.'
+            .format(self.support_op_sets, decoder.op_set, run_op_set))
+        return eval(OpMapper)(decoder)

x2paddle/op_mapper/onnx_op_mapper.py → x2paddle/op_mapper/onnx/opset9.py

-#   Copyright (c) 2019  PaddlePaddle Authors. All Rights Reserved.
+# Copyright (c) 2019  PaddlePaddle Authors. All Rights Reserved.
 #
 # Licensed under the Apache License, Version 2.0 (the "License"
 # you may not use this file except in compliance with the License.
@@ -17,22 +17,19 @@ from x2paddle.core.op_mapper import OpMapper
 from x2paddle.core.fluid_code import Layer
 from x2paddle.core.fluid_code import FluidCode
 from x2paddle.decoder.onnx_decoder import ONNXGraph, ONNXGraphNode, ONNXGraphDataNode
-from x2paddle.op_mapper.onnx_directly_map import default_op_mapping_field_values
-from x2paddle.op_mapper.onnx_directly_map import default_op_mapping
-from x2paddle.op_mapper.onnx_directly_map import default_ioa_constraint
-from x2paddle.op_mapper.onnx_custom_layer import *
+from x2paddle.op_mapper.onnx.custom_layer import *
 from x2paddle.core.util import string
 import numpy as np
 import onnx
 import onnx.numpy_helper as numpy_helper
 from onnx.mapping import TENSOR_TYPE_TO_NP_TYPE
 import logging as _logging
-from collections import OrderedDict as _dict
+from collections import OrderedDict
 import math
 import os
 import shutil
 from functools import reduce
-import onnxruntime as rt
+
 _logger = _logging.getLogger(__name__)
 
 
@@ -52,7 +49,24 @@ def get_same_padding(in_size, kernel_size, stride):
     return [pad0, pad1]
 
 
-class ONNXOpMapper(OpMapper):
+def print_mapping_info(func):
+    def run_mapping(*args, **kwargs):
+        node = args[1]
+        try:
+            res = func(*args, **kwargs)
+        except:
+            print("convert failed node:{}, op_type is {}".format(
+                node.layer_name[9:], node.layer_type))
+            raise
+        else:
+            #print("convert successfully node:{}, op_type is {}".format(
+            #    node.layer_name[9:], node.layer_type))
+            return res
+
+    return run_mapping
+
+
+class ONNXOpMapperOpSet9(OpMapper):
     elementwise_ops = {
         'Add': 'elementwise_add',
         'Div': 'elementwise_div',
@@ -61,18 +75,76 @@ class ONNXOpMapper(OpMapper):
         'Pow': 'elementwise_pow',
     }
 
-    def __init__(self, decoder, save_dir):
-        super(ONNXOpMapper, self).__init__()
-        self.decoder = decoder
-        self.graph = decoder.onnx_graph
+    default_op_mapping_field_values = OrderedDict()
+    default_op_mapping_field_values['FLUID_OP'] = ''
+    default_op_mapping_field_values['FLUID_INPUT_ARGS'] = None
+    default_op_mapping_field_values['FLUID_OUTPUT_ARGS'] = None
+    default_op_mapping_field_values['ATTR_MAPPING'] = dict()
+    default_op_mapping_field_values['DEFAULTS'] = dict()
+    default_op_mapping_field_values['INPUT_PERM'] = None
+    default_op_mapping_field_values['OUTPUT_PERM'] = None
+    default_op_mapping_field_values['FILL_NAME_FIELD'] = True
+
+    default_op_mapping = {
+        'Shape': ['shape', ['X'], ['Out']],
+        'Clip': [
+            'clip', ['X'], ['Out'], dict(), dict(
+                min=(np.asarray(
+                    [255, 255, 127, 255], dtype=np.uint8).view(np.float32)[0]),
+                max=(np.asarray(
+                    [255, 255, 127, 127], dtype=np.uint8).view(np.float32)[0]),
+            )
+        ],
+        'Erf': ['erf', ['X'], ['Out']],
+        'Ceil': ['ceil', ['X'], ['Out']],
+        'ReduceMean': [
+            'reduce_mean', ['X'], ['Out'], dict(
+                axes='dim', keepdims='keep_dim'), dict(keep_dim=1)
+        ],
+        'ReduceSum': [
+            'reduce_sum', ['X'], ['Out'], dict(
+                axes='dim', keepdims='keep_dim'), dict(keep_dim=1)
+        ],
+        'ReduceMin': [
+            'reduce_min', ['X'], ['Out'], dict(
+                axes='dim', keepdims='keep_dim'), dict(keep_dim=1)
+        ],
+        #active function
+        'Relu': ['relu', ['X'], ['Out']],
+        'LeakyRelu': ['leaky_relu', ['X'], ['Out'], dict(), dict(alpha=.01)],
+        'Elu': ['elu', ['X'], ['Out'], dict(), dict(alpha=1.)],
+        'ThresholdedRelu': [
+            'thresholded_relu', ['X'], ['Out'], dict(alpha='threshold'),
+            dict(alpha=1.)
+        ],
+        'Tanh': ['tanh', ['X'], ['Out']],
+        'Sigmoid': ['sigmoid', ['X'], ['Out']],
+        'HardSigmoid': [
+            'hard_sigmoid', ['X'], ['Out'], dict(
+                alpha='slope', beta='offset'), dict(
+                    slope=.2, offset=.5)
+        ],
+        'Softsign': ['softsign', ['X'], ['Out']],
+        'Softplus': ['softplus', ['X'], ['Out']],
+        'Exp': ['exp', ['X'], ['Out']],
+        'Softmax': ['softmax', ['X'], ['Out'], dict(), dict(axis=1)],
+        'Sqrt': ['sqrt', ['X'], ['Out']],
+        'Floor': ['floor', ['X'], ['Out']],
+        'Abs': ['abs', ['X'], ['Out']],
+    }
+
+    default_ioa_constraint = {
+        'Gather':
+        [(lambda i, o, a: a.get('axis', 0) == 0, 'only axis = 0 is supported')],
+    }
+
+    def __init__(self, decoder):
+        super(ONNXOpMapperOpSet9, self).__init__()
+        self.graph = decoder.graph
         self.input_shapes = []
         self.weights = dict()
         self.omit_nodes = list()
         self.used_custom_layers = dict()
-        self.is_inference = False
-        self.tmp_data_dir = os.path.join(save_dir, 'tmp_data')
-        self.tmp_outputs_dict = {}
-        self.get_output_shapes()
 
         if not self.op_checker():
             raise Exception("Model are not supported yet.")
@@ -83,20 +155,21 @@ class ONNXOpMapper(OpMapper):
                 isinstance(node, ONNXGraphNode)
                 for name, node in self.graph.node_map.items()
             ])))
+
+        print("Nodes converting ...")
         for node_name in self.graph.topo_sort:
             node = self.graph.get_node(node_name)
             op = node.layer_type
             if hasattr(self, op):
                 func = getattr(self, op)
                 func(node)
-            elif op in default_op_mapping:
+            elif op in self.default_op_mapping:
                 self.directly_map(node)
             elif op in custom_layers:
                 self.deal_custom_layer(node)
             elif op in self.elementwise_ops:
                 self.elementwise_map(node)
-
-        self.remove_tmp_data()
+        print("Nodes converted.")
 
     def op_checker(self):
         unsupported_ops = set()
@@ -104,7 +177,7 @@ class ONNXOpMapper(OpMapper):
             node = self.graph.get_node(node_name)
             op = node.layer_type
             if not hasattr(self, op) and \
-                op not in default_op_mapping and \
+                op not in self.default_op_mapping and \
                 op not in custom_layers and \
                 op not in self.elementwise_ops:
                 unsupported_ops.add(op)
@@ -117,98 +190,15 @@ class ONNXOpMapper(OpMapper):
                 print(op)
             return False
 
-    def get_results_of_inference(self, model, value_infos, data_nodes):
-        if not os.path.exists(self.tmp_data_dir):
-            os.makedirs(self.tmp_data_dir)
-        inputs_dict = {}
-        for data_node in data_nodes:
-            value_info = value_infos[data_node]
-            shape = value_info['shape']
-            for i, dim_shape in enumerate(shape):
-                if dim_shape == 0 and i == 0:
-                    shape[i] = 1
-                if dim_shape == 0 and i != 0:
-                    assert 'shape of input is not assigned'
-            ipt = np.random.random(shape).astype(value_info['dtype'])
-            inputs_dict[data_node] = ipt
-
-        model = onnx.shape_inference.infer_shapes(model)
-        outputs = []
-
-        for value_info in model.graph.value_info:
-            outputs.append(value_info.name)
-
-        model.graph.ClearField('output')
-        model.graph.output.MergeFrom(model.graph.value_info)
-        onnx.save(model, os.path.join(self.tmp_data_dir,
-                                      'onnx_model_infer.onnx'))
-        sess = rt.InferenceSession(
-            os.path.join(self.tmp_data_dir, 'onnx_model_infer.onnx'))
-        res = sess.run(None, input_feed=inputs_dict)
-        self.tmp_outputs_dict = dict(zip(outputs, res))
-
-        return
-
-    def get_dynamic_shape(self, layer):
-        """
-        get dynamic shape from infer_result
-        """
-        if layer not in self.tmp_outputs_dict:
-            return [None, None, None]
-        output = self.tmp_outputs_dict[layer]
-        return output.tolist(), output.dtype, output.shape
-
-    def get_output_shapes(self):
-        """
-        build topo_sort of ONNX model
-        """
-        nodes = self.decoder.model.graph.node
-        node_map = self.decoder.onnx_graph.node_map
-        value_infos = self.decoder.onnx_graph.value_infos
-        onnx_model = self.decoder.model
-        for layer in nodes:
-            node = node_map[layer.name]
-            for opt in layer.output:
-                if opt in value_infos:
-                    value_info = value_infos[opt]
-                    if len(value_info['shape']) == 0 or value_info[
-                            'dtype'] is None or 0 in value_info['shape']:
-                        if self.is_inference == False:
-                            self.get_results_of_inference(
-                                onnx_model, value_infos,
-                                self.decoder.onnx_graph.place_holder_nodes)
-                            self.is_inference = True
-                        _, dtype, shape = self.get_dynamic_shape(opt)
-                        node.out_shapes.append(shape)
-                        node.dtype = dtype
-                    else:
-                        node.dtype = value_info['dtype']
-                        node.out_shapes.append(value_info['shape'])
-                else:
-                    if self.is_inference == False:
-                        self.get_results_of_inference(
-                            onnx_model, value_infos,
-                            self.decoder.onnx_graph.place_holder_nodes)
-                        self.is_inference = True
-                    _, dtype, shape = self.get_dynamic_shape(opt)
-                    node.dtype = dtype
-                    node.out_shapes.append(shape)
-
-    def remove_tmp_data(self):
-        """
-        remove temporarily generated file
-        """
-        if os.path.exists(self.tmp_data_dir):
-            import shutil
-            shutil.rmtree(self.tmp_data_dir)
-
+    @print_mapping_info
     def directly_map(self, node, name='', *args, **kwargs):
         inputs = node.layer.input
         outputs = node.layer.output
         op_type = node.layer_type
         attrs = node.attr_map
-        info = default_op_mapping[op_type]
-        info.extend(list(default_op_mapping_field_values.values())[len(info):])
+        info = self.default_op_mapping[op_type]
+        info.extend(
+            list(self.default_op_mapping_field_values.values())[len(info):])
         (
             fluid_op,
             fluid_input_args,
@@ -217,11 +207,10 @@ class ONNXOpMapper(OpMapper):
             default_attrs,
             input_perm,
             output_perm,
-            fill_name_field,
-        ) = info
+            fill_name_field, ) = info
 
-        if fluid_op in default_ioa_constraint:
-            for predicate, message in default_ioa_constraint[fluid_op]:
+        if fluid_op in self.default_ioa_constraint:
+            for predicate, message in self.default_ioa_constraint[fluid_op]:
                 assert predicate(inputs, outputs, attrs), message
 
         mapped_attrs = {
@@ -244,13 +233,18 @@ class ONNXOpMapper(OpMapper):
             map(lambda i: outputs[i], output_perm))
         attr = fluid_attrs
         assert len(val_inps) == 1, 'directly_map error with multi inputs'
-        if fluid_op not in ['shape']:
+        if fluid_op not in ['shape', 'erf']:
             attr['name'] = string(node.layer_name)
-        node.fluid_code.add_layer(fluid_op,
-                                  inputs=val_inps[0],
-                                  output=val_outs[0],
-                                  param_attr=attr)
-
+        node.fluid_code.add_layer(
+            fluid_op, inputs=val_inps[0], output=val_outs[0], param_attr=attr)
+        if fluid_op in ['shape']:
+            node.fluid_code.add_layer(
+                'cast',
+                inputs=val_outs[0],
+                output=val_outs[0],
+                param_attr={'dtype': string('int64')})
+
+    @print_mapping_info
     def deal_custom_layer(self, node):
         op = node.layer_type
         custom_code, func = make_custom_layer(node)
@@ -258,11 +252,12 @@ class ONNXOpMapper(OpMapper):
         params = get_params(node.layer, node.layer_type)
         arg_names, kwargs = set_args(func, params)
         kwargs['name'] = string(node.layer_name)
-        node.fluid_code.add_layer(func.__code__.co_name,
-                                  inputs=node.inputs,
-                                  output=node,
-                                  param_attr=kwargs,
-                                  is_custom_layer=True)
+        node.fluid_code.add_layer(
+            func.__code__.co_name,
+            inputs=node.inputs,
+            output=node,
+            param_attr=kwargs,
+            is_custom_layer=True)
         if op not in self.used_custom_layers:
             self.used_custom_layers[op] = custom_code
             if op + '_child_func' not in self.used_custom_layers:
@@ -270,6 +265,7 @@ class ONNXOpMapper(OpMapper):
                     self.used_custom_layers[op +
                                             '_child_func'] = child_func_code
 
+    @print_mapping_info
     def elementwise_map(self, node):
         assert node.layer_type in self.elementwise_ops
         op_type = self.elementwise_ops[node.layer_type]
@@ -281,6 +277,7 @@ class ONNXOpMapper(OpMapper):
 
         if len(val_x_shape) < len(val_y_shape):
             val_x, val_y = val_y, val_x
+            val_y_shape, val_x_shape = val_x_shape, val_y_shape
 
         str_y_shape = ','.join(str(e) for e in val_y_shape)
         str_x_shape = ','.join(str(e) for e in val_x_shape)
@@ -299,22 +296,20 @@ class ONNXOpMapper(OpMapper):
                 'shape': val_y_reshaped,
                 'name': string(var_y_reshaped)
             }
-            node.fluid_code.add_layer('reshape',
-                                      inputs=val_y,
-                                      output=var_y_reshaped,
-                                      param_attr=attr_reshaped)
+            node.fluid_code.add_layer(
+                'reshape',
+                inputs=val_y,
+                output=var_y_reshaped,
+                param_attr=attr_reshaped)
             inputs = {'x': val_x, 'y': var_y_reshaped}
-            node.fluid_code.add_layer(op_type,
-                                      inputs=inputs,
-                                      output=node,
-                                      param_attr=attr)
+            node.fluid_code.add_layer(
+                op_type, inputs=inputs, output=node, param_attr=attr)
         else:
             inputs = {'x': val_x, 'y': val_y}
-            node.fluid_code.add_layer(op_type,
-                                      inputs=inputs,
-                                      output=node,
-                                      param_attr=attr)
+            node.fluid_code.add_layer(
+                op_type, inputs=inputs, output=node, param_attr=attr)
 
+    @print_mapping_info
     def place_holder(self, node):
         self.input_shapes.append(node.out_shapes[0])
 
@@ -331,11 +326,10 @@ class ONNXOpMapper(OpMapper):
             "append_batch_size": 'False'
         }
 
-        node.fluid_code.add_layer("data",
-                                  inputs=None,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            "data", inputs=None, output=node, param_attr=attr)
 
+    @print_mapping_info
     def create_parameter(self, node, parameter=None):
         if parameter is not None:
             node = parameter
@@ -348,13 +342,24 @@ class ONNXOpMapper(OpMapper):
             'dtype': string(dtype),
             'shape': shape,
             'name': string(node.layer_name),
-            'attr': string(node.layer_name),
             'default_initializer': 'Constant(0.0)'
         }
-        node.fluid_code.add_layer("create_parameter",
-                                  inputs=None,
-                                  output=node,
-                                  param_attr=attr)
+        if dtype == 'bool':
+            attr['dtype'] = string('int64')
+            node.fluid_code.add_layer(
+                "create_parameter", inputs=None, output=node, param_attr=attr)
+            node.fluid_code.add_layer(
+                "cast",
+                inputs=node,
+                output=node,
+                param_attr={'dtype': string('bool')})
+        elif dtype == 'uint8':
+            attr['dtype'] = string('float32')
+            node.fluid_code.add_layer(
+                "create_parameter", inputs=None, output=node, param_attr=attr)
+        else:
+            node.fluid_code.add_layer(
+                "create_parameter", inputs=None, output=node, param_attr=attr)
 
     def _pad_if_asymmetric(self, node, pads, val_name):  # pads: SSEE
         assert len(pads) & 1 == 0
@@ -371,41 +376,13 @@ class ONNXOpMapper(OpMapper):
 
     def _interpolate(self, node):
         val_x = self.graph.get_input_node(node, idx=0, copy=True)
-        val_scales = self.graph.get_input_node(node, idx=1, copy=True)
-        val_y = self.graph.get_node(node.layer.output[0], copy=True)
-        out_shape = val_y.out_shapes[0]
-        if out_shape is not None:
-            assert len(out_shape) == 4, 'only 4-D Tensor as X and Y supported'
-            out_shape = out_shape[2:]
+        if node.layer_type == 'Resize':
+            val_scales = self.graph.get_input_node(node, idx=2, copy=True)
+        elif node.layer_type == 'Upsample':
+            val_scales = self.graph.get_input_node(node, idx=1, copy=True)
 
-        scales = _const_weight_or_none(val_scales)
-
-        if isinstance(val_scales, ONNXGraphNode):
-            scales, _, _ = self.get_dynamic_shape(val_scales.layer_name)
         attr = {'name': string(node.layer_name)}
-        use_scales = True
-        if scales is not None:
-            try:
-                assert len(scales) == 4, 'only 4-D Tensor as X and Y supported'
-                assert scales[0] == 1 and scales[
-                    1] == 1, 'only scale on (NC)HW supported'
-                assert scales[2] == scales[
-                    3], 'only aspect-ratio-invariant scale supported'
-            except:
-                use_scales = False
-        scale = scales[2] if scales else None
-        if scale is None:
-            assert out_shape, 'neither scales nor output shape is available'
-        else:
-            if out_shape is None:
-                in_shape = val_x.out_shapes[0]
-                assert in_shape is not None, 'out_shape required but not inferrable'
-                assert len(
-                    in_shape) == 4, 'only 4-D Tensor as X and Y supported'
-                out_shape = [in_shape[2] * scale, in_shape[3] * scale]
-
         mode = node.get_attr('mode', 'nearest')
-
         fluid_op = 'resize_{}'.format(mode)
         if 'linear' in mode:
             print(
@@ -413,16 +390,14 @@ class ONNXOpMapper(OpMapper):
             )
             fluid_op = 'resize_bilinear'
 
-        if use_scales and scale is not None:
-            attr['scale'] = scale
-        else:
-            attr['out_shape'] = out_shape
-
-        node.fluid_code.add_layer(fluid_op,
-                                  inputs=val_x,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            fluid_op,
+            inputs={'input': val_x,
+                    'scale': val_scales},
+            output=node,
+            param_attr=attr)
 
+    @print_mapping_info
     def RoiAlign(self, node):
         val_x = self.graph.get_input_node(node, idx=0, copy=True)
         val_rois = self.graph.get_input_node(node, idx=1, copy=True)
@@ -437,14 +412,14 @@ class ONNXOpMapper(OpMapper):
             'spatial_scale': spatial_scale,
             'sampling_ratio': sampling_ratio,
         }
-        node.fluid_code.add_layer('roi_align',
-                                  inputs={
-                                      'input': val_x,
-                                      'rois': val_rois
-                                  },
-                                  output=node,
-                                  param_attr=attr)
-
+        node.fluid_code.add_layer(
+            'roi_align',
+            inputs={'input': val_x,
+                    'rois': val_rois},
+            output=node,
+            param_attr=attr)
+
+    @print_mapping_info
     def MaxRoiPool(self, node):
         val_x = self.graph.get_input_node(node, idx=0, copy=True)
         val_rois = self.graph.get_input_node(node, idx=1, copy=True)
@@ -456,14 +431,14 @@ class ONNXOpMapper(OpMapper):
             'pooled_width': pooled_width,
             'spatial_scale': spatial_scale,
         }
-        node.fluid_code.add_layer('roi_pool',
-                                  inputs={
-                                      'input': val_x,
-                                      'rois': val_rois
-                                  },
-                                  output=node,
-                                  param_attr=attr)
-
+        node.fluid_code.add_layer(
+            'roi_pool',
+            inputs={'input': val_x,
+                    'rois': val_rois},
+            output=node,
+            param_attr=attr)
+
+    @print_mapping_info
     def Pad(self, node, op_independent=True):
         val_x = self.graph.get_input_node(node, idx=0, copy=True)
         pads = node.get_attr('pads')
@@ -499,44 +474,54 @@ class ONNXOpMapper(OpMapper):
         attr['paddings'] = paddings
         if op_independent:
             attr['name'] = string(node.layer_name)
-            node.fluid_code.add_layer(fluid_op,
-                                      inputs=val_x,
-                                      output=node,
-                                      param_attr=attr)
+            node.fluid_code.add_layer(
+                fluid_op, inputs=val_x, output=node, param_attr=attr)
         else:
             attr['name'] = string(node.layer_name + '_paded')
-            node.fluid_code.add_layer(fluid_op,
-                                      inputs=val_x,
-                                      output=node.layer_name + '_paded',
-                                      param_attr=attr)
+            node.fluid_code.add_layer(
+                fluid_op,
+                inputs=val_x,
+                output=node.layer_name + '_paded',
+                param_attr=attr)
             return node.layer_name + '_paded'
 
+    @print_mapping_info
     def Unsqueeze(self, node):
         val_x = self.graph.get_input_node(node, idx=0, copy=True)
         axes = node.get_attr('axes')
+        attr = {'axes': axes, 'name': string(node.layer_name)}
         if len(val_x.out_shapes[0]) == 0:
-            node.fluid_code.add_layer('assign',
-                                      inputs=val_x,
-                                      output=node,
-                                      param_attr=None)
+            if node.layer_name:
+                node.fluid_code.add_layer(
+                    'reshape',
+                    inputs=val_x,
+                    output=node,
+                    param_attr={'shape': [1]})
         else:
-            attr = {'axes': axes, 'name': string(node.layer_name)}
-            node.fluid_code.add_layer('unsqueeze',
-                                      inputs=val_x,
-                                      output=node,
-                                      param_attr=attr)
+            node.fluid_code.add_layer(
+                'unsqueeze', inputs=val_x, output=node, param_attr=attr)
 
+    @print_mapping_info
     def Shrink(self, node):
         val_x = self.graph.get_input_node(node, idx=0, copy=True)
         bias = node.get_attr('bias')
         lambd = node.get_attr('lambd')
         assert bias == 0.0, 'not support bias!=0'
         attr = {'threshold': lambd, 'name': node.layer_name}
-        node.fluid_code.add_layer('hard_shrink',
-                                  inputs=val_x,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            'hard_shrink', inputs=val_x, output=node, param_attr=attr)
 
+    def Greater(self, node):
+        val_x = self.graph.get_input_node(node, idx=0, copy=True)
+        val_y = self.graph.get_input_node(node, idx=1, copy=True)
+        node.fluid_code.add_layer(
+            'greater_than',
+            inputs={'x': val_x,
+                    'y': val_y},
+            output=node,
+            param_attr=None)
+
+    @print_mapping_info
     def Constant(self, node):
         val_output = self.graph.get_node(node.layer.output[0], copy=True)
 
@@ -552,11 +537,10 @@ class ONNXOpMapper(OpMapper):
             shape = val_output.out_shapes[0]
         if shape is None:
             shape = list(value.shape)
-            _logger.warning(
-                'in (Constant -> %s): '
-                'attribute "shape" of %s not inferred, '
-                'using value as 1-D tensor may lead to fails',
-                val_output.layer_name, val_output.layer_name)
+            _logger.warning('in (Constant -> %s): '
+                            'attribute "shape" of %s not inferred, '
+                            'using value as 1-D tensor may lead to fails',
+                            val_output.layer_name, val_output.layer_name)
 
         if len(value) == 1:
             value = value.tolist()
@@ -565,31 +549,45 @@ class ONNXOpMapper(OpMapper):
             if dtype.name == 'int64':
                 dtype = 'int32'
             attr = {'shape': shape, 'dtype': string(dtype), 'value': value}
-            node.fluid_code.add_layer('fill_constant',
-                                      inputs=None,
-                                      output=node,
-                                      param_attr=attr)
+            node.fluid_code.add_layer(
+                'fill_constant', inputs=None, output=node, param_attr=attr)
         else:
+            if dtype.name == 'uint8':
+                dtype = 'int64'
             value = np.reshape(value, shape)
             self.weights[node.layer_name] = value
             attr = {
                 'dtype': string(dtype),
                 'shape': shape,
                 'name': string(node.layer_name),
-                'attr': string(node.layer_name),
                 'default_initializer': 'Constant(0.0)'
             }
-            node.fluid_code.add_layer("create_parameter",
-                                      inputs=None,
-                                      output=node,
-                                      param_attr=attr)
+            node.fluid_code.add_layer(
+                "create_parameter", inputs=None, output=node, param_attr=attr)
 
+    @print_mapping_info
     def Resize(self, node):
         self._interpolate(node)
 
+    @print_mapping_info
     def Upsample(self, node):
         self._interpolate(node)
 
+    @print_mapping_info
+    def InstanceNormalization(self, node):
+        val_x = self.graph.get_input_node(node, idx=0, copy=True)
+        val_scale = self.graph.get_input_node(node, idx=1, copy=True)
+        val_b = self.graph.get_input_node(node, idx=2, copy=True)
+        epsilon = node.get_attr('epsilon', 1e-5)
+        attr = {
+            'epsilon': epsilon,
+            'param_attr': string(val_scale.layer_name),
+            'bias_attr': string(val_b.layer_name)
+        }
+        node.fluid_code.add_layer(
+            "instance_norm", inputs=val_x, output=node, param_attr=attr)
+
+    @print_mapping_info
     def Expand(self, node):
         val_x = self.graph.get_input_node(node, idx=0, copy=True)
         val_shape = self.graph.get_input_node(node, idx=1, copy=True)
@@ -603,159 +601,225 @@ class ONNXOpMapper(OpMapper):
 
         name_ones = node.layer_name + '_ones'
         attr_ones = {'shape': out_shape, 'dtype': string(val_x_dtype)}
-        node.fluid_code.add_layer('ones',
-                                  inputs=None,
-                                  output=name_ones,
-                                  param_attr=attr_ones)
+        node.fluid_code.add_layer(
+            'ones', inputs=None, output=name_ones, param_attr=attr_ones)
         inputs = {'x': name_ones, 'y': val_x}
         attr = {'name': string(node.layer_name)}
-        node.fluid_code.add_layer('elementwise_mul',
-                                  inputs=inputs,
-                                  output=node.layer_name,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            'elementwise_mul',
+            inputs=inputs,
+            output=node.layer_name,
+            param_attr=attr)
 
+    @print_mapping_info
     def Gather(self, node):
         val_x = self.graph.get_input_node(node, idx=0, copy=True)
         indices = self.graph.get_input_node(node, idx=1, copy=True)
         indices_shape = indices.out_shapes[0]
         axis = node.get_attr('axis', 0)
-        assert len(
-            indices_shape) <= 2, "Gather op don't support dim of indice >2 "
+        #assert len(
+        #    indices_shape) <= 2, "Gather op don't support dim of indice >2 "
         if axis == 0 and len(indices_shape) <= 1:
-            node.fluid_code.add_layer('gather',
-                                      inputs={
-                                          'input': val_x,
-                                          'index': indices
-                                      },
-                                      output=node,
-                                      param_attr=None)
+            node.fluid_code.add_layer(
+                'gather',
+                inputs={'input': val_x,
+                        'index': indices},
+                output=node,
+                param_attr=None)
         elif axis > 0 and len(indices_shape) <= 1:
             perm = list(range(len(val_x.out_shapes[0])))
             perm = [axis] + perm[:axis] + perm[axis + 1:]
             attr_trans = {'perm': perm}
             name_trans = val_x.layer_name + '_trans'
-            node.fluid_code.add_layer('transpose',
-                                      inputs=val_x,
-                                      output=name_trans,
-                                      param_attr=attr_trans)
-            node.fluid_code.add_layer('gather',
-                                      inputs={
-                                          'input': name_trans,
-                                          'index': indices
-                                      },
-                                      output=node,
-                                      param_attr=None)
-            node.fluid_code.add_layer('transpose',
-                                      inputs=node,
-                                      output=node,
-                                      param_attr=attr_trans)
-        elif len(indices_shape) > 1:
+            node.fluid_code.add_layer(
+                'transpose',
+                inputs=val_x,
+                output=name_trans,
+                param_attr=attr_trans)
+            node.fluid_code.add_layer(
+                'gather',
+                inputs={'input': name_trans,
+                        'index': indices},
+                output=node,
+                param_attr=None)
+            node.fluid_code.add_layer(
+                'transpose', inputs=node, output=node, param_attr=attr_trans)
+        elif axis == 0 and len(indices_shape) > 1:
+            if val_x.out_shapes[0] is not None and isinstance(
+                    val_x, ONNXGraphDataNode):
+                node.fluid_code.add_layer(
+                    'embedding',
+                    inputs=indices,
+                    output=node,
+                    use_fluid=True,
+                    param_attr={
+                        'param_attr': string(val_x.layer_name),
+                        'size': val_x.out_shapes[0]
+                    })
+            else:
+                from functools import reduce
+                #indices_shape = [1,7]
+                reshape_shape = reduce(lambda x, y: x * y, indices_shape)
+                indices_reshape = indices.layer_name + '_shape'
+                node.fluid_code.add_layer(
+                    'reshape',
+                    inputs=indices,
+                    output=indices_reshape,
+                    param_attr={'shape': [reshape_shape, ]})
+
+                perm = list(range(len(val_x.out_shapes[0])))
+                node.fluid_code.add_layer(
+                    'gather',
+                    inputs={'input': val_x,
+                            'index': indices_reshape},
+                    output=node,
+                    param_attr=None)
+                val_x_shape = val_x.out_shapes[0]
+                reshaped_shape = []
+                for i in perm:
+                    reshaped_shape.append(indices_shape[i])
+                for i in val_x_shape[:axis] + val_x_shape[axis + 1:]:
+                    reshaped_shape.append(i)
+                node.fluid_code.add_layer(
+                    'reshape',
+                    inputs=node,
+                    output=node,
+                    param_attr={'shape': reshaped_shape})
+        elif axis > 0 and len(indices_shape) > 1:
             from functools import reduce
             reshape_shape = reduce(lambda x, y: x * y, indices_shape)
-            node.fluid_code.add_layer('reshape',
-                                      inputs=indices,
-                                      output=indices,
-                                      param_attr={'shape': [
-                                          reshape_shape,
-                                      ]})
+            indices_reshape = indices.layer_name + '_shape'
+            node.fluid_code.add_layer(
+                'reshape',
+                inputs=indices,
+                output=indices_reshape,
+                param_attr={'shape': [reshape_shape, ]})
 
             perm = list(range(len(val_x.out_shapes[0])))
             perm = [axis] + perm[:axis] + perm[axis + 1:]
             attr_trans = {'perm': perm}
             name_trans = val_x.layer_name + '_trans'
-            node.fluid_code.add_layer('transpose',
-                                      inputs=val_x,
-                                      output=name_trans,
-                                      param_attr=attr_trans)
-            node.fluid_code.add_layer('gather',
-                                      inputs={
-                                          'input': name_trans,
-                                          'index': indices
-                                      },
-                                      output=node,
-                                      param_attr=None)
-            node.fluid_code.add_layer('transpose',
-                                      inputs=node,
-                                      output=node,
-                                      param_attr=attr_trans)
+            node.fluid_code.add_layer(
+                'transpose',
+                inputs=val_x,
+                output=name_trans,
+                param_attr=attr_trans)
+            node.fluid_code.add_layer(
+                'gather',
+                inputs={'input': name_trans,
+                        'index': indices_reshape},
+                output=node,
+                param_attr=None)
+            node.fluid_code.add_layer(
+                'transpose', inputs=node, output=node, param_attr=attr_trans)
             val_x_shape = val_x.out_shapes[0]
             reshaped_shape = []
             for i in perm:
                 reshaped_shape.append(indices_shape[i])
             for i in val_x_shape[:axis] + val_x_shape[axis + 1:]:
                 reshaped_shape.append(i)
-            node.fluid_code.add_layer('reshape',
-                                      inputs=node,
-                                      output=node,
-                                      param_attr={'shape': reshaped_shape})
-
+            node.fluid_code.add_layer(
+                'reshape',
+                inputs=node,
+                output=node,
+                param_attr={'shape': reshaped_shape})
+
+    @print_mapping_info
+    def Range(self, node):
+        val_start = self.graph.get_input_node(node, idx=0, copy=True)
+        val_limit = self.graph.get_input_node(node, idx=1, copy=True)
+        val_delta = self.graph.get_input_node(node, idx=2, copy=True)
+        dtype = val_start.dtype
+        inputs = {'start': val_start, 'end': val_limit, 'step': val_delta}
+        node.fluid_code.add_layer(
+            'range',
+            inputs=inputs,
+            output=node,
+            param_attr={'dtype': string(dtype)})
+
+    @print_mapping_info
     def Slice(self, node):
         val_x = self.graph.get_input_node(node, idx=0, copy=True)
         starts, ends, axes, steps = None, None, None, None
+        attr = {}
         if len(node.inputs) > 1:
             starts = self.graph.get_input_node(node, idx=1, copy=True)
             ends = self.graph.get_input_node(node, idx=2, copy=True)
             if len(node.inputs) > 3:
                 axes = self.graph.get_input_node(node, idx=3, copy=True)
-                self.omit_nodes.append(axes.layer_name)
                 axes = _const_weight_or_none(axes)
             if len(node.inputs) > 4:
                 steps = self.graph.get_input_node(node, idx=4, copy=True)
-                self.omit_nodes.append(steps.layer_name)
                 steps = _const_weight_or_none(steps)
-
-            self.omit_nodes.append(starts.layer_name)
-            self.omit_nodes.append(ends.layer_name)
-            starts = _const_weight_or_none(starts).copy()
-            ends = _const_weight_or_none(ends).copy()
+                if steps is not None:
+                    assert steps == 1, "Only support convert op:Slice, which attribute:steps == 1"
+            attr = {
+                "axes": axes,
+                "starts": starts.layer_name,
+                "ends": ends.layer_name
+            }
+            starts_value = _const_weight_or_none(starts)
+            ends_value = _const_weight_or_none(ends)
+            if starts_value is not None and ends_value is not None:
+                self.omit_nodes.append(starts.layer_name)
+                self.omit_nodes.append(ends.layer_name)
+                ends_value = ends_value.copy()
+                for idx in range(len(ends_value)):
+                    if ends_value[idx] > 2**31 - 1:
+                        ends_value[idx] = 2**31 - 1
+                attr = {
+                    "axes": axes,
+                    "starts": starts_value,
+                    "ends": ends_value
+                }
+            else:
+                if starts.dtype != 'int32':
+                    node.fluid_code.add_layer(
+                        'cast',
+                        inputs=starts,
+                        output=starts,
+                        param_attr={'dtype': string('int32')})
+                if ends.dtype != 'int32':
+                    node.fluid_code.add_layer(
+                        'cast',
+                        inputs=ends,
+                        output=ends,
+                        param_attr={'dtype': string('int32')})
         else:
             starts = node.get_attr('starts')
             ends = node.get_attr('ends')
             axes = node.get_attr('axes')
+            for idx in range(len(ends)):
+                if ends[idx] > 2**31 - 1:
+                    ends[idx] = 2**31 - 1
+            attr = {"axes": axes, "starts": starts, "ends": ends}
 
-        val_y = self.graph.get_node(node.layer.output[0], copy=True)
-
-        shape = val_x.out_shapes[0]
-
-        if shape is not None:
-            for idx, value in enumerate(starts):
-                if value > shape[axes[idx]]:
-                    starts[idx] = shape[axes[idx]]
-            for idx, value in enumerate(ends):
-                if value > shape[axes[idx]]:
-                    ends[idx] = shape[axes[idx]]
-        attr = {"axes": axes, "starts": starts, "ends": ends}
-        node.fluid_code.add_layer('slice',
-                                  inputs=val_x,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            'slice', inputs=val_x, output=node, param_attr=attr)
 
+    @print_mapping_info
     def ConstantOfShape(self, node):
         val_shape = self.graph.get_input_node(node, idx=0, copy=True)
         val_y = self.graph.get_node(node.layer.output[0], copy=True)
-        shape = _const_weight_or_none(val_shape)
-
-        if shape is None:
-            shape = node.out_shapes[0]
-
-        assert shape is not None, (
-            'given shape is neither const value nor deductible from output, '
-            'this is not supported')
 
         value = node.get_attr('value')
         dtype = value.dtype
         value = value.tolist()
+        assert len(value) == 1, ('given value not Scalar, shape of value > 1, '
+                                 'this is not supported')
         if len(value) == 1:
-            shape = [1]
             value = value[0]
             if dtype.name == 'int64':
                 dtype = 'int32'
-            attr = {'shape': shape, 'dtype': string(dtype), 'value': value}
-            node.fluid_code.add_layer('fill_constant',
-                                      inputs=None,
-                                      output=node,
-                                      param_attr=attr)
+            attr = {
+                'shape': val_shape.layer_name,
+                'dtype': string(dtype),
+                'value': value
+            }
+            node.fluid_code.add_layer(
+                'fill_constant', inputs=None, output=node, param_attr=attr)
 
+    @print_mapping_info
     def Split(self, node):
         val_x = self.graph.get_input_node(node, idx=0, copy=True)
         val_y = self.graph.get_node(node.layer.output[0], copy=True)
@@ -769,52 +833,56 @@ class ONNXOpMapper(OpMapper):
             'name': string(node.layer_name)
         }
 
-        node.fluid_code.add_layer('split',
-                                  inputs=val_x,
-                                  output=val_y,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            'split', inputs=val_x, output=val_y, param_attr=attr)
 
+    @print_mapping_info
     def Reshape(self, node):
         val_x = self.graph.get_input_node(node, idx=0, copy=True)
         val_shape = self.graph.get_input_node(node, idx=1, copy=True)
         val_reshaped = self.graph.get_node(node.layer.output[0], copy=True)
-        shape = None
-
-        if isinstance(val_shape, ONNXGraphDataNode):
-            self.omit_nodes.append(val_shape.layer_name)
-
-        attr = {'name': string(node.layer_name)}
-        # catch dynamic graph shape
-        if isinstance(val_shape, ONNXGraphNode):
-            shape, _, _ = self.get_dynamic_shape(val_shape.layer_name)
-            if val_shape.dtype == 'int64':
-                val_shape_cast = val_shape.layer_name + '_cast'
-                node.fluid_code.add_layer('cast',
-                                          inputs=val_shape,
-                                          output=val_shape_cast,
-                                          param_attr={'dtype': string('int32')})
-
-                attr['actual_shape'] = val_shape_cast
-            else:
-                attr['actual_shape'] = val_shape
-
-        if shape is None:
-            shape = val_reshaped.out_shapes[0]
-
-        if shape is None:
-            shape = [1, -1]
-            _logger.warning(
-                'in %s(%s -> Reshape -> %s): '
-                'input "shape" not inferred, use [1, -1] as dummy value, '
-                'the behavior of Paddle fluid maybe undefined', node.layer_name,
-                val_x.layer_name, val_reshaped.layer_name)
-
-        attr['shape'] = shape
-        node.fluid_code.add_layer('reshape',
-                                  inputs=val_x,
-                                  output=node,
-                                  param_attr=attr)
-
+        attr = {}
+        shape_value = _const_weight_or_none(val_shape)
+        shape_dims = len(val_shape.out_shapes[0])
+
+        if shape_value is not None:
+            node.fluid_code.add_layer(
+                'reshape',
+                inputs={'x': val_x},
+                output=node,
+                param_attr={'shape': shape_value.tolist()})
+        elif val_shape.dtype == 'int64':
+            val_shape_cast = val_shape.layer_name + '_cast'
+            node.fluid_code.add_layer(
+                'cast',
+                inputs=val_shape,
+                output=val_shape_cast,
+                param_attr={'dtype': string('int32')})
+            node.fluid_code.add_layer(
+                'reshape',
+                inputs=val_shape_cast,
+                output=val_shape_cast,
+                param_attr={'shape': val_shape.out_shapes[0]})
+            node.fluid_code.add_layer(
+                'reshape',
+                inputs={'x': val_x,
+                        'shape': val_shape_cast},
+                output=node,
+                param_attr=attr)
+        else:
+            node.fluid_code.add_layer(
+                'reshape',
+                inputs=val_shape,
+                output=val_shape,
+                param_attr={'shape': val_shape.out_shapes[0]})
+            node.fluid_code.add_layer(
+                'reshape',
+                inputs={'x': val_x,
+                        'shape': val_shape},
+                output=node,
+                param_attr=attr)
+
+    @print_mapping_info
     def Cast(self, node):
         val_input = self.graph.get_input_node(node, idx=0, copy=True)
         val_output = self.graph.get_node(node.layer.output[0], copy=True)
@@ -827,11 +895,10 @@ class ONNXOpMapper(OpMapper):
         if output_dtype:
             assert dtype == output_dtype, 'dtype of to unmatches output'
         attr = {'dtype': string(dtype)}
-        node.fluid_code.add_layer('cast',
-                                  inputs=val_input,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            'cast', inputs=val_input, output=node, param_attr=attr)
 
+    @print_mapping_info
     def AveragePool(self, node):
         val_x = self.graph.get_input_node(node, idx=0, copy=True)
 
@@ -865,11 +932,10 @@ class ONNXOpMapper(OpMapper):
             "name": string(node.layer_name)
         }
 
-        node.fluid_code.add_layer(fluid_op,
-                                  inputs=val_x,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            fluid_op, inputs=val_x, output=node, param_attr=attr)
 
+    @print_mapping_info
     def Concat(self, node):
         inputs = []
         for i in range(len(node.layer.input)):
@@ -880,20 +946,18 @@ class ONNXOpMapper(OpMapper):
                 inputs.append(ipt.layer_name)
         axis = node.get_attr('axis')
         attr = {'axis': axis}
-        node.fluid_code.add_layer('concat',
-                                  inputs=inputs,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            'concat', inputs=inputs, output=node, param_attr=attr)
 
+    @print_mapping_info
     def Flatten(self, node):
         val_x = self.graph.get_input_node(node, idx=0, copy=True)
         axis = node.get_attr('axis', 1)
         attr = {"axis": str(axis), "name": string(node.layer_name)}
-        node.fluid_code.add_layer('flatten',
-                                  inputs=val_x,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            'flatten', inputs=val_x, output=node, param_attr=attr)
 
+    @print_mapping_info
     def Gemm(self, node):
         val_a = self.graph.get_input_node(node, idx=0, copy=True)
         val_b = self.graph.get_input_node(node, idx=1, copy=True)
@@ -911,39 +975,46 @@ class ONNXOpMapper(OpMapper):
             "alpha": alpha,
             "name": string(val_mm)
         }
-        node.fluid_code.add_layer('matmul',
-                                  inputs=matmul_inputs,
-                                  output=val_mm,
-                                  param_attr=attr_matmul)
+        node.fluid_code.add_layer(
+            'matmul',
+            inputs=matmul_inputs,
+            output=val_mm,
+            param_attr=attr_matmul)
 
         if beta != 0:
             if beta == 1.:
                 add_inputs = {"x": val_mm, "y": val_c}
                 attr = {"name": string(node.layer_name)}
-                node.fluid_code.add_layer("elementwise_add",
-                                          inputs=add_inputs,
-                                          output=node,
-                                          param_attr=attr)
+                node.fluid_code.add_layer(
+                    "elementwise_add",
+                    inputs=add_inputs,
+                    output=node,
+                    param_attr=attr)
             else:
                 var_beta = node.layer_name + '_beta'
                 matmul_beta_inputs = {"x": val_c, "y": var_beta}
-                node.fluid_code.add_layer("Constant",
-                                          inputs=matmul_beta_inputs,
-                                          output=var_beta,
-                                          param_attr={'value': beta})
+                node.fluid_code.add_layer(
+                    "Constant",
+                    inputs=matmul_beta_inputs,
+                    output=var_beta,
+                    param_attr={'value': beta})
 
                 add_inputs = {"x": val_mm, "y": var_beta}
                 attr = {"name": string(node.layer_name)}
-                node.fluid_code.add_layer("elementwise_add",
-                                          inputs=add_inputs,
-                                          output=node,
-                                          param_attr=attr)
+                node.fluid_code.add_layer(
+                    "elementwise_add",
+                    inputs=add_inputs,
+                    output=node,
+                    param_attr=attr)
 
+    @print_mapping_info
     def Sum(self, node):
         val_inps = node.layer.input
         inputs = {
-            "x": self.graph.get_input_node(node, idx=0, copy=True),
-            "y": self.graph.get_input_node(node, idx=1, copy=True),
+            "x": self.graph.get_input_node(
+                node, idx=0, copy=True),
+            "y": self.graph.get_input_node(
+                node, idx=1, copy=True),
         }
         node.fluid_code.add_layer("elementwise_add", inputs=inputs, output=node)
 
@@ -953,20 +1024,19 @@ class ONNXOpMapper(OpMapper):
                 "x": node.layer_name,
                 "y": y,
             }
-            node.fluid_code.add_layer("elementwise_add",
-                                      inputs=inputs,
-                                      output=node)
+            node.fluid_code.add_layer(
+                "elementwise_add", inputs=inputs, output=node)
 
+    @print_mapping_info
     def MatMul(self, node):
         val_x = self.graph.get_input_node(node, idx=0, copy=True)
         val_y = self.graph.get_input_node(node, idx=1, copy=True)
         inputs = {"x": val_x, "y": val_y}
         attr = {"name": string(node.layer_name)}
-        node.fluid_code.add_layer("matmul",
-                                  inputs=inputs,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            "matmul", inputs=inputs, output=node, param_attr=attr)
 
+    @print_mapping_info
     def BatchNormalization(self, node):
         val_x = self.graph.get_input_node(node, idx=0, copy=True)
         val_scale = self.graph.get_input_node(node, idx=1, copy=True)
@@ -996,28 +1066,25 @@ class ONNXOpMapper(OpMapper):
             "use_global_stats": spatial,
             "name": string(node.layer_name)
         }
-        node.fluid_code.add_layer("batch_norm",
-                                  inputs=val_x,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            "batch_norm", inputs=val_x, output=node, param_attr=attr)
 
+    @print_mapping_info
     def Transpose(self, node):
         val_x = self.graph.get_input_node(node, idx=0, copy=True)
         perm = node.get_attr('perm')
         attr = {'perm': perm, "name": string(node.layer_name)}
-        node.fluid_code.add_layer("transpose",
-                                  inputs=val_x,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            "transpose", inputs=val_x, output=node, param_attr=attr)
 
+    @print_mapping_info
     def Relu(self, node):
         val_x = self.graph.get_input_node(node, idx=0, copy=True)
         attr = {"name": string(node.layer_name)}
-        node.fluid_code.add_layer("relu",
-                                  inputs=val_x,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            "relu", inputs=val_x, output=node, param_attr=attr)
 
+    @print_mapping_info
     def PRelu(self, node):
         val_x = self.graph.get_input_node(node, idx=0, copy=True)
         val_slope = self.graph.get_input_node(node, idx=1, copy=True)
@@ -1032,133 +1099,130 @@ class ONNXOpMapper(OpMapper):
             "param_attr": string(val_slope.layer_name),
             'mode': string(mode)
         }
-        node.fluid_code.add_layer("prelu",
-                                  inputs=val_x,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            "prelu", inputs=val_x, output=node, param_attr=attr)
 
+    @print_mapping_info
     def Squeeze(self, node):
         val_x = self.graph.get_input_node(node, idx=0, copy=True)
         axes = node.get_attr('axes')
         attr = {'axes': axes, "name": string(node.layer_name)}
-        node.fluid_code.add_layer("squeeze",
-                                  inputs=val_x,
-                                  output=node,
-                                  param_attr=attr)
+        if len(val_x.out_shapes[0]) == 1:
+            node.fluid_code.add_layer(
+                "cast",
+                inputs=val_x,
+                output=node,
+                param_attr={'dtype': string(val_x.dtype)})
+        else:
+            node.fluid_code.add_layer(
+                "squeeze", inputs=val_x, output=node, param_attr=attr)
 
+    @print_mapping_info
     def Equal(self, node):
         val_x = self.graph.get_input_node(node, idx=0, copy=True)
         val_y = self.graph.get_input_node(node, idx=1, copy=True)
-        node.fluid_code.add_layer("equal",
-                                  inputs={
-                                      'x': val_x,
-                                      'y': val_y
-                                  },
-                                  output=node,
-                                  param_attr=None)
-
+        node.fluid_code.add_layer(
+            "equal",
+            inputs={'x': val_x,
+                    'y': val_y},
+            output=node,
+            param_attr=None)
+
+    @print_mapping_info
     def Where(self, node):
         condition = self.graph.get_input_node(node, idx=0, copy=True)
         val_x = self.graph.get_input_node(node, idx=1, copy=True)
         val_y = self.graph.get_input_node(node, idx=2, copy=True)
 
         not_condition = condition.layer_name + '_not'
-        node.fluid_code.add_layer("logical_not",
-                                  inputs=condition,
-                                  output=not_condition,
-                                  param_attr=None)
+        node.fluid_code.add_layer(
+            "logical_not",
+            inputs=condition,
+            output=not_condition,
+            param_attr=None)
         cast_not_condition = not_condition + '_cast'
-        node.fluid_code.add_layer("cast",
-                                  inputs=not_condition,
-                                  output=cast_not_condition,
-                                  param_attr={'dtype': string(val_x.dtype)})
+        node.fluid_code.add_layer(
+            "cast",
+            inputs=not_condition,
+            output=cast_not_condition,
+            param_attr={'dtype': string(val_x.dtype)})
         cast_condition = condition.layer_name + '_cast'
-        node.fluid_code.add_layer("cast",
-                                  inputs=condition,
-                                  output=cast_condition,
-                                  param_attr={'dtype': string(val_x.dtype)})
+        node.fluid_code.add_layer(
+            "cast",
+            inputs=condition,
+            output=cast_condition,
+            param_attr={'dtype': string(val_x.dtype)})
         mul_val_x = val_x.layer_name + '_mul'
-        node.fluid_code.add_layer("elementwise_mul",
-                                  inputs={
-                                      'x': val_x,
-                                      'y': cast_condition
-                                  },
-                                  output=mul_val_x,
-                                  param_attr=None)
+        node.fluid_code.add_layer(
+            "elementwise_mul",
+            inputs={'x': val_x,
+                    'y': cast_condition},
+            output=mul_val_x,
+            param_attr=None)
 
         mul_val_y = val_y.layer_name + '_mul'
-        node.fluid_code.add_layer("elementwise_mul",
-                                  inputs={
-                                      'x': val_y,
-                                      'y': cast_not_condition
-                                  },
-                                  output=mul_val_y,
-                                  param_attr=None)
-
-        node.fluid_code.add_layer("elementwise_add",
-                                  inputs={
-                                      'x': mul_val_x,
-                                      'y': mul_val_y
-                                  },
-                                  output=node,
-                                  param_attr=None)
-
+        node.fluid_code.add_layer(
+            "elementwise_mul",
+            inputs={'x': val_y,
+                    'y': cast_not_condition},
+            output=mul_val_y,
+            param_attr=None)
+
+        node.fluid_code.add_layer(
+            "elementwise_add",
+            inputs={'x': mul_val_x,
+                    'y': mul_val_y},
+            output=node,
+            param_attr=None)
+
+    @print_mapping_info
     def NonZero(self, node):
         val_x = self.graph.get_input_node(node, idx=0, copy=True)
-        where_name = node.layer_name + '_where'
-        node.fluid_code.add_layer("where",
-                                  inputs=val_x.layer_name + '!=0',
-                                  output=where_name)
-        dims = len(val_x.out_shapes[0])
-        elements_count_val_x = reduce(lambda x, y: x * y, val_x.out_shapes[0])
-        flatten_names = []
-        for dim in range(dims):
-            slice_name = node.layer_name + '_slice' + str(dim)
-            flatten_name = node.layer_name + '_flatten' + str(dim)
-            flatten_names.append(flatten_name)
-            attr = {
-                'axes': list(range(dims)),
-                'starts': [0, dim],
-                'ends': [elements_count_val_x, dim + 1]
-            }
-            node.fluid_code.add_layer("slice",
-                                      inputs=where_name,
-                                      output=slice_name,
-                                      param_attr=attr)
-            node.fluid_code.add_layer("flatten",
-                                      inputs=slice_name,
-                                      output=flatten_name,
-                                      param_attr={'axis': 0})
-        node.fluid_code.add_layer("concat",
-                                  inputs=flatten_names,
-                                  output=node,
-                                  param_attr={'axis': 0})
-
+        val_x_dim = len(val_x.out_shapes[0])
+        print(val_x.layer_name, val_x.out_shapes[0])
+        if val_x_dim == 1:
+            node.fluid_code.add_layer("nonzero", inputs=val_x, output=val_x)
+            node.fluid_code.add_layer(
+                "transpose",
+                inputs=val_x,
+                output=node,
+                param_attr={'perm': [1, 0]})
+        if val_x_dim > 1:
+            node.fluid_code.add_layer("nonzero", inputs=val_x, output=val_x)
+            node.fluid_code.add_layer(
+                "split",
+                inputs=val_x,
+                output=val_x,
+                param_attr={'num_or_sections': 1,
+                            'dim': val_x_dim})
+            node.fluid_code.add_layer("concat", inputs=val_x, output=node)
+
+    @print_mapping_info
     def Identity(self, node):
         val_x = self.graph.get_input_node(node, idx=0, copy=True)
         node.fluid_code.add_layer("assign", inputs=val_x, output=node)
 
+    @print_mapping_info
     def Tile(self, node):
         val_x = self.graph.get_input_node(node, idx=0, copy=True)
         val_repeats = self.graph.get_input_node(node, idx=1, copy=True)
         repeats = _const_weight_or_none(val_repeats)
-        assert repeats is not None, 'for OP:Tile, only const repeats supported'
 
-        if isinstance(repeats, int):
+        if repeats is None:
+            repeats = val_repeats.layer_name
+        elif isinstance(repeats, int):
             repeats = [repeats]
 
         attr = {
             'expand_times': repeats,
             "name": string(node.layer_name),
         }
-        node.fluid_code.add_layer("expand",
-                                  inputs=val_x,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            "expand", inputs=val_x, output=node, param_attr=attr)
 
+    @print_mapping_info
     def MaxPool(self, node):
         val_x = self.graph.get_input_node(node, idx=0, copy=True)
-
         auto_pad = node.get_attr('auto_pad', 'NOTSET')
         assert node.get_attr(
             "dilations") is None, 'only dilations = 0 is supported'  # optional
@@ -1191,24 +1255,13 @@ class ONNXOpMapper(OpMapper):
             "name": string(node.layer_name),
             "exclusive": False
         }
-        node.fluid_code.add_layer(fluid_op,
-                                  inputs=val_x,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            fluid_op, inputs=val_x, output=node, param_attr=attr)
 
     def _global_pool(self, node):
         val_x = self.graph.get_input_node(node, idx=0, copy=True)
         val_y = self.graph.get_node(node.layer.output[0], copy=True)
-        input_shape = val_x.out_shapes[0]
-        output_shape = val_y.out_shapes[0]
-        assert input_shape is not None or output_shape is not None, 'poolnd not inferred'  # N
-        if input_shape:
-            poolnd = len(input_shape) - 2  # NC...
-        elif output_shape:
-            poolnd = len(output_shape) - 2  # NC...
-        assert 2 <= poolnd <= 3, 'only pool2d and pool3d is supported'
-        fluid_op = 'pool{}d'.format(poolnd)
-
+        fluid_op = 'pool2d'
         pool_type = None
         if node.layer.op_type == 'GlobalMaxPool':
             pool_type = 'max'
@@ -1220,17 +1273,18 @@ class ONNXOpMapper(OpMapper):
             "global_pooling": True,
             "name": string(node.layer_name)
         }
-        node.fluid_code.add_layer(fluid_op,
-                                  inputs=val_x,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            fluid_op, inputs=val_x, output=node, param_attr=attr)
 
+    @print_mapping_info
     def GlobalMaxPool(self, node):
         self._global_pool(node)
 
+    @print_mapping_info
     def GlobalAveragePool(self, node):
         self._global_pool(node)
 
+    @print_mapping_info
     def Conv(self, node):
         val_x = self.graph.get_input_node(node, idx=0, copy=True)
         val_w = self.graph.get_input_node(node, idx=1, copy=True)
@@ -1279,11 +1333,10 @@ class ONNXOpMapper(OpMapper):
             attr["bias_attr"] = string(val_b.layer_name)
         else:
             attr["bias_attr"] = False
-        node.fluid_code.add_layer(fluid_op,
-                                  inputs=val_x,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            fluid_op, inputs=val_x, output=node, param_attr=attr)
 
+    @print_mapping_info
     def ConvTranspose(self, node):
         val_x = self.graph.get_input_node(node, idx=0, copy=True)
         val_w = self.graph.get_input_node(node, idx=1, copy=True)
@@ -1314,11 +1367,11 @@ class ONNXOpMapper(OpMapper):
 
         output_size = [0, 0]
 
-        output_size[0] = (val_x.out_shapes[0][2] -
-                          1) * strides[0] - 2 * paddings[0] + dilations[0] * (
+        output_size[0] = (val_x.out_shapes[0][2] - 1
+                          ) * strides[0] - 2 * paddings[0] + dilations[0] * (
                               kernel_shape[0] - 1) + 1 + out_padding[0]
-        output_size[1] = (val_x.out_shapes[0][3] -
-                          1) * strides[1] - 2 * paddings[1] + dilations[1] * (
+        output_size[1] = (val_x.out_shapes[0][3] - 1
+                          ) * strides[1] - 2 * paddings[1] + dilations[1] * (
                               kernel_shape[1] - 1) + 1 + out_padding[1]
         attr = {
             'num_filters': num_out_channels,
@@ -1332,11 +1385,10 @@ class ONNXOpMapper(OpMapper):
             'bias_attr': None if val_b is None else string(val_b.layer_name),
             'name': string(node.layer_name),
         }
-        node.fluid_code.add_layer(fluid_op,
-                                  inputs=val_x,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            fluid_op, inputs=val_x, output=node, param_attr=attr)
 
+    @print_mapping_info
     def GRU(self, node):
         val_x = self.graph.get_input_node(node, idx=0, copy=True)
         val_w = self.graph.get_input_node(node, idx=1, copy=True)
@@ -1352,17 +1404,13 @@ class ONNXOpMapper(OpMapper):
         else:
             miss_arg_num += 1
         if num_ipt > 4 and node.layer.input[4] != '':
-            val_len = self.graph.get_input_node(node,
-                                                idx=4 - miss_arg_num,
-                                                copy=True)
+            val_len = self.graph.get_input_node(
+                node, idx=4 - miss_arg_num, copy=True)
         else:
             miss_arg_num += 1
         if num_ipt > 5 and node.layer.input[5] != '':
-            val_xh = self.graph.get_input_node(node,
-                                               idx=5 - miss_arg_num,
-                                               copy=True)
-
-        data, dtype, shape = self.get_dynamic_shape(val_x.layer_name)
+            val_xh = self.graph.get_input_node(
+                node, idx=5 - miss_arg_num, copy=True)
 
         x_shape = val_x.out_shapes[0]
 
@@ -1401,97 +1449,87 @@ class ONNXOpMapper(OpMapper):
         is_reverse = direction == 'reverse'
 
         var_x0 = node.layer_name + '_x0'
-        node.fluid_code.add_layer('squeeze',
-                                  inputs=val_x,
-                                  output=var_x0,
-                                  param_attr={
-                                      'axes': [1],
-                                      'name': string(var_x0)
-                                  })
+        node.fluid_code.add_layer(
+            'squeeze',
+            inputs=val_x,
+            output=var_x0,
+            param_attr={'axes': [1],
+                        'name': string(var_x0)})
 
         var_w0 = node.layer_name + '_w0'
-        node.fluid_code.add_layer('squeeze',
-                                  inputs=val_w,
-                                  output=var_w0,
-                                  param_attr={
-                                      'axes': [0],
-                                      'name': string(var_w0)
-                                  })
+        node.fluid_code.add_layer(
+            'squeeze',
+            inputs=val_w,
+            output=var_w0,
+            param_attr={'axes': [0],
+                        'name': string(var_w0)})
 
         var_fc = node.layer_name + '_fc'
         var_mm = (node.layer_name + '_mm') if val_b else var_fc
-        node.fluid_code.add_layer('matmul',
-                                  inputs={
-                                      'x': var_x0,
-                                      'y': var_w0
-                                  },
-                                  output=var_mm,
-                                  param_attr={
-                                      'transpose_x': 0,
-                                      'transpose_y': 1,
-                                      'name': string(var_mm)
-                                  })
+        node.fluid_code.add_layer(
+            'matmul',
+            inputs={'x': var_x0,
+                    'y': var_w0},
+            output=var_mm,
+            param_attr={
+                'transpose_x': 0,
+                'transpose_y': 1,
+                'name': string(var_mm)
+            })
 
         var_r0 = node.layer_name + '_r0'
-        node.fluid_code.add_layer('squeeze',
-                                  inputs=val_r,
-                                  output=var_r0,
-                                  param_attr={
-                                      'axes': [0],
-                                      'name': string(var_r0)
-                                  })
+        node.fluid_code.add_layer(
+            'squeeze',
+            inputs=val_r,
+            output=var_r0,
+            param_attr={'axes': [0],
+                        'name': string(var_r0)})
 
         var_r0t = node.layer_name + '_r0t'
 
-        node.fluid_code.add_layer('transpose',
-                                  inputs=var_r0,
-                                  output=var_r0t,
-                                  param_attr={
-                                      'perm': [1, 0],
-                                      'name': string(var_r0t)
-                                  })
+        node.fluid_code.add_layer(
+            'transpose',
+            inputs=var_r0,
+            output=var_r0t,
+            param_attr={'perm': [1, 0],
+                        'name': string(var_r0t)})
         if val_b:
             var_bi = node.layer_name + '_bi'
             var_bh = node.layer_name + '_bh'
-            node.fluid_code.add_layer('split',
-                                      inputs=val_b,
-                                      output=var_bi + ',' + var_bh,
-                                      param_attr={
-                                          'axis':
-                                          1,
-                                          'split':
-                                          [hidden_size * 3, hidden_size * 3],
-                                          'name':
-                                          string(node.layer_name + '.b/split')
-                                      })
+            node.fluid_code.add_layer(
+                'split',
+                inputs=val_b,
+                output=var_bi + ',' + var_bh,
+                param_attr={
+                    'axis': 1,
+                    'split': [hidden_size * 3, hidden_size * 3],
+                    'name': string(node.layer_name + '.b/split')
+                })
             var_bi0 = node.layer_name + '_bi0'
-            node.fluid_code.add_layer('squeeze',
-                                      inputs=var_bi,
-                                      output=var_bi0,
-                                      param_attr={
-                                          'axes': [0],
-                                          'name': string(var_bi0)
-                                      })
-
-            node.fluid_code.add_layer('elmentwise_add',
-                                      inputs=[var_mm, var_bi0],
-                                      output=var_fc,
-                                      param_attr={
-                                          'axes':
-                                          1,
-                                          'name':
-                                          string(node.layer_name + '.i/bias')
-                                      })
+            node.fluid_code.add_layer(
+                'squeeze',
+                inputs=var_bi,
+                output=var_bi0,
+                param_attr={'axes': [0],
+                            'name': string(var_bi0)})
+
+            node.fluid_code.add_layer(
+                'elmentwise_add',
+                inputs=[var_mm, var_bi0],
+                output=var_fc,
+                param_attr={
+                    'axes': 1,
+                    'name': string(node.layer_name + '.i/bias')
+                })
 
         if val_xh:
             var_xh0 = node.layer_name + '_xh0'
-            node.fluid_code.add_layer('squeeze',
-                                      inputs=val_xh,
-                                      output=var_xh0,
-                                      param_attr={
-                                          'axes': [1],
-                                          'name': string(var_xh0)
-                                      })
+            node.fluid_code.add_layer(
+                'squeeze',
+                inputs=val_xh,
+                output=var_xh0,
+                param_attr={'axes': [1],
+                            'name': string(var_xh0)})
         var_y00 = node.layer_name + '_y00'
 
         attr = {
@@ -1503,26 +1541,29 @@ class ONNXOpMapper(OpMapper):
             'param_attr': string(var_r0t),
             'bias_attr': string(var_bh) if val_b else False,
         }
-        node.fluid_code.add_layer('dynamic_gru',
-                                  inputs=var_fc + ',' + str(hidden_size),
-                                  output=var_y00,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            'dynamic_gru',
+            inputs=var_fc + ',' + str(hidden_size),
+            output=var_y00,
+            param_attr=attr)
 
         num_opt = len(node.layer.output)
 
         if num_opt > 0 and node.layer.output[0] != '':
-            node.fluid_code.add_layer('unsqueeze',
-                                      inputs=var_y00,
-                                      output=node.layer.output[0],
-                                      param_attr={
-                                          'axes': [1, 1],
-                                          'name': string(node.layer.output[0])
-                                      })
+            node.fluid_code.add_layer(
+                'unsqueeze',
+                inputs=var_y00,
+                output=node.layer.output[0],
+                param_attr={
+                    'axes': [1, 1],
+                    'name': string(node.layer.output[0])
+                })
         if num_opt > 1 and node.layer.output[1] != '':
-            node.fluid_code.add_layer('unsqueeze',
-                                      inputs=var_y00,
-                                      output=node.layer.output[1],
-                                      param_attr={
-                                          'axes': [1, 1],
-                                          'name': string(node.layer.output[1])
-                                      })
+            node.fluid_code.add_layer(
+                'unsqueeze',
+                inputs=var_y00,
+                output=node.layer.output[1],
+                param_attr={
+                    'axes': [1, 1],
+                    'name': string(node.layer.output[1])
+                })

x2paddle/op_mapper/onnx_directly_map.py

@@ -28,61 +28,55 @@ default_op_mapping_field_values['FILL_NAME_FIELD'] = True
 default_op_mapping = {
     'Shape': ['shape', ['X'], ['Out']],
     'Clip': [
-        'clip', ['X'], ['Out'],
-        dict(),
-        dict(
-            min=(_np.asarray([255, 255, 127, 255],
-                             dtype=_np.uint8).view(_np.float32)[0]),
-            max=(_np.asarray([255, 255, 127, 127],
-                             dtype=_np.uint8).view(_np.float32)[0]),
-        )
+        'clip', ['X'], ['Out'], dict(), dict(
+            min=(_np.asarray(
+                [255, 255, 127, 255], dtype=_np.uint8).view(_np.float32)[0]),
+            max=(_np.asarray(
+                [255, 255, 127, 127], dtype=_np.uint8).view(_np.float32)[0]), )
     ],
     'Erf': ['erf', ['X'], ['Out']],
     'Ceil': ['ceil', ['X'], ['Out']],
     'ReduceMean': [
-        'reduce_mean', ['X'], ['Out'],
-        dict(axes='dim', keepdims='keep_dim'),
-        dict(keep_dim=1)
+        'reduce_mean', ['X'], ['Out'], dict(
+            axes='dim', keepdims='keep_dim'), dict(keep_dim=1)
     ],
     'ReduceSum': [
-        'reduce_sum', ['X'], ['Out'],
-        dict(axes='dim', keepdims='keep_dim'),
-        dict(keep_dim=1)
+        'reduce_sum', ['X'], ['Out'], dict(
+            axes='dim', keepdims='keep_dim'), dict(keep_dim=1)
     ],
     'ReduceMin': [
-        'reduce_min', ['X'], ['Out'],
-        dict(axes='dim', keepdims='keep_dim'),
-        dict(keep_dim=1)
+        'reduce_min', ['X'], ['Out'], dict(
+            axes='dim', keepdims='keep_dim'), dict(keep_dim=1)
+    ],
+    'ReduceMax': [
+        'reduce_max', ['X'], ['Out'], dict(
+            axes='dim', keepdims='keep_dim'), dict(keep_dim=1)
     ],
     #active function
     'Relu': ['relu', ['X'], ['Out']],
-    'LeakyRelu': ['leaky_relu', ['X'], ['Out'],
-                  dict(), dict(alpha=.01)],
-    'Elu': ['elu', ['X'], ['Out'],
-            dict(), dict(alpha=1.)],
+    'LeakyRelu': ['leaky_relu', ['X'], ['Out'], dict(), dict(alpha=.01)],
+    'Elu': ['elu', ['X'], ['Out'], dict(), dict(alpha=1.)],
     'ThresholdedRelu': [
-        'thresholded_relu', ['X'], ['Out'],
-        dict(alpha='threshold'),
+        'thresholded_relu', ['X'], ['Out'], dict(alpha='threshold'),
         dict(alpha=1.)
     ],
     'Tanh': ['tanh', ['X'], ['Out']],
     'Sigmoid': ['sigmoid', ['X'], ['Out']],
     'HardSigmoid': [
-        'hard_sigmoid', ['X'], ['Out'],
-        dict(alpha='slope', beta='offset'),
-        dict(slope=.2, offset=.5)
+        'hard_sigmoid', ['X'], ['Out'], dict(
+            alpha='slope', beta='offset'), dict(
+                slope=.2, offset=.5)
     ],
     'Softsign': ['softsign', ['X'], ['Out']],
     'Softplus': ['softplus', ['X'], ['Out']],
     'Exp': ['exp', ['X'], ['Out']],
-    'Softmax': ['softmax', ['X'], ['Out'],
-                dict(), dict(axis=1)],
+    'Softmax': ['softmax', ['X'], ['Out'], dict(), dict(axis=1)],
     'Sqrt': ['sqrt', ['X'], ['Out']],
     'Floor': ['floor', ['X'], ['Out']],
     'Abs': ['abs', ['X'], ['Out']],
 }
 
 default_ioa_constraint = {
-    'Gather':
-    [(lambda i, o, a: a.get('axis', 0) == 0, 'only axis = 0 is supported')],
+    'Gather': [(lambda i, o, a: a.get('axis', 0) == 0,
+                'only axis = 0 is supported')],
 }

x2paddle/op_mapper/paddle_custom_layer/im2sequence.py

+import onnx
+import numpy as np
+from onnx import onnx_pb, helper
+
+im2seq_counter = 0
+
+
+def im2sequence(op, block):
+    global im2sequence_counter
+    n, c, h, w = block.var(op.input('X')[0]).shape
+    assert h > 0 and w > 0, "Only supported fixed input shape for im2sequence operator."
+    stride_h, stride_w = op.attr('strides')
+    paddings = op.attr('paddings')
+    assert op.attr(
+        'out_stride'
+    ) != 1, "Only out_stride==1 is supported for im2sequence operator."
+    h = h + paddings[0] + paddings[1]
+    w = w + paddings[1] + paddings[2]
+    kernel_h, kernel_w = op.attr('kernels')
+    out_h = 1 + (h - kernel_h + stride_h - 1) // stride_h
+    out_w = 1 + (w - kernel_w + stride_w - 1) // stride_w
+    h_steps = list()
+    for i in range(out_h):
+        h_steps.append([i * stride_h, i * stride_h + kernel_h])
+    w_steps = list()
+    for i in range(out_w):
+        w_steps.append([i * stride_w, i * stride_w + kernel_w])
+
+    nodes = list()
+    slice_blocks = list()
+    for i in range(out_h):
+        for j in range(out_w):
+            starts_name = "im2sequence.starts.{}.{}.{}".format(im2seq_counter,
+                                                               i, j)
+            starts_tensor = helper.make_tensor(
+                name=starts_name,
+                data_type=onnx_pb.TensorProto.INT64,
+                dims=[4],
+                vals=[0, 0, h_steps[i][0], w_steps[j][0]])
+            ends_name = "im2sequence.ends.{}.{}.{}".format(im2seq_counter, i, j)
+            ends_tensor = helper.make_tensor(
+                name=ends_name,
+                data_type=onnx_pb.TensorProto.INT64,
+                dims=[4],
+                vals=[999999, 999999, h_steps[i][1], w_steps[j][1]])
+            starts_node = helper.make_node(
+                'Constant',
+                inputs=[],
+                outputs=[starts_name],
+                value=starts_tensor)
+            ends_node = helper.make_node(
+                'Constant', inputs=[], outputs=[ends_name], value=ends_tensor)
+            nodes.extend([starts_node, ends_node])
+
+            slice_block_name = "im2sequence.slice.{}.{}.{}".format(
+                im2seq_counter, i, j)
+            slice_block_node = helper.make_node(
+                'Slice',
+                inputs=[op.input('X')[0], starts_name, ends_name],
+                outputs=[slice_block_name])
+            flatten_block_name = "im2sequence.flatten.{}.{}.{}".format(
+                im2seq_counter, i, j)
+            flatten_block_node = helper.make_node(
+                "Flatten",
+                inputs=[slice_block_name],
+                outputs=[flatten_block_name],
+                axis=0)
+            nodes.extend([slice_block_node, flatten_block_node])
+            slice_blocks.append(flatten_block_name)
+    concat_block_name = "im2sequence.concat_block.{}".format(im2seq_counter)
+    #    concat_block_node = helper.make_node("Concat", inputs=slice_blocks, outputs=[concat_block_name], axis=0)
+    concat_block_node = helper.make_node(
+        "Concat", inputs=slice_blocks, outputs=op.output('Out'), axis=0)
+    nodes.append(concat_block_node)
+    print("\n\n==========Importance Notice===========")
+    print(
+        "Since im2sequence operator is used in your paddlepaddle model, the translated onnx model only support input data with batch_size=1."
+    )
+    print("======================================\n")
+    return nodes

x2paddle/op_mapper/paddle_custom_layer/multiclass_nms.py

+#   Copyright (c) 2019  PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License"
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import math
+import sys
+import os
+import numpy as np
+import paddle.fluid.core as core
+import paddle.fluid as fluid
+import onnx
+import warnings
+from onnx import helper, onnx_pb
+
+
+def multiclass_nms(op, block):
+    """
+    Convert the paddle multiclass_nms to onnx op.
+    This op is get the select boxes from origin boxes.
+    """
+    inputs = dict()
+    outputs = dict()
+    attrs = dict()
+    for name in op.input_names:
+        inputs[name] = op.input(name)
+    for name in op.output_names:
+        outputs[name] = op.output(name)
+    for name in op.attr_names:
+        attrs[name] = op.attr(name)
+
+    result_name = outputs['Out'][0]
+    background = attrs['background_label']
+    normalized = attrs['normalized']
+    if normalized == False:
+        warnings.warn(
+            'The parameter normalized of multiclass_nms OP of Paddle is False, which has diff with ONNX. \
+                         Please set normalized=True in multiclass_nms of Paddle')
+
+    #convert the paddle attribute to onnx tensor
+    name_score_threshold = [outputs['Out'][0] + "@score_threshold"]
+    name_iou_threshold = [outputs['Out'][0] + "@iou_threshold"]
+    name_keep_top_k = [outputs['Out'][0] + '@keep_top_k']
+    name_keep_top_k_2D = [outputs['Out'][0] + '@keep_top_k_1D']
+
+    node_score_threshold = onnx.helper.make_node(
+        'Constant',
+        inputs=[],
+        outputs=name_score_threshold,
+        value=onnx.helper.make_tensor(
+            name=name_score_threshold[0] + "@const",
+            data_type=onnx.TensorProto.FLOAT,
+            dims=(),
+            vals=[float(attrs['score_threshold'])]))
+
+    node_iou_threshold = onnx.helper.make_node(
+        'Constant',
+        inputs=[],
+        outputs=name_iou_threshold,
+        value=onnx.helper.make_tensor(
+            name=name_iou_threshold[0] + "@const",
+            data_type=onnx.TensorProto.FLOAT,
+            dims=(),
+            vals=[float(attrs['nms_threshold'])]))
+
+    node_keep_top_k = onnx.helper.make_node(
+        'Constant',
+        inputs=[],
+        outputs=name_keep_top_k,
+        value=onnx.helper.make_tensor(
+            name=name_keep_top_k[0] + "@const",
+            data_type=onnx.TensorProto.INT64,
+            dims=(),
+            vals=[np.int64(attrs['keep_top_k'])]))
+
+    node_keep_top_k_2D = onnx.helper.make_node(
+        'Constant',
+        inputs=[],
+        outputs=name_keep_top_k_2D,
+        value=onnx.helper.make_tensor(
+            name=name_keep_top_k_2D[0] + "@const",
+            data_type=onnx.TensorProto.INT64,
+            dims=[1, 1],
+            vals=[np.int64(attrs['keep_top_k'])]))
+
+    # the paddle data format is x1,y1,x2,y2
+    kwargs = {'center_point_box': 0}
+
+    name_select_nms = [outputs['Out'][0] + "@select_index"]
+    node_select_nms= onnx.helper.make_node(
+        'NonMaxSuppression',
+        inputs=inputs['BBoxes'] + inputs['Scores'] + name_keep_top_k +\
+            name_iou_threshold + name_score_threshold,
+        outputs=name_select_nms)
+    # step 1 nodes select the nms class
+    node_list = [
+        node_score_threshold, node_iou_threshold, node_keep_top_k,
+        node_keep_top_k_2D, node_select_nms
+    ]
+
+    # create some const value to use
+    name_const_value = [result_name+"@const_0",
+        result_name+"@const_1",\
+        result_name+"@const_2",\
+        result_name+"@const_-1"]
+    value_const_value = [0, 1, 2, -1]
+    for name, value in zip(name_const_value, value_const_value):
+        node = onnx.helper.make_node(
+            'Constant',
+            inputs=[],
+            outputs=[name],
+            value=onnx.helper.make_tensor(
+                name=name + "@const",
+                data_type=onnx.TensorProto.INT64,
+                dims=[1],
+                vals=[value]))
+        node_list.append(node)
+
+    # Ine this code block, we will deocde the raw score data, reshape N * C * M to 1 * N*C*M
+    # and the same time, decode the select indices to 1 * D, gather the select_indices
+    outputs_gather_1 = [result_name + "@gather_1"]
+    node_gather_1 = onnx.helper.make_node(
+        'Gather',
+        inputs=name_select_nms + [result_name + "@const_1"],
+        outputs=outputs_gather_1,
+        axis=1)
+    node_list.append(node_gather_1)
+
+    outputs_squeeze_gather_1 = [result_name + "@sequeeze_gather_1"]
+    node_squeeze_gather_1 = onnx.helper.make_node(
+        'Squeeze',
+        inputs=outputs_gather_1,
+        outputs=outputs_squeeze_gather_1,
+        axes=[1])
+    node_list.append(node_squeeze_gather_1)
+
+    outputs_gather_2 = [result_name + "@gather_2"]
+    node_gather_2 = onnx.helper.make_node(
+        'Gather',
+        inputs=name_select_nms + [result_name + "@const_2"],
+        outputs=outputs_gather_2,
+        axis=1)
+    node_list.append(node_gather_2)
+
+    #slice the class is not 0
+    if background == 0:
+        outputs_nonzero = [result_name + "@nonzero"]
+        node_nonzero = onnx.helper.make_node(
+            'NonZero', inputs=outputs_squeeze_gather_1, outputs=outputs_nonzero)
+        node_list.append(node_nonzero)
+    else:
+        name_thresh = [result_name + "@thresh"]
+        node_thresh = onnx.helper.make_node(
+            'Constant',
+            inputs=[],
+            outputs=name_thresh,
+            value=onnx.helper.make_tensor(
+                name=name_thresh[0] + "@const",
+                data_type=onnx.TensorProto.INT32,
+                dims=[1],
+                vals=[-1]))
+        node_list.append(node_thresh)
+
+        outputs_cast = [result_name + "@cast"]
+        node_cast = onnx.helper.make_node(
+            'Cast', inputs=outputs_squeeze_gather_1, outputs=outputs_cast, to=6)
+        node_list.append(node_cast)
+
+        outputs_greater = [result_name + "@greater"]
+        node_greater = onnx.helper.make_node(
+            'Greater',
+            inputs=outputs_cast + name_thresh,
+            outputs=outputs_greater)
+        node_list.append(node_greater)
+
+        outputs_nonzero = [result_name + "@nonzero"]
+        node_nonzero = onnx.helper.make_node(
+            'NonZero', inputs=outputs_greater, outputs=outputs_nonzero)
+        node_list.append(node_nonzero)
+
+    outputs_gather_1_nonzero = [result_name + "@gather_1_nonzero"]
+    node_gather_1_nonzero = onnx.helper.make_node(
+        'Gather',
+        inputs=outputs_gather_1 + outputs_nonzero,
+        outputs=outputs_gather_1_nonzero,
+        axis=0)
+    node_list.append(node_gather_1_nonzero)
+
+    outputs_gather_2_nonzero = [result_name + "@gather_2_nonzero"]
+    node_gather_2_nonzero = onnx.helper.make_node(
+        'Gather',
+        inputs=outputs_gather_2 + outputs_nonzero,
+        outputs=outputs_gather_2_nonzero,
+        axis=0)
+    node_list.append(node_gather_2_nonzero)
+
+    # reshape scores N * C * M to (N*C*M) * 1
+    outputs_reshape_scores_rank1 = [result_name + "@reshape_scores_rank1"]
+    node_reshape_scores_rank1 = onnx.helper.make_node(
+        "Reshape",
+        inputs=inputs['Scores'] + [result_name + "@const_-1"],
+        outputs=outputs_reshape_scores_rank1)
+    node_list.append(node_reshape_scores_rank1)
+
+    # get the shape of scores
+    outputs_shape_scores = [result_name + "@shape_scores"]
+    node_shape_scores = onnx.helper.make_node(
+        'Shape', inputs=inputs['Scores'], outputs=outputs_shape_scores)
+    node_list.append(node_shape_scores)
+
+    # gather the index: 2 shape of scores
+    outputs_gather_scores_dim1 = [result_name + "@gather_scores_dim1"]
+    node_gather_scores_dim1 = onnx.helper.make_node(
+        'Gather',
+        inputs=outputs_shape_scores + [result_name + "@const_2"],
+        outputs=outputs_gather_scores_dim1,
+        axis=0)
+    node_list.append(node_gather_scores_dim1)
+
+    # mul class * M
+    outputs_mul_classnum_boxnum = [result_name + "@mul_classnum_boxnum"]
+    node_mul_classnum_boxnum = onnx.helper.make_node(
+        'Mul',
+        inputs=outputs_gather_1_nonzero + outputs_gather_scores_dim1,
+        outputs=outputs_mul_classnum_boxnum)
+    node_list.append(node_mul_classnum_boxnum)
+
+    # add class * M * index
+    outputs_add_class_M_index = [result_name + "@add_class_M_index"]
+    node_add_class_M_index = onnx.helper.make_node(
+        'Add',
+        inputs=outputs_mul_classnum_boxnum + outputs_gather_2_nonzero,
+        outputs=outputs_add_class_M_index)
+    node_list.append(node_add_class_M_index)
+
+    # Squeeze the indices to 1 dim
+    outputs_squeeze_select_index = [result_name + "@squeeze_select_index"]
+    node_squeeze_select_index = onnx.helper.make_node(
+        'Squeeze',
+        inputs=outputs_add_class_M_index,
+        outputs=outputs_squeeze_select_index,
+        axes=[0, 2])
+    node_list.append(node_squeeze_select_index)
+
+    # gather the data from flatten scores
+    outputs_gather_select_scores = [result_name + "@gather_select_scores"]
+    node_gather_select_scores = onnx.helper.make_node('Gather',
+        inputs=outputs_reshape_scores_rank1 + \
+            outputs_squeeze_select_index,
+        outputs=outputs_gather_select_scores,
+        axis=0)
+    node_list.append(node_gather_select_scores)
+
+    # get nums to input TopK
+    outputs_shape_select_num = [result_name + "@shape_select_num"]
+    node_shape_select_num = onnx.helper.make_node(
+        'Shape',
+        inputs=outputs_gather_select_scores,
+        outputs=outputs_shape_select_num)
+    node_list.append(node_shape_select_num)
+
+    outputs_gather_select_num = [result_name + "@gather_select_num"]
+    node_gather_select_num = onnx.helper.make_node(
+        'Gather',
+        inputs=outputs_shape_select_num + [result_name + "@const_0"],
+        outputs=outputs_gather_select_num,
+        axis=0)
+    node_list.append(node_gather_select_num)
+
+    outputs_unsqueeze_select_num = [result_name + "@unsqueeze_select_num"]
+    node_unsqueeze_select_num = onnx.helper.make_node(
+        'Unsqueeze',
+        inputs=outputs_gather_select_num,
+        outputs=outputs_unsqueeze_select_num,
+        axes=[0])
+    node_list.append(node_unsqueeze_select_num)
+
+    outputs_concat_topK_select_num = [result_name + "@conat_topK_select_num"]
+    node_conat_topK_select_num = onnx.helper.make_node(
+        'Concat',
+        inputs=outputs_unsqueeze_select_num + name_keep_top_k_2D,
+        outputs=outputs_concat_topK_select_num,
+        axis=0)
+    node_list.append(node_conat_topK_select_num)
+
+    outputs_cast_concat_topK_select_num = [
+        result_name + "@concat_topK_select_num"
+    ]
+    node_outputs_cast_concat_topK_select_num = onnx.helper.make_node(
+        'Cast',
+        inputs=outputs_concat_topK_select_num,
+        outputs=outputs_cast_concat_topK_select_num,
+        to=6)
+    node_list.append(node_outputs_cast_concat_topK_select_num)
+    # get min(topK, num_select)
+    outputs_compare_topk_num_select = [result_name + "@compare_topk_num_select"]
+    node_compare_topk_num_select = onnx.helper.make_node(
+        'ReduceMin',
+        inputs=outputs_cast_concat_topK_select_num,
+        outputs=outputs_compare_topk_num_select,
+        keepdims=0)
+    node_list.append(node_compare_topk_num_select)
+
+    # unsqueeze the indices to 1D tensor
+    outputs_unsqueeze_topk_select_indices = [
+        result_name + "@unsqueeze_topk_select_indices"
+    ]
+    node_unsqueeze_topk_select_indices = onnx.helper.make_node(
+        'Unsqueeze',
+        inputs=outputs_compare_topk_num_select,
+        outputs=outputs_unsqueeze_topk_select_indices,
+        axes=[0])
+    node_list.append(node_unsqueeze_topk_select_indices)
+
+    # cast the indices to INT64
+    outputs_cast_topk_indices = [result_name + "@cast_topk_indices"]
+    node_cast_topk_indices = onnx.helper.make_node(
+        'Cast',
+        inputs=outputs_unsqueeze_topk_select_indices,
+        outputs=outputs_cast_topk_indices,
+        to=7)
+    node_list.append(node_cast_topk_indices)
+
+    # select topk scores  indices
+    outputs_topk_select_topk_indices = [result_name + "@topk_select_topk_values",\
+        result_name + "@topk_select_topk_indices"]
+    node_topk_select_topk_indices = onnx.helper.make_node(
+        'TopK',
+        inputs=outputs_gather_select_scores + outputs_cast_topk_indices,
+        outputs=outputs_topk_select_topk_indices)
+    node_list.append(node_topk_select_topk_indices)
+
+    # gather topk label, scores, boxes
+    outputs_gather_topk_scores = [result_name + "@gather_topk_scores"]
+    node_gather_topk_scores = onnx.helper.make_node(
+        'Gather',
+        inputs=outputs_gather_select_scores +
+        [outputs_topk_select_topk_indices[1]],
+        outputs=outputs_gather_topk_scores,
+        axis=0)
+    node_list.append(node_gather_topk_scores)
+
+    outputs_gather_topk_class = [result_name + "@gather_topk_class"]
+    node_gather_topk_class = onnx.helper.make_node(
+        'Gather',
+        inputs=outputs_gather_1_nonzero +
+        [outputs_topk_select_topk_indices[1]],
+        outputs=outputs_gather_topk_class,
+        axis=1)
+    node_list.append(node_gather_topk_class)
+
+    # gather the boxes need to gather the boxes id, then get boxes
+    outputs_gather_topk_boxes_id = [result_name + "@gather_topk_boxes_id"]
+    node_gather_topk_boxes_id = onnx.helper.make_node(
+        'Gather',
+        inputs=outputs_gather_2_nonzero +
+        [outputs_topk_select_topk_indices[1]],
+        outputs=outputs_gather_topk_boxes_id,
+        axis=1)
+    node_list.append(node_gather_topk_boxes_id)
+
+    # squeeze the gather_topk_boxes_id to 1 dim
+    outputs_squeeze_topk_boxes_id = [result_name + "@squeeze_topk_boxes_id"]
+    node_squeeze_topk_boxes_id = onnx.helper.make_node(
+        'Squeeze',
+        inputs=outputs_gather_topk_boxes_id,
+        outputs=outputs_squeeze_topk_boxes_id,
+        axes=[0, 2])
+    node_list.append(node_squeeze_topk_boxes_id)
+
+    outputs_gather_select_boxes = [result_name + "@gather_select_boxes"]
+    node_gather_select_boxes = onnx.helper.make_node(
+        'Gather',
+        inputs=inputs['BBoxes'] + outputs_squeeze_topk_boxes_id,
+        outputs=outputs_gather_select_boxes,
+        axis=1)
+    node_list.append(node_gather_select_boxes)
+
+    # concat the final result
+    # before concat need to cast the class to float
+    outputs_cast_topk_class = [result_name + "@cast_topk_class"]
+    node_cast_topk_class = onnx.helper.make_node(
+        'Cast',
+        inputs=outputs_gather_topk_class,
+        outputs=outputs_cast_topk_class,
+        to=1)
+    node_list.append(node_cast_topk_class)
+
+    outputs_unsqueeze_topk_scores = [result_name + "@unsqueeze_topk_scores"]
+    node_unsqueeze_topk_scores = onnx.helper.make_node(
+        'Unsqueeze',
+        inputs=outputs_gather_topk_scores,
+        outputs=outputs_unsqueeze_topk_scores,
+        axes=[0, 2])
+    node_list.append(node_unsqueeze_topk_scores)
+
+    inputs_concat_final_results = outputs_cast_topk_class + outputs_unsqueeze_topk_scores +\
+        outputs_gather_select_boxes
+    outputs_concat_final_results = outputs['Out']
+    node_concat_final_results = onnx.helper.make_node(
+        'Concat',
+        inputs=inputs_concat_final_results,
+        outputs=outputs_concat_final_results,
+        axis=2)
+    node_list.append(node_concat_final_results)
+
+    return node_list

x2paddle/op_mapper/paddle_custom_layer/yolo_box.py

+import onnx
+import numpy as np
+from onnx import onnx_pb, helper
+
+
+def get_old_name(arg, name_prefix=''):
+    prefix_index = arg.find(name_prefix)
+
+    if prefix_index != -1:
+        last_prefix = arg[len(name_prefix):]
+    else:
+        last_prefix = arg
+    idx = last_prefix.find('@')
+    if idx != -1:
+        last_prefix = last_prefix[:idx]
+    return name_prefix + last_prefix
+
+
+def yolo_box(op, block):
+    inputs = dict()
+    outputs = dict()
+    attrs = dict()
+    for name in op.input_names:
+        inputs[name] = op.input(name)
+    for name in op.output_names:
+        outputs[name] = op.output(name)
+    for name in op.attr_names:
+        attrs[name] = op.attr(name)
+    model_name = outputs['Boxes'][0]
+    input_shape = block.vars[get_old_name(inputs['X'][0])].shape
+    image_size = inputs['ImgSize']
+    input_height = input_shape[2]
+    input_width = input_shape[3]
+
+    class_num = attrs['class_num']
+    anchors = attrs['anchors']
+    num_anchors = int(len(anchors)) // 2
+    downsample_ratio = attrs['downsample_ratio']
+    input_size = input_height * downsample_ratio
+    conf_thresh = attrs['conf_thresh']
+    conf_thresh_mat = np.ones([num_anchors * input_height *
+                               input_width]) * conf_thresh
+
+    node_list = []
+    im_outputs = []
+
+    x_shape = [1, num_anchors, 5 + class_num, input_height, input_width]
+    name_x_shape = [model_name + "@x_shape"]
+    node_x_shape = onnx.helper.make_node(
+        'Constant',
+        inputs=[],
+        outputs=name_x_shape,
+        value=onnx.helper.make_tensor(
+            name=name_x_shape[0] + "@const",
+            data_type=onnx.TensorProto.INT64,
+            dims=[5],
+            vals=x_shape))
+    node_list.append(node_x_shape)
+
+    outputs_x_reshape = [model_name + "@reshape"]
+    node_x_reshape = onnx.helper.make_node(
+        'Reshape', inputs=inputs['X'] + name_x_shape, outputs=outputs_x_reshape)
+    node_list.append(node_x_reshape)
+
+    outputs_x_transpose = [model_name + "@x_transpose"]
+    node_x_transpose = onnx.helper.make_node(
+        'Transpose',
+        inputs=outputs_x_reshape,
+        outputs=outputs_x_transpose,
+        perm=[0, 1, 3, 4, 2])
+    node_list.append(node_x_transpose)
+
+    range_x = []
+    range_y = []
+    for i in range(0, input_width):
+        range_x.append(i)
+    for j in range(0, input_height):
+        range_y.append(j)
+
+    name_range_x = [model_name + "@range_x"]
+    node_range_x = onnx.helper.make_node(
+        'Constant',
+        inputs=[],
+        outputs=name_range_x,
+        value=onnx.helper.make_tensor(
+            name=name_range_x[0] + "@const",
+            data_type=onnx.TensorProto.FLOAT,
+            dims=[input_width],
+            vals=range_x))
+    node_list.append(node_range_x)
+
+    name_range_y = [model_name + "@range_y"]
+    node_range_y = onnx.helper.make_node(
+        'Constant',
+        inputs=[],
+        outputs=name_range_y,
+        value=onnx.helper.make_tensor(
+            name=name_range_y[0] + "@const",
+            data_type=onnx.TensorProto.FLOAT,
+            dims=[input_height],
+            vals=range_y))
+    node_list.append(node_range_y)
+
+    range_x_new_shape = [1, input_width]
+    range_y_new_shape = [input_height, 1]
+
+    name_range_x_new_shape = [model_name + "@range_x_new_shape"]
+    node_range_x_new_shape = onnx.helper.make_node(
+        'Constant',
+        inputs=[],
+        outputs=name_range_x_new_shape,
+        value=onnx.helper.make_tensor(
+            name=name_range_x_new_shape[0] + "@const",
+            data_type=onnx.TensorProto.INT64,
+            dims=[len(range_x_new_shape)],
+            vals=range_x_new_shape))
+    node_list.append(node_range_x_new_shape)
+
+    name_range_y_new_shape = [model_name + "@range_y_new_shape"]
+    node_range_y_new_shape = onnx.helper.make_node(
+        'Constant',
+        inputs=[],
+        outputs=name_range_y_new_shape,
+        value=onnx.helper.make_tensor(
+            name=name_range_y_new_shape[0] + "@const",
+            data_type=onnx.TensorProto.INT64,
+            dims=[len(range_y_new_shape)],
+            vals=range_y_new_shape))
+    node_list.append(node_range_y_new_shape)
+
+    outputs_range_x_reshape = [model_name + "@range_x_reshape"]
+    node_range_x_reshape = onnx.helper.make_node(
+        'Reshape',
+        inputs=name_range_x + name_range_x_new_shape,
+        outputs=outputs_range_x_reshape)
+    node_list.append(node_range_x_reshape)
+
+    outputs_range_y_reshape = [model_name + "@range_y_reshape"]
+    node_range_y_reshape = onnx.helper.make_node(
+        'Reshape',
+        inputs=name_range_y + name_range_y_new_shape,
+        outputs=outputs_range_y_reshape)
+    node_list.append(node_range_y_reshape)
+
+    outputs_grid_x = [model_name + "@grid_x"]
+    node_grid_x = onnx.helper.make_node(
+        "Tile",
+        inputs=outputs_range_x_reshape + name_range_y_new_shape,
+        outputs=outputs_grid_x)
+    node_list.append(node_grid_x)
+
+    outputs_grid_y = [model_name + "@grid_y"]
+    node_grid_y = onnx.helper.make_node(
+        "Tile",
+        inputs=outputs_range_y_reshape + name_range_x_new_shape,
+        outputs=outputs_grid_y)
+    node_list.append(node_grid_y)
+
+    outputs_box_x = [model_name + "@box_x"]
+    outputs_box_y = [model_name + "@box_y"]
+    outputs_box_w = [model_name + "@box_w"]
+    outputs_box_h = [model_name + "@box_h"]
+    outputs_conf = [model_name + "@conf"]
+    outputs_prob = [model_name + "@prob"]
+
+    node_split_input = onnx.helper.make_node(
+        "Split",
+        inputs=outputs_x_transpose,
+        outputs=outputs_box_x + outputs_box_y + outputs_box_w\
+                + outputs_box_h + outputs_conf + outputs_prob,
+        axis=-1,
+        split=[1, 1, 1, 1, 1, class_num])
+    node_list.append(node_split_input)
+
+    outputs_box_x_sigmoid = [model_name + "@box_x_sigmoid"]
+    outputs_box_y_sigmoid = [model_name + "@box_y_sigmoid"]
+
+    node_box_x_sigmoid = onnx.helper.make_node(
+        "Sigmoid", inputs=outputs_box_x, outputs=outputs_box_x_sigmoid)
+    node_list.append(node_box_x_sigmoid)
+
+    node_box_y_sigmoid = onnx.helper.make_node(
+        "Sigmoid", inputs=outputs_box_y, outputs=outputs_box_y_sigmoid)
+    node_list.append(node_box_y_sigmoid)
+
+    outputs_box_x_squeeze = [model_name + "@box_x_squeeze"]
+    outputs_box_y_squeeze = [model_name + "@box_y_squeeze"]
+
+    node_box_x_squeeze = onnx.helper.make_node(
+        'Squeeze',
+        inputs=outputs_box_x_sigmoid,
+        outputs=outputs_box_x_squeeze,
+        axes=[4])
+    node_list.append(node_box_x_squeeze)
+
+    node_box_y_squeeze = onnx.helper.make_node(
+        'Squeeze',
+        inputs=outputs_box_y_sigmoid,
+        outputs=outputs_box_y_squeeze,
+        axes=[4])
+    node_list.append(node_box_y_squeeze)
+
+    outputs_box_x_add_grid = [model_name + "@box_x_add_grid"]
+    outputs_box_y_add_grid = [model_name + "@box_y_add_grid"]
+
+    node_box_x_add_grid = onnx.helper.make_node(
+        "Add",
+        inputs=outputs_grid_x + outputs_box_x_squeeze,
+        outputs=outputs_box_x_add_grid)
+    node_list.append(node_box_x_add_grid)
+
+    node_box_y_add_grid = onnx.helper.make_node(
+        "Add",
+        inputs=outputs_grid_y + outputs_box_y_squeeze,
+        outputs=outputs_box_y_add_grid)
+    node_list.append(node_box_y_add_grid)
+
+    name_input_h = [model_name + "@input_h"]
+    name_input_w = [model_name + "@input_w"]
+
+    node_input_h = onnx.helper.make_node(
+        'Constant',
+        inputs=[],
+        outputs=name_input_h,
+        value=onnx.helper.make_tensor(
+            name=name_input_w[0] + "@const",
+            data_type=onnx.TensorProto.FLOAT,
+            dims=(),
+            vals=[input_height]))
+    node_list.append(node_input_h)
+
+    node_input_w = onnx.helper.make_node(
+        'Constant',
+        inputs=[],
+        outputs=name_input_w,
+        value=onnx.helper.make_tensor(
+            name=name_input_w[0] + "@const",
+            data_type=onnx.TensorProto.FLOAT,
+            dims=(),
+            vals=[input_width]))
+    node_list.append(node_input_w)
+
+    outputs_box_x_encode = [model_name + "@box_x_encode"]
+    outputs_box_y_encode = [model_name + "@box_y_encode"]
+
+    node_box_x_encode = onnx.helper.make_node(
+        'Div',
+        inputs=outputs_box_x_add_grid + name_input_w,
+        outputs=outputs_box_x_encode)
+    node_list.append(node_box_x_encode)
+
+    node_box_y_encode = onnx.helper.make_node(
+        'Div',
+        inputs=outputs_box_y_add_grid + name_input_h,
+        outputs=outputs_box_y_encode)
+    node_list.append(node_box_y_encode)
+
+    name_anchor_tensor = [model_name + "@anchor_tensor"]
+    node_anchor_tensor = onnx.helper.make_node(
+        "Constant",
+        inputs=[],
+        outputs=name_anchor_tensor,
+        value=onnx.helper.make_tensor(
+            name=name_anchor_tensor[0] + "@const",
+            data_type=onnx.TensorProto.FLOAT,
+            dims=[len(anchors)],
+            vals=anchors))
+    node_list.append(node_anchor_tensor)
+
+    anchor_shape = [int(num_anchors), 2]
+    name_anchor_shape = [model_name + "@anchor_shape"]
+    node_anchor_shape = onnx.helper.make_node(
+        "Constant",
+        inputs=[],
+        outputs=name_anchor_shape,
+        value=onnx.helper.make_tensor(
+            name=name_anchor_shape[0] + "@const",
+            data_type=onnx.TensorProto.INT64,
+            dims=[2],
+            vals=anchor_shape))
+    node_list.append(node_anchor_shape)
+
+    outputs_anchor_tensor_reshape = [model_name + "@anchor_tensor_reshape"]
+    node_anchor_tensor_reshape = onnx.helper.make_node(
+        "Reshape",
+        inputs=name_anchor_tensor + name_anchor_shape,
+        outputs=outputs_anchor_tensor_reshape)
+    node_list.append(node_anchor_tensor_reshape)
+
+    name_input_size = [model_name + "@input_size"]
+    node_input_size = onnx.helper.make_node(
+        "Constant",
+        inputs=[],
+        outputs=name_input_size,
+        value=onnx.helper.make_tensor(
+            name=name_input_size[0] + "@const",
+            data_type=onnx.TensorProto.FLOAT,
+            dims=(),
+            vals=[input_size]))
+    node_list.append(node_input_size)
+
+    outputs_anchors_div_input_size = [model_name + "@anchors_div_input_size"]
+    node_anchors_div_input_size = onnx.helper.make_node(
+        "Div",
+        inputs=outputs_anchor_tensor_reshape + name_input_size,
+        outputs=outputs_anchors_div_input_size)
+    node_list.append(node_anchors_div_input_size)
+
+    outputs_anchor_w = [model_name + "@anchor_w"]
+    outputs_anchor_h = [model_name + "@anchor_h"]
+
+    node_anchor_split = onnx.helper.make_node(
+        'Split',
+        inputs=outputs_anchors_div_input_size,
+        outputs=outputs_anchor_w + outputs_anchor_h,
+        axis=1,
+        split=[1, 1])
+    node_list.append(node_anchor_split)
+
+    new_anchor_shape = [1, int(num_anchors), 1, 1]
+    name_new_anchor_shape = [model_name + "@new_anchor_shape"]
+    node_new_anchor_shape = onnx.helper.make_node(
+        'Constant',
+        inputs=[],
+        outputs=name_new_anchor_shape,
+        value=onnx.helper.make_tensor(
+            name=name_new_anchor_shape[0] + "@const",
+            data_type=onnx.TensorProto.INT64,
+            dims=[len(new_anchor_shape)],
+            vals=new_anchor_shape))
+    node_list.append(node_new_anchor_shape)
+
+    outputs_anchor_w_reshape = [model_name + "@anchor_w_reshape"]
+    outputs_anchor_h_reshape = [model_name + "@anchor_h_reshape"]
+
+    node_anchor_w_reshape = onnx.helper.make_node(
+        'Reshape',
+        inputs=outputs_anchor_w + name_new_anchor_shape,
+        outputs=outputs_anchor_w_reshape)
+    node_list.append(node_anchor_w_reshape)
+
+    node_anchor_h_reshape = onnx.helper.make_node(
+        'Reshape',
+        inputs=outputs_anchor_h + name_new_anchor_shape,
+        outputs=outputs_anchor_h_reshape)
+    node_list.append(node_anchor_h_reshape)
+
+    outputs_box_w_squeeze = [model_name + "@box_w_squeeze"]
+    node_box_w_squeeze = onnx.helper.make_node(
+        'Squeeze',
+        inputs=outputs_box_w,
+        outputs=outputs_box_w_squeeze,
+        axes=[4])
+    node_list.append(node_box_w_squeeze)
+
+    outputs_box_h_squeeze = [model_name + "@box_h_squeeze"]
+    node_box_h_squeeze = onnx.helper.make_node(
+        'Squeeze',
+        inputs=outputs_box_h,
+        outputs=outputs_box_h_squeeze,
+        axes=[4])
+    node_list.append(node_box_h_squeeze)
+
+    outputs_box_w_exp = [model_name + "@box_w_exp"]
+    node_box_w_exp = onnx.helper.make_node(
+        "Exp", inputs=outputs_box_w_squeeze, outputs=outputs_box_w_exp)
+    node_list.append(node_box_w_exp)
+
+    outputs_box_h_exp = [model_name + "@box_h_exp"]
+    node_box_h_exp = onnx.helper.make_node(
+        "Exp", inputs=outputs_box_h_squeeze, outputs=outputs_box_h_exp)
+    node_list.append(node_box_h_exp)
+
+    outputs_box_w_encode = [model_name + "box_w_encode"]
+    outputs_box_h_encode = [model_name + "box_h_encode"]
+
+    node_box_w_encode = onnx.helper.make_node(
+        'Mul',
+        inputs=outputs_box_w_exp + outputs_anchor_w_reshape,
+        outputs=outputs_box_w_encode)
+    node_list.append(node_box_w_encode)
+
+    node_box_h_encode = onnx.helper.make_node(
+        'Mul',
+        inputs=outputs_box_h_exp + outputs_anchor_h_reshape,
+        outputs=outputs_box_h_encode)
+    node_list.append(node_box_h_encode)
+
+    outputs_conf_sigmoid = [model_name + "@conf_sigmoid"]
+    node_conf_sigmoid = onnx.helper.make_node(
+        'Sigmoid', inputs=outputs_conf, outputs=outputs_conf_sigmoid)
+    node_list.append(node_conf_sigmoid)
+
+    name_conf_thresh = [model_name + "@conf_thresh"]
+    node_conf_thresh = onnx.helper.make_node(
+        'Constant',
+        inputs=[],
+        outputs=name_conf_thresh,
+        value=onnx.helper.make_tensor(
+            name=name_conf_thresh[0] + "@const",
+            data_type=onnx.TensorProto.FLOAT,
+            dims=[num_anchors * input_height * input_width],
+            vals=conf_thresh_mat))
+    node_list.append(node_conf_thresh)
+
+    conf_shape = [1, int(num_anchors), input_height, input_width, 1]
+    name_conf_shape = [model_name + "@conf_shape"]
+    node_conf_shape = onnx.helper.make_node(
+        'Constant',
+        inputs=[],
+        outputs=name_conf_shape,
+        value=onnx.helper.make_tensor(
+            name=name_conf_shape[0] + "@const",
+            data_type=onnx.TensorProto.INT64,
+            dims=[len(conf_shape)],
+            vals=conf_shape))
+    node_list.append(node_conf_shape)
+
+    outputs_conf_thresh_reshape = [model_name + "@conf_thresh_reshape"]
+    node_conf_thresh_reshape = onnx.helper.make_node(
+        'Reshape',
+        inputs=name_conf_thresh + name_conf_shape,
+        outputs=outputs_conf_thresh_reshape)
+    node_list.append(node_conf_thresh_reshape)
+
+    outputs_conf_sub = [model_name + "@conf_sub"]
+    node_conf_sub = onnx.helper.make_node(
+        'Sub',
+        inputs=outputs_conf_sigmoid + outputs_conf_thresh_reshape,
+        outputs=outputs_conf_sub)
+    node_list.append(node_conf_sub)
+
+    outputs_conf_clip = [model_name + "@conf_clip"]
+    node_conf_clip = onnx.helper.make_node(
+        'Clip', inputs=outputs_conf_sub, outputs=outputs_conf_clip)
+    node_list.append(node_conf_clip)
+
+    zeros = [0]
+    name_zeros = [model_name + "@zeros"]
+    node_zeros = onnx.helper.make_node(
+        'Constant',
+        inputs=[],
+        outputs=name_zeros,
+        value=onnx.helper.make_tensor(
+            name=name_zeros[0] + "@const",
+            data_type=onnx.TensorProto.FLOAT,
+            dims=(),
+            vals=zeros))
+    node_list.append(node_zeros)
+
+    outputs_conf_clip_bool = [model_name + "@conf_clip_bool"]
+    node_conf_clip_bool = onnx.helper.make_node(
+        'Greater',
+        inputs=outputs_conf_clip + name_zeros,
+        outputs=outputs_conf_clip_bool)
+    node_list.append(node_conf_clip_bool)
+
+    outputs_conf_clip_cast = [model_name + "@conf_clip_cast"]
+    node_conf_clip_cast = onnx.helper.make_node(
+        'Cast',
+        inputs=outputs_conf_clip_bool,
+        outputs=outputs_conf_clip_cast,
+        to=1)
+    node_list.append(node_conf_clip_cast)
+
+    outputs_conf_set_zero = [model_name + "@conf_set_zero"]
+    node_conf_set_zero = onnx.helper.make_node(
+        'Mul',
+        inputs=outputs_conf_sigmoid + outputs_conf_clip_cast,
+        outputs=outputs_conf_set_zero)
+    node_list.append(node_conf_set_zero)
+
+    outputs_prob_sigmoid = [model_name + "@prob_sigmoid"]
+    node_prob_sigmoid = onnx.helper.make_node(
+        'Sigmoid', inputs=outputs_prob, outputs=outputs_prob_sigmoid)
+    node_list.append(node_prob_sigmoid)
+
+    new_shape = [1, int(num_anchors), input_height, input_width, 1]
+    name_new_shape = [model_name + "@new_shape"]
+    node_new_shape = onnx.helper.make_node(
+        'Constant',
+        inputs=[],
+        outputs=name_new_shape,
+        value=onnx.helper.make_tensor(
+            name=name_new_shape[0] + "@const",
+            data_type=onnx.TensorProto.INT64,
+            dims=[len(new_shape)],
+            vals=new_shape))
+    node_list.append(node_new_shape)
+
+    outputs_conf_new_shape = [model_name + "@_conf_new_shape"]
+    node_conf_new_shape = onnx.helper.make_node(
+        'Reshape',
+        inputs=outputs_conf_set_zero + name_new_shape,
+        outputs=outputs_conf_new_shape)
+    node_list.append(node_conf_new_shape)
+
+    outputs_score = [model_name + "@score"]
+    node_score = onnx.helper.make_node(
+        'Mul',
+        inputs=outputs_prob_sigmoid + outputs_conf_new_shape,
+        outputs=outputs_score)
+    node_list.append(node_score)
+
+    outputs_conf_bool = [model_name + "@conf_bool"]
+    node_conf_bool = onnx.helper.make_node(
+        'Greater',
+        inputs=outputs_conf_new_shape + name_zeros,
+        outputs=outputs_conf_bool)
+    node_list.append(node_conf_bool)
+
+    outputs_box_x_new_shape = [model_name + "@box_x_new_shape"]
+    node_box_x_new_shape = onnx.helper.make_node(
+        'Reshape',
+        inputs=outputs_box_x_encode + name_new_shape,
+        outputs=outputs_box_x_new_shape)
+    node_list.append(node_box_x_new_shape)
+
+    outputs_box_y_new_shape = [model_name + "@box_y_new_shape"]
+    node_box_y_new_shape = onnx.helper.make_node(
+        'Reshape',
+        inputs=outputs_box_y_encode + name_new_shape,
+        outputs=outputs_box_y_new_shape)
+    node_list.append(node_box_y_new_shape)
+
+    outputs_box_w_new_shape = [model_name + "@box_w_new_shape"]
+    node_box_w_new_shape = onnx.helper.make_node(
+        'Reshape',
+        inputs=outputs_box_w_encode + name_new_shape,
+        outputs=outputs_box_w_new_shape)
+    node_list.append(node_box_w_new_shape)
+
+    outputs_box_h_new_shape = [model_name + "@box_h_new_shape"]
+    node_box_h_new_shape = onnx.helper.make_node(
+        'Reshape',
+        inputs=outputs_box_h_encode + name_new_shape,
+        outputs=outputs_box_h_new_shape)
+    node_list.append(node_box_h_new_shape)
+
+    outputs_pred_box = [model_name + "@pred_box"]
+    node_pred_box = onnx.helper.make_node(
+        'Concat',
+        inputs=outputs_box_x_new_shape + outputs_box_y_new_shape + \
+               outputs_box_w_new_shape + outputs_box_h_new_shape,
+        outputs=outputs_pred_box,
+        axis=4)
+    node_list.append(node_pred_box)
+
+    outputs_conf_cast = [model_name + "conf_cast"]
+    node_conf_cast = onnx.helper.make_node(
+        'Cast', inputs=outputs_conf_bool, outputs=outputs_conf_cast, to=1)
+    node_list.append(node_conf_cast)
+
+    outputs_pred_box_mul_conf = [model_name + "@pred_box_mul_conf"]
+    node_pred_box_mul_conf = onnx.helper.make_node(
+        'Mul',
+        inputs=outputs_pred_box + outputs_conf_cast,
+        outputs=outputs_pred_box_mul_conf)
+    node_list.append(node_pred_box_mul_conf)
+
+    box_shape = [1, int(num_anchors) * input_height * input_width, 4]
+    name_box_shape = [model_name + "@box_shape"]
+    node_box_shape = onnx.helper.make_node(
+        'Constant',
+        inputs=[],
+        outputs=name_box_shape,
+        value=onnx.helper.make_tensor(
+            name=name_box_shape[0] + "@const",
+            data_type=onnx.TensorProto.INT64,
+            dims=[len(box_shape)],
+            vals=box_shape))
+    node_list.append(node_box_shape)
+
+    outputs_pred_box_new_shape = [model_name + "@pred_box_new_shape"]
+    node_pred_box_new_shape = onnx.helper.make_node(
+        'Reshape',
+        inputs=outputs_pred_box_mul_conf + name_box_shape,
+        outputs=outputs_pred_box_new_shape)
+    node_list.append(node_pred_box_new_shape)
+
+    outputs_pred_box_x = [model_name + "@_pred_box_x"]
+    outputs_pred_box_y = [model_name + "@_pred_box_y"]
+    outputs_pred_box_w = [model_name + "@_pred_box_w"]
+    outputs_pred_box_h = [model_name + "@_pred_box_h"]
+
+    node_pred_box_split = onnx.helper.make_node(
+        'Split',
+        inputs=outputs_pred_box_new_shape,
+        outputs=outputs_pred_box_x + outputs_pred_box_y + outputs_pred_box_w +
+        outputs_pred_box_h,
+        axis=2)
+    node_list.append(node_pred_box_split)
+
+    name_number_two = [model_name + "@number_two"]
+    node_number_two = onnx.helper.make_node(
+        "Constant",
+        inputs=[],
+        outputs=name_number_two,
+        value=onnx.helper.make_tensor(
+            name=name_number_two[0] + "@const",
+            data_type=onnx.TensorProto.FLOAT,
+            dims=(),
+            vals=[2]))
+    node_list.append(node_number_two)
+
+    outputs_half_w = [model_name + "@half_w"]
+    node_half_w = onnx.helper.make_node(
+        "Div",
+        inputs=outputs_pred_box_w + name_number_two,
+        outputs=outputs_half_w)
+    node_list.append(node_half_w)
+
+    outputs_half_h = [model_name + "@half_h"]
+    node_half_h = onnx.helper.make_node(
+        "Div",
+        inputs=outputs_pred_box_h + name_number_two,
+        outputs=outputs_half_h)
+    node_list.append(node_half_h)
+
+    outputs_pred_box_x1 = [model_name + "@pred_box_x1"]
+    node_pred_box_x1 = onnx.helper.make_node(
+        'Sub',
+        inputs=outputs_pred_box_x + outputs_half_w,
+        outputs=outputs_pred_box_x1)
+    node_list.append(node_pred_box_x1)
+
+    outputs_pred_box_y1 = [model_name + "@pred_box_y1"]
+    node_pred_box_y1 = onnx.helper.make_node(
+        'Sub',
+        inputs=outputs_pred_box_y + outputs_half_h,
+        outputs=outputs_pred_box_y1)
+    node_list.append(node_pred_box_y1)
+
+    outputs_pred_box_x2 = [model_name + "@pred_box_x2"]
+    node_pred_box_x2 = onnx.helper.make_node(
+        'Add',
+        inputs=outputs_pred_box_x + outputs_half_w,
+        outputs=outputs_pred_box_x2)
+    node_list.append(node_pred_box_x2)
+
+    outputs_pred_box_y2 = [model_name + "@pred_box_y2"]
+    node_pred_box_y2 = onnx.helper.make_node(
+        'Add',
+        inputs=outputs_pred_box_y + outputs_half_h,
+        outputs=outputs_pred_box_y2)
+    node_list.append(node_pred_box_y2)
+
+    outputs_sqeeze_image_size = [model_name + "@sqeeze_image_size"]
+    node_sqeeze_image_size = onnx.helper.make_node(
+        "Squeeze",
+        axes=[0],
+        inputs=image_size,
+        outputs=outputs_sqeeze_image_size)
+    node_list.append(node_sqeeze_image_size)
+
+    output_img_height = [model_name + "@img_height"]
+    output_img_width = [model_name + "@img_width"]
+    node_image_size_split = onnx.helper.make_node(
+        "Split",
+        inputs=outputs_sqeeze_image_size,
+        outputs=output_img_height + output_img_width,
+        axis=-1,
+        split=[1, 1])
+    node_list.append(node_image_size_split)
+
+    output_img_width_cast = [model_name + "@img_width_cast"]
+    node_img_width_cast = onnx.helper.make_node(
+        'Cast', inputs=output_img_width, outputs=output_img_width_cast, to=1)
+    node_list.append(node_img_width_cast)
+
+    output_img_height_cast = [model_name + "@img_height_cast"]
+    node_img_height_cast = onnx.helper.make_node(
+        'Cast', inputs=output_img_height, outputs=output_img_height_cast, to=1)
+    node_list.append(node_img_height_cast)
+
+    outputs_pred_box_x1_decode = [model_name + "@pred_box_x1_decode"]
+    outputs_pred_box_y1_decode = [model_name + "@pred_box_y1_decode"]
+    outputs_pred_box_x2_decode = [model_name + "@pred_box_x2_decode"]
+    outputs_pred_box_y2_decode = [model_name + "@pred_box_y2_decode"]
+
+    node_pred_box_x1_decode = onnx.helper.make_node(
+        'Mul',
+        inputs=outputs_pred_box_x1 + output_img_width_cast,
+        outputs=outputs_pred_box_x1_decode)
+    node_list.append(node_pred_box_x1_decode)
+
+    node_pred_box_y1_decode = onnx.helper.make_node(
+        'Mul',
+        inputs=outputs_pred_box_y1 + output_img_height_cast,
+        outputs=outputs_pred_box_y1_decode)
+    node_list.append(node_pred_box_y1_decode)
+
+    node_pred_box_x2_decode = onnx.helper.make_node(
+        'Mul',
+        inputs=outputs_pred_box_x2 + output_img_width_cast,
+        outputs=outputs_pred_box_x2_decode)
+    node_list.append(node_pred_box_x2_decode)
+
+    node_pred_box_y2_decode = onnx.helper.make_node(
+        'Mul',
+        inputs=outputs_pred_box_y2 + output_img_height_cast,
+        outputs=outputs_pred_box_y2_decode)
+    node_list.append(node_pred_box_y2_decode)
+
+    name_number_one = [model_name + "@one"]
+    node_number_one = onnx.helper.make_node(
+        'Constant',
+        inputs=[],
+        outputs=name_number_one,
+        value=onnx.helper.make_tensor(
+            name=name_number_one[0] + "@const",
+            data_type=onnx.TensorProto.FLOAT,
+            dims=(),
+            vals=[1]))
+    node_list.append(node_number_one)
+
+    output_new_img_height = [model_name + "@new_img_height"]
+    node_new_img_height = onnx.helper.make_node(
+        'Sub',
+        inputs=output_img_height_cast + name_number_one,
+        outputs=output_new_img_height)
+    node_list.append(node_new_img_height)
+
+    output_new_img_width = [model_name + "@new_img_width"]
+    node_new_img_width = onnx.helper.make_node(
+        'Sub',
+        inputs=output_img_width_cast + name_number_one,
+        outputs=output_new_img_width)
+    node_list.append(node_new_img_width)
+
+    outputs_pred_box_x2_sub_w = [model_name + "@pred_box_x2_sub_w"]
+    node_pred_box_x2_sub_w = onnx.helper.make_node(
+        'Sub',
+        inputs=outputs_pred_box_x2_decode + output_new_img_width,
+        outputs=outputs_pred_box_x2_sub_w)
+    node_list.append(node_pred_box_x2_sub_w)
+
+    outputs_pred_box_y2_sub_h = [model_name + "@pred_box_y2_sub_h"]
+    node_pred_box_y2_sub_h = onnx.helper.make_node(
+        'Sub',
+        inputs=outputs_pred_box_y2_decode + output_new_img_height,
+        outputs=outputs_pred_box_y2_sub_h)
+    node_list.append(node_pred_box_y2_sub_h)
+
+    outputs_pred_box_x1_clip = [model_name + "@pred_box_x1_clip"]
+    outputs_pred_box_y1_clip = [model_name + "@pred_box_y1_clip"]
+    outputs_pred_box_x2_clip = [model_name + "@pred_box_x2_clip"]
+    outputs_pred_box_y2_clip = [model_name + "@pred_box_y2_clip"]
+
+    node_pred_box_x1_clip = onnx.helper.make_node(
+        'Clip',
+        inputs=outputs_pred_box_x1_decode,
+        outputs=outputs_pred_box_x1_clip,
+        min=0.0,
+        max=float(np.inf))
+    node_list.append(node_pred_box_x1_clip)
+
+    node_pred_box_y1_clip = onnx.helper.make_node(
+        'Clip',
+        inputs=outputs_pred_box_y1_decode,
+        outputs=outputs_pred_box_y1_clip,
+        min=0.0,
+        max=float(np.inf))
+    node_list.append(node_pred_box_y1_clip)
+
+    node_pred_box_x2_clip = onnx.helper.make_node(
+        'Clip',
+        inputs=outputs_pred_box_x2_sub_w,
+        outputs=outputs_pred_box_x2_clip,
+        min=0.0,
+        max=float(np.inf))
+    node_list.append(node_pred_box_x2_clip)
+
+    node_pred_box_y2_clip = onnx.helper.make_node(
+        'Clip',
+        inputs=outputs_pred_box_y2_sub_h,
+        outputs=outputs_pred_box_y2_clip,
+        min=0.0,
+        max=float(np.inf))
+    node_list.append(node_pred_box_y2_clip)
+
+    outputs_pred_box_x2_res = [model_name + "@box_x2_res"]
+    node_pred_box_x2_res = onnx.helper.make_node(
+        'Sub',
+        inputs=outputs_pred_box_x2_decode + outputs_pred_box_x2_clip,
+        outputs=outputs_pred_box_x2_res)
+    node_list.append(node_pred_box_x2_res)
+
+    outputs_pred_box_y2_res = [model_name + "@box_y2_res"]
+    node_pred_box_y2_res = onnx.helper.make_node(
+        'Sub',
+        inputs=outputs_pred_box_y2_decode + outputs_pred_box_y2_clip,
+        outputs=outputs_pred_box_y2_res)
+    node_list.append(node_pred_box_y2_res)
+
+    node_pred_box_result = onnx.helper.make_node(
+        'Concat',
+        inputs=outputs_pred_box_x1_clip + outputs_pred_box_y1_clip +
+        outputs_pred_box_x2_res + outputs_pred_box_y2_res,
+        outputs=outputs['Boxes'],
+        axis=-1)
+    node_list.append(node_pred_box_result)
+
+    score_shape = [1, input_height * input_width * int(num_anchors), class_num]
+    name_score_shape = [model_name + "@score_shape"]
+    node_score_shape = onnx.helper.make_node(
+        "Constant",
+        inputs=[],
+        outputs=name_score_shape,
+        value=onnx.helper.make_tensor(
+            name=name_score_shape[0] + "@const",
+            data_type=onnx.TensorProto.INT64,
+            dims=[len(score_shape)],
+            vals=score_shape))
+    node_list.append(node_score_shape)
+
+    node_score_new_shape = onnx.helper.make_node(
+        'Reshape',
+        inputs=outputs_score + name_score_shape,
+        outputs=outputs['Scores'])
+    node_list.append(node_score_new_shape)
+    return node_list

x2paddle/op_mapper/paddle_op_mapper.py

+#   Copyright (c) 2019  PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License"
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import math
+import sys
+import x2paddle
+import os
+import numpy as np
+import paddle.fluid.core as core
+import paddle.fluid as fluid
+import onnx
+from onnx import helper, onnx_pb
+
+
+class PaddleOpMapper(object):
+    def __init__(self):
+        self.paddle_onnx_dtype_map = {
+            core.VarDesc.VarType.FP32: onnx_pb.TensorProto.FLOAT,
+            core.VarDesc.VarType.FP64: onnx_pb.TensorProto.DOUBLE,
+            core.VarDesc.VarType.INT32: onnx_pb.TensorProto.INT32,
+            core.VarDesc.VarType.INT16: onnx_pb.TensorProto.INT16,
+            core.VarDesc.VarType.INT16: onnx_pb.TensorProto.UINT16,
+            core.VarDesc.VarType.INT64: onnx_pb.TensorProto.INT64,
+            core.VarDesc.VarType.BOOL: onnx_pb.TensorProto.BOOL
+        }
+
+        self.name_counter = dict()
+
+    def convert(self, program, save_dir):
+        weight_nodes = self.convert_weights(program)
+        op_nodes = list()
+        input_nodes = list()
+        output_nodes = list()
+
+        unsupported_ops = set()
+
+        print("Translating PaddlePaddle to ONNX...\n")
+        for block in program.blocks:
+            for i, op in enumerate(block.ops):
+                sys.stdout.write(
+                    "\rTotal:{}, Current:{} : {}                   ".format(
+                        len(block.ops), i + 1, op.type))
+                sys.stdout.flush()
+                if not hasattr(self, op.type):
+                    unsupported_ops.add(op.type)
+                    continue
+                if len(unsupported_ops) > 0:
+                    continue
+                node = getattr(self, op.type)(op, block)
+                if op.type == 'feed':
+                    input_nodes.append(node)
+                elif op.type == 'fetch':
+                    output_nodes.append(node)
+                else:
+                    if isinstance(node, list):
+                        op_nodes = op_nodes + node
+                    else:
+                        op_nodes.append(node)
+
+        if len(unsupported_ops) > 0:
+            print("\nThere's {} ops are not supported yet".format(
+                len(unsupported_ops)))
+            for op in unsupported_ops:
+                print("=========== {} ===========".format(op))
+            return
+
+        graph = helper.make_graph(
+            nodes=weight_nodes + op_nodes,
+            name='onnx_model_from_paddle',
+            initializer=[],
+            inputs=input_nodes,
+            outputs=output_nodes)
+        model = helper.make_model(graph, producer_name='X2Paddle')
+        onnx.checker.check_model(model)
+
+        if not os.path.isdir(save_dir):
+            os.makedirs(save_dir)
+        with open(os.path.join(save_dir, 'x2paddle_model.onnx'), 'wb') as f:
+            f.write(model.SerializeToString())
+        print("\nTranslated model saved in {}".format(
+            os.path.join(save_dir, 'x2paddle_model.onnx')))
+
+    def get_name(self, op_name, var_name):
+        name = 'p2o.{}.{}'.format(op_name, var_name)
+        if name not in self.name_counter:
+            self.name_counter[name] = 0
+        else:
+            self.name_counter[name] += 1
+        return name + '.{}'.format(self.name_counter[name])
+
+    def convert_weights(self, program):
+        var_names = program.global_block().vars
+        nodes = list()
+        for name in var_names:
+            var = program.global_block().var(name)
+            if name.endswith('feed') or name.endswith('fetch'):
+                continue
+            if not var.persistable:
+                continue
+            weight = np.array(fluid.global_scope().find_var(name).get_tensor())
+            tensor = helper.make_tensor(
+                name=name,
+                dims=var.shape,
+                data_type=self.paddle_onnx_dtype_map[var.dtype],
+                vals=weight.flatten().tolist())
+            node = helper.make_node(
+                'Constant', inputs=[], outputs=[name], value=tensor)
+            nodes.append(node)
+        return nodes
+
+    def make_constant_node(self, name, dtype, value=None):
+        if isinstance(value, list):
+            dims = (len(value), )
+        elif value is None:
+            dims = ()
+            value = []
+        else:
+            dims = ()
+            value = [value]
+        tensor = helper.make_tensor(
+            name=name, data_type=dtype, dims=dims, vals=value)
+        node = helper.make_node(
+            'Constant', inputs=[], outputs=[name], value=tensor)
+        return node
+
+    def conv2d(self, op, block):
+        kernel_shape = block.var(op.input('Filter')[0]).shape
+        node = helper.make_node(
+            'Conv',
+            inputs=op.input('Input') + op.input('Filter'),
+            outputs=op.output('Output'),
+            dilations=op.attr('dilations'),
+            kernel_shape=kernel_shape[-2:],
+            strides=op.attr('strides'),
+            group=op.attr('groups'),
+            pads=op.attr('paddings') + op.attr('paddings'))
+        return node
+
+    def conv2d_transpose(self, op, block):
+        kernel_shape = block.var(op.input('Filter')[0]).shape
+        node = helper.make_node(
+            'ConvTranspose',
+            inputs=op.input('Input') + op.input('Filter'),
+            outputs=op.output('Output'),
+            dilations=op.attr('dilations'),
+            kernel_shape=kernel_shape[-2:],
+            strides=op.attr('strides'),
+            group=1,
+            pads=op.attr('paddings') + op.attr('paddings'))
+        return node
+
+    def relu(self, op, block):
+        node = helper.make_node(
+            'Relu', inputs=op.input('X'), outputs=op.output('Out'))
+        return node
+
+    def sigmoid(self, op, block):
+        node = helper.make_node(
+            'Sigmoid', inputs=op.input('X'), outputs=op.output('Out'))
+        return node
+
+    def exp(self, op, block):
+        node = helper.make_node(
+            'Exp', inputs=op.input('X'), outputs=op.output('Out'))
+        return node
+
+    def leaky_relu(self, op, block):
+        node = helper.make_node(
+            'LeakyRelu',
+            inputs=op.input('X'),
+            outputs=op.output('Out'),
+            alpha=op.attr('alpha'))
+        return node
+
+    def elementwise_add(self, op, block):
+        axis = op.attr('axis')
+        x_shape = block.var(op.input('X')[0]).shape
+        y_shape = block.var(op.input('Y')[0]).shape
+        if len(y_shape) == 1 and axis == 1:
+            shape_name = self.get_name(op.type, 'shape')
+            shape_value = [1] * len(x_shape)
+            shape_value[axis] = y_shape[0]
+            shape_node = self.make_constant_node(
+                shape_name, onnx_pb.TensorProto.INT64, shape_value)
+            temp_value = self.get_name(op.type, 'temp')
+            y_node = helper.make_node(
+                'Reshape',
+                inputs=[op.input('Y')[0], shape_name],
+                outputs=[temp_value])
+            node = helper.make_node(
+                'Add',
+                inputs=[op.input('X')[0], temp_value],
+                outputs=op.output('Out'))
+            return [shape_node, y_node, node]
+        elif len(x_shape) == len(y_shape):
+            node = helper.make_node(
+                'Add',
+                inputs=[op.input('X')[0], op.input('Y')[0]],
+                outputs=op.output('Out'))
+            return node
+        else:
+            raise Excpetion("Unexpected situation happend in elementwise_add")
+
+    def elementwise_sub(self, op, block):
+        axis = op.attr('axis')
+        x_shape = block.var(op.input('X')[0]).shape
+        y_shape = block.var(op.input('Y')[0]).shape
+        if len(y_shape) == 1 and axis == 1:
+            shape_name = self.get_name(op.type, 'shape')
+            shape_value = [1] * len(x_shape)
+            shape_value[axis] = y_shape[0]
+            shape_node = self.make_constant_node(
+                shape_name, onnx_pb.TensorProto.INT64, shape_value)
+            temp_value = self.get_name(op.type, 'temp')
+            y_node = helper.make_node(
+                'Reshape',
+                inputs=[op.input('Y')[0], shape_name],
+                outputs=[temp_value])
+            node = helper.make_node(
+                'Sub',
+                inputs=[op.input('X')[0], temp_value],
+                outputs=op.output('Out'))
+            return [shape_node, y_node, node]
+        elif len(x_shape) == len(y_shape):
+            node = helper.make_node(
+                'Sub',
+                inputs=[op.input('X')[0], op.input('Y')[0]],
+                outputs=op.output('Out'))
+            return node
+        else:
+            raise Excpetion("Unexpected situation happend in elementwise_sub")
+
+    def pool2d(self, op, block):
+        pool_type = {
+            'max': ('MaxPool', 'GlobalMaxPool'),
+            'avg': ('AveragePool', 'GlobalAveragePool')
+        }
+        if op.attr('global_pooling'):
+            node = helper.make_node(
+                pool_type[op.attr('pooling_type')][1],
+                inputs=op.input('X'),
+                outputs=op.output('Out'), )
+        else:
+            input_shape = block.var(op.input('X')[0]).shape
+            k_size = op.attr('ksize')
+            paddings = op.attr('paddings')
+            if input_shape[2] > 0 and input_shape[2] + paddings[0] < k_size[0]:
+                k_size[0] = input_shape[2] + paddings[0]
+            if input_shape[3] > 0 and input_shape[3] + paddings[1] < k_size[1]:
+                k_size[1] = input_shape[3] + paddings[1]
+            node = helper.make_node(
+                pool_type[op.attr('pooling_type')][0],
+                inputs=op.input('X'),
+                outputs=op.output('Out'),
+                kernel_shape=k_size,
+                strides=op.attr('strides'),
+                pads=op.attr('paddings') + op.attr('paddings'))
+        return node
+
+    def softmax(self, op, block):
+        axis = op.attr('axis')
+        shape = block.var(op.output('Out')[0]).shape
+        if axis < 0:
+            axis += len(shape)
+        if axis == len(shape) - 1:
+            node = helper.make_node(
+                'Softmax',
+                inputs=op.input('X'),
+                outputs=op.output('Out'),
+                axis=op.attr('axis'))
+            return node
+        else:
+            perm = [i for i in range(len(shape))]
+            perm[-1] = axis
+            perm[axis] = len(shape) - 1
+            transpose_name0 = self.get_name(op.type, 'transpose')
+            transpose_node0 = helper.make_node(
+                'Transpose',
+                inputs=op.input('X'),
+                outputs=[transpose_name0],
+                perm=perm)
+            softmax_name = self.get_name(op.type, 'softmax')
+            softmax_node = helper.make_node(
+                'Softmax',
+                inputs=[transpose_name0],
+                outputs=[softmax_name],
+                axis=-1)
+            transpose_name1 = self.get_name(op.type, 'transpose')
+            transpose_node1 = helper.make_node(
+                'Transpose',
+                inputs=[softmax_name],
+                outputs=op.output('Out'),
+                perm=perm)
+            return [transpose_node0, softmax_node, transpose_node1]
+
+    def scale(self, op, block):
+        scale = op.attr('scale')
+        bias = op.attr('bias')
+        if math.fabs(scale - 1.0) < 1e-06 and math.fabs(bias - 0.0) < 1e-06:
+            node = helper.make_node(
+                'Identity', inputs=op.input('X'), outputs=op.output('Out'))
+            return node
+        else:
+            scale_name = self.get_name(op.type, 'scale')
+            bias_name = self.get_name(op.type, 'bias')
+            scale_node = self.make_constant_node(
+                scale_name, onnx_pb.TensorProto.FLOAT, scale)
+            bias_node = self.make_constant_node(bias_name,
+                                                onnx_pb.TensorProto.FLOAT, bias)
+            temp_tensor_name = self.get_name(op.type, 'temporary')
+            if op.attr('bias_after_scale'):
+                node1 = helper.make_node(
+                    'Mul',
+                    inputs=[scale_name, op.input('X')[0]],
+                    outputs=[temp_tensor_name])
+                node2 = helper.make_node(
+                    'Add',
+                    inputs=[bias_name, temp_tensor_name],
+                    outputs=op.output('Out'))
+            else:
+                node1 = helper.make_node(
+                    'Add',
+                    inputs=[bias_name, op.input('X')[0]],
+                    outputs=temp_tensor_name)
+                node2 = helper.make_node(
+                    'Mul',
+                    inputs=[scale_name, temp_tensor_name],
+                    outputs=[op.output('Out')])
+            return [scale_node, bias_node, node1, node2]
+
+    def mul(self, op, block):
+        x_shape = block.var(op.input('X')[0]).shape
+        y_shape = block.var(op.input('Y')[0]).shape
+        out_shape = list(block.var(op.output('Out')[0]).shape)
+        x_num_col_dims = op.attr('x_num_col_dims')
+        y_num_col_dims = op.attr('y_num_col_dims')
+        flatten_x_name = 'flatten_{}'.format(op.input('X')[0])
+        flatten_y_name = 'flatten_{}'.format(op.input('Y')[0])
+        shape_name = 'temp_shape_{}'.format(op.output('Out')[0])
+        temp_out_name = 'temp_{}'.format(op.output('Out')[0])
+        flatten_x = helper.make_node(
+            'Flatten',
+            inputs=op.input('X'),
+            outputs=[flatten_x_name],
+            axis=x_num_col_dims)
+        flatten_y = helper.make_node(
+            'Flatten',
+            inputs=op.input('Y'),
+            outputs=[flatten_y_name],
+            axis=y_num_col_dims)
+        shape_node = self.make_constant_node(
+            shape_name, onnx_pb.TensorProto.INT64, out_shape)
+        node = helper.make_node(
+            'MatMul',
+            inputs=[flatten_x_name, flatten_y_name],
+            outputs=[temp_out_name])
+        reshape_out = helper.make_node(
+            'Reshape',
+            inputs=[temp_out_name, shape_name],
+            outputs=op.output('Out'))
+        return [flatten_x, flatten_y, shape_node, node, reshape_out]
+
+    def batch_norm(self, op, block):
+        kwargs = {
+            'epsilon': op.attr('epsilon'),
+            'momentum': op.attr('momentum')
+        }
+        inputs = op.input('X') + op.input('Scale') + op.input(
+            'Bias') + op.input('Mean') + op.input('Variance')
+        node = helper.make_node(
+            'BatchNormalization',
+            inputs=inputs,
+            outputs=op.output('Y'),
+            **kwargs)
+        return node
+
+    def concat(self, op, block):
+        node = helper.make_node(
+            'Concat',
+            inputs=op.input('X'),
+            outputs=op.output('Out'),
+            axis=op.attr('axis'))
+        return node
+
+    def depthwise_conv2d(self, op, block):
+        return self.conv2d(op, block)
+
+    def relu6(self, op, block):
+        min_name = self.get_name(op.type, 'min')
+        max_name = self.get_name(op.type, 'max')
+        min_node = self.make_constant_node(min_name, onnx_pb.TensorProto.FLOAT,
+                                           0)
+        max_node = self.make_constant_node(max_name, onnx_pb.TensorProto.FLOAT,
+                                           op.attr('threshold'))
+        node = helper.make_node(
+            'Clip',
+            inputs=[op.input('X')[0], min_name, max_name],
+            outputs=op.output('Out'), )
+        return [min_node, max_node, node]
+
+    def shape(self, op, block):
+        node = helper.make_node(
+            'Shape', inputs=op.input('Input'), outputs=op.output('Out'))
+        return node
+
+    def split(self, op, block):
+        sections = op.attr('sections')
+        if len(sections) > 0:
+            node = helper.make_node(
+                'Split',
+                inputs=op.input('X'),
+                outputs=op.output('Out'),
+                axis=op.attr('axis'),
+                split=sections)
+        else:
+            node = helper.make_node(
+                'Split',
+                inputs=op.input('X'),
+                outputs=op.output('Out'),
+                axis=op.attr('axis'))
+        return node
+
+    def slice(self, op, block):
+        axes = op.attr('axes')
+        starts = op.attr('starts')
+        ends = op.attr('ends')
+        axes_name = self.get_name(op.type, 'axes')
+        starts_name = self.get_name(op.type, 'starts')
+        ends_name = self.get_name(op.type, 'ends')
+
+        axes_node = self.make_constant_node(axes_name,
+                                            onnx_pb.TensorProto.INT64, axes)
+        starts_node = self.make_constant_node(starts_name,
+                                              onnx_pb.TensorProto.INT64, starts)
+        ends_node = self.make_constant_node(ends_name,
+                                            onnx_pb.TensorProto.INT64, ends)
+        node = helper.make_node(
+            "Slice",
+            inputs=[op.input('Input')[0], starts_name, ends_name, axes_name],
+            outputs=op.output('Out'), )
+        return [starts_node, ends_node, axes_node, node]
+
+    def fill_constant(self, op, block):
+        value = op.attr('value')
+        dtype = op.attr('dtype')
+        shape = op.attr('shape')
+        value = np.ones(shape) * value
+        if dtype == 2:
+            value = value.astype('int32')
+        node = helper.make_node(
+            'Constant',
+            inputs=[],
+            outputs=op.output('Out'),
+            value=helper.make_tensor(
+                name=op.output('Out')[0],
+                data_type=self.paddle_onnx_dtype_map[dtype],
+                dims=shape,
+                vals=value.tolist()))
+        return node
+
+    def transpose2(self, op, block):
+        node = helper.make_node(
+            'Transpose',
+            inputs=op.input('X'),
+            outputs=op.output('Out'),
+            perm=op.attr('axis'))
+        return node
+
+    def reshape2(self, op, block):
+        input_names = op.input_names
+        if len(op.input('ShapeTensor')) > 1:
+            cast_shape_nodes = list()
+            cast_shape_names = list()
+            for i in range(len(op.input('ShapeTensor'))):
+                dim = op.input('ShapeTensor')[i]
+                temp_name = self.get_name(op.type, 'shape.cast')
+                node = helper.make_node(
+                    'Cast',
+                    inputs=[dim],
+                    outputs=[temp_name],
+                    to=onnx_pb.TensorProto.INT64)
+                cast_shape_nodes.append(node)
+                cast_shape_names.append(temp_name)
+
+            temp_name = self.get_name(op.type, 'shape.concat')
+            shape_node = helper.make_node(
+                'Concat', inputs=cast_shape_names, outputs=[temp_name], axis=-1)
+            node = helper.make_node(
+                'Reshape',
+                inputs=[op.input('X')[0], temp_name],
+                outputs=op.output('Out'))
+            return cast_shape_nodes + [shape_node, node]
+        else:
+            temp_name = self.get_name(op.type, 'shape.cast')
+            cast_shape_node = helper.make_node(
+                'Cast',
+                inputs=op.input('ShapeTensor'),
+                outputs=[temp_name],
+                to=onnx_pb.TensorProto.INT64)
+            node = helper.make_node(
+                'Reshape',
+                inputs=[op.input('X')[0], temp_name],
+                outputs=op.output('Out'))
+            return [cast_shape_node, node]
+
+    def dropout(self, op, block):
+        dropout_mode = op.attr('dropout_implementation')
+        dropout_prob = op.attr('dropout_prob')
+        if dropout_mode == 'upscale_in_train':
+            node = helper.make_node(
+                'Identity', inputs=op.input('X'), outputs=op.output('Out'))
+            return node
+        elif dropout_mode == 'downgrade_in_infer':
+            scale_name = self.get_name(op.type, 'scale')
+            scale_node = self.make_constant_node(
+                scale_name, onnx_pb.TensorProto.FLOAT, 1 - dropout_prob)
+            node = helper.make_node(
+                "Mul",
+                inputs=[op.input('X')[0], scale_name],
+                outputs=op.output('Out'))
+            return [scale_node, node]
+        else:
+            raise Exception("Unexpected situation happend")
+
+    def reduce_mean(self, op, block):
+        node = helper.make_node(
+            'ReduceMean',
+            inputs=op.input('X'),
+            outputs=op.output('Out'),
+            axes=op.attr('dim'),
+            keepdims=op.attr('keep_dim'))
+        return node
+
+    def bilinear_interp(self, op, block):
+        input_names = op.input_names
+        coordinate_transformation_mode = 'half_pixel'
+        if op.attr('align_corners'):
+            coordinate_transformation_mode = 'align_corners'
+        if ('OutSize' in input_names and len(op.input('OutSize')) > 0) or (
+                'SizeTensor' in input_names and
+                len(op.input('SizeTensor')) > 0):
+            node_list = list()
+            roi_node = self.make_constant_node(
+                self.get_name(op.type, 'roi'), onnx_pb.TensorProto.FLOAT,
+                [1, 1, 1, 1, 1, 1, 1, 1])
+            roi_name = self.get_name(op.type, 'roi')
+            roi_node = self.make_constant_node(
+                roi_name, onnx_pb.TensorProto.FLOAT, [1, 1, 1, 1, 1, 1, 1, 1])
+            empty_name = self.get_name(op.type, 'empty')
+            empty_tensor = helper.make_tensor(
+                empty_name,
+                onnx_pb.TensorProto.FLOAT, (0, ),
+                np.array([]).astype('float32'),
+                raw=False)
+            empty_node = helper.make_node(
+                'Constant', [], outputs=[empty_name], value=empty_tensor)
+            shape_name0 = self.get_name(op.type, 'shape')
+            shape_node0 = helper.make_node(
+                'Shape', inputs=op.input('X'), outputs=[shape_name0])
+            starts_name = self.get_name(op.type, 'slice.starts')
+            starts_node = self.make_constant_node(
+                starts_name, onnx_pb.TensorProto.INT64, [0])
+            ends_name = self.get_name(op.type, 'slice.ends')
+            ends_node = self.make_constant_node(ends_name,
+                                                onnx_pb.TensorProto.INT64, [2])
+            shape_name1 = self.get_name(op.type, 'shape')
+            shape_node1 = helper.make_node(
+                'Slice',
+                inputs=[shape_name0, starts_name, ends_name],
+                outputs=[shape_name1])
+            node_list.extend([
+                roi_node, empty_node, shape_node0, starts_node, ends_node,
+                shape_node1
+            ])
+            #            shape_name2 = self.get_name(op.type, "shape.cast")
+            #            shape_node2 = helper.make_node(
+            #                'Cast',
+            #                inputs=op.input('OutSize'),
+            #                outputs=[shape_name2],
+            #                to=onnx_pb.TensorProto.INT64)
+            if 'OutSize' in input_names and len(op.input('OutSize')) > 0:
+                cast_shape_name = self.get_name(op.type, "shape.cast")
+                cast_shape_node = helper.make_node(
+                    'Cast',
+                    inputs=op.input('OutSize'),
+                    outputs=[cast_shape_name],
+                    to=onnx_pb.TensorProto.INT64)
+                node_list.append(cast_shape_node)
+            else:
+                concat_shape_name = self.get_name(op.type, "shape.concat")
+                concat_shape_node = helper.make_node(
+                    "Concat",
+                    inputs=op.input('SizeTensor'),
+                    outputs=[concat_shape_name],
+                    axis=0)
+                cast_shape_name = self.get_name(op.type, "shape.cast")
+                cast_shape_node = helper.make_node(
+                    'Cast',
+                    inputs=[concat_shape_name],
+                    outputs=[cast_shape_name],
+                    to=onnx_pb.TensorProto.INT64)
+                node_list.extend([concat_shape_node, cast_shape_node])
+            shape_name3 = self.get_name(op.type, "shape.concat")
+            shape_node3 = helper.make_node(
+                'Concat',
+                inputs=[shape_name1, cast_shape_name],
+                outputs=[shape_name3],
+                axis=0)
+            result_node = helper.make_node(
+                'Resize',
+                inputs=[op.input('X')[0], roi_name, empty_name, shape_name3],
+                outputs=op.output('Out'),
+                mode='linear',
+                coordinate_transformation_mode=coordinate_transformation_mode)
+            node_list.extend([shape_node3, result_node])
+            return node_list
+        elif 'Scale' in input_names and len(op.input('Scale')) > 0:
+            node = helper.make_node(
+                'Resize',
+                inputs=[op.input('X')[0], op.input('Scale')[0]],
+                outputs=op.output('Out'),
+                mode='linear',
+                coordinate_transformation_mode=coordinate_transformation_mode)
+        else:
+            out_shape = [op.attr('out_h'), op.attr('out_w')]
+            scale = op.attr('scale')
+            if out_shape.count(-1) > 0:
+                scale_name = self.get_name(op.type, 'scale')
+                scale_node = self.make_constant_node(scale_name,
+                                                     onnx_pb.TensorProto.FLOAT,
+                                                     [1, 1, scale, scale])
+                roi_name = self.get_name(op.type, 'roi')
+                roi_node = self.make_constant_node(roi_name,
+                                                   onnx_pb.TensorProto.FLOAT,
+                                                   [1, 1, 1, 1, 1, 1, 1, 1])
+                node = helper.make_node(
+                    'Resize',
+                    inputs=[op.input('X')[0], roi_name, scale_name],
+                    outputs=op.output('Out'),
+                    mode='nearest',
+                    coordinate_transformation_mode=coordinate_transformation_mode
+                )
+                return [scale_node, roi_node, node]
+            else:
+                raise Exception("Unexpected situation happend")
+        return node
+
+    def nearest_interp(self, op, block):
+        input_names = op.input_names
+        coordinate_transformation_mode = 'half_pixel'
+        if op.attr('align_corners'):
+            coordinate_transformation_mode = 'align_corners'
+        if 'OutSize' in input_names and len(op.input('OutSize')) > 0:
+            node = helper.make_node(
+                'Resize',
+                inputs=[op.input('X')[0], '', op.input('OutSize')[0]],
+                outputs=op.output('Out'),
+                mode='nearest',
+                coordinate_transformation_mode=coordinate_transformation_mode)
+        elif 'Scale' in input_names and len(op.input('Scale')) > 0:
+            node = helper.make_node(
+                'Resize',
+                inputs=[op.input('X')[0], op.input('Scale')[0]],
+                outputs=op.output('Out'),
+                mode='nearest',
+                coordinate_transformation_mode=coordinate_transformation_mode)
+        else:
+            out_shape = [op.attr('out_h'), op.attr('out_w')]
+            scale = op.attr('scale')
+            if out_shape.count(-1) > 0:
+                scale_name = self.get_name(op.type, 'scale')
+                scale_node = self.make_constant_node(scale_name,
+                                                     onnx_pb.TensorProto.FLOAT,
+                                                     [1, 1, scale, scale])
+                roi_name = self.get_name(op.type, 'roi')
+                roi_node = self.make_constant_node(roi_name,
+                                                   onnx_pb.TensorProto.FLOAT,
+                                                   [1, 1, 1, 1, 1, 1, 1, 1])
+                node = helper.make_node(
+                    'Resize',
+                    inputs=[op.input('X')[0], roi_name, scale_name],
+                    outputs=op.output('Out'),
+                    mode='nearest',
+                    coordinate_transformation_mode=coordinate_transformation_mode
+                )
+                return [scale_node, roi_node, node]
+            else:
+                raise Exception("Unexpected situation happend")
+        return node
+
+    def hard_sigmoid(self, op, block):
+        slope = op.attr('slope')
+        offset = op.attr('offset')
+        node = helper.make_node(
+            'HardSigmoid',
+            inputs=op.input('X'),
+            outputs=op.output('Out'),
+            alpha=slope,
+            beta=offset)
+        return node
+
+    def hard_swish(self, op, block):
+        min_name = self.get_name(op.type, 'min')
+        max_name = self.get_name(op.type, 'max')
+        scale_name = self.get_name(op.type, 'scale')
+        offset_name = self.get_name(op.type, 'offset')
+        min_node = self.make_constant_node(min_name, onnx_pb.TensorProto.FLOAT,
+                                           0)
+        max_node = self.make_constant_node(max_name, onnx_pb.TensorProto.FLOAT,
+                                           op.attr('threshold'))
+        scale_node = self.make_constant_node(scale_name,
+                                             onnx_pb.TensorProto.FLOAT,
+                                             op.attr('scale'))
+        offset_node = self.make_constant_node(offset_name,
+                                              onnx_pb.TensorProto.FLOAT,
+                                              op.attr('offset'))
+
+        name0 = self.get_name(op.type, 'add')
+        node0 = helper.make_node(
+            'Add', inputs=[op.input('X')[0], offset_name], outputs=[name0])
+        name1 = self.get_name(op.type, 'relu')
+        node1 = helper.make_node(
+            'Clip',
+            inputs=[name0, min_name, max_name],
+            outputs=[name1], )
+        name2 = self.get_name(op.type, 'mul')
+        node2 = helper.make_node(
+            'Mul', inputs=[op.input('X')[0], name1], outputs=[name2])
+        node3 = helper.make_node(
+            'Div', inputs=[name2, scale_name], outputs=op.output('Out'))
+        return [
+            min_node, max_node, scale_node, offset_node, node0, node1, node2,
+            node3
+        ]
+
+    def elementwise_mul(self, op, block):
+        axis = op.attr('axis')
+        x_shape = block.var(op.input('X')[0]).shape
+        y_shape = block.var(op.input('Y')[0]).shape
+        if len(y_shape) == 1 and axis == 1:
+            shape_name = self.get_name(op.type, 'shape')
+            shape_value = [1] * len(x_shape)
+            shape_value[axis] = y_shape[0]
+            shape_node = self.make_constant_node(
+                shape_name, onnx_pb.TensorProto.INT64, shape_value)
+            temp_value = self.get_name(op.type, 'temp')
+            y_node = helper.make_node(
+                'Reshape',
+                inputs=[op.input('Y')[0], shape_name],
+                outputs=[temp_value])
+            node = helper.make_node(
+                'Mul',
+                inputs=[op.input('X')[0], temp_value],
+                outputs=op.output('Out'))
+            return [shape_node, y_node, node]
+        elif len(x_shape) == len(y_shape):
+            node = helper.make_node(
+                'Mul',
+                inputs=[op.input('X')[0], op.input('Y')[0]],
+                outputs=op.output('Out'))
+            return node
+        else:
+            raise Excpetion("Unexpected situation happend in elementwise_add")
+        return node
+
+    def feed(self, op, block):
+        name = op.output('Out')[0]
+        var = block.var(name)
+        tensor_info = helper.make_tensor_value_info(
+            name=name,
+            shape=var.shape,
+            elem_type=self.paddle_onnx_dtype_map[var.dtype])
+        return tensor_info
+
+    def fetch(self, op, block):
+        name = op.input('X')[0]
+        var = block.var(name)
+        tensor_info = helper.make_tensor_value_info(
+            name=name,
+            shape=var.shape,
+            elem_type=self.paddle_onnx_dtype_map[var.dtype])
+        return tensor_info
+
+    def unsqueeze2(self, op, block):
+        node = helper.make_node(
+            'Unsqueeze',
+            inputs=op.input('X'),
+            outputs=op.output('Out'),
+            axes=op.attr('axes'))
+        return node
+
+    def arg_max(self, op, block):
+        node = helper.make_node(
+            'ArgMax',
+            inputs=op.input('X'),
+            outputs=op.output('Out'),
+            axis=op.attr('axis'),
+            keepdims=0)
+        return node
+
+    def reciprocal(self, op, block):
+        inputs = op.input(op.input_names[0])
+        outputs = op.output(op.output_names[0])
+        node = helper.make_node('Reciprocal', inputs=inputs, outputs=outputs)
+        return node
+
+    def im2sequence(self, op, block):
+        from .paddle_custom_layer.im2sequence import im2sequence
+        return im2sequence(op, block)

x2paddle/op_mapper/tf_op_mapper.py

@@ -85,7 +85,8 @@ class TFOpMapper(OpMapper):
 
         not_placeholder = list()
         for name in self.graph.input_nodes:
-            if self.graph.get_node(name).layer_type != "Placeholder":
+            if self.graph.get_node(name).layer_type != "Placeholder" \
+               and self.graph.get_node(name).layer_type != "OneShotIterator":
                 not_placeholder.append(name)
         for name in not_placeholder:
             idx = self.graph.input_nodes.index(name)
@@ -113,9 +114,8 @@ class TFOpMapper(OpMapper):
             else:
                 unsupported_ops.add(op)
         if len(unsupported_ops) > 0:
-            sys.stderr.write(
-                "=========={} Ops are not supported yet======\n".format(
-                    len(unsupported_ops)))
+            sys.stderr.write("=========={} Ops are not supported yet======\n".
+                             format(len(unsupported_ops)))
             for op in unsupported_ops:
                 sys.stderr.write("========== {} ==========\n".format(op))
             sys.exit(-1)
@@ -140,10 +140,8 @@ class TFOpMapper(OpMapper):
             pd_param_name = list(param.values())[0]
             tf_param = node.get_attr(tf_param_name)
             attr[pd_param_name] = tf_param
-        node.fluid_code.add_layer(op_info[0],
-                                  inputs=input,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            op_info[0], inputs=input, output=node, param_attr=attr)
 
     def elementwise_map(self, node):
         assert node.layer_type in self.elementwise_ops
@@ -178,21 +176,21 @@ class TFOpMapper(OpMapper):
                         0] == y_shape[-1] and y_shape.count(-1) < 1:
                     shape = [1, x_shape[0], 1, 1]
                     attr = {"shape": shape}
-                    node.fluid_code.add_layer("reshape",
-                                              inputs=x_input,
-                                              output="reshape_x",
-                                              param_attr=attr)
+                    node.fluid_code.add_layer(
+                        "reshape",
+                        inputs=x_input,
+                        output="reshape_x",
+                        param_attr=attr)
                     if y_shape[0] != 1:
                         attr = {"expand_times": [y_shape[0], 1, 1, 1]}
-                        node.fluid_code.add_layer("expand",
-                                                  inputs="reshape_x",
-                                                  output="reshape_x",
-                                                  param_attr=attr)
+                        node.fluid_code.add_layer(
+                            "expand",
+                            inputs="reshape_x",
+                            output="reshape_x",
+                            param_attr=attr)
                     inputs = {"x": "reshape_x", "y": y_input}
-                    node.fluid_code.add_layer(op_type,
-                                              inputs=inputs,
-                                              output=node,
-                                              param_attr=None)
+                    node.fluid_code.add_layer(
+                        op_type, inputs=inputs, output=node, param_attr=None)
                     return
                 else:
                     raise Exception("Unexpected situation happend")
@@ -204,10 +202,8 @@ class TFOpMapper(OpMapper):
                 axis = -1
             attr = {"axis": axis}
             inputs = {"x": x_input, "y": y_input}
-            node.fluid_code.add_layer(op_type,
-                                      inputs=inputs,
-                                      output=node,
-                                      param_attr=attr)
+            node.fluid_code.add_layer(
+                op_type, inputs=inputs, output=node, param_attr=attr)
             return
 
         is_sub_seq = True
@@ -241,10 +237,8 @@ class TFOpMapper(OpMapper):
                 if len(x_expand_times) == 4 and x.tf_data_format == "NHWC":
                     x_expand_times = [x_expand_times[i] for i in [0, 3, 1, 2]]
                 attr = {"expand_times": x_expand_times}
-                node.fluid_code.add_layer("expand",
-                                          inputs=x_input,
-                                          output="x_tmp",
-                                          param_attr=attr)
+                node.fluid_code.add_layer(
+                    "expand", inputs=x_input, output="x_tmp", param_attr=attr)
                 x_input = "x_tmp"
             if y_need_expand:
                 if len(y_expand_times) == 3 and y.tf_data_format == "NHWC":
@@ -252,16 +246,12 @@ class TFOpMapper(OpMapper):
                 if len(y_expand_times) == 4 and y.tf_data_format == "NHWC":
                     y_expand_times = [y_expand_times[i] for i in [0, 3, 1, 2]]
                 attr = {"expand_times": y_expand_times}
-                node.fluid_code.add_layer("expand",
-                                          inputs=y_input,
-                                          output="y_tmp",
-                                          param_attr=attr)
+                node.fluid_code.add_layer(
+                    "expand", inputs=y_input, output="y_tmp", param_attr=attr)
                 y_input = "y_tmp"
         inputs = {"x": x_input, "y": y_input}
-        node.fluid_code.add_layer(op_type,
-                                  inputs=inputs,
-                                  output=node,
-                                  param_attr=None)
+        node.fluid_code.add_layer(
+            op_type, inputs=inputs, output=node, param_attr=None)
 
     def Placeholder(self, node):
         shape = node.out_shapes[0]
@@ -282,10 +272,11 @@ class TFOpMapper(OpMapper):
         if shape[0] < 0:
             self.batch_node = node
 
-        node.fluid_code.add_layer("data",
-                                  inputs=None,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            "data", inputs=None, output=node, param_attr=attr)
+
+    def OneShotIterator(self, node):
+        return self.Placeholder(node)
 
     def Const(self, node):
         shape = node.out_shapes[0]
@@ -304,8 +295,8 @@ class TFOpMapper(OpMapper):
                 shape = [shape[i] for i in [0, 3, 1, 2]]
             if len(shape) == 3:
                 shape = [shape[i] for i in [2, 0, 1]]
-                self.weights[node.layer_name] = numpy.transpose(
-                    node.value, (2, 0, 1))
+                self.weights[node.layer_name] = numpy.transpose(node.value,
+                                                                (2, 0, 1))
         elif node.tf_data_format == "NCHW":
             if len(shape) == 4:
                 self.graph.data_format_propagation(node)
@@ -316,10 +307,8 @@ class TFOpMapper(OpMapper):
             'name': string(node.layer_name),
             'default_initializer': initializer
         }
-        node.fluid_code.add_layer("create_parameter",
-                                  inputs=None,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            "create_parameter", inputs=None, output=node, param_attr=attr)
 
     def Transpose(self, node):
         input = self.graph.get_node(node.layer.input[0], copy=True)
@@ -358,16 +347,12 @@ class TFOpMapper(OpMapper):
             node.tf_data_format = [tf_data_format[i] for i in perm]
             node.pd_data_format = [pd_data_format[i] for i in perm]
             attr = {'perm': new_perm}
-            node.fluid_code.add_layer("transpose",
-                                      inputs=input,
-                                      output=node,
-                                      param_attr=attr)
+            node.fluid_code.add_layer(
+                "transpose", inputs=input, output=node, param_attr=attr)
         elif len(node.out_shapes[0]) != 4:
             attr = {'perm': perm}
-            node.fluid_code.add_layer("transpose",
-                                      inputs=input,
-                                      output=node,
-                                      param_attr=attr)
+            node.fluid_code.add_layer(
+                "transpose", inputs=input, output=node, param_attr=attr)
         else:
             raise Exception("Unexpected situation happend in Transpose OP")
 
@@ -397,10 +382,8 @@ class TFOpMapper(OpMapper):
             "pool_padding": string(pad_mode),
             "pool_stride": strides[2:4]
         }
-        node.fluid_code.add_layer("pool2d",
-                                  inputs=input,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            "pool2d", inputs=input, output=node, param_attr=attr)
 
     def Conv2D(self, node):
         input = self.graph.get_node(node.layer.input[0], copy=True)
@@ -440,10 +423,8 @@ class TFOpMapper(OpMapper):
             "dilation": dilations[2:4],
             "padding": string(pad_mode)
         }
-        node.fluid_code.add_layer("conv2d",
-                                  inputs=input,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            "conv2d", inputs=input, output=node, param_attr=attr)
 
     def BiasAdd(self, node):
         input = self.graph.get_node(node.layer.input[0], copy=True)
@@ -453,10 +434,8 @@ class TFOpMapper(OpMapper):
             axis = 1
         inputs = {"x": input, "y": bias}
         attr = {"axis": axis}
-        node.fluid_code.add_layer("elementwise_add",
-                                  inputs=inputs,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            "elementwise_add", inputs=inputs, output=node, param_attr=attr)
 
     def FusedBatchNorm(self, node):
         input = self.graph.get_node(node.layer.input[0], copy=True)
@@ -487,10 +466,11 @@ class TFOpMapper(OpMapper):
             "is_test": True
         }
 
-        node.fluid_code.add_layer("batch_norm",
-                                  inputs=input,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            "batch_norm", inputs=input, output=node, param_attr=attr)
+
+    def FusedBatchNormV3(self, node):
+        return self.FusedBatchNorm(node)
 
     def DepthwiseConv2dNative(self, node):
         input = self.graph.get_node(node.layer.input[0], copy=True)
@@ -532,10 +512,8 @@ class TFOpMapper(OpMapper):
             "use_cudnn": False,
             "padding": string(pad_mode)
         }
-        node.fluid_code.add_layer("conv2d",
-                                  inputs=input,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            "conv2d", inputs=input, output=node, param_attr=attr)
 
     def Reshape(self, node):
         input = self.graph.get_node(node.layer.input[0], copy=True)
@@ -555,18 +533,17 @@ class TFOpMapper(OpMapper):
                 attr = {"shape": shape}
                 self.add_omit_nodes(param.layer_name, node.layer_name)
             else:
-                assert len(param.out_shapes[0]
-                           ) == 1, "Unexpected situation of shape parameter"
+                assert len(param.out_shapes[
+                    0]) == 1, "Unexpected situation of shape parameter"
                 attr = {"shape": [-1]}
-                node.fluid_code.add_layer("reshape",
-                                          inputs=param,
-                                          output="shape_param",
-                                          param_attr=attr)
+                node.fluid_code.add_layer(
+                    "reshape",
+                    inputs=param,
+                    output="shape_param",
+                    param_attr=attr)
                 attr = {"num_or_sections": param.out_shapes[0][0], "dim": 0}
-                node.fluid_code.add_layer("split",
-                                          inputs="shape_param",
-                                          output=node,
-                                          param_attr=attr)
+                node.fluid_code.add_layer(
+                    "split", inputs="shape_param", output=node, param_attr=attr)
                 new_param = "["
                 for i in range(param.out_shapes[0][0]):
                     new_param += (node.layer_name + "[{}]".format(i) + ", ")
@@ -594,14 +571,10 @@ class TFOpMapper(OpMapper):
         if len(input.out_shapes[0]) == 4 and node.tf_data_format == "NHWC":
             if len(attr["shape"]) < 3:
                 perm = {"perm": [0, 2, 3, 1]}
-                node.fluid_code.add_layer("transpose",
-                                          inputs=input,
-                                          output=node,
-                                          param_attr=perm)
-                node.fluid_code.add_layer("reshape",
-                                          inputs=node,
-                                          output=node,
-                                          param_attr=attr)
+                node.fluid_code.add_layer(
+                    "transpose", inputs=input, output=node, param_attr=perm)
+                node.fluid_code.add_layer(
+                    "reshape", inputs=node, output=node, param_attr=attr)
                 return
 
         if len(attr["shape"]) == 4 and node.tf_data_format == "NHWC":
@@ -610,27 +583,19 @@ class TFOpMapper(OpMapper):
                 attr["shape"] = [attr["shape"][i] for i in [0, 3, 1, 2]]
             else:
                 perm = {"perm": [0, 2, 3, 1]}
-                node.fluid_code.add_layer("transpose",
-                                          inputs=input,
-                                          output=node,
-                                          param_attr=perm)
-                node.fluid_code.add_layer("reshape",
-                                          inputs=node,
-                                          output=node,
-                                          param_attr=attr)
+                node.fluid_code.add_layer(
+                    "transpose", inputs=input, output=node, param_attr=perm)
+                node.fluid_code.add_layer(
+                    "reshape", inputs=node, output=node, param_attr=attr)
                 perm = {"perm": [0, 3, 1, 2]}
-                node.fluid_code.add_layer("transpose",
-                                          inputs=node,
-                                          output=node,
-                                          param_attr=perm)
+                node.fluid_code.add_layer(
+                    "transpose", inputs=node, output=node, param_attr=perm)
                 return
         if len(attr["shape"]) == 5:
             attr["shape"] = [attr["shape"][i] for i in [0, 1, 4, 2, 3]]
 
-        node.fluid_code.add_layer("reshape",
-                                  inputs=input,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            "reshape", inputs=input, output=node, param_attr=attr)
 
     def AvgPool(self, node):
         input = self.graph.get_node(node.layer.input[0], copy=True)
@@ -658,10 +623,8 @@ class TFOpMapper(OpMapper):
             "pool_stride": strides[2:4],
             "pool_padding": string(pad_mode)
         }
-        node.fluid_code.add_layer("pool2d",
-                                  inputs=input,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            "pool2d", inputs=input, output=node, param_attr=attr)
 
     def SplitV(self, node):
         input = self.graph.get_node(node.layer.input[0], copy=True)
@@ -678,28 +641,24 @@ class TFOpMapper(OpMapper):
             "num_or_sections": num_sections.value.tolist(),
             "dim": dim.value
         }
-        node.fluid_code.add_layer("split",
-                                  inputs=input,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            "split", inputs=input, output=node, param_attr=attr)
 
     def ConcatV2(self, node):
         inputs = [
-            self.graph.get_node(name, copy=True)
-            for name in node.layer.input[:-1]
+            self.graph.get_node(
+                name, copy=True) for name in node.layer.input[:-1]
         ]
         axis = self.graph.get_node(node.layer.input[-1], copy=True)
         assert axis.layer_type == "Const"
         self.add_omit_nodes(axis.layer_name, node.layer_name)
         axis = axis.value
-        if inputs[0].tf_data_format == "NHWC" and len(
-                inputs[0].out_shapes[0]) == 4:
+        if inputs[0].tf_data_format == "NHWC" and len(inputs[0].out_shapes[
+                0]) == 4:
             axis = nhwc_dim_to_nchw(inputs[0], axis)
         attr = {"axis": axis}
-        node.fluid_code.add_layer("concat",
-                                  inputs=inputs,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            "concat", inputs=inputs, output=node, param_attr=attr)
 
     def Tile(self, node):
         input = self.graph.get_node(node.layer.input[0], copy=True)
@@ -712,25 +671,24 @@ class TFOpMapper(OpMapper):
         if input.tf_data_format == "NHWC":
             if len(input.out_shapes[0]) == 4:
                 expand_times = [expand_times[i] for i in [0, 3, 1, 2]]
-            elif len(input.out_shape[0]) == 3:
+            elif len(input.out_shapes[0]) == 3:
                 expand_times = [expand_times[i] for i in [2, 0, 1]]
         for i in range(len(expand_times)):
             if expand_times[i] < 0:
                 expand_times[i] = 1
 
         attr = {"expand_times": expand_times}
-        node.fluid_code.add_layer("expand",
-                                  inputs=input,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            "expand", inputs=input, output=node, param_attr=attr)
 
     def Pack(self, node):
         inputs = [
-            self.graph.get_node(name, copy=True) for name in node.layer.input
+            self.graph.get_node(
+                name, copy=True) for name in node.layer.input
         ]
         axis = node.get_attr("axis")
-        if inputs[0].tf_data_format == "NHWC" and len(
-                inputs[0].out_shapes[0]) == 4:
+        if inputs[0].tf_data_format == "NHWC" and len(inputs[0].out_shapes[
+                0]) == 4:
             tf_data_format = list(inputs[0].tf_data_format)
             tf_data_format.insert(axis, str(len(tf_data_format)))
             axis = nhwc_dim_to_nchw(inputs[0], axis)
@@ -740,10 +698,8 @@ class TFOpMapper(OpMapper):
             node.pd_data_format = "".join(pd_data_format)
 
         attr = {"axis": axis}
-        node.fluid_code.add_layer("stack",
-                                  inputs=inputs,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            "stack", inputs=inputs, output=node, param_attr=attr)
 
     def Pad(self, node):
         input = self.graph.get_node(node.layer.input[0], copy=True)
@@ -760,10 +716,8 @@ class TFOpMapper(OpMapper):
                 paddings = paddings[4:]
                 pad_op = "pad2d"
         attr = {"paddings": paddings}
-        node.fluid_code.add_layer(pad_op,
-                                  inputs=input,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            pad_op, inputs=input, output=node, param_attr=attr)
 
     def MirrorPad(self, node):
         input = self.graph.get_node(node.layer.input[0], copy=True)
@@ -782,10 +736,8 @@ class TFOpMapper(OpMapper):
                 paddings = paddings[4:]
                 pad_op = "pad2d"
         attr = {"paddings": paddings, "mode": string("reflect")}
-        node.fluid_code.add_layer(pad_op,
-                                  inputs=input,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            pad_op, inputs=input, output=node, param_attr=attr)
 
     def Range(self, node):
         start = self.graph.get_node(node.layer.input[0], copy=True)
@@ -809,10 +761,8 @@ class TFOpMapper(OpMapper):
 
         inputs = {"start": start, "end": limit, "step": delta}
         attr = {"dtype": string(node.dtype)}
-        node.fluid_code.add_layer("range",
-                                  inputs=inputs,
-                                  output=node,
-                                  param_attr=None)
+        node.fluid_code.add_layer(
+            "range", inputs=inputs, output=node, param_attr=attr)
 
     def Mean(self, node):
         input = self.graph.get_node(node.layer.input[0], copy=True)
@@ -826,10 +776,8 @@ class TFOpMapper(OpMapper):
                 dims[i] = nhwc_dim_to_nchw(input, dims[i])
 
         attr = {"dim": dims, "keep_dim": keep_dims}
-        node.fluid_code.add_layer("reduce_mean",
-                                  inputs=input,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            "reduce_mean", inputs=input, output=node, param_attr=attr)
 
     def MatMul(self, node):
         x = self.graph.get_node(node.layer.input[0], copy=True)
@@ -843,15 +791,11 @@ class TFOpMapper(OpMapper):
             shape = x.out_shapes[0]
             shape[-1] = y.out_shapes[0][0]
             attr = {"shape": shape}
-            node.fluid_code.add_layer("reshape",
-                                      inputs=x,
-                                      output=x,
-                                      param_attr=attr)
+            node.fluid_code.add_layer(
+                "reshape", inputs=x, output=x, param_attr=attr)
         attr = {"transpose_x": transpose_a, "transpose_y": transpose_b}
-        node.fluid_code.add_layer("matmul",
-                                  inputs=inputs,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            "matmul", inputs=inputs, output=node, param_attr=attr)
 
     def ArgMax(self, node):
         input = self.graph.get_node(node.layer.input[0], copy=True)
@@ -862,10 +806,8 @@ class TFOpMapper(OpMapper):
         if input.tf_data_format == "NHWC" and len(input.out_shapes[0]) == 4:
             axis = nhwc_dim_to_nchw(input, axis)
         attr = {"axis": axis}
-        node.fluid_code.add_layer("argmax",
-                                  inputs=input,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            "argmax", inputs=input, output=node, param_attr=attr)
 
     def StridedSlice(self, node):
         input = self.graph.get_node(node.layer.input[0], copy=True)
@@ -903,16 +845,12 @@ class TFOpMapper(OpMapper):
             x = shrink_axis_mask >> i & 1
             if x == 1:
                 squeeze_dims.append(i)
-        node.fluid_code.add_layer("slice",
-                                  inputs=input,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            "slice", inputs=input, output=node, param_attr=attr)
         if shrink_axis_mask > 0 and len(input.out_shapes[0]) == 5:
             attr = {"axes": squeeze_dims}
-            node.fluid_code.add_layer("squeeze",
-                                      inputs=node,
-                                      output=node,
-                                      param_attr=attr)
+            node.fluid_code.add_layer(
+                "squeeze", inputs=node, output=node, param_attr=attr)
 
     def Slice(self, node):
         input = self.graph.get_node(node.layer.input[0], copy=True)
@@ -944,10 +882,8 @@ class TFOpMapper(OpMapper):
             "starts": begin,
             "ends": size
         }
-        node.fluid_code.add_layer("slice",
-                                  inputs=input,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            "slice", inputs=input, output=node, param_attr=attr)
 
     def Conv2DBackpropInput(self, node):
         out_shape = self.graph.get_node(node.layer.input[0], copy=True)
@@ -997,10 +933,8 @@ class TFOpMapper(OpMapper):
             "padding": string(pad_mode),
             "output_size": out_shape[1:3]
         }
-        node.fluid_code.add_layer("conv2d_transpose",
-                                  inputs=input,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            "conv2d_transpose", inputs=input, output=node, param_attr=attr)
 
     def Max(self, node):
         input = self.graph.get_node(node.layer.input[0], copy=True)
@@ -1012,10 +946,8 @@ class TFOpMapper(OpMapper):
             dim = nhwc_dim_to_nchw(input, dim)
 
         attr = {"dim": dim, "keep_dim": keep_dims}
-        node.fluid_code.add_layer("reduce_max",
-                                  inputs=input,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            "reduce_max", inputs=input, output=node, param_attr=attr)
 
     def Sum(self, node):
         input = self.graph.get_node(node.layer.input[0], copy=True)
@@ -1027,19 +959,15 @@ class TFOpMapper(OpMapper):
             dim = nhwc_dim_to_nchw(input, dim)
 
         attr = {"dim": dim, "keep_dim": keep_dims}
-        node.fluid_code.add_layer("reduce_sum",
-                                  inputs=input,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            "reduce_sum", inputs=input, output=node, param_attr=attr)
 
     def Cast(self, node):
         input = self.graph.get_node(node.layer.input[0], copy=True)
         dtype = node.dtype_map[node.get_attr('DstT')]
         attr = {"dtype": string(dtype)}
-        node.fluid_code.add_layer("cast",
-                                  inputs=input,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            "cast", inputs=input, output=node, param_attr=attr)
 
     def Split(self, node):
         dim = self.graph.get_node(node.layer.input[0], copy=True)
@@ -1051,10 +979,8 @@ class TFOpMapper(OpMapper):
             dim = nhwc_dim_to_nchw(input, dim)
 
         attr = {"num_or_sections": num_split, "dim": dim}
-        node.fluid_code.add_layer("split",
-                                  inputs=input,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            "split", inputs=input, output=node, param_attr=attr)
 
     def Squeeze(self, node):
         input = self.graph.get_node(node.layer.input[0], copy=True)
@@ -1063,10 +989,8 @@ class TFOpMapper(OpMapper):
             for i in range(len(squeeze_dims)):
                 squeeze_dims[i] = nhwc_dim_to_nchw(input, squeeze_dims[i])
         attr = {"axes": squeeze_dims}
-        node.fluid_code.add_layer("squeeze",
-                                  inputs=input,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            "squeeze", inputs=input, output=node, param_attr=attr)
 
     def Softmax(self, node):
         input = self.graph.get_node(node.layer.input[0], copy=True)
@@ -1076,10 +1000,8 @@ class TFOpMapper(OpMapper):
         if input.tf_data_format == "NHWC" and len(input.out_shapes[0]) == 4:
             axis = nhwc_dim_to_nchw(input, axis)
         attr = {"axis": axis}
-        node.fluid_code.add_layer("softmax",
-                                  inputs=input,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            "softmax", inputs=input, output=node, param_attr=attr)
 
     def ResizeNearestNeighbor(self, node):
         input = self.graph.get_node(node.layer.input[0], copy=True)
@@ -1088,14 +1010,12 @@ class TFOpMapper(OpMapper):
         if resize_shape.layer_type == "Const":
             resize_shape = resize_shape.value.tolist()
         else:
-            resize_shape = self.decoder.infer_shape_tensor(
-                resize_shape, node.out_shapes[0])
+            resize_shape = self.decoder.infer_shape_tensor(resize_shape,
+                                                           node.out_shapes[0])
         align_corners = node.get_attr("align_corners")
         attr = {"align_corners": align_corners, "out_shape": resize_shape}
-        node.fluid_code.add_layer("resize_nearest",
-                                  inputs=input,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            "resize_nearest", inputs=input, output=node, param_attr=attr)
 
     def ResizeBilinear(self, node):
         input = self.graph.get_node(node.layer.input[0], copy=True)
@@ -1104,27 +1024,23 @@ class TFOpMapper(OpMapper):
         if resize_shape.layer_type == "Const":
             resize_shape = resize_shape.value.tolist()
         else:
-            resize_shape = self.decoder.infer_shape_tensor(
-                resize_shape, node.out_shapes[0])
+            resize_shape = self.decoder.infer_shape_tensor(resize_shape,
+                                                           node.out_shapes[0])
         align_corners = node.get_attr("align_corners")
         attr = {
             "align_corners": align_corners,
             "out_shape": resize_shape,
             "align_mode": 1
         }
-        node.fluid_code.add_layer("resize_bilinear",
-                                  inputs=input,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            "resize_bilinear", inputs=input, output=node, param_attr=attr)
 
     def GreaterEqual(self, node):
         x = self.graph.get_node(node.layer.input[0], copy=True)
         y = self.graph.get_node(node.layer.input[1], copy=True)
         inputs = {"x": x, "y": y}
-        node.fluid_code.add_layer("greater_equal",
-                                  inputs=inputs,
-                                  output=node,
-                                  param_attr=None)
+        node.fluid_code.add_layer(
+            "greater_equal", inputs=inputs, output=node, param_attr=None)
 
     def RandomUniform(self, node):
         shape = self.graph.get_node(node.layer.input[0], copy=True)
@@ -1138,26 +1054,21 @@ class TFOpMapper(OpMapper):
         attr = {"shape": shape, "min": 0.0, "max": 0.9999}
         if shape[0] < 0:
             input = self.batch_node
-            node.fluid_code.add_layer("uniform_random_batch_size_like",
-                                      inputs=input,
-                                      output=node,
-                                      param_attr=attr)
+            node.fluid_code.add_layer(
+                "uniform_random_batch_size_like",
+                inputs=input,
+                output=node,
+                param_attr=attr)
         else:
-            node.fluid_code.add_layer("uniform_random",
-                                      inputs=None,
-                                      output=node,
-                                      param_attr=attr)
+            node.fluid_code.add_layer(
+                "uniform_random", inputs=None, output=node, param_attr=attr)
 
     def SquaredDifference(self, node):
         x = self.graph.get_node(node.layer.input[0], copy=True)
         y = self.graph.get_node(node.layer.input[1], copy=True)
         inputs = {"x": x, "y": y}
-        node.fluid_code.add_layer("elementwise_sub",
-                                  inputs=inputs,
-                                  output=node,
-                                  param_attr=None)
+        node.fluid_code.add_layer(
+            "elementwise_sub", inputs=inputs, output=node, param_attr=None)
         inputs = {"x": node, "y": node}
-        node.fluid_code.add_layer("elementwise_mul",
-                                  inputs=inputs,
-                                  output=node,
-                                  param_attr=None)
+        node.fluid_code.add_layer(
+            "elementwise_mul", inputs=inputs, output=node, param_attr=None)

x2paddle/op_mapper/tf_op_mapper_nhwc.py

@@ -43,6 +43,7 @@ class TFOpMapperNHWC(OpMapper):
         'Sqrt': ['sqrt'],
         'swish_f32': ['swish'],
         'Tanh': ['tanh'],
+        'Softplus': ['softplus'],
         'LeakyRelu': ['leaky_relu', {
             'alpha': 'alpha'
         }]
@@ -128,26 +129,18 @@ class TFOpMapperNHWC(OpMapper):
 
         if len(input.out_shapes[0]) == 4 and op_info[0] != 'shape':
             attr1 = {"perm": [0, 3, 1, 2]}
-            node.fluid_code.add_layer('transpose',
-                                      inputs=input,
-                                      output=node,
-                                      param_attr=attr1)
+            node.fluid_code.add_layer(
+                'transpose', inputs=input, output=node, param_attr=attr1)
             input = node
-            node.fluid_code.add_layer(op_info[0],
-                                      inputs=input,
-                                      output=node,
-                                      param_attr=attr)
+            node.fluid_code.add_layer(
+                op_info[0], inputs=input, output=node, param_attr=attr)
             input = node
             attr2 = {"perm": [0, 2, 3, 1]}
-            node.fluid_code.add_layer('transpose',
-                                      inputs=input,
-                                      output=node,
-                                      param_attr=attr2)
+            node.fluid_code.add_layer(
+                'transpose', inputs=input, output=node, param_attr=attr2)
         else:
-            node.fluid_code.add_layer(op_info[0],
-                                      inputs=input,
-                                      output=node,
-                                      param_attr=attr)
+            node.fluid_code.add_layer(
+                op_info[0], inputs=input, output=node, param_attr=attr)
 
     def elementwise_map(self, node):
         assert node.layer_type in self.elementwise_ops
@@ -208,42 +201,37 @@ class TFOpMapperNHWC(OpMapper):
                         raise Exception("Unexpected situation happend")
             if x_need_expand:
                 attr = {"expand_times": x_expand_times}
-                node.fluid_code.add_layer("expand",
-                                          inputs=x_input,
-                                          output="x_tmp",
-                                          param_attr=attr)
+                node.fluid_code.add_layer(
+                    "expand", inputs=x_input, output="x_tmp", param_attr=attr)
                 x_input = "x_tmp"
             if y_need_expand:
                 attr = {"expand_times": y_expand_times}
-                node.fluid_code.add_layer("expand",
-                                          inputs=y_input,
-                                          output="y_tmp",
-                                          param_attr=attr)
+                node.fluid_code.add_layer(
+                    "expand", inputs=y_input, output="y_tmp", param_attr=attr)
                 y_input = "y_tmp"
         if len(x_shape) == 4 and len(y_shape) == 4:
-            node.fluid_code.add_layer("transpose",
-                                      inputs=x_input,
-                                      output=x_input,
-                                      param_attr={'perm': [0, 3, 1, 2]})
-            node.fluid_code.add_layer("transpose",
-                                      inputs=y_input,
-                                      output=y_input,
-                                      param_attr={'perm': [0, 3, 1, 2]})
+            node.fluid_code.add_layer(
+                "transpose",
+                inputs=x_input,
+                output=x_input,
+                param_attr={'perm': [0, 3, 1, 2]})
+            node.fluid_code.add_layer(
+                "transpose",
+                inputs=y_input,
+                output=y_input,
+                param_attr={'perm': [0, 3, 1, 2]})
             inputs = {"x": x_input, "y": y_input}
-            node.fluid_code.add_layer(op_type,
-                                      inputs=inputs,
-                                      output=node,
-                                      param_attr=None)
-            node.fluid_code.add_layer("transpose",
-                                      inputs=node,
-                                      output=node,
-                                      param_attr={'perm': [0, 2, 3, 1]})
+            node.fluid_code.add_layer(
+                op_type, inputs=inputs, output=node, param_attr=None)
+            node.fluid_code.add_layer(
+                "transpose",
+                inputs=node,
+                output=node,
+                param_attr={'perm': [0, 2, 3, 1]})
         else:
             inputs = {"x": x_input, "y": y_input}
-            node.fluid_code.add_layer(op_type,
-                                      inputs=inputs,
-                                      output=node,
-                                      param_attr=None)
+            node.fluid_code.add_layer(
+                op_type, inputs=inputs, output=node, param_attr=None)
 
     def Placeholder(self, node):
         shape = node.out_shapes[0]
@@ -259,10 +247,8 @@ class TFOpMapperNHWC(OpMapper):
             'append_batch_size': False
         }
 
-        node.fluid_code.add_layer("data",
-                                  inputs=None,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            "data", inputs=None, output=node, param_attr=attr)
 
     def Const(self, node):
         shape = node.out_shapes[0]
@@ -282,10 +268,8 @@ class TFOpMapperNHWC(OpMapper):
             'name': string(node.layer_name),
             'default_initializer': initializer
         }
-        node.fluid_code.add_layer("create_parameter",
-                                  inputs=None,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            "create_parameter", inputs=None, output=node, param_attr=attr)
 
     def Transpose(self, node):
         input = self.graph.get_node(node.layer.input[0], copy=True)
@@ -296,10 +280,8 @@ class TFOpMapperNHWC(OpMapper):
         perm = perm.value.tolist()
 
         attr = {'perm': perm}
-        node.fluid_code.add_layer("transpose",
-                                  inputs=input,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            "transpose", inputs=input, output=node, param_attr=attr)
 
     def MaxPool(self, node):
         input = self.graph.get_node(node.layer.input[0], copy=True)
@@ -316,10 +298,8 @@ class TFOpMapperNHWC(OpMapper):
 
         if not channel_first:
             attr = {"perm": [0, 3, 1, 2]}
-            node.fluid_code.add_layer("transpose",
-                                      inputs=input,
-                                      output=node,
-                                      param_attr=attr)
+            node.fluid_code.add_layer(
+                "transpose", inputs=input, output=node, param_attr=attr)
             in_shape = [in_shape[i] for i in [0, 3, 1, 2]]
             strides = [strides[i] for i in [0, 3, 1, 2]]
             k_size = [k_size[i] for i in [0, 3, 1, 2]]
@@ -331,17 +311,13 @@ class TFOpMapperNHWC(OpMapper):
             "pool_stride": strides[2:4],
             "pool_padding": string(pad_mode)
         }
-        node.fluid_code.add_layer("pool2d",
-                                  inputs=input,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            "pool2d", inputs=input, output=node, param_attr=attr)
 
         if not channel_first:
             attr = {"perm": [0, 2, 3, 1]}
-            node.fluid_code.add_layer("transpose",
-                                      inputs=node,
-                                      output=node,
-                                      param_attr=attr)
+            node.fluid_code.add_layer(
+                "transpose", inputs=node, output=node, param_attr=attr)
 
     def Conv2D(self, node):
         input = self.graph.get_node(node.layer.input[0], copy=True)
@@ -373,10 +349,8 @@ class TFOpMapperNHWC(OpMapper):
             strides = [strides[i] for i in [0, 3, 1, 2]]
             dilations = [dilations[i] for i in [0, 3, 1, 2]]
             attr = {"perm": [0, 3, 1, 2]}
-            node.fluid_code.add_layer("transpose",
-                                      inputs=input,
-                                      output=node,
-                                      param_attr=attr)
+            node.fluid_code.add_layer(
+                "transpose", inputs=input, output=node, param_attr=attr)
             input = node
 
         attr = {
@@ -393,25 +367,19 @@ class TFOpMapperNHWC(OpMapper):
             if len(node.dilation) == 1:
                 attr['dilation'] = [1, node.dilation[0]]
 
-        node.fluid_code.add_layer("conv2d",
-                                  inputs=input,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            "conv2d", inputs=input, output=node, param_attr=attr)
         if not channel_first:
             attr = {"perm": [0, 2, 3, 1]}
-            node.fluid_code.add_layer("transpose",
-                                      inputs=node,
-                                      output=node,
-                                      param_attr=attr)
+            node.fluid_code.add_layer(
+                "transpose", inputs=node, output=node, param_attr=attr)
 
     def BiasAdd(self, node):
         input = self.graph.get_node(node.layer.input[0], copy=True)
         bias = self.graph.get_node(node.layer.input[1], copy=True)
         inputs = {"x": input, "y": bias}
-        node.fluid_code.add_layer("elementwise_add",
-                                  inputs=inputs,
-                                  output=node,
-                                  param_attr=None)
+        node.fluid_code.add_layer(
+            "elementwise_add", inputs=inputs, output=node, param_attr=None)
 
     def FusedBatchNorm(self, node):
         input = self.graph.get_node(node.layer.input[0], copy=True)
@@ -433,10 +401,8 @@ class TFOpMapperNHWC(OpMapper):
 
         if not channel_first:
             attr = {"perm": [0, 3, 1, 2]}
-            node.fluid_code.add_layer("transpose",
-                                      inputs=input,
-                                      output=node,
-                                      param_attr=attr)
+            node.fluid_code.add_layer(
+                "transpose", inputs=input, output=node, param_attr=attr)
             input = node
 
         attr = {
@@ -448,17 +414,13 @@ class TFOpMapperNHWC(OpMapper):
             "is_test": True
         }
 
-        node.fluid_code.add_layer("batch_norm",
-                                  inputs=input,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            "batch_norm", inputs=input, output=node, param_attr=attr)
 
         if not channel_first:
             attr = {"perm": [0, 2, 3, 1]}
-            node.fluid_code.add_layer("transpose",
-                                      inputs=node,
-                                      output=node,
-                                      param_attr=attr)
+            node.fluid_code.add_layer(
+                "transpose", inputs=node, output=node, param_attr=attr)
 
     def DepthwiseConv2dNative(self, node):
         input = self.graph.get_node(node.layer.input[0], copy=True)
@@ -487,10 +449,8 @@ class TFOpMapperNHWC(OpMapper):
             strides = [strides[i] for i in [0, 3, 1, 2]]
             dilations = [dilations[i] for i in [0, 3, 1, 2]]
             attr = {"perm": [0, 3, 1, 2]}
-            node.fluid_code.add_layer("transpose",
-                                      inputs=input,
-                                      output=node,
-                                      param_attr=attr)
+            node.fluid_code.add_layer(
+                "transpose", inputs=input, output=node, param_attr=attr)
             input = node
 
         attr = {
@@ -504,17 +464,13 @@ class TFOpMapperNHWC(OpMapper):
             "use_cudnn": False,
             "padding": string(pad_mode)
         }
-        node.fluid_code.add_layer("conv2d",
-                                  inputs=input,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            "conv2d", inputs=input, output=node, param_attr=attr)
 
         if not channel_first:
             attr = {"perm": [0, 2, 3, 1]}
-            node.fluid_code.add_layer("transpose",
-                                      inputs=node,
-                                      output=node,
-                                      param_attr=attr)
+            node.fluid_code.add_layer(
+                "transpose", inputs=node, output=node, param_attr=attr)
 
     def Reshape(self, node):
         input = self.graph.get_node(node.layer.input[0], copy=True)
@@ -530,18 +486,17 @@ class TFOpMapperNHWC(OpMapper):
                 attr = {"shape": shape}
                 self.add_omit_nodes(param.layer_name, node.layer_name)
             else:
-                assert len(param.out_shapes[0]
-                           ) == 1, "Unexpected situation of shape parameter"
+                assert len(param.out_shapes[
+                    0]) == 1, "Unexpected situation of shape parameter"
                 attr = {"shape": [-1]}
-                node.fluid_code.add_layer("reshape",
-                                          inputs=param,
-                                          output="shape_param",
-                                          param_attr=attr)
+                node.fluid_code.add_layer(
+                    "reshape",
+                    inputs=param,
+                    output="shape_param",
+                    param_attr=attr)
                 attr = {"num_or_sections": param.out_shapes[0][0], "dim": 0}
-                node.fluid_code.add_layer("split",
-                                          inputs="shape_param",
-                                          output=node,
-                                          param_attr=attr)
+                node.fluid_code.add_layer(
+                    "split", inputs="shape_param", output=node, param_attr=attr)
                 new_param = "["
                 for i in range(param.out_shapes[0][0]):
                     new_param += (node.layer_name + "[{}]".format(i) + ", ")
@@ -565,10 +520,8 @@ class TFOpMapperNHWC(OpMapper):
                 attr["shape"][index] = int(total_size)
                 attr["shape"][0] = -1
 
-        node.fluid_code.add_layer("reshape",
-                                  inputs=input,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            "reshape", inputs=input, output=node, param_attr=attr)
 
     def AvgPool(self, node):
         input = self.graph.get_node(node.layer.input[0], copy=True)
@@ -588,10 +541,8 @@ class TFOpMapperNHWC(OpMapper):
             strides = [strides[i] for i in [0, 3, 1, 2]]
             k_size = [k_size[i] for i in [0, 3, 1, 2]]
             attr = {"perm": [0, 3, 1, 2]}
-            node.fluid_code.add_layer("transpose",
-                                      inputs=input,
-                                      output=node,
-                                      param_attr=attr)
+            node.fluid_code.add_layer(
+                "transpose", inputs=input, output=node, param_attr=attr)
             input = node
 
         attr = {
@@ -600,17 +551,13 @@ class TFOpMapperNHWC(OpMapper):
             "pool_stride": strides[2:4],
             "pool_padding": string(pad_mode)
         }
-        node.fluid_code.add_layer("pool2d",
-                                  inputs=input,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            "pool2d", inputs=input, output=node, param_attr=attr)
 
         if not channel_first:
             attr = {"perm": [0, 2, 3, 1]}
-            node.fluid_code.add_layer("transpose",
-                                      inputs=node,
-                                      output=node,
-                                      param_attr=attr)
+            node.fluid_code.add_layer(
+                "transpose", inputs=node, output=node, param_attr=attr)
 
     def SplitV(self, node):
         input = self.graph.get_node(node.layer.input[0], copy=True)
@@ -625,15 +572,13 @@ class TFOpMapperNHWC(OpMapper):
             "num_or_sections": num_sections.value.tolist(),
             "dim": dim.value
         }
-        node.fluid_code.add_layer("split",
-                                  inputs=input,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            "split", inputs=input, output=node, param_attr=attr)
 
     def ConcatV2(self, node):
         inputs = [
-            self.graph.get_node(name, copy=True)
-            for name in node.layer.input[:-1]
+            self.graph.get_node(
+                name, copy=True) for name in node.layer.input[:-1]
         ]
         axis = self.graph.get_node(node.layer.input[-1], copy=True)
         assert axis.layer_type == "Const"
@@ -643,10 +588,8 @@ class TFOpMapperNHWC(OpMapper):
             axis += len(inputs[0].out_shapes[0])
 
         attr = {"axis": axis}
-        node.fluid_code.add_layer("concat",
-                                  inputs=inputs,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            "concat", inputs=inputs, output=node, param_attr=attr)
 
     def Tile(self, node):
         input = self.graph.get_node(node.layer.input[0], copy=True)
@@ -660,21 +603,18 @@ class TFOpMapperNHWC(OpMapper):
             if expand_times[i] < 0:
                 expand_times[i] = 1
         attr = {"expand_times": expand_times}
-        node.fluid_code.add_layer("expand",
-                                  inputs=input,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            "expand", inputs=input, output=node, param_attr=attr)
 
     def Pack(self, node):
         inputs = [
-            self.graph.get_node(name, copy=True) for name in node.layer.input
+            self.graph.get_node(
+                name, copy=True) for name in node.layer.input
         ]
         axis = node.get_attr("axis")
         attr = {"axis": axis}
-        node.fluid_code.add_layer("stack",
-                                  inputs=inputs,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            "stack", inputs=inputs, output=node, param_attr=attr)
 
     def Pad(self, node):
         input = self.graph.get_node(node.layer.input[0], copy=True)
@@ -695,30 +635,22 @@ class TFOpMapperNHWC(OpMapper):
             if new_padding is not None:
                 if input.tf_data_format == "NHWC":
                     attr = {"perm": [0, 3, 1, 2]}
-                    node.fluid_code.add_layer("transpose",
-                                              inputs=input,
-                                              output=node,
-                                              param_attr=attr)
+                    node.fluid_code.add_layer(
+                        "transpose", inputs=input, output=node, param_attr=attr)
                     input = node
                 attr = {"paddings": new_padding}
-                node.fluid_code.add_layer("pad2d",
-                                          inputs=input,
-                                          output=node,
-                                          param_attr=attr)
+                node.fluid_code.add_layer(
+                    "pad2d", inputs=input, output=node, param_attr=attr)
                 if input.tf_data_format == "NHWC":
                     attr = {"perm": [0, 2, 3, 1]}
-                    node.fluid_code.add_layer("transpose",
-                                              inputs=node,
-                                              output=node,
-                                              param_attr=attr)
+                    node.fluid_code.add_layer(
+                        "transpose", inputs=node, output=node, param_attr=attr)
 
                 return
 
         attr = {"paddings": paddings}
-        node.fluid_code.add_layer("pad",
-                                  inputs=input,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            "pad", inputs=input, output=node, param_attr=attr)
 
     def Range(self, node):
         start = self.graph.get_node(node.layer.input[0], copy=True)
@@ -744,13 +676,10 @@ class TFOpMapperNHWC(OpMapper):
             "start": start,
             "end": limit,
             "step": delta,
-            "dtype": string(dtype)
         }
         attr = {"dtype": string(node.dtype)}
-        node.fluid_code.add_layer("range",
-                                  inputs=inputs,
-                                  output=node,
-                                  param_attr=None)
+        node.fluid_code.add_layer(
+            "range", inputs=inputs, output=node, param_attr=attr)
 
     def Mean(self, node):
         input = self.graph.get_node(node.layer.input[0], copy=True)
@@ -760,10 +689,8 @@ class TFOpMapperNHWC(OpMapper):
         keep_dims = node.get_attr("keep_dims")
 
         attr = {"dim": dims, "keep_dim": keep_dims}
-        node.fluid_code.add_layer("reduce_mean",
-                                  inputs=input,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            "reduce_mean", inputs=input, output=node, param_attr=attr)
 
     def MatMul(self, node):
         x = self.graph.get_node(node.layer.input[0], copy=True)
@@ -777,15 +704,11 @@ class TFOpMapperNHWC(OpMapper):
             shape = x.out_shapes[0]
             shape[-1] = y.out_shapes[0][0]
             attr = {"shape": shape}
-            node.fluid_code.add_layer("reshape",
-                                      inputs=x,
-                                      output=x,
-                                      param_attr=attr)
+            node.fluid_code.add_layer(
+                "reshape", inputs=x, output=x, param_attr=attr)
         attr = {"transpose_x": transpose_a, "transpose_y": transpose_b}
-        node.fluid_code.add_layer("matmul",
-                                  inputs=inputs,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            "matmul", inputs=inputs, output=node, param_attr=attr)
 
     def ArgMax(self, node):
         input = self.graph.get_node(node.layer.input[0], copy=True)
@@ -794,10 +717,8 @@ class TFOpMapperNHWC(OpMapper):
         self.add_omit_nodes(axis.layer_name, node.layer_name)
         axis = axis.value
         attr = {"axis": axis}
-        node.fluid_code.add_layer("argmax",
-                                  inputs=input,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            "argmax", inputs=input, output=node, param_attr=attr)
 
     def StridedSlice(self, node):
         input = self.graph.get_node(node.layer.input[0], copy=True)
@@ -863,25 +784,19 @@ class TFOpMapperNHWC(OpMapper):
             "starts": new_begin,
             "ends": new_end
         }
-        node.fluid_code.add_layer("slice",
-                                  inputs=input,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            "slice", inputs=input, output=node, param_attr=attr)
         if len(new_axes) > 0:
             attr = {"axes": new_axes}
-            node.fluid_code.add_layer("unsqueeze",
-                                      inputs=node,
-                                      output=node,
-                                      param_attr=attr)
+            node.fluid_code.add_layer(
+                "unsqueeze", inputs=node, output=node, param_attr=attr)
         if len(shrink_axes) > 0:
             if len(input.out_shapes[0]) + len(new_axes) <= 1:
                 pass
             else:
                 attr = {"axes": shrink_axes}
-                node.fluid_code.add_layer("squeeze",
-                                          inputs=node,
-                                          output=node,
-                                          param_attr=attr)
+                node.fluid_code.add_layer(
+                    "squeeze", inputs=node, output=node, param_attr=attr)
 
     def Slice(self, node):
         input = self.graph.get_node(node.layer.input[0], copy=True)
@@ -910,10 +825,8 @@ class TFOpMapperNHWC(OpMapper):
             "ends": size
         }
 
-        node.fluid_code.add_layer("slice",
-                                  inputs=input,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            "slice", inputs=input, output=node, param_attr=attr)
 
     def Conv2DBackpropInput(self, node):
         out_shape = self.graph.get_node(node.layer.input[0], copy=True)
@@ -951,10 +864,8 @@ class TFOpMapperNHWC(OpMapper):
             strides = [strides[i] for i in [0, 3, 1, 2]]
             dilations = [dilations[i] for i in [0, 3, 1, 2]]
             attr = {"perm": [0, 3, 1, 2]}
-            node.fluid_code.add_layer("transpose",
-                                      inputs=input,
-                                      output=node,
-                                      param_attr=attr)
+            node.fluid_code.add_layer(
+                "transpose", inputs=input, output=node, param_attr=attr)
             input = node
         else:
             self.data_format_propagation(node)
@@ -969,17 +880,13 @@ class TFOpMapperNHWC(OpMapper):
             "padding": string(pad_mode),
             "output_size": out_shape[1:3]
         }
-        node.fluid_code.add_layer("conv2d_transpose",
-                                  inputs=input,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            "conv2d_transpose", inputs=input, output=node, param_attr=attr)
 
         if not channel_first:
             attr = {"perm": [0, 2, 3, 1]}
-            node.fluid_code.add_layer("transpose",
-                                      inputs=node,
-                                      output=node,
-                                      param_attr=attr)
+            node.fluid_code.add_layer(
+                "transpose", inputs=node, output=node, param_attr=attr)
 
     def Max(self, node):
         input = self.graph.get_node(node.layer.input[0], copy=True)
@@ -989,10 +896,8 @@ class TFOpMapperNHWC(OpMapper):
         dim = reduce_idx.value.tolist()
 
         attr = {"dim": dim, "keep_dim": keep_dims}
-        node.fluid_code.add_layer("reduce_max",
-                                  inputs=input,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            "reduce_max", inputs=input, output=node, param_attr=attr)
 
     def Sum(self, node):
         input = self.graph.get_node(node.layer.input[0], copy=True)
@@ -1002,19 +907,15 @@ class TFOpMapperNHWC(OpMapper):
         dim = reduce_idx.value.tolist()
 
         attr = {"dim": dim, "keep_dim": keep_dims}
-        node.fluid_code.add_layer("reduce_sum",
-                                  inputs=input,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            "reduce_sum", inputs=input, output=node, param_attr=attr)
 
     def Cast(self, node):
         input = self.graph.get_node(node.layer.input[0], copy=True)
         dtype = node.dtype_map[node.get_attr('DstT')]
         attr = {"dtype": string(dtype)}
-        node.fluid_code.add_layer("cast",
-                                  inputs=input,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            "cast", inputs=input, output=node, param_attr=attr)
 
     def Split(self, node):
         dim = self.graph.get_node(node.layer.input[0], copy=True)
@@ -1025,28 +926,22 @@ class TFOpMapperNHWC(OpMapper):
         dim = dim.value
 
         attr = {"num_or_sections": num_split, "dim": dim}
-        node.fluid_code.add_layer("split",
-                                  inputs=input,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            "split", inputs=input, output=node, param_attr=attr)
 
     def Squeeze(self, node):
         input = self.graph.get_node(node.layer.input[0], copy=True)
         squeeze_dims = node.get_attr('squeeze_dims')
         attr = {"axes": squeeze_dims}
-        node.fluid_code.add_layer("squeeze",
-                                  inputs=input,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            "squeeze", inputs=input, output=node, param_attr=attr)
 
     def Softmax(self, node):
         input = self.graph.get_node(node.layer.input[0], copy=True)
         axis = node.get_attr("axis")
         attr = {"axis": axis}
-        node.fluid_code.add_layer("softmax",
-                                  inputs=input,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            "softmax", inputs=input, output=node, param_attr=attr)
 
     def ResizeNearestNeighbor(self, node):
         input = self.graph.get_node(node.layer.input[0], copy=True)
@@ -1055,24 +950,18 @@ class TFOpMapperNHWC(OpMapper):
         if resize_shape.layer_type == "Const":
             resize_shape = resize_shape.value.tolist()
         else:
-            resize_shape = self.decoder.infer_shape_tensor(
-                resize_shape, node.out_shapes[0])
+            resize_shape = self.decoder.infer_shape_tensor(resize_shape,
+                                                           node.out_shapes[0])
         align_corners = node.get_attr("align_corners")
         attr = {"perm": [0, 3, 1, 2]}
-        node.fluid_code.add_layer("transpose",
-                                  inputs=input,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            "transpose", inputs=input, output=node, param_attr=attr)
         attr = {"align_corners": align_corners, "out_shape": resize_shape}
-        node.fluid_code.add_layer("resize_nearest",
-                                  inputs=node,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            "resize_nearest", inputs=node, output=node, param_attr=attr)
         attr = {"perm": [0, 2, 3, 1]}
-        node.fluid_code.add_layer("transpose",
-                                  inputs=node,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            "transpose", inputs=node, output=node, param_attr=attr)
 
     def ResizeBilinear(self, node):
         input = self.graph.get_node(node.layer.input[0], copy=True)
@@ -1081,37 +970,29 @@ class TFOpMapperNHWC(OpMapper):
         if resize_shape.layer_type == "Const":
             resize_shape = resize_shape.value.tolist()
         else:
-            resize_shape = self.decoder.infer_shape_tensor(
-                resize_shape, node.out_shapes[0])
+            resize_shape = self.decoder.infer_shape_tensor(resize_shape,
+                                                           node.out_shapes[0])
         align_corners = node.get_attr("align_corners")
         attr = {"perm": [0, 3, 1, 2]}
-        node.fluid_code.add_layer("transpose",
-                                  inputs=input,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            "transpose", inputs=input, output=node, param_attr=attr)
         attr = {
             "align_corners": align_corners,
             "out_shape": resize_shape,
             "align_mode": 1
         }
-        node.fluid_code.add_layer("resize_bilinear",
-                                  inputs=node,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            "resize_bilinear", inputs=node, output=node, param_attr=attr)
         attr = {"perm": [0, 2, 3, 1]}
-        node.fluid_code.add_layer("transpose",
-                                  inputs=node,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            "transpose", inputs=node, output=node, param_attr=attr)
 
     def GreaterEqual(self, node):
         x = self.graph.get_node(node.layer.input[0], copy=True)
         y = self.graph.get_node(node.layer.input[1], copy=True)
         inputs = {"x": x, "y": y}
-        node.fluid_code.add_layer("greater_equal",
-                                  inputs=inputs,
-                                  output=node,
-                                  param_attr=None)
+        node.fluid_code.add_layer(
+            "greater_equal", inputs=inputs, output=node, param_attr=None)
 
     def RandomUniform(self, node):
         shape = self.graph.get_node(node.layer.input[0], copy=True)
@@ -1124,29 +1005,24 @@ class TFOpMapperNHWC(OpMapper):
 
         if shape[0] < 0:
             input = self.batch_node
-            node.fluid_code.add_layer("uniform_random_batch_size_like",
-                                      inputs=input,
-                                      output=node,
-                                      param_attr=attr)
+            node.fluid_code.add_layer(
+                "uniform_random_batch_size_like",
+                inputs=input,
+                output=node,
+                param_attr=attr)
         else:
-            node.fluid_code.add_layer("uniform_random",
-                                      inputs=None,
-                                      output=node,
-                                      param_attr=attr)
+            node.fluid_code.add_layer(
+                "uniform_random", inputs=None, output=node, param_attr=attr)
 
     def SquaredDifference(self, node):
         x = self.graph.get_node(node.layer.input[0], copy=True)
         y = self.graph.get_node(node.layer.input[1], copy=True)
         inputs = {"x": x, "y": y}
-        node.fluid_code.add_layer("elementwise_sub",
-                                  inputs=inputs,
-                                  output=node,
-                                  param_attr=None)
+        node.fluid_code.add_layer(
+            "elementwise_sub", inputs=inputs, output=node, param_attr=None)
         inputs = {"x": node, "y": node}
-        node.fluid_code.add_layer("elementwise_mul",
-                                  inputs=inputs,
-                                  output=node,
-                                  param_attr=None)
+        node.fluid_code.add_layer(
+            "elementwise_mul", inputs=inputs, output=node, param_attr=None)
 
     def ExpandDims(self, node):
         x = self.graph.get_node(node.layer.input[0], copy=True)
@@ -1157,19 +1033,15 @@ class TFOpMapperNHWC(OpMapper):
             dim = self.decoder.infer_tensor(y)
         self.add_omit_nodes(y.layer_name, node.layer_name)
         attr = {'axes': [dim]}
-        node.fluid_code.add_layer("unsqueeze",
-                                  inputs=x,
-                                  output=node,
-                                  param_attr=attr)
+        node.fluid_code.add_layer(
+            "unsqueeze", inputs=x, output=node, param_attr=attr)
 
     def BatchToSpaceND(self, node):
         x = self.graph.get_node(node.layer.input[0], copy=True)
         y = self.graph.get_node(node.layer.input[1], copy=True)
         if hasattr(node, 'skip') and node.skip:
-            node.fluid_code.add_layer("=",
-                                      inputs=x,
-                                      output=node,
-                                      param_attr=None)
+            node.fluid_code.add_layer(
+                "=", inputs=x, output=node, param_attr=None)
         else:
             raise Exception("BatchToSpaceND is not supported")
 
@@ -1177,9 +1049,7 @@ class TFOpMapperNHWC(OpMapper):
         x = self.graph.get_node(node.layer.input[0], copy=True)
         y = self.graph.get_node(node.layer.input[1], copy=True)
         if hasattr(node, 'skip') and node.skip:
-            node.fluid_code.add_layer("=",
-                                      inputs=x,
-                                      output=node,
-                                      param_attr=None)
+            node.fluid_code.add_layer(
+                "=", inputs=x, output=node, param_attr=None)
         else:
             raise Exception("SpaceToBatchND is not supported")

x2paddle/optimizer/caffe_optimizer.py

@@ -41,10 +41,11 @@ class CaffeOptimizer(object):
                     if is_delete_node:
                         parent_node.fluid_code.clear()
                     node.fluid_code.clear()
-                    node.fluid_code.add_layer("batch_norm",
-                                              inputs=input,
-                                              output=node,
-                                              param_attr=parent_param_attr)
+                    node.fluid_code.add_layer(
+                        "batch_norm",
+                        inputs=input,
+                        output=node,
+                        param_attr=parent_param_attr)
 
     def merge_op_activation(self):
         for node_name in self.graph.topo_sort:
@@ -62,7 +63,8 @@ class CaffeOptimizer(object):
                     if is_delete_node:
                         parent_node.fluid_code.clear()
                     node.fluid_code.clear()
-                    node.fluid_code.add_layer(op,
-                                              inputs=input,
-                                              output=node,
-                                              param_attr=parent_param_attr)
+                    node.fluid_code.add_layer(
+                        op,
+                        inputs=input,
+                        output=node,
+                        param_attr=parent_param_attr)

x2paddle/optimizer/onnx_optimizer.py

@@ -13,7 +13,6 @@
 # limitations under the License.
 
 # TODO useless node remove
-from x2paddle.op_mapper.onnx_op_mapper import ONNXOpMapper
 
 
 class ONNXOptimizer(object):

x2paddle/optimizer/tf_optimizer.py

@@ -554,10 +554,11 @@ class TFOptimizer(object):
                 node.fluid_code.layers[0].param_attr["shape"] = shape
                 node.fluid_code.layers[0].output = "nhwc_" + name
                 attr = {"perm": [0, 2, 3, 1]}
-                node.fluid_code.add_layer("transpose",
-                                          inputs="nhwc_" + name,
-                                          output=node,
-                                          param_attr=attr)
+                node.fluid_code.add_layer(
+                    "transpose",
+                    inputs="nhwc_" + name,
+                    output=node,
+                    param_attr=attr)
                 self.graph.input_nodes[i] = "nhwc_" + name
         for i, name in enumerate(self.graph.output_nodes):
             node = self.graph.get_node(name)
@@ -767,8 +768,8 @@ class TFOptimizer(object):
                     is_prelu = False
                     continue
 
-                if len(in_nodes0[0].outputs) != 1 or len(
-                        in_nodes0[1].outputs) != 1:
+                if len(in_nodes0[0].outputs) != 1 or len(in_nodes0[1]
+                                                         .outputs) != 1:
                     is_prelu = False
                     continue
 
@@ -777,8 +778,8 @@ class TFOptimizer(object):
                     self.graph.get_node(in_name)
                     for in_name in in_nodes0[1].inputs
                 ]
-                if in_nodes2[1].layer_type != "Const" or numpy.fabs(
-                        in_nodes2[1].value - 0.5) > 1e-06:
+                if in_nodes2[1].layer_type != "Const" or numpy.fabs(in_nodes2[
+                        1].value - 0.5) > 1e-06:
                     is_prelu = False
                     continue
                 if in_nodes2[0].layer_type != "Mul":
@@ -787,8 +788,8 @@ class TFOptimizer(object):
                 if exist_act(in_nodes2[0]):
                     is_prelu = False
                     continue
-                if len(in_nodes2[1].outputs) != 1 or len(
-                        in_nodes2[0].outputs) != 1:
+                if len(in_nodes2[1].outputs) != 1 or len(in_nodes2[0]
+                                                         .outputs) != 1:
                     is_prelu = False
                     continue
 
@@ -803,8 +804,8 @@ class TFOptimizer(object):
                 if exist_act(in_nodes3[1]):
                     is_prelu = False
                     continue
-                if len(in_nodes3[0].outputs) != 1 or len(
-                        in_nodes3[1].outputs) != 1:
+                if len(in_nodes3[0].outputs) != 1 or len(in_nodes3[1]
+                                                         .outputs) != 1:
                     is_prelu = False
                     continue
 
@@ -856,12 +857,12 @@ class TFOptimizer(object):
                     mode = "element"
                 elif len(in_nodes3[0].value.shape) == 0:
                     mode = "all"
-                elif len(in_nodes3[0].value.shape
-                         ) == 1 and in_nodes3[0].value.shape[0] == 1:
+                elif len(in_nodes3[0].value.shape) == 1 and in_nodes3[
+                        0].value.shape[0] == 1:
                     mode = "all"
-                elif len(in_shape) == 4 and len(
-                        in_nodes3[0].value.shape
-                ) == 1 and in_nodes3[0].value.shape[0] == in_shape[-1]:
+                elif len(in_shape) == 4 and len(in_nodes3[
+                        0].value.shape) == 1 and in_nodes3[0].value.shape[
+                            0] == in_shape[-1]:
                     mode = "channel"
                     weight = self.op_mapper.weights[in_nodes3[0].layer_name]
                     weight = numpy.expand_dims(weight, 0)
@@ -916,14 +917,15 @@ class TFOptimizer(object):
                     self.graph.get_node(in_name) for in_name in node.inputs
                 ]
                 if in_nodes0[0].layer_type != "Mul" or in_nodes0[
-                        1].layer_type != "Const" or in_nodes0[1].value.size != 1:
+                        1].layer_type != "Const" or in_nodes0[
+                            1].value.size != 1:
                     is_scale = False
                     continue
                 if exist_act(in_nodes0[0]):
                     is_scale = False
                     continue
-                if len(in_nodes0[0].outputs) != 1 or len(
-                        in_nodes0[1].outputs) != 1:
+                if len(in_nodes0[0].outputs) != 1 or len(in_nodes0[1]
+                                                         .outputs) != 1:
                     is_scale = False
                     continue
 
@@ -939,8 +941,8 @@ class TFOptimizer(object):
                 if exist_act(in_nodes1[1]):
                     is_scale = False
                     continue
-                if len(in_nodes1[0].outputs) != 1 or len(
-                        in_nodes1[1].outputs) != 1:
+                if len(in_nodes1[0].outputs) != 1 or len(in_nodes1[1]
+                                                         .outputs) != 1:
                     is_scale = False
                     continue
 
@@ -962,8 +964,8 @@ class TFOptimizer(object):
                     scale = 1.0 / in_nodes2[1].value * in_nodes1[0].value
                     act = None
                     if node.fluid_code.layers[0].param_attr is not None:
-                        act = node.fluid_code.layers[0].param_attr.get(
-                            "act", None)
+                        act = node.fluid_code.layers[0].param_attr.get("act",
+                                                                       None)
                     node.fluid_code.clear()
 
                     attr = {
@@ -972,10 +974,8 @@ class TFOptimizer(object):
                         "bias_after_scale": True,
                         "act": act
                     }
-                    node.fluid_code.add_layer("scale",
-                                              inputs=in_node,
-                                              output=node,
-                                              param_attr=attr)
+                    node.fluid_code.add_layer(
+                        "scale", inputs=in_node, output=node, param_attr=attr)
 
                     del self.graph.node_map[in_nodes0[0].layer_name]
                     del self.graph.node_map[in_nodes0[1].layer_name]
@@ -1004,17 +1004,17 @@ class TFOptimizer(object):
                 if exist_act(in_nodes0[0]):
                     is_affine_channel = False
                     continue
-                if len(in_nodes0[0].outputs) != 1 or len(
-                        in_nodes0[1].outputs) != 1:
+                if len(in_nodes0[0].outputs) != 1 or len(in_nodes0[1]
+                                                         .outputs) != 1:
                     is_affine_channel = False
                     continue
                 in_nodes1 = [
                     self.graph.get_node(in_name)
                     for in_name in in_nodes0[0].inputs
                 ]
-                if len(in_nodes1[0].out_shapes[0]
-                       ) != 4 or in_nodes1[1].layer_type != "Const" or len(
-                           in_nodes1[1].value.shape) != 3:
+                if len(in_nodes1[0].out_shapes[0]) != 4 or in_nodes1[
+                        1].layer_type != "Const" or len(in_nodes1[1]
+                                                        .value.shape) != 3:
                     is_affine_channel = False
                     continue
                 if len(in_nodes1[1].outputs) != 1:
@@ -1037,8 +1037,8 @@ class TFOptimizer(object):
                     node.layer_type = "AffineChannel"
                     node.inputs = [in_node.layer_name]
                     scale = 1.0 / in_nodes0[1].value.flatten()
-                    bias = in_nodes1[1].value.flatten(
-                    ) / in_nodes0[1].value.flatten()
+                    bias = in_nodes1[1].value.flatten() / in_nodes0[
+                        1].value.flatten()
                     if not bias_add:
                         bias *= -1.0
                     self.op_mapper.weights[node.layer_name + "_scale"] = scale
@@ -1046,8 +1046,8 @@ class TFOptimizer(object):
 
                     act = None
                     if node.fluid_code.layers[0].param_attr is not None:
-                        act = node.fluid_code.layers[0].param_attr.get(
-                            "act", None)
+                        act = node.fluid_code.layers[0].param_attr.get("act",
+                                                                       None)
                     node.fluid_code.clear()
 
                     attr = {
@@ -1055,29 +1055,32 @@ class TFOptimizer(object):
                         "shape": [channel],
                         "name": string(node.layer_name + "_scale")
                     }
-                    node.fluid_code.add_layer("create_parameter",
-                                              inputs=None,
-                                              output=node.layer_name + "_scale",
-                                              param_attr=attr)
+                    node.fluid_code.add_layer(
+                        "create_parameter",
+                        inputs=None,
+                        output=node.layer_name + "_scale",
+                        param_attr=attr)
                     attr = {
                         "dtype": string(scale.dtype),
                         "shape": [channel],
                         "name": string(node.layer_name + "_bias")
                     }
-                    node.fluid_code.add_layer("create_parameter",
-                                              inputs=None,
-                                              output=node.layer_name + "_bias",
-                                              param_attr=attr)
+                    node.fluid_code.add_layer(
+                        "create_parameter",
+                        inputs=None,
+                        output=node.layer_name + "_bias",
+                        param_attr=attr)
                     inputs = {
                         "x": in_node,
                         "scale": node.layer_name + "_scale",
                         "bias": node.layer_name + "_bias"
                     }
                     attr = {"act": act}
-                    node.fluid_code.add_layer("affine_channel",
-                                              inputs=inputs,
-                                              output=node,
-                                              param_attr=attr)
+                    node.fluid_code.add_layer(
+                        "affine_channel",
+                        inputs=inputs,
+                        output=node,
+                        param_attr=attr)
 
                     del self.graph.node_map[in_nodes0[0].layer_name]
                     del self.graph.node_map[in_nodes0[1].layer_name]

x2paddle_model_zoo.md

@@ -13,6 +13,7 @@
 | ShuffleNet | [code](https://github.com/TropComplique/shufflenet-v2-tensorflow) |-|
 | mNASNet | [code](https://github.com/tensorflow/tpu/tree/master/models/official/mnasnet) |-|
 | EfficientNet | [code](https://github.com/tensorflow/tpu/tree/master/models/official/efficientnet) |-|
+| Inception_V3 | [code](https://github.com/tensorflow/models/blob/master/research/slim/nets/inception_v3.py) |-|
 | Inception_V4 | [code](https://github.com/tensorflow/models/blob/master/research/slim/nets/inception_v4.py) |-|
 | Inception_ResNet_V2 | [code](https://github.com/tensorflow/models/blob/master/research/slim/nets/inception_resnet_v2.py) |-|
 | VGG16 | [code](https://github.com/tensorflow/models/tree/master/research/slim/nets) |-|
@@ -47,8 +48,8 @@
 ## ONNX
 **注：** 部分模型来源于PyTorch，PyTorch的转换可参考[pytorch_to_onnx.md](pytorch_to_onnx.md)
 
-| 模型 | 来源 | operator version|
-|-------|--------|---------|
+| 模型 | 来源 | operator version|备注|
+|-------|--------|---------|---------|
 | ResNet18 | [torchvison.model.resnet18](https://github.com/pytorch/vision/blob/master/torchvision/models/resnet.py) |9|
 | ResNet34 | [torchvison.model.resnet34](https://github.com/pytorch/vision/blob/master/torchvision/models/resnet.py) |9|
 | ResNet50 | [torchvison.model.resnet50](https://github.com/pytorch/vision/blob/master/torchvision/models/resnet.py) |9|
@@ -64,4 +65,6 @@
 | mNASNet | [pytorch(personal practice)](https://github.com/rwightman/gen-efficientnet-pytorch) |9|
 | EfficientNet | [pytorch(personal practice)](https://github.com/rwightman/gen-efficientnet-pytorch) |9|
 | SqueezeNet | [onnx official](https://s3.amazonaws.com/download.onnx/models/opset_9/squeezenet.tar.gz) |9|
-|Ultra-Light-Fast-Generic-Face-Detector-1MB| [onnx_model](https://github.com/Linzaer/Ultra-Light-Fast-Generic-Face-Detector-1MB/tree/master/models/onnx)| |
+|Ultra-Light-Fast-Generic-Face-Detector-1MB| [onnx_model](https://github.com/Linzaer/Ultra-Light-Fast-Generic-Face-Detector-1MB/tree/master/models/onnx)|9 |
+|BERT| [pytorch(huggingface)](https://github.com/huggingface/transformers/blob/master/notebooks/04-onnx-export.ipynb)|11|转换时需指定input shape，见[文档Q3](FAQ.md)|
+|GPT2| [pytorch(huggingface)](https://github.com/huggingface/transformers/blob/master/notebooks/04-onnx-export.ipynb)|11|转换时需指定input shape，见[文档Q3](FAQ.md)|