Merge pull request #4 from PaddlePaddle/develop_jason

Develop jason

.github/ISSUE_TEMPLATE/caffe2paddle.md

+---
+name: caffe2paddle
+about: Caffe模型转换至PaddlePaddle，请说明Caffe模型的来源，模型类型（例如图像分类、目标检测等）
+---
+
+Caffe模型转换至PaddlePaddle，请说明Caffe模型的来源，模型类型（例如图像分类、目标检测等）  
+
+如有原模型文件或github链接，如方便可一并附上，方便开发人员分析。

.github/ISSUE_TEMPLATE/onnx2paddle.md

+---
+name: onnx2paddle
+about: ONNX模型转换至PaddlePaddle，请说明ONNX模型的来源，模型类型（例如图像分类、目标检测等)
+---
+ONNX模型转换至PaddlePaddle，请说明ONNX模型的来源，模型类型（例如图像分类、目标检测等）  
+
+如有原模型文件或github链接，如方便可一并附上，方便开发人员分析。

.github/ISSUE_TEMPLATE/other.md

+---
+name: 其它类型
+about: 例如Bug类，需求建议类
+---

.github/ISSUE_TEMPLATE/paddle2onnx.md

+---
+name: paddle2onnx
+about: Paddle模型转换至ONNX，请说明Paddle模型的来源，模型类型（例如图像分类、目标检测等）
+---
+Paddle模型转换至ONNX，请说明Paddle模型的来源，模型类型（例如图像分类、目标检测等）  
+
+如有原模型文件或github链接，如方便可一并附上，方便开发人员分析，同时建议说明模型转换的使用场景:)

.github/ISSUE_TEMPLATE/tensorflow2paddle.md

+---
+name: tensorflow2paddle
+about: TensorFlow模型转换至PaddlePaddle，请说明TensorFlow模型的来源，模型类型（例如图像分类、目标检测等）
+---
+
+TensorFlow模型转换至PaddlePaddle，请说明TensorFlow模型的来源，模型类型（例如图像分类、目标检测等）  
+
+如有原模型文件或github链接，可以一并附上，方便开发人员分析。

FAQ.md

@@ -11,3 +11,6 @@ x2paddle -f tensorflow -m tf.pb -s pd-model --without_data_format_optimization -
 ```
 
 > 1. 目前Tensorflow的CV模型大部分均为`NHWC`的输入格式，而Paddle的默认输入格式为`NCHW`，因此X2Paddle在转换过程中，会对如`axis`， `shape`等参数进行转换，适应Paddle的NCHW格式。但在这种情况下，可能会由于TensorFlow模型太复杂，导致出错。  指定`--without_data_format_optimization`后，会停止对`axis`，`shape`等参数的优化（这可能会带来一定数量的transpose操作）
+
+**Q3. ONNX模型转换过程中，提示『Unknown shape for input tensor[tensor name: "input"] -> shape: ['batch', 'sequence']， Please define shape of input here』**  
+A：该提示信息表示从ONNX的模型中获取到输入tensor(tensor名为"input:)的shape是语义象征性的['batch', 'sequence']，而不是dim为int类型的shape，从而可能会因为部分node的shape无法推理，导致转换失败。所以用户可以尝试手动在提示后输入详细的shape信息，如:-1,3,224,224  其中-1表示Batch

README.md

@@ -10,12 +10,12 @@ X2Paddle在多个主流的CV模型上，测试过TensorFlow/Caffe/ONNX模型的
 ## 环境依赖
 
 python == 2.7 | python >= 3.5  
-paddlepaddle >= 1.6.0  
+paddlepaddle >= 1.8.0  
 
 **按需安装以下依赖**  
 tensorflow ： tensorflow == 1.14.0  
 caffe ： 无  
-onnx ： onnx == 1.6.0  onnxruntime == 1.0.0
+onnx ： onnx == 1.6.0
 
 ## 安装
 ### 安装方式一（推荐）
@@ -61,6 +61,8 @@ x2paddle --framework=paddle2onnx --model=paddle_infer_model_dir --save_dir=onnx_
 |--without_data_format_optimization | **[可选]** For TensorFlow, 当指定该参数时，关闭NHWC->NCHW的优化，见[文档Q2](FAQ.md) |
 |--define_input_shape | **[可选]** For TensorFlow, 当指定该参数时，强制用户输入每个Placeholder的shape，见[文档Q2](FAQ.md) |
 |--params_merge | **[可选]** 当指定该参数时，转换完成后，inference_model中的所有模型参数将合并保存为一个文件__params__ |
+|--onnx_opset | **[可选]** 当framework为paddle2onnx时，该参数可设置转换为ONNX的OpSet版本，目前支持9、10、11，默认为10 |
+
 
 
 ## 使用转换后的模型

op_list.md

 # X2Paddle支持OP列表
-> 目前X2Paddle支持40+的TensorFlow OP，30+的Caffe Layer，覆盖了大部分CV分类模型常用的操作。我们在如下列表中给出了目前X2Paddle支持的全部OP。
+> 目前X2Paddle支持50+的TensorFlow OP，30+的Caffe Layer，覆盖了大部分CV分类模型常用的操作。我们在如下列表中给出了目前X2Paddle支持的全部OP。
 
 **注：** 目前，部分OP暂未支持，如您在转换过程中出现OP不支持的情况，可自行添加或反馈给我们。欢迎通过[ISSUE反馈](https://github.com/PaddlePaddle/X2Paddle/issues/new)的方式告知我们(模型名，代码实现或模型获取方式)，我们会及时跟进：）
 
@@ -20,7 +20,7 @@
 | 41 | Cast        | 42 | Split     | 43 | Squeeze   | 44 | ResizeNearestNeighbor |
 | 45 | Softmax     | 46 | Range     | 47 | ConcatV2  | 48 | MirrorPad             |
 | 49 | Identity    | 50 | GreaterEqual  | 51 | StopGradient | 52 | Minimum |
-| 53 | RadnomUniform | | | | | | |
+| 53 | RadnomUniform | 54 | Fill | 55 | Floor | 56 | DepthToSpace |
 
 ## Caffe
 

x2paddle/__init__.py

 __version__ = "0.7.4"
+
+from .core.program import PaddleProgram
+
+program = PaddleProgram()
+
+name_counter = dict()
+
+
+def gen_name(op_name, var_name):
+    name = "{}_{}".format(op_name, var_name)
+    if name not in name_counter:
+        name_counter[name] = 0
+    else:
+        name_counter[name] += 1
+    name = name + "_" + str(name_counter[name])
+    return name

x2paddle/convert.py

@@ -13,6 +13,7 @@
 # limitations under the License.
 
 from six import text_type as _text_type
+from x2paddle import program
 import argparse
 import sys
 
@@ -75,6 +76,12 @@ def arg_parser():
         action="store_true",
         default=False,
         help="define input shape for tf model")
+    parser.add_argument(
+        "--onnx_opset",
+        "-oo",
+        type=int,
+        default=10,
+        help="when paddle2onnx set onnx opset version to export")
     parser.add_argument(
         "--params_merge",
         "-pm",
@@ -114,29 +121,10 @@ def tf2paddle(model_path,
 
     print("Now translating model from tensorflow to paddle.")
     model = TFDecoder(model_path, define_input_shape=define_input_shape)
-    if not without_data_format_optimization:
-        mapper = TFOpMapper(model)
-        optimizer = TFOptimizer(mapper)
-        # neccesary optimization
-        optimizer.delete_redundance_code()
-        # optimizer below is experimental
-        optimizer.optimize_elementwise_op()
-        optimizer.merge_activation()
-        optimizer.merge_bias()
-        optimizer.optimize_sub_graph()
 
-#        optimizer.merge_batch_norm()
-#        optimizer.merge_prelu()
-    else:
-        mapper = TFOpMapperNHWC(model)
-        optimizer = TFOptimizer(mapper)
-        optimizer.delete_redundance_code()
-        optimizer.strip_graph()
-        optimizer.merge_activation()
-        optimizer.merge_bias()
-        optimizer.make_nchw_input_output()
-        optimizer.remove_transpose()
-    mapper.save_inference_model(save_dir, params_merge)
+    mapper = TFOpMapperNHWC(model)
+    program.build()
+    program.gen_model(save_dir)
 
 
 def caffe2paddle(proto, weight, save_dir, caffe_proto, params_merge=False):
@@ -172,24 +160,26 @@ def onnx2paddle(model_path, save_dir, params_merge=False):
         return
     print("Now translating model from onnx to paddle.")
 
-    from x2paddle.op_mapper.onnx_op_mapper import ONNXOpMapper
+    from x2paddle.op_mapper.onnx2paddle.onnx_op_mapper import ONNXOpMapper
     from x2paddle.decoder.onnx_decoder import ONNXDecoder
     from x2paddle.optimizer.onnx_optimizer import ONNXOptimizer
-    import onnxruntime
     model = ONNXDecoder(model_path)
-    mapper = ONNXOpMapper(model, save_dir)
+    mapper = ONNXOpMapper(model)
+    print("Model optimizing ...")
     optimizer = ONNXOptimizer(mapper)
+    print("Model optimized.")
 
-    optimizer.delete_redundance_code()
+    print("Paddle model and code generating ...")
     mapper.save_inference_model(save_dir, params_merge)
+    print("Paddle model and code generated.")
 
 
-def paddle2onnx(model_path, save_dir):
+def paddle2onnx(model_path, save_dir, opset_version=10):
     from x2paddle.decoder.paddle_decoder import PaddleDecoder
-    from x2paddle.op_mapper.paddle_op_mapper import PaddleOpMapper
+    from x2paddle.op_mapper.paddle2onnx.paddle_op_mapper import PaddleOpMapper
     model = PaddleDecoder(model_path, '__model__', '__params__')
     mapper = PaddleOpMapper()
-    mapper.convert(model.program, save_dir)
+    mapper.convert(model.program, save_dir, opset_number=opset_version)
 
 
 def main():
@@ -211,18 +201,6 @@ def main():
     assert args.framework is not None, "--framework is not defined(support tensorflow/caffe/onnx)"
     assert args.save_dir is not None, "--save_dir is not defined"
 
-    if args.framework == "onnx":
-        try:
-            import onnxruntime as rt
-            version = rt.__version__
-            if version != '1.0.0':
-                print("[ERROR] onnxruntime==1.0.0 is required")
-                return
-        except:
-            print(
-                "[ERROR] onnxruntime is not installed, use \"pip install onnxruntime==1.0.0\"."
-            )
-
     try:
         import paddle
         v0, v1, v2 = paddle.__version__.split('.')
@@ -261,13 +239,14 @@ def main():
     elif args.framework == "onnx":
         assert args.model is not None, "--model should be defined while translating onnx model"
         params_merge = False
+
         if args.params_merge:
             params_merge = True
         onnx2paddle(args.model, args.save_dir, params_merge)
 
     elif args.framework == "paddle2onnx":
         assert args.model is not None, "--model should be defined while translating paddle model to onnx"
-        paddle2onnx(args.model, args.save_dir)
+        paddle2onnx(args.model, args.save_dir, args.onnx_opset)
 
     else:
         raise Exception(

x2paddle/core/fluid_code.py

@@ -25,6 +25,7 @@ class Layer(object):
         self.inputs = dict()
         self.output = None
         self.is_custom_layer = False
+        self.use_fluid = False
 
     def get_code(self):
         layer_code = ""
@@ -38,6 +39,8 @@ class Layer(object):
             layer_code = layer_code + self.op + "("
         elif self.op == "=":
             layer_code = layer_code
+        elif self.use_fluid:
+            layer_code = layer_code + "fluid." + self.op + "("
         else:
             layer_code = layer_code + "fluid.layers." + self.op + "("
 
@@ -108,9 +111,11 @@ class FluidCode(object):
                   inputs,
                   output,
                   param_attr=None,
+                  use_fluid=False,
                   is_custom_layer=False):
         layer = Layer()
         layer.op = op
+        layer.use_fluid = use_fluid
         layer.is_custom_layer = is_custom_layer
         if inputs is not None:
             layer.inputs = inputs

x2paddle/core/op_mapper.py

@@ -29,11 +29,14 @@ def export_paddle_param(param, param_name, dir):
         "bool": [framework_pb2.VarType.BOOL, None]
     }
     shape = param.shape
+    if str(param.dtype) in ['uint8', 'uint_8', 'bool']:
+        param = param.astype('int64')
     if len(shape) == 0:
         assert param.size == 1, "Unexpected situation happend!"
         shape = [1]
-    assert str(param.dtype) in dtype_map, "Unknown dtype of params."
-
+    assert str(
+        param.dtype) in dtype_map, "Unknown dtype {} of params: {}.".format(
+            str(param.dtype), param_name)
     fp = open(os.path.join(dir, param_name), 'wb')
     numpy.array([0], dtype='int32').tofile(fp)
     numpy.array([0], dtype='int64').tofile(fp)
@@ -52,6 +55,24 @@ def export_paddle_param(param, param_name, dir):
 def run_net(param_dir="./"):
     import os
     inputs, outputs = x2paddle_net()
+
+    ops = fluid.default_main_program().global_block().ops
+    used_vars = list()
+    for op in ops:
+        used_vars += op.input_arg_names
+
+    tmp = list()
+    for input in inputs:
+        if isinstance(input, list):
+            for ipt in input:
+                if ipt.name not in used_vars:
+                    continue
+                tmp.append(ipt)
+        else:
+            if input.name not in used_vars:
+                continue
+            tmp.append(input)
+    inputs = tmp
     for i, out in enumerate(outputs):
         if isinstance(out, list):
             for out_part in out:
@@ -119,12 +140,30 @@ class OpMapper(object):
         import model
         try:
             inputs, outputs = model.x2paddle_net()
+
+            ops = fluid.default_main_program().global_block().ops
+            used_vars = list()
+            for op in ops:
+                used_vars += op.input_arg_names
+
             for i, out in enumerate(outputs):
                 if isinstance(out, list):
                     for out_part in out:
                         outputs.append(out_part)
                     del outputs[i]
-            input_names = [input.name for input in inputs]
+
+            input_names = list()
+            for input in inputs:
+                if isinstance(input, list):
+                    for ipt in input:
+                        if ipt.name not in used_vars:
+                            continue
+                        input_names.append(ipt.name)
+                else:
+                    if input.name not in used_vars:
+                        continue
+                    input_names.append(input.name)
+
             exe = fluid.Executor(fluid.CPUPlace())
             exe.run(fluid.default_startup_program())
 

x2paddle/core/program.py

+#   Copyright (c) 2019  PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License"
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from __future__ import print_function
+from __future__ import division
+import paddle.fluid as fluid
+from paddle.fluid.proto import framework_pb2
+import numpy
+import collections
+import sys
+import os
+import six
+
+
+class PaddleLayer(object):
+    def __init__(self, kernel, inputs, outputs, **kwargs):
+        assert isinstance(
+            inputs,
+            dict), "parameter 'inputs' for PaddleLayer should be type of dict"
+        assert isinstance(
+            outputs,
+            list), "parameter 'outputs' for PaddleLayer should be type of list"
+        for k, v in inputs.items():
+            if isinstance(v, list):
+                for i in v:
+                    assert isinstance(
+                        i, six.string_types
+                    ), "value in inputs should be type of string or list of string"
+            else:
+                assert isinstance(v, six.string_types) or isinstance(
+                    v, list
+                ), "value in inputs should be type of string or list of string"
+        for v in outputs:
+            assert isinstance(
+                v, six.
+                string_types), "elements in outputs should be type of string"
+        self.kernel = kernel
+        self.inputs = inputs
+        self.outputs = outputs
+        self.attrs = kwargs
+
+
+class PaddleProgram(object):
+    def __init__(self):
+        self.layers = list()
+        self.edges_out = dict()
+        self.edges_in = dict()
+        self.inputs = list()
+        self.outputs = list()
+        self.parameters = dict()
+
+    def clear(self):
+        self.layers = list()
+        self.edges_out = dict()
+        self.edges_in = dict()
+        self.inputs = list()
+        self.outputs = list()
+        self.parameters = dict()
+
+    def add_layer(self, kernel, inputs, outputs, **kwargs):
+        layer = PaddleLayer(kernel, inputs, outputs, **kwargs)
+        index = len(self.layers)
+        self.layers.append(layer)
+        return index
+
+    def build(self):
+        outputs_from_nodes = dict()
+        for i, layer in enumerate(self.layers):
+            for input_key, input_var in layer.inputs.items():
+                vs = input_var
+                if not isinstance(vs, list):
+                    vs = [vs]
+                for v in vs:
+                    assert v in outputs_from_nodes, "Couldn't find {} in previous layers, the layers should be make by topological sort".format(
+                        v)
+                    in_layer_index = outputs_from_nodes[v]
+                    if in_layer_index not in self.edges_out:
+                        self.edges_out[in_layer_index] = list()
+                    self.edges_out[in_layer_index].append(i)
+
+                    if i not in self.edges_in:
+                        self.edges_in[i] = list()
+                    self.edges_in[i].append(in_layer_index)
+            for output in layer.outputs:
+                outputs_from_nodes[output] = i
+
+    def get_layer_outputs(self, i):
+        return self.edges_out[i]
+
+    def get_layer_inputs(self, i):
+        return self.edges_in[i]
+
+    def gen_code(self, code_dir):
+        def write_code(f, code_list, indent=0):
+            indent_blank = "    " * indent
+            for code_line in code_list:
+                if code_line.strip() == "":
+                    f.write('\n')
+                else:
+                    f.write(indent_blank + code_line + '\n')
+
+        if not os.path.exists(code_dir):
+            os.makedirs(code_dir)
+        f = open(os.path.join(code_dir, 'x2paddle_model.py'), 'w')
+
+        write_code(
+            f, [
+                "from paddle.fluid.initializer import Constant",
+                "from paddle.fluid.param_attr import ParamAttr",
+                "import paddle.fluid as fluid"
+                "", "def x2paddle_net():"
+            ],
+            indent=0)
+        for i, layer in enumerate(self.layers):
+            edges_in = self.edges_in.get(i, [])
+            edges_out = self.edges_out.get(i, [])
+            if len(edges_in) == 0 and len(edges_out) == 0:
+                continue
+
+            line = ""
+
+            if len(layer.outputs) == 1:
+                line = layer.outputs[0]
+            else:
+                for output in layer.outputs:
+                    line += "{}, ".format(output)
+                line = line.strip(", ")
+
+            line += " = {}(".format(layer.kernel)
+            for k, v in layer.inputs.items():
+                if isinstance(v, list):
+                    line += "{}=[{}], ".format(k, ", ".join(v))
+                else:
+                    line += "{}={}, ".format(k, v)
+            for k, v in layer.attrs.items():
+                line += "{}={}, ".format(k, v)
+            line = line.strip(", ")
+            line += ")"
+            write_code(f, [line], indent=1)
+
+        write_code(
+            f, [
+                "return [{}], [{}]".format(", ".join(self.inputs),
+                                           ", ".join(self.outputs))
+            ],
+            indent=1)
+        f.close()
+
+    def gen_model(self, save_dir):
+        code_dir = os.path.join(save_dir, 'model_with_code')
+        infer_dir = os.path.join(save_dir, 'inference_model')
+        self.gen_code(code_dir)
+        sys.path.append(code_dir)
+        import x2paddle_model
+        scope = fluid.Scope()
+        startup_program = fluid.Program()
+        main_program = fluid.Program()
+        with fluid.scope_guard(scope):
+            with fluid.program_guard(main_program, startup_program):
+                inputs, outputs = x2paddle_model.x2paddle_net()
+                exe = fluid.Executor(fluid.CPUPlace())
+                exe.run(startup_program)
+
+                param_dir = os.path.join(code_dir, 'weights')
+                for k, v in self.parameters.items():
+                    if scope.find_var(k):
+                        self.dump_parameter(k, v, param_dir)
+
+                def if_exist(var):
+                    b = os.path.exists(
+                        os.path.join(os.path.join(param_dir, var.name)))
+                    return b
+
+                fluid.io.load_vars(
+                    exe, param_dir, main_program, predicate=if_exist)
+                fluid.io.save_inference_model(
+                    dirname=infer_dir,
+                    feeded_var_names=[i.name for i in inputs],
+                    target_vars=outputs,
+                    executor=exe)
+
+    def dump_parameter(self, param_name, param, save_dir):
+        if not os.path.exists(save_dir):
+            os.makedirs(save_dir)
+        dtype_map = {
+            "int16": [framework_pb2.VarType.INT16, 'h'],
+            "int32": [framework_pb2.VarType.INT32, 'i'],
+            "int64": [framework_pb2.VarType.INT64, 'q'],
+            "float16": [framework_pb2.VarType.FP16, 'e'],
+            "float32": [framework_pb2.VarType.FP32, 'f'],
+            "float64": [framework_pb2.VarType.FP64, 'd'],
+            "bool": [framework_pb2.VarType.BOOL, None]
+        }
+        shape = param.shape
+        if str(param.dtype) in ['uint8', 'uint_8', 'bool']:
+            param = param.astype('int64')
+        if len(shape) == 0:
+            assert param.size == 1, "Unexpected situation happend!"
+            shape = [1]
+        assert str(
+            param.dtype) in dtype_map, "Unknown dtype {} of params: {}.".format(
+                str(param.dtype), param_name)
+        fp = open(os.path.join(save_dir, param_name), 'wb')
+        numpy.array([0], dtype='int32').tofile(fp)
+        numpy.array([0], dtype='int64').tofile(fp)
+        numpy.array([0], dtype='int32').tofile(fp)
+        tensor_desc = framework_pb2.VarType.TensorDesc()
+        tensor_desc.data_type = dtype_map[str(param.dtype)][0]
+        tensor_desc.dims.extend(shape)
+        desc_size = tensor_desc.ByteSize()
+        numpy.array([desc_size], dtype='int32').tofile(fp)
+        fp.write(tensor_desc.SerializeToString())
+        param.tofile(fp)
+        fp.close()

x2paddle/decoder/onnx_decoder.py

@@ -14,6 +14,7 @@
 
 from x2paddle.core.graph import GraphNode, Graph
 from x2paddle.core.fluid_code import FluidCode
+from x2paddle.decoder.onnx_shape_inference import SymbolicShapeInference
 from onnx.checker import ValidationError
 from onnx.checker import check_model
 from onnx.utils import polish_model
@@ -53,7 +54,7 @@ class ONNXGraphNode(GraphNode):
         convert ONNX node attributes to dict
         """
         return {
-            attr.name: self.get_attribute_value2(attr)
+            attr.name: self.get_attribute_value(attr)
             for attr in self.layer.attribute
         }
 
@@ -64,7 +65,7 @@ class ONNXGraphNode(GraphNode):
             return None
         return self.attr_map['value']
 
-    def get_attribute_value2(self, attr):
+    def get_attribute_value(self, attr):
         """
         get_attribute_value enhanced
         """
@@ -130,43 +131,90 @@ class ONNXGraphDataNode(GraphNode):
 
 class ONNXGraph(Graph):
     def __init__(self, onnx_model):
-        super(ONNXGraph, self).__init__(onnx_model.graph)
-        self.onnx_model = onnx_model
+        super(ONNXGraph, self).__init__(onnx_model)
+        self.fixed_input_shape = {}
         self.initializer = {}
         self.place_holder_nodes = list()
+        self.value_infos = {}
+        self.graph = onnx_model.graph
         self.get_place_holder_nodes()
-        self.value_infos = self.inferred_model_value_info(self.model)
-        self.results_of_inference = dict()
+        print("shape inferencing ...")
+        self.graph = SymbolicShapeInference.infer_shapes(
+            onnx_model, fixed_input_shape=self.fixed_input_shape)
+        print("shape inferenced.")
+        self.build()
+        self.collect_value_infos()
+        self.allocate_shapes()
 
     def get_inner_nodes(self):
         """
         generate inner node of ONNX model
         """
         inner_nodes = []
-        if not isinstance(self.model, onnx.GraphProto):
+        if not isinstance(self.graph, onnx.GraphProto):
             logger.error('graph is not a GraphProto instance')
             return
-        for initializer in self.model.initializer:
+        for initializer in self.graph.initializer:
             name = initializer.name
             inner_nodes.append(name)
         return inner_nodes
 
+    def get_symbolic_shape(self, dims):
+        shape = []
+        for dim in dims:
+            if dim.HasField('dim_param'):
+                shape.append(dim.dim_param)
+            else:
+                shape.append(dim.dim_value)
+        return shape
+
+    def check_input_shape(self, vi):
+        if vi.type.HasField('tensor_type'):
+            for dim in vi.type.tensor_type.shape.dim:
+                if dim.HasField(
+                        'dim_param') and vi.name not in self.fixed_input_shape:
+                    shape = self.get_symbolic_shape(
+                        vi.type.tensor_type.shape.dim)
+                    print(
+                        "Unknown shape for input tensor[tensor name: '{}'] -> shape: {}, Please define shape of input here,\nNote:you can use visualization tools like Netron to check input shape."
+                        .format(vi.name, shape))
+                    right_shape_been_input = False
+                    while not right_shape_been_input:
+                        try:
+                            shape = raw_input(
+                                "Shape of Input(e.g. -1,3,224,224), enter 'N' to skip: "
+                            )
+                        except:
+                            shape = input(
+                                "Shape of Input(e.g. -1,3,224,224), enter 'N' to skip: "
+                            )
+                        if shape.count("-1") > 1:
+                            print("Only 1 dimension can be -1, type again:)")
+                        else:
+                            right_shape_been_input = True
+                    if shape == 'N':
+                        break
+                    shape = [int(dim) for dim in shape.strip().split(',')]
+                    assert shape.count(-1) <= 1, "Only one dimension can be -1"
+                    self.fixed_input_shape[vi.name] = shape
+                    break
+
     def get_place_holder_nodes(self):
         """
         generate place_holder node of ONNX model
         """
         inner_nodes = self.get_inner_nodes()
-        input_nodes = [value.name for value in self.model.input]
-        for ipt_data in input_nodes:
-            if ipt_data not in inner_nodes:
-                self.place_holder_nodes.append(ipt_data)
+        for ipt_vi in self.graph.input:
+            if ipt_vi.name not in inner_nodes:
+                self.check_input_shape(ipt_vi)
+                self.place_holder_nodes.append(ipt_vi.name)
 
     def get_output_nodes(self):
         """
         generate output_nodes node of ONNX model
         """
         inner_nodes = self.get_inner_nodes()
-        output_nodes = [value.name for value in self.model.output]
+        output_nodes = [value.name for value in self.graph.output]
         for opt_data in output_nodes:
             if opt_data not in inner_nodes:
                 self.output_nodes.append(opt_data)
@@ -183,11 +231,11 @@ class ONNXGraph(Graph):
         """
         build topo_sort of ONNX model
         """
-        for layer in self.model.node:
+        for layer in self.graph.node:
             node = ONNXGraphNode(layer)
             self.node_map[layer.name] = node
 
-        for layer in self.model.input:
+        for layer in self.graph.input:
             if layer.name not in self.node_map:
                 is_place_holder = self.is_place_holder_nodes(layer.name)
                 self.node_map[layer.name] = ONNXGraphDataNode(
@@ -196,7 +244,7 @@ class ONNXGraph(Graph):
                     is_global_input=is_place_holder)
 
         #set data node's weight
-        for initializer in self.model.initializer:
+        for initializer in self.graph.initializer:
             name = initializer.name
             weight = to_array(initializer)
             if name in self.node_map:
@@ -228,7 +276,7 @@ class ONNXGraph(Graph):
                 continue
             if in_node not in self.node_map:
                 flag = 0
-                for nd in self.model.node:
+                for nd in self.graph.node:
                     for idx, opt in enumerate(nd.output):
                         if opt == in_node:
                             self.connect(nd.name, layer_name)
@@ -256,81 +304,68 @@ class ONNXGraph(Graph):
                 ipt_node.index = node.which_child[ipt_node.layer_name]
             return ipt_node
 
-    def graph_weights(self, graph):
+    def graph_weights(self):
         """
         generator for weights
         """
 
-        if not isinstance(graph, onnx.GraphProto):
+        if not isinstance(self.graph, onnx.GraphProto):
             logger.error('graph is not a GraphProto instance')
             return
 
-        for initializer in graph.initializer:
+        for initializer in self.graph.initializer:
             name = initializer.name
             weight = to_array(initializer)
             yield name, weight
 
-    def inferred_model_value_info(self, graph):
+    def collect_value_infos(self):
         """
         collect value/type info for an ONNX model
         """
-        assert isinstance(graph,
+        assert isinstance(self.graph,
                           onnx.GraphProto), 'model is not a ModelProto instance'
 
-        value_info = Dict()
-        for item in graph.value_info:
-            value_info[item.name] = {
+        for item in self.graph.value_info:
+            self.value_infos[item.name] = {
                 'dtype':
                 TENSOR_TYPE_TO_NP_TYPE[item.type.tensor_type.elem_type],
                 'shape':
                 [dim.dim_value for dim in item.type.tensor_type.shape.dim],
                 'external': False
             }
-        for item in graph.input:
-            assert item.name not in value_info
-            value_info[item.name] = {
-                'dtype':
-                TENSOR_TYPE_TO_NP_TYPE[item.type.tensor_type.elem_type],
-                'shape':
-                [dim.dim_value for dim in item.type.tensor_type.shape.dim],
-                'external': True
-            }
-        for item in graph.output:
-            assert item.name not in value_info
-            value_info[item.name] = {
-                'dtype':
-                TENSOR_TYPE_TO_NP_TYPE[item.type.tensor_type.elem_type],
-                'shape':
-                [dim.dim_value for dim in item.type.tensor_type.shape.dim],
-                'external': True
-            }
-        return value_info
+
+    def allocate_shapes(self):
+        """
+        run shape inference
+        """
+        for layer in self.graph.node:
+            node = self.node_map[layer.name]
+            for opt in layer.output:
+                if opt in self.value_infos:
+                    value_info = self.value_infos[opt]
+                    #if len(value_info['shape']) == 0 or value_info[
+                    #        'dtype'] is None or 0 in value_info['shape']:
+                    #    #TODO add node shape inference
+                    node.dtype = value_info['dtype']
+                    node.out_shapes.append(value_info['shape'])
+                else:
+                    node.out_shapes.append([])
 
 
 class ONNXDecoder(object):
     def __init__(self, onnx_model):
-        model = onnx.load(onnx_model)
+        onnx_model = onnx.load(onnx_model)
         print('model ir_version: {}, op version: {}'.format(
-            model.ir_version, model.opset_import[0].version))
-        if model.opset_import[0].version < 9:
-            _logger.warning(
-                'Now, onnx2paddle support convert onnx model opset_verison == 9,'
-                'opset_verison of your onnx model is %d < 9,'
-                'some operator maybe unsuccessful in convertion.',
-                model.opset_import[0].version)
-
-        check_model(model)
-        self.check_model_running_state(onnx_model)
-
-        model = onnx.shape_inference.infer_shapes(model)
-        model = self.optimize_model_skip_op_for_inference(model)
-        model = self.optimize_model_strip_initializer(model)
-        self.standardize_variable_name(model.graph)
-
-        self.model = model
-        graph = model.graph
-        self.onnx_graph = ONNXGraph(model)
-        self.onnx_graph.build()
+            onnx_model.ir_version, onnx_model.opset_import[0].version))
+        self.op_set = onnx_model.opset_import[0].version
+
+        check_model(onnx_model)
+
+        onnx_model = self.optimize_model_skip_op(onnx_model)
+        onnx_model = self.optimize_model_strip_initializer(onnx_model)
+        onnx_model = self.optimize_node_name(onnx_model)
+        self.graph = ONNXGraph(onnx_model)
+        #self.onnx_model = onnx_model
 
     def build_value_refs(self, nodes):
         """
@@ -373,14 +408,13 @@ class ONNXDecoder(object):
                     processed += 1
         return processed
 
-    def optimize_model_skip_op_for_inference(self, model, op_list=None):
+    def optimize_model_skip_op(self, model, op_list=None):
         """
         skip ops can be bypassed for inference
         """
+        nodes = model.graph.node
         if op_list is None:
             op_list = ['Dropout']
-
-        nodes = model.graph.node
         input_refs, output_refs = self.build_value_refs(nodes)
         ret = type(model)()
         ret.CopyFrom(model)
@@ -473,38 +507,11 @@ class ONNXDecoder(object):
             name = name.replace(s, '_')
         return 'x2paddle_' + name
 
-    def check_model_running_state(self, model_path):
-        import onnxruntime as rt
-        model = onnx.load(model_path)
-        model = onnx.shape_inference.infer_shapes(model)
-        if len(model.graph.value_info) < len(model.graph.node) - 1:
-            _logger.warning(
-                'During conversion of your  model, some operators will be assignd node.out_shape==None, '
-                'refer to https://github.com/onnx/onnx/blob/master/docs/ShapeInference.md'
-            )
-        try:
-            datatype_map = {
-                'tensor(int64)': 'int',
-                'tensor(float)': 'float32',
-                'tensor(int32)': 'int32'
-            }
-            input_dict = {}
-            sess = rt.InferenceSession(model_path)
-            for ipt in sess.get_inputs():
-                datatype = datatype_map[ipt.type]
-                input_dict[ipt.name] = np.random.random(ipt.shape).astype(
-                    datatype)
-
-            res = sess.run(None, input_feed=input_dict)
-        except:
-            raise Exception(
-                "onnxruntime inference onnx model failed, Please confirm the correctness of onnx model by onnxruntime, if onnx model is correct, please submit issue in github."
-            )
-
-    def standardize_variable_name(self, graph):
+    def optimize_node_name(self, model):
         """
         standardize variable name for paddle's code
         """
+        graph = model.graph
         for initializer in graph.initializer:
             initializer.name = self.make_variable_name(initializer.name)
         for ipt in graph.input:
@@ -523,3 +530,4 @@ class ONNXDecoder(object):
                     node.input[i] = self.make_variable_name(node.input[i])
             for i in range(len(node.output)):
                 node.output[i] = self.make_variable_name(node.output[i])
+        return model

x2paddle/decoder/onnx_shape_inference.py

+# Copyright (c) 2019  PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License"
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# Reference Code from https://github.com/microsoft/onnxruntime,  Licensed under the MIT License.
+
+# -*- coding: UTF-8 -*-
+import argparse
+import numpy as np
+import onnx
+import sys
+from onnx import helper, numpy_helper, shape_inference
+import sympy
+
+from packaging import version
+assert version.parse(onnx.__version__) >= version.parse("1.5.0")
+
+
+def get_attribute(node, attr_name, default_value=None):
+    found = [attr for attr in node.attribute if attr.name == attr_name]
+    if found:
+        return helper.get_attribute_value(found[0])
+    return default_value
+
+
+def get_dim_from_type_proto(dim):
+    return getattr(dim, dim.WhichOneof('value')) if type(
+        dim.WhichOneof('value')) == str else None
+
+
+def get_shape_from_type_proto(type_proto):
+    return [
+        get_dim_from_type_proto(d) for d in type_proto.tensor_type.shape.dim
+    ]
+
+
+def get_shape_from_sympy_shape(sympy_shape):
+    sympy_shape = [
+        None if i is None else (int(i) if is_literal(i) else str(i))
+        for i in sympy_shape
+    ]
+    return sympy_shape
+
+
+def is_literal(dim):
+    return type(dim) in [int, np.int64, np.int32, sympy.Integer] or (
+        hasattr(dim, 'is_number') and
+        dim.is_number)  # or (hasattr(dim, 'is_integer') and dim.is_integer)
+
+
+def handle_negative_axis(axis, rank):
+    assert axis < rank and axis >= -rank
+    return axis if axis >= 0 else rank + axis
+
+
+def get_opset(mp, domain=['', 'onnx', 'ai.onnx']):
+    if type(domain) != list:
+        domain = [domain]
+    for opset in mp.opset_import:
+        if opset.domain in domain:
+            return opset.version
+    return None
+
+
+def as_scalar(x):
+    if type(x) == list:
+        assert len(x) == 1
+        return x[0]
+    elif type(x) == np.ndarray:
+        return np.asscalar(x)
+    else:
+        return x
+
+
+def as_list(x, keep_none):
+    if type(x) == list:
+        return x
+    elif type(x) == np.ndarray:
+        return list(x)
+    elif keep_none and x is None:
+        return None
+    else:
+        return [x]
+
+
+def sympy_reduce_product(x):
+    if type(x) == list:
+        value = sympy.Integer(1)
+        for v in x:
+            value = value * v
+    else:
+        value = x
+    return value
+
+
+class SymbolicShapeInference:
+    def __init__(self, int_max, auto_merge, guess_output_rank, verbose):
+        self.dispatcher_ = {
+            'Add': self._infer_symbolic_compute_ops,
+            'ArrayFeatureExtractor': self._infer_ArrayFeatureExtractor,
+            'AveragePool': self._infer_Pool,
+            'Cast': self._infer_Cast,
+            'CategoryMapper': self._infer_CategoryMapper,
+            'Compress': self._infer_Compress,
+            'Concat': self._infer_Concat,
+            'ConstantOfShape': self._infer_ConstantOfShape,
+            'Conv': self._infer_Conv,
+            'CumSum': self._pass_on_shape_and_type,
+            'Div': self._infer_symbolic_compute_ops,
+            'Expand': self._infer_Expand,
+            'Equal': self._infer_symbolic_compute_ops,
+            'Gather': self._infer_Gather,
+            'GatherElements': self._infer_GatherElements,
+            'GatherND': self._infer_GatherND,
+            'If': self._infer_If,
+            'Loop': self._infer_Loop,
+            'MatMul': self._infer_MatMul,
+            'MatMulInteger16': self._infer_MatMulInteger,
+            'MaxPool': self._infer_Pool,
+            'Max': self._infer_symbolic_compute_ops,
+            'Min': self._infer_symbolic_compute_ops,
+            'Mul': self._infer_symbolic_compute_ops,
+            'NonMaxSuppression': self._infer_NonMaxSuppression,
+            'NonZero': self._infer_NonZero,
+            'OneHot': self._infer_OneHot,
+            'Pad': self._infer_Pad,
+            'Range': self._infer_Range,
+            'ReduceProd': self._infer_ReduceProd,
+            'Reshape': self._infer_Reshape,
+            'Resize': self._infer_Resize,
+            'Round': self._pass_on_shape_and_type,
+            'Scan': self._infer_Scan,
+            'ScatterElements': self._infer_ScatterElements,
+            'Shape': self._infer_Shape,
+            'Size': self._infer_Size,
+            'Slice': self._infer_Slice,
+            'Split': self._infer_Split,
+            'Squeeze': self._infer_Squeeze,
+            'Sub': self._infer_symbolic_compute_ops,
+            'Tile': self._infer_Tile,
+            'TopK': self._infer_TopK,
+            'Unsqueeze': self._infer_Unsqueeze,
+            'Where': self._infer_symbolic_compute_ops,
+            'Transpose': self._infer_Transpose,
+            'ZipMap': self._infer_ZipMap
+        }
+        self.run_ = True
+        self.suggested_merge_ = {}
+        self.symbolic_dims_ = {}
+        self.input_symbols_ = {}
+        self.auto_merge_ = auto_merge
+        self.guess_output_rank_ = guess_output_rank
+        self.verbose_ = verbose
+        self.int_max_ = int_max
+
+    def _add_suggested_merge(self, symbols, apply=False):
+        assert all([(type(s) == str and s in self.symbolic_dims_) or
+                    is_literal(s) for s in symbols])
+        symbols = set(symbols)
+        for k, v in self.suggested_merge_.items():
+            if k in symbols:
+                symbols.remove(k)
+                symbols.add(v)
+        map_to = None
+        # if there is literal, map to it first
+        for s in symbols:
+            if is_literal(s):
+                map_to = s
+                break
+        # when no literals, map to input symbolic dims, then existing symbolic dims
+        if map_to is None:
+            for s in symbols:
+                if s in self.input_symbols_:
+                    map_to = s
+                    break
+        if map_to is None:
+            for s in symbols:
+                if type(self.symbolic_dims_[s]) == sympy.Symbol:
+                    map_to = s
+                    break
+        # when nothing to map to, use the shorter one
+        if map_to is None:
+            if self.verbose_ > 0:
+                print(
+                    'Potential unsafe merge between symbolic expressions: ({})'.
+                    format(','.join(symbols)))
+            symbols_list = list(symbols)
+            lens = [len(s) for s in symbols_list]
+            map_to = symbols_list[lens.index(min(lens))]
+            symbols.remove(map_to)
+
+        for s in symbols:
+            if s == map_to:
+                continue
+            if is_literal(map_to) and is_literal(s):
+                assert int(map_to) == int(s)
+            self.suggested_merge_[s] = int(map_to) if is_literal(
+                map_to) else map_to
+            for k, v in self.suggested_merge_.items():
+                if v == s:
+                    self.suggested_merge_[k] = map_to
+        if apply and self.auto_merge_:
+            self._apply_suggested_merge()
+
+    def _apply_suggested_merge(self, graph_input_only=False):
+        if not self.suggested_merge_:
+            return
+        for i in list(self.out_mp_.graph.input) + (
+            [] if graph_input_only else list(self.out_mp_.graph.value_info)):
+            for d in i.type.tensor_type.shape.dim:
+                if d.dim_param in self.suggested_merge_:
+                    v = self.suggested_merge_[d.dim_param]
+                    if is_literal(v):
+                        d.dim_value = int(v)
+                    else:
+                        d.dim_param = v
+
+    def _preprocess(self, in_mp, input_shapes=None):
+        out_mp = onnx.ModelProto()
+        out_mp.CopyFrom(in_mp)
+        out_mp.graph.ClearField('node')
+        self.out_mp_ = out_mp
+
+        defined = set([
+            i.name
+            for i in list(in_mp.graph.input) + list(in_mp.graph.initializer)
+        ])
+        pending_nodes = []
+
+        # returns True if no more ready nodes
+        def _insert_ready_nodes():
+            ready_nodes = [
+                pn for pn in pending_nodes
+                if all([i in defined for i in pn.input if i])
+            ]
+            for rn in ready_nodes:
+                self.out_mp_.graph.node.add().CopyFrom(rn)
+                for o in rn.output:
+                    defined.add(o)
+                pending_nodes.remove(rn)
+            return not ready_nodes
+
+        # constant op -> initializer, topological sort
+        for in_n in in_mp.graph.node:
+            if in_n.op_type == 'Constant':
+                t = get_attribute(in_n, 'value')
+                t.name = in_n.output[0]
+                self.out_mp_.graph.initializer.add().CopyFrom(t)
+                defined.add(t.name)
+            else:
+                pending_nodes.append(in_n)
+            _insert_ready_nodes()
+
+        while pending_nodes:
+            if _insert_ready_nodes():
+                break
+
+        if pending_nodes and self.verbose_ > 0:
+            print('SymbolicShapeInference: orphaned nodes discarded: ')
+            print(
+                * [n.op_type + ': ' + n.output[0] for n in pending_nodes],
+                sep='\n')
+        if input_shapes is not None:
+            for input_name, shape in input_shapes.items():
+                for idx in range(len(self.out_mp_.graph.input)):
+                    if self.out_mp_.graph.input[idx].name == input_name:
+                        value_info = self.out_mp_.graph.input[idx]
+                        del self.out_mp_.graph.input[idx]
+                        self.out_mp_.graph.input.append(
+                            helper.make_tensor_value_info(
+                                value_info.name,
+                                value_info.type.tensor_type.elem_type, shape))
+        self.initializers_ = dict(
+            [(i.name, i) for i in self.out_mp_.graph.initializer])
+        self.known_vi_ = dict(
+            [(i.name, i) for i in list(self.out_mp_.graph.input)])
+        self.known_vi_.update(
+            dict([(i.name, helper.make_tensor_value_info(i.name, i.data_type,
+                                                         list(i.dims)))
+                  for i in self.out_mp_.graph.initializer]))
+
+    def _merge_symbols(self, dims):
+        if not all([type(d) == str for d in dims]):
+            if self.auto_merge_:
+                assert len(
+                    dims
+                ) == 2  # only allow symbol->int merge in binary ops for now
+                is_int = [is_literal(d) for d in dims]
+                if sum(is_int) == 1:
+                    int_dim = is_int.index(1)
+                    if self.verbose_ > 0:
+                        print('dim {} has been merged with value {}'.format(
+                            dims[1 - int_dim], dims[int_dim]))
+                    self._check_merged_dims(dims, allow_broadcast=False)
+                    return dims[int_dim]
+                else:
+                    if self.verbose_ > 0:
+                        print('dim {} has been mergd with dim {}'.format(dims[
+                            0], dims[1]))
+                    return dims[0]
+            else:
+                return None
+        if all([d == dims[0] for d in dims]):
+            return dims[0]
+        merged = [
+            self.suggested_merge_[d] if d in self.suggested_merge_ else d
+            for d in dims
+        ]
+        if all([d == merged[0] for d in merged]):
+            assert merged[0] in self.symbolic_dims_
+            return merged[0]
+        else:
+            return None
+
+    # broadcast from right to left, and merge symbolic dims if needed
+    def _broadcast_shapes(self, shape1, shape2):
+        new_shape = []
+        rank1 = len(shape1)
+        rank2 = len(shape2)
+        new_rank = max(rank1, rank2)
+        for i in range(new_rank):
+            dim1 = shape1[rank1 - 1 - i] if i < rank1 else 1
+            dim2 = shape2[rank2 - 1 - i] if i < rank2 else 1
+            if dim1 == 1 or dim1 == dim2:
+                new_dim = dim2
+            elif dim2 == 1:
+                new_dim = dim1
+            else:
+                new_dim = self._merge_symbols([dim1, dim2])
+                if not new_dim:
+                    # warning about unsupported broadcast when not auto merge
+                    # note that auto merge has the risk of incorrectly merge symbols while one of them being 1
+                    # for example, 'a' = 1, 'b' = 5 at runtime is valid broadcasting, but with auto merge 'a' == 'b'
+                    if self.auto_merge_:
+                        self._add_suggested_merge([dim1, dim2], apply=True)
+                    else:
+                        print('unsupported broadcast between ' + str(dim1) + ' '
+                              + str(dim2))
+            new_shape = [new_dim] + new_shape
+        return new_shape
+
+    def _get_shape(self, node, idx):
+        name = node.input[idx]
+        shape = []
+        if name in self.known_vi_:
+            shape = get_shape_from_type_proto(self.known_vi_[name].type)
+        elif name in self.initializers_:
+            assert name in self.initializers_
+            shape = list(self.initializers_[name].dims)
+        return shape
+
+    def _get_initializer_value(self, node, idx):
+        name = node.input[idx]
+        if name in self.initializers_:
+            value = numpy_helper.to_array(self.initializers_[name])
+            return value
+        else:
+            return False
+
+    def _get_shape_rank(self, node, idx):
+        return len(self._get_shape(node, idx))
+
+    def _get_sympy_shape(self, node, idx):
+        sympy_shape = []
+        for d in self._get_shape(node, idx):
+            if type(d) is str:
+                sympy_shape.append(self.symbolic_dims_[d] if d in
+                                   self.symbolic_dims_ else sympy.Symbol(
+                                       d, integer=True))
+            else:
+                assert None != d
+                sympy_shape.append(d)
+        return sympy_shape
+
+    def _get_value(self, node, idx):
+        name = node.input[idx]
+        assert name in self.sympy_data_ or name in self.initializers_
+        return self.sympy_data_[
+            name] if name in self.sympy_data_ else numpy_helper.to_array(
+                self.initializers_[name])
+
+    def _try_get_value(self, node, idx):
+        if idx >= len(node.input):
+            return None
+        name = node.input[idx]
+        if name in self.sympy_data_ or name in self.initializers_:
+            return self._get_value(node, idx)
+        return None
+
+    def _update_computed_dims(self, new_sympy_shape):
+        for i, new_dim in enumerate(new_sympy_shape):
+            if not is_literal(new_dim) and not type(new_dim) == str:
+                str_dim = str(new_dim)
+                if str_dim in self.suggested_merge_:
+                    new_sympy_shape[i] = self.symbolic_dims_[
+                        self.suggested_merge_[str_dim]]
+                else:
+                    # add new_dim if it's a computational expression
+                    if not str(new_dim) in self.symbolic_dims_:
+                        self.symbolic_dims_[str(new_dim)] = new_dim
+
+    def _onnx_infer_single_node(self, node):
+        # skip onnx shape inference for Scan/Loop
+        skip_infer = node.op_type in ['Scan', 'Loop']
+        if not skip_infer:
+            # run single node inference with self.known_vi_ shapes
+            # note that inference rely on initializer values is not handled
+            # as we don't copy initializer weights to tmp_graph for inference speed purpose
+            tmp_graph = helper.make_graph(
+                [node], 'tmp', [self.known_vi_[i] for i in node.input if i], [
+                    helper.make_tensor_value_info(i, onnx.TensorProto.UNDEFINED,
+                                                  None) for i in node.output
+                ])
+            self.tmp_mp_.graph.CopyFrom(tmp_graph)
+            self.tmp_mp_ = shape_inference.infer_shapes(self.tmp_mp_)
+        for i_o in range(len(node.output)):
+            o = node.output[i_o]
+            vi = self.out_mp_.graph.value_info.add()
+            if not skip_infer:
+                vi.CopyFrom(self.tmp_mp_.graph.output[i_o])
+            self.known_vi_[o] = vi
+
+    def _onnx_infer_subgraph(self, node, subgraph, use_node_input=True):
+        if self.verbose_ > 2:
+            print('Inferencing subgraph of node {} with output({}...): {}'.
+                  format(node.name, node.output[0], node.op_type))
+        # node inputs are not passed directly to the subgraph
+        # it's up to the node dispatcher to prepare subgraph input
+        # for example, with Scan/Loop, subgraph input shape would be trimmed from node input shape
+        # besides, inputs in subgraph could shadow implicit inputs
+        subgraph_inputs = set([
+            i.name for i in list(subgraph.initializer) + list(subgraph.input)
+        ])
+        subgraph_implicit_input = set([
+            name for name in self.known_vi_.keys()
+            if not name in subgraph_inputs
+        ])
+        tmp_graph = helper.make_graph(
+            list(subgraph.node), 'tmp',
+            list(subgraph.input) +
+            [self.known_vi_[i] for i in subgraph_implicit_input], [
+                helper.make_tensor_value_info(i.name,
+                                              onnx.TensorProto.UNDEFINED, None)
+                for i in subgraph.output
+            ])
+        tmp_graph.initializer.extend([
+            i for i in self.out_mp_.graph.initializer
+            if i.name in subgraph_implicit_input
+        ])
+        tmp_graph.initializer.extend(subgraph.initializer)
+        self.tmp_mp_.graph.CopyFrom(tmp_graph)
+
+        symbolic_shape_inference = SymbolicShapeInference(
+            self.int_max_, self.auto_merge_, self.guess_output_rank_,
+            self.verbose_)
+        all_shapes_inferred = False
+        symbolic_shape_inference._preprocess(self.tmp_mp_)
+        symbolic_shape_inference.suggested_merge_ = self.suggested_merge_.copy()
+        while symbolic_shape_inference.run_:
+            all_shapes_inferred = symbolic_shape_inference._infer_impl(
+                self.tmp_mp_, self.sympy_data_.copy())
+        symbolic_shape_inference._update_output_from_vi()
+        if use_node_input:
+            # if subgraph uses node input, it needs to update to merged dims
+            subgraph.ClearField('input')
+            subgraph.input.extend(
+                symbolic_shape_inference.out_mp_.graph.input[:len(node.input)])
+        subgraph.ClearField('output')
+        subgraph.output.extend(symbolic_shape_inference.out_mp_.graph.output)
+        subgraph.ClearField('value_info')
+        subgraph.value_info.extend(
+            symbolic_shape_inference.out_mp_.graph.value_info)
+        subgraph.ClearField('node')
+        subgraph.node.extend(symbolic_shape_inference.out_mp_.graph.node)
+        # for new symbolic dims from subgraph output, add to main graph symbolic dims
+        subgraph_shapes = [
+            get_shape_from_type_proto(o.type)
+            for o in symbolic_shape_inference.out_mp_.graph.output
+        ]
+        subgraph_new_symbolic_dims = set([
+            d for s in subgraph_shapes
+            if s for d in s if type(d) == str and not d in self.symbolic_dims_
+        ])
+        new_dims = {}
+        for d in subgraph_new_symbolic_dims:
+            assert d in symbolic_shape_inference.symbolic_dims_
+            new_dims[d] = symbolic_shape_inference.symbolic_dims_[d]
+        self.symbolic_dims_.update(new_dims)
+        return symbolic_shape_inference
+
+    def _get_int_values(self, node, broadcast=False):
+        values = [self._try_get_value(node, i) for i in range(len(node.input))]
+        if all([v is not None for v in values]):
+            # some shape compute is in floating point, cast to int for sympy
+            for i, v in enumerate(values):
+                if type(v) != np.ndarray:
+                    continue
+                if len(v.shape) > 1:
+                    new_v = None  # ignore value for rank > 1
+                elif len(v.shape) == 0:
+                    new_v = int(np.asscalar(v))
+                else:
+                    assert len(v.shape) == 1
+                    new_v = [int(vv) for vv in v]
+                values[i] = new_v
+        values_len = [len(v) if type(v) == list else 0 for v in values]
+        max_len = max(values_len)
+        if max_len >= 1 and broadcast:
+            # broadcast
+            for i, v in enumerate(values):
+                if v is None:
+                    continue  # don't broadcast if value is unknown
+                if type(v) == list:
+                    if len(v) < max_len:
+                        values[i] = v * max_len
+                    else:
+                        assert len(v) == max_len
+                else:
+                    values[i] = [v] * max_len
+        return values
+
+    def _compute_on_sympy_data(self, node, op_func):
+        assert len(node.output) == 1
+        values = self._get_int_values(node, broadcast=True)
+        if all([v is not None for v in values]):
+            new_shape = []
+            is_list = [type(v) == list for v in values]
+            as_list = any(is_list)
+            if as_list:
+                data = [op_func(vs) for vs in zip(*values)]
+                self.sympy_data_[node.output[0]] = data
+                new_shape = np.array(data).shape
+            else:
+                data = op_func(values)
+                self.sympy_data_[node.output[0]] = data
+                new_shape = np.array(data).shape
+            vi = self.known_vi_[node.output[0]]
+            #print(node.output[0])
+            #print(new_shape)
+            #vi.CopyFrom(helper.make_tensor_value_info(node.output[0], self.known_vi_[node.input[0]].type.tensor_type.elem_type, list(new_shape)))
+
+    def _pass_on_sympy_data(self, node):
+        assert len(node.input) == 1 or node.op_type == 'Reshape'
+        self._compute_on_sympy_data(node, lambda x: x[0])
+
+    def _pass_on_shape_and_type(self, node):
+        vi = self.known_vi_[node.output[0]]
+        vi.CopyFrom(
+            helper.make_tensor_value_info(node.output[0], self.known_vi_[
+                node.input[0]].type.tensor_type.elem_type,
+                                          self._get_shape(node, 0)))
+
+    def _new_symbolic_dim(self, prefix, dim):
+        new_dim = '{}_d{}'.format(prefix, dim)
+        if new_dim in self.suggested_merge_:
+            v = self.suggested_merge_[new_dim]
+            new_dim = sympy.Integer(int(v)) if is_literal(v) else v
+        else:
+            self.symbolic_dims_[new_dim] = sympy.Symbol(new_dim, integer=True)
+        return new_dim
+
+    def _new_symbolic_dim_from_output(self, node, out_idx=0, dim=0):
+        return self._new_symbolic_dim('{}{}_o{}_'.format(
+            node.op_type, list(self.out_mp_.graph.node).index(node), out_idx),
+                                      dim)
+
+    def _new_symbolic_shape(self, rank, node, out_idx=0):
+        return [
+            self._new_symbolic_dim_from_output(node, out_idx, i)
+            for i in range(rank)
+        ]
+
+    def _compute_conv_pool_shape(self, node):
+        sympy_shape = self._get_sympy_shape(node, 0)
+        if len(node.input) > 1:
+            W_shape = self._get_sympy_shape(node, 1)
+            rank = len(W_shape) - 2  # number of spatial axes
+            kernel_shape = W_shape[-rank:]
+            sympy_shape[1] = W_shape[0]
+        else:
+            W_shape = None
+            kernel_shape = get_attribute(node, 'kernel_shape')
+            rank = len(kernel_shape)
+
+        assert len(sympy_shape) == rank + 2
+
+        # only need to symbolic shape inference if input has symbolic dims in spatial axes
+        is_symbolic_dims = [not is_literal(i) for i in sympy_shape[-rank:]]
+
+        if not any(is_symbolic_dims):
+            shape = get_shape_from_type_proto(self.known_vi_[node.output[0]]
+                                              .type)
+            if len(shape) > 0:
+                assert len(sympy_shape) == len(shape)
+                sympy_shape[-rank:] = [sympy.Integer(d) for d in shape[-rank:]]
+                return sympy_shape
+
+        dilations = get_attribute(node, 'dilations', [1] * rank)
+        strides = get_attribute(node, 'strides', [1] * rank)
+        effective_kernel_shape = [(k - 1) * d + 1
+                                  for k, d in zip(kernel_shape, dilations)]
+        pads = get_attribute(node, 'pads')
+        if pads is None:
+            pads = [0] * (2 * rank)
+            auto_pad = get_attribute(node, 'auto_pad',
+                                     b'NOTSET').decode('utf-8')
+            if auto_pad != 'VALID' and auto_pad != 'NOTSET':
+                try:
+                    residual = [
+                        sympy.Mod(d, s)
+                        for d, s in zip(sympy_shape[-rank:], strides)
+                    ]
+                    total_pads = [
+                        max(0, (k - s) if r == 0 else (k - r))
+                        for k, s, r in zip(effective_kernel_shape, strides,
+                                           residual)
+                    ]
+                except TypeError:  # sympy may throw TypeError: cannot determine truth value of Relational
+                    total_pads = [
+                        max(0, (k - s))
+                        for k, s in zip(effective_kernel_shape, strides)
+                    ]  # assuming no residual if sympy throws error
+            elif auto_pad == 'VALID':
+                total_pads = []
+            else:
+                total_pads = [0] * rank
+        else:
+            assert len(pads) == 2 * rank
+            total_pads = [p1 + p2 for p1, p2 in zip(pads[:rank], pads[rank:])]
+
+        ceil_mode = get_attribute(node, 'ceil_mode', 0)
+        for i in range(rank):
+            effective_input_size = sympy_shape[-rank + i]
+            if len(total_pads) > 0:
+                effective_input_size = effective_input_size + total_pads[i]
+            if ceil_mode:
+                strided_kernel_positions = sympy.ceiling(
+                    (effective_input_size - effective_kernel_shape[i]) /
+                    strides[i])
+            else:
+                strided_kernel_positions = (
+                    effective_input_size - effective_kernel_shape[i]
+                ) // strides[i]
+            sympy_shape[-rank + i] = strided_kernel_positions + 1
+        return sympy_shape
+
+    def _check_merged_dims(self, dims, allow_broadcast=True):
+        if allow_broadcast:
+            dims = [d for d in dims if not (is_literal(d) and int(d) <= 1)]
+        if not all([d == dims[0] for d in dims]):
+            self._add_suggested_merge(dims, apply=True)
+
+    def _compute_matmul_shape(self, node, output_dtype=None):
+        lhs_shape = self._get_shape(node, 0)
+        rhs_shape = self._get_shape(node, 1)
+        lhs_rank = len(lhs_shape)
+        rhs_rank = len(rhs_shape)
+        lhs_reduce_dim = 0
+        rhs_reduce_dim = 0
+        assert lhs_rank > 0 and rhs_rank > 0
+        if lhs_rank == 1 and rhs_rank == 1:
+            new_shape = []
+        elif lhs_rank == 1:
+            rhs_reduce_dim = -2
+            new_shape = rhs_shape[:rhs_reduce_dim] + [rhs_shape[-1]]
+        elif rhs_rank == 1:
+            lhs_reduce_dim = -1
+            new_shape = lhs_shape[:lhs_reduce_dim]
+        else:
+            lhs_reduce_dim = -1
+            rhs_reduce_dim = -2
+            new_shape = self._broadcast_shapes(
+                lhs_shape[:-2], rhs_shape[:-2]) + [lhs_shape[-2]
+                                                   ] + [rhs_shape[-1]]
+        # merge reduce dim
+        self._check_merged_dims(
+            [lhs_shape[lhs_reduce_dim], rhs_shape[rhs_reduce_dim]],
+            allow_broadcast=False)
+        if output_dtype is None:
+            # infer output_dtype from input type when not specified
+            output_dtype = self.known_vi_[node.input[
+                0]].type.tensor_type.elem_type
+        vi = self.known_vi_[node.output[0]]
+        vi.CopyFrom(
+            helper.make_tensor_value_info(node.output[0], output_dtype,
+                                          new_shape))
+
+    def _infer_ArrayFeatureExtractor(self, node):
+        data_shape = self._get_shape(node, 0)
+        indices_shape = self._get_shape(node, 1)
+        vi = self.known_vi_[node.output[0]]
+        vi.CopyFrom(
+            helper.make_tensor_value_info(node.output[0], self.known_vi_[
+                node.input[0]].type.tensor_type.elem_type, data_shape[:-1] +
+                                          indices_shape))
+
+    def _infer_symbolic_compute_ops(self, node):
+        funcs = {
+            'Add': lambda l: l[0] + l[1],
+            'Div': lambda l: l[0] // l[1],  # integer div in sympy
+            'Equal': lambda l: l[0] == l[1],
+            'Max':
+            lambda l: l[1] if is_literal(l[0]) and int(l[0]) < -self.int_max_ else (l[0] if is_literal(l[1]) and int(l[1]) < -self.int_max_ else sympy.Max(l[0], l[1])),
+            'Min':
+            lambda l: l[1] if is_literal(l[0]) and int(l[0]) > self.int_max_ else (l[0] if is_literal(l[1]) and int(l[1]) > self.int_max_ else sympy.Min(l[0], l[1])),
+            'Mul': lambda l: l[0] * l[1],
+            'Sub': lambda l: l[0] - l[1],
+            'Where': lambda l: l[1] if l[0] else l[2]
+        }
+        assert node.op_type in funcs
+        self._compute_on_sympy_data(node, funcs[node.op_type])
+
+    def _infer_Cast(self, node):
+        self._pass_on_sympy_data(node)
+
+    def _infer_CategoryMapper(self, node):
+        input_type = self.known_vi_[node.input[0]].type.tensor_type.elem_type
+        if input_type == onnx.TensorProto.STRING:
+            output_type = onnx.TensorProto.INT64
+        else:
+            output_type = onnx.TensorProto.STRING
+        vi = self.known_vi_[node.output[0]]
+        vi.CopyFrom(
+            helper.make_tensor_value_info(node.output[0], output_type,
+                                          self._get_shape(node, 0)))
+
+    def _infer_Transpose(self, node):
+        input_shape = self._get_shape(node, 0)
+        perm = get_attribute(node, 'perm')
+        output_shape = np.array(input_shape)[perm].tolist()
+        vi = self.known_vi_[node.output[0]]
+        vi.CopyFrom(
+            helper.make_tensor_value_info(node.output[0], self.known_vi_[
+                node.input[0]].type.tensor_type.elem_type, output_shape))
+
+    def _infer_Compress(self, node):
+        input_shape = self._get_shape(node, 0)
+        # create a new symbolic dimension for Compress output
+        compress_len = self._new_symbolic_dim_from_output(node)
+        axis = get_attribute(node, 'axis')
+        if axis == None:
+            # when axis is not specified, input is flattened before compress so output is 1D
+            output_shape = [compress_len]
+        else:
+            output_shape = input_shape
+            output_shape[handle_negative_axis(axis, len(
+                input_shape))] = compress_len
+        vi = self.known_vi_[node.output[0]]
+        vi.CopyFrom(
+            helper.make_tensor_value_info(node.output[0], self.known_vi_[
+                node.input[0]].type.tensor_type.elem_type, output_shape))
+
+    def _infer_Concat(self, node):
+        if any([i in self.sympy_data_ for i in node.input]):
+            values = self._get_int_values(node)
+            if all([v is not None for v in values]):
+                assert 0 == get_attribute(node, 'axis')
+                self.sympy_data_[node.output[0]] = []
+                for i in range(len(node.input)):
+                    value = values[i]
+                    if type(value) == list:
+                        self.sympy_data_[node.output[0]].extend(value)
+                    else:
+                        self.sympy_data_[node.output[0]].append(value)
+
+        sympy_shape = self._get_sympy_shape(node, 0)
+        axis = handle_negative_axis(
+            get_attribute(node, 'axis'), len(sympy_shape))
+        for i_idx in range(1, len(node.input)):
+            input_shape = self._get_sympy_shape(node, i_idx)
+            if input_shape:
+                sympy_shape[axis] = sympy_shape[axis] + input_shape[axis]
+        self._update_computed_dims(sympy_shape)
+        # merge symbolic dims for non-concat axes
+        for d in range(len(sympy_shape)):
+            if d == axis:
+                continue
+            dims = [
+                self._get_shape(node, i_idx)[d]
+                for i_idx in range(len(node.input))
+                if self._get_shape(node, i_idx)
+            ]
+            if all([d == dims[0] for d in dims]):
+                continue
+            merged = self._merge_symbols(dims)
+            if type(merged) == str:
+                sympy_shape[d] = self.symbolic_dims_[merged] if merged else None
+            else:
+                sympy_shape[d] = merged
+        vi = self.known_vi_[node.output[0]]
+        vi.CopyFrom(
+            helper.make_tensor_value_info(
+                node.output[0], self.known_vi_[node.input[0]].type.tensor_type.
+                elem_type, get_shape_from_sympy_shape(sympy_shape)))
+
+    def _infer_Conv(self, node):
+        sympy_shape = self._compute_conv_pool_shape(node)
+        self._update_computed_dims(sympy_shape)
+        vi = self.known_vi_[node.output[0]]
+        vi.CopyFrom(
+            helper.make_tensor_value_info(
+                node.output[0], vi.type.tensor_type.elem_type,
+                get_shape_from_sympy_shape(sympy_shape)))
+
+    def _infer_ConstantOfShape(self, node):
+        sympy_shape = self._get_int_values(node)[0]
+        vi = self.known_vi_[node.output[0]]
+        if sympy_shape is not None:
+            if type(sympy_shape) != list:
+                sympy_shape = [sympy_shape]
+            self._update_computed_dims(sympy_shape)
+            # update sympy data if output type is int, and shape is known
+            if vi.type.tensor_type.elem_type == onnx.TensorProto.INT64 and all(
+                [is_literal(x) for x in sympy_shape]):
+                self.sympy_data_[node.output[0]] = np.ones(
+                    [int(x) for x in sympy_shape],
+                    dtype=np.int64) * numpy_helper.to_array(
+                        get_attribute(node, 'value', 0))
+        else:
+            # create new dynamic shape
+            sympy_shape = self._new_symbolic_shape(
+                self._get_shape_rank(node, 0), node)
+
+        vi.CopyFrom(
+            helper.make_tensor_value_info(
+                node.output[0], vi.type.tensor_type.elem_type,
+                get_shape_from_sympy_shape(sympy_shape)))
+
+    def _infer_Expand(self, node):
+        expand_to_shape = self._try_get_value(node, 1)
+        if expand_to_shape is not None:
+            # new_shape's dim can come from shape value
+            self._update_computed_dims(expand_to_shape)
+            shape = self._get_shape(node, 0)
+            new_shape = self._broadcast_shapes(
+                shape, get_shape_from_sympy_shape(expand_to_shape))
+            vi = self.known_vi_[node.output[0]]
+            vi.CopyFrom(
+                helper.make_tensor_value_info(node.output[0], self.known_vi_[
+                    node.input[0]].type.tensor_type.elem_type, new_shape))
+
+    def _infer_Gather(self, node):
+        data_shape = self._get_shape(node, 0)
+        axis = handle_negative_axis(
+            get_attribute(node, 'axis', 0), len(data_shape))
+        indices_shape = self._get_shape(node, 1)
+        #if indices_shape == []:
+        #    value = self._get_initializer_value(node, 1)
+        #    if isinstance(value.tolist(), int):
+        #        indices_shape = [1]
+        new_shape = data_shape[:axis] + indices_shape + data_shape[axis + 1:]
+        #print(new_shape)
+        vi = self.known_vi_[node.output[0]]
+        vi.CopyFrom(
+            helper.make_tensor_value_info(node.output[
+                0], vi.type.tensor_type.elem_type, new_shape))
+        if node.input[0] in self.sympy_data_:
+            assert 0 == get_attribute(node, 'axis',
+                                      0)  # only handle 1D sympy compute
+            idx = self._get_value(node, 1)
+            data = self.sympy_data_[node.input[0]]
+            if type(data) == list:
+                if type(idx) == np.ndarray and len(idx.shape) == 1:
+                    self.sympy_data_[node.output[0]] = [
+                        data[int(i)] for i in idx
+                    ]
+                else:
+                    self.sympy_data_[node.output[0]] = data[int(idx)]
+            else:
+                assert idx == 0
+                self.sympy_data_[node.output[0]] = data
+
+    def _infer_GatherElements(self, node):
+        indices_shape = self._get_shape(node, 1)
+        vi = self.known_vi_[node.output[0]]
+        vi.CopyFrom(
+            helper.make_tensor_value_info(node.output[0], self.known_vi_[
+                node.input[0]].type.tensor_type.elem_type, indices_shape))
+
+    def _infer_GatherND(self, node):
+        data_shape = self._get_shape(node, 0)
+        data_rank = len(data_shape)
+        indices_shape = self._get_shape(node, 1)
+        indices_rank = len(indices_shape)
+        last_index_dimension = indices_shape[-1]
+        assert is_literal(
+            last_index_dimension) and last_index_dimension <= data_rank
+        new_shape = indices_shape[:-1] + data_shape[last_index_dimension:]
+        vi = self.known_vi_[node.output[0]]
+        vi.CopyFrom(
+            helper.make_tensor_value_info(node.output[0], self.known_vi_[
+                node.input[0]].type.tensor_type.elem_type, new_shape))
+
+    def _infer_If(self, node):
+        # special case for constant condition, in case there are mismatching shape from the non-executed branch
+        subgraphs = [
+            get_attribute(node, 'then_branch'),
+            get_attribute(node, 'else_branch')
+        ]
+        cond = self._try_get_value(node, 0)
+        if cond is not None:
+            if cond > 0:
+                subgraphs[1].CopyFrom(subgraphs[0])
+            else:
+                subgraphs[0].CopyFrom(subgraphs[1])
+
+        for i_sub, subgraph in enumerate(subgraphs):
+            subgraph_infer = self._onnx_infer_subgraph(
+                node, subgraph, use_node_input=False)
+            for i_out in range(len(node.output)):
+                vi = self.known_vi_[node.output[i_out]]
+                if i_sub == 0:
+                    vi.CopyFrom(subgraph.output[i_out])
+                    vi.name = node.output[i_out]
+                else:
+                    assert all([
+                        d1 == d2
+                        for d1, d2 in zip(vi.type.tensor_type.shape.dim,
+                                          subgraph.output[
+                                              i_out].type.tensor_type.shape.dim)
+                    ])
+                # pass on sympy data from subgraph, if cond is constant
+                if cond is not None and i_sub == (0 if cond > 0 else 1):
+                    if subgraph.output[
+                            i_out].name in subgraph_infer.sympy_data_:
+                        self.sympy_data_[vi.name] = subgraph_infer.sympy_data_[
+                            subgraph.output[i_out].name]
+
+    def _infer_Loop(self, node):
+        subgraph = get_attribute(node, 'body')
+        assert len(subgraph.input) == len(node.input)
+        for i, si in enumerate(subgraph.input):
+            subgraph_name = si.name
+            si.CopyFrom(self.known_vi_[node.input[i]])
+            si.name = subgraph_name
+        self._onnx_infer_subgraph(node, subgraph)
+        # create a new symbolic dimension for iteration dependent dimension
+        loop_iter_dim = self._new_symbolic_dim_from_output(node)
+        num_loop_carried = len(node.input) - 2
+        for i in range(len(node.output)):
+            vi = self.known_vi_[node.output[i]]
+            vi.CopyFrom(
+                subgraph.output[i + 1]
+            )  # first subgraph output is condition, not in node output
+            if i >= num_loop_carried:
+                subgraph_vi_dim = subgraph.output[i +
+                                                  1].type.tensor_type.shape.dim
+                vi.type.tensor_type.shape.ClearField('dim')
+                vi_dim = vi.type.tensor_type.shape.dim
+                vi_dim.add().dim_param = loop_iter_dim
+                vi_dim.extend(list(subgraph_vi_dim))
+            vi.name = node.output[i]
+
+    def _infer_MatMul(self, node):
+        self._compute_matmul_shape(node)
+
+    def _infer_MatMulInteger(self, node):
+        self._compute_matmul_shape(node, onnx.TensorProto.INT32)
+
+    def _infer_NonMaxSuppression(self, node):
+        selected = self._new_symbolic_dim_from_output(node)
+        vi = self.known_vi_[node.output[0]]
+        vi.CopyFrom(
+            helper.make_tensor_value_info(node.output[
+                0], onnx.TensorProto.INT64, [selected, 3]))
+
+    def _infer_NonZero(self, node):
+        input_rank = self._get_shape_rank(node, 0)
+        # create a new symbolic dimension for NonZero output
+        nz_len = self._new_symbolic_dim_from_output(node, 0, 1)
+        vi = self.known_vi_[node.output[0]]
+        vi.CopyFrom(
+            helper.make_tensor_value_info(node.output[
+                0], vi.type.tensor_type.elem_type, [input_rank, nz_len]))
+
+    def _infer_OneHot(self, node):
+        shape = self._get_shape(node, 0)
+        axis = get_attribute(node, 'axis', -1)
+        axis = handle_negative_axis(axis, len(shape) + 1)
+        new_shape = shape[:axis] + [self._new_symbolic_dim_from_output(node)
+                                    ] + shape[axis:]
+        vi = self.known_vi_[node.output[0]]
+        vi.CopyFrom(
+            helper.make_tensor_value_info(node.output[0], self.known_vi_[
+                node.input[2]].type.tensor_type.elem_type, new_shape))
+
+    def _infer_Pad(self, node):
+        if get_opset(self.out_mp_) <= 10:
+            pads = get_attribute(node, 'pads')
+        else:
+            pads = self._try_get_value(node, 1)
+
+        vi = self.known_vi_[node.output[0]]
+        output_shape = get_shape_from_type_proto(vi.type)
+        if len(output_shape) == 0 or None in output_shape:
+            sympy_shape = self._get_sympy_shape(node, 0)
+            rank = len(sympy_shape)
+            if pads is not None:
+                assert len(pads) == 2 * rank
+                new_sympy_shape = [
+                    d + pad_up + pad_down
+                    for d, pad_up, pad_down in zip(sympy_shape, pads[:rank],
+                                                   pads[rank:])
+                ]
+                self._update_computed_dims(new_sympy_shape)
+            else:
+                # dynamic pads, create new symbolic dimensions
+                new_sympy_shape = self._new_symbolic_shape(rank, node)
+            output_tp = self.known_vi_[node.input[0]].type.tensor_type.elem_type
+            vi.CopyFrom(
+                helper.make_tensor_value_info(node.output[
+                    0], output_tp, get_shape_from_sympy_shape(new_sympy_shape)))
+
+    def _infer_Pool(self, node):
+        sympy_shape = self._compute_conv_pool_shape(node)
+        self._update_computed_dims(sympy_shape)
+        for o in node.output:
+            if not o:
+                continue
+            vi = self.known_vi_[o]
+            vi.CopyFrom(
+                helper.make_tensor_value_info(o, vi.type.tensor_type.elem_type,
+                                              get_shape_from_sympy_shape(
+                                                  sympy_shape)))
+
+    def _infer_Range(self, node):
+        vi = self.known_vi_[node.output[0]]
+        input_data = self._get_int_values(node)
+        if all([i is not None for i in input_data]):
+            start = as_scalar(input_data[0])
+            limit = as_scalar(input_data[1])
+            delta = as_scalar(input_data[2])
+            new_sympy_shape = [
+                sympy.Max(sympy.ceiling((limit - start) / delta), 0)
+            ]
+        else:
+            new_dim = self._new_symbolic_dim_from_output(node)
+            new_sympy_shape = [self.symbolic_dims_[new_dim]]
+        self._update_computed_dims(new_sympy_shape)
+        vi.CopyFrom(
+            helper.make_tensor_value_info(
+                node.output[0], self.known_vi_[node.input[0]].type.tensor_type.
+                elem_type, get_shape_from_sympy_shape(new_sympy_shape)))
+
+    def _infer_ReduceProd(self, node):
+        axes = get_attribute(node, 'axes')
+        keep_dims = get_attribute(node, 'keepdims')
+        if keep_dims == 0 and axes == [0]:
+            data = self._get_int_values(node)[0]
+            if data is not None:
+                self.sympy_data_[node.output[0]] = sympy_reduce_product(data)
+
+    def _infer_Reshape(self, node):
+        shape_value = self._try_get_value(node, 1)
+        vi = self.known_vi_[node.output[0]]
+        if shape_value is None:
+            shape_shape = self._get_shape(node, 1)
+            assert len(shape_shape) == 1
+            shape_rank = shape_shape[0]
+            assert is_literal(shape_rank)
+            vi.CopyFrom(
+                helper.make_tensor_value_info(
+                    node.output[0], vi.type.tensor_type.elem_type,
+                    get_shape_from_sympy_shape(
+                        self._new_symbolic_shape(shape_rank, node))))
+        else:
+            input_shape = self._get_shape(node, 0)
+            input_sympy_shape = self._get_sympy_shape(node, 0)
+            total = int(1)
+            for d in input_sympy_shape:
+                total = total * d
+            new_sympy_shape = []
+            deferred_dim_idx = -1
+            non_deferred_size = int(1)
+            for i, d in enumerate(shape_value):
+                if type(d) == sympy.Symbol:
+                    new_sympy_shape.append(d)
+                elif d == 0:
+                    new_sympy_shape.append(input_sympy_shape[i])
+                    non_deferred_size = non_deferred_size * input_sympy_shape[i]
+                else:
+                    new_sympy_shape.append(d)
+                if d == -1:
+                    deferred_dim_idx = i
+                elif d != 0:
+                    non_deferred_size = non_deferred_size * d
+
+            assert new_sympy_shape.count(-1) < 2
+            if -1 in new_sympy_shape:
+                new_dim = total // non_deferred_size
+                new_sympy_shape[deferred_dim_idx] = new_dim
+                self._update_computed_dims(new_sympy_shape)
+
+            vi.CopyFrom(
+                helper.make_tensor_value_info(
+                    node.output[0], vi.type.tensor_type.elem_type,
+                    get_shape_from_sympy_shape(new_sympy_shape)))
+
+        self._pass_on_sympy_data(node)
+
+    def _infer_Resize(self, node):
+        vi = self.known_vi_[node.output[0]]
+        input_sympy_shape = self._get_sympy_shape(node, 0)
+        if get_opset(self.out_mp_) <= 10:
+            scales = self._try_get_value(node, 1)
+            if scales is not None:
+                new_sympy_shape = [
+                    sympy.simplify(sympy.floor(d * s))
+                    for d, s in zip(input_sympy_shape, scales)
+                ]
+                self._update_computed_dims(new_sympy_shape)
+                vi.CopyFrom(
+                    helper.make_tensor_value_info(
+                        node.output[0], self.known_vi_[node.input[
+                            0]].type.tensor_type.elem_type,
+                        get_shape_from_sympy_shape(new_sympy_shape)))
+        else:
+            roi = self._try_get_value(node, 1)
+            scales = self._try_get_value(node, 2)
+            sizes = self._try_get_value(node, 3)
+            if sizes is not None:
+                new_sympy_shape = [
+                    sympy.simplify(sympy.floor(s)) for s in sizes
+                ]
+                self._update_computed_dims(new_sympy_shape)
+            elif roi is not None and scales is not None:
+                rank = len(scales)
+                assert len(roi) == 2 * rank
+                roi_start = list(roi)[:rank]
+                roi_end = list(roi)[rank:]
+                scales = list(scales)
+                new_sympy_shape = [
+                    sympy.simplify(sympy.floor(d * (end - start) * scale))
+                    for d, start, end, scale in zip(input_sympy_shape,
+                                                    roi_start, roi_end, scales)
+                ]
+                self._update_computed_dims(new_sympy_shape)
+            else:
+                new_sympy_shape = self._new_symbolic_shape(
+                    self._get_shape_rank(node, 0), node)
+
+            vi.CopyFrom(
+                helper.make_tensor_value_info(node.output[0], self.known_vi_[
+                    node.input[0]].type.tensor_type.elem_type,
+                                              get_shape_from_sympy_shape(
+                                                  new_sympy_shape)))
+
+    def _infer_Scan(self, node):
+        subgraph = get_attribute(node, 'body')
+        num_scan_inputs = get_attribute(node, 'num_scan_inputs')
+        scan_input_axes = get_attribute(node, 'scan_input_axes',
+                                        [0] * num_scan_inputs)
+        num_scan_states = len(node.input) - num_scan_inputs
+        scan_input_axes = [
+            handle_negative_axis(
+                ax, self._get_shape_rank(node, i + num_scan_states))
+            for i, ax in enumerate(scan_input_axes)
+        ]
+        # We may have cases where the subgraph has optionial inputs that appear in both subgraph's input and initializer,
+        # but not in the node's input. In such cases, the input model might be invalid, but let's skip those optional inputs.
+        assert len(subgraph.input) >= len(node.input)
+        subgraph_inputs = subgraph.input[:len(node.input)]
+        for i, si in enumerate(subgraph_inputs):
+            subgraph_name = si.name
+            si.CopyFrom(self.known_vi_[node.input[i]])
+            if i >= num_scan_states:
+                scan_input_dim = si.type.tensor_type.shape.dim
+                scan_input_dim.remove(scan_input_dim[scan_input_axes[
+                    i - num_scan_states]])
+            si.name = subgraph_name
+        self._onnx_infer_subgraph(node, subgraph)
+        num_scan_outputs = len(node.output) - num_scan_states
+        scan_output_axes = get_attribute(node, 'scan_output_axes',
+                                         [0] * num_scan_outputs)
+        scan_input_dim = get_shape_from_type_proto(self.known_vi_[node.input[
+            -1]].type)[scan_input_axes[-1]]
+        for i, o in enumerate(node.output):
+            vi = self.known_vi_[o]
+            if i >= num_scan_states:
+                shape = get_shape_from_type_proto(subgraph.output[i].type)
+                new_dim = handle_negative_axis(
+                    scan_output_axes[i - num_scan_states], len(shape) + 1)
+                shape = shape[:new_dim] + [scan_input_dim] + shape[new_dim:]
+                vi.CopyFrom(
+                    helper.make_tensor_value_info(o, subgraph.output[
+                        i].type.tensor_type.elem_type, shape))
+            else:
+                vi.CopyFrom(subgraph.output[i])
+            vi.name = o
+
+    def _infer_ScatterElements(self, node):
+        data_shape = self._get_shape(node, 0)
+        vi = self.known_vi_[node.output[0]]
+        vi.CopyFrom(
+            helper.make_tensor_value_info(node.output[0], self.known_vi_[
+                node.input[0]].type.tensor_type.elem_type, data_shape))
+
+    def _infer_Shape(self, node):
+        self.sympy_data_[node.output[0]] = self._get_sympy_shape(node, 0)
+
+    def _infer_Size(self, node):
+        sympy_shape = self._get_sympy_shape(node, 0)
+        self.sympy_data_[node.output[0]] = sympy_reduce_product(sympy_shape)
+        self.known_vi_[node.output[0]].CopyFrom(
+            helper.make_tensor_value_info(node.output[0],
+                                          onnx.TensorProto.INT64, []))
+
+    def _infer_Slice(self, node):
+        if get_opset(self.out_mp_) <= 9:
+            axes = get_attribute(node, 'axes')
+            starts = get_attribute(node, 'starts')
+            ends = get_attribute(node, 'ends')
+            steps = [1] * len(axes)
+        else:
+            starts = as_list(self._try_get_value(node, 1), keep_none=True)
+            ends = as_list(self._try_get_value(node, 2), keep_none=True)
+            axes = self._try_get_value(node, 3)
+            steps = self._try_get_value(node, 4)
+            if axes is None and not (starts is None and ends is None):
+                axes = list(
+                    range(0, len(starts if starts is not None else ends)))
+            if steps is None and not (starts is None and ends is None):
+                steps = [1] * len(starts if starts is not None else ends)
+            axes = as_list(axes, keep_none=True)
+            steps = as_list(steps, keep_none=True)
+
+        new_sympy_shape = self._get_sympy_shape(node, 0)
+        if starts is None or ends is None:
+            if axes is None:
+                for i in range(len(new_sympy_shape)):
+                    new_sympy_shape[i] = self._new_symbolic_dim_from_output(
+                        node, 0, i)
+            else:
+                new_sympy_shape = get_shape_from_sympy_shape(new_sympy_shape)
+                for i in axes:
+                    new_sympy_shape[i] = self._new_symbolic_dim_from_output(
+                        node, 0, i)
+        else:
+            for i, s, e, t in zip(axes, starts, ends, steps):
+                idx = handle_negative_axis(i, len(new_sympy_shape))
+                if is_literal(e):
+                    if e >= self.int_max_:
+                        e = new_sympy_shape[i]
+                    elif e <= -self.int_max_:
+                        e = 0 if s > 0 else -1
+                    elif is_literal(new_sympy_shape[i]):
+                        if e < 0:
+                            e = e + new_sympy_shape[i]
+                        e = min(e, new_sympy_shape[i])
+                    else:
+                        if e > 0:
+                            e = sympy.Min(
+                                e, new_sympy_shape[i]
+                            ) if e > 1 else e  #special case for slicing first to make computation easier
+                        else:
+                            e = new_sympy_shape[i] + e
+                else:
+                    if is_literal(new_sympy_shape[i]):
+                        e = sympy.Min(e, new_sympy_shape[i])
+                    else:
+                        try:
+                            if e >= new_sympy_shape[i]:
+                                e = new_sympy_shape[i]
+                        except Exception:
+                            print(
+                                'Unable to determine if {} <= {}, treat as equal'
+                                .format(e, new_sympy_shape[i]))
+                            e = new_sympy_shape[i]
+
+                if is_literal(s) and int(s) < 0:
+                    s = new_sympy_shape[i] + s
+
+                new_sympy_shape[idx] = (e - s + t + (-1 if t > 0 else 1)) // t
+
+            self._update_computed_dims(new_sympy_shape)
+
+        vi = self.known_vi_[node.output[0]]
+        vi.CopyFrom(
+            helper.make_tensor_value_info(
+                node.output[0], vi.type.tensor_type.elem_type,
+                get_shape_from_sympy_shape(new_sympy_shape)))
+
+        # handle sympy_data if needed, for slice in shape computation
+        if node.input[0] in self.sympy_data_:
+            assert [0] == axes
+            assert len(starts) == 1
+            assert len(ends) == 1
+            self.sympy_data_[node.output[0]] = self.sympy_data_[node.input[0]][
+                starts[0]:ends[0]]
+
+    def _infer_Split(self, node):
+        input_sympy_shape = self._get_sympy_shape(node, 0)
+        axis = handle_negative_axis(
+            get_attribute(node, 'axis', 0), len(input_sympy_shape))
+        split = get_attribute(node, 'split')
+        if not split:
+            num_outputs = len(node.output)
+            split = [input_sympy_shape[axis] /
+                     sympy.Integer(num_outputs)] * num_outputs
+            self._update_computed_dims(split)
+        else:
+            split = [sympy.Integer(s) for s in split]
+
+        for i_o in range(len(split)):
+            vi = self.known_vi_[node.output[i_o]]
+            vi.CopyFrom(
+                helper.make_tensor_value_info(
+                    node.output[i_o], self.known_vi_[node.input[
+                        0]].type.tensor_type.elem_type,
+                    get_shape_from_sympy_shape(input_sympy_shape[:axis] + [
+                        split[i_o]
+                    ] + input_sympy_shape[axis + 1:])))
+            self.known_vi_[vi.name] = vi
+
+    def _infer_Squeeze(self, node):
+        self._pass_on_sympy_data(node)
+
+    def _infer_Tile(self, node):
+        repeats_value = self._get_value(node, 1)
+        input_sympy_shape = self._get_sympy_shape(node, 0)
+        new_sympy_shape = []
+        for i, d in enumerate(input_sympy_shape):
+            new_dim = d * repeats_value[i]
+            new_sympy_shape.append(new_dim)
+        self._update_computed_dims(new_sympy_shape)
+        vi = self.known_vi_[node.output[0]]
+        vi.CopyFrom(
+            helper.make_tensor_value_info(
+                node.output[0], vi.type.tensor_type.elem_type,
+                get_shape_from_sympy_shape(new_sympy_shape)))
+
+    def _infer_TopK(self, node):
+        rank = self._get_shape_rank(node, 0)
+        axis = handle_negative_axis(get_attribute(node, 'axis', -1), rank)
+        new_shape = self._get_shape(node, 0)
+
+        if get_opset(self.out_mp_) <= 9:
+            k = get_attribute(node, 'k')
+        else:
+            k = self._get_int_values(node)[1]
+
+        if k == None:
+            k = self._new_symbolic_dim_from_output(node)
+        else:
+            k = as_scalar(k)
+
+        if type(k) in [int, str]:
+            new_shape[axis] = k
+        else:
+            new_sympy_shape = self._get_sympy_shape(node, 0)
+            new_sympy_shape[axis] = k
+            self._update_computed_dims(
+                new_sympy_shape
+            )  # note that TopK dim could be computed in sympy_data, so need to update computed_dims when it enters shape
+            new_shape = get_shape_from_sympy_shape(new_sympy_shape)
+
+        for i_o in range(len(node.output)):
+            vi = self.known_vi_[node.output[i_o]]
+            vi.CopyFrom(
+                helper.make_tensor_value_info(node.output[
+                    i_o], vi.type.tensor_type.elem_type, new_shape))
+
+    def _infer_Unsqueeze(self, node):
+        self._pass_on_sympy_data(node)
+
+    def _infer_ZipMap(self, node):
+        map_key_type = None
+        if get_attribute(node, 'classlabels_int64s') is not None:
+            map_key_type = onnx.TensorProto.INT64
+        elif get_attribute(node, 'classlabels_strings') is not None:
+            map_key_type = onnx.TensorProto.STRING
+
+        assert map_key_type is not None
+        new_vi = onnx.ValueInfoProto()
+        new_vi.name = node.output[0]
+        new_vi.type.sequence_type.elem_type.map_type.value_type.tensor_type.elem_type = onnx.TensorProto.FLOAT
+        new_vi.type.sequence_type.elem_type.map_type.key_type = map_key_type
+        vi = self.known_vi_[node.output[0]]
+        vi.CopyFrom(new_vi)
+
+    def _infer_impl(self, in_mp, start_sympy_data={}):
+        self.sympy_data_ = start_sympy_data
+        self.out_mp_.graph.ClearField('value_info')
+        self._apply_suggested_merge(graph_input_only=True)
+        self.input_symbols_ = set()
+        for i in self.out_mp_.graph.input:
+            input_dims = i.type.tensor_type.shape.dim
+            for i_dim in range(len(input_dims)):
+                if get_dim_from_type_proto(input_dims[i_dim]) is None:
+                    # some models use None for symbolic dim in input, replace it with a string
+                    input_dims[i_dim].dim_param = self._new_symbolic_dim(i.name,
+                                                                         i_dim)
+            self.input_symbols_.update([
+                d for d in get_shape_from_type_proto(i.type) if type(d) == str
+            ])
+
+        for s in self.input_symbols_:
+            if s in self.suggested_merge_:
+                s_merge = self.suggested_merge_[s]
+                assert s_merge in self.symbolic_dims_
+                self.symbolic_dims_[s] = self.symbolic_dims_[s_merge]
+            else:
+                self.symbolic_dims_[s] = sympy.Symbol(s, integer=True)
+
+        # create a temporary ModelProto for single node inference
+        # note that we remove initializer to have faster inference
+        # for tensor ops like Reshape/Tile/Expand that read initializer, we need to do sympy computation based inference anyways
+        self.tmp_mp_ = onnx.ModelProto()
+        self.tmp_mp_.CopyFrom(self.out_mp_)
+        self.tmp_mp_.graph.ClearField('initializer')
+
+        for node in self.out_mp_.graph.node:
+            assert all([i in self.known_vi_ for i in node.input if i])
+            self._onnx_infer_single_node(node)
+            if node.op_type in self.dispatcher_:
+                self.dispatcher_[node.op_type](node)
+            if self.verbose_ > 2:
+                print(node.op_type + ': ' + node.name)
+                for i, name in enumerate(node.input):
+                    print('  Input {}: {} {}€5€5€5€5€5'.format(
+                        i, name, 'initializer'
+                        if name in self.initializers_ else ''))
+
+            # onnx automatically merge dims with value, i.e. Mul(['aaa', 'bbb'], [1000, 1]) -> [1000, 'bbb']
+            # symbolic shape inference needs to apply merge of 'aaa' -> 1000 in this case
+            if node.op_type in [
+                    'Add', 'Sub', 'Mul', 'Div', 'MatMul', 'MatMulInteger',
+                    'MatMulInteger16', 'Where', 'Sum'
+            ]:
+                vi = self.known_vi_[node.output[0]]
+                out_rank = len(get_shape_from_type_proto(vi.type))
+                in_shapes = [
+                    self._get_shape(node, i) for i in range(len(node.input))
+                ]
+                for d in range(out_rank - (2 if node.op_type in [
+                        'MatMul', 'MatMulInteger', 'MatMulInteger16'
+                ] else 0)):
+                    in_dims = [
+                        s[len(s) - out_rank + d] for s in in_shapes
+                        if len(s) + d >= out_rank
+                    ]
+                    if len(in_dims) > 1:
+                        self._check_merged_dims(in_dims, allow_broadcast=True)
+            for i_o in range(len(node.output)):
+                vi = self.known_vi_[node.output[i_o]]
+                out_type = vi.type
+                out_type_kind = out_type.WhichOneof('value')
+                # only TensorProto and SparseTensorProto have shape
+                if out_type_kind != 'tensor_type' and out_type_kind != 'sparse_tensor_type':
+                    continue
+                out_shape = get_shape_from_type_proto(vi.type)
+                out_type_undefined = out_type.tensor_type.elem_type == onnx.TensorProto.UNDEFINED
+                if self.verbose_ > 2:
+                    print('  {}: {} {}'.format(node.output[
+                        i_o], str(out_shape), vi.type.tensor_type.elem_type))
+                    if node.output[i_o] in self.sympy_data_:
+                        print('  Sympy Data: ' + str(self.sympy_data_[
+                            node.output[i_o]]))
+
+                if None in out_shape or out_type_undefined:
+                    if self.auto_merge_:
+                        if node.op_type in [
+                                'Add', 'Sub', 'Mul', 'Div', 'MatMul',
+                                'MatMulInteger', 'MatMulInteger16', 'Concat',
+                                'Where', 'Sum'
+                        ]:
+                            shapes = [
+                                self._get_shape(node, i)
+                                for i in range(len(node.input))
+                            ]
+                            if node.op_type in [
+                                    'MatMul', 'MatMulInteger', 'MatMulInteger16'
+                            ]:
+                                # only support auto merge for MatMul for dim < rank-2 when rank > 2
+                                assert len(shapes[0]) > 2 and dim_idx[0] < len(
+                                    shapes[0]) - 2
+                                assert len(shapes[1]) > 2 and dim_idx[1] < len(
+                                    shapes[1]) - 2
+                        elif node.op_type == 'Expand':
+                            # auto merge for cases like Expand([min(batch, 1), min(seq, 512)], [batch, seq])
+                            shapes = [
+                                self._get_shape(node, 0),
+                                self._get_value(node, 1)
+                            ]
+                        else:
+                            shapes = []
+
+                        if shapes:
+                            for idx in range(len(out_shape)):
+                                if out_shape[idx] is not None:
+                                    continue
+                                dim_idx = [
+                                    len(s) - len(out_shape) + idx
+                                    for s in shapes
+                                ]
+                                assert all([d >= 0 for d in dim_idx])
+                                self._add_suggested_merge([
+                                    s[i] if is_literal(s[i]) else str(s[i])
+                                    for s, i in zip(shapes, dim_idx)
+                                ])
+                            self.run_ = True
+                        else:
+                            self.run_ = False
+                    else:
+                        self.run_ = False
+
+                    # create new dynamic dims for ops not handled by symbolic shape inference
+                    if self.run_ == False and not node.op_type in self.dispatcher_:
+                        is_unknown_op = (out_type_undefined and
+                                         len(out_shape) == 0)
+                        if is_unknown_op:
+                            # unknown op to ONNX, maybe from higher opset or other domain
+                            # only guess the output rank from input 0 when using guess_output_rank option
+                            out_rank = self._get_shape_rank(
+                                node, 0) if self.guess_output_rank_ else -1
+                        else:
+                            # valid ONNX op, but not handled by symbolic shape inference, just assign dynamic shape
+                            out_rank = len(out_shape)
+
+                        if out_rank >= 0:
+                            new_shape = self._new_symbolic_shape(out_rank, node,
+                                                                 i_o)
+                            vi.CopyFrom(
+                                helper.make_tensor_value_info(
+                                    vi.name, self.known_vi_[node.input[
+                                        0]].type.tensor_type.elem_type,
+                                    get_shape_from_sympy_shape(new_shape)))
+
+                            if self.verbose_ > 0:
+                                if is_unknown_op:
+                                    print(
+                                        "Possible unknown op: {} node: {}, guessing {} shape"
+                                        .format(node.op_type, node.name,
+                                                vi.name))
+                                if self.verbose_ > 2:
+                                    print('  {}: {} {}'.format(
+                                        node.output[i_o],
+                                        str(new_shape),
+                                        vi.type.tensor_type.elem_type))
+
+                            self.run_ = True
+                            continue  # continue the inference after guess, no need to stop as no merge is needed
+
+                    if self.verbose_ > 0 or not self.auto_merge_ or out_type_undefined:
+                        print('Stopping at incomplete shape inference at ' +
+                              node.op_type + ': ' + node.name)
+                        print('node inputs:')
+                        for i in node.input:
+                            print(self.known_vi_[i])
+                        print('node outputs:')
+                        for o in node.output:
+                            print(self.known_vi_[o])
+                        if self.auto_merge_ and not out_type_undefined:
+                            print('Merging: ' + str(self.suggested_merge_))
+                    return False
+        self.run_ = False
+        return True
+
+    def _update_output_from_vi(self):
+        for output in self.out_mp_.graph.output:
+            if output.name in self.known_vi_:
+                tmp_output = self.known_vi_[output.name]
+                output.CopyFrom(tmp_output)
+
+    @staticmethod
+    def infer_shapes(in_mp,
+                     int_max=2**31 - 1,
+                     fixed_input_shape=None,
+                     auto_merge=True,
+                     guess_output_rank=False,
+                     verbose=0):
+        if get_opset(in_mp) < 7:
+            print('Only support shape inferencing models of opset 7 and above.')
+            return
+        symbolic_shape_inference = SymbolicShapeInference(
+            int_max, auto_merge, guess_output_rank, verbose)
+        all_shapes_inferred = False
+        symbolic_shape_inference._preprocess(
+            in_mp, input_shapes=fixed_input_shape)
+        try:
+            while symbolic_shape_inference.run_:
+                all_shapes_inferred = symbolic_shape_inference._infer_impl(
+                    in_mp)
+            symbolic_shape_inference._update_output_from_vi()
+            if not all_shapes_inferred:
+                print('!' * 10)
+                symbolic_shape_inference.out_mp_ = shape_inference.infer_shapes(
+                    symbolic_shape_inference.out_mp_)
+            #onnx.save(symbolic_shape_inference.out_mp_, 'tmp.onnx')
+        except:
+            print('Stopping at incomplete shape inference')
+            symbolic_shape_inference.out_mp_ = shape_inference.infer_shapes(
+                symbolic_shape_inference.out_mp_)
+        return symbolic_shape_inference.out_mp_.graph

x2paddle/decoder/tf_decoder.py

@@ -48,7 +48,7 @@ class TFGraphNode(GraphNode):
 
     @property
     def out_shapes(self):
-        if self.layer_type == "OneShotIterator":
+        if self.layer_type == "OneShotIterator" or self.layer_type == "IteratorV2":
             values = self.layer.attr["output_shapes"].list.shape
         else:
             values = self.layer.attr["_output_shapes"].list.shape
@@ -68,7 +68,8 @@ class TFGraphNode(GraphNode):
         if dtype == 0:
             dtype = self.layer.attr['output_types'].list.type[0]
         if dtype not in self.dtype_map:
-            raise Exception("Dtype[{}] not in dtype_map".format(dtype))
+            raise Exception("Dtype[{}] of node({}) not in dtype_map".format(
+                dtype, self.layer.name))
         return self.dtype_map[dtype]
 
     @property
@@ -88,6 +89,16 @@ class TFGraphNode(GraphNode):
         field = getattr(attr, attr.WhichOneof('value'))
         return tensor_util.MakeNdarray(field)
 
+    @property
+    def name(self):
+        multi_out_ops = ['Split', 'SplitV', 'IteratorV2']
+        if self.layer_type in multi_out_ops:
+            if self.layer_name.count(':') > 0:
+                return self.layer_name.replace(':', '_p')
+            else:
+                return "{}_p0".format(self.layer_name)
+        return self.layer_name
+
     def get_attr(self, name):
         if name not in self.layer.attr:
             return None
@@ -114,16 +125,20 @@ class TFGraph(Graph):
     def __init__(self, model, data_format="NHWC"):
         super(TFGraph, self).__init__(model)
         self.identity_map = dict()
-        self.multi_out_ops = ['Split', 'SplitV']
+        self.multi_out_ops = ['Split', 'SplitV', 'IteratorV2']
         self.tf_data_format = data_format
 
     def build(self):
         for layer in self.model.node:
+            if layer.op == 'Assert':
+                continue
             self.node_map[layer.name.replace('/', '_').replace(
                 '-', '_')] = TFGraphNode(
                     layer, data_format=self.tf_data_format)
 
         for layer_name, node in self.node_map.items():
+            if node.layer_type == 'Const':
+                continue
             for in_node in node.layer.input:
                 in_node = in_node.replace('/', '_').replace('-', '_').replace(
                     '^', '')
@@ -139,6 +154,14 @@ class TFGraph(Graph):
 
         super(TFGraph, self).build()
 
+        for layer in self.model.node:
+            if layer.op == 'Assert':
+                for ipt in layer.input:
+                    ipt_name = ipt.replace('-', '_').replace('/', '_')
+                    if ipt_name in self.output_nodes:
+                        idx = self.output_nodes.index(ipt_name)
+                        del self.output_nodes[idx]
+
         # tensorflow graph optimize
         self._remove_isolated_node()
         self._optimize_dialiation_conv()
@@ -272,7 +295,6 @@ class TFGraph(Graph):
 
     def data_format_propagation(self, node):
         current_node = self.node_map[node.layer_name]
-        current_node = node.tf_data_format
         outputs = current_node.outputs
         if len(outputs) == 0:
             return
@@ -322,7 +344,7 @@ class TFDecoder(object):
         graph_def = cp.deepcopy(graph_def)
         input_map = dict()
         for layer in graph_def.node:
-            if layer.op != "Placeholder" and layer.op != "OneShotIterator":
+            if layer.op != "Placeholder" and layer.op != "OneShotIterator" and layer.op != "IteratorV2":
                 continue
             graph_node = TFGraphNode(layer)
             dtype = graph_node.layer.attr['dtype'].type
@@ -403,7 +425,7 @@ class TFDecoder(object):
             else:
                 value = graph_node.layer.attr["shape"].shape
                 shape = [dim.size for dim in value.dim]
-                self.input_info[graph_node.layer_name] = (shape, dtype)
+                self.input_info[layer.name] = (shape, dtype)
 
         return input_map
 

x2paddle/op_mapper/onnx2paddle/onnx_op_mapper.py

+# Copyright (c) 2019  PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License"
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from x2paddle.op_mapper.onnx2paddle.opset9 import OpSet9, custom_layers
+from x2paddle.core.op_mapper import OpMapper
+from x2paddle.decoder.onnx_decoder import ONNXGraph, ONNXGraphNode, ONNXGraphDataNode
+
+
+class ONNXOpMapper(OpMapper):
+    def __init__(self, decoder):
+        super(ONNXOpMapper, self).__init__()
+        self.support_op_sets = [9, ]
+        self.default_op_set = 9
+        self.graph = decoder.graph
+        self.opset = self.create_opset(decoder)
+        if not self.op_checker():
+            raise Exception("Model are not supported yet.")
+        #mapping op
+        print("Total nodes: {}".format(
+            sum([
+                isinstance(node, ONNXGraphNode)
+                for name, node in self.graph.node_map.items()
+            ])))
+
+        print("Nodes converting ...")
+        for node_name in self.graph.topo_sort:
+            node = self.graph.get_node(node_name)
+            op = node.layer_type
+            if hasattr(self.opset, op):
+                func = getattr(self.opset, op)
+                func(node)
+            elif op in self.opset.default_op_mapping:
+                self.opset.directly_map(node)
+            elif op in custom_layers:
+                self.opset.deal_custom_layer(node)
+            elif op in self.opset.elementwise_ops:
+                self.opset.elementwise_map(node)
+        print("Nodes converted.")
+        self.weights = self.opset.weights
+        self.omit_nodes = self.opset.omit_nodes
+        self.used_custom_layers = self.opset.used_custom_layers
+
+    def op_checker(self):
+        unsupported_ops = set()
+        for node_name in self.graph.topo_sort:
+            node = self.graph.get_node(node_name)
+            op = node.layer_type
+            if not hasattr(self.opset, op) and \
+                op not in self.opset.default_op_mapping and \
+                op not in custom_layers and \
+                op not in self.opset.elementwise_ops:
+                unsupported_ops.add(op)
+        if len(unsupported_ops) == 0:
+            return True
+        else:
+            print("There are {} ops not supported yet, list as below".format(
+                len(unsupported_ops)))
+            for op in unsupported_ops:
+                print(op)
+            return False
+
+    def create_opset(self, decoder):
+        run_op_set = self.default_op_set
+        opset = ''
+        if decoder.op_set in self.support_op_sets:
+            opset = 'OpSet' + str(decoder.op_set)
+        elif decoder.op_set < self.default_op_set:
+            opset = 'OpSet' + str(self.default_op_set)
+        else:
+            for op_set in self.support_op_sets:
+                if decoder.op_set > op_set:
+                    run_op_set = op_set
+                else:
+                    break
+            opset = 'OpSet' + str(run_op_set)
+        print(
+            'Now, onnx2paddle support convert onnx model opset_verison {},'
+            'opset_verison of your onnx model is {}, automatically treated as op_set: {}.'
+            .format(self.support_op_sets, decoder.op_set, run_op_set))
+        return eval(opset)(decoder)

x2paddle/op_mapper/onnx2paddle/opset9/__init__.py

+from .opset import OpSet9
+from .custom_layer import custom_layers

x2paddle/op_mapper/onnx_custom_layer/__init__.py → x2paddle/op_mapper/onnx2paddle/opset9/custom_layer/__init__.py

@@ -13,10 +13,6 @@
 # limitations under the License.
 
 from .register import get_registered_layers
-#custom layer import begins
-
-from . import InstanceNormalization
-#custom layer import ends
 
 custom_layers = get_registered_layers()
 

x2paddle/op_mapper/onnx_op_mapper.py → x2paddle/op_mapper/onnx2paddle/opset9/opset.py

-#   Copyright (c) 2019  PaddlePaddle Authors. All Rights Reserved.
+# Copyright (c) 2019  PaddlePaddle Authors. All Rights Reserved.
 #
 # Licensed under the Apache License, Version 2.0 (the "License"
 # you may not use this file except in compliance with the License.
@@ -12,27 +12,23 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
+from x2paddle.decoder.onnx_decoder import ONNXGraph, ONNXGraphNode, ONNXGraphDataNode
 from x2paddle.core.graph import GraphNode
-from x2paddle.core.op_mapper import OpMapper
 from x2paddle.core.fluid_code import Layer
 from x2paddle.core.fluid_code import FluidCode
-from x2paddle.decoder.onnx_decoder import ONNXGraph, ONNXGraphNode, ONNXGraphDataNode
-from x2paddle.op_mapper.onnx_directly_map import default_op_mapping_field_values
-from x2paddle.op_mapper.onnx_directly_map import default_op_mapping
-from x2paddle.op_mapper.onnx_directly_map import default_ioa_constraint
-from x2paddle.op_mapper.onnx_custom_layer import *
 from x2paddle.core.util import string
+from x2paddle.op_mapper.onnx2paddle.opset9.custom_layer import *
+from functools import reduce
 import numpy as np
 import onnx
 import onnx.numpy_helper as numpy_helper
 from onnx.mapping import TENSOR_TYPE_TO_NP_TYPE
 import logging as _logging
-from collections import OrderedDict as _dict
+from collections import OrderedDict
 import math
 import os
 import shutil
-from functools import reduce
-import onnxruntime as rt
+
 _logger = _logging.getLogger(__name__)
 
 
@@ -52,7 +48,24 @@ def get_same_padding(in_size, kernel_size, stride):
     return [pad0, pad1]
 
 
-class ONNXOpMapper(OpMapper):
+def print_mapping_info(func):
+    def run_mapping(*args, **kwargs):
+        node = args[1]
+        try:
+            res = func(*args, **kwargs)
+        except:
+            print("convert failed node:{}, op_type is {}".format(
+                node.layer_name[9:], node.layer_type))
+            raise
+        else:
+            #print("convert successfully node:{}, op_type is {}".format(
+            #    node.layer_name[9:], node.layer_type))
+            return res
+
+    return run_mapping
+
+
+class OpSet9():
     elementwise_ops = {
         'Add': 'elementwise_add',
         'Div': 'elementwise_div',
@@ -61,154 +74,87 @@ class ONNXOpMapper(OpMapper):
         'Pow': 'elementwise_pow',
     }
 
-    def __init__(self, decoder, save_dir):
-        super(ONNXOpMapper, self).__init__()
-        self.decoder = decoder
-        self.graph = decoder.onnx_graph
+    default_op_mapping_field_values = OrderedDict()
+    default_op_mapping_field_values['FLUID_OP'] = ''
+    default_op_mapping_field_values['FLUID_INPUT_ARGS'] = None
+    default_op_mapping_field_values['FLUID_OUTPUT_ARGS'] = None
+    default_op_mapping_field_values['ATTR_MAPPING'] = dict()
+    default_op_mapping_field_values['DEFAULTS'] = dict()
+    default_op_mapping_field_values['INPUT_PERM'] = None
+    default_op_mapping_field_values['OUTPUT_PERM'] = None
+    default_op_mapping_field_values['FILL_NAME_FIELD'] = True
+
+    default_op_mapping = {
+        'Shape': ['shape', ['X'], ['Out']],
+        'Clip': [
+            'clip', ['X'], ['Out'], dict(), dict(
+                min=(np.asarray(
+                    [255, 255, 127, 255], dtype=np.uint8).view(np.float32)[0]),
+                max=(np.asarray(
+                    [255, 255, 127, 127], dtype=np.uint8).view(np.float32)[0]),
+            )
+        ],
+        'Erf': ['erf', ['X'], ['Out']],
+        'Ceil': ['ceil', ['X'], ['Out']],
+        'ReduceMean': [
+            'reduce_mean', ['X'], ['Out'], dict(
+                axes='dim', keepdims='keep_dim'), dict(keep_dim=1)
+        ],
+        'ReduceSum': [
+            'reduce_sum', ['X'], ['Out'], dict(
+                axes='dim', keepdims='keep_dim'), dict(keep_dim=1)
+        ],
+        'ReduceMin': [
+            'reduce_min', ['X'], ['Out'], dict(
+                axes='dim', keepdims='keep_dim'), dict(keep_dim=1)
+        ],
+        'ReduceMax': [
+            'reduce_max', ['X'], ['Out'], dict(
+                axes='dim', keepdims='keep_dim'), dict(keep_dim=1)
+        ],
+        #active function
+        'Relu': ['relu', ['X'], ['Out']],
+        'LeakyRelu': ['leaky_relu', ['X'], ['Out'], dict(), dict(alpha=.01)],
+        'Elu': ['elu', ['X'], ['Out'], dict(), dict(alpha=1.)],
+        'ThresholdedRelu': [
+            'thresholded_relu', ['X'], ['Out'], dict(alpha='threshold'),
+            dict(alpha=1.)
+        ],
+        'Tanh': ['tanh', ['X'], ['Out']],
+        'Sigmoid': ['sigmoid', ['X'], ['Out']],
+        'HardSigmoid': [
+            'hard_sigmoid', ['X'], ['Out'], dict(
+                alpha='slope', beta='offset'), dict(
+                    slope=.2, offset=.5)
+        ],
+        'Softsign': ['softsign', ['X'], ['Out']],
+        'Softplus': ['softplus', ['X'], ['Out']],
+        'Exp': ['exp', ['X'], ['Out']],
+        'Softmax': ['softmax', ['X'], ['Out'], dict(), dict(axis=1)],
+        'Sqrt': ['sqrt', ['X'], ['Out']],
+        'Floor': ['floor', ['X'], ['Out']],
+        'Abs': ['abs', ['X'], ['Out']],
+    }
+
+    default_ioa_constraint = {}
+
+    def __init__(self, decoder):
+        super(OpSet9, self).__init__()
+        self.graph = decoder.graph
         self.input_shapes = []
         self.weights = dict()
         self.omit_nodes = list()
         self.used_custom_layers = dict()
-        self.is_inference = False
-        self.tmp_data_dir = os.path.join(save_dir, 'tmp_data')
-        self.tmp_outputs_dict = {}
-        self.get_output_shapes()
-
-        if not self.op_checker():
-            raise Exception("Model are not supported yet.")
-
-        #mapping op
-        print("Total nodes: {}".format(
-            sum([
-                isinstance(node, ONNXGraphNode)
-                for name, node in self.graph.node_map.items()
-            ])))
-        for node_name in self.graph.topo_sort:
-            node = self.graph.get_node(node_name)
-            op = node.layer_type
-            if hasattr(self, op):
-                func = getattr(self, op)
-                func(node)
-            elif op in default_op_mapping:
-                self.directly_map(node)
-            elif op in custom_layers:
-                self.deal_custom_layer(node)
-            elif op in self.elementwise_ops:
-                self.elementwise_map(node)
-
-        self.remove_tmp_data()
-
-    def op_checker(self):
-        unsupported_ops = set()
-        for node_name in self.graph.topo_sort:
-            node = self.graph.get_node(node_name)
-            op = node.layer_type
-            if not hasattr(self, op) and \
-                op not in default_op_mapping and \
-                op not in custom_layers and \
-                op not in self.elementwise_ops:
-                unsupported_ops.add(op)
-        if len(unsupported_ops) == 0:
-            return True
-        else:
-            print("There are {} ops not supported yet, list as below".format(
-                len(unsupported_ops)))
-            for op in unsupported_ops:
-                print(op)
-            return False
-
-    def get_results_of_inference(self, model, value_infos, data_nodes):
-        if not os.path.exists(self.tmp_data_dir):
-            os.makedirs(self.tmp_data_dir)
-        inputs_dict = {}
-        for data_node in data_nodes:
-            value_info = value_infos[data_node]
-            shape = value_info['shape']
-            for i, dim_shape in enumerate(shape):
-                if dim_shape == 0 and i == 0:
-                    shape[i] = 1
-                if dim_shape == 0 and i != 0:
-                    assert 'shape of input is not assigned'
-            ipt = np.random.random(shape).astype(value_info['dtype'])
-            inputs_dict[data_node] = ipt
-
-        model = onnx.shape_inference.infer_shapes(model)
-        outputs = []
-
-        for value_info in model.graph.value_info:
-            outputs.append(value_info.name)
-
-        model.graph.ClearField('output')
-        model.graph.output.MergeFrom(model.graph.value_info)
-        onnx.save(model,
-                  os.path.join(self.tmp_data_dir, 'onnx_model_infer.onnx'))
-        sess = rt.InferenceSession(
-            os.path.join(self.tmp_data_dir, 'onnx_model_infer.onnx'))
-        res = sess.run(None, input_feed=inputs_dict)
-        self.tmp_outputs_dict = dict(zip(outputs, res))
-
-        return
-
-    def get_dynamic_shape(self, layer):
-        """
-        get dynamic shape from infer_result
-        """
-        if layer not in self.tmp_outputs_dict:
-            return [None, None, None]
-        output = self.tmp_outputs_dict[layer]
-        return output.tolist(), output.dtype, output.shape
-
-    def get_output_shapes(self):
-        """
-        build topo_sort of ONNX model
-        """
-        nodes = self.decoder.model.graph.node
-        node_map = self.decoder.onnx_graph.node_map
-        value_infos = self.decoder.onnx_graph.value_infos
-        onnx_model = self.decoder.model
-        for layer in nodes:
-            node = node_map[layer.name]
-            for opt in layer.output:
-                if opt in value_infos:
-                    value_info = value_infos[opt]
-                    if len(value_info['shape']) == 0 or value_info[
-                            'dtype'] is None or 0 in value_info['shape']:
-                        if self.is_inference == False:
-                            self.get_results_of_inference(
-                                onnx_model, value_infos,
-                                self.decoder.onnx_graph.place_holder_nodes)
-                            self.is_inference = True
-                        _, dtype, shape = self.get_dynamic_shape(opt)
-                        node.out_shapes.append(shape)
-                        node.dtype = dtype
-                    else:
-                        node.dtype = value_info['dtype']
-                        node.out_shapes.append(value_info['shape'])
-                else:
-                    if self.is_inference == False:
-                        self.get_results_of_inference(
-                            onnx_model, value_infos,
-                            self.decoder.onnx_graph.place_holder_nodes)
-                        self.is_inference = True
-                    _, dtype, shape = self.get_dynamic_shape(opt)
-                    node.dtype = dtype
-                    node.out_shapes.append(shape)
-
-    def remove_tmp_data(self):
-        """
-        remove temporarily generated file
-        """
-        if os.path.exists(self.tmp_data_dir):
-            import shutil
-            shutil.rmtree(self.tmp_data_dir)
 
+    @print_mapping_info
     def directly_map(self, node, name='', *args, **kwargs):
         inputs = node.layer.input
         outputs = node.layer.output
         op_type = node.layer_type
         attrs = node.attr_map
-        info = default_op_mapping[op_type]
-        info.extend(list(default_op_mapping_field_values.values())[len(info):])
+        info = self.default_op_mapping[op_type]
+        info.extend(
+            list(self.default_op_mapping_field_values.values())[len(info):])
         (
             fluid_op,
             fluid_input_args,
@@ -219,8 +165,8 @@ class ONNXOpMapper(OpMapper):
             output_perm,
             fill_name_field, ) = info
 
-        if fluid_op in default_ioa_constraint:
-            for predicate, message in default_ioa_constraint[fluid_op]:
+        if fluid_op in self.default_ioa_constraint:
+            for predicate, message in self.default_ioa_constraint[fluid_op]:
                 assert predicate(inputs, outputs, attrs), message
 
         mapped_attrs = {
@@ -243,11 +189,18 @@ class ONNXOpMapper(OpMapper):
             map(lambda i: outputs[i], output_perm))
         attr = fluid_attrs
         assert len(val_inps) == 1, 'directly_map error with multi inputs'
-        if fluid_op not in ['shape']:
+        if fluid_op not in ['shape', 'erf']:
             attr['name'] = string(node.layer_name)
         node.fluid_code.add_layer(
             fluid_op, inputs=val_inps[0], output=val_outs[0], param_attr=attr)
+        if fluid_op in ['shape']:
+            node.fluid_code.add_layer(
+                'cast',
+                inputs=val_outs[0],
+                output=val_outs[0],
+                param_attr={'dtype': string('int64')})
 
+    @print_mapping_info
     def deal_custom_layer(self, node):
         op = node.layer_type
         custom_code, func = make_custom_layer(node)
@@ -268,6 +221,7 @@ class ONNXOpMapper(OpMapper):
                     self.used_custom_layers[op +
                                             '_child_func'] = child_func_code
 
+    @print_mapping_info
     def elementwise_map(self, node):
         assert node.layer_type in self.elementwise_ops
         op_type = self.elementwise_ops[node.layer_type]
@@ -279,6 +233,7 @@ class ONNXOpMapper(OpMapper):
 
         if len(val_x_shape) < len(val_y_shape):
             val_x, val_y = val_y, val_x
+            val_y_shape, val_x_shape = val_x_shape, val_y_shape
 
         str_y_shape = ','.join(str(e) for e in val_y_shape)
         str_x_shape = ','.join(str(e) for e in val_x_shape)
@@ -310,6 +265,7 @@ class ONNXOpMapper(OpMapper):
             node.fluid_code.add_layer(
                 op_type, inputs=inputs, output=node, param_attr=attr)
 
+    @print_mapping_info
     def place_holder(self, node):
         self.input_shapes.append(node.out_shapes[0])
 
@@ -329,6 +285,7 @@ class ONNXOpMapper(OpMapper):
         node.fluid_code.add_layer(
             "data", inputs=None, output=node, param_attr=attr)
 
+    @print_mapping_info
     def create_parameter(self, node, parameter=None):
         if parameter is not None:
             node = parameter
@@ -341,11 +298,24 @@ class ONNXOpMapper(OpMapper):
             'dtype': string(dtype),
             'shape': shape,
             'name': string(node.layer_name),
-            'attr': string(node.layer_name),
             'default_initializer': 'Constant(0.0)'
         }
-        node.fluid_code.add_layer(
-            "create_parameter", inputs=None, output=node, param_attr=attr)
+        if dtype == 'bool':
+            attr['dtype'] = string('int64')
+            node.fluid_code.add_layer(
+                "create_parameter", inputs=None, output=node, param_attr=attr)
+            node.fluid_code.add_layer(
+                "cast",
+                inputs=node,
+                output=node,
+                param_attr={'dtype': string('bool')})
+        elif dtype == 'uint8':
+            attr['dtype'] = string('float32')
+            node.fluid_code.add_layer(
+                "create_parameter", inputs=None, output=node, param_attr=attr)
+        else:
+            node.fluid_code.add_layer(
+                "create_parameter", inputs=None, output=node, param_attr=attr)
 
     def _pad_if_asymmetric(self, node, pads, val_name):  # pads: SSEE
         assert len(pads) & 1 == 0
@@ -362,41 +332,13 @@ class ONNXOpMapper(OpMapper):
 
     def _interpolate(self, node):
         val_x = self.graph.get_input_node(node, idx=0, copy=True)
-        val_scales = self.graph.get_input_node(node, idx=1, copy=True)
-        val_y = self.graph.get_node(node.layer.output[0], copy=True)
-        out_shape = val_y.out_shapes[0]
-        if out_shape is not None:
-            assert len(out_shape) == 4, 'only 4-D Tensor as X and Y supported'
-            out_shape = out_shape[2:]
-
-        scales = _const_weight_or_none(val_scales)
+        if node.layer_type == 'Resize':
+            val_scales = self.graph.get_input_node(node, idx=2, copy=True)
+        elif node.layer_type == 'Upsample':
+            val_scales = self.graph.get_input_node(node, idx=1, copy=True)
 
-        if isinstance(val_scales, ONNXGraphNode):
-            scales, _, _ = self.get_dynamic_shape(val_scales.layer_name)
         attr = {'name': string(node.layer_name)}
-        use_scales = True
-        if scales is not None:
-            try:
-                assert len(scales) == 4, 'only 4-D Tensor as X and Y supported'
-                assert scales[0] == 1 and scales[
-                    1] == 1, 'only scale on (NC)HW supported'
-                assert scales[2] == scales[
-                    3], 'only aspect-ratio-invariant scale supported'
-            except:
-                use_scales = False
-        scale = scales[2] if scales else None
-        if scale is None:
-            assert out_shape, 'neither scales nor output shape is available'
-        else:
-            if out_shape is None:
-                in_shape = val_x.out_shapes[0]
-                assert in_shape is not None, 'out_shape required but not inferrable'
-                assert len(
-                    in_shape) == 4, 'only 4-D Tensor as X and Y supported'
-                out_shape = [in_shape[2] * scale, in_shape[3] * scale]
-
         mode = node.get_attr('mode', 'nearest')
-
         fluid_op = 'resize_{}'.format(mode)
         if 'linear' in mode:
             print(
@@ -404,14 +346,14 @@ class ONNXOpMapper(OpMapper):
             )
             fluid_op = 'resize_bilinear'
 
-        if use_scales and scale is not None:
-            attr['scale'] = scale
-        else:
-            attr['out_shape'] = out_shape
-
         node.fluid_code.add_layer(
-            fluid_op, inputs=val_x, output=node, param_attr=attr)
+            fluid_op,
+            inputs={'input': val_x,
+                    'scale': val_scales},
+            output=node,
+            param_attr=attr)
 
+    @print_mapping_info
     def RoiAlign(self, node):
         val_x = self.graph.get_input_node(node, idx=0, copy=True)
         val_rois = self.graph.get_input_node(node, idx=1, copy=True)
@@ -433,6 +375,7 @@ class ONNXOpMapper(OpMapper):
             output=node,
             param_attr=attr)
 
+    @print_mapping_info
     def MaxRoiPool(self, node):
         val_x = self.graph.get_input_node(node, idx=0, copy=True)
         val_rois = self.graph.get_input_node(node, idx=1, copy=True)
@@ -451,6 +394,7 @@ class ONNXOpMapper(OpMapper):
             output=node,
             param_attr=attr)
 
+    @print_mapping_info
     def Pad(self, node, op_independent=True):
         val_x = self.graph.get_input_node(node, idx=0, copy=True)
         pads = node.get_attr('pads')
@@ -497,17 +441,23 @@ class ONNXOpMapper(OpMapper):
                 param_attr=attr)
             return node.layer_name + '_paded'
 
+    @print_mapping_info
     def Unsqueeze(self, node):
         val_x = self.graph.get_input_node(node, idx=0, copy=True)
         axes = node.get_attr('axes')
+        attr = {'axes': axes, 'name': string(node.layer_name)}
         if len(val_x.out_shapes[0]) == 0:
-            node.fluid_code.add_layer(
-                'assign', inputs=val_x, output=node, param_attr=None)
+            if node.layer_name:
+                node.fluid_code.add_layer(
+                    'reshape',
+                    inputs=val_x,
+                    output=node,
+                    param_attr={'shape': [1]})
         else:
-            attr = {'axes': axes, 'name': string(node.layer_name)}
             node.fluid_code.add_layer(
                 'unsqueeze', inputs=val_x, output=node, param_attr=attr)
 
+    @print_mapping_info
     def Shrink(self, node):
         val_x = self.graph.get_input_node(node, idx=0, copy=True)
         bias = node.get_attr('bias')
@@ -527,6 +477,7 @@ class ONNXOpMapper(OpMapper):
             output=node,
             param_attr=None)
 
+    @print_mapping_info
     def Constant(self, node):
         val_output = self.graph.get_node(node.layer.output[0], copy=True)
 
@@ -557,24 +508,42 @@ class ONNXOpMapper(OpMapper):
             node.fluid_code.add_layer(
                 'fill_constant', inputs=None, output=node, param_attr=attr)
         else:
+            if dtype.name == 'uint8':
+                dtype = 'int64'
             value = np.reshape(value, shape)
             self.weights[node.layer_name] = value
             attr = {
                 'dtype': string(dtype),
                 'shape': shape,
                 'name': string(node.layer_name),
-                'attr': string(node.layer_name),
                 'default_initializer': 'Constant(0.0)'
             }
             node.fluid_code.add_layer(
                 "create_parameter", inputs=None, output=node, param_attr=attr)
 
+    @print_mapping_info
     def Resize(self, node):
         self._interpolate(node)
 
+    @print_mapping_info
     def Upsample(self, node):
         self._interpolate(node)
 
+    @print_mapping_info
+    def InstanceNormalization(self, node):
+        val_x = self.graph.get_input_node(node, idx=0, copy=True)
+        val_scale = self.graph.get_input_node(node, idx=1, copy=True)
+        val_b = self.graph.get_input_node(node, idx=2, copy=True)
+        epsilon = node.get_attr('epsilon', 1e-5)
+        attr = {
+            'epsilon': epsilon,
+            'param_attr': string(val_scale.layer_name),
+            'bias_attr': string(val_b.layer_name)
+        }
+        node.fluid_code.add_layer(
+            "instance_norm", inputs=val_x, output=node, param_attr=attr)
+
+    @print_mapping_info
     def Expand(self, node):
         val_x = self.graph.get_input_node(node, idx=0, copy=True)
         val_shape = self.graph.get_input_node(node, idx=1, copy=True)
@@ -598,13 +567,14 @@ class ONNXOpMapper(OpMapper):
             output=node.layer_name,
             param_attr=attr)
 
+    @print_mapping_info
     def Gather(self, node):
         val_x = self.graph.get_input_node(node, idx=0, copy=True)
         indices = self.graph.get_input_node(node, idx=1, copy=True)
         indices_shape = indices.out_shapes[0]
         axis = node.get_attr('axis', 0)
-        assert len(
-            indices_shape) <= 2, "Gather op don't support dim of indice >2 "
+        #assert len(
+        #    indices_shape) <= 2, "Gather op don't support dim of indice >2 "
         if axis == 0 and len(indices_shape) <= 1:
             node.fluid_code.add_layer(
                 'gather',
@@ -630,13 +600,55 @@ class ONNXOpMapper(OpMapper):
                 param_attr=None)
             node.fluid_code.add_layer(
                 'transpose', inputs=node, output=node, param_attr=attr_trans)
-        elif len(indices_shape) > 1:
+        elif axis == 0 and len(indices_shape) > 1:
+            if val_x.out_shapes[0] is not None and isinstance(
+                    val_x, ONNXGraphDataNode):
+                node.fluid_code.add_layer(
+                    'embedding',
+                    inputs=indices,
+                    output=node,
+                    use_fluid=True,
+                    param_attr={
+                        'param_attr': string(val_x.layer_name),
+                        'size': val_x.out_shapes[0]
+                    })
+            else:
+                from functools import reduce
+                #indices_shape = [1,7]
+                reshape_shape = reduce(lambda x, y: x * y, indices_shape)
+                indices_reshape = indices.layer_name + '_shape'
+                node.fluid_code.add_layer(
+                    'reshape',
+                    inputs=indices,
+                    output=indices_reshape,
+                    param_attr={'shape': [reshape_shape, ]})
+
+                perm = list(range(len(val_x.out_shapes[0])))
+                node.fluid_code.add_layer(
+                    'gather',
+                    inputs={'input': val_x,
+                            'index': indices_reshape},
+                    output=node,
+                    param_attr=None)
+                val_x_shape = val_x.out_shapes[0]
+                reshaped_shape = []
+                for i in perm:
+                    reshaped_shape.append(indices_shape[i])
+                for i in val_x_shape[:axis] + val_x_shape[axis + 1:]:
+                    reshaped_shape.append(i)
+                node.fluid_code.add_layer(
+                    'reshape',
+                    inputs=node,
+                    output=node,
+                    param_attr={'shape': reshaped_shape})
+        elif axis > 0 and len(indices_shape) > 1:
             from functools import reduce
             reshape_shape = reduce(lambda x, y: x * y, indices_shape)
+            indices_reshape = indices.layer_name + '_shape'
             node.fluid_code.add_layer(
                 'reshape',
                 inputs=indices,
-                output=indices,
+                output=indices_reshape,
                 param_attr={'shape': [reshape_shape, ]})
 
             perm = list(range(len(val_x.out_shapes[0])))
@@ -651,7 +663,7 @@ class ONNXOpMapper(OpMapper):
             node.fluid_code.add_layer(
                 'gather',
                 inputs={'input': name_trans,
-                        'index': indices},
+                        'index': indices_reshape},
                 output=node,
                 param_attr=None)
             node.fluid_code.add_layer(
@@ -668,69 +680,102 @@ class ONNXOpMapper(OpMapper):
                 output=node,
                 param_attr={'shape': reshaped_shape})
 
+    @print_mapping_info
+    def Range(self, node):
+        val_start = self.graph.get_input_node(node, idx=0, copy=True)
+        val_limit = self.graph.get_input_node(node, idx=1, copy=True)
+        val_delta = self.graph.get_input_node(node, idx=2, copy=True)
+        dtype = val_start.dtype
+        inputs = {'start': val_start, 'end': val_limit, 'step': val_delta}
+        node.fluid_code.add_layer(
+            'range',
+            inputs=inputs,
+            output=node,
+            param_attr={'dtype': string(dtype)})
+
+    @print_mapping_info
     def Slice(self, node):
         val_x = self.graph.get_input_node(node, idx=0, copy=True)
         starts, ends, axes, steps = None, None, None, None
+        attr = {}
         if len(node.inputs) > 1:
             starts = self.graph.get_input_node(node, idx=1, copy=True)
             ends = self.graph.get_input_node(node, idx=2, copy=True)
             if len(node.inputs) > 3:
                 axes = self.graph.get_input_node(node, idx=3, copy=True)
-                self.omit_nodes.append(axes.layer_name)
                 axes = _const_weight_or_none(axes)
             if len(node.inputs) > 4:
                 steps = self.graph.get_input_node(node, idx=4, copy=True)
-                self.omit_nodes.append(steps.layer_name)
                 steps = _const_weight_or_none(steps)
-
-            self.omit_nodes.append(starts.layer_name)
-            self.omit_nodes.append(ends.layer_name)
-            starts = _const_weight_or_none(starts).copy()
-            ends = _const_weight_or_none(ends).copy()
+                if steps is not None:
+                    assert steps == 1, "Only support convert op:Slice, which attribute:steps == 1"
+            attr = {
+                "axes": axes,
+                "starts": starts.layer_name,
+                "ends": ends.layer_name
+            }
+            starts_value = _const_weight_or_none(starts)
+            ends_value = _const_weight_or_none(ends)
+            if starts_value is not None and ends_value is not None:
+                self.omit_nodes.append(starts.layer_name)
+                self.omit_nodes.append(ends.layer_name)
+                ends_value = ends_value.copy()
+                for idx in range(len(ends_value)):
+                    if ends_value[idx] > 2**31 - 1:
+                        ends_value[idx] = 2**31 - 1
+                attr = {
+                    "axes": axes,
+                    "starts": starts_value,
+                    "ends": ends_value
+                }
+            else:
+                if starts.dtype != 'int32':
+                    node.fluid_code.add_layer(
+                        'cast',
+                        inputs=starts,
+                        output=starts,
+                        param_attr={'dtype': string('int32')})
+                if ends.dtype != 'int32':
+                    node.fluid_code.add_layer(
+                        'cast',
+                        inputs=ends,
+                        output=ends,
+                        param_attr={'dtype': string('int32')})
         else:
             starts = node.get_attr('starts')
             ends = node.get_attr('ends')
             axes = node.get_attr('axes')
+            for idx in range(len(ends)):
+                if ends[idx] > 2**31 - 1:
+                    ends[idx] = 2**31 - 1
+            attr = {"axes": axes, "starts": starts, "ends": ends}
 
-        val_y = self.graph.get_node(node.layer.output[0], copy=True)
-
-        shape = val_x.out_shapes[0]
-
-        if shape is not None:
-            for idx, value in enumerate(starts):
-                if value > shape[axes[idx]]:
-                    starts[idx] = shape[axes[idx]]
-            for idx, value in enumerate(ends):
-                if value > shape[axes[idx]]:
-                    ends[idx] = shape[axes[idx]]
-        attr = {"axes": axes, "starts": starts, "ends": ends}
         node.fluid_code.add_layer(
             'slice', inputs=val_x, output=node, param_attr=attr)
 
+    @print_mapping_info
     def ConstantOfShape(self, node):
         val_shape = self.graph.get_input_node(node, idx=0, copy=True)
         val_y = self.graph.get_node(node.layer.output[0], copy=True)
-        shape = _const_weight_or_none(val_shape)
-
-        if shape is None:
-            shape = node.out_shapes[0]
-
-        assert shape is not None, (
-            'given shape is neither const value nor deductible from output, '
-            'this is not supported')
 
         value = node.get_attr('value')
         dtype = value.dtype
         value = value.tolist()
+        assert len(value) == 1, ('given value not Scalar, shape of value > 1, '
+                                 'this is not supported')
         if len(value) == 1:
-            shape = [1]
             value = value[0]
             if dtype.name == 'int64':
                 dtype = 'int32'
-            attr = {'shape': shape, 'dtype': string(dtype), 'value': value}
+            attr = {
+                'shape': val_shape.layer_name,
+                'dtype': string(dtype),
+                'value': value
+            }
             node.fluid_code.add_layer(
                 'fill_constant', inputs=None, output=node, param_attr=attr)
 
+    @print_mapping_info
     def Split(self, node):
         val_x = self.graph.get_input_node(node, idx=0, copy=True)
         val_y = self.graph.get_node(node.layer.output[0], copy=True)
@@ -747,46 +792,53 @@ class ONNXOpMapper(OpMapper):
         node.fluid_code.add_layer(
             'split', inputs=val_x, output=val_y, param_attr=attr)
 
+    @print_mapping_info
     def Reshape(self, node):
         val_x = self.graph.get_input_node(node, idx=0, copy=True)
         val_shape = self.graph.get_input_node(node, idx=1, copy=True)
         val_reshaped = self.graph.get_node(node.layer.output[0], copy=True)
-        shape = None
-
-        if isinstance(val_shape, ONNXGraphDataNode):
-            self.omit_nodes.append(val_shape.layer_name)
-
-        attr = {'name': string(node.layer_name)}
-        # catch dynamic graph shape
-        if isinstance(val_shape, ONNXGraphNode):
-            shape, _, _ = self.get_dynamic_shape(val_shape.layer_name)
-            if val_shape.dtype == 'int64':
-                val_shape_cast = val_shape.layer_name + '_cast'
-                node.fluid_code.add_layer(
-                    'cast',
-                    inputs=val_shape,
-                    output=val_shape_cast,
-                    param_attr={'dtype': string('int32')})
-
-                attr['actual_shape'] = val_shape_cast
-            else:
-                attr['actual_shape'] = val_shape
-
-        if shape is None:
-            shape = val_reshaped.out_shapes[0]
+        attr = {}
+        shape_value = _const_weight_or_none(val_shape)
+        shape_dims = len(val_shape.out_shapes[0])
 
-        if shape is None:
-            shape = [1, -1]
-            _logger.warning(
-                'in %s(%s -> Reshape -> %s): '
-                'input "shape" not inferred, use [1, -1] as dummy value, '
-                'the behavior of Paddle fluid maybe undefined', node.layer_name,
-                val_x.layer_name, val_reshaped.layer_name)
-
-        attr['shape'] = shape
-        node.fluid_code.add_layer(
-            'reshape', inputs=val_x, output=node, param_attr=attr)
+        if shape_value is not None:
+            node.fluid_code.add_layer(
+                'reshape',
+                inputs={'x': val_x},
+                output=node,
+                param_attr={'shape': shape_value.tolist()})
+        elif val_shape.dtype == 'int64':
+            val_shape_cast = val_shape.layer_name + '_cast'
+            node.fluid_code.add_layer(
+                'cast',
+                inputs=val_shape,
+                output=val_shape_cast,
+                param_attr={'dtype': string('int32')})
+            node.fluid_code.add_layer(
+                'reshape',
+                inputs=val_shape_cast,
+                output=val_shape_cast,
+                param_attr={'shape': val_shape.out_shapes[0]})
+            node.fluid_code.add_layer(
+                'reshape',
+                inputs={'x': val_x,
+                        'shape': val_shape_cast},
+                output=node,
+                param_attr=attr)
+        else:
+            node.fluid_code.add_layer(
+                'reshape',
+                inputs=val_shape,
+                output=val_shape,
+                param_attr={'shape': val_shape.out_shapes[0]})
+            node.fluid_code.add_layer(
+                'reshape',
+                inputs={'x': val_x,
+                        'shape': val_shape},
+                output=node,
+                param_attr=attr)
 
+    @print_mapping_info
     def Cast(self, node):
         val_input = self.graph.get_input_node(node, idx=0, copy=True)
         val_output = self.graph.get_node(node.layer.output[0], copy=True)
@@ -802,6 +854,7 @@ class ONNXOpMapper(OpMapper):
         node.fluid_code.add_layer(
             'cast', inputs=val_input, output=node, param_attr=attr)
 
+    @print_mapping_info
     def AveragePool(self, node):
         val_x = self.graph.get_input_node(node, idx=0, copy=True)
 
@@ -838,6 +891,7 @@ class ONNXOpMapper(OpMapper):
         node.fluid_code.add_layer(
             fluid_op, inputs=val_x, output=node, param_attr=attr)
 
+    @print_mapping_info
     def Concat(self, node):
         inputs = []
         for i in range(len(node.layer.input)):
@@ -851,6 +905,7 @@ class ONNXOpMapper(OpMapper):
         node.fluid_code.add_layer(
             'concat', inputs=inputs, output=node, param_attr=attr)
 
+    @print_mapping_info
     def Flatten(self, node):
         val_x = self.graph.get_input_node(node, idx=0, copy=True)
         axis = node.get_attr('axis', 1)
@@ -858,6 +913,7 @@ class ONNXOpMapper(OpMapper):
         node.fluid_code.add_layer(
             'flatten', inputs=val_x, output=node, param_attr=attr)
 
+    @print_mapping_info
     def Gemm(self, node):
         val_a = self.graph.get_input_node(node, idx=0, copy=True)
         val_b = self.graph.get_input_node(node, idx=1, copy=True)
@@ -907,6 +963,7 @@ class ONNXOpMapper(OpMapper):
                     output=node,
                     param_attr=attr)
 
+    @print_mapping_info
     def Sum(self, node):
         val_inps = node.layer.input
         inputs = {
@@ -926,6 +983,7 @@ class ONNXOpMapper(OpMapper):
             node.fluid_code.add_layer(
                 "elementwise_add", inputs=inputs, output=node)
 
+    @print_mapping_info
     def MatMul(self, node):
         val_x = self.graph.get_input_node(node, idx=0, copy=True)
         val_y = self.graph.get_input_node(node, idx=1, copy=True)
@@ -934,6 +992,7 @@ class ONNXOpMapper(OpMapper):
         node.fluid_code.add_layer(
             "matmul", inputs=inputs, output=node, param_attr=attr)
 
+    @print_mapping_info
     def BatchNormalization(self, node):
         val_x = self.graph.get_input_node(node, idx=0, copy=True)
         val_scale = self.graph.get_input_node(node, idx=1, copy=True)
@@ -966,6 +1025,7 @@ class ONNXOpMapper(OpMapper):
         node.fluid_code.add_layer(
             "batch_norm", inputs=val_x, output=node, param_attr=attr)
 
+    @print_mapping_info
     def Transpose(self, node):
         val_x = self.graph.get_input_node(node, idx=0, copy=True)
         perm = node.get_attr('perm')
@@ -973,12 +1033,14 @@ class ONNXOpMapper(OpMapper):
         node.fluid_code.add_layer(
             "transpose", inputs=val_x, output=node, param_attr=attr)
 
+    @print_mapping_info
     def Relu(self, node):
         val_x = self.graph.get_input_node(node, idx=0, copy=True)
         attr = {"name": string(node.layer_name)}
         node.fluid_code.add_layer(
             "relu", inputs=val_x, output=node, param_attr=attr)
 
+    @print_mapping_info
     def PRelu(self, node):
         val_x = self.graph.get_input_node(node, idx=0, copy=True)
         val_slope = self.graph.get_input_node(node, idx=1, copy=True)
@@ -996,13 +1058,22 @@ class ONNXOpMapper(OpMapper):
         node.fluid_code.add_layer(
             "prelu", inputs=val_x, output=node, param_attr=attr)
 
+    @print_mapping_info
     def Squeeze(self, node):
         val_x = self.graph.get_input_node(node, idx=0, copy=True)
         axes = node.get_attr('axes')
         attr = {'axes': axes, "name": string(node.layer_name)}
-        node.fluid_code.add_layer(
-            "squeeze", inputs=val_x, output=node, param_attr=attr)
+        if len(val_x.out_shapes[0]) == 1:
+            node.fluid_code.add_layer(
+                "cast",
+                inputs=val_x,
+                output=node,
+                param_attr={'dtype': string(val_x.dtype)})
+        else:
+            node.fluid_code.add_layer(
+                "squeeze", inputs=val_x, output=node, param_attr=attr)
 
+    @print_mapping_info
     def Equal(self, node):
         val_x = self.graph.get_input_node(node, idx=0, copy=True)
         val_y = self.graph.get_input_node(node, idx=1, copy=True)
@@ -1013,6 +1084,18 @@ class ONNXOpMapper(OpMapper):
             output=node,
             param_attr=None)
 
+    @print_mapping_info
+    def Greater(self, node):
+        val_x = self.graph.get_input_node(node, idx=0, copy=True)
+        val_y = self.graph.get_input_node(node, idx=1, copy=True)
+        node.fluid_code.add_layer(
+            "greater_than",
+            inputs={'x': val_x,
+                    'y': val_y},
+            output=node,
+            param_attr=None)
+
+    @print_mapping_info
     def Where(self, node):
         condition = self.graph.get_input_node(node, idx=0, copy=True)
         val_x = self.graph.get_input_node(node, idx=1, copy=True)
@@ -1059,45 +1142,42 @@ class ONNXOpMapper(OpMapper):
             output=node,
             param_attr=None)
 
+    @print_mapping_info
     def NonZero(self, node):
         val_x = self.graph.get_input_node(node, idx=0, copy=True)
-        where_name = node.layer_name + '_where'
-        node.fluid_code.add_layer(
-            "where", inputs=val_x.layer_name + '!=0', output=where_name)
-        dims = len(val_x.out_shapes[0])
-        elements_count_val_x = reduce(lambda x, y: x * y, val_x.out_shapes[0])
-        flatten_names = []
-        for dim in range(dims):
-            slice_name = node.layer_name + '_slice' + str(dim)
-            flatten_name = node.layer_name + '_flatten' + str(dim)
-            flatten_names.append(flatten_name)
-            attr = {
-                'axes': list(range(dims)),
-                'starts': [0, dim],
-                'ends': [elements_count_val_x, dim + 1]
-            }
+        val_x_dim = len(val_x.out_shapes[0])
+        print(val_x.layer_name, val_x.out_shapes[0])
+        if val_x_dim == 1:
+            node.fluid_code.add_layer("nonzero", inputs=val_x, output=val_x)
             node.fluid_code.add_layer(
-                "slice", inputs=where_name, output=slice_name, param_attr=attr)
+                "transpose",
+                inputs=val_x,
+                output=node,
+                param_attr={'perm': [1, 0]})
+        if val_x_dim > 1:
+            node.fluid_code.add_layer("nonzero", inputs=val_x, output=val_x)
             node.fluid_code.add_layer(
-                "flatten",
-                inputs=slice_name,
-                output=flatten_name,
-                param_attr={'axis': 0})
-        node.fluid_code.add_layer(
-            "concat", inputs=flatten_names, output=node,
-            param_attr={'axis': 0})
+                "split",
+                inputs=val_x,
+                output=val_x,
+                param_attr={'num_or_sections': 1,
+                            'dim': val_x_dim})
+            node.fluid_code.add_layer("concat", inputs=val_x, output=node)
 
+    @print_mapping_info
     def Identity(self, node):
         val_x = self.graph.get_input_node(node, idx=0, copy=True)
         node.fluid_code.add_layer("assign", inputs=val_x, output=node)
 
+    @print_mapping_info
     def Tile(self, node):
         val_x = self.graph.get_input_node(node, idx=0, copy=True)
         val_repeats = self.graph.get_input_node(node, idx=1, copy=True)
         repeats = _const_weight_or_none(val_repeats)
-        assert repeats is not None, 'for OP:Tile, only const repeats supported'
 
-        if isinstance(repeats, int):
+        if repeats is None:
+            repeats = val_repeats.layer_name
+        elif isinstance(repeats, int):
             repeats = [repeats]
 
         attr = {
@@ -1107,9 +1187,9 @@ class ONNXOpMapper(OpMapper):
         node.fluid_code.add_layer(
             "expand", inputs=val_x, output=node, param_attr=attr)
 
+    @print_mapping_info
     def MaxPool(self, node):
         val_x = self.graph.get_input_node(node, idx=0, copy=True)
-
         auto_pad = node.get_attr('auto_pad', 'NOTSET')
         assert node.get_attr(
             "dilations") is None, 'only dilations = 0 is supported'  # optional
@@ -1148,16 +1228,7 @@ class ONNXOpMapper(OpMapper):
     def _global_pool(self, node):
         val_x = self.graph.get_input_node(node, idx=0, copy=True)
         val_y = self.graph.get_node(node.layer.output[0], copy=True)
-        input_shape = val_x.out_shapes[0]
-        output_shape = val_y.out_shapes[0]
-        assert input_shape is not None or output_shape is not None, 'poolnd not inferred'  # N
-        if input_shape:
-            poolnd = len(input_shape) - 2  # NC...
-        elif output_shape:
-            poolnd = len(output_shape) - 2  # NC...
-        assert 2 <= poolnd <= 3, 'only pool2d and pool3d is supported'
-        fluid_op = 'pool{}d'.format(poolnd)
-
+        fluid_op = 'pool2d'
         pool_type = None
         if node.layer.op_type == 'GlobalMaxPool':
             pool_type = 'max'
@@ -1172,12 +1243,15 @@ class ONNXOpMapper(OpMapper):
         node.fluid_code.add_layer(
             fluid_op, inputs=val_x, output=node, param_attr=attr)
 
+    @print_mapping_info
     def GlobalMaxPool(self, node):
         self._global_pool(node)
 
+    @print_mapping_info
     def GlobalAveragePool(self, node):
         self._global_pool(node)
 
+    @print_mapping_info
     def Conv(self, node):
         val_x = self.graph.get_input_node(node, idx=0, copy=True)
         val_w = self.graph.get_input_node(node, idx=1, copy=True)
@@ -1229,6 +1303,7 @@ class ONNXOpMapper(OpMapper):
         node.fluid_code.add_layer(
             fluid_op, inputs=val_x, output=node, param_attr=attr)
 
+    @print_mapping_info
     def ConvTranspose(self, node):
         val_x = self.graph.get_input_node(node, idx=0, copy=True)
         val_w = self.graph.get_input_node(node, idx=1, copy=True)
@@ -1280,6 +1355,7 @@ class ONNXOpMapper(OpMapper):
         node.fluid_code.add_layer(
             fluid_op, inputs=val_x, output=node, param_attr=attr)
 
+    @print_mapping_info
     def GRU(self, node):
         val_x = self.graph.get_input_node(node, idx=0, copy=True)
         val_w = self.graph.get_input_node(node, idx=1, copy=True)
@@ -1303,8 +1379,6 @@ class ONNXOpMapper(OpMapper):
             val_xh = self.graph.get_input_node(
                 node, idx=5 - miss_arg_num, copy=True)
 
-        data, dtype, shape = self.get_dynamic_shape(val_x.layer_name)
-
         x_shape = val_x.out_shapes[0]
 
         assert x_shape[1] == 1, 'only X with batch_size = 1 supported'
@@ -1394,8 +1468,8 @@ class ONNXOpMapper(OpMapper):
                 inputs=val_b,
                 output=var_bi + ',' + var_bh,
                 param_attr={
-                    'axis': 1,
-                    'split': [hidden_size * 3, hidden_size * 3],
+                    'dim': 1,
+                    'num_or_sections': [hidden_size * 3, hidden_size * 3],
                     'name': string(node.layer_name + '.b/split')
                 })
             var_bi0 = node.layer_name + '_bi0'
@@ -1407,7 +1481,7 @@ class ONNXOpMapper(OpMapper):
                             'name': string(var_bi0)})
 
             node.fluid_code.add_layer(
-                'elmentwise_add',
+                'elementwise_add',
                 inputs=[var_mm, var_bi0],
                 output=var_fc,
                 param_attr={

x2paddle/op_mapper/onnx_directly_map.py

-#   Copyright (c) 2019  PaddlePaddle Authors. All Rights Reserved.
-#
-# Licensed under the Apache License, Version 2.0 (the "License"
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-from collections import OrderedDict as _dict
-import numpy as _np
-
-default_op_mapping_field_values = _dict()
-default_op_mapping_field_values['FLUID_OP'] = ''
-default_op_mapping_field_values['FLUID_INPUT_ARGS'] = None
-default_op_mapping_field_values['FLUID_OUTPUT_ARGS'] = None
-default_op_mapping_field_values['ATTR_MAPPING'] = dict()
-default_op_mapping_field_values['DEFAULTS'] = dict()
-default_op_mapping_field_values['INPUT_PERM'] = None
-default_op_mapping_field_values['OUTPUT_PERM'] = None
-default_op_mapping_field_values['FILL_NAME_FIELD'] = True
-
-default_op_mapping = {
-    'Shape': ['shape', ['X'], ['Out']],
-    'Clip': [
-        'clip', ['X'], ['Out'], dict(), dict(
-            min=(_np.asarray(
-                [255, 255, 127, 255], dtype=_np.uint8).view(_np.float32)[0]),
-            max=(_np.asarray(
-                [255, 255, 127, 127], dtype=_np.uint8).view(_np.float32)[0]), )
-    ],
-    'Erf': ['erf', ['X'], ['Out']],
-    'Ceil': ['ceil', ['X'], ['Out']],
-    'ReduceMean': [
-        'reduce_mean', ['X'], ['Out'], dict(
-            axes='dim', keepdims='keep_dim'), dict(keep_dim=1)
-    ],
-    'ReduceSum': [
-        'reduce_sum', ['X'], ['Out'], dict(
-            axes='dim', keepdims='keep_dim'), dict(keep_dim=1)
-    ],
-    'ReduceMin': [
-        'reduce_min', ['X'], ['Out'], dict(
-            axes='dim', keepdims='keep_dim'), dict(keep_dim=1)
-    ],
-    'ReduceMax': [
-        'reduce_max', ['X'], ['Out'], dict(
-            axes='dim', keepdims='keep_dim'), dict(keep_dim=1)
-    ],
-    #active function
-    'Relu': ['relu', ['X'], ['Out']],
-    'LeakyRelu': ['leaky_relu', ['X'], ['Out'], dict(), dict(alpha=.01)],
-    'Elu': ['elu', ['X'], ['Out'], dict(), dict(alpha=1.)],
-    'ThresholdedRelu': [
-        'thresholded_relu', ['X'], ['Out'], dict(alpha='threshold'),
-        dict(alpha=1.)
-    ],
-    'Tanh': ['tanh', ['X'], ['Out']],
-    'Sigmoid': ['sigmoid', ['X'], ['Out']],
-    'HardSigmoid': [
-        'hard_sigmoid', ['X'], ['Out'], dict(
-            alpha='slope', beta='offset'), dict(
-                slope=.2, offset=.5)
-    ],
-    'Softsign': ['softsign', ['X'], ['Out']],
-    'Softplus': ['softplus', ['X'], ['Out']],
-    'Exp': ['exp', ['X'], ['Out']],
-    'Softmax': ['softmax', ['X'], ['Out'], dict(), dict(axis=1)],
-    'Sqrt': ['sqrt', ['X'], ['Out']],
-    'Floor': ['floor', ['X'], ['Out']],
-    'Abs': ['abs', ['X'], ['Out']],
-}
-
-default_ioa_constraint = {
-    'Gather': [(lambda i, o, a: a.get('axis', 0) == 0,
-                'only axis = 0 is supported')],
-}

x2paddle/op_mapper/paddle2onnx/opset10/opset.py

+#   Copyright (c) 2019  PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License"
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import math
+import sys
+import x2paddle
+import os
+import numpy as np
+import paddle.fluid.core as core
+import paddle.fluid as fluid
+import onnx
+from onnx import helper, onnx_pb
+from x2paddle.op_mapper.paddle2onnx.opset9.opset import OpSet9
+
+
+class OpSet10(OpSet9):
+    def __init__(self):
+        super(OpSet10, self).__init__()
+
+    def slice(self, op, block):
+        axes = op.attr('axes')
+        starts = op.attr('starts')
+        ends = op.attr('ends')
+        axes_name = self.get_name(op.type, 'axes')
+        starts_name = self.get_name(op.type, 'starts')
+        ends_name = self.get_name(op.type, 'ends')
+
+        axes_node = self.make_constant_node(axes_name,
+                                            onnx_pb.TensorProto.INT64, axes)
+        starts_node = self.make_constant_node(starts_name,
+                                              onnx_pb.TensorProto.INT64, starts)
+        ends_node = self.make_constant_node(ends_name,
+                                            onnx_pb.TensorProto.INT64, ends)
+        node = helper.make_node(
+            "Slice",
+            inputs=[op.input('Input')[0], starts_name, ends_name, axes_name],
+            outputs=op.output('Out'), )
+        return [starts_node, ends_node, axes_node, node]
+
+    def im2sequence(self, op, block):
+        from .paddle_custom_layer.im2sequence import im2sequence
+        return im2sequence(op, block)
+
+    def yolo_box(self, op, block):
+        from .paddle_custom_layer.yolo_box import yolo_box
+        return yolo_box(op, block)
+
+    def multiclass_nms(self, op, block):
+        from .paddle_custom_layer.multiclass_nms import multiclass_nms
+        return multiclass_nms(op, block)

x2paddle/op_mapper/paddle2onnx/opset10/paddle_custom_layer/im2sequence.py

+# Copyright (c) 2020  PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License"
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import onnx
+import numpy as np
+from onnx import onnx_pb, helper
+from x2paddle.op_mapper.paddle2onnx.opset9.paddle_custom_layer.im2sequence import im2sequence as im2sequence9
+
+
+def im2sequence(op, block):
+    return im2sequence9(op, block)

x2paddle/op_mapper/onnx_custom_layer/InstanceNormalization.py → x2paddle/op_mapper/paddle2onnx/opset10/paddle_custom_layer/multiclass_nms.py

@@ -12,45 +12,21 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-from .register import register
-
-
-def InstanceNormalization_shape(input_shape):
-    return input_shape
-
-
-def InstanceNormalization_layer(inputs, name=None):
-    # TODO(lvmengsi@baidu.com): Check the accuracy when using fluid.layers.layer_norm.
-    epsilon = 1e-5
-    input_ = inputs[0]
-    mean = fluid.layers.reduce_mean(input_, dim=[2, 3], keep_dim=True)
-    var = fluid.layers.reduce_mean(
-        fluid.layers.square(input_ - mean), dim=[2, 3], keep_dim=True)
-    if name is not None:
-        scale_name = name + "_scale"
-        offset_name = name + "_offset"
-
-    scale_param = inputs[1]
-    offset_param = inputs[2]
-    scale = fluid.layers.create_parameter(
-        name=scale_param.name, shape=input_.shape[1:2], dtype="float32")
-    offset = fluid.layers.create_parameter(
-        name=offset_param.name, shape=input_.shape[1:2], dtype="float32")
-
-    tmp = fluid.layers.elementwise_mul(x=(input_ - mean), y=scale, axis=1)
-    tmp = tmp / fluid.layers.sqrt(var + epsilon)
-    tmp = fluid.layers.elementwise_add(tmp, offset, axis=1)
-    return tmp
-
-
-def InstanceNormalization_weights(name, data=None):
-    weights_name = [name + '_scale']
-    return weights_name
-
-
-register(
-    kind='InstanceNormalization',
-    shape=InstanceNormalization_shape,
-    layer=InstanceNormalization_layer,
-    child_func=None,
-    weights=InstanceNormalization_weights)
+import math
+import sys
+import os
+import numpy as np
+import paddle.fluid.core as core
+import paddle.fluid as fluid
+import onnx
+import warnings
+from onnx import helper, onnx_pb
+from x2paddle.op_mapper.paddle2onnx.opset9.paddle_custom_layer.multiclass_nms import multiclass_nms as multiclass_nms9
+
+
+def multiclass_nms(op, block):
+    """
+    Convert the paddle multiclass_nms to onnx op.
+    This op is get the select boxes from origin boxes.
+    """
+    return multiclass_nms9(op, block)

x2paddle/op_mapper/paddle2onnx/opset10/paddle_custom_layer/yolo_box.py

+# Copyright (c) 2020  PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License"
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import onnx
+import numpy as np
+from onnx import onnx_pb, helper
+from x2paddle.op_mapper.paddle2onnx.opset9.paddle_custom_layer.yolo_box import yolo_box as yolo_box9
+
+
+def yolo_box(op, block):
+    return yolo_box9(op, block)

x2paddle/op_mapper/paddle2onnx/opset11/opset.py

+# Copyright (c) 2019  PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License"
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import math
+import sys
+import x2paddle
+import os
+import numpy as np
+import paddle.fluid.core as core
+import paddle.fluid as fluid
+import onnx
+from onnx import helper, onnx_pb
+from x2paddle.op_mapper.paddle2onnx.opset10.opset import OpSet10
+
+
+class OpSet11(OpSet10):
+    def __init__(self):
+        super(OpSet11, self).__init__()
+
+    def relu6(self, op, block):
+        min_name = self.get_name(op.type, 'min')
+        max_name = self.get_name(op.type, 'max')
+        min_node = self.make_constant_node(min_name, onnx_pb.TensorProto.FLOAT,
+                                           0)
+        max_node = self.make_constant_node(max_name, onnx_pb.TensorProto.FLOAT,
+                                           op.attr('threshold'))
+        node = helper.make_node(
+            'Clip',
+            inputs=[op.input('X')[0], min_name, max_name],
+            outputs=op.output('Out'), )
+        return [min_node, max_node, node]
+
+    def pad2d(self, op, block):
+        x_shape = block.var(op.input('X')[0]).shape
+        paddings = op.attr('paddings')
+        onnx_pads = []
+        #TODO support pads is Variable
+        if op.attr('data_format') == 'NCHW':
+            pads = [
+                0, 0, paddings[0], paddings[2], 0, 0, paddings[1], paddings[3]
+            ]
+        else:
+            pads = [
+                0, paddings[0], paddings[2], 0, 0, paddings[1], paddings[3], 0
+            ]
+        pads_name = self.get_name(op.type, 'pads')
+        pads_node = self.make_constant_node(pads_name,
+                                            onnx_pb.TensorProto.INT64, pads)
+        constant_value_name = self.get_name(op.type, 'constant_value')
+        constant_value_node = self.make_constant_node(constant_value_name,
+                                                      onnx_pb.TensorProto.FLOAT,
+                                                      op.attr('pad_value'))
+        node = helper.make_node(
+            'Pad',
+            inputs=op.input('X') + [pads_name, constant_value_name],
+            outputs=op.output('Out'),
+            mode=op.attr('mode'))
+        return [pads_node, constant_value_node, node]
+
+    def bilinear_interp(self, op, block):
+        input_names = op.input_names
+        coordinate_transformation_mode = ''
+        align_corners = op.attr('align_corners')
+        align_mode = op.attr('align_mode')
+        if align_corners:
+            coordinate_transformation_mode = 'align_corners'
+        elif align_mode == 1:
+            coordinate_transformation_mode = 'asymmetric'
+        else:
+            coordinate_transformation_mode = 'half_pixel'
+
+        if ('OutSize' in input_names and len(op.input('OutSize')) > 0) or (
+                'SizeTensor' in input_names and
+                len(op.input('SizeTensor')) > 0):
+            node_list = list()
+            roi_node = self.make_constant_node(
+                self.get_name(op.type, 'roi'), onnx_pb.TensorProto.FLOAT,
+                [1, 1, 1, 1, 1, 1, 1, 1])
+            roi_name = self.get_name(op.type, 'roi')
+            roi_node = self.make_constant_node(
+                roi_name, onnx_pb.TensorProto.FLOAT, [1, 1, 1, 1, 1, 1, 1, 1])
+            empty_name = self.get_name(op.type, 'empty')
+            empty_tensor = helper.make_tensor(
+                empty_name,
+                onnx_pb.TensorProto.FLOAT, (0, ),
+                np.array([]).astype('float32'),
+                raw=False)
+            empty_node = helper.make_node(
+                'Constant', [], outputs=[empty_name], value=empty_tensor)
+            shape_name0 = self.get_name(op.type, 'shape')
+            shape_node0 = helper.make_node(
+                'Shape', inputs=op.input('X'), outputs=[shape_name0])
+            starts_name = self.get_name(op.type, 'slice.starts')
+            starts_node = self.make_constant_node(
+                starts_name, onnx_pb.TensorProto.INT64, [0])
+            ends_name = self.get_name(op.type, 'slice.ends')
+            ends_node = self.make_constant_node(ends_name,
+                                                onnx_pb.TensorProto.INT64, [2])
+            shape_name1 = self.get_name(op.type, 'shape')
+            shape_node1 = helper.make_node(
+                'Slice',
+                inputs=[shape_name0, starts_name, ends_name],
+                outputs=[shape_name1])
+            node_list.extend([
+                roi_node, empty_node, shape_node0, starts_node, ends_node,
+                shape_node1
+            ])
+            if 'OutSize' in input_names and len(op.input('OutSize')) > 0:
+                cast_shape_name = self.get_name(op.type, "shape.cast")
+                cast_shape_node = helper.make_node(
+                    'Cast',
+                    inputs=op.input('OutSize'),
+                    outputs=[cast_shape_name],
+                    to=onnx_pb.TensorProto.INT64)
+                node_list.append(cast_shape_node)
+            else:
+                concat_shape_name = self.get_name(op.type, "shape.concat")
+                concat_shape_node = helper.make_node(
+                    "Concat",
+                    inputs=op.input('SizeTensor'),
+                    outputs=[concat_shape_name],
+                    axis=0)
+                cast_shape_name = self.get_name(op.type, "shape.cast")
+                cast_shape_node = helper.make_node(
+                    'Cast',
+                    inputs=[concat_shape_name],
+                    outputs=[cast_shape_name],
+                    to=onnx_pb.TensorProto.INT64)
+                node_list.extend([concat_shape_node, cast_shape_node])
+            shape_name3 = self.get_name(op.type, "shape.concat")
+            shape_node3 = helper.make_node(
+                'Concat',
+                inputs=[shape_name1, cast_shape_name],
+                outputs=[shape_name3],
+                axis=0)
+            result_node = helper.make_node(
+                'Resize',
+                inputs=[op.input('X')[0], roi_name, empty_name, shape_name3],
+                outputs=op.output('Out'),
+                mode='linear',
+                coordinate_transformation_mode=coordinate_transformation_mode)
+            node_list.extend([shape_node3, result_node])
+            return node_list
+        elif 'Scale' in input_names and len(op.input('Scale')) > 0:
+            node = helper.make_node(
+                'Resize',
+                inputs=[op.input('X')[0], op.input('Scale')[0]],
+                outputs=op.output('Out'),
+                mode='linear',
+                coordinate_transformation_mode=coordinate_transformation_mode)
+        else:
+            out_shape = [op.attr('out_h'), op.attr('out_w')]
+            scale = op.attr('scale')
+            if out_shape.count(-1) > 0:
+                scale_name = self.get_name(op.type, 'scale')
+                scale_node = self.make_constant_node(scale_name,
+                                                     onnx_pb.TensorProto.FLOAT,
+                                                     [1, 1, scale, scale])
+                roi_name = self.get_name(op.type, 'roi')
+                roi_node = self.make_constant_node(roi_name,
+                                                   onnx_pb.TensorProto.FLOAT,
+                                                   [1, 1, 1, 1, 1, 1, 1, 1])
+                node = helper.make_node(
+                    'Resize',
+                    inputs=[op.input('X')[0], roi_name, scale_name],
+                    outputs=op.output('Out'),
+                    mode='nearest',
+                    coordinate_transformation_mode=coordinate_transformation_mode
+                )
+                return [scale_node, roi_node, node]
+            else:
+                raise Exception("Unexpected situation happend")
+        return node
+
+    def nearest_interp(self, op, block):
+        input_names = op.input_names
+        coordinate_transformation_mode = ''
+        align_corners = op.attr('align_corners')
+        if align_corners:
+            coordinate_transformation_mode = 'align_corners'
+        else:
+            coordinate_transformation_mode = 'asymmetric'
+        if 'OutSize' in input_names and len(op.input('OutSize')) > 0:
+            node = helper.make_node(
+                'Resize',
+                inputs=[op.input('X')[0], '', op.input('OutSize')[0]],
+                outputs=op.output('Out'),
+                mode='nearest',
+                coordinate_transformation_mode=coordinate_transformation_mode)
+        elif 'Scale' in input_names and len(op.input('Scale')) > 0:
+            node = helper.make_node(
+                'Resize',
+                inputs=[op.input('X')[0], op.input('Scale')[0]],
+                outputs=op.output('Out'),
+                mode='nearest',
+                coordinate_transformation_mode=coordinate_transformation_mode)
+        else:
+            out_shape = [op.attr('out_h'), op.attr('out_w')]
+            scale = op.attr('scale')
+            if out_shape.count(-1) > 0:
+                scale_name = self.get_name(op.type, 'scale')
+                scale_node = self.make_constant_node(scale_name,
+                                                     onnx_pb.TensorProto.FLOAT,
+                                                     [1, 1, scale, scale])
+                roi_name = self.get_name(op.type, 'roi')
+                roi_node = self.make_constant_node(roi_name,
+                                                   onnx_pb.TensorProto.FLOAT,
+                                                   [1, 1, 1, 1, 1, 1, 1, 1])
+                node = helper.make_node(
+                    'Resize',
+                    inputs=[op.input('X')[0], roi_name, scale_name],
+                    outputs=op.output('Out'),
+                    mode='nearest',
+                    coordinate_transformation_mode=coordinate_transformation_mode
+                )
+                return [scale_node, roi_node, node]
+            else:
+                raise Exception("Unexpected situation happend")
+        return node
+
+    def hard_swish(self, op, block):
+        min_name = self.get_name(op.type, 'min')
+        max_name = self.get_name(op.type, 'max')
+        scale_name = self.get_name(op.type, 'scale')
+        offset_name = self.get_name(op.type, 'offset')
+        min_node = self.make_constant_node(min_name, onnx_pb.TensorProto.FLOAT,
+                                           0)
+        max_node = self.make_constant_node(max_name, onnx_pb.TensorProto.FLOAT,
+                                           op.attr('threshold'))
+        scale_node = self.make_constant_node(scale_name,
+                                             onnx_pb.TensorProto.FLOAT,
+                                             op.attr('scale'))
+        offset_node = self.make_constant_node(offset_name,
+                                              onnx_pb.TensorProto.FLOAT,
+                                              op.attr('offset'))
+
+        name0 = self.get_name(op.type, 'add')
+        node0 = helper.make_node(
+            'Add', inputs=[op.input('X')[0], offset_name], outputs=[name0])
+        name1 = self.get_name(op.type, 'relu')
+        node1 = helper.make_node(
+            'Clip',
+            inputs=[name0, min_name, max_name],
+            outputs=[name1], )
+        name2 = self.get_name(op.type, 'mul')
+        node2 = helper.make_node(
+            'Mul', inputs=[op.input('X')[0], name1], outputs=[name2])
+        node3 = helper.make_node(
+            'Div', inputs=[name2, scale_name], outputs=op.output('Out'))
+        return [
+            min_node, max_node, scale_node, offset_node, node0, node1, node2,
+            node3
+        ]
+
+    def im2sequence(self, op, block):
+        from .paddle_custom_layer.im2sequence import im2sequence
+        return im2sequence(op, block)
+
+    def yolo_box(self, op, block):
+        from .paddle_custom_layer.yolo_box import yolo_box
+        return yolo_box(op, block)
+
+    def multiclass_nms(self, op, block):
+        from .paddle_custom_layer.multiclass_nms import multiclass_nms
+        return multiclass_nms(op, block)

x2paddle/op_mapper/paddle2onnx/opset11/paddle_custom_layer/im2sequence.py

+# Copyright (c) 2020  PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License"
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import onnx
+import numpy as np
+from onnx import onnx_pb, helper
+from x2paddle.op_mapper.paddle2onnx.opset10.paddle_custom_layer.im2sequence import im2sequence as im2sequence10
+
+
+def im2sequence(op, block):
+    return im2sequence10(op, block)

x2paddle/op_mapper/paddle_custom_layer/multiclass_nms.py → x2paddle/op_mapper/paddle2onnx/opset11/paddle_custom_layer/multiclass_nms.py

@@ -125,7 +125,7 @@ def multiclass_nms(op, block):
                 vals=[value]))
         node_list.append(node)
 
-    # Ine this code block, we will deocde the raw score data, reshape N * C * M to 1 * N*C*M
+    # In this code block, we will deocde the raw score data, reshape N * C * M to 1 * N*C*M
     # and the same time, decode the select indices to 1 * D, gather the select_indices
     outputs_gather_1 = [result_name + "@gather_1"]
     node_gather_1 = onnx.helper.make_node(
@@ -405,12 +405,36 @@ def multiclass_nms(op, block):
 
     inputs_concat_final_results = outputs_cast_topk_class + outputs_unsqueeze_topk_scores +\
         outputs_gather_select_boxes
-    outputs_concat_final_results = outputs['Out']
-    node_concat_final_results = onnx.helper.make_node(
+    outputs_sort_by_socre_results = [result_name + "@concat_topk_scores"]
+    node_sort_by_socre_results = onnx.helper.make_node(
         'Concat',
         inputs=inputs_concat_final_results,
-        outputs=outputs_concat_final_results,
+        outputs=outputs_sort_by_socre_results,
         axis=2)
-    node_list.append(node_concat_final_results)
+    node_list.append(node_sort_by_socre_results)
 
+    # select topk classes indices
+    outputs_squeeze_cast_topk_class = [result_name + "@squeeze_cast_topk_class"]
+    node_squeeze_cast_topk_class = onnx.helper.make_node(
+        'Squeeze',
+        inputs=outputs_cast_topk_class,
+        outputs=outputs_squeeze_cast_topk_class,
+        axes=[0, 2])
+    node_list.append(node_squeeze_cast_topk_class)
+    outputs_topk_select_classes_indices = [result_name + "@topk_select_topk_classes_scores",\
+        result_name + "@topk_select_topk_classes_indices"]
+    node_topk_select_topk_indices = onnx.helper.make_node(
+        'TopK',
+        inputs=outputs_squeeze_cast_topk_class + outputs_cast_topk_indices,
+        outputs=outputs_topk_select_classes_indices,
+        largest=0)
+    node_list.append(node_topk_select_topk_indices)
+    outputs_concat_final_results = outputs['Out']
+    node_concat_final_results = onnx.helper.make_node(
+        'Gather',
+        inputs=outputs_sort_by_socre_results +
+        [outputs_topk_select_classes_indices[1]],
+        outputs=outputs_concat_final_results,
+        axis=1)
+    node_list.append(node_concat_final_results)
     return node_list

x2paddle/op_mapper/paddle2onnx/opset11/paddle_custom_layer/yolo_box.py

+# Copyright (c) 2020  PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License"
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import onnx
+import numpy as np
+from onnx import onnx_pb, helper
+
+MAX_FLOAT = np.asarray([255, 255, 127, 127], dtype=np.uint8).view(np.float32)[0]
+
+
+def get_old_name(arg, name_prefix=''):
+    prefix_index = arg.find(name_prefix)
+
+    if prefix_index != -1:
+        last_prefix = arg[len(name_prefix):]
+    else:
+        last_prefix = arg
+    idx = last_prefix.find('@')
+    if idx != -1:
+        last_prefix = last_prefix[:idx]
+    return name_prefix + last_prefix
+
+
+def yolo_box(op, block):
+    inputs = dict()
+    outputs = dict()
+    attrs = dict()
+    for name in op.input_names:
+        inputs[name] = op.input(name)
+    for name in op.output_names:
+        outputs[name] = op.output(name)
+    for name in op.attr_names:
+        attrs[name] = op.attr(name)
+    model_name = outputs['Boxes'][0]
+    input_shape = block.vars[get_old_name(inputs['X'][0])].shape
+    image_size = inputs['ImgSize']
+    input_height = input_shape[2]
+    input_width = input_shape[3]
+
+    class_num = attrs['class_num']
+    anchors = attrs['anchors']
+    num_anchors = int(len(anchors)) // 2
+    downsample_ratio = attrs['downsample_ratio']
+    input_size = input_height * downsample_ratio
+    conf_thresh = attrs['conf_thresh']
+    conf_thresh_mat = np.ones([num_anchors * input_height *
+                               input_width]) * conf_thresh
+
+    node_list = []
+    im_outputs = []
+
+    x_shape = [1, num_anchors, 5 + class_num, input_height, input_width]
+    name_x_shape = [model_name + "@x_shape"]
+    node_x_shape = onnx.helper.make_node(
+        'Constant',
+        inputs=[],
+        outputs=name_x_shape,
+        value=onnx.helper.make_tensor(
+            name=name_x_shape[0] + "@const",
+            data_type=onnx.TensorProto.INT64,
+            dims=[5],
+            vals=x_shape))
+    node_list.append(node_x_shape)
+
+    outputs_x_reshape = [model_name + "@reshape"]
+    node_x_reshape = onnx.helper.make_node(
+        'Reshape', inputs=inputs['X'] + name_x_shape, outputs=outputs_x_reshape)
+    node_list.append(node_x_reshape)
+
+    outputs_x_transpose = [model_name + "@x_transpose"]
+    node_x_transpose = onnx.helper.make_node(
+        'Transpose',
+        inputs=outputs_x_reshape,
+        outputs=outputs_x_transpose,
+        perm=[0, 1, 3, 4, 2])
+    node_list.append(node_x_transpose)
+
+    range_x = []
+    range_y = []
+    for i in range(0, input_width):
+        range_x.append(i)
+    for j in range(0, input_height):
+        range_y.append(j)
+
+    name_range_x = [model_name + "@range_x"]
+    node_range_x = onnx.helper.make_node(
+        'Constant',
+        inputs=[],
+        outputs=name_range_x,
+        value=onnx.helper.make_tensor(
+            name=name_range_x[0] + "@const",
+            data_type=onnx.TensorProto.FLOAT,
+            dims=[input_width],
+            vals=range_x))
+    node_list.append(node_range_x)
+
+    name_range_y = [model_name + "@range_y"]
+    node_range_y = onnx.helper.make_node(
+        'Constant',
+        inputs=[],
+        outputs=name_range_y,
+        value=onnx.helper.make_tensor(
+            name=name_range_y[0] + "@const",
+            data_type=onnx.TensorProto.FLOAT,
+            dims=[input_height],
+            vals=range_y))
+    node_list.append(node_range_y)
+
+    range_x_new_shape = [1, input_width]
+    range_y_new_shape = [input_height, 1]
+
+    name_range_x_new_shape = [model_name + "@range_x_new_shape"]
+    node_range_x_new_shape = onnx.helper.make_node(
+        'Constant',
+        inputs=[],
+        outputs=name_range_x_new_shape,
+        value=onnx.helper.make_tensor(
+            name=name_range_x_new_shape[0] + "@const",
+            data_type=onnx.TensorProto.INT64,
+            dims=[len(range_x_new_shape)],
+            vals=range_x_new_shape))
+    node_list.append(node_range_x_new_shape)
+
+    name_range_y_new_shape = [model_name + "@range_y_new_shape"]
+    node_range_y_new_shape = onnx.helper.make_node(
+        'Constant',
+        inputs=[],
+        outputs=name_range_y_new_shape,
+        value=onnx.helper.make_tensor(
+            name=name_range_y_new_shape[0] + "@const",
+            data_type=onnx.TensorProto.INT64,
+            dims=[len(range_y_new_shape)],
+            vals=range_y_new_shape))
+    node_list.append(node_range_y_new_shape)
+
+    outputs_range_x_reshape = [model_name + "@range_x_reshape"]
+    node_range_x_reshape = onnx.helper.make_node(
+        'Reshape',
+        inputs=name_range_x + name_range_x_new_shape,
+        outputs=outputs_range_x_reshape)
+    node_list.append(node_range_x_reshape)
+
+    outputs_range_y_reshape = [model_name + "@range_y_reshape"]
+    node_range_y_reshape = onnx.helper.make_node(
+        'Reshape',
+        inputs=name_range_y + name_range_y_new_shape,
+        outputs=outputs_range_y_reshape)
+    node_list.append(node_range_y_reshape)
+
+    outputs_grid_x = [model_name + "@grid_x"]
+    node_grid_x = onnx.helper.make_node(
+        "Tile",
+        inputs=outputs_range_x_reshape + name_range_y_new_shape,
+        outputs=outputs_grid_x)
+    node_list.append(node_grid_x)
+
+    outputs_grid_y = [model_name + "@grid_y"]
+    node_grid_y = onnx.helper.make_node(
+        "Tile",
+        inputs=outputs_range_y_reshape + name_range_x_new_shape,
+        outputs=outputs_grid_y)
+    node_list.append(node_grid_y)
+
+    outputs_box_x = [model_name + "@box_x"]
+    outputs_box_y = [model_name + "@box_y"]
+    outputs_box_w = [model_name + "@box_w"]
+    outputs_box_h = [model_name + "@box_h"]
+    outputs_conf = [model_name + "@conf"]
+    outputs_prob = [model_name + "@prob"]
+
+    node_split_input = onnx.helper.make_node(
+        "Split",
+        inputs=outputs_x_transpose,
+        outputs=outputs_box_x + outputs_box_y + outputs_box_w\
+                + outputs_box_h + outputs_conf + outputs_prob,
+        axis=-1,
+        split=[1, 1, 1, 1, 1, class_num])
+    node_list.append(node_split_input)
+
+    outputs_box_x_sigmoid = [model_name + "@box_x_sigmoid"]
+    outputs_box_y_sigmoid = [model_name + "@box_y_sigmoid"]
+
+    node_box_x_sigmoid = onnx.helper.make_node(
+        "Sigmoid", inputs=outputs_box_x, outputs=outputs_box_x_sigmoid)
+    node_list.append(node_box_x_sigmoid)
+
+    node_box_y_sigmoid = onnx.helper.make_node(
+        "Sigmoid", inputs=outputs_box_y, outputs=outputs_box_y_sigmoid)
+    node_list.append(node_box_y_sigmoid)
+
+    outputs_box_x_squeeze = [model_name + "@box_x_squeeze"]
+    outputs_box_y_squeeze = [model_name + "@box_y_squeeze"]
+
+    node_box_x_squeeze = onnx.helper.make_node(
+        'Squeeze',
+        inputs=outputs_box_x_sigmoid,
+        outputs=outputs_box_x_squeeze,
+        axes=[4])
+    node_list.append(node_box_x_squeeze)
+
+    node_box_y_squeeze = onnx.helper.make_node(
+        'Squeeze',
+        inputs=outputs_box_y_sigmoid,
+        outputs=outputs_box_y_squeeze,
+        axes=[4])
+    node_list.append(node_box_y_squeeze)
+
+    outputs_box_x_add_grid = [model_name + "@box_x_add_grid"]
+    outputs_box_y_add_grid = [model_name + "@box_y_add_grid"]
+
+    node_box_x_add_grid = onnx.helper.make_node(
+        "Add",
+        inputs=outputs_grid_x + outputs_box_x_squeeze,
+        outputs=outputs_box_x_add_grid)
+    node_list.append(node_box_x_add_grid)
+
+    node_box_y_add_grid = onnx.helper.make_node(
+        "Add",
+        inputs=outputs_grid_y + outputs_box_y_squeeze,
+        outputs=outputs_box_y_add_grid)
+    node_list.append(node_box_y_add_grid)
+
+    name_input_h = [model_name + "@input_h"]
+    name_input_w = [model_name + "@input_w"]
+
+    node_input_h = onnx.helper.make_node(
+        'Constant',
+        inputs=[],
+        outputs=name_input_h,
+        value=onnx.helper.make_tensor(
+            name=name_input_w[0] + "@const",
+            data_type=onnx.TensorProto.FLOAT,
+            dims=(),
+            vals=[input_height]))
+    node_list.append(node_input_h)
+
+    node_input_w = onnx.helper.make_node(
+        'Constant',
+        inputs=[],
+        outputs=name_input_w,
+        value=onnx.helper.make_tensor(
+            name=name_input_w[0] + "@const",
+            data_type=onnx.TensorProto.FLOAT,
+            dims=(),
+            vals=[input_width]))
+    node_list.append(node_input_w)
+
+    outputs_box_x_encode = [model_name + "@box_x_encode"]
+    outputs_box_y_encode = [model_name + "@box_y_encode"]
+
+    node_box_x_encode = onnx.helper.make_node(
+        'Div',
+        inputs=outputs_box_x_add_grid + name_input_w,
+        outputs=outputs_box_x_encode)
+    node_list.append(node_box_x_encode)
+
+    node_box_y_encode = onnx.helper.make_node(
+        'Div',
+        inputs=outputs_box_y_add_grid + name_input_h,
+        outputs=outputs_box_y_encode)
+    node_list.append(node_box_y_encode)
+
+    name_anchor_tensor = [model_name + "@anchor_tensor"]
+    node_anchor_tensor = onnx.helper.make_node(
+        "Constant",
+        inputs=[],
+        outputs=name_anchor_tensor,
+        value=onnx.helper.make_tensor(
+            name=name_anchor_tensor[0] + "@const",
+            data_type=onnx.TensorProto.FLOAT,
+            dims=[len(anchors)],
+            vals=anchors))
+    node_list.append(node_anchor_tensor)
+
+    anchor_shape = [int(num_anchors), 2]
+    name_anchor_shape = [model_name + "@anchor_shape"]
+    node_anchor_shape = onnx.helper.make_node(
+        "Constant",
+        inputs=[],
+        outputs=name_anchor_shape,
+        value=onnx.helper.make_tensor(
+            name=name_anchor_shape[0] + "@const",
+            data_type=onnx.TensorProto.INT64,
+            dims=[2],
+            vals=anchor_shape))
+    node_list.append(node_anchor_shape)
+
+    outputs_anchor_tensor_reshape = [model_name + "@anchor_tensor_reshape"]
+    node_anchor_tensor_reshape = onnx.helper.make_node(
+        "Reshape",
+        inputs=name_anchor_tensor + name_anchor_shape,
+        outputs=outputs_anchor_tensor_reshape)
+    node_list.append(node_anchor_tensor_reshape)
+
+    name_input_size = [model_name + "@input_size"]
+    node_input_size = onnx.helper.make_node(
+        "Constant",
+        inputs=[],
+        outputs=name_input_size,
+        value=onnx.helper.make_tensor(
+            name=name_input_size[0] + "@const",
+            data_type=onnx.TensorProto.FLOAT,
+            dims=(),
+            vals=[input_size]))
+    node_list.append(node_input_size)
+
+    outputs_anchors_div_input_size = [model_name + "@anchors_div_input_size"]
+    node_anchors_div_input_size = onnx.helper.make_node(
+        "Div",
+        inputs=outputs_anchor_tensor_reshape + name_input_size,
+        outputs=outputs_anchors_div_input_size)
+    node_list.append(node_anchors_div_input_size)
+
+    outputs_anchor_w = [model_name + "@anchor_w"]
+    outputs_anchor_h = [model_name + "@anchor_h"]
+
+    node_anchor_split = onnx.helper.make_node(
+        'Split',
+        inputs=outputs_anchors_div_input_size,
+        outputs=outputs_anchor_w + outputs_anchor_h,
+        axis=1,
+        split=[1, 1])
+    node_list.append(node_anchor_split)
+
+    new_anchor_shape = [1, int(num_anchors), 1, 1]
+    name_new_anchor_shape = [model_name + "@new_anchor_shape"]
+    node_new_anchor_shape = onnx.helper.make_node(
+        'Constant',
+        inputs=[],
+        outputs=name_new_anchor_shape,
+        value=onnx.helper.make_tensor(
+            name=name_new_anchor_shape[0] + "@const",
+            data_type=onnx.TensorProto.INT64,
+            dims=[len(new_anchor_shape)],
+            vals=new_anchor_shape))
+    node_list.append(node_new_anchor_shape)
+
+    outputs_anchor_w_reshape = [model_name + "@anchor_w_reshape"]
+    outputs_anchor_h_reshape = [model_name + "@anchor_h_reshape"]
+
+    node_anchor_w_reshape = onnx.helper.make_node(
+        'Reshape',
+        inputs=outputs_anchor_w + name_new_anchor_shape,
+        outputs=outputs_anchor_w_reshape)
+    node_list.append(node_anchor_w_reshape)
+
+    node_anchor_h_reshape = onnx.helper.make_node(
+        'Reshape',
+        inputs=outputs_anchor_h + name_new_anchor_shape,
+        outputs=outputs_anchor_h_reshape)
+    node_list.append(node_anchor_h_reshape)
+
+    outputs_box_w_squeeze = [model_name + "@box_w_squeeze"]
+    node_box_w_squeeze = onnx.helper.make_node(
+        'Squeeze',
+        inputs=outputs_box_w,
+        outputs=outputs_box_w_squeeze,
+        axes=[4])
+    node_list.append(node_box_w_squeeze)
+
+    outputs_box_h_squeeze = [model_name + "@box_h_squeeze"]
+    node_box_h_squeeze = onnx.helper.make_node(
+        'Squeeze',
+        inputs=outputs_box_h,
+        outputs=outputs_box_h_squeeze,
+        axes=[4])
+    node_list.append(node_box_h_squeeze)
+
+    outputs_box_w_exp = [model_name + "@box_w_exp"]
+    node_box_w_exp = onnx.helper.make_node(
+        "Exp", inputs=outputs_box_w_squeeze, outputs=outputs_box_w_exp)
+    node_list.append(node_box_w_exp)
+
+    outputs_box_h_exp = [model_name + "@box_h_exp"]
+    node_box_h_exp = onnx.helper.make_node(
+        "Exp", inputs=outputs_box_h_squeeze, outputs=outputs_box_h_exp)
+    node_list.append(node_box_h_exp)
+
+    outputs_box_w_encode = [model_name + "box_w_encode"]
+    outputs_box_h_encode = [model_name + "box_h_encode"]
+
+    node_box_w_encode = onnx.helper.make_node(
+        'Mul',
+        inputs=outputs_box_w_exp + outputs_anchor_w_reshape,
+        outputs=outputs_box_w_encode)
+    node_list.append(node_box_w_encode)
+
+    node_box_h_encode = onnx.helper.make_node(
+        'Mul',
+        inputs=outputs_box_h_exp + outputs_anchor_h_reshape,
+        outputs=outputs_box_h_encode)
+    node_list.append(node_box_h_encode)
+
+    outputs_conf_sigmoid = [model_name + "@conf_sigmoid"]
+    node_conf_sigmoid = onnx.helper.make_node(
+        'Sigmoid', inputs=outputs_conf, outputs=outputs_conf_sigmoid)
+    node_list.append(node_conf_sigmoid)
+
+    name_conf_thresh = [model_name + "@conf_thresh"]
+    node_conf_thresh = onnx.helper.make_node(
+        'Constant',
+        inputs=[],
+        outputs=name_conf_thresh,
+        value=onnx.helper.make_tensor(
+            name=name_conf_thresh[0] + "@const",
+            data_type=onnx.TensorProto.FLOAT,
+            dims=[num_anchors * input_height * input_width],
+            vals=conf_thresh_mat))
+    node_list.append(node_conf_thresh)
+
+    conf_shape = [1, int(num_anchors), input_height, input_width, 1]
+    name_conf_shape = [model_name + "@conf_shape"]
+    node_conf_shape = onnx.helper.make_node(
+        'Constant',
+        inputs=[],
+        outputs=name_conf_shape,
+        value=onnx.helper.make_tensor(
+            name=name_conf_shape[0] + "@const",
+            data_type=onnx.TensorProto.INT64,
+            dims=[len(conf_shape)],
+            vals=conf_shape))
+    node_list.append(node_conf_shape)
+
+    outputs_conf_thresh_reshape = [model_name + "@conf_thresh_reshape"]
+    node_conf_thresh_reshape = onnx.helper.make_node(
+        'Reshape',
+        inputs=name_conf_thresh + name_conf_shape,
+        outputs=outputs_conf_thresh_reshape)
+    node_list.append(node_conf_thresh_reshape)
+
+    outputs_conf_sub = [model_name + "@conf_sub"]
+    node_conf_sub = onnx.helper.make_node(
+        'Sub',
+        inputs=outputs_conf_sigmoid + outputs_conf_thresh_reshape,
+        outputs=outputs_conf_sub)
+    node_list.append(node_conf_sub)
+
+    outputs_conf_clip = [model_name + "@conf_clip"]
+    node_conf_clip = onnx.helper.make_node(
+        'Clip', inputs=outputs_conf_sub, outputs=outputs_conf_clip)
+    node_list.append(node_conf_clip)
+
+    zeros = [0]
+    name_zeros = [model_name + "@zeros"]
+    node_zeros = onnx.helper.make_node(
+        'Constant',
+        inputs=[],
+        outputs=name_zeros,
+        value=onnx.helper.make_tensor(
+            name=name_zeros[0] + "@const",
+            data_type=onnx.TensorProto.FLOAT,
+            dims=(),
+            vals=zeros))
+    node_list.append(node_zeros)
+
+    outputs_conf_clip_bool = [model_name + "@conf_clip_bool"]
+    node_conf_clip_bool = onnx.helper.make_node(
+        'Greater',
+        inputs=outputs_conf_clip + name_zeros,
+        outputs=outputs_conf_clip_bool)
+    node_list.append(node_conf_clip_bool)
+
+    outputs_conf_clip_cast = [model_name + "@conf_clip_cast"]
+    node_conf_clip_cast = onnx.helper.make_node(
+        'Cast',
+        inputs=outputs_conf_clip_bool,
+        outputs=outputs_conf_clip_cast,
+        to=1)
+    node_list.append(node_conf_clip_cast)
+
+    outputs_conf_set_zero = [model_name + "@conf_set_zero"]
+    node_conf_set_zero = onnx.helper.make_node(
+        'Mul',
+        inputs=outputs_conf_sigmoid + outputs_conf_clip_cast,
+        outputs=outputs_conf_set_zero)
+    node_list.append(node_conf_set_zero)
+
+    outputs_prob_sigmoid = [model_name + "@prob_sigmoid"]
+    node_prob_sigmoid = onnx.helper.make_node(
+        'Sigmoid', inputs=outputs_prob, outputs=outputs_prob_sigmoid)
+    node_list.append(node_prob_sigmoid)
+
+    new_shape = [1, int(num_anchors), input_height, input_width, 1]
+    name_new_shape = [model_name + "@new_shape"]
+    node_new_shape = onnx.helper.make_node(
+        'Constant',
+        inputs=[],
+        outputs=name_new_shape,
+        value=onnx.helper.make_tensor(
+            name=name_new_shape[0] + "@const",
+            data_type=onnx.TensorProto.INT64,
+            dims=[len(new_shape)],
+            vals=new_shape))
+    node_list.append(node_new_shape)
+
+    outputs_conf_new_shape = [model_name + "@_conf_new_shape"]
+    node_conf_new_shape = onnx.helper.make_node(
+        'Reshape',
+        inputs=outputs_conf_set_zero + name_new_shape,
+        outputs=outputs_conf_new_shape)
+    node_list.append(node_conf_new_shape)
+
+    outputs_score = [model_name + "@score"]
+    node_score = onnx.helper.make_node(
+        'Mul',
+        inputs=outputs_prob_sigmoid + outputs_conf_new_shape,
+        outputs=outputs_score)
+    node_list.append(node_score)
+
+    outputs_conf_bool = [model_name + "@conf_bool"]
+    node_conf_bool = onnx.helper.make_node(
+        'Greater',
+        inputs=outputs_conf_new_shape + name_zeros,
+        outputs=outputs_conf_bool)
+    node_list.append(node_conf_bool)
+
+    outputs_box_x_new_shape = [model_name + "@box_x_new_shape"]
+    node_box_x_new_shape = onnx.helper.make_node(
+        'Reshape',
+        inputs=outputs_box_x_encode + name_new_shape,
+        outputs=outputs_box_x_new_shape)
+    node_list.append(node_box_x_new_shape)
+
+    outputs_box_y_new_shape = [model_name + "@box_y_new_shape"]
+    node_box_y_new_shape = onnx.helper.make_node(
+        'Reshape',
+        inputs=outputs_box_y_encode + name_new_shape,
+        outputs=outputs_box_y_new_shape)
+    node_list.append(node_box_y_new_shape)
+
+    outputs_box_w_new_shape = [model_name + "@box_w_new_shape"]
+    node_box_w_new_shape = onnx.helper.make_node(
+        'Reshape',
+        inputs=outputs_box_w_encode + name_new_shape,
+        outputs=outputs_box_w_new_shape)
+    node_list.append(node_box_w_new_shape)
+
+    outputs_box_h_new_shape = [model_name + "@box_h_new_shape"]
+    node_box_h_new_shape = onnx.helper.make_node(
+        'Reshape',
+        inputs=outputs_box_h_encode + name_new_shape,
+        outputs=outputs_box_h_new_shape)
+    node_list.append(node_box_h_new_shape)
+
+    outputs_pred_box = [model_name + "@pred_box"]
+    node_pred_box = onnx.helper.make_node(
+        'Concat',
+        inputs=outputs_box_x_new_shape + outputs_box_y_new_shape + \
+               outputs_box_w_new_shape + outputs_box_h_new_shape,
+        outputs=outputs_pred_box,
+        axis=4)
+    node_list.append(node_pred_box)
+
+    outputs_conf_cast = [model_name + "conf_cast"]
+    node_conf_cast = onnx.helper.make_node(
+        'Cast', inputs=outputs_conf_bool, outputs=outputs_conf_cast, to=1)
+    node_list.append(node_conf_cast)
+
+    outputs_pred_box_mul_conf = [model_name + "@pred_box_mul_conf"]
+    node_pred_box_mul_conf = onnx.helper.make_node(
+        'Mul',
+        inputs=outputs_pred_box + outputs_conf_cast,
+        outputs=outputs_pred_box_mul_conf)
+    node_list.append(node_pred_box_mul_conf)
+
+    box_shape = [1, int(num_anchors) * input_height * input_width, 4]
+    name_box_shape = [model_name + "@box_shape"]
+    node_box_shape = onnx.helper.make_node(
+        'Constant',
+        inputs=[],
+        outputs=name_box_shape,
+        value=onnx.helper.make_tensor(
+            name=name_box_shape[0] + "@const",
+            data_type=onnx.TensorProto.INT64,
+            dims=[len(box_shape)],
+            vals=box_shape))
+    node_list.append(node_box_shape)
+
+    outputs_pred_box_new_shape = [model_name + "@pred_box_new_shape"]
+    node_pred_box_new_shape = onnx.helper.make_node(
+        'Reshape',
+        inputs=outputs_pred_box_mul_conf + name_box_shape,
+        outputs=outputs_pred_box_new_shape)
+    node_list.append(node_pred_box_new_shape)
+
+    outputs_pred_box_x = [model_name + "@_pred_box_x"]
+    outputs_pred_box_y = [model_name + "@_pred_box_y"]
+    outputs_pred_box_w = [model_name + "@_pred_box_w"]
+    outputs_pred_box_h = [model_name + "@_pred_box_h"]
+
+    node_pred_box_split = onnx.helper.make_node(
+        'Split',
+        inputs=outputs_pred_box_new_shape,
+        outputs=outputs_pred_box_x + outputs_pred_box_y + outputs_pred_box_w +
+        outputs_pred_box_h,
+        axis=2)
+    node_list.append(node_pred_box_split)
+
+    name_number_two = [model_name + "@number_two"]
+    node_number_two = onnx.helper.make_node(
+        "Constant",
+        inputs=[],
+        outputs=name_number_two,
+        value=onnx.helper.make_tensor(
+            name=name_number_two[0] + "@const",
+            data_type=onnx.TensorProto.FLOAT,
+            dims=(),
+            vals=[2]))
+    node_list.append(node_number_two)
+
+    outputs_half_w = [model_name + "@half_w"]
+    node_half_w = onnx.helper.make_node(
+        "Div",
+        inputs=outputs_pred_box_w + name_number_two,
+        outputs=outputs_half_w)
+    node_list.append(node_half_w)
+
+    outputs_half_h = [model_name + "@half_h"]
+    node_half_h = onnx.helper.make_node(
+        "Div",
+        inputs=outputs_pred_box_h + name_number_two,
+        outputs=outputs_half_h)
+    node_list.append(node_half_h)
+
+    outputs_pred_box_x1 = [model_name + "@pred_box_x1"]
+    node_pred_box_x1 = onnx.helper.make_node(
+        'Sub',
+        inputs=outputs_pred_box_x + outputs_half_w,
+        outputs=outputs_pred_box_x1)
+    node_list.append(node_pred_box_x1)
+
+    outputs_pred_box_y1 = [model_name + "@pred_box_y1"]
+    node_pred_box_y1 = onnx.helper.make_node(
+        'Sub',
+        inputs=outputs_pred_box_y + outputs_half_h,
+        outputs=outputs_pred_box_y1)
+    node_list.append(node_pred_box_y1)
+
+    outputs_pred_box_x2 = [model_name + "@pred_box_x2"]
+    node_pred_box_x2 = onnx.helper.make_node(
+        'Add',
+        inputs=outputs_pred_box_x + outputs_half_w,
+        outputs=outputs_pred_box_x2)
+    node_list.append(node_pred_box_x2)
+
+    outputs_pred_box_y2 = [model_name + "@pred_box_y2"]
+    node_pred_box_y2 = onnx.helper.make_node(
+        'Add',
+        inputs=outputs_pred_box_y + outputs_half_h,
+        outputs=outputs_pred_box_y2)
+    node_list.append(node_pred_box_y2)
+
+    outputs_sqeeze_image_size = [model_name + "@sqeeze_image_size"]
+    node_sqeeze_image_size = onnx.helper.make_node(
+        "Squeeze",
+        axes=[0],
+        inputs=image_size,
+        outputs=outputs_sqeeze_image_size)
+    node_list.append(node_sqeeze_image_size)
+
+    output_img_height = [model_name + "@img_height"]
+    output_img_width = [model_name + "@img_width"]
+    node_image_size_split = onnx.helper.make_node(
+        "Split",
+        inputs=outputs_sqeeze_image_size,
+        outputs=output_img_height + output_img_width,
+        axis=-1,
+        split=[1, 1])
+    node_list.append(node_image_size_split)
+
+    output_img_width_cast = [model_name + "@img_width_cast"]
+    node_img_width_cast = onnx.helper.make_node(
+        'Cast', inputs=output_img_width, outputs=output_img_width_cast, to=1)
+    node_list.append(node_img_width_cast)
+
+    output_img_height_cast = [model_name + "@img_height_cast"]
+    node_img_height_cast = onnx.helper.make_node(
+        'Cast', inputs=output_img_height, outputs=output_img_height_cast, to=1)
+    node_list.append(node_img_height_cast)
+
+    outputs_pred_box_x1_decode = [model_name + "@pred_box_x1_decode"]
+    outputs_pred_box_y1_decode = [model_name + "@pred_box_y1_decode"]
+    outputs_pred_box_x2_decode = [model_name + "@pred_box_x2_decode"]
+    outputs_pred_box_y2_decode = [model_name + "@pred_box_y2_decode"]
+
+    node_pred_box_x1_decode = onnx.helper.make_node(
+        'Mul',
+        inputs=outputs_pred_box_x1 + output_img_width_cast,
+        outputs=outputs_pred_box_x1_decode)
+    node_list.append(node_pred_box_x1_decode)
+
+    node_pred_box_y1_decode = onnx.helper.make_node(
+        'Mul',
+        inputs=outputs_pred_box_y1 + output_img_height_cast,
+        outputs=outputs_pred_box_y1_decode)
+    node_list.append(node_pred_box_y1_decode)
+
+    node_pred_box_x2_decode = onnx.helper.make_node(
+        'Mul',
+        inputs=outputs_pred_box_x2 + output_img_width_cast,
+        outputs=outputs_pred_box_x2_decode)
+    node_list.append(node_pred_box_x2_decode)
+
+    node_pred_box_y2_decode = onnx.helper.make_node(
+        'Mul',
+        inputs=outputs_pred_box_y2 + output_img_height_cast,
+        outputs=outputs_pred_box_y2_decode)
+    node_list.append(node_pred_box_y2_decode)
+
+    name_number_one = [model_name + "@one"]
+    node_number_one = onnx.helper.make_node(
+        'Constant',
+        inputs=[],
+        outputs=name_number_one,
+        value=onnx.helper.make_tensor(
+            name=name_number_one[0] + "@const",
+            data_type=onnx.TensorProto.FLOAT,
+            dims=(),
+            vals=[1]))
+    node_list.append(node_number_one)
+
+    output_new_img_height = [model_name + "@new_img_height"]
+    node_new_img_height = onnx.helper.make_node(
+        'Sub',
+        inputs=output_img_height_cast + name_number_one,
+        outputs=output_new_img_height)
+    node_list.append(node_new_img_height)
+
+    output_new_img_width = [model_name + "@new_img_width"]
+    node_new_img_width = onnx.helper.make_node(
+        'Sub',
+        inputs=output_img_width_cast + name_number_one,
+        outputs=output_new_img_width)
+    node_list.append(node_new_img_width)
+
+    outputs_pred_box_x2_sub_w = [model_name + "@pred_box_x2_sub_w"]
+    node_pred_box_x2_sub_w = onnx.helper.make_node(
+        'Sub',
+        inputs=outputs_pred_box_x2_decode + output_new_img_width,
+        outputs=outputs_pred_box_x2_sub_w)
+    node_list.append(node_pred_box_x2_sub_w)
+
+    outputs_pred_box_y2_sub_h = [model_name + "@pred_box_y2_sub_h"]
+    node_pred_box_y2_sub_h = onnx.helper.make_node(
+        'Sub',
+        inputs=outputs_pred_box_y2_decode + output_new_img_height,
+        outputs=outputs_pred_box_y2_sub_h)
+    node_list.append(node_pred_box_y2_sub_h)
+
+    outputs_pred_box_x1_clip = [model_name + "@pred_box_x1_clip"]
+    outputs_pred_box_y1_clip = [model_name + "@pred_box_y1_clip"]
+    outputs_pred_box_x2_clip = [model_name + "@pred_box_x2_clip"]
+    outputs_pred_box_y2_clip = [model_name + "@pred_box_y2_clip"]
+
+    min_const_name = model_name + "@pred_box_min_const"
+    max_const_name = model_name + "@pred_box_max_const"
+
+    min_const = onnx.helper.make_node(
+        'Constant',
+        inputs=[],
+        outputs=[min_const_name],
+        value=onnx.helper.make_tensor(
+            name=min_const_name,
+            data_type=onnx.TensorProto.FLOAT,
+            dims=(),
+            vals=[0.0]))
+    node_list.append(min_const)
+
+    max_const = onnx.helper.make_node(
+        'Constant',
+        inputs=[],
+        outputs=[max_const_name],
+        value=onnx.helper.make_tensor(
+            name=max_const_name,
+            data_type=onnx.TensorProto.FLOAT,
+            dims=(),
+            vals=[MAX_FLOAT]))
+    node_list.append(max_const)
+
+    node_pred_box_x1_clip = onnx.helper.make_node(
+        'Clip',
+        inputs=outputs_pred_box_x1_decode + [min_const_name, max_const_name],
+        outputs=outputs_pred_box_x1_clip)
+    node_list.append(node_pred_box_x1_clip)
+
+    node_pred_box_y1_clip = onnx.helper.make_node(
+        'Clip',
+        inputs=outputs_pred_box_y1_decode + [min_const_name, max_const_name],
+        outputs=outputs_pred_box_y1_clip)
+    node_list.append(node_pred_box_y1_clip)
+
+    node_pred_box_x2_clip = onnx.helper.make_node(
+        'Clip',
+        inputs=outputs_pred_box_x2_sub_w + [min_const_name, max_const_name],
+        outputs=outputs_pred_box_x2_clip)
+    node_list.append(node_pred_box_x2_clip)
+
+    node_pred_box_y2_clip = onnx.helper.make_node(
+        'Clip',
+        inputs=outputs_pred_box_y2_sub_h + [min_const_name, max_const_name],
+        outputs=outputs_pred_box_y2_clip)
+    node_list.append(node_pred_box_y2_clip)
+
+    outputs_pred_box_x2_res = [model_name + "@box_x2_res"]
+    node_pred_box_x2_res = onnx.helper.make_node(
+        'Sub',
+        inputs=outputs_pred_box_x2_decode + outputs_pred_box_x2_clip,
+        outputs=outputs_pred_box_x2_res)
+    node_list.append(node_pred_box_x2_res)
+
+    outputs_pred_box_y2_res = [model_name + "@box_y2_res"]
+    node_pred_box_y2_res = onnx.helper.make_node(
+        'Sub',
+        inputs=outputs_pred_box_y2_decode + outputs_pred_box_y2_clip,
+        outputs=outputs_pred_box_y2_res)
+    node_list.append(node_pred_box_y2_res)
+
+    node_pred_box_result = onnx.helper.make_node(
+        'Concat',
+        inputs=outputs_pred_box_x1_clip + outputs_pred_box_y1_clip +
+        outputs_pred_box_x2_res + outputs_pred_box_y2_res,
+        outputs=outputs['Boxes'],
+        axis=-1)
+    node_list.append(node_pred_box_result)
+
+    score_shape = [1, input_height * input_width * int(num_anchors), class_num]
+    name_score_shape = [model_name + "@score_shape"]
+    node_score_shape = onnx.helper.make_node(
+        "Constant",
+        inputs=[],
+        outputs=name_score_shape,
+        value=onnx.helper.make_tensor(
+            name=name_score_shape[0] + "@const",
+            data_type=onnx.TensorProto.INT64,
+            dims=[len(score_shape)],
+            vals=score_shape))
+    node_list.append(node_score_shape)
+
+    node_score_new_shape = onnx.helper.make_node(
+        'Reshape',
+        inputs=outputs_score + name_score_shape,
+        outputs=outputs['Scores'])
+    node_list.append(node_score_new_shape)
+    return node_list

x2paddle/op_mapper/paddle_op_mapper.py → x2paddle/op_mapper/paddle2onnx/opset9/opset.py

-#   Copyright (c) 2019  PaddlePaddle Authors. All Rights Reserved.
+# Copyright (c) 2019  PaddlePaddle Authors. All Rights Reserved.
 #
 # Licensed under the Apache License, Version 2.0 (the "License"
 # you may not use this file except in compliance with the License.
@@ -23,7 +23,7 @@ import onnx
 from onnx import helper, onnx_pb
 
 
-class PaddleOpMapper(object):
+class OpSet9(object):
     def __init__(self):
         self.paddle_onnx_dtype_map = {
             core.VarDesc.VarType.FP32: onnx_pb.TensorProto.FLOAT,
@@ -34,63 +34,8 @@ class PaddleOpMapper(object):
             core.VarDesc.VarType.INT64: onnx_pb.TensorProto.INT64,
             core.VarDesc.VarType.BOOL: onnx_pb.TensorProto.BOOL
         }
-
         self.name_counter = dict()
 
-    def convert(self, program, save_dir):
-        weight_nodes = self.convert_weights(program)
-        op_nodes = list()
-        input_nodes = list()
-        output_nodes = list()
-
-        unsupported_ops = set()
-
-        print("Translating PaddlePaddle to ONNX...\n")
-        for block in program.blocks:
-            for i, op in enumerate(block.ops):
-                sys.stdout.write(
-                    "\rTotal:{}, Current:{} : {}                   ".format(
-                        len(block.ops), i + 1, op.type))
-                sys.stdout.flush()
-                if not hasattr(self, op.type):
-                    unsupported_ops.add(op.type)
-                    continue
-                if len(unsupported_ops) > 0:
-                    continue
-                node = getattr(self, op.type)(op, block)
-                if op.type == 'feed':
-                    input_nodes.append(node)
-                elif op.type == 'fetch':
-                    output_nodes.append(node)
-                else:
-                    if isinstance(node, list):
-                        op_nodes = op_nodes + node
-                    else:
-                        op_nodes.append(node)
-
-        if len(unsupported_ops) > 0:
-            print("\nThere's {} ops are not supported yet".format(
-                len(unsupported_ops)))
-            for op in unsupported_ops:
-                print("=========== {} ===========".format(op))
-            return
-
-        graph = helper.make_graph(
-            nodes=weight_nodes + op_nodes,
-            name='onnx_model_from_paddle',
-            initializer=[],
-            inputs=input_nodes,
-            outputs=output_nodes)
-        model = helper.make_model(graph, producer_name='X2Paddle')
-        onnx.checker.check_model(model)
-
-        if not os.path.isdir(save_dir):
-            os.makedirs(save_dir)
-        with open(os.path.join(save_dir, 'x2paddle_model.onnx'), 'wb') as f:
-            f.write(model.SerializeToString())
-        print("\nTranslated model saved in {}".format(
-            os.path.join(save_dir, 'x2paddle_model.onnx')))
-
     def get_name(self, op_name, var_name):
         name = 'p2o.{}.{}'.format(op_name, var_name)
         if name not in self.name_counter:
@@ -99,6 +44,21 @@ class PaddleOpMapper(object):
             self.name_counter[name] += 1
         return name + '.{}'.format(self.name_counter[name])
 
+    def make_constant_node(self, name, dtype, value=None):
+        if isinstance(value, list):
+            dims = (len(value), )
+        elif value is None:
+            dims = ()
+            value = []
+        else:
+            dims = ()
+            value = [value]
+        tensor = helper.make_tensor(
+            name=name, data_type=dtype, dims=dims, vals=value)
+        node = helper.make_node(
+            'Constant', inputs=[], outputs=[name], value=tensor)
+        return node
+
     def convert_weights(self, program):
         var_names = program.global_block().vars
         nodes = list()
@@ -119,21 +79,6 @@ class PaddleOpMapper(object):
             nodes.append(node)
         return nodes
 
-    def make_constant_node(self, name, dtype, value=None):
-        if isinstance(value, list):
-            dims = (len(value), )
-        elif value is None:
-            dims = ()
-            value = []
-        else:
-            dims = ()
-            value = [value]
-        tensor = helper.make_tensor(
-            name=name, data_type=dtype, dims=dims, vals=value)
-        node = helper.make_node(
-            'Constant', inputs=[], outputs=[name], value=tensor)
-        return node
-
     def conv2d(self, op, block):
         kernel_shape = block.var(op.input('Filter')[0]).shape
         node = helper.make_node(
@@ -251,6 +196,8 @@ class PaddleOpMapper(object):
                 pool_type[op.attr('pooling_type')][1],
                 inputs=op.input('X'),
                 outputs=op.output('Out'), )
+        elif op.attr('adaptive'):
+            raise Excpetion("ONNX cannot support adaptive pool")
         else:
             input_shape = block.var(op.input('X')[0]).shape
             k_size = op.attr('ksize')
@@ -268,6 +215,28 @@ class PaddleOpMapper(object):
                 pads=op.attr('paddings') + op.attr('paddings'))
         return node
 
+    def pad2d(self, op, block):
+        x_shape = block.var(op.input('X')[0]).shape
+        paddings = op.attr('paddings')
+        onnx_pads = []
+        if op.attr('data_format') == 'NCHW':
+            pads = [
+                0, 0, paddings[0], paddings[2], 0, 0, paddings[1], paddings[3]
+            ]
+        else:
+            pads = [
+                0, paddings[0], paddings[2], 0, 0, paddings[1], paddings[3], 0
+            ]
+        #TODO support pads is Variable
+        node = helper.make_node(
+            'Pad',
+            inputs=op.input('X'),
+            outputs=op.output('Out'),
+            mode=op.attr('mode'),
+            value=op.attr('pad_value'),
+            pads=pads)
+        return node
+
     def softmax(self, op, block):
         axis = op.attr('axis')
         shape = block.var(op.output('Out')[0]).shape
@@ -397,17 +366,14 @@ class PaddleOpMapper(object):
         return self.conv2d(op, block)
 
     def relu6(self, op, block):
-        min_name = self.get_name(op.type, 'min')
-        max_name = self.get_name(op.type, 'max')
-        min_node = self.make_constant_node(min_name, onnx_pb.TensorProto.FLOAT,
-                                           0)
-        max_node = self.make_constant_node(max_name, onnx_pb.TensorProto.FLOAT,
-                                           op.attr('threshold'))
+        threshold = op.attr('threshold')
         node = helper.make_node(
             'Clip',
-            inputs=[op.input('X')[0], min_name, max_name],
-            outputs=op.output('Out'), )
-        return [min_node, max_node, node]
+            inputs=[op.input('X')[0]],
+            outputs=op.output('Out'),
+            max=threshold,
+            min=0.0)
+        return [node]
 
     def shape(self, op, block):
         node = helper.make_node(
@@ -435,21 +401,14 @@ class PaddleOpMapper(object):
         axes = op.attr('axes')
         starts = op.attr('starts')
         ends = op.attr('ends')
-        axes_name = self.get_name(op.type, 'axes')
-        starts_name = self.get_name(op.type, 'starts')
-        ends_name = self.get_name(op.type, 'ends')
-
-        axes_node = self.make_constant_node(axes_name,
-                                            onnx_pb.TensorProto.INT64, axes)
-        starts_node = self.make_constant_node(starts_name,
-                                              onnx_pb.TensorProto.INT64, starts)
-        ends_node = self.make_constant_node(ends_name,
-                                            onnx_pb.TensorProto.INT64, ends)
         node = helper.make_node(
             "Slice",
             inputs=[op.input('Input')[0], starts_name, ends_name, axes_name],
-            outputs=op.output('Out'), )
-        return [starts_node, ends_node, axes_node, node]
+            outputs=op.output('Out'),
+            axes=axes,
+            starts=starts,
+            ends=ends)
+        return [node]
 
     def fill_constant(self, op, block):
         value = op.attr('value')
@@ -544,27 +503,15 @@ class PaddleOpMapper(object):
 
     def bilinear_interp(self, op, block):
         input_names = op.input_names
-        coordinate_transformation_mode = 'half_pixel'
-        if op.attr('align_corners'):
-            coordinate_transformation_mode = 'align_corners'
+        input_shape = block.vars[op.input('X')[0]].shape
+        if op.attr('align_corners') or op.attr('align_mode') == 0:
+            raise Exception(
+                "Resize in onnx(opset<=10) only support coordinate_transformation_mode: 'asymmetric'."
+            )
         if ('OutSize' in input_names and len(op.input('OutSize')) > 0) or (
                 'SizeTensor' in input_names and
                 len(op.input('SizeTensor')) > 0):
             node_list = list()
-            roi_node = self.make_constant_node(
-                self.get_name(op.type, 'roi'), onnx_pb.TensorProto.FLOAT,
-                [1, 1, 1, 1, 1, 1, 1, 1])
-            roi_name = self.get_name(op.type, 'roi')
-            roi_node = self.make_constant_node(
-                roi_name, onnx_pb.TensorProto.FLOAT, [1, 1, 1, 1, 1, 1, 1, 1])
-            empty_name = self.get_name(op.type, 'empty')
-            empty_tensor = helper.make_tensor(
-                empty_name,
-                onnx_pb.TensorProto.FLOAT, (0, ),
-                np.array([]).astype('float32'),
-                raw=False)
-            empty_node = helper.make_node(
-                'Constant', [], outputs=[empty_name], value=empty_tensor)
             shape_name0 = self.get_name(op.type, 'shape')
             shape_node0 = helper.make_node(
                 'Shape', inputs=op.input('X'), outputs=[shape_name0])
@@ -579,16 +526,7 @@ class PaddleOpMapper(object):
                 'Slice',
                 inputs=[shape_name0, starts_name, ends_name],
                 outputs=[shape_name1])
-            node_list.extend([
-                roi_node, empty_node, shape_node0, starts_node, ends_node,
-                shape_node1
-            ])
-            #            shape_name2 = self.get_name(op.type, "shape.cast")
-            #            shape_node2 = helper.make_node(
-            #                'Cast',
-            #                inputs=op.input('OutSize'),
-            #                outputs=[shape_name2],
-            #                to=onnx_pb.TensorProto.INT64)
+            node_list.extend([shape_node0, starts_node, ends_node, shape_node1])
             if 'OutSize' in input_names and len(op.input('OutSize')) > 0:
                 cast_shape_name = self.get_name(op.type, "shape.cast")
                 cast_shape_node = helper.make_node(
@@ -598,7 +536,8 @@ class PaddleOpMapper(object):
                     to=onnx_pb.TensorProto.INT64)
                 node_list.append(cast_shape_node)
             else:
-                concat_shape_name = self.get_name(op.type, "shape.concat")
+                concat_shape_name = self.get_name(
+                    op.type, op.output('Out')[0] + "shape.concat")
                 concat_shape_node = helper.make_node(
                     "Concat",
                     inputs=op.input('SizeTensor'),
@@ -611,27 +550,46 @@ class PaddleOpMapper(object):
                     outputs=[cast_shape_name],
                     to=onnx_pb.TensorProto.INT64)
                 node_list.extend([concat_shape_node, cast_shape_node])
-            shape_name3 = self.get_name(op.type, "shape.concat")
-            shape_node3 = helper.make_node(
+            shape_name2 = self.get_name(op.type, "shape.concat")
+            shape_node2 = helper.make_node(
                 'Concat',
                 inputs=[shape_name1, cast_shape_name],
-                outputs=[shape_name3],
+                outputs=[shape_name2],
                 axis=0)
+            node_list.append(shape_node2)
+            cast_shape_name2 = self.get_name(op.type, "shape.cast")
+            cast_shape_node2 = helper.make_node(
+                'Cast',
+                inputs=[shape_name2],
+                outputs=[cast_shape_name2],
+                to=onnx_pb.TensorProto.FLOAT)
+            node_list.append(cast_shape_node2)
+            cast_shape_name0 = self.get_name(op.type, "shape.cast")
+            cast_shape_node0 = helper.make_node(
+                'Cast',
+                inputs=[shape_name0],
+                outputs=[cast_shape_name0],
+                to=onnx_pb.TensorProto.FLOAT)
+            node_list.append(cast_shape_node0)
+            outputs_h_w_scales = op.output('Out')[0] + "@out_hw_scales"
+            node_h_w_scales = helper.make_node(
+                'Div',
+                inputs=[cast_shape_name2, cast_shape_name0],
+                outputs=[outputs_h_w_scales])
+            node_list.append(node_h_w_scales)
             result_node = helper.make_node(
                 'Resize',
-                inputs=[op.input('X')[0], roi_name, empty_name, shape_name3],
+                inputs=[op.input('X')[0], outputs_h_w_scales],
                 outputs=op.output('Out'),
-                mode='linear',
-                coordinate_transformation_mode=coordinate_transformation_mode)
-            node_list.extend([shape_node3, result_node])
+                mode='linear')
+            node_list.extend([result_node])
             return node_list
         elif 'Scale' in input_names and len(op.input('Scale')) > 0:
             node = helper.make_node(
                 'Resize',
                 inputs=[op.input('X')[0], op.input('Scale')[0]],
                 outputs=op.output('Out'),
-                mode='linear',
-                coordinate_transformation_mode=coordinate_transformation_mode)
+                mode='linear')
         else:
             out_shape = [op.attr('out_h'), op.attr('out_w')]
             scale = op.attr('scale')
@@ -640,41 +598,34 @@ class PaddleOpMapper(object):
                 scale_node = self.make_constant_node(scale_name,
                                                      onnx_pb.TensorProto.FLOAT,
                                                      [1, 1, scale, scale])
-                roi_name = self.get_name(op.type, 'roi')
-                roi_node = self.make_constant_node(roi_name,
-                                                   onnx_pb.TensorProto.FLOAT,
-                                                   [1, 1, 1, 1, 1, 1, 1, 1])
                 node = helper.make_node(
                     'Resize',
-                    inputs=[op.input('X')[0], roi_name, scale_name],
+                    inputs=[op.input('X')[0], scale_name],
                     outputs=op.output('Out'),
-                    mode='nearest',
-                    coordinate_transformation_mode=coordinate_transformation_mode
-                )
-                return [scale_node, roi_node, node]
+                    mode='linear')
+                return [scale_node, node]
             else:
                 raise Exception("Unexpected situation happend")
         return node
 
     def nearest_interp(self, op, block):
         input_names = op.input_names
-        coordinate_transformation_mode = 'half_pixel'
         if op.attr('align_corners'):
-            coordinate_transformation_mode = 'align_corners'
+            raise Exception(
+                "Resize in onnx(opset<=10) only support coordinate_transformation_mode: 'asymmetric'."
+            )
         if 'OutSize' in input_names and len(op.input('OutSize')) > 0:
             node = helper.make_node(
                 'Resize',
-                inputs=[op.input('X')[0], '', op.input('OutSize')[0]],
+                inputs=[op.input('X')[0], op.input('OutSize')[0]],
                 outputs=op.output('Out'),
-                mode='nearest',
-                coordinate_transformation_mode=coordinate_transformation_mode)
+                mode='nearest')
         elif 'Scale' in input_names and len(op.input('Scale')) > 0:
             node = helper.make_node(
                 'Resize',
                 inputs=[op.input('X')[0], op.input('Scale')[0]],
                 outputs=op.output('Out'),
-                mode='nearest',
-                coordinate_transformation_mode=coordinate_transformation_mode)
+                mode='nearest')
         else:
             out_shape = [op.attr('out_h'), op.attr('out_w')]
             scale = op.attr('scale')
@@ -683,18 +634,12 @@ class PaddleOpMapper(object):
                 scale_node = self.make_constant_node(scale_name,
                                                      onnx_pb.TensorProto.FLOAT,
                                                      [1, 1, scale, scale])
-                roi_name = self.get_name(op.type, 'roi')
-                roi_node = self.make_constant_node(roi_name,
-                                                   onnx_pb.TensorProto.FLOAT,
-                                                   [1, 1, 1, 1, 1, 1, 1, 1])
                 node = helper.make_node(
                     'Resize',
-                    inputs=[op.input('X')[0], roi_name, scale_name],
+                    inputs=[op.input('X')[0], scale_name],
                     outputs=op.output('Out'),
-                    mode='nearest',
-                    coordinate_transformation_mode=coordinate_transformation_mode
-                )
-                return [scale_node, roi_node, node]
+                    mode='nearest')
+                return [scale_node, node]
             else:
                 raise Exception("Unexpected situation happend")
         return node
@@ -711,14 +656,8 @@ class PaddleOpMapper(object):
         return node
 
     def hard_swish(self, op, block):
-        min_name = self.get_name(op.type, 'min')
-        max_name = self.get_name(op.type, 'max')
         scale_name = self.get_name(op.type, 'scale')
         offset_name = self.get_name(op.type, 'offset')
-        min_node = self.make_constant_node(min_name, onnx_pb.TensorProto.FLOAT,
-                                           0)
-        max_node = self.make_constant_node(max_name, onnx_pb.TensorProto.FLOAT,
-                                           op.attr('threshold'))
         scale_node = self.make_constant_node(scale_name,
                                              onnx_pb.TensorProto.FLOAT,
                                              op.attr('scale'))
@@ -730,19 +669,20 @@ class PaddleOpMapper(object):
         node0 = helper.make_node(
             'Add', inputs=[op.input('X')[0], offset_name], outputs=[name0])
         name1 = self.get_name(op.type, 'relu')
+        min_value = op.attr('min')
+        max_value = op.attr('max')
         node1 = helper.make_node(
             'Clip',
-            inputs=[name0, min_name, max_name],
-            outputs=[name1], )
+            inputs=[name0],
+            outputs=[name1],
+            max=max_value,
+            min=min_value)
         name2 = self.get_name(op.type, 'mul')
         node2 = helper.make_node(
             'Mul', inputs=[op.input('X')[0], name1], outputs=[name2])
         node3 = helper.make_node(
             'Div', inputs=[name2, scale_name], outputs=op.output('Out'))
-        return [
-            min_node, max_node, scale_node, offset_node, node0, node1, node2,
-            node3
-        ]
+        return [scale_node, offset_node, node0, node1, node2, node3]
 
     def elementwise_mul(self, op, block):
         axis = op.attr('axis')
@@ -818,3 +758,11 @@ class PaddleOpMapper(object):
     def im2sequence(self, op, block):
         from .paddle_custom_layer.im2sequence import im2sequence
         return im2sequence(op, block)
+
+    def yolo_box(self, op, block):
+        from .paddle_custom_layer.yolo_box import yolo_box
+        return yolo_box(op, block)
+
+    def multiclass_nms(self, op, block):
+        from .paddle_custom_layer.multiclass_nms import multiclass_nms
+        return multiclass_nms(op, block)

x2paddle/op_mapper/paddle_custom_layer/im2sequence.py → x2paddle/op_mapper/paddle2onnx/opset9/paddle_custom_layer/im2sequence.py

+# Copyright (c) 2020  PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License"
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
 import onnx
 import numpy as np
 from onnx import onnx_pb, helper

x2paddle/op_mapper/paddle2onnx/opset9/paddle_custom_layer/multiclass_nms.py

+# Copyright (c) 2019  PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License"
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import math
+import sys
+import os
+import numpy as np
+import paddle.fluid.core as core
+import paddle.fluid as fluid
+import onnx
+import warnings
+from onnx import helper, onnx_pb
+
+
+def multiclass_nms(op, block):
+    """
+    Convert the paddle multiclass_nms to onnx op.
+    This op is get the select boxes from origin boxes.
+    """
+    inputs = dict()
+    outputs = dict()
+    attrs = dict()
+    for name in op.input_names:
+        inputs[name] = op.input(name)
+    for name in op.output_names:
+        outputs[name] = op.output(name)
+    for name in op.attr_names:
+        attrs[name] = op.attr(name)
+
+    result_name = outputs['Out'][0]
+    background = attrs['background_label']
+    normalized = attrs['normalized']
+    if normalized == False:
+        warnings.warn(
+            'The parameter normalized of multiclass_nms OP of Paddle is False, which has diff with ONNX. \
+                         Please set normalized=True in multiclass_nms of Paddle')
+
+    #convert the paddle attribute to onnx tensor
+    name_score_threshold = [outputs['Out'][0] + "@score_threshold"]
+    name_iou_threshold = [outputs['Out'][0] + "@iou_threshold"]
+    name_keep_top_k = [outputs['Out'][0] + '@keep_top_k']
+    name_keep_top_k_2D = [outputs['Out'][0] + '@keep_top_k_1D']
+
+    node_score_threshold = onnx.helper.make_node(
+        'Constant',
+        inputs=[],
+        outputs=name_score_threshold,
+        value=onnx.helper.make_tensor(
+            name=name_score_threshold[0] + "@const",
+            data_type=onnx.TensorProto.FLOAT,
+            dims=(),
+            vals=[float(attrs['score_threshold'])]))
+
+    node_iou_threshold = onnx.helper.make_node(
+        'Constant',
+        inputs=[],
+        outputs=name_iou_threshold,
+        value=onnx.helper.make_tensor(
+            name=name_iou_threshold[0] + "@const",
+            data_type=onnx.TensorProto.FLOAT,
+            dims=(),
+            vals=[float(attrs['nms_threshold'])]))
+
+    node_keep_top_k = onnx.helper.make_node(
+        'Constant',
+        inputs=[],
+        outputs=name_keep_top_k,
+        value=onnx.helper.make_tensor(
+            name=name_keep_top_k[0] + "@const",
+            data_type=onnx.TensorProto.INT64,
+            dims=(),
+            vals=[np.int64(attrs['keep_top_k'])]))
+
+    node_keep_top_k_2D = onnx.helper.make_node(
+        'Constant',
+        inputs=[],
+        outputs=name_keep_top_k_2D,
+        value=onnx.helper.make_tensor(
+            name=name_keep_top_k_2D[0] + "@const",
+            data_type=onnx.TensorProto.INT64,
+            dims=[1, 1],
+            vals=[np.int64(attrs['keep_top_k'])]))
+
+    # the paddle data format is x1,y1,x2,y2
+    kwargs = {'center_point_box': 0}
+
+    name_select_nms = [outputs['Out'][0] + "@select_index"]
+    node_select_nms= onnx.helper.make_node(
+        'NonMaxSuppression',
+        inputs=inputs['BBoxes'] + inputs['Scores'] + name_keep_top_k +\
+            name_iou_threshold + name_score_threshold,
+        outputs=name_select_nms)
+    # step 1 nodes select the nms class
+    node_list = [
+        node_score_threshold, node_iou_threshold, node_keep_top_k,
+        node_keep_top_k_2D, node_select_nms
+    ]
+
+    # create some const value to use
+    name_const_value = [result_name+"@const_0",
+        result_name+"@const_1",\
+        result_name+"@const_2",\
+        result_name+"@const_-1"]
+    value_const_value = [0, 1, 2, -1]
+    for name, value in zip(name_const_value, value_const_value):
+        node = onnx.helper.make_node(
+            'Constant',
+            inputs=[],
+            outputs=[name],
+            value=onnx.helper.make_tensor(
+                name=name + "@const",
+                data_type=onnx.TensorProto.INT64,
+                dims=[1],
+                vals=[value]))
+        node_list.append(node)
+
+    # In this code block, we will deocde the raw score data, reshape N * C * M to 1 * N*C*M
+    # and the same time, decode the select indices to 1 * D, gather the select_indices
+    outputs_gather_1 = [result_name + "@gather_1"]
+    node_gather_1 = onnx.helper.make_node(
+        'Gather',
+        inputs=name_select_nms + [result_name + "@const_1"],
+        outputs=outputs_gather_1,
+        axis=1)
+    node_list.append(node_gather_1)
+
+    outputs_squeeze_gather_1 = [result_name + "@sequeeze_gather_1"]
+    node_squeeze_gather_1 = onnx.helper.make_node(
+        'Squeeze',
+        inputs=outputs_gather_1,
+        outputs=outputs_squeeze_gather_1,
+        axes=[1])
+    node_list.append(node_squeeze_gather_1)
+
+    outputs_gather_2 = [result_name + "@gather_2"]
+    node_gather_2 = onnx.helper.make_node(
+        'Gather',
+        inputs=name_select_nms + [result_name + "@const_2"],
+        outputs=outputs_gather_2,
+        axis=1)
+    node_list.append(node_gather_2)
+
+    #slice the class is not 0
+    if background == 0:
+        outputs_nonzero = [result_name + "@nonzero"]
+        node_nonzero = onnx.helper.make_node(
+            'NonZero', inputs=outputs_squeeze_gather_1, outputs=outputs_nonzero)
+        node_list.append(node_nonzero)
+    else:
+        name_thresh = [result_name + "@thresh"]
+        node_thresh = onnx.helper.make_node(
+            'Constant',
+            inputs=[],
+            outputs=name_thresh,
+            value=onnx.helper.make_tensor(
+                name=name_thresh[0] + "@const",
+                data_type=onnx.TensorProto.INT32,
+                dims=[1],
+                vals=[-1]))
+        node_list.append(node_thresh)
+
+        outputs_cast = [result_name + "@cast"]
+        node_cast = onnx.helper.make_node(
+            'Cast', inputs=outputs_squeeze_gather_1, outputs=outputs_cast, to=6)
+        node_list.append(node_cast)
+
+        outputs_greater = [result_name + "@greater"]
+        node_greater = onnx.helper.make_node(
+            'Greater',
+            inputs=outputs_cast + name_thresh,
+            outputs=outputs_greater)
+        node_list.append(node_greater)
+
+        outputs_nonzero = [result_name + "@nonzero"]
+        node_nonzero = onnx.helper.make_node(
+            'NonZero', inputs=outputs_greater, outputs=outputs_nonzero)
+        node_list.append(node_nonzero)
+
+    outputs_gather_1_nonzero = [result_name + "@gather_1_nonzero"]
+    node_gather_1_nonzero = onnx.helper.make_node(
+        'Gather',
+        inputs=outputs_gather_1 + outputs_nonzero,
+        outputs=outputs_gather_1_nonzero,
+        axis=0)
+    node_list.append(node_gather_1_nonzero)
+
+    outputs_gather_2_nonzero = [result_name + "@gather_2_nonzero"]
+    node_gather_2_nonzero = onnx.helper.make_node(
+        'Gather',
+        inputs=outputs_gather_2 + outputs_nonzero,
+        outputs=outputs_gather_2_nonzero,
+        axis=0)
+    node_list.append(node_gather_2_nonzero)
+
+    # reshape scores N * C * M to (N*C*M) * 1
+    outputs_reshape_scores_rank1 = [result_name + "@reshape_scores_rank1"]
+    node_reshape_scores_rank1 = onnx.helper.make_node(
+        "Reshape",
+        inputs=inputs['Scores'] + [result_name + "@const_-1"],
+        outputs=outputs_reshape_scores_rank1)
+    node_list.append(node_reshape_scores_rank1)
+
+    # get the shape of scores
+    outputs_shape_scores = [result_name + "@shape_scores"]
+    node_shape_scores = onnx.helper.make_node(
+        'Shape', inputs=inputs['Scores'], outputs=outputs_shape_scores)
+    node_list.append(node_shape_scores)
+
+    # gather the index: 2 shape of scores
+    outputs_gather_scores_dim1 = [result_name + "@gather_scores_dim1"]
+    node_gather_scores_dim1 = onnx.helper.make_node(
+        'Gather',
+        inputs=outputs_shape_scores + [result_name + "@const_2"],
+        outputs=outputs_gather_scores_dim1,
+        axis=0)
+    node_list.append(node_gather_scores_dim1)
+
+    # mul class * M
+    outputs_mul_classnum_boxnum = [result_name + "@mul_classnum_boxnum"]
+    node_mul_classnum_boxnum = onnx.helper.make_node(
+        'Mul',
+        inputs=outputs_gather_1_nonzero + outputs_gather_scores_dim1,
+        outputs=outputs_mul_classnum_boxnum)
+    node_list.append(node_mul_classnum_boxnum)
+
+    # add class * M * index
+    outputs_add_class_M_index = [result_name + "@add_class_M_index"]
+    node_add_class_M_index = onnx.helper.make_node(
+        'Add',
+        inputs=outputs_mul_classnum_boxnum + outputs_gather_2_nonzero,
+        outputs=outputs_add_class_M_index)
+    node_list.append(node_add_class_M_index)
+
+    # Squeeze the indices to 1 dim
+    outputs_squeeze_select_index = [result_name + "@squeeze_select_index"]
+    node_squeeze_select_index = onnx.helper.make_node(
+        'Squeeze',
+        inputs=outputs_add_class_M_index,
+        outputs=outputs_squeeze_select_index,
+        axes=[0, 2])
+    node_list.append(node_squeeze_select_index)
+
+    # gather the data from flatten scores
+    outputs_gather_select_scores = [result_name + "@gather_select_scores"]
+    node_gather_select_scores = onnx.helper.make_node('Gather',
+        inputs=outputs_reshape_scores_rank1 + \
+            outputs_squeeze_select_index,
+        outputs=outputs_gather_select_scores,
+        axis=0)
+    node_list.append(node_gather_select_scores)
+
+    # get nums to input TopK
+    outputs_shape_select_num = [result_name + "@shape_select_num"]
+    node_shape_select_num = onnx.helper.make_node(
+        'Shape',
+        inputs=outputs_gather_select_scores,
+        outputs=outputs_shape_select_num)
+    node_list.append(node_shape_select_num)
+
+    outputs_gather_select_num = [result_name + "@gather_select_num"]
+    node_gather_select_num = onnx.helper.make_node(
+        'Gather',
+        inputs=outputs_shape_select_num + [result_name + "@const_0"],
+        outputs=outputs_gather_select_num,
+        axis=0)
+    node_list.append(node_gather_select_num)
+
+    outputs_unsqueeze_select_num = [result_name + "@unsqueeze_select_num"]
+    node_unsqueeze_select_num = onnx.helper.make_node(
+        'Unsqueeze',
+        inputs=outputs_gather_select_num,
+        outputs=outputs_unsqueeze_select_num,
+        axes=[0])
+    node_list.append(node_unsqueeze_select_num)
+
+    outputs_concat_topK_select_num = [result_name + "@conat_topK_select_num"]
+    node_conat_topK_select_num = onnx.helper.make_node(
+        'Concat',
+        inputs=outputs_unsqueeze_select_num + name_keep_top_k_2D,
+        outputs=outputs_concat_topK_select_num,
+        axis=0)
+    node_list.append(node_conat_topK_select_num)
+
+    outputs_cast_concat_topK_select_num = [
+        result_name + "@concat_topK_select_num"
+    ]
+    node_outputs_cast_concat_topK_select_num = onnx.helper.make_node(
+        'Cast',
+        inputs=outputs_concat_topK_select_num,
+        outputs=outputs_cast_concat_topK_select_num,
+        to=6)
+    node_list.append(node_outputs_cast_concat_topK_select_num)
+    # get min(topK, num_select)
+    outputs_compare_topk_num_select = [result_name + "@compare_topk_num_select"]
+    node_compare_topk_num_select = onnx.helper.make_node(
+        'ReduceMin',
+        inputs=outputs_cast_concat_topK_select_num,
+        outputs=outputs_compare_topk_num_select,
+        keepdims=0)
+    node_list.append(node_compare_topk_num_select)
+
+    # unsqueeze the indices to 1D tensor
+    outputs_unsqueeze_topk_select_indices = [
+        result_name + "@unsqueeze_topk_select_indices"
+    ]
+    node_unsqueeze_topk_select_indices = onnx.helper.make_node(
+        'Unsqueeze',
+        inputs=outputs_compare_topk_num_select,
+        outputs=outputs_unsqueeze_topk_select_indices,
+        axes=[0])
+    node_list.append(node_unsqueeze_topk_select_indices)
+
+    # cast the indices to INT64
+    outputs_cast_topk_indices = [result_name + "@cast_topk_indices"]
+    node_cast_topk_indices = onnx.helper.make_node(
+        'Cast',
+        inputs=outputs_unsqueeze_topk_select_indices,
+        outputs=outputs_cast_topk_indices,
+        to=7)
+    node_list.append(node_cast_topk_indices)
+
+    # select topk scores  indices
+    outputs_topk_select_topk_indices = [result_name + "@topk_select_topk_values",\
+        result_name + "@topk_select_topk_indices"]
+    node_topk_select_topk_indices = onnx.helper.make_node(
+        'TopK',
+        inputs=outputs_gather_select_scores + outputs_cast_topk_indices,
+        outputs=outputs_topk_select_topk_indices)
+    node_list.append(node_topk_select_topk_indices)
+
+    # gather topk label, scores, boxes
+    outputs_gather_topk_scores = [result_name + "@gather_topk_scores"]
+    node_gather_topk_scores = onnx.helper.make_node(
+        'Gather',
+        inputs=outputs_gather_select_scores +
+        [outputs_topk_select_topk_indices[1]],
+        outputs=outputs_gather_topk_scores,
+        axis=0)
+    node_list.append(node_gather_topk_scores)
+
+    outputs_gather_topk_class = [result_name + "@gather_topk_class"]
+    node_gather_topk_class = onnx.helper.make_node(
+        'Gather',
+        inputs=outputs_gather_1_nonzero +
+        [outputs_topk_select_topk_indices[1]],
+        outputs=outputs_gather_topk_class,
+        axis=1)
+    node_list.append(node_gather_topk_class)
+
+    # gather the boxes need to gather the boxes id, then get boxes
+    outputs_gather_topk_boxes_id = [result_name + "@gather_topk_boxes_id"]
+    node_gather_topk_boxes_id = onnx.helper.make_node(
+        'Gather',
+        inputs=outputs_gather_2_nonzero +
+        [outputs_topk_select_topk_indices[1]],
+        outputs=outputs_gather_topk_boxes_id,
+        axis=1)
+    node_list.append(node_gather_topk_boxes_id)
+
+    # squeeze the gather_topk_boxes_id to 1 dim
+    outputs_squeeze_topk_boxes_id = [result_name + "@squeeze_topk_boxes_id"]
+    node_squeeze_topk_boxes_id = onnx.helper.make_node(
+        'Squeeze',
+        inputs=outputs_gather_topk_boxes_id,
+        outputs=outputs_squeeze_topk_boxes_id,
+        axes=[0, 2])
+    node_list.append(node_squeeze_topk_boxes_id)
+
+    outputs_gather_select_boxes = [result_name + "@gather_select_boxes"]
+    node_gather_select_boxes = onnx.helper.make_node(
+        'Gather',
+        inputs=inputs['BBoxes'] + outputs_squeeze_topk_boxes_id,
+        outputs=outputs_gather_select_boxes,
+        axis=1)
+    node_list.append(node_gather_select_boxes)
+
+    # concat the final result
+    # before concat need to cast the class to float
+    outputs_cast_topk_class = [result_name + "@cast_topk_class"]
+    node_cast_topk_class = onnx.helper.make_node(
+        'Cast',
+        inputs=outputs_gather_topk_class,
+        outputs=outputs_cast_topk_class,
+        to=1)
+    node_list.append(node_cast_topk_class)
+
+    outputs_unsqueeze_topk_scores = [result_name + "@unsqueeze_topk_scores"]
+    node_unsqueeze_topk_scores = onnx.helper.make_node(
+        'Unsqueeze',
+        inputs=outputs_gather_topk_scores,
+        outputs=outputs_unsqueeze_topk_scores,
+        axes=[0, 2])
+    node_list.append(node_unsqueeze_topk_scores)
+
+    inputs_concat_final_results = outputs_cast_topk_class + outputs_unsqueeze_topk_scores +\
+        outputs_gather_select_boxes
+    outputs_sort_by_socre_results = [result_name + "@concat_topk_scores"]
+    node_sort_by_socre_results = onnx.helper.make_node(
+        'Concat',
+        inputs=inputs_concat_final_results,
+        outputs=outputs_sort_by_socre_results,
+        axis=2)
+    node_list.append(node_sort_by_socre_results)
+
+    # select topk classes indices
+    outputs_squeeze_cast_topk_class = [result_name + "@squeeze_cast_topk_class"]
+    node_squeeze_cast_topk_class = onnx.helper.make_node(
+        'Squeeze',
+        inputs=outputs_cast_topk_class,
+        outputs=outputs_squeeze_cast_topk_class,
+        axes=[0, 2])
+    node_list.append(node_squeeze_cast_topk_class)
+    outputs_neg_squeeze_cast_topk_class = [
+        result_name + "@neg_squeeze_cast_topk_class"
+    ]
+    node_neg_squeeze_cast_topk_class = onnx.helper.make_node(
+        'Neg',
+        inputs=outputs_squeeze_cast_topk_class,
+        outputs=outputs_neg_squeeze_cast_topk_class)
+    node_list.append(node_neg_squeeze_cast_topk_class)
+    outputs_topk_select_classes_indices = [result_name + "@topk_select_topk_classes_scores",\
+        result_name + "@topk_select_topk_classes_indices"]
+    node_topk_select_topk_indices = onnx.helper.make_node(
+        'TopK',
+        inputs=outputs_neg_squeeze_cast_topk_class + outputs_cast_topk_indices,
+        outputs=outputs_topk_select_classes_indices)
+    node_list.append(node_topk_select_topk_indices)
+    outputs_concat_final_results = outputs['Out']
+    node_concat_final_results = onnx.helper.make_node(
+        'Gather',
+        inputs=outputs_sort_by_socre_results +
+        [outputs_topk_select_classes_indices[1]],
+        outputs=outputs_concat_final_results,
+        axis=1)
+    node_list.append(node_concat_final_results)
+    return node_list

x2paddle/op_mapper/paddle_custom_layer/yolo_box.py → x2paddle/op_mapper/paddle2onnx/opset9/paddle_custom_layer/yolo_box.py

+# Copyright (c) 2020  PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License"
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
 import onnx
 import numpy as np
 from onnx import onnx_pb, helper

x2paddle/op_mapper/paddle2onnx/paddle_op_mapper.py

+#   Copyright (c) 2019  PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License"
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import math
+import sys
+import x2paddle
+import os
+import numpy as np
+import paddle.fluid.core as core
+import paddle.fluid as fluid
+import onnx
+from onnx import helper, onnx_pb
+from x2paddle.op_mapper.paddle2onnx.opset9.opset import OpSet9
+from x2paddle.op_mapper.paddle2onnx.opset10.opset import OpSet10
+from x2paddle.op_mapper.paddle2onnx.opset11.opset import OpSet11
+
+
+class PaddleOpMapper(object):
+    def __init__(self):
+        self.support_opsets = [9, 10, 11]
+        self.default_opset = 10
+        self.name_counter = dict()
+        self.op_set = None
+
+    def convert(self, program, save_dir, opset_number=10):
+        self.op_set = self.create_opset(opset_number)
+        weight_nodes = self.op_set.convert_weights(program)
+        op_nodes = list()
+        input_nodes = list()
+        output_nodes = list()
+        unsupported_ops = set()
+
+        print("Translating PaddlePaddle to ONNX...\n")
+        for block in program.blocks:
+            for i, op in enumerate(block.ops):
+                sys.stdout.write("\rTotal:{}, Current:{} : {} ".format(
+                    len(block.ops), i + 1, op.type))
+                sys.stdout.flush()
+                if not hasattr(self.op_set, op.type):
+                    unsupported_ops.add(op.type)
+                    continue
+                if len(unsupported_ops) > 0:
+                    continue
+                node = getattr(self.op_set, op.type)(op, block)
+                if op.type == 'feed':
+                    print(node.name)
+                    input_nodes.append(node)
+                elif op.type == 'fetch':
+                    output_nodes.append(node)
+                else:
+                    if isinstance(node, list):
+                        op_nodes = op_nodes + node
+                    else:
+                        op_nodes.append(node)
+
+        if len(unsupported_ops) > 0:
+            print("\nThere's {} ops are not supported yet".format(
+                len(unsupported_ops)))
+            for op in unsupported_ops:
+                print("=========== {} ===========".format(op))
+            return
+
+        graph = helper.make_graph(
+            nodes=weight_nodes + op_nodes,
+            name='onnx_model_from_paddle',
+            initializer=[],
+            inputs=input_nodes,
+            outputs=output_nodes)
+        opset_imports = [helper.make_opsetid("", opset_number)]
+        model = helper.make_model(
+            graph, producer_name='X2Paddle', opset_imports=opset_imports)
+        onnx.checker.check_model(model)
+
+        if not os.path.isdir(save_dir):
+            os.makedirs(save_dir)
+        with open(os.path.join(save_dir, 'x2paddle_model.onnx'), 'wb') as f:
+            f.write(model.SerializeToString())
+        print("\nTranslated model saved in {}".format(
+            os.path.join(save_dir, 'x2paddle_model.onnx')))
+
+    def create_opset(self, opset_number):
+        run_opset = self.default_opset
+        opset = ''
+        if opset_number in self.support_opsets:
+            run_opset = opset_number
+        else:
+            for support_opset_number in self.support_opsets:
+                if support_opset_number < opset_number:
+                    run_opset = support_opset_number
+                else:
+                    break
+        print(
+            'Now, onnx2paddle support convert onnx model opset_verison {},'
+            'opset_verison of your onnx model is {}, automatically treated as op_set: {}.'
+            .format(self.support_opsets, opset_number, run_opset))
+        opset = 'OpSet' + str(run_opset)
+        return eval(opset)()

x2paddle/op_mapper/tf_op_mapper_nhwc.py

@@ -15,6 +15,10 @@
 from x2paddle.decoder.tf_decoder import TFGraph
 from x2paddle.core.op_mapper import OpMapper
 from x2paddle.core.util import *
+from x2paddle import program
+from x2paddle import gen_name
+import traceback
+import math
 import inspect
 import numpy
 import sys
@@ -35,7 +39,6 @@ class TFOpMapperNHWC(OpMapper):
     directly_map_ops = {
         'Relu': ['relu'],
         'Relu6': ['relu6'],
-        'Shape': ['shape'],
         'Abs': ['abs'],
         'Sigmoid': ['sigmoid'],
         'Exp': ['exp'],
@@ -46,7 +49,9 @@ class TFOpMapperNHWC(OpMapper):
         'Softplus': ['softplus'],
         'LeakyRelu': ['leaky_relu', {
             'alpha': 'alpha'
-        }]
+        }],
+        'Floor': ['floor'],
+        'Erf': ['erf']
     }
     elementwise_ops = {
         'Add': 'elementwise_add',
@@ -54,6 +59,9 @@ class TFOpMapperNHWC(OpMapper):
         'RealDiv': 'elementwise_div',
         'Sub': 'elementwise_sub',
         'Maximum': 'elementwise_max',
+        'Minimum': 'elementwise_min',
+        'LessEqual': 'less_equal',
+        'GreaterEqual': 'greater_equal',
         'Mul': 'elementwise_mul',
         'FloorDiv': 'elementwise_floordiv'
     }
@@ -63,18 +71,25 @@ class TFOpMapperNHWC(OpMapper):
         self.decoder = decoder
         self.graph = decoder.tf_graph
         self.weights = dict()
-        self.batch_node = None
         self.omit_nodes = list()
         self.used_custom_layers = dict()
+        program.clear()
 
         not_placeholder = list()
         for name in self.graph.input_nodes:
-            if self.graph.get_node(name).layer_type != "Placeholder":
+            if self.graph.get_node(
+                    name).layer_type != "Placeholder" and self.graph.get_node(
+                        name
+                    ).layer_type != "OneShotIterator" and self.graph.get_node(
+                        name).layer_type != "IteratorV2":
                 not_placeholder.append(name)
         for name in not_placeholder:
             idx = self.graph.input_nodes.index(name)
             del self.graph.input_nodes[idx]
 
+        program.inputs = self.graph.input_nodes
+        program.outputs = self.graph.output_nodes
+
         unsupported_ops = set()
         sys.stderr.write("Total nodes: {}\n".format(len(self.graph.topo_sort)))
         for i, node_name in enumerate(self.graph.topo_sort):
@@ -95,31 +110,23 @@ class TFOpMapperNHWC(OpMapper):
                 func = getattr(self, op)
                 try:
                     func(node)
-                except:
+                except Exception as e:
                     unsupported_ops.add(op)
+                    print("\n{}\n".format(traceback.format_exc()))
             else:
                 unsupported_ops.add(op)
         if len(unsupported_ops) > 0:
-            print("========= {} OPs are not supported yet ===========".format(
+            print("\n========= {} OPs are not supported yet ===========".format(
                 len(unsupported_ops)))
             for op in unsupported_ops:
                 print("========== {} ============".format(op))
             sys.exit(-1)
         sys.stderr.write("\nDone!\n")
 
-    def add_omit_nodes(self, in_node_name, out_node_name):
-        in_node = self.graph.get_node(in_node_name)
-        out_node = self.graph.get_node(out_node_name)
-        index = in_node.outputs.index(out_node_name)
-        del in_node.outputs[index]
-        index = out_node.inputs.index(in_node_name)
-        del out_node.inputs[index]
-        self.omit_nodes.append(in_node.layer_name)
-
     def directly_map(self, node):
         assert node.layer_type in self.directly_map_ops
         op_info = self.directly_map_ops[node.layer_type]
-        input = self.graph.get_node(node.layer.input[0], copy=True)
+        input = self.graph.get_node(node.layer.input[0])
         attr = dict()
         for param in op_info[1:]:
             tf_param_name = list(param.keys())[0]
@@ -127,128 +134,35 @@ class TFOpMapperNHWC(OpMapper):
             tf_param = node.get_attr(tf_param_name)
             attr[pd_param_name] = tf_param
 
-        if len(input.out_shapes[0]) == 4 and op_info[0] != 'shape':
-            attr1 = {"perm": [0, 3, 1, 2]}
-            node.fluid_code.add_layer(
-                'transpose', inputs=input, output=node, param_attr=attr1)
-            input = node
-            node.fluid_code.add_layer(
-                op_info[0], inputs=input, output=node, param_attr=attr)
-            input = node
-            attr2 = {"perm": [0, 2, 3, 1]}
-            node.fluid_code.add_layer(
-                'transpose', inputs=input, output=node, param_attr=attr2)
-        else:
-            node.fluid_code.add_layer(
-                op_info[0], inputs=input, output=node, param_attr=attr)
+        program.add_layer(
+            kernel="fluid.layers.{}".format(op_info[0]),
+            inputs={"x": input.name},
+            outputs=[node.name],
+            **attr)
 
     def elementwise_map(self, node):
         assert node.layer_type in self.elementwise_ops
         op_type = self.elementwise_ops[node.layer_type]
-        x = self.graph.get_node(node.layer.input[0], copy=True)
-        y = self.graph.get_node(node.layer.input[1], copy=True)
-        x_shape = x.out_shapes[0]
-        y_shape = y.out_shapes[0]
-        if len(x_shape) == 0:
-            x_shape = [1]
-        if len(y_shape) == 0:
-            y_shape = [1]
-        # incomplement broadcasting support for paddle
-        x_input = x
-        y_input = y
-        if len(x_shape) < len(y_shape):
-            unrevertable_ops = [
-                "elementwise_sub", "elementwise_div", "elementwise_floordiv",
-                "elementwise_mod", "elementwise_pow"
-            ]
-            if op_type not in unrevertable_ops:
-                x_input = y
-                y_input = x
-                x_shape = y.out_shapes[0]
-                if len(x_shape) == 0:
-                    x_shape = [1]
-                y_shape = x.out_shapes[0]
-                if len(y_shape) == 0:
-                    y_shape = [1]
-            else:
-                raise Exception("Unexpected situation happend")
-
-        if len(x_shape) == 4 and len(y_shape) == 1:
-            inputs = {"x": x_input, "y": y_input}
-            node.fluid_code.add_layer(op_type, inputs=inputs, output=node)
-            return
-
-        is_sub_seq = True
-        for i in range(len(y_shape)):
-            index = -1 * i - 1
-            if y_shape[index] != x_shape[index]:
-                is_sub_seq = False
-        if not is_sub_seq:
-            x_expand_times = [1] * len(x_shape)
-            y_expand_times = [1] * len(y_shape)
-            x_need_expand = False
-            y_need_expand = False
-            for i in range(len(y_shape)):
-                index = -1 * i - 1
-                if y_shape[index] != x_shape[index]:
-                    if y_shape[index] == 1:
-                        y_expand_times[index] = x_shape[index]
-                        y_need_expand = True
-                    elif x_shape[index] == 1:
-                        x_expand_times[index] = y_shape[index]
-                        x_need_expand = True
-                    else:
-                        raise Exception("Unexpected situation happend")
-            if x_need_expand:
-                attr = {"expand_times": x_expand_times}
-                node.fluid_code.add_layer(
-                    "expand", inputs=x_input, output="x_tmp", param_attr=attr)
-                x_input = "x_tmp"
-            if y_need_expand:
-                attr = {"expand_times": y_expand_times}
-                node.fluid_code.add_layer(
-                    "expand", inputs=y_input, output="y_tmp", param_attr=attr)
-                y_input = "y_tmp"
-        if len(x_shape) == 4 and len(y_shape) == 4:
-            node.fluid_code.add_layer(
-                "transpose",
-                inputs=x_input,
-                output=x_input,
-                param_attr={'perm': [0, 3, 1, 2]})
-            node.fluid_code.add_layer(
-                "transpose",
-                inputs=y_input,
-                output=y_input,
-                param_attr={'perm': [0, 3, 1, 2]})
-            inputs = {"x": x_input, "y": y_input}
-            node.fluid_code.add_layer(
-                op_type, inputs=inputs, output=node, param_attr=None)
-            node.fluid_code.add_layer(
-                "transpose",
-                inputs=node,
-                output=node,
-                param_attr={'perm': [0, 2, 3, 1]})
-        else:
-            inputs = {"x": x_input, "y": y_input}
-            node.fluid_code.add_layer(
-                op_type, inputs=inputs, output=node, param_attr=None)
+        x = self.graph.get_node(node.layer.input[0])
+        y = self.graph.get_node(node.layer.input[1])
+        program.add_layer(
+            kernel="fluid.layers.{}".format(op_type),
+            inputs={"x": x.name,
+                    "y": y.name},
+            outputs=[node.name])
 
     def Placeholder(self, node):
         shape = node.out_shapes[0]
         assert len(shape) != 0, "Unknown shape of input nodes[{}].".format(
             node.layer_name)
         dtype = node.dtype
-        if shape[0] < 0:
-            self.batch_node = node
-        attr = {
-            'dtype': string(dtype),
-            'shape': shape,
-            'name': string(node.layer_name),
-            'append_batch_size': False
-        }
-
-        node.fluid_code.add_layer(
-            "data", inputs=None, output=node, param_attr=attr)
+        program.add_layer(
+            kernel="fluid.data",
+            inputs={},
+            outputs=[node.name],
+            dtype=string(dtype),
+            shape=shape,
+            name=string(node.name))
 
     def Const(self, node):
         shape = node.out_shapes[0]
@@ -260,477 +174,475 @@ class TFOpMapperNHWC(OpMapper):
             shape = [1]
             initializer = "Constant({})".format(value)
 
-        self.weights[node.layer_name] = node.value
-
-        attr = {
-            'dtype': string(dtype),
-            'shape': shape,
-            'name': string(node.layer_name),
-            'default_initializer': initializer
-        }
-        node.fluid_code.add_layer(
-            "create_parameter", inputs=None, output=node, param_attr=attr)
+        program.parameters[node.name] = node.value
+        program.add_layer(
+            kernel="fluid.layers.create_parameter",
+            inputs={},
+            outputs=[node.name],
+            dtype=string(dtype),
+            shape=shape,
+            name=string(node.name),
+            default_initializer=initializer)
 
     def Transpose(self, node):
-        input = self.graph.get_node(node.layer.input[0], copy=True)
-        perm = self.graph.get_node(node.layer.input[1], copy=True)
+        input = self.graph.get_node(node.layer.input[0])
+        perm = self.graph.get_node(node.layer.input[1])
         assert perm.layer_type == "Const", "Perm of transpose OP should be Const"
-        del self.weights[perm.layer_name.replace('/', '_')]
-        perm.fluid_code.clear()
         perm = perm.value.tolist()
 
-        attr = {'perm': perm}
-        node.fluid_code.add_layer(
-            "transpose", inputs=input, output=node, param_attr=attr)
+        program.add_layer(
+            kernel="fluid.layers.transpose",
+            inputs={"x": input.name},
+            outputs=[node.name],
+            perm=perm)
+
+    def Fill(self, node):
+        dims = self.graph.get_node(node.layer.input[0], copy=True)
+        input_value = self.graph.get_node(node.layer.input[1], copy=True)
+        assert input_value.layer_type == "Const", "Value of fill OP should be Const"
+
+        input_value = input_value.value
+        input_dtype = string(input_value.dtype)
+        program.add_layer(
+            "fluid.layers.fill_constant",
+            inputs={},
+            outputs=[node.name],
+            shape=dims,
+            dtype=string(input_dtype),
+            value=input_value)
+
+    def DepthToSpace(self, node):
+        input = self.graph.get_node(node.layer.input[0], copy=True)
+
+        block_size = node.get_attr("block_size")
+        data_format = node.get_attr("data_format").decode()
+        n, h, w, c = input.out_shapes[0]
+
+        input_name = input.name
+        if data_format == "NHWC":
+            transpose_name = gen_name("depth_to_space", "transpose")
+            program.add_layer(
+                kernel="fluid.layers.transpose",
+                inputs={"x": input.name},
+                outputs=[transpose_name],
+                perm=[0, 3, 1, 2])
+            input_name = transpose_name
+
+        shape = [0, block_size * block_size, -1, h, w]
+        reshape_name = gen_name("depth_to_space", "reshape")
+        program.add_layer(
+            kernel="fluid.layers.reshape",
+            inputs={"x": input_name},
+            outputs=[reshape_name],
+            shape=shape)
+
+        transpose_name = gen_name("depth_to_space", "transpose")
+        program.add_layer(
+            kernel="fluid.layers.transpose",
+            inputs={"x": reshape_name},
+            outputs=[transpose_name],
+            perm=[0, 2, 1, 3, 4])
+
+        reshape_name = gen_name("depth_to_space", "reshape")
+        program.add_layer(
+            kernel="fluid.layers.reshape",
+            inputs={"x": transpose_name},
+            outputs=[reshape_name],
+            shape=[0, c, h, w])
+
+        program.add_layer(
+            kernel="fluid.layers.pixed_shuffle",
+            inputs={"input": reshape_name},
+            outputs=[node.name],
+            upscale_factor=block_size)
+
+        if data_format == "NHWC":
+            program.add_layer(
+                kernel="fluid.layers.transpose",
+                inputs={"x": node.name},
+                outputs=[node.name],
+                perm=[0, 2, 3, 1])
 
     def MaxPool(self, node):
         input = self.graph.get_node(node.layer.input[0], copy=True)
 
-        in_shape = input.out_shapes[0]
-        if in_shape.count(-1) > 2:
-            in_shape = self.decoder.infer_tensor(input).shape
-
         k_size = node.get_attr("ksize")
         strides = node.get_attr("strides")
         data_format = node.get_attr("data_format").decode()
         pad_mode = node.get_attr("padding").decode()
-        channel_first = data_format == "NCHW"
 
-        if not channel_first:
-            attr = {"perm": [0, 3, 1, 2]}
-            node.fluid_code.add_layer(
-                "transpose", inputs=input, output=node, param_attr=attr)
-            in_shape = [in_shape[i] for i in [0, 3, 1, 2]]
+        input_name = input.name
+        if data_format == "NHWC":
+            transpose_name = gen_name("max_pool", "transpose")
+            program.add_layer(
+                kernel="fluid.layers.transpose",
+                inputs={"x": input.name},
+                outputs=[transpose_name],
+                perm=[0, 3, 1, 2])
             strides = [strides[i] for i in [0, 3, 1, 2]]
             k_size = [k_size[i] for i in [0, 3, 1, 2]]
-            input = node
-
-        attr = {
-            "pool_size": k_size[2:4],
-            "pool_type": string("max"),
-            "pool_stride": strides[2:4],
-            "pool_padding": string(pad_mode)
-        }
-        node.fluid_code.add_layer(
-            "pool2d", inputs=input, output=node, param_attr=attr)
-
-        if not channel_first:
-            attr = {"perm": [0, 2, 3, 1]}
-            node.fluid_code.add_layer(
-                "transpose", inputs=node, output=node, param_attr=attr)
+            input_name = transpose_name
+
+        program.add_layer(
+            kernel="fluid.layers.pool2d",
+            inputs={"input": input_name},
+            outputs=[node.name],
+            pool_size=k_size[2:4],
+            pool_type=string("max"),
+            pool_stride=strides[2:4],
+            pool_padding=string(pad_mode))
+
+        if data_format == "NHWC":
+            program.add_layer(
+                kernel="fluid.layers.transpose",
+                inputs={"x": node.name},
+                outputs=[node.name],
+                perm=[0, 2, 3, 1])
 
     def Conv2D(self, node):
-        input = self.graph.get_node(node.layer.input[0], copy=True)
-        kernel = self.graph.get_node(node.layer.input[1], copy=True)
-        self.add_omit_nodes(kernel.layer_name, node.layer_name)
+        input = self.graph.get_node(node.layer.input[0])
+        kernel = self.graph.get_node(node.layer.input[1])
 
-        in_shape = input.out_shapes[0]
-        if in_shape.count(-1) > 2:
-            in_shape = self.decoder.infer_tensor(input).shape
         k_size = kernel.out_shapes[0]
-        if k_size.count(-1) > 2:
-            k_size = self.decoder.infer_tensor(kernel).shape
-
         strides = node.get_attr("strides")
         dilations = node.get_attr("dilations")
         data_format = node.get_attr("data_format").decode()
         pad_mode = node.get_attr("padding").decode()
-        channel_first = data_format == "NCHW"
 
         if kernel.layer_type == 'Const':
             kernel_value = kernel.value
+            kernel_weight_name = kernel.layer_name.replace('/', '_')
         else:
             kernel_value = self.decoder.infer_tensor(kernel)
-        self.weights[kernel.layer_name.replace('/', '_')] = numpy.transpose(
-            kernel_value, (3, 2, 0, 1))
+            if kernel.layer_type == 'Split':
+                kernel_weight_name = "{}_{}_kernel".format(node.layer_name,
+                                                           kernel.layer_name)
+            else:
+                kernel_weight_name = kernel.layer_name.replace('/', '_')
+        program.parameters[kernel_weight_name] = numpy.transpose(kernel_value,
+                                                                 (3, 2, 0, 1))
 
-        if not channel_first:
-            in_shape = [in_shape[i] for i in [0, 3, 1, 2]]
+        input_name = input.name
+        if data_format == "NHWC":
             strides = [strides[i] for i in [0, 3, 1, 2]]
             dilations = [dilations[i] for i in [0, 3, 1, 2]]
-            attr = {"perm": [0, 3, 1, 2]}
-            node.fluid_code.add_layer(
-                "transpose", inputs=input, output=node, param_attr=attr)
-            input = node
-
-        attr = {
-            "bias_attr": False,
-            "param_attr": string(kernel.layer_name),
-            "num_filters": k_size[3],
-            "filter_size": k_size[0:2],
-            "stride": strides[2:4],
-            "dilation": dilations[2:4],
-            "padding": string(pad_mode)
-        }
-
-        if hasattr(node, 'dilation') and attr['dilation'] == [1, 1]:
-            if len(node.dilation) == 1:
-                attr['dilation'] = [1, node.dilation[0]]
-
-        node.fluid_code.add_layer(
-            "conv2d", inputs=input, output=node, param_attr=attr)
-        if not channel_first:
-            attr = {"perm": [0, 2, 3, 1]}
-            node.fluid_code.add_layer(
-                "transpose", inputs=node, output=node, param_attr=attr)
+            transpose_name = gen_name("conv2d", "transpose")
+            program.add_layer(
+                kernel="fluid.layers.transpose",
+                inputs={"x": input.name},
+                outputs=[transpose_name],
+                perm=[0, 3, 1, 2])
+            input_name = transpose_name
+
+        program.add_layer(
+            kernel="fluid.layers.conv2d",
+            inputs={"input": input_name},
+            outputs=[node.name],
+            bias_attr=False,
+            param_attr=string(kernel_weight_name),
+            num_filters=k_size[3],
+            filter_size=k_size[0:2],
+            stride=strides[2:4],
+            dilation=dilations[2:4],
+            padding=string(pad_mode))
+
+        if data_format == "NHWC":
+            program.add_layer(
+                kernel="fluid.layers.transpose",
+                inputs={"x": node.name},
+                outputs=[node.name],
+                perm=[0, 2, 3, 1])
 
     def BiasAdd(self, node):
-        input = self.graph.get_node(node.layer.input[0], copy=True)
-        bias = self.graph.get_node(node.layer.input[1], copy=True)
-        inputs = {"x": input, "y": bias}
-        node.fluid_code.add_layer(
-            "elementwise_add", inputs=inputs, output=node, param_attr=None)
+        input = self.graph.get_node(node.layer.input[0])
+        bias = self.graph.get_node(node.layer.input[1])
+        program.add_layer(
+            kernel="fluid.layers.elementwise_add",
+            inputs={"x": input.name,
+                    "y": bias.name},
+            outputs=[node.name])
 
     def FusedBatchNorm(self, node):
-        input = self.graph.get_node(node.layer.input[0], copy=True)
-        gamma = self.graph.get_node(node.layer.input[1], copy=True)
-        beta = self.graph.get_node(node.layer.input[2], copy=True)
-        moving_mean = self.graph.get_node(node.layer.input[3], copy=True)
-        moving_var = self.graph.get_node(node.layer.input[4], copy=True)
+        input = self.graph.get_node(node.layer.input[0])
+        gamma = self.graph.get_node(node.layer.input[1])
+        beta = self.graph.get_node(node.layer.input[2])
+        moving_mean = self.graph.get_node(node.layer.input[3])
+        moving_var = self.graph.get_node(node.layer.input[4])
         data_format = node.get_attr("data_format").decode()
-        channel_first = data_format == "NCHW"
 
         assert gamma.layer_type == "Const"
         assert beta.layer_type == "Const"
         assert moving_mean.layer_type == "Const"
         assert moving_var.layer_type == "Const"
-        self.add_omit_nodes(gamma.layer_name, node.layer_name)
-        self.add_omit_nodes(beta.layer_name, node.layer_name)
-        self.add_omit_nodes(moving_mean.layer_name, node.layer_name)
-        self.add_omit_nodes(moving_var.layer_name, node.layer_name)
-
-        if not channel_first:
-            attr = {"perm": [0, 3, 1, 2]}
-            node.fluid_code.add_layer(
-                "transpose", inputs=input, output=node, param_attr=attr)
-            input = node
-
-        attr = {
-            "epsilon": node.get_attr("epsilon"),
-            "param_attr": string(gamma.layer_name),
-            "bias_attr": string(beta.layer_name),
-            "moving_mean_name": string(moving_mean.layer_name),
-            "moving_variance_name": string(moving_var.layer_name),
-            "is_test": True
-        }
-
-        node.fluid_code.add_layer(
-            "batch_norm", inputs=input, output=node, param_attr=attr)
-
-        if not channel_first:
-            attr = {"perm": [0, 2, 3, 1]}
-            node.fluid_code.add_layer(
-                "transpose", inputs=node, output=node, param_attr=attr)
+
+        input_name = input.name
+        if data_format == "NHWC":
+            transpose_name = gen_name("batch_norm", "transpose")
+            program.add_layer(
+                kernel="fluid.layers.transpose",
+                inputs={"x": input.name},
+                outputs=[transpose_name],
+                perm=[0, 3, 1, 2])
+            input_name = transpose_name
+
+        program.add_layer(
+            kernel="fluid.layers.batch_norm",
+            inputs={"input": input_name},
+            outputs=[node.name],
+            epsilon=node.get_attr("epsilon"),
+            param_attr=string(gamma.name),
+            bias_attr=string(beta.name),
+            moving_mean_name=string(moving_mean.name),
+            moving_variance_name=string(moving_var.name),
+            is_test=True)
+
+        if data_format == "NHWC":
+            program.add_layer(
+                kernel="fluid.layers.transpose",
+                inputs={"x": node.name},
+                outputs=[node.name],
+                perm=[0, 2, 3, 1])
+
+    def Mean(self, node):
+        input = self.graph.get_node(node.layer.input[0])
+        reduce_idx = self.graph.get_node(node.layer.input[1])
+        assert reduce_idx.layer_type == "Const", "Only support Const parameter[reduce_idx]"
+        dims = reduce_idx.value.tolist()
+        keep_dims = node.get_attr("keep_dims")
+
+        program.add_layer(
+            kernel="fluid.layers.reduce_mean",
+            inputs={"input": input.name},
+            outputs=[node.name],
+            dim=dims,
+            keep_dim=keep_dims)
+
+    def Reshape(self, node):
+        input = self.graph.get_node(node.layer.input[0])
+        param = self.graph.get_node(node.layer.input[1])
+        if param.layer_type == "Const":
+            shape = param.value.tolist()
+            program.add_layer(
+                kernel="fluid.layers.reshape",
+                inputs={"x": input.name},
+                outputs=[node.name],
+                shape=shape)
+        else:
+            program.add_layer(
+                kernel="fluid.layers.reshape",
+                inputs={"x": input.name,
+                        "shape": param.name},
+                outputs=[node.name])
+        if param.layer_type != "Const":
+            out_shape = numpy.array(node.out_shapes[0])
+            if (out_shape > 0).any():
+                out_shape[out_shape < 0] = 0
+                program.add_layer(
+                    kernel="fluid.layers.reshape",
+                    inputs={"x": node.name},
+                    outputs=[node.name],
+                    shape=out_shape.tolist())
+
+    def Pad(self, node):
+        input = self.graph.get_node(node.layer.input[0])
+        paddings = self.graph.get_node(node.layer.input[1])
+        assert paddings.layer_type == "Const", "Padding should be Const"
+        paddings = paddings.value.flatten().tolist()
+
+        if len(input.out_shapes[0]) == 4:
+            if paddings[0] + paddings[1] + paddings[6] + paddings[7] == 0:
+                new_padding = paddings[2:6]
+                transpose_name = gen_name("pad", "transpose")
+                program.add_layer(
+                    kernel="fluid.layers.transpose",
+                    inputs={"x": input.name},
+                    outputs=[transpose_name],
+                    perm=[0, 3, 1, 2])
+                program.add_layer(
+                    kernel="fluid.layers.pad2d",
+                    inputs={"input": transpose_name},
+                    outputs=[node.name],
+                    paddings=new_padding)
+                program.add_layer(
+                    kernel="fluid.layers.transpose",
+                    inputs={"x": node.name},
+                    outputs=[node.name],
+                    perm=[0, 2, 3, 1])
+                return
+
+        program.add_layer(
+            kernel="fluid.layers.pad",
+            inputs={"input": input.name},
+            outputs=[node.name],
+            paddings=paddings)
+
+    def Squeeze(self, node):
+        input = self.graph.get_node(node.layer.input[0])
+        squeeze_dims = node.get_attr('squeeze_dims')
+        program.add_layer(
+            kernel="fluid.layers.squeeze",
+            inputs={"input": input.name},
+            outputs=[node.name],
+            axes=squeeze_dims)
+
+    def Softmax(self, node):
+        input = self.graph.get_node(node.layer.input[0])
+        axis = node.get_attr("axis")
+        program.add_layer(
+            kernel="fluid.layers.softmax",
+            inputs={"input": input.name},
+            outputs=[node.name],
+            axis=axis)
+
+    def Shape(self, node):
+        input = self.graph.get_node(node.layer.input[0])
+        program.add_layer(
+            kernel="fluid.layers.shape",
+            inputs={"input": input.name},
+            outputs=[node.name])
+
+    def ArgMax(self, node):
+        input = self.graph.get_node(node.layer.input[0])
+        axis = self.graph.get_node(node.layer.input[1])
+        assert axis.layer_type == "Const", "ArgMax only support Const parameter"
+        axis = axis.value
+        program.add_layer(
+            kernel="fluid.layers.argmax",
+            inputs={"x": input.name},
+            outputs=[node.name],
+            axis=axis)
+
+    def MatMul(self, node):
+        x = self.graph.get_node(node.layer.input[0])
+        y = self.graph.get_node(node.layer.input[1])
+        transpose_a = node.get_attr('transpose_a')
+        transpose_b = node.get_attr('transpose_b')
+        if transpose_a is None:
+            transpose_a = node.get_attr('adj_x')
+        if transpose_b is None:
+            transpose_b = node.get_attr('adj_y')
+        program.add_layer(
+            kernel="fluid.layers.matmul",
+            inputs={"x": x.name,
+                    "y": y.name},
+            outputs=[node.name],
+            transpose_x=transpose_a,
+            transpose_y=transpose_b)
 
     def DepthwiseConv2dNative(self, node):
-        input = self.graph.get_node(node.layer.input[0], copy=True)
-        kernel = self.graph.get_node(node.layer.input[1], copy=True)
+        input = self.graph.get_node(node.layer.input[0])
+        kernel = self.graph.get_node(node.layer.input[1])
         assert kernel.layer_type == "Const", "Kernel of DepthwiseConv2DNative should be Const"
-        self.add_omit_nodes(kernel.layer_name, node.layer_name)
 
         in_shape = input.out_shapes[0]
-        if in_shape.count(-1) > 2:
-            in_shape = self.decoder.infer_tensor(input).shape
         k_size = kernel.out_shapes[0]
-        if k_size.count(-1) > 2:
-            k_size = self.decoder.infer_tensor(kernel).shape
-
         strides = node.get_attr("strides")
         dilations = node.get_attr("dilations")
         data_format = node.get_attr("data_format").decode()
         pad_mode = node.get_attr("padding").decode()
-        channel_first = data_format == "NCHW"
 
-        self.weights[kernel.layer_name.replace('/', '_')] = numpy.transpose(
-            kernel.value, (2, 3, 0, 1))
+        program.parameters[kernel.layer_name.replace(
+            '/', '_')] = numpy.transpose(kernel.value, (2, 3, 0, 1))
 
-        if not channel_first:
+        input_name = input.name
+        if data_format == "NHWC":
             in_shape = [in_shape[i] for i in [0, 3, 1, 2]]
             strides = [strides[i] for i in [0, 3, 1, 2]]
             dilations = [dilations[i] for i in [0, 3, 1, 2]]
-            attr = {"perm": [0, 3, 1, 2]}
-            node.fluid_code.add_layer(
-                "transpose", inputs=input, output=node, param_attr=attr)
-            input = node
-
-        attr = {
-            "bias_attr": False,
-            "param_attr": string(kernel.layer_name),
-            "num_filters": in_shape[1],
-            "filter_size": k_size[0:2],
-            "stride": strides[2:4],
-            "dilation": dilations[2:4],
-            "groups": k_size[3] * in_shape[1],
-            "use_cudnn": False,
-            "padding": string(pad_mode)
-        }
-        node.fluid_code.add_layer(
-            "conv2d", inputs=input, output=node, param_attr=attr)
-
-        if not channel_first:
-            attr = {"perm": [0, 2, 3, 1]}
-            node.fluid_code.add_layer(
-                "transpose", inputs=node, output=node, param_attr=attr)
-
-    def Reshape(self, node):
-        input = self.graph.get_node(node.layer.input[0], copy=True)
-        param = self.graph.get_node(node.layer.input[1], copy=True)
-        is_variable = False
-        if param.layer_type == "Const":
-            attr = {"shape": param.value.tolist()}
-            self.add_omit_nodes(param.layer_name, node.layer_name)
-        else:
-            # Here is a trick method to solove tensor parameter in tensorflow
-            shape = self.decoder.infer_shape_tensor(param, node.out_shapes[0])
-            if shape.count(-1) <= 1:
-                attr = {"shape": shape}
-                self.add_omit_nodes(param.layer_name, node.layer_name)
-            else:
-                assert len(param.out_shapes[
-                    0]) == 1, "Unexpected situation of shape parameter"
-                attr = {"shape": [-1]}
-                node.fluid_code.add_layer(
-                    "reshape",
-                    inputs=param,
-                    output="shape_param",
-                    param_attr=attr)
-                attr = {"num_or_sections": param.out_shapes[0][0], "dim": 0}
-                node.fluid_code.add_layer(
-                    "split", inputs="shape_param", output=node, param_attr=attr)
-                new_param = "["
-                for i in range(param.out_shapes[0][0]):
-                    new_param += (node.layer_name + "[{}]".format(i) + ", ")
-                new_param = new_param.strip(", ") + "]"
-                attr = {"shape": new_param}
-                is_variable = True
-        # to change [192, -1]->[-1, 192], allways put -1 in the first dimension
-        # optimization for Paddle-Lite
-        in_shape = input.out_shapes[0]
-        if not is_variable and in_shape.count(-1) < 1:
-            total_size = 1
-            for i in range(len(in_shape)):
-                total_size *= in_shape[i]
-            for i in range(len(attr["shape"])):
-                if attr["shape"][i] == 0:
-                    attr["shape"][i] = in_shape[i]
-                if attr["shape"][i] != -1:
-                    total_size /= attr["shape"][i]
-            if attr["shape"].count(-1) > 0:
-                index = attr["shape"].index(-1)
-                attr["shape"][index] = int(total_size)
-                attr["shape"][0] = -1
-
-        node.fluid_code.add_layer(
-            "reshape", inputs=input, output=node, param_attr=attr)
+            transpose_name = gen_name('depthwise_conv2d', 'transpose')
+            program.add_layer(
+                kernel="fluid.layers.transpose",
+                inputs={"x": input.name},
+                outputs=[transpose_name],
+                perm=[0, 3, 1, 2])
+            input_name = transpose_name
+
+        program.add_layer(
+            kernel="fluid.layers.conv2d",
+            inputs={"input": input_name},
+            outputs=[node.name],
+            num_filters=in_shape[1],
+            filter_size=k_size[0:2],
+            stride=strides[2:4],
+            dilation=dilations[2:4],
+            groups=k_size[3] * in_shape[1],
+            padding=string(pad_mode),
+            param_attr=string(kernel.layer_name),
+            bias_attr=False)
+
+        if data_format == "NHWC":
+            program.add_layer(
+                kernel="fluid.layers.transpose",
+                inputs={"x": node.name},
+                outputs=[node.name],
+                perm=[0, 2, 3, 1])
 
     def AvgPool(self, node):
         input = self.graph.get_node(node.layer.input[0], copy=True)
 
-        in_shape = input.out_shapes[0]
-        if in_shape.count(-1) > 2:
-            in_shape = self.decoder.infer_tensor(input).shape
-
         k_size = node.get_attr("ksize")
         strides = node.get_attr("strides")
         data_format = node.get_attr("data_format").decode()
         pad_mode = node.get_attr("padding").decode()
-        channel_first = data_format == "NCHW"
 
-        if not channel_first:
-            in_shape = [in_shape[i] for i in [0, 3, 1, 2]]
+        input_name = input.name
+        if data_format == "NHWC":
+            transpose_name = gen_name("avg_pool", "transpose")
+            program.add_layer(
+                kernel="fluid.layers.transpose",
+                inputs={"x": input.name},
+                outputs=[transpose_name],
+                perm=[0, 3, 1, 2])
             strides = [strides[i] for i in [0, 3, 1, 2]]
             k_size = [k_size[i] for i in [0, 3, 1, 2]]
-            attr = {"perm": [0, 3, 1, 2]}
-            node.fluid_code.add_layer(
-                "transpose", inputs=input, output=node, param_attr=attr)
-            input = node
-
-        attr = {
-            "pool_size": k_size[2:4],
-            "pool_type": string("avg"),
-            "pool_stride": strides[2:4],
-            "pool_padding": string(pad_mode)
-        }
-        node.fluid_code.add_layer(
-            "pool2d", inputs=input, output=node, param_attr=attr)
-
-        if not channel_first:
-            attr = {"perm": [0, 2, 3, 1]}
-            node.fluid_code.add_layer(
-                "transpose", inputs=node, output=node, param_attr=attr)
-
-    def SplitV(self, node):
-        input = self.graph.get_node(node.layer.input[0], copy=True)
-        num_sections = self.graph.get_node(node.layer.input[1], copy=True)
-        dim = self.graph.get_node(node.layer.input[2], copy=True)
-        assert num_sections.layer_type == "Const"
-        assert dim.layer_type == "Const"
-        self.add_omit_nodes(num_sections.layer_name, node.layer_name)
-        self.add_omit_nodes(dim.layer_name, node.layer_name)
-        dim = dim.value
-        attr = {
-            "num_or_sections": num_sections.value.tolist(),
-            "dim": dim.value
-        }
-        node.fluid_code.add_layer(
-            "split", inputs=input, output=node, param_attr=attr)
-
-    def ConcatV2(self, node):
-        inputs = [
-            self.graph.get_node(
-                name, copy=True) for name in node.layer.input[:-1]
-        ]
-        axis = self.graph.get_node(node.layer.input[-1], copy=True)
-        assert axis.layer_type == "Const"
-        self.add_omit_nodes(axis.layer_name, node.layer_name)
-        axis = axis.value
-        if axis < 0:
-            axis += len(inputs[0].out_shapes[0])
-
-        attr = {"axis": axis}
-        node.fluid_code.add_layer(
-            "concat", inputs=inputs, output=node, param_attr=attr)
-
-    def Tile(self, node):
-        input = self.graph.get_node(node.layer.input[0], copy=True)
-        expand_times = self.graph.get_node(node.layer.input[1], copy=True)
-        self.add_omit_nodes(expand_times.layer_name, node.layer_name)
-        if expand_times.layer_type == "Const":
-            expand_times = expand_times.value.tolist()
-        else:
-            expand_times = self.decoder.infer_shape_tensor(expand_times)
-        for i in range(len(expand_times)):
-            if expand_times[i] < 0:
-                expand_times[i] = 1
-        attr = {"expand_times": expand_times}
-        node.fluid_code.add_layer(
-            "expand", inputs=input, output=node, param_attr=attr)
+            input_name = transpose_name
+
+        program.add_layer(
+            kernel="fluid.layers.pool2d",
+            inputs={"input": input_name},
+            outputs=[node.name],
+            pool_size=k_size[2:4],
+            pool_type=string("avg"),
+            pool_stride=strides[2:4],
+            pool_padding=string(pad_mode))
+
+        if data_format == "NHWC":
+            program.add_layer(
+                kernel="fluid.layers.transpose",
+                inputs={"x": node.name},
+                outputs=[node.name],
+                perm=[0, 2, 3, 1])
 
     def Pack(self, node):
-        inputs = [
-            self.graph.get_node(
-                name, copy=True) for name in node.layer.input
-        ]
+        inputs = [self.graph.get_node(name) for name in node.layer.input]
         axis = node.get_attr("axis")
-        attr = {"axis": axis}
-        node.fluid_code.add_layer(
-            "stack", inputs=inputs, output=node, param_attr=attr)
+        program.add_layer(
+            kernel="fluid.layers.stack",
+            inputs={"x": [i.name for i in inputs]},
+            outputs=[node.name],
+            axis=axis)
 
-    def Pad(self, node):
-        input = self.graph.get_node(node.layer.input[0], copy=True)
-        paddings = self.graph.get_node(node.layer.input[1], copy=True)
-        assert paddings.layer_type == "Const", "Padding should be Const"
-        self.add_omit_nodes(paddings.layer_name, node.layer_name)
-        paddings = paddings.value.flatten().tolist()
-        data_format = input.tf_data_format
-
-        if len(input.out_shapes[0]) == 4:
-            new_padding = None
-            if input.tf_data_format == "NHWC":
-                if paddings[0] + paddings[1] + paddings[6] + paddings[7] == 0:
-                    new_padding = paddings[2:6]
-            else:
-                if paddings[0] + paddings[1] + paddings[2] + paddings[3] == 0:
-                    new_padding = paddings[4:]
-            if new_padding is not None:
-                if input.tf_data_format == "NHWC":
-                    attr = {"perm": [0, 3, 1, 2]}
-                    node.fluid_code.add_layer(
-                        "transpose", inputs=input, output=node, param_attr=attr)
-                    input = node
-                attr = {"paddings": new_padding}
-                node.fluid_code.add_layer(
-                    "pad2d", inputs=input, output=node, param_attr=attr)
-                if input.tf_data_format == "NHWC":
-                    attr = {"perm": [0, 2, 3, 1]}
-                    node.fluid_code.add_layer(
-                        "transpose", inputs=node, output=node, param_attr=attr)
-
-                return
-
-        attr = {"paddings": paddings}
-        node.fluid_code.add_layer(
-            "pad", inputs=input, output=node, param_attr=attr)
-
-    def Range(self, node):
-        start = self.graph.get_node(node.layer.input[0], copy=True)
-        limit = self.graph.get_node(node.layer.input[1], copy=True)
-        delta = self.graph.get_node(node.layer.input[2], copy=True)
-        self.add_omit_nodes(start.layer_name, node.layer_name)
-        self.add_omit_nodes(limit.layer_name, node.layer_name)
-        self.add_omit_nodes(delta.layer_name, node.layer_name)
-        if start.layer_type == "Const":
-            start = start.value
-        else:
-            start = self.decoder.infer_tensor(start)
-        if limit.layer_type == "Const":
-            limit = limit.value
-        else:
-            limit = self.decoder.infer_tensor(limit)
-        if delta.layer_type == "Const":
-            delta = delta.value
-        else:
-            delta = self.decoder.infer_tensor(delta)
-        dtype = node.dtype
-        inputs = {
-            "start": start,
-            "end": limit,
-            "step": delta,
-        }
-        attr = {"dtype": string(node.dtype)}
-        node.fluid_code.add_layer(
-            "range", inputs=inputs, output=node, param_attr=attr)
-
-    def Mean(self, node):
-        input = self.graph.get_node(node.layer.input[0], copy=True)
-        reduce_idx = self.graph.get_node(node.layer.input[1], copy=True)
-        assert reduce_idx.layer_type == "Const", "Only support Const parameter[reduce_idx]"
-        dims = reduce_idx.value.tolist()
-        keep_dims = node.get_attr("keep_dims")
-
-        attr = {"dim": dims, "keep_dim": keep_dims}
-        node.fluid_code.add_layer(
-            "reduce_mean", inputs=input, output=node, param_attr=attr)
-
-    def MatMul(self, node):
-        x = self.graph.get_node(node.layer.input[0], copy=True)
-        y = self.graph.get_node(node.layer.input[1], copy=True)
-        transpose_a = node.get_attr('transpose_a')
-        transpose_b = node.get_attr('transpose_b')
-        inputs = {"x": x, "y": y}
-        # fix paddle shape infer problem
-        # should be removed after paddle 1.6
-        if x.out_shapes[0][-1] < 0 and y.out_shapes[0][0] > 0:
-            shape = x.out_shapes[0]
-            shape[-1] = y.out_shapes[0][0]
-            attr = {"shape": shape}
-            node.fluid_code.add_layer(
-                "reshape", inputs=x, output=x, param_attr=attr)
-        attr = {"transpose_x": transpose_a, "transpose_y": transpose_b}
-        node.fluid_code.add_layer(
-            "matmul", inputs=inputs, output=node, param_attr=attr)
-
-    def ArgMax(self, node):
-        input = self.graph.get_node(node.layer.input[0], copy=True)
-        axis = self.graph.get_node(node.layer.input[1], copy=True)
-        assert axis.layer_type == "Const", "ArgMax only support Const parameter"
-        self.add_omit_nodes(axis.layer_name, node.layer_name)
+    def ConcatV2(self, node):
+        inputs = [self.graph.get_node(name) for name in node.layer.input[:-1]]
+        axis = self.graph.get_node(node.layer.input[-1])
+        assert axis.layer_type == "Const", "axis for ConcatV2 must be type Const"
         axis = axis.value
-        attr = {"axis": axis}
-        node.fluid_code.add_layer(
-            "argmax", inputs=input, output=node, param_attr=attr)
+        if axis < 0:
+            axis += len(inputs[0].out_shapes[0])
+        program.add_layer(
+            kernel="fluid.layers.concat",
+            inputs={"input": [i.name for i in inputs]},
+            outputs=[node.name],
+            axis=axis)
 
     def StridedSlice(self, node):
-        input = self.graph.get_node(node.layer.input[0], copy=True)
-        begin = self.graph.get_node(node.layer.input[1], copy=True)
-        end = self.graph.get_node(node.layer.input[2], copy=True)
-        strides = self.graph.get_node(node.layer.input[3], copy=True)
+        input = self.graph.get_node(node.layer.input[0])
+        begin = self.graph.get_node(node.layer.input[1])
+        end = self.graph.get_node(node.layer.input[2])
+        strides = self.graph.get_node(node.layer.input[3])
         assert begin.layer_type == "Const"
         assert end.layer_type == "Const"
         assert strides.layer_type == "Const"
-        self.add_omit_nodes(begin.layer_name, node.layer_name)
-        self.add_omit_nodes(end.layer_name, node.layer_name)
-        self.add_omit_nodes(strides.layer_name, node.layer_name)
         strides = strides.value.tolist()
         assert len(set(strides)) == 1 and strides[
             0] == 1, "Only support strides be 1 in StridedSlice OP"
@@ -779,65 +691,228 @@ class TFOpMapperNHWC(OpMapper):
             else:
                 new_end.append(end[i])
 
-        attr = {
-            "axes": [i for i in range(len(new_begin))],
-            "starts": new_begin,
-            "ends": new_end
-        }
-        node.fluid_code.add_layer(
-            "slice", inputs=input, output=node, param_attr=attr)
+        program.add_layer(
+            kernel="fluid.layers.slice",
+            inputs={"input": input.name},
+            outputs=[node.name],
+            axes=[i for i in range(len(new_begin))],
+            starts=new_begin,
+            ends=new_end)
         if len(new_axes) > 0:
-            attr = {"axes": new_axes}
-            node.fluid_code.add_layer(
-                "unsqueeze", inputs=node, output=node, param_attr=attr)
+            program.add_layer(
+                kernel="fluid.layers.unsqueeze",
+                inputs={"x": node.name},
+                outputs=[node.name],
+                axes=new_axes)
         if len(shrink_axes) > 0:
             if len(input.out_shapes[0]) + len(new_axes) <= 1:
                 pass
             else:
-                attr = {"axes": shrink_axes}
-                node.fluid_code.add_layer(
-                    "squeeze", inputs=node, output=node, param_attr=attr)
+                program.add_layer(
+                    kernel="fluid.layers.unsqueeze",
+                    inputs={"x": node.name},
+                    outputs=[node.name],
+                    axes=new_axes)
+
+    def Split(self, node):
+        dim = self.graph.get_node(node.layer.input[0])
+        input = self.graph.get_node(node.layer.input[1])
+        assert dim.layer_type == "Const"
+        num_split = node.get_attr('num_split')
+        dim = dim.value
+
+        program.add_layer(
+            kernel="fluid.layers.split",
+            inputs={"input": input.name},
+            outputs=[
+                "{}_p{}".format(node.layer_name, i) for i in range(num_split)
+            ],
+            num_or_sections=num_split,
+            dim=dim)
 
     def Slice(self, node):
-        input = self.graph.get_node(node.layer.input[0], copy=True)
-        begin = self.graph.get_node(node.layer.input[1], copy=True)
-        size = self.graph.get_node(node.layer.input[2], copy=True)
-        self.add_omit_nodes(begin.layer_name, node.layer_name)
-        self.add_omit_nodes(size.layer_name, node.layer_name)
+        input = self.graph.get_node(node.layer.input[0])
+        begin = self.graph.get_node(node.layer.input[1])
+        size = self.graph.get_node(node.layer.input[2])
+
+        inputs = {"x": input.name}
+        attrs = {}
         if begin.layer_type == "Const":
             begin = begin.value.tolist()
+            attrs['offsets'] = begin
         else:
-            begin = self.decoder.infer_tensor(begin).tolist()
-        if size.layer_type == "const":
+            shape = begin.out_shapes[0]
+            reshape_name = gen_name("slice", "reshape")
+            program.add_layer(
+                kernel="fluid.layers.reshape",
+                inputs={"x": begin.name},
+                outputs=[reshape_name],
+                shape=shape)
+            inputs['offsets'] = reshape_name
+        if size.layer_type == "Const":
             size = size.value.tolist()
+            attrs['shape'] = size
         else:
-            size = self.decoder.infer_tensor(size).tolist()
+            shape = size.out_shapes[0]
+            reshape_name = gen_name("slice", "reshape")
+            program.add_layer(
+                kernel="fluid.layers.reshape",
+                inputs={"x": size.name},
+                outputs=[reshape_name],
+                shape=shape)
+            inputs['shape'] = reshape_name
+        program.add_layer(
+            kernel="fluid.layers.crop_tensor",
+            inputs=inputs,
+            outputs=[node.name],
+            **attrs)
 
-        for i in range(len(size)):
-            if size[i] < 0:
-                size[i] = 99999999
-            else:
-                size[i] = size[i] + begin[i]
+    def ResizeNearestNeighbor(self, node):
+        input = self.graph.get_node(node.layer.input[0])
+        resize_shape = self.graph.get_node(node.layer.input[1])
+        data_format = "NHWC"
+        inputs = {"input": input.name}
+        attrs = {"align_corners": node.get_attr("align_corners")}
+
+        if resize_shape.layer_type == "Const":
+            resize_shape = resize_shape.value.tolist()
+            attrs["out_shape"] = resize_shape
+        else:
+            shape = resize_shape.out_shapes[0]
+            reshape_name = gen_name("resize_nearest", "reshape")
+            program.add_layer(
+                kernel="fluid.layers.reshape",
+                inputs={"x": resize_shape.name},
+                outputs=[reshape_name],
+                shape=shape)
+            inputs["out_shape"] = reshape_name
+
+        if data_format == "NHWC":
+            transpose_name = gen_name("resize_nearest", "reshape")
+            program.add_layer(
+                kernel="fluid.layers.transpose",
+                inputs={"x": input.name},
+                outputs=[transpose_name],
+                perm=[0, 3, 1, 2])
+            inputs["input"] = transpose_name
+
+        program.add_layer(
+            kernel="fluid.layers.resize_nearest",
+            inputs=inputs,
+            outputs=[node.name],
+            **attrs)
+
+        if data_format == "NHWC":
+            program.add_layer(
+                kernel="fluid.layers.transpose",
+                inputs={"x": node.name},
+                outputs=[node.name],
+                perm=[0, 2, 3, 1])
+
+    def ResizeBilinear(self, node):
+        input = self.graph.get_node(node.layer.input[0])
+        resize_shape = self.graph.get_node(node.layer.input[1])
+        data_format = "NHWC"
+        inputs = {"input": input.name}
+        attrs = {"align_corners": node.get_attr("align_corners")}
+
+        if resize_shape.layer_type == "Const":
+            resize_shape = resize_shape.value.tolist()
+            attrs["out_shape"] = resize_shape
+        else:
+            shape = resize_shape.out_shapes[0]
+            reshape_name = gen_name("resize_bilinear", "reshape")
+            program.add_layer(
+                kernel="fluid.layers.reshape",
+                inputs={"x": resize_shape.name},
+                outputs=[reshape_name],
+                shape=shape)
+            inputs["out_shape"] = reshape_name
+
+        if data_format == "NHWC":
+            transpose_name = gen_name("resize_bilinear", "reshape")
+            program.add_layer(
+                kernel="fluid.layers.transpose",
+                inputs={"x": input.name},
+                outputs=[transpose_name],
+                perm=[0, 3, 1, 2])
+            inputs["input"] = transpose_name
+
+        program.add_layer(
+            kernel="fluid.layers.resize_bilinear",
+            inputs=inputs,
+            outputs=[node.name],
+            **attrs)
+
+        if data_format == "NHWC":
+            program.add_layer(
+                kernel="fluid.layers.transpose",
+                inputs={"x": node.name},
+                outputs=[node.name],
+                perm=[0, 2, 3, 1])
 
-        attr = {
-            "axes": [i for i in range(len(size))],
-            "starts": begin,
-            "ends": size
-        }
+    def Cast(self, node):
+        input = self.graph.get_node(node.layer.input[0])
+        dtype = node.dtype
+        program.add_layer(
+            kernel="fluid.layers.cast",
+            inputs={"x": input.name},
+            outputs=[node.name],
+            dtype=string(dtype))
+
+    def Sum(self, node):
+        input = self.graph.get_node(node.layer.input[0])
+        reduce_idx = self.graph.get_node(node.layer.input[1])
+        assert reduce_idx.layer_type == "Const", "Only support Const parameter[reduce_idx]"
+        keep_dims = node.get_attr("keep_dims")
+        dim = reduce_idx.value.tolist()
 
-        node.fluid_code.add_layer(
-            "slice", inputs=input, output=node, param_attr=attr)
+        program.add_layer(
+            kernel="fluid.layers.reduce_sum",
+            inputs={"input": input.name},
+            outputs=[node.name],
+            dim=dim,
+            keep_dim=keep_dims)
+
+    def Max(self, node):
+        input = self.graph.get_node(node.layer.input[0])
+        reduce_idx = self.graph.get_node(node.layer.input[1])
+        assert reduce_idx.layer_type == "Const", "Only support Const parameter[reduce_idx]"
+        keep_dims = node.get_attr("keep_dims")
+        dim = reduce_idx.value.tolist()
+        program.add_layer(
+            kernel="fluid.layers.reduce_max",
+            inputs={"input": input.name},
+            outputs=[node.name],
+            dim=dim,
+            keep_dim=keep_dims)
+
+    def RandomUniform(self, node):
+        shape = self.graph.get_node(node.layer.input[0], copy=True)
+        if shape.layer_type == "Const":
+            shape = shape.value.tolist()
+            program.add_layer(
+                kernel="fluid.layers.uniform_random",
+                inputs={},
+                outputs=[node.name],
+                shape=shape,
+                min=0.0,
+                max=0.9999)
+        else:
+            program.add_layer(
+                kernel="fluid.layers.uniform_random",
+                inputs={'shape': shape.name},
+                outputs=[node.name],
+                min=0.0,
+                max=0.9999)
 
     def Conv2DBackpropInput(self, node):
-        out_shape = self.graph.get_node(node.layer.input[0], copy=True)
-        kernel = self.graph.get_node(node.layer.input[1], copy=True)
-        input = self.graph.get_node(node.layer.input[2], copy=True)
+        out_shape = self.graph.get_node(node.layer.input[0])
+        kernel = self.graph.get_node(node.layer.input[1])
+        input = self.graph.get_node(node.layer.input[2])
 
         assert kernel.layer_type == "Const", "Kernel of Conv2DBackpropInput should be Const"
 
-        self.add_omit_nodes(kernel.layer_name, node.layer_name)
-        self.add_omit_nodes(out_shape.layer_name, node.layer_name)
-
         if out_shape.layer_type == "Const":
             out_shape = out_shape.value.tolist()
         else:
@@ -855,201 +930,39 @@ class TFOpMapperNHWC(OpMapper):
         strides = node.get_attr("strides")
         dilations = node.get_attr("dilations")
         data_format = node.get_attr("data_format").decode()
-        channel_first = data_format == "NCHW"
 
-        self.weights[kernel.layer_name.replace('/', '_')] = numpy.transpose(
-            kernel.value, (3, 2, 0, 1))
-        if not channel_first:
+        program.parameters[kernel.layer_name.replace(
+            '/', '_')] = numpy.transpose(kernel.value, (3, 2, 0, 1))
+
+        input_name = input.name
+        if data_format == "NHWC":
             in_shape = [in_shape[i] for i in [0, 3, 1, 2]]
             strides = [strides[i] for i in [0, 3, 1, 2]]
             dilations = [dilations[i] for i in [0, 3, 1, 2]]
-            attr = {"perm": [0, 3, 1, 2]}
-            node.fluid_code.add_layer(
-                "transpose", inputs=input, output=node, param_attr=attr)
-            input = node
-        else:
-            self.data_format_propagation(node)
-
-        attr = {
-            "bias_attr": False,
-            "param_attr": string(kernel.layer_name),
-            "num_filters": k_size[2],
-            "filter_size": k_size[0:2],
-            "stride": strides[2:4],
-            "dilation": dilations[2:4],
-            "padding": string(pad_mode),
-            "output_size": out_shape[1:3]
-        }
-        node.fluid_code.add_layer(
-            "conv2d_transpose", inputs=input, output=node, param_attr=attr)
-
-        if not channel_first:
-            attr = {"perm": [0, 2, 3, 1]}
-            node.fluid_code.add_layer(
-                "transpose", inputs=node, output=node, param_attr=attr)
-
-    def Max(self, node):
-        input = self.graph.get_node(node.layer.input[0], copy=True)
-        reduce_idx = self.graph.get_node(node.layer.input[1], copy=True)
-        assert reduce_idx.layer_type == "Const", "Only support Const parameter[reduce_idx]"
-        keep_dims = node.get_attr("keep_dims")
-        dim = reduce_idx.value.tolist()
-
-        attr = {"dim": dim, "keep_dim": keep_dims}
-        node.fluid_code.add_layer(
-            "reduce_max", inputs=input, output=node, param_attr=attr)
-
-    def Sum(self, node):
-        input = self.graph.get_node(node.layer.input[0], copy=True)
-        reduce_idx = self.graph.get_node(node.layer.input[1], copy=True)
-        assert reduce_idx.layer_type == "Const", "Only support Const parameter[reduce_idx]"
-        keep_dims = node.get_attr("keep_dims")
-        dim = reduce_idx.value.tolist()
-
-        attr = {"dim": dim, "keep_dim": keep_dims}
-        node.fluid_code.add_layer(
-            "reduce_sum", inputs=input, output=node, param_attr=attr)
-
-    def Cast(self, node):
-        input = self.graph.get_node(node.layer.input[0], copy=True)
-        dtype = node.dtype_map[node.get_attr('DstT')]
-        attr = {"dtype": string(dtype)}
-        node.fluid_code.add_layer(
-            "cast", inputs=input, output=node, param_attr=attr)
-
-    def Split(self, node):
-        dim = self.graph.get_node(node.layer.input[0], copy=True)
-        input = self.graph.get_node(node.layer.input[1], copy=True)
-        assert dim.layer_type == "Const"
-        self.add_omit_nodes(dim.layer_name, node.layer_name)
-        num_split = node.get_attr('num_split')
-        dim = dim.value
-
-        attr = {"num_or_sections": num_split, "dim": dim}
-        node.fluid_code.add_layer(
-            "split", inputs=input, output=node, param_attr=attr)
-
-    def Squeeze(self, node):
-        input = self.graph.get_node(node.layer.input[0], copy=True)
-        squeeze_dims = node.get_attr('squeeze_dims')
-        attr = {"axes": squeeze_dims}
-        node.fluid_code.add_layer(
-            "squeeze", inputs=input, output=node, param_attr=attr)
-
-    def Softmax(self, node):
-        input = self.graph.get_node(node.layer.input[0], copy=True)
-        axis = node.get_attr("axis")
-        attr = {"axis": axis}
-        node.fluid_code.add_layer(
-            "softmax", inputs=input, output=node, param_attr=attr)
-
-    def ResizeNearestNeighbor(self, node):
-        input = self.graph.get_node(node.layer.input[0], copy=True)
-        resize_shape = self.graph.get_node(node.layer.input[1], copy=True)
-        self.add_omit_nodes(resize_shape.layer_name, node.layer_name)
-        if resize_shape.layer_type == "Const":
-            resize_shape = resize_shape.value.tolist()
-        else:
-            resize_shape = self.decoder.infer_shape_tensor(resize_shape,
-                                                           node.out_shapes[0])
-        align_corners = node.get_attr("align_corners")
-        attr = {"perm": [0, 3, 1, 2]}
-        node.fluid_code.add_layer(
-            "transpose", inputs=input, output=node, param_attr=attr)
-        attr = {"align_corners": align_corners, "out_shape": resize_shape}
-        node.fluid_code.add_layer(
-            "resize_nearest", inputs=node, output=node, param_attr=attr)
-        attr = {"perm": [0, 2, 3, 1]}
-        node.fluid_code.add_layer(
-            "transpose", inputs=node, output=node, param_attr=attr)
-
-    def ResizeBilinear(self, node):
-        input = self.graph.get_node(node.layer.input[0], copy=True)
-        resize_shape = self.graph.get_node(node.layer.input[1], copy=True)
-        self.add_omit_nodes(resize_shape.layer_name, node.layer_name)
-        if resize_shape.layer_type == "Const":
-            resize_shape = resize_shape.value.tolist()
-        else:
-            resize_shape = self.decoder.infer_shape_tensor(resize_shape,
-                                                           node.out_shapes[0])
-        align_corners = node.get_attr("align_corners")
-        attr = {"perm": [0, 3, 1, 2]}
-        node.fluid_code.add_layer(
-            "transpose", inputs=input, output=node, param_attr=attr)
-        attr = {
-            "align_corners": align_corners,
-            "out_shape": resize_shape,
-            "align_mode": 1
-        }
-        node.fluid_code.add_layer(
-            "resize_bilinear", inputs=node, output=node, param_attr=attr)
-        attr = {"perm": [0, 2, 3, 1]}
-        node.fluid_code.add_layer(
-            "transpose", inputs=node, output=node, param_attr=attr)
-
-    def GreaterEqual(self, node):
-        x = self.graph.get_node(node.layer.input[0], copy=True)
-        y = self.graph.get_node(node.layer.input[1], copy=True)
-        inputs = {"x": x, "y": y}
-        node.fluid_code.add_layer(
-            "greater_equal", inputs=inputs, output=node, param_attr=None)
-
-    def RandomUniform(self, node):
-        shape = self.graph.get_node(node.layer.input[0], copy=True)
-        self.add_omit_nodes(shape.layer_name, node.layer_name)
-        if shape.layer_type == "Const":
-            shape = shape.value.tolist()
-        else:
-            shape = self.decoder.infer_shape_tensor(shape)
-        attr = {"shape": shape, "min": 0.0, "max": 0.9999}
-
-        if shape[0] < 0:
-            input = self.batch_node
-            node.fluid_code.add_layer(
-                "uniform_random_batch_size_like",
-                inputs=input,
-                output=node,
-                param_attr=attr)
-        else:
-            node.fluid_code.add_layer(
-                "uniform_random", inputs=None, output=node, param_attr=attr)
-
-    def SquaredDifference(self, node):
-        x = self.graph.get_node(node.layer.input[0], copy=True)
-        y = self.graph.get_node(node.layer.input[1], copy=True)
-        inputs = {"x": x, "y": y}
-        node.fluid_code.add_layer(
-            "elementwise_sub", inputs=inputs, output=node, param_attr=None)
-        inputs = {"x": node, "y": node}
-        node.fluid_code.add_layer(
-            "elementwise_mul", inputs=inputs, output=node, param_attr=None)
-
-    def ExpandDims(self, node):
-        x = self.graph.get_node(node.layer.input[0], copy=True)
-        y = self.graph.get_node(node.layer.input[1], copy=True)
-        if y.layer_type == 'Const':
-            dim = y.value.tolist()
-        else:
-            dim = self.decoder.infer_tensor(y)
-        self.add_omit_nodes(y.layer_name, node.layer_name)
-        attr = {'axes': [dim]}
-        node.fluid_code.add_layer(
-            "unsqueeze", inputs=x, output=node, param_attr=attr)
-
-    def BatchToSpaceND(self, node):
-        x = self.graph.get_node(node.layer.input[0], copy=True)
-        y = self.graph.get_node(node.layer.input[1], copy=True)
-        if hasattr(node, 'skip') and node.skip:
-            node.fluid_code.add_layer(
-                "=", inputs=x, output=node, param_attr=None)
-        else:
-            raise Exception("BatchToSpaceND is not supported")
-
-    def SpaceToBatchND(self, node):
-        x = self.graph.get_node(node.layer.input[0], copy=True)
-        y = self.graph.get_node(node.layer.input[1], copy=True)
-        if hasattr(node, 'skip') and node.skip:
-            node.fluid_code.add_layer(
-                "=", inputs=x, output=node, param_attr=None)
-        else:
-            raise Exception("SpaceToBatchND is not supported")
+            transpose_name = gen_name("conv2dbackpropinput", "transpose")
+            program.add_layer(
+                kernel="fluid.layers.transpose",
+                inputs={"x": input.name},
+                outputs=[transpose_name],
+                perm=[0, 3, 1, 2])
+            input_name = transpose_name
+
+        program.add_layer(
+            kernel="fluid.layers.conv2d_transpose",
+            inputs={"input": input_name},
+            outputs=[node.name],
+            bias_attr=False,
+            param_attr=string(kernel.layer_name),
+            num_filters=k_size[2],
+            filter_size=k_size[0:2],
+            stride=strides[2:4],
+            dilation=dilations[2:4],
+            padding=string(pad_mode),
+            output_size=out_shape[1:3])
+
+        if data_format == "NHWC":
+            program.add_layer(
+                kernel="fluid.layers.transpose",
+                inputs={"x": node.name},
+                outputs=[node.name],
+                perm=[0, 2, 3, 1])

x2paddle/optimizer/onnx_optimizer.py

@@ -13,7 +13,6 @@
 # limitations under the License.
 
 # TODO useless node remove
-from x2paddle.op_mapper.onnx_op_mapper import ONNXOpMapper
 
 
 class ONNXOptimizer(object):

x2paddle/optimizer/tf_optimizer.py

@@ -236,26 +236,18 @@ class TFOptimizer(object):
 
     def remove_transpose(self):
         graph_copy = cp.deepcopy(self.graph)
-        nhwc_insensitive_ops = [
-            'Relu', 'Relu6', 'Abs', 'Sigmoid', 'Exp', 'Rsqrt', 'swish_f32',
-            'LeakyRelu', 'Cast', 'Tanh'
-        ]
         elementwise_ops = [
             'Sub', 'Add', 'RealDiv', 'Maximum', 'Mul', 'FloorDiv',
-            'GreaterEqual'
-        ]
-        optimize_ops = [
-            'Conv2D', 'MaxPool', 'FusedBatchNorm', 'DepthwiseConv2dNative',
-            'AvgPool', 'Pad', 'Conv2DBackpropInput', 'ResizeNearestNeighbor',
-            'ResizeBilinear', "Placeholder"
+            'GreateerEqual'
         ]
         can_be_optimized_ops = [
             'Conv2D', 'MaxPool', 'FusedBatchNorm', 'DepthwiseConv2dNative',
             'AvgPool', 'Pad', 'Conv2DBackpropInput', 'ResizeNearestNeighbor',
-            'ResizeBilinear', "Placeholder", 'Relu', 'Relu6', 'Abs', 'Sigmoid',
-            'Exp', 'Rsqrt', 'swish_f32', 'LeakyRelu', 'Cast', 'Tanh'
+            'Placeholder', 'Relu', 'Relu6', 'Abs', 'Sigmoid', 'Exp', 'Rsqrt',
+            'swish_f32', 'LeakyRelu', 'Cast', 'Tanh'
         ]
-
+        # These ops may have one more Variable input
+        can_be_optimized_special_ops = ['ResizeBilinear']
         for node_name in self.graph.topo_sort:
             node = graph_copy.get_node(node_name)
             if node is None:
@@ -278,9 +270,10 @@ class TFOptimizer(object):
                                 0].param_attr["perm"] != [0, 3, 1, 2]:
                         can_be_removed = False
                         break
-                    elif out_node.layer_type in elementwise_ops:
+                    elif out_node.layer_type in elementwise_ops or out_node.layer_type in can_be_optimized_special_ops:
                         can_be_removed = False
                         break
+
                 if can_be_removed and len(node.fluid_code.layers) > 1:
                     true_node = self.graph.get_node(node_name)
                     if true_node.layer_type == "Placeholder":
@@ -298,6 +291,7 @@ class TFOptimizer(object):
                         -2].output = true_node.fluid_code.layers[-1].output
                     node.removed = True
                     del true_node.fluid_code.layers[-1]
+
                     for out_name in output_names:
                         out_node = self.graph.get_node(out_name)
                         out_node.fluid_code.layers[

x2paddle_model_zoo.md

 # X2Paddle模型测试库
-> 目前X2Paddle支持40+的TensorFlow OP，40+的Caffe Layer，覆盖了大部分CV分类模型常用的操作。我们在如下模型列表中测试了X2Paddle的转换。
+> 目前X2Paddle支持50+的TensorFlow OP，40+的Caffe Layer，覆盖了大部分CV分类模型常用的操作。我们在如下模型列表中测试了X2Paddle的转换。
 
 **注：** 受限于不同框架的差异，部分模型可能会存在目前无法转换的情况，如TensorFlow中包含控制流的模型，NLP模型等。对于CV常见的模型，如若您发现无法转换或转换失败，存在较大diff等问题，欢迎通过[ISSUE反馈](https://github.com/PaddlePaddle/X2Paddle/issues/new)的方式告知我们(模型名，代码实现或模型获取方式)，我们会及时跟进：）
 
@@ -22,7 +22,8 @@
 | UNet | [code1](https://github.com/jakeret/tf_unet )/[code2](https://github.com/lyatdawn/Unet-Tensorflow) |-|
 |MTCNN | [code](https://github.com/AITTSMD/MTCNN-Tensorflow) |-|
 |YOLO-V3| [code](https://github.com/YunYang1994/tensorflow-yolov3) | 转换需要关闭NHWC->NCHW的优化，见[文档Q2](FAQ.md) |
-|Inception_ResNet_V2| [code](https://github.com/tensorflow/models/tree/master/research/slim/nets) | - |
+| FALSR | [code](https://github.com/xiaomi-automl/FALSR) | - |
+| DCSCN | [code](https://modelzoo.co/model/dcscn-super-resolution) | - |
 
 ## Caffe
 
@@ -48,8 +49,8 @@
 ## ONNX
 **注：** 部分模型来源于PyTorch，PyTorch的转换可参考[pytorch_to_onnx.md](pytorch_to_onnx.md)
 
-| 模型 | 来源 | operator version|
-|-------|--------|---------|
+| 模型 | 来源 | operator version|备注|
+|-------|--------|---------|---------|
 | ResNet18 | [torchvison.model.resnet18](https://github.com/pytorch/vision/blob/master/torchvision/models/resnet.py) |9|
 | ResNet34 | [torchvison.model.resnet34](https://github.com/pytorch/vision/blob/master/torchvision/models/resnet.py) |9|
 | ResNet50 | [torchvison.model.resnet50](https://github.com/pytorch/vision/blob/master/torchvision/models/resnet.py) |9|
@@ -65,4 +66,6 @@
 | mNASNet | [pytorch(personal practice)](https://github.com/rwightman/gen-efficientnet-pytorch) |9|
 | EfficientNet | [pytorch(personal practice)](https://github.com/rwightman/gen-efficientnet-pytorch) |9|
 | SqueezeNet | [onnx official](https://s3.amazonaws.com/download.onnx/models/opset_9/squeezenet.tar.gz) |9|
-|Ultra-Light-Fast-Generic-Face-Detector-1MB| [onnx_model](https://github.com/Linzaer/Ultra-Light-Fast-Generic-Face-Detector-1MB/tree/master/models/onnx)| |
+|Ultra-Light-Fast-Generic-Face-Detector-1MB| [onnx_model](https://github.com/Linzaer/Ultra-Light-Fast-Generic-Face-Detector-1MB/tree/master/models/onnx)|9 |
+|BERT| [pytorch(huggingface)](https://github.com/huggingface/transformers/blob/master/notebooks/04-onnx-export.ipynb)|11|转换时需指定input shape，见[文档Q3](FAQ.md)|
+|GPT2| [pytorch(huggingface)](https://github.com/huggingface/transformers/blob/master/notebooks/04-onnx-export.ipynb)|11|转换时需指定input shape，见[文档Q3](FAQ.md)|