Merge pull request #3 from PaddlePaddle/develop

dev

FAQ.md

@@ -5,10 +5,9 @@ A：该提示信息表示无法从TensorFlow的pb模型中获取到输入tensor(
 
 
 **Q2. TensorFlow模型转换失败怎么解决?**  
-A: 如果并非是由缺少OP导致，那可能是由于TensorFlow模型转换时（NHWC->NCHW格式转换导致），在这种情况下，可以采用关闭格式优化，同时固化输入大小的方式，继续尝试转换，见如下命令，转换过程中，根据提示，输入相应tensor的固化shape大小
+A: 如果并非是由缺少OP导致，那可能是由于TensorFlow模型转换时（NHWC->NCHW格式转换导致），在这种情况下，采用如下方式进行转换，同时固化输入大小的方式，继续尝试转换，见如下命令，转换过程中，根据提示，输入相应tensor的固化shape大小
 ```
 x2paddle -f tensorflow -m tf.pb -s pd-model --without_data_format_optimization --define_input_shape
 ```
 
-> 1. 目前Tensorflow的CV模型大部分均为`NHWC`的输入格式，而Paddle的默认输入格式为`NCHW`，因此X2Paddle在转换过程中，会对如`axis`， `shape`等参数进行转换，适应Paddle的NCHW格式。但在这种情况下，可能会由于TensorFlow模型太复杂，导致出错。  
-> 2. X2Paddle默认情况，TensorFlow模型转换后得到的Paddle模型为`NCHW`的输入格式。但在指定`--withou_data_format_optimization`后，转换后的Paddle模型输入格式也同样为`NHWC`。
+> 1. 目前Tensorflow的CV模型大部分均为`NHWC`的输入格式，而Paddle的默认输入格式为`NCHW`，因此X2Paddle在转换过程中，会对如`axis`， `shape`等参数进行转换，适应Paddle的NCHW格式。但在这种情况下，可能会由于TensorFlow模型太复杂，导致出错。  指定`--without_data_format_optimization`后，会停止对`axis`，`shape`等参数的优化（这可能会带来一定数量的transpose操作）

README.md

@@ -14,7 +14,7 @@ paddlepaddle >= 1.5.0
 
 **按需安装以下依赖**  
 tensorflow ： tensorflow == 1.14.0  
-caffe ： caffe == 1.0.0  
+caffe ： 无  
 onnx ： onnx == 1.5.0  pytorch == 1.1.0
 
 ## 安装
@@ -79,6 +79,7 @@ X2Paddle提供了工具解决如下问题，详见[tools/README.md](tools/README
 2. [如何导出TensorFlow的pb模型](export_tf_model.md)
 3. [X2Paddle测试模型库](x2paddle_model_zoo.md)  
 4. [PyTorch模型导出为ONNX模型](pytorch_to_onnx.md)
+5. [X2Paddle内置的Caffe自定义层](caffe_custom_layer.md)
 
 ## 更新历史
 2019.08.05  

caffe_custom_layer.md

+目前，代码中已经提供了8个非官方op（不在[官网](http://caffe.berkeleyvision.org/tutorial/layers)上的op）的转换，这些op对应的Caffe实现源码如下：
+
+| op | 该版本实现源码 |
+|-------|--------|
+| PriorBox | [code](https://github.com/weiliu89/caffe/blob/ssd/src/caffe/layers/prior_box_layer.cpp) |
+| DetectionOutput | [code](https://github.com/weiliu89/caffe/blob/ssd/src/caffe/layers/detection_output_layer.cpp) |
+| ConvolutionDepthwise | [code](https://github.com/farmingyard/caffe-mobilenet/blob/master/conv_dw_layer.cpp) |
+| ShuffleChannel | [code](https://github.com/farmingyard/ShuffleNet/blob/master/shuffle_channel_layer.cpp) |
+| Permute | [code](https://github.com/weiliu89/caffe/blob/ssd/src/caffe/layers/permute_layer.cpp) |
+| Normalize | [code](https://github.com/weiliu89/caffe/blob/ssd/src/caffe/layers/normalize_layer.cpp) |
+| ROIPooling | [code](https://github.com/rbgirshick/caffe-fast-rcnn/blob/0dcd397b29507b8314e252e850518c5695efbb83/src/caffe/layers/roi_pooling_layer.cpp) |
+| Axpy | [code](https://github.com/hujie-frank/SENet/blob/master/src/caffe/layers/axpy_layer.cpp) |
+
+

x2paddle/convert.py

@@ -124,7 +124,10 @@ def caffe2paddle(proto, weight, save_dir, caffe_proto):
     from x2paddle.decoder.caffe_decoder import CaffeDecoder
     from x2paddle.op_mapper.caffe_op_mapper import CaffeOpMapper
     from x2paddle.optimizer.caffe_optimizer import CaffeOptimizer
-
+    import google.protobuf as gpb
+    ver_str = gpb.__version__.replace('.', '')
+    ver_int = int(ver_str[0:2])
+    assert ver_int >= 36, 'The version of protobuf must be larger than 3.6.0!'
     print("Now translating model from caffe to paddle.")
     model = CaffeDecoder(proto, weight, caffe_proto)
     mapper = CaffeOpMapper(model)

x2paddle/decoder/caffe_decoder.py

@@ -190,7 +190,6 @@ class CaffeGraph(Graph):
                     top_layer[out_name] = [layer.name]
                 else:
                     top_layer[out_name].append(layer.name)
-
         for layer_name, data in self.params:
             if layer_name in self.node_map:
                 node = self.node_map[layer_name]

x2paddle/op_mapper/caffe_custom_layer/convolutiondepthwise.py

@@ -19,20 +19,29 @@ def convolutiondepthwise_shape(input_shape,
     if isinstance(kernel_size, numbers.Number):
         [k_h, k_w] = [kernel_size] * 2
     elif len(kernel_size) > 0:
-        k_h = kernel_h if kernel_h else kernel_size[0]
-        k_w = kernel_w if kernel_w else kernel_size[len(kernel_size) - 1]
+        k_h = kernel_h if kernel_h > 0 else kernel_size[0]
+        k_w = kernel_w if kernel_w > 0 else kernel_size[len(kernel_size) - 1]
+    elif kernel_h > 0 or kernel_w > 0:
+        k_h = kernel_h
+        k_w = kernel_w
     [s_h, s_w] = [1, 1]
     if isinstance(stride, numbers.Number):
         [s_h, s_w] = [stride] * 2
     elif len(stride) > 0:
-        s_h = stride_h if stride_h else stride[0]
-        s_w = stride_w if stride_w else stride[len(stride) - 1]
+        s_h = stride_h if stride_h > 0 else stride[0]
+        s_w = stride_w if stride_w > 0 else stride[len(stride) - 1]
+    elif stride_h > 0 or stride_w > 0:
+        s_h = stride_h
+        s_w = stride_w
     [p_h, p_w] = [0, 0]
     if isinstance(pad, numbers.Number):
         [p_h, p_w] = [pad] * 2
     elif len(pad) > 0:
-        p_h = pad_h if pad_h else pad[0]
-        p_w = pad_w if pad_w else pad[len(pad) - 1]
+        p_h = pad_h if pad_h > 0 else pad[0]
+        p_w = pad_w if pad_w > 0 else pad[len(pad) - 1]
+    elif pad_h > 0 or pad_w > 0:
+        p_h = pad_h
+        p_w = pad_w
     dila_len = len(dilation)
     dila_h = 1
     dila_w = 1
@@ -74,20 +83,29 @@ def convolutiondepthwise_layer(inputs,
     if isinstance(kernel_size, numbers.Number):
         [k_h, k_w] = [kernel_size] * 2
     elif len(kernel_size) > 0:
-        k_h = kernel_h if kernel_h else kernel_size[0]
-        k_w = kernel_w if kernel_w else kernel_size[len(kernel_size) - 1]
+        k_h = kernel_h if kernel_h > 0 else kernel_size[0]
+        k_w = kernel_w if kernel_w > 0 else kernel_size[len(kernel_size) - 1]
+    elif kernel_h > 0 or kernel_w > 0:
+        k_h = kernel_h
+        k_w = kernel_w
     [s_h, s_w] = [1, 1]
     if isinstance(stride, numbers.Number):
         [s_h, s_w] = [stride] * 2
     elif len(stride) > 0:
-        s_h = stride_h if stride_h else stride[0]
-        s_w = stride_w if stride_w else stride[len(stride) - 1]
+        s_h = stride_h if stride_h > 0 else stride[0]
+        s_w = stride_w if stride_w > 0 else stride[len(stride) - 1]
+    elif stride_h > 0 or stride_w > 0:
+        s_h = stride_h
+        s_w = stride_w
     [p_h, p_w] = [0, 0]
     if isinstance(pad, numbers.Number):
         [p_h, p_w] = [pad] * 2
     elif len(pad) > 0:
-        p_h = pad_h if pad_h else pad[0]
-        p_w = pad_w if pad_w else pad[len(pad) - 1]
+        p_h = pad_h if pad_h > 0 else pad[0]
+        p_w = pad_w if pad_w > 0 else pad[len(pad) - 1]
+    elif pad_h > 0 or pad_w > 0:
+        p_h = pad_h
+        p_w = pad_w
     input = inputs[0]
     dila_len = len(dilation)
     dila_h = 1

x2paddle/op_mapper/caffe_op_mapper.py

@@ -39,6 +39,8 @@ class CaffeOpMapper(OpMapper):
         print("Total nodes: {}".format(len(self.graph.topo_sort)))
         for node_name in self.graph.topo_sort:
             node = self.graph.get_node(node_name)
+            if node.layer_type == 'DepthwiseConvolution':
+                node.layer_type = 'ConvolutionDepthwise'
             op = node.layer_type
             if hasattr(self, op):
                 self.set_node_shape(node)
@@ -133,23 +135,32 @@ class CaffeOpMapper(OpMapper):
         if isinstance(params.kernel_size, numbers.Number):
             [k_h, k_w] = [params.kernel_size] * 2
         elif len(params.kernel_size) > 0:
-            k_h = params.kernel_h if params.kernel_h else params.kernel_size[0]
-            k_w = params.kernel_w if params.kernel_w else params.kernel_size[
+            k_h = params.kernel_h if params.kernel_h > 0 else params.kernel_size[0]
+            k_w = params.kernel_w if params.kernel_w > 0 else params.kernel_size[
                 len(params.kernel_size) - 1]
+        elif params.kernel_h > 0 or params.kernel_w > 0:
+            k_h = params.kernel_h
+            k_w = params.kernel_w
         [s_h, s_w] = [1, 1]
         if isinstance(params.stride, numbers.Number):
             [s_h, s_w] = [params.stride] * 2
         elif len(params.stride) > 0:
-            s_h = params.stride_h if params.stride_h else params.stride[0]
-            s_w = params.stride_w if params.stride_w else params.stride[
+            s_h = params.stride_h if params.stride_h > 0 else params.stride[0]
+            s_w = params.stride_w if params.stride_w > 0 else params.stride[
                 len(params.stride) - 1]
+        elif params.stride_h > 0 or params.stride_w > 0:
+            s_h = params.stride_h
+            s_w = params.stride_w
         [p_h, p_w] = [0, 0]
         if isinstance(params.pad, numbers.Number):
             [p_h, p_w] = [params.pad] * 2
         elif len(params.pad) > 0:
-            p_h = params.pad_h if params.pad_h else params.pad[0]
-            p_w = params.pad_w if params.pad_w else params.pad[len(params.pad) -
-                                                               1]
+            p_h = params.pad_h if params.pad_h > 0 else params.pad[0]
+            p_w = params.pad_w if params.pad_w > 0 else params.pad[len(params.pad) -
+                                                                  1]
+        elif params.pad_h > 0 or params.pad_w > 0:
+            p_h = params.pad_h
+            p_w = params.pad_w
         dila_h = dila_w = 1
         group = 1
         c_o = 1
@@ -209,15 +220,22 @@ class CaffeOpMapper(OpMapper):
 
     def Convolution(self, node):
         data = node.data
-        assert data is not None, 'The parameter of {} (type is {}) is not set. You need to use python package of caffe to set the default value.'.format(
-            node.layer_name, node.layer_type)
-        data = self.adjust_parameters(node)
-        self.weights[node.layer_name + '_weights'] = data[0]
-        if len(data) == 2:
-            self.weights[node.layer_name + '_bias'] = data[1]
         params = node.layer.convolution_param
         channel, kernel, stride, pad, dilation, group = self.get_kernel_parameters(
             node.layer_type, params)
+        if data is None:
+            data = []
+            print('The parameter of {} (type is {}) is not set. So we set the parameters as 0'.format(
+            node.layer_name, node.layer_type))
+            input_c = node.input_shape[0][1]
+            output_c = channel
+            data.append(np.zeros([output_c, input_c, kernel[0], kernel[1]]).astype('float32'))
+            data.append(np.zeros([output_c,])).astype('float32')
+        else:
+            data = self.adjust_parameters(node)
+        self.weights[node.layer_name + '_weights'] = data[0]
+        if len(data) == 2:
+            self.weights[node.layer_name + '_bias'] = data[1]
         assert len(node.inputs
                    ) == 1, 'The count of Convolution node\'s input is not 1.'
         input = self.graph.get_bottom_node(node, idx=0, copy=True)
@@ -249,15 +267,22 @@ class CaffeOpMapper(OpMapper):
 
     def Deconvolution(self, node):
         data = node.data
-        assert data is not None, 'The parameter of {} (type is {}) is not set. You need to use python package of caffe to set the default value.'.format(
-            node.layer_name, node.layer_type)
-        data = self.adjust_parameters(node)
-        self.weights[node.layer_name + '_weights'] = data[0]
-        if len(data) == 2:
-            self.weights[node.layer_name + '_bias'] = data[1]
         params = node.layer.convolution_param
         channel, kernel, stride, pad, dilation, group = self.get_kernel_parameters(
             node.layer_type, params)
+        if data is None:
+            data = []
+            print('The parameter of {} (type is {}) is not set. So we set the parameters as 0'.format(
+            node.layer_name, node.layer_type))
+            input_c = node.input_shape[0][1]
+            output_c = channel
+            data.append(np.zeros([output_c, input_c, kernel[0], kernel[1]]).astype('float32'))
+            data.append(np.zeros([output_c,]).astype('float32'))
+        else:
+            data = self.adjust_parameters(node)
+        self.weights[node.layer_name + '_weights'] = data[0]
+        if len(data) == 2:
+            self.weights[node.layer_name + '_bias'] = data[1]
         assert len(node.inputs
                    ) == 1, 'The count of Deconvolution node\'s input is not 1.'
         input = self.graph.get_bottom_node(node, idx=0, copy=True)
@@ -342,24 +367,32 @@ class CaffeOpMapper(OpMapper):
 
     def InnerProduct(self, node):
         data = node.data
-        assert data is not None, 'The parameter of {} (type is {}) is not set. You need to use python package of caffe to set the default value.'.format(
-            node.layer_name, node.layer_type)
-        data = self.adjust_parameters(node)
-        # Reshape the parameters to Paddle's ordering
-        transpose_order = (1, 0)
-        w = data[0]
-        fc_shape = w.shape
-        output_channels = fc_shape[0]
-        w = w.reshape((output_channels, -1))
-        w = w.transpose(transpose_order)
-        data[0] = w
+        params = node.layer.inner_product_param
+        if data is None:
+            print('The parameter of {} (type is {}) is not set. So we set the parameters as 0.'.format(
+            node.layer_name, node.layer_type))
+            input_c = node.input_shape[0][1]
+            output_c = params.num_output
+            data = []
+            data.append(np.zeros([input_c, output_c]).astype('float32').astype('float32'))
+            data.append(np.zeros([output_c]).astype('float32').astype('float32'))
+        else:
+            data = self.adjust_parameters(node)
+            # Reshape the parameters to Paddle's ordering
+            transpose_order = (1, 0)
+            w = data[0]
+            fc_shape = w.shape
+            output_channels = fc_shape[0]
+            w = w.reshape((output_channels, -1))
+            w = w.transpose(transpose_order)
+            data[0] = w
 
         self.weights[node.layer_name + '_weights'] = data[0]
         if len(data) == 2:
             self.weights[node.layer_name + '_bias'] = data[1]
         assert len(node.inputs
                    ) == 1, 'The count of InnerProduct node\'s input is not 1.'
-        params = node.layer.inner_product_param
+        #params = node.layer.inner_product_param
         assert params.axis == 1
         assert params.bias_term == True
         input = self.graph.get_bottom_node(node, idx=0, copy=True)
@@ -590,9 +623,16 @@ class CaffeOpMapper(OpMapper):
             eps = params.eps
         else:
             eps = 1e-5
-        assert len(node.data) == 3
-        node.data = [np.squeeze(i) for i in node.data]
-        mean, variance, scale = node.data
+        if node.data is None or len(node.data) != 3:
+            print('The parameter of {} (type is {}) is not set. So we set the parameters as 0'.format(
+            node.layer_name, node.layer_type))
+            input_c = node.input_shape[0][1]
+            mean = np.zeros([input_c,]).astype('float32')
+            variance = np.zeros([input_c,]).astype('float32')
+            scale = 0
+        else:
+            node.data = [np.squeeze(i) for i in node.data]
+            mean, variance, scale = node.data
         # Prescale the stats
         scaling_factor = 1.0 / scale if scale != 0 else 0
         mean *= scaling_factor
@@ -614,9 +654,15 @@ class CaffeOpMapper(OpMapper):
                                   param_attr=attr)
 
     def Scale(self, node):
-
-        self.weights[node.layer_name + '_scale'] = np.squeeze(node.data[0])
-        self.weights[node.layer_name + '_offset'] = np.squeeze(node.data[1])
+        if node.data is None:
+            print('The parameter of {} (type is {}) is not set. So we set the parameters as 0'.format(
+            node.layer_name, node.layer_type))
+            input_c = node.input_shape[0][1]
+            self.weights[node.layer_name + '_scale'] = np.zeros([input_c,]).astype('float32')
+            self.weights[node.layer_name + '_offset'] = np.zeros([input_c,]).astype('float32')
+        else:
+            self.weights[node.layer_name + '_scale'] = np.squeeze(node.data[0])
+            self.weights[node.layer_name + '_offset'] = np.squeeze(node.data[1])
         params = node.layer.scale_param
         axis = params.axis
         num_axes = params.num_axes

x2paddle/op_mapper/caffe_shape.py

@@ -22,22 +22,31 @@ def get_kernel_parameters(params):
     if isinstance(params.kernel_size, numbers.Number):
         [k_h, k_w] = [params.kernel_size] * 2
     elif len(params.kernel_size) > 0:
-        k_h = params.kernel_h if params.kernel_h else params.kernel_size[0]
-        k_w = params.kernel_w if params.kernel_w else params.kernel_size[
+        k_h = params.kernel_h if params.kernel_h > 0 else params.kernel_size[0]
+        k_w = params.kernel_w if params.kernel_w > 0 else params.kernel_size[
             len(params.kernel_size) - 1]
+    elif params.kernel_h > 0 or params.kernel_w > 0:
+        k_h = params.kernel_h
+        k_w = params.kernel_w
     [s_h, s_w] = [1, 1]
     if isinstance(params.stride, numbers.Number):
         [s_h, s_w] = [params.stride] * 2
     elif len(params.stride) > 0:
-        s_h = params.stride_h if params.stride_h else params.stride[0]
-        s_w = params.stride_w if params.stride_w else params.stride[
+        s_h = params.stride_h if params.stride_h > 0 else params.stride[0]
+        s_w = params.stride_w if params.stride_w > 0 else params.stride[
             len(params.stride) - 1]
+    elif params.stride_h > 0 or params.stride_w > 0:
+        s_h = params.stride_h
+        s_w = params.stride_w
     [p_h, p_w] = [0, 0]
     if isinstance(params.pad, numbers.Number):
         [p_h, p_w] = [params.pad] * 2
     elif len(params.pad) > 0:
-        p_h = params.pad_h if params.pad_h else params.pad[0]
-        p_w = params.pad_w if params.pad_w else params.pad[len(params.pad) - 1]
+        p_h = params.pad_h if params.pad_h > 0 else params.pad[0]
+        p_w = params.pad_w if params.pad_w > 0 else params.pad[len(params.pad) - 1]
+    elif params.pad_h > 0 or params.pad_w > 0:
+        p_h = params.pad_h
+        p_w = params.pad_w
     dila_h = dila_w = 1
     if hasattr(params, 'dilation'):
         dila_len = len(params.dilation)

x2paddle_model_zoo.md

@@ -31,9 +31,19 @@
 | SqueezeNet | [code](https://github.com/DeepScale/SqueezeNet/tree/master/SqueezeNet_v1.1) |
 | MobileNet_V1 | [code](https://github.com/shicai/MobileNet-Caffe) |
 | MobileNet_V2 | [code](https://github.com/shicai/MobileNet-Caffe) |
-| ShuffleNet | [code](https://github.com/miaow1988/ShuffleNet_V2_pytorch_caffe/releases/tag/v0.1.0) |
+| ShuffleNet_v2 | [code](https://github.com/miaow1988/ShuffleNet_V2_pytorch_caffe/releases/tag/v0.1.0) |
 | mNASNet | [code](https://github.com/LiJianfei06/MnasNet-caffe) |
 | MTCNN | [code](https://github.com/kpzhang93/MTCNN_face_detection_alignment/tree/master/code/codes/MTCNNv1/model) |
+| Mobilenet_SSD | [code](https://github.com/chuanqi305/MobileNet-SSD) |
+| ResNet18 | [code](https://github.com/HolmesShuan/ResNet-18-Caffemodel-on-ImageNet/blob/master/deploy.prototxt) |
+| ResNet50 | [code](https://github.com/soeaver/caffe-model/blob/master/cls/resnet/deploy_resnet50.prototxt) |
+| Unet | [code](https://github.com/jolibrain/deepdetect/blob/master/templates/caffe/unet/deploy.prototxt) |
+| VGGNet | [code](https://gist.github.com/ksimonyan/211839e770f7b538e2d8#file-vgg_ilsvrc_16_layers_deploy-prototxt) |
+
+
+
+
+
 
 ## ONNX
 **注：** 部分模型来源于PyTorch，PyTorch的转换可参考[pytorch_to_onnx.md](pytorch_to_onnx.md)