Created by: luotao1

Motivation

For models not trained by MKLDNN and trained in old PaddlePaddle version, in their ProgramDesc:

• In some op, their use_mkldnn flag is False. The reason is that models were not trained by MKLDNN. For example: the use_mkldnn flag of relu_op is False.

   ops {
       inputs {
         parameter: "X"
         arguments: "conv2d_7.tmp_1"
       }
       outputs {
         parameter: "Out"
         arguments: "conv2d_7.tmp_2"
       }
       type: "relu"
       attrs {
         name: "use_mkldnn"
         type: BOOLEAN
         b: false
       }
     }

• In some other op, their use_mkldnn flag is missing. The reason is that models were trained in a old version, where use_mkldnn flag is not added to these ops. For example, elementwise_add_op lacks use_mkldnn flag, but it will support use_mkldnn flag later.

  ops {
       inputs {
         parameter: "X"
         arguments: "fc_0.tmp_0"
       }
       inputs {
         parameter: "Y"
         arguments: "fc_0.b_0"
       }
       outputs {
         parameter: "Out"
         arguments: "fc_0.tmp_1"
       }
       type: "elementwise_add"
       attrs {
         name: "axis"
         type: INT
         i: 1
       }
     }

To enable the inference ON MKLDNN, we should do transpiler on ProgramDesc.

Implemenation

At first, we can add a enable_mkldnn function in inference_transpiler.py in Python end. (We will re-implement in C++ end later).

This function will go over the ProgramDesc of a model, and for each OpDesc:

if this OP is enable by MKLDNN kernel:
   if use_mkldnn of this OP is False:
       set use_mkldnn = True
   if use_mkldnn attr of this Op is missing:
       add use_mkldnn attr, and set it True