Created by: pzelazko-intel

DO NOT MERGE IT

Please take a look at MKLDNN layout Proof of Concept and assess if we can stay with this design. This code is not finished - there are debug VLOG prints, some parts are missing and the code needs cleanup.

We need MKLDNN layouts for MKLDNN kernels to be performed efficiently - especially for convolution and fully-connected OPs.

In #8305 (closed) it was recommended to create MKLDNNLoDTensor class deriving from LoDTensor. But that approach was causing too many problems - in many places Tensor class is used explicitly and I would need to differentiate between LoDTensor and MKLDNNLoDTensor.

I came up with a different approach - I've added Tensor::ExtendedData interface and a pointer to this interface in Tensor class. It is a placeholder for additional data. I use it for MKLDNN layout case - mkldnn::memory::format and mkldnn::engine are held in MKLDNNTensorData, which derives from Tensor::ExtendedData. If Tensor layout is set to kMKLDNN, then I can treat this tensor as holding data in MKLDNN format layout. To make use of such a tensor, I've created decorator classes MKLDNNTensor and MutableMKLDNNTensor. The data is kept still in Placeholder within Tensor.

Reorders to MKLDNN format happen within convolution and fully-connected, because:

• For these two OPs MKLDNN library can decide which layout is best based on input parameters like the size of the input, stride, padding etc.
• Other MKLDNN OPs don't offer significant performance boost on MKLDNN layouts

When the next OP expected kernel is not a MKLDNN one, then we transform tensor to NCHW layout.