mkldnn.html



<!DOCTYPE html>
<!--[if IE 8]><html class="no-js lt-ie9" lang="en" > <![endif]-->
<!--[if gt IE 8]><!--> <html class="no-js" lang="en" > <!--<![endif]-->
<head>
  <meta charset="utf-8">
  
  <meta name="viewport" content="width=device-width, initial-scale=1.0">
  
  <title>Intel® MKL-DNN on PaddlePaddle: Design Doc &mdash; PaddlePaddle  documentation</title>
  

    <link rel="stylesheet" href="../../_static/css/theme.css" type="text/css" />
  

        <link rel="index" title="Index"
              href="../../genindex.html"/>
        <link rel="search" title="Search" href="../../search.html"/>
    <link rel="top" title="PaddlePaddle  documentation" href="../../index.html"/> 
<script>
var _hmt = _hmt || [];
(function() {
  var hm = document.createElement("script");
  hm.src = "//hm.baidu.com/hm.js?b9a314ab40d04d805655aab1deee08ba";
  var s = document.getElementsByTagName("script")[0]; 
  s.parentNode.insertBefore(hm, s);
})();
</script>


  <script src="../../_static/js/modernizr.min.js"></script>

</head>

<body class="wy-body-for-nav" role="document">

  <div class="wy-grid-for-nav">

    
    <nav data-toggle="wy-nav-shift" class="wy-nav-side">
      <div class="wy-side-scroll">
        <div class="wy-side-nav-search">
          

            <a href="../../index_en.html" class="icon icon-home"> PaddlePaddle
          

          </a>

          
<div role="search">
  <form id="rtd-search-form" class="wy-form" action="../../search.html" method="get">
    <input type="text" name="q" placeholder="Search docs" />
    <input type="hidden" name="check_keywords" value="yes" />
    <input type="hidden" name="area" value="default" />
  </form>
</div>

          
        </div>

        <div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="main navigation">
          
<nav class="doc-menu-vertical" role="navigation">

<ul>
<li class="toctree-l1"><a class="reference internal" href="../../getstarted/index_en.html">GET STARTED</a><ul>
<li class="toctree-l2"><a class="reference internal" href="../../getstarted/quickstart_en.html">Quick Start</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../getstarted/concepts/use_concepts_en.html">Basic Concept</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="../../build_and_install/index_en.html">Install and Build</a><ul>
<li class="toctree-l2"><a class="reference internal" href="../../build_and_install/pip_install_en.html">Install using pip</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../build_and_install/docker_install_en.html">Run in Docker Containers</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../build_and_install/build_from_source_en.html">Build from Sources</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="../../howto/index_en.html">HOW TO</a><ul>
<li class="toctree-l2"><a class="reference internal" href="../../howto/cmd_parameter/index_en.html">Set Command-line Parameters</a><ul>
<li class="toctree-l3"><a class="reference internal" href="../../howto/cmd_parameter/use_case_en.html">Use Case</a></li>
<li class="toctree-l3"><a class="reference internal" href="../../howto/cmd_parameter/arguments_en.html">Argument Outline</a></li>
<li class="toctree-l3"><a class="reference internal" href="../../howto/cmd_parameter/detail_introduction_en.html">Detail Description</a></li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="../../howto/cluster/index_en.html">Distributed Training</a><ul>
<li class="toctree-l3"><a class="reference internal" href="../../howto/cluster/preparations_en.html">Preparations</a></li>
<li class="toctree-l3"><a class="reference internal" href="../../howto/cluster/cmd_argument_en.html">Command-line arguments</a></li>
<li class="toctree-l3"><a class="reference internal" href="../../howto/cluster/multi_cluster/index_en.html">Use different clusters</a><ul>
<li class="toctree-l4"><a class="reference internal" href="../../howto/cluster/multi_cluster/fabric_en.html">Fabric</a></li>
<li class="toctree-l4"><a class="reference internal" href="../../howto/cluster/multi_cluster/openmpi_en.html">OpenMPI</a></li>
<li class="toctree-l4"><a class="reference internal" href="../../howto/cluster/multi_cluster/k8s_en.html">Kubernetes</a></li>
<li class="toctree-l4"><a class="reference internal" href="../../howto/cluster/multi_cluster/k8s_aws_en.html">Kubernetes on AWS</a></li>
</ul>
</li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="../../howto/rnn/index_en.html">RNN Models</a><ul>
<li class="toctree-l3"><a class="reference internal" href="../../howto/rnn/rnn_config_en.html">RNN Configuration</a></li>
<li class="toctree-l3"><a class="reference internal" href="../../howto/rnn/recurrent_group_en.html">Recurrent Group Tutorial</a></li>
<li class="toctree-l3"><a class="reference internal" href="../../howto/rnn/hierarchical_layer_en.html">Layers supporting hierarchical sequence as input</a></li>
<li class="toctree-l3"><a class="reference internal" href="../../howto/rnn/hrnn_rnn_api_compare_en.html">API comparision between RNN and hierarchical RNN</a></li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="../../howto/optimization/gpu_profiling_en.html">Tune GPU Performance</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="../../dev/index_en.html">Development</a><ul>
<li class="toctree-l2"><a class="reference internal" href="../../dev/contribute_to_paddle_en.html">Contribute Code</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../dev/write_docs_en.html">Contribute Documentation</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="../../faq/index_en.html">FAQ</a><ul>
<li class="toctree-l2"><a class="reference internal" href="../../faq/build_and_install/index_en.html">Install, Build and Unit test</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../faq/model/index_en.html">Model Configuration</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../faq/parameter/index_en.html">Parameter Setting</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../faq/local/index_en.html">Local Training and Prediction</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../faq/cluster/index_en.html">Cluster Training and Prediction</a></li>
</ul>
</li>
</ul>

</nav>

        </div>
      </div>
    </nav>

    <section data-toggle="wy-nav-shift" class="wy-nav-content-wrap">

      
      <nav class="wy-nav-top" role="navigation" aria-label="top navigation">
        <i data-toggle="wy-nav-top" class="fa fa-bars"></i>
        <a href="../../index_en.html">PaddlePaddle</a>
      </nav>


      <div class="wy-nav-content">
        <div class="rst-content">
          

<div role="navigation" aria-label="breadcrumbs navigation">
  <ul class="wy-breadcrumbs">
    <li><a href="../../index_en.html">Docs</a> &raquo;</li>
      
    <li>Intel® MKL-DNN on PaddlePaddle: Design Doc</li>
      <li class="wy-breadcrumbs-aside">
        
          
            <a href="../../_sources/design/mkl/mkldnn.md.txt" rel="nofollow"> View page source</a>
          
        
      </li>
  </ul>
  <hr/>
</div>
          <div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article">
           <div itemprop="articleBody">
            
  <div class="section" id="intel-mkl-dnn-on-paddlepaddle-design-doc">
<span id="intel-mkl-dnn-on-paddlepaddle-design-doc"></span><h1>Intel® MKL-DNN on PaddlePaddle: Design Doc<a class="headerlink" href="#intel-mkl-dnn-on-paddlepaddle-design-doc" title="Permalink to this headline">¶</a></h1>
<p>我们计划将英特尔深度神经网络数学库<a class="reference external" href="https://github.com/01org/mkl-dnn">Intel MKL-DNN</a>
(Intel Math Kernel Library for Deep Neural Networks)集成到PaddlePaddle，
充分展现英特尔平台的优势，有效提升PaddlePaddle在英特尔架构上的性能。</p>
<div align="center">
<img src="image/overview.png"><br/>
Figure 1. PaddlePaddle on IA
</div><p>近期目标</p>
<ul class="simple">
<li>完成常用Layer的MKL-DNN实现。</li>
<li>完成常见深度神经网络VGG，GoogLeNet 和 ResNet的MKL-DNN实现。</li>
</ul>
<p>目前的优化，主要针对PaddlePaddle在重构之前的代码框架以及V1的API。
具体的完成状态可以参见<a class="reference external" href="https://github.com/PaddlePaddle/Paddle/projects/21">这里</a>。</p>
<div class="section" id="contents">
<span id="contents"></span><h2>Contents<a class="headerlink" href="#contents" title="Permalink to this headline">¶</a></h2>
<ul class="simple">
<li><a class="reference external" href="#overview">Overview</a></li>
<li><a class="reference external" href="#actions">Actions</a><ul>
<li><a class="reference external" href="#cmake">CMake</a></li>
<li><a class="reference external" href="#matrix">Matrix</a></li>
<li><a class="reference external" href="#layers">Layers</a></li>
<li><a class="reference external" href="#activations">Activations</a></li>
<li><a class="reference external" href="#parameters">Parameters</a></li>
<li><a class="reference external" href="#gradients">Gradients</a></li>
<li><a class="reference external" href="#unit-tests">Unit Tests</a></li>
<li><a class="reference external" href="#python-api">Python API</a></li>
<li><a class="reference external" href="#benchmarking">Benchmarking</a></li>
<li><a class="reference external" href="#others">Others</a></li>
</ul>
</li>
<li><a class="reference external" href="#design-concerns">Design Concerns</a></li>
</ul>
</div>
<div class="section" id="overview">
<span id="overview"></span><h2>Overview<a class="headerlink" href="#overview" title="Permalink to this headline">¶</a></h2>
<p>我们会把MKL-DNN会作为第三方库集成进PaddlePaddle，与其他第三方库一样，会在编译PaddlePaddle的时候下载并编译MKL-DNN。</p>
<p>同时，为了进一步提升PaddlePaddle在基本数学运算的计算速度，我们也将MKLML即(MKL small library[<a class="reference external" href="#references">1</a>])
作为另一个第三方库集成进PaddlePaddle，它只会包括生成好的动态库和头文件。</p>
<p>MKL，MKLML以及MKL-DNN三者关系如下表：</p>
<p>| Name        |  Open Source     | License     | Descriptions  |
| :&#8212;&#8212;&#8212;- | :&#8212;&#8212;&#8212;&#8212;&#8212; | :&#8212;&#8212;&#8212;- | :&#8212;&#8212;&#8212;&#8212; |
|   MKL       |     No           | Proprietary | Accelerate math processing routines |
|   MKLML     |     No           | Proprietary | Small package of MKL, especially for Machine Learning |
|   MKL-DNN   |     Yes          | Apache 2.0  | Accelerate primitives processing routines especially for Deep Neural Networks  |</p>
<p>MKLML可以与MKL-DNN共同使用，以此达到最好的性能。</p>
<div align="center">
<img src="image/engine.png"><br/>
Figure 2. PaddlePaddle with MKL Engines
</div></div>
<div class="section" id="actions">
<span id="actions"></span><h2>Actions<a class="headerlink" href="#actions" title="Permalink to this headline">¶</a></h2>
<p>添加的相关文件和目录结构如下：</p>
<div class="highlight-txt"><div class="highlight"><pre><span></span>PaddlePaddle/Paddle
├── ...
├── cmake/
│   ├── external/
│   │   ├── ...
│   │   ├── mkldnn.cmake
│   │   └── mklml.cmake
└── paddle/
    ├── ...
    ├── math/
    │   ├── ...
    │   └── MKLDNNMatrix.*
    └── gserver/
        ├── ...
        ├── layers/
        │   ├── ...
        │   └── MKLDNN*Layer.*
        ├── activations/
        │   ├── ...
        │   └── MKLDNNActivations.*
        └── tests/
            ├── ...
            ├── MKLDNNTester.*
            └── test_MKLDNN.cpp
</pre></div>
</div>
<div class="section" id="cmake">
<span id="cmake"></span><h3>CMake<a class="headerlink" href="#cmake" title="Permalink to this headline">¶</a></h3>
<p>在<code class="docutils literal"><span class="pre">CMakeLists.txt</span></code>中提供一个与MKL有关的总开关：<code class="docutils literal"><span class="pre">WITH_MKL</span></code>，它负责决定编译时是否使用MKLML和MKL-DNN</p>
<ul class="simple">
<li><code class="docutils literal"><span class="pre">WITH_MKLML</span></code> 控制是否使用MKLML库。
当打开<code class="docutils literal"><span class="pre">WITH_MKL</span></code>时，会自动使用MKLML库作为PaddlePaddle的CBLAS和LAPACK库，同时会开启Intel OpenMP用于提高MKLML的性能。
编译时会把对应的头文件和库放在<code class="docutils literal"><span class="pre">build/third_party/install/mklml/*</span></code>目录下对应的地方。
MKLML的库目前都是动态库，主要包括<code class="docutils literal"><span class="pre">libiomp5.so</span></code>和<code class="docutils literal"><span class="pre">libmklml_intel.so</span></code>。</li>
<li><code class="docutils literal"><span class="pre">WITH_MKLDNN</span></code> 控制是否使用MKL-DNN。
当开启<code class="docutils literal"><span class="pre">WITH_MKL</span></code>时，会自动根据硬件配置[<a class="reference external" href="#references">2</a>]选择是否编译MKL-DNN。
编译时会把对应的头文件和库放在<code class="docutils literal"><span class="pre">build/third_party/install/mkldnn/*</span></code>目录下对应的地方。
MKL-DNN的库目前只有动态库<code class="docutils literal"><span class="pre">libmkldnn.so</span></code>。</li>
</ul>
</div>
<div class="section" id="matrix">
<span id="matrix"></span><h3>Matrix<a class="headerlink" href="#matrix" title="Permalink to this headline">¶</a></h3>
<p>目前在PaddlePaddle中数据都是以<code class="docutils literal"><span class="pre">NCHW</span></code>的格式存储，但是在MKL-DNN中的排列方式不止这一种。
所以我们定义了一个<code class="docutils literal"><span class="pre">MKLDNNMatrix</span></code>用于管理MKL-DNN数据的不同格式以及相互之间的转换。</p>
<div align="center">
<img src="image/matrix.png"><br/>
Figure 3. MKLDNNMatrix
</div></div>
<div class="section" id="layers">
<span id="layers"></span><h3>Layers<a class="headerlink" href="#layers" title="Permalink to this headline">¶</a></h3>
<p>所有MKL-DNN的Layers都会继承于<code class="docutils literal"><span class="pre">MKLDNNLayer</span></code>，该类继承于PaddlePaddle的基类<code class="docutils literal"><span class="pre">Layer</span></code>。
在<code class="docutils literal"><span class="pre">MKLDNNLayer</span></code>中会提供一些必要的接口和函数，并且会写好<code class="docutils literal"><span class="pre">forward</span></code>和<code class="docutils literal"><span class="pre">backward</span></code>的基本逻辑，
子类只需要使用定义好的接口，实现具体的函数功能即可。</p>
<div align="center">
<img src="image/layers.png"><br/>
Figure 4. MKLDNNLayer
</div><p>每个MKLDNNLayer都包含用于内部存储和外部存储的一系列MKLDNNMatrix：</p>
<ul class="simple">
<li>内部存储（internel memory）：<code class="docutils literal"><span class="pre">inVal_</span></code>,<code class="docutils literal"><span class="pre">inGrad_</span></code>,<code class="docutils literal"><span class="pre">outVal_</span></code>和<code class="docutils literal"><span class="pre">outGrad_</span></code>，分别代表输入数据，输入梯度，输出数据和输出梯度。</li>
<li>外部存储（external memory）：都是以ext开头，比如<code class="docutils literal"><span class="pre">extInVal_</span></code>和<code class="docutils literal"><span class="pre">extInGrad_</span></code>，它们主要是用于，
当数据格式与PaddlePaddle默认的<code class="docutils literal"><span class="pre">NCHW</span></code>格式不匹配时，转换内存的工作。
需要注意的是，PaddlePaddle的activation会直接使用<code class="docutils literal"><span class="pre">output_.value</span></code>和<code class="docutils literal"><span class="pre">output_.grad</span></code>，
所以<code class="docutils literal"><span class="pre">extOutVal_</span></code>和<code class="docutils literal"><span class="pre">extOutGrad_</span></code>必须分别与<code class="docutils literal"><span class="pre">output_.value</span></code>和<code class="docutils literal"><span class="pre">output_.grad</span></code>共享内存，
如果不需要外部存储用于转换，那么对应的内部存储也会与它们共享内存。</li>
<li>转换函数（resetXXX）： 包括<code class="docutils literal"><span class="pre">resetInValue</span></code>，<code class="docutils literal"><span class="pre">resetInGrad</span></code>，<code class="docutils literal"><span class="pre">resetOutValue</span></code>和<code class="docutils literal"><span class="pre">resetOutGrad</span></code>，
表示对输入数据，输入梯度，输出数据和输出梯度的转换。
这些函数会根据输入参数重新设置内部和外部存储，当然这两者也可以相等，即表示不需要转换。</li>
</ul>
<p>注意：每个<code class="docutils literal"><span class="pre">MKLDNNlayer</span></code>的子类只需要使用内部存储就可以了，所有外部的转换工作都会在reset系列函数中都准备好。</p>
</div>
<div class="section" id="activations">
<span id="activations"></span><h3>Activations<a class="headerlink" href="#activations" title="Permalink to this headline">¶</a></h3>
<p>在重构前的PaddlePaddle中，激活函数是独立于<code class="docutils literal"><span class="pre">Layer</span></code>的概念，并且输入输出都是共用一块内存，
所以添加了对应的<code class="docutils literal"><span class="pre">MKLDNNActivation</span></code>来实现，方式类似于<code class="docutils literal"><span class="pre">MKLDNNLayer</span></code>。</p>
</div>
<div class="section" id="parameters">
<span id="parameters"></span><h3>Parameters<a class="headerlink" href="#parameters" title="Permalink to this headline">¶</a></h3>
<p>对于有参数的层，我们会保证<code class="docutils literal"><span class="pre">MKLDNNLayer</span></code>使用的参数与PaddlePaddle申请的buffer共用一块内存。
如果存在数据排列格式不一样的情况时，我们会在网络训练之前把格式转换为MKL-DNN希望的格式，
在训练结束的时候再保存为PaddlePaddle的格式，但是整个训练过程中不需要任何转换。
这样既使得最终保存的参数格式与PaddlePaddle一致，又可以避免不必要的转换。</p>
</div>
<div class="section" id="gradients">
<span id="gradients"></span><h3>Gradients<a class="headerlink" href="#gradients" title="Permalink to this headline">¶</a></h3>
<p>由于MKL-DNN的操作都是直接覆盖的形式，也就是说输出的结果不会在原来的数据上累加，
这样带来的好处就是不需要一直清空memory，节省了不必要的操作。
但是注意的是，当网络出现分支且在<code class="docutils literal"><span class="pre">backward</span></code>的时候，需要累加不同Layer传过来的梯度。
所以在<code class="docutils literal"><span class="pre">MKLDNNlayer</span></code>中实现了一个merge的方法，此时每个小分支的<code class="docutils literal"><span class="pre">Input</span> <span class="pre">Gradient</span></code>
会先临时保存在<code class="docutils literal"><span class="pre">MKLDNNMatrix</span></code>中，由分支处的Layer负责求和，并把结果放到当前层的<code class="docutils literal"><span class="pre">output_.grad</span></code>中。
所以整体上，在实现每个子类的时候就不需要关心分支的事情了。</p>
<div align="center">
<img src="image/gradients.png"><br/>
Figure 5. Merge Gradients
</div></div>
<div class="section" id="unit-tests">
<span id="unit-tests"></span><h3>Unit Tests<a class="headerlink" href="#unit-tests" title="Permalink to this headline">¶</a></h3>
<p>我们会添加<code class="docutils literal"><span class="pre">test_MKLDNN.cpp</span></code>和<code class="docutils literal"><span class="pre">MKLDNNTester.*</span></code>用于MKL-DNN的测试。
测试分为每个Layer（或Activation）的单元测试和简单网络的整体测试。
每个测试会对比PaddlePaddle中CPU算出的结果与MKL-DNN的结果，小于某个比较小的阈值认为通过。</p>
</div>
<div class="section" id="python-api">
<span id="python-api"></span><h3>Python API<a class="headerlink" href="#python-api" title="Permalink to this headline">¶</a></h3>
<p>目前只考虑<strong>v1 API</strong>。</p>
<p>计划在<code class="docutils literal"><span class="pre">python/paddle/trainer/config_parser.py</span></code>里面添加<code class="docutils literal"><span class="pre">use_mkldnn</span></code>这个选择，方便用户选择使用MKL-DNN的layers。</p>
<p>具体实现方式比如：</p>
<div class="highlight-python"><div class="highlight"><pre><span></span><span class="n">use_mkldnn</span> <span class="o">=</span> <span class="nb">bool</span><span class="p">(</span><span class="nb">int</span><span class="p">(</span><span class="n">g_command_config_args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;use_mkldnn&quot;</span><span class="p">,</span> <span class="mi">0</span><span class="p">)))</span>
<span class="k">if</span> <span class="n">use_mkldnn</span>
    <span class="bp">self</span><span class="o">.</span><span class="n">layer_type</span> <span class="o">=</span> <span class="n">mkldnn_</span><span class="o">*</span>
</pre></div>
</div>
<p>所有MKL-DNN的<code class="docutils literal"><span class="pre">layer_type</span></code>会以*mkldnn_*开头，这些会在<code class="docutils literal"><span class="pre">MKLDNN*Layer</span></code>注册layer的时候保证，以示区分。</p>
<p>同时,会在<code class="docutils literal"><span class="pre">paddle/utils.Flags</span></code>中添加一个<code class="docutils literal"><span class="pre">use_mkldnn</span></code>的flag，用于选择是否使用MKL-DNN的相关功能。</p>
</div>
<div class="section" id="benchmarking">
<span id="benchmarking"></span><h3>Benchmarking<a class="headerlink" href="#benchmarking" title="Permalink to this headline">¶</a></h3>
<p>会添加相应的脚本在<a class="reference external" href="https://github.com/PaddlePaddle/Paddle/tree/develop/benchmark/paddle/image">这里</a>，用于测试和对比在使用MKL-DNN前后的CNN网络性能。
测试的性能对比结果会在<a class="reference external" href="https://github.com/PaddlePaddle/Paddle/blob/develop/benchmark/IntelOptimizedPaddle.md">IntelOptimizedPaddle.md</a></p>
</div>
<div class="section" id="others">
<span id="others"></span><h3>Others<a class="headerlink" href="#others" title="Permalink to this headline">¶</a></h3>
<ol class="simple">
<li>如果在使用MKL-DNN的情况下，会把CPU的Buffer对齐为4096，具体可以参考MKL-DNN中的<a class="reference external" href="https://github.com/01org/mkl-dnn/blob/master/include/mkldnn.hpp#L673">memory</a>。</li>
<li>深入PaddlePaddle，寻找有没有其他可以优化的可能，进一步优化。比如可能会用OpenMP改进SGD的更新性能。</li>
</ol>
</div>
</div>
<div class="section" id="design-concerns">
<span id="design-concerns"></span><h2>Design Concerns<a class="headerlink" href="#design-concerns" title="Permalink to this headline">¶</a></h2>
<p>为了更好的符合PaddlePaddle的代码风格[<a class="reference external" href="#references">3</a>]，同时又尽可能少的牺牲MKL-DNN的性能[<a class="reference external" href="#references">4</a>]。</p>
<p>我们总结出一些特别需要注意的点：</p>
<ol class="simple">
<li>使用**deviceId_**。为了尽可能少的在父类Layer中添加变量或者函数，
我们决定使用已有的<code class="docutils literal"><span class="pre">deviceId_</span></code>变量来区分layer的属性，定义<code class="docutils literal"><span class="pre">-2</span></code>为<code class="docutils literal"><span class="pre">MKLDNNLayer</span></code>特有的设备ID。</li>
<li>重写父类Layer的<strong>init</strong>函数，修改<code class="docutils literal"><span class="pre">deviceId_</span></code>为<code class="docutils literal"><span class="pre">-2</span></code>，代表这个layer是用于跑在MKL-DNN的环境下。</li>
<li>创建<code class="docutils literal"><span class="pre">MKLDNNBase</span></code>，定义一些除了layer和memory相关的类和函数。
包括MKL-DNN会用到<code class="docutils literal"><span class="pre">MKLDNNStream</span></code>和<code class="docutils literal"><span class="pre">CPUEngine</span></code>，和未来可能还会用到<code class="docutils literal"><span class="pre">FPGAEngine</span></code>等。</li>
<li>如果MKL-DNN layer的后面接有cpu device，那么就会使<code class="docutils literal"><span class="pre">output_.value</span></code>与<code class="docutils literal"><span class="pre">extOutVal_</span></code>共享内存，
同时数据格式就是<code class="docutils literal"><span class="pre">NCHW</span></code>，这样下一个cpu device就能拿到正确的数据。
在有普通的CPU layer时， <code class="docutils literal"><span class="pre">extOutVal_</span></code>和<code class="docutils literal"><span class="pre">extOutGrad_</span></code>的格式始终是<code class="docutils literal"><span class="pre">NCHW</span></code>或者<code class="docutils literal"><span class="pre">NC</span></code>。</li>
</ol>
</div>
<div class="section" id="references">
<span id="references"></span><h2>References<a class="headerlink" href="#references" title="Permalink to this headline">¶</a></h2>
<ol class="simple">
<li><a class="reference external" href="https://github.com/01org/mkl-dnn#linking-your-application">MKL small library</a>是<a class="reference external" href="https://software.intel.com/en-us/mkl">Intel MKL</a>的一个子集。
主要包括了深度学习相关的数学原语与操作，一般由MKL-DNN在发布<a class="reference external" href="https://github.com/01org/mkl-dnn/releases">新版本</a>时一起更新。</li>
<li><a class="reference external" href="https://github.com/01org/mkl-dnn#system-requirements">MKL-DNN System Requirements</a>。
目前在PaddlePaddle中，仅会在支持AVX2指令集及以上的机器才使用MKL-DNN。</li>
<li><a class="reference external" href="https://github.com/PaddlePaddle/Paddle/pull/3096">原来的方案</a>会引入<strong>nextLayer</strong>的信息。
但是在PaddlePaddle中，无论是重构前的layer还是重构后的op，都不会想要知道next layer/op的信息。</li>
<li>MKL-DNN的高性能格式与PaddlePaddle原有的<code class="docutils literal"><span class="pre">NCHW</span></code>不同(PaddlePaddle中的cuDNN部分使用的也是<code class="docutils literal"><span class="pre">NCHW</span></code>，所以不存在这个问题)。
所以需要引入一个转换方法，并且只需要在必要的时候转换这种格式，才能更好的发挥MKL-DNN的性能。</li>
</ol>
</div>
</div>


           </div>
          </div>
          <footer>
  

  <hr/>

  <div role="contentinfo">
    <p>
        &copy; Copyright 2016, PaddlePaddle developers.

    </p>
  </div>
  Built with <a href="http://sphinx-doc.org/">Sphinx</a> using a <a href="https://github.com/snide/sphinx_rtd_theme">theme</a> provided by <a href="https://readthedocs.org">Read the Docs</a>. 

</footer>

        </div>
      </div>

    </section>

  </div>
  

    <script type="text/javascript">
        var DOCUMENTATION_OPTIONS = {
            URL_ROOT:'../../',
            VERSION:'',
            COLLAPSE_INDEX:false,
            FILE_SUFFIX:'.html',
            HAS_SOURCE:  true
        };
    </script>
      <script type="text/javascript" src="../../_static/jquery.js"></script>
      <script type="text/javascript" src="../../_static/underscore.js"></script>
      <script type="text/javascript" src="../../_static/doctools.js"></script>
      <script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.0/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script>

  
    <script type="text/javascript" src="../../_static/js/theme.js"></script>
  

  <script type="text/javascript">
      jQuery(function () {
          SphinxRtdTheme.StickyNav.enable();
      });
  </script>
   

</body>
</html>