Skip to content

P
Paddle

PaddlePaddle / Paddle
大约 1 年 前同步成功

通知 2299
Star 20931
Fork 5422
P
Paddle
切换分支/标签
查找文件 普通视图历史永久链接Permalink
mkldnn.html 30.7 KB
Newer Older
T
Deploy to GitHub Pages: c13805e923fb4fbe1eeae06c2acc0b60a633d5a6  
Travis CI 已提交
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 


<!DOCTYPE html>
<!--[if IE 8]><html class="no-js lt-ie9" lang="en" > <![endif]-->
<!--[if gt IE 8]><!--> <html class="no-js" lang="en" > <!--<![endif]-->
<head>
  <meta charset="utf-8">
  
  <meta name="viewport" content="width=device-width, initial-scale=1.0">
  
  <title>Intel® MKL-DNN on PaddlePaddle: Design Doc &mdash; PaddlePaddle  documentation</title>
  

  
  

  

  
  
    

  

  
  
    <link rel="stylesheet" href="../../_static/css/theme.css" type="text/css" />
  

  
  
        <link rel="index" title="Index"
              href="../../genindex.html"/>
        <link rel="search" title="Search" href="../../search.html"/>
    <link rel="top" title="PaddlePaddle  documentation" href="../../index.html"/> 

  <link rel="stylesheet" href="https://cdn.jsdelivr.net/perfect-scrollbar/0.6.14/css/perfect-scrollbar.min.css" type="text/css" />
  <link rel="stylesheet" href="../../_static/css/override.css" type="text/css" />
  <script>
  var _hmt = _hmt || [];
  (function() {
    var hm = document.createElement("script");
    hm.src = "//hm.baidu.com/hm.js?b9a314ab40d04d805655aab1deee08ba";
    var s = document.getElementsByTagName("script")[0]; 
    s.parentNode.insertBefore(hm, s);
  })();
  </script>

  

  
  <script src="../../_static/js/modernizr.min.js"></script>

</head>

<body class="wy-body-for-nav" role="document">

  
  <header class="site-header">
    <div class="site-logo">
      <a href="/"><img src="../../_static/images/PP_w.png"></a>
    </div>
    <div class="site-nav-links">
      <div class="site-menu">
        <a class="fork-on-github" href="https://github.com/PaddlePaddle/Paddle" target="_blank"><i class="fa fa-github"></i>Fork me on Github</a>
        <div class="language-switcher dropdown">
          <a type="button" data-toggle="dropdown">
            <span>English</span>
            <i class="fa fa-angle-up"></i>
            <i class="fa fa-angle-down"></i>
          </a>
          <ul class="dropdown-menu">
            <li><a href="/doc_cn">中文</a></li>
            <li><a href="/doc">English</a></li>
          </ul>
        </div>
        <ul class="site-page-links">
          <li><a href="/">Home</a></li>
        </ul>
      </div>
      <div class="doc-module">
        
        <ul>
<li class="toctree-l1"><a class="reference internal" href="../../getstarted/index_en.html">GET STARTED</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../howto/index_en.html">HOW TO</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../api/index_en.html">API</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../mobile/index_en.html">MOBILE</a></li>
</ul>

        
<div role="search">
  <form id="rtd-search-form" class="wy-form" action="../../search.html" method="get">
    <input type="text" name="q" placeholder="Search docs" />
    <input type="hidden" name="check_keywords" value="yes" />
    <input type="hidden" name="area" value="default" />
  </form>
</div>        
      </div>
    </div>
  </header>
  
  <div class="main-content-wrap">

    
    <nav class="doc-menu-vertical" role="navigation">
        
          
          <ul>
<li class="toctree-l1"><a class="reference internal" href="../../getstarted/index_en.html">GET STARTED</a><ul>
<li class="toctree-l2"><a class="reference internal" href="../../getstarted/build_and_install/index_en.html">Install and Build</a><ul>
<li class="toctree-l3"><a class="reference internal" href="../../getstarted/build_and_install/pip_install_en.html">Install Using pip</a></li>
<li class="toctree-l3"><a class="reference internal" href="../../getstarted/build_and_install/docker_install_en.html">Run in Docker Containers</a></li>
<li class="toctree-l3"><a class="reference internal" href="../../howto/dev/build_en.html">Build using Docker</a></li>
<li class="toctree-l3"><a class="reference internal" href="../../getstarted/build_and_install/build_from_source_en.html">Build from Sources</a></li>
</ul>
</li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="../../howto/index_en.html">HOW TO</a><ul>
<li class="toctree-l2"><a class="reference internal" href="../../howto/usage/cmd_parameter/index_en.html">Set Command-line Parameters</a><ul>
<li class="toctree-l3"><a class="reference internal" href="../../howto/usage/cmd_parameter/use_case_en.html">Use Case</a></li>
<li class="toctree-l3"><a class="reference internal" href="../../howto/usage/cmd_parameter/arguments_en.html">Argument Outline</a></li>
<li class="toctree-l3"><a class="reference internal" href="../../howto/usage/cmd_parameter/detail_introduction_en.html">Detail Description</a></li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="../../howto/usage/cluster/cluster_train_en.html">PaddlePaddle Distributed Training</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../howto/usage/k8s/k8s_en.html">Paddle On Kubernetes</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../howto/usage/k8s/k8s_aws_en.html">Distributed PaddlePaddle Training on AWS with Kubernetes</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../howto/dev/new_layer_en.html">Write New Layers</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../howto/dev/contribute_to_paddle_en.html">Contribute Code</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../howto/dev/write_docs_en.html">Contribute Documentation</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../howto/deep_model/rnn/index_en.html">RNN Models</a><ul>
<li class="toctree-l3"><a class="reference internal" href="../../howto/deep_model/rnn/rnn_config_en.html">RNN Configuration</a></li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="../../howto/optimization/gpu_profiling_en.html">Tune GPU Performance</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="../../api/index_en.html">API</a><ul>
<li class="toctree-l2"><a class="reference internal" href="../../api/v2/model_configs.html">Model Configuration</a><ul>
<li class="toctree-l3"><a class="reference internal" href="../../api/v2/config/activation.html">Activation</a></li>
<li class="toctree-l3"><a class="reference internal" href="../../api/v2/config/layer.html">Layers</a></li>
<li class="toctree-l3"><a class="reference internal" href="../../api/v2/config/evaluators.html">Evaluators</a></li>
<li class="toctree-l3"><a class="reference internal" href="../../api/v2/config/optimizer.html">Optimizer</a></li>
<li class="toctree-l3"><a class="reference internal" href="../../api/v2/config/pooling.html">Pooling</a></li>
<li class="toctree-l3"><a class="reference internal" href="../../api/v2/config/networks.html">Networks</a></li>
<li class="toctree-l3"><a class="reference internal" href="../../api/v2/config/attr.html">Parameter Attribute</a></li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="../../api/v2/data.html">Data Reader Interface and DataSets</a><ul>
<li class="toctree-l3"><a class="reference internal" href="../../api/v2/data/data_reader.html">Data Reader Interface</a></li>
<li class="toctree-l3"><a class="reference internal" href="../../api/v2/data/image.html">Image Interface</a></li>
<li class="toctree-l3"><a class="reference internal" href="../../api/v2/data/dataset.html">Dataset</a></li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="../../api/v2/run_logic.html">Training and Inference</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../api/v2/fluid.html">Fluid</a><ul>
<li class="toctree-l3"><a class="reference internal" href="../../api/v2/fluid/layers.html">Layers</a></li>
<li class="toctree-l3"><a class="reference internal" href="../../api/v2/fluid/data_feeder.html">DataFeeder</a></li>
<li class="toctree-l3"><a class="reference internal" href="../../api/v2/fluid/executor.html">Executor</a></li>
<li class="toctree-l3"><a class="reference internal" href="../../api/v2/fluid/initializer.html">Initializer</a></li>
<li class="toctree-l3"><a class="reference internal" href="../../api/v2/fluid/evaluator.html">Evaluator</a></li>
<li class="toctree-l3"><a class="reference internal" href="../../api/v2/fluid/nets.html">Nets</a></li>
<li class="toctree-l3"><a class="reference internal" href="../../api/v2/fluid/optimizer.html">Optimizer</a></li>
<li class="toctree-l3"><a class="reference internal" href="../../api/v2/fluid/param_attr.html">ParamAttr</a></li>
<li class="toctree-l3"><a class="reference internal" href="../../api/v2/fluid/profiler.html">Profiler</a></li>
<li class="toctree-l3"><a class="reference internal" href="../../api/v2/fluid/regularizer.html">Regularizer</a></li>
</ul>
</li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="../../mobile/index_en.html">MOBILE</a><ul>
<li class="toctree-l2"><a class="reference internal" href="../../mobile/cross_compiling_for_android_en.html">Build PaddlePaddle for Android</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../mobile/cross_compiling_for_ios_en.html">PaddlePaddle Compiling Guide for iOS</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../mobile/cross_compiling_for_raspberry_en.html">Build PaddlePaddle for Raspberry Pi</a></li>
</ul>
</li>
</ul>

        
    </nav>
    
    <section class="doc-content-wrap">

      

 







<div role="navigation" aria-label="breadcrumbs navigation">
  <ul class="wy-breadcrumbs">
      
    <li>Intel® MKL-DNN on PaddlePaddle: Design Doc</li>
  </ul>
</div>
      
      <div class="wy-nav-content" id="doc-content">
        <div class="rst-content">
          <div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article">
           <div itemprop="articleBody">
            
  <div class="section" id="intel-mkl-dnn-on-paddlepaddle-design-doc">
<span id="intel-mkl-dnn-on-paddlepaddle-design-doc"></span><h1>Intel® MKL-DNN on PaddlePaddle: Design Doc<a class="headerlink" href="#intel-mkl-dnn-on-paddlepaddle-design-doc" title="Permalink to this headline"></a></h1>
<p>我们计划将英特尔深度神经网络数学库<a class="reference external" href="https://github.com/01org/mkl-dnn">Intel MKL-DNN</a>
(Intel Math Kernel Library for Deep Neural Networks)集成到PaddlePaddle，
充分展现英特尔平台的优势，有效提升PaddlePaddle在英特尔架构上的性能。</p>
<div align="center">
<img src="image/overview.png"><br/>
Figure 1. PaddlePaddle on IA
</div><p>近期目标</p>
<ul class="simple">
<li>完成常用Layer的MKL-DNN实现。</li>
<li>完成常见深度神经网络VGG，GoogLeNet 和 ResNet的MKL-DNN实现。</li>
</ul>
<p>目前的优化，主要针对PaddlePaddle在重构之前的代码框架以及V1的API。
具体的完成状态可以参见<a class="reference external" href="https://github.com/PaddlePaddle/Paddle/projects/21">这里</a>。</p>
<div class="section" id="contents">
<span id="contents"></span><h2>Contents<a class="headerlink" href="#contents" title="Permalink to this headline"></a></h2>
<ul class="simple">
<li><a class="reference external" href="#overview">Overview</a></li>
<li><a class="reference external" href="#actions">Actions</a><ul>
<li><a class="reference external" href="#cmake">CMake</a></li>
<li><a class="reference external" href="#matrix">Matrix</a></li>
<li><a class="reference external" href="#layers">Layers</a></li>
<li><a class="reference external" href="#activations">Activations</a></li>
<li><a class="reference external" href="#parameters">Parameters</a></li>
<li><a class="reference external" href="#gradients">Gradients</a></li>
<li><a class="reference external" href="#unit-tests">Unit Tests</a></li>
<li><a class="reference external" href="#python-api">Python API</a></li>
<li><a class="reference external" href="#benchmarking">Benchmarking</a></li>
<li><a class="reference external" href="#others">Others</a></li>
</ul>
</li>
<li><a class="reference external" href="#design-concerns">Design Concerns</a></li>
</ul>
</div>
<div class="section" id="overview">
<span id="overview"></span><h2>Overview<a class="headerlink" href="#overview" title="Permalink to this headline"></a></h2>
<p>我们会把MKL-DNN会作为第三方库集成进PaddlePaddle，与其他第三方库一样，会在编译PaddlePaddle的时候下载并编译MKL-DNN。</p>
<p>同时，为了进一步提升PaddlePaddle在基本数学运算的计算速度，我们也将MKLML即(MKL small library[<a class="reference external" href="#references">1</a>])
作为另一个第三方库集成进PaddlePaddle，它只会包括生成好的动态库和头文件。</p>
<p>MKL，MKLML以及MKL-DNN三者关系如下表：</p>
<p>| Name        |  Open Source     | License     | Descriptions  |
| :&#8212;&#8212;&#8212;- | :&#8212;&#8212;&#8212;&#8212;&#8212; | :&#8212;&#8212;&#8212;- | :&#8212;&#8212;&#8212;&#8212; |
|   MKL       |     No           | Proprietary | Accelerate math processing routines |
|   MKLML     |     No           | Proprietary | Small package of MKL, especially for Machine Learning |
|   MKL-DNN   |     Yes          | Apache 2.0  | Accelerate primitives processing routines especially for Deep Neural Networks  |</p>
<p>MKLML可以与MKL-DNN共同使用，以此达到最好的性能。</p>
<div align="center">
<img src="image/engine.png"><br/>
Figure 2. PaddlePaddle with MKL Engines
</div></div>
<div class="section" id="actions">
<span id="actions"></span><h2>Actions<a class="headerlink" href="#actions" title="Permalink to this headline"></a></h2>
<p>添加的相关文件和目录结构如下：</p>
<div class="highlight-txt"><div class="highlight"><pre><span></span>PaddlePaddle/Paddle
├── ...
├── cmake/
│   ├── external/
│   │   ├── ...
│   │   ├── mkldnn.cmake
│   │   └── mklml.cmake
└── paddle/
    ├── ...
    ├── math/
    │   ├── ...
    │   └── MKLDNNMatrix.*
    └── gserver/
        ├── ...
        ├── layers/
        │   ├── ...
        │   └── MKLDNN*Layer.*
        ├── activations/
        │   ├── ...
        │   └── MKLDNNActivations.*
        └── tests/
            ├── ...
            ├── MKLDNNTester.*
            └── test_MKLDNN.cpp
</pre></div>
</div>
<div class="section" id="cmake">
<span id="cmake"></span><h3>CMake<a class="headerlink" href="#cmake" title="Permalink to this headline"></a></h3>
<p>在<code class="docutils literal"><span class="pre">CMakeLists.txt</span></code>中提供一个与MKL有关的总开关：<code class="docutils literal"><span class="pre">WITH_MKL</span></code>，它负责决定编译时是否使用MKLML和MKL-DNN</p>
<ul class="simple">
<li><code class="docutils literal"><span class="pre">WITH_MKLML</span></code> 控制是否使用MKLML库。
当打开<code class="docutils literal"><span class="pre">WITH_MKL</span></code>时，会自动使用MKLML库作为PaddlePaddle的CBLAS和LAPACK库，同时会开启Intel OpenMP用于提高MKLML的性能。
编译时会把对应的头文件和库放在<code class="docutils literal"><span class="pre">build/third_party/install/mklml/*</span></code>目录下对应的地方。
MKLML的库目前都是动态库，主要包括<code class="docutils literal"><span class="pre">libiomp5.so</span></code>和<code class="docutils literal"><span class="pre">libmklml_intel.so</span></code>。</li>
<li><code class="docutils literal"><span class="pre">WITH_MKLDNN</span></code> 控制是否使用MKL-DNN。
当开启<code class="docutils literal"><span class="pre">WITH_MKL</span></code>时，会自动根据硬件配置[<a class="reference external" href="#references">2</a>]选择是否编译MKL-DNN。
编译时会把对应的头文件和库放在<code class="docutils literal"><span class="pre">build/third_party/install/mkldnn/*</span></code>目录下对应的地方。
MKL-DNN的库目前只有动态库<code class="docutils literal"><span class="pre">libmkldnn.so</span></code>。</li>
</ul>
</div>
<div class="section" id="matrix">
<span id="matrix"></span><h3>Matrix<a class="headerlink" href="#matrix" title="Permalink to this headline"></a></h3>
<p>目前在PaddlePaddle中数据都是以<code class="docutils literal"><span class="pre">NCHW</span></code>的格式存储，但是在MKL-DNN中的排列方式不止这一种。
所以我们定义了一个<code class="docutils literal"><span class="pre">MKLDNNMatrix</span></code>用于管理MKL-DNN数据的不同格式以及相互之间的转换。</p>
<div align="center">
<img src="image/matrix.png"><br/>
Figure 3. MKLDNNMatrix
</div></div>
<div class="section" id="layers">
<span id="layers"></span><h3>Layers<a class="headerlink" href="#layers" title="Permalink to this headline"></a></h3>
<p>所有MKL-DNN的Layers都会继承于<code class="docutils literal"><span class="pre">MKLDNNLayer</span></code>，该类继承于PaddlePaddle的基类<code class="docutils literal"><span class="pre">Layer</span></code>。
在<code class="docutils literal"><span class="pre">MKLDNNLayer</span></code>中会提供一些必要的接口和函数，并且会写好<code class="docutils literal"><span class="pre">forward</span></code>和<code class="docutils literal"><span class="pre">backward</span></code>的基本逻辑，
子类只需要使用定义好的接口，实现具体的函数功能即可。</p>
<div align="center">
<img src="image/layers.png"><br/>
Figure 4. MKLDNNLayer
</div><p>每个MKLDNNLayer都包含用于内部存储和外部存储的一系列MKLDNNMatrix：</p>
<ul class="simple">
<li>内部存储（internel memory）：<code class="docutils literal"><span class="pre">inVal_</span></code>,<code class="docutils literal"><span class="pre">inGrad_</span></code>,<code class="docutils literal"><span class="pre">outVal_</span></code>和<code class="docutils literal"><span class="pre">outGrad_</span></code>，分别代表输入数据，输入梯度，输出数据和输出梯度。</li>
<li>外部存储（external memory）：都是以ext开头，比如<code class="docutils literal"><span class="pre">extInVal_</span></code>和<code class="docutils literal"><span class="pre">extInGrad_</span></code>，它们主要是用于，
当数据格式与PaddlePaddle默认的<code class="docutils literal"><span class="pre">NCHW</span></code>格式不匹配时，转换内存的工作。
需要注意的是，PaddlePaddle的activation会直接使用<code class="docutils literal"><span class="pre">output_.value</span></code>和<code class="docutils literal"><span class="pre">output_.grad</span></code>，
所以<code class="docutils literal"><span class="pre">extOutVal_</span></code>和<code class="docutils literal"><span class="pre">extOutGrad_</span></code>必须分别与<code class="docutils literal"><span class="pre">output_.value</span></code>和<code class="docutils literal"><span class="pre">output_.grad</span></code>共享内存，
如果不需要外部存储用于转换，那么对应的内部存储也会与它们共享内存。</li>
<li>转换函数（resetXXX）： 包括<code class="docutils literal"><span class="pre">resetInValue</span></code>，<code class="docutils literal"><span class="pre">resetInGrad</span></code>，<code class="docutils literal"><span class="pre">resetOutValue</span></code>和<code class="docutils literal"><span class="pre">resetOutGrad</span></code>，
表示对输入数据，输入梯度，输出数据和输出梯度的转换。
这些函数会根据输入参数重新设置内部和外部存储，当然这两者也可以相等，即表示不需要转换。</li>
</ul>
<p>注意：每个<code class="docutils literal"><span class="pre">MKLDNNlayer</span></code>的子类只需要使用内部存储就可以了，所有外部的转换工作都会在reset系列函数中都准备好。</p>
</div>
<div class="section" id="activations">
<span id="activations"></span><h3>Activations<a class="headerlink" href="#activations" title="Permalink to this headline"></a></h3>
<p>在重构前的PaddlePaddle中，激活函数是独立于<code class="docutils literal"><span class="pre">Layer</span></code>的概念，并且输入输出都是共用一块内存，
所以添加了对应的<code class="docutils literal"><span class="pre">MKLDNNActivation</span></code>来实现，方式类似于<code class="docutils literal"><span class="pre">MKLDNNLayer</span></code>。</p>
</div>
<div class="section" id="parameters">
<span id="parameters"></span><h3>Parameters<a class="headerlink" href="#parameters" title="Permalink to this headline"></a></h3>
<p>对于有参数的层，我们会保证<code class="docutils literal"><span class="pre">MKLDNNLayer</span></code>使用的参数与PaddlePaddle申请的buffer共用一块内存。
如果存在数据排列格式不一样的情况时，我们会在网络训练之前把格式转换为MKL-DNN希望的格式，
在训练结束的时候再保存为PaddlePaddle的格式，但是整个训练过程中不需要任何转换。
这样既使得最终保存的参数格式与PaddlePaddle一致，又可以避免不必要的转换。</p>
</div>
<div class="section" id="gradients">
<span id="gradients"></span><h3>Gradients<a class="headerlink" href="#gradients" title="Permalink to this headline"></a></h3>
<p>由于MKL-DNN的操作都是直接覆盖的形式，也就是说输出的结果不会在原来的数据上累加，
这样带来的好处就是不需要一直清空memory，节省了不必要的操作。
但是注意的是，当网络出现分支且在<code class="docutils literal"><span class="pre">backward</span></code>的时候，需要累加不同Layer传过来的梯度。
所以在<code class="docutils literal"><span class="pre">MKLDNNlayer</span></code>中实现了一个merge的方法，此时每个小分支的<code class="docutils literal"><span class="pre">Input</span> <span class="pre">Gradient</span></code>
会先临时保存在<code class="docutils literal"><span class="pre">MKLDNNMatrix</span></code>中，由分支处的Layer负责求和，并把结果放到当前层的<code class="docutils literal"><span class="pre">output_.grad</span></code>中。
所以整体上，在实现每个子类的时候就不需要关心分支的事情了。</p>
<div align="center">
<img src="image/gradients.png"><br/>
Figure 5. Merge Gradients
</div></div>
<div class="section" id="unit-tests">
<span id="unit-tests"></span><h3>Unit Tests<a class="headerlink" href="#unit-tests" title="Permalink to this headline"></a></h3>
<p>我们会添加<code class="docutils literal"><span class="pre">test_MKLDNN.cpp</span></code>和<code class="docutils literal"><span class="pre">MKLDNNTester.*</span></code>用于MKL-DNN的测试。
测试分为每个Layer（或Activation）的单元测试和简单网络的整体测试。
每个测试会对比PaddlePaddle中CPU算出的结果与MKL-DNN的结果，小于某个比较小的阈值认为通过。</p>
</div>
<div class="section" id="python-api">
<span id="python-api"></span><h3>Python API<a class="headerlink" href="#python-api" title="Permalink to this headline"></a></h3>
<p>目前只考虑<strong>v1 API</strong>。</p>
<p>计划在<code class="docutils literal"><span class="pre">python/paddle/trainer/config_parser.py</span></code>里面添加<code class="docutils literal"><span class="pre">use_mkldnn</span></code>这个选择，方便用户选择使用MKL-DNN的layers。</p>
<p>具体实现方式比如：</p>
<div class="highlight-python"><div class="highlight"><pre><span></span><span class="n">use_mkldnn</span> <span class="o">=</span> <span class="nb">bool</span><span class="p">(</span><span class="nb">int</span><span class="p">(</span><span class="n">g_command_config_args</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;use_mkldnn&quot;</span><span class="p">,</span> <span class="mi">0</span><span class="p">)))</span>
<span class="k">if</span> <span class="n">use_mkldnn</span>
    <span class="bp">self</span><span class="o">.</span><span class="n">layer_type</span> <span class="o">=</span> <span class="n">mkldnn_</span><span class="o">*</span>
</pre></div>
</div>
<p>所有MKL-DNN的<code class="docutils literal"><span class="pre">layer_type</span></code>会以*mkldnn_*开头，这些会在<code class="docutils literal"><span class="pre">MKLDNN*Layer</span></code>注册layer的时候保证，以示区分。</p>
<p>同时,会在<code class="docutils literal"><span class="pre">paddle/utils.Flags</span></code>中添加一个<code class="docutils literal"><span class="pre">use_mkldnn</span></code>的flag，用于选择是否使用MKL-DNN的相关功能。</p>
</div>
<div class="section" id="benchmarking">
<span id="benchmarking"></span><h3>Benchmarking<a class="headerlink" href="#benchmarking" title="Permalink to this headline"></a></h3>
<p>会添加相应的脚本在<a class="reference external" href="https://github.com/PaddlePaddle/Paddle/tree/develop/benchmark/paddle/image">这里</a>，用于测试和对比在使用MKL-DNN前后的CNN网络性能。
测试的性能对比结果会在<a class="reference external" href="https://github.com/PaddlePaddle/Paddle/blob/develop/benchmark/IntelOptimizedPaddle.md">IntelOptimizedPaddle.md</a></p>
</div>
<div class="section" id="others">
<span id="others"></span><h3>Others<a class="headerlink" href="#others" title="Permalink to this headline"></a></h3>
<ol class="simple">
<li>如果在使用MKL-DNN的情况下，会把CPU的Buffer对齐为4096，具体可以参考MKL-DNN中的<a class="reference external" href="https://github.com/01org/mkl-dnn/blob/master/include/mkldnn.hpp#L673">memory</a>。</li>
<li>深入PaddlePaddle，寻找有没有其他可以优化的可能，进一步优化。比如可能会用OpenMP改进SGD的更新性能。</li>
</ol>
</div>
</div>
<div class="section" id="design-concerns">
<span id="design-concerns"></span><h2>Design Concerns<a class="headerlink" href="#design-concerns" title="Permalink to this headline"></a></h2>
<p>为了更好的符合PaddlePaddle的代码风格[<a class="reference external" href="#references">3</a>]，同时又尽可能少的牺牲MKL-DNN的性能[<a class="reference external" href="#references">4</a>]。</p>
<p>我们总结出一些特别需要注意的点：</p>
<ol class="simple">
<li>使用**deviceId_**。为了尽可能少的在父类Layer中添加变量或者函数，
我们决定使用已有的<code class="docutils literal"><span class="pre">deviceId_</span></code>变量来区分layer的属性，定义<code class="docutils literal"><span class="pre">-2</span></code>为<code class="docutils literal"><span class="pre">MKLDNNLayer</span></code>特有的设备ID。</li>
<li>重写父类Layer的<strong>init</strong>函数，修改<code class="docutils literal"><span class="pre">deviceId_</span></code>为<code class="docutils literal"><span class="pre">-2</span></code>，代表这个layer是用于跑在MKL-DNN的环境下。</li>
<li>创建<code class="docutils literal"><span class="pre">MKLDNNBase</span></code>，定义一些除了layer和memory相关的类和函数。
包括MKL-DNN会用到<code class="docutils literal"><span class="pre">MKLDNNStream</span></code>和<code class="docutils literal"><span class="pre">CPUEngine</span></code>，和未来可能还会用到<code class="docutils literal"><span class="pre">FPGAEngine</span></code>等。</li>
<li>如果MKL-DNN layer的后面接有cpu device，那么就会使<code class="docutils literal"><span class="pre">output_.value</span></code>与<code class="docutils literal"><span class="pre">extOutVal_</span></code>共享内存，
同时数据格式就是<code class="docutils literal"><span class="pre">NCHW</span></code>，这样下一个cpu device就能拿到正确的数据。
在有普通的CPU layer时， <code class="docutils literal"><span class="pre">extOutVal_</span></code>和<code class="docutils literal"><span class="pre">extOutGrad_</span></code>的格式始终是<code class="docutils literal"><span class="pre">NCHW</span></code>或者<code class="docutils literal"><span class="pre">NC</span></code>。</li>
</ol>
</div>
<div class="section" id="references">
<span id="references"></span><h2>References<a class="headerlink" href="#references" title="Permalink to this headline"></a></h2>
<ol class="simple">
<li><a class="reference external" href="https://github.com/01org/mkl-dnn#linking-your-application">MKL small library</a>是<a class="reference external" href="https://software.intel.com/en-us/mkl">Intel MKL</a>的一个子集。
主要包括了深度学习相关的数学原语与操作，一般由MKL-DNN在发布<a class="reference external" href="https://github.com/01org/mkl-dnn/releases">新版本</a>时一起更新。</li>
<li><a class="reference external" href="https://github.com/01org/mkl-dnn#system-requirements">MKL-DNN System Requirements</a>。
目前在PaddlePaddle中，仅会在支持AVX2指令集及以上的机器才使用MKL-DNN。</li>
<li><a class="reference external" href="https://github.com/PaddlePaddle/Paddle/pull/3096">原来的方案</a>会引入<strong>nextLayer</strong>的信息。
但是在PaddlePaddle中，无论是重构前的layer还是重构后的op，都不会想要知道next layer/op的信息。</li>
<li>MKL-DNN的高性能格式与PaddlePaddle原有的<code class="docutils literal"><span class="pre">NCHW</span></code>不同(PaddlePaddle中的cuDNN部分使用的也是<code class="docutils literal"><span class="pre">NCHW</span></code>，所以不存在这个问题)。
所以需要引入一个转换方法，并且只需要在必要的时候转换这种格式，才能更好的发挥MKL-DNN的性能。</li>
</ol>
</div>
</div>


           </div>
          </div>
          <footer>
  

  <hr/>

  <div role="contentinfo">
    <p>
        &copy; Copyright 2016, PaddlePaddle developers.

    </p>
  </div>
  Built with <a href="http://sphinx-doc.org/">Sphinx</a> using a <a href="https://github.com/snide/sphinx_rtd_theme">theme</a> provided by <a href="https://readthedocs.org">Read the Docs</a>. 

</footer>

        </div>
      </div>

    </section>

  </div>
  


  

    <script type="text/javascript">
        var DOCUMENTATION_OPTIONS = {
            URL_ROOT:'../../',
            VERSION:'',
            COLLAPSE_INDEX:false,
            FILE_SUFFIX:'.html',
            HAS_SOURCE:  true,
            SOURCELINK_SUFFIX: ".txt",
        };
    </script>
      <script type="text/javascript" src="../../_static/jquery.js"></script>
      <script type="text/javascript" src="../../_static/underscore.js"></script>
      <script type="text/javascript" src="../../_static/doctools.js"></script>
      <script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.0/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script>
       
  

  
  
    <script type="text/javascript" src="../../_static/js/theme.js"></script>
  
  
  <script src="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/js/bootstrap.min.js" integrity="sha384-Tc5IQib027qvyjSMfHjOMaLkfuWVxZxUPnCJA7l2mCWNIpG9mGCD8wGNIcPD7Txa" crossorigin="anonymous"></script>
  <script src="https://cdn.jsdelivr.net/perfect-scrollbar/0.6.14/js/perfect-scrollbar.jquery.min.js"></script>
  <script src="../../_static/js/paddle_doc_init.js"></script> 

</body>
</html>