01.inference_implementation.html



<!DOCTYPE html>
<!--[if IE 8]><html class="no-js lt-ie9" lang="en" > <![endif]-->
<!--[if gt IE 8]><!--> <html class="no-js" lang="en" > <!--<![endif]-->
<head>
  <meta charset="utf-8">
  
  <meta name="viewport" content="width=device-width, initial-scale=1.0">
  
  <title>C-API 模型推断实现文档 &mdash; PaddlePaddle  文档</title>
  

    <link rel="stylesheet" href="../../_static/css/theme.css" type="text/css" />
  

        <link rel="index" title="索引"
              href="../../genindex.html"/>
        <link rel="search" title="搜索" href="../../search.html"/>
    <link rel="top" title="PaddlePaddle  文档" href="../../index.html"/> 
<script>
var _hmt = _hmt || [];
(function() {
  var hm = document.createElement("script");
  hm.src = "//hm.baidu.com/hm.js?b9a314ab40d04d805655aab1deee08ba";
  var s = document.getElementsByTagName("script")[0]; 
  s.parentNode.insertBefore(hm, s);
})();
</script>


  <script src="../../_static/js/modernizr.min.js"></script>

</head>

<body class="wy-body-for-nav" role="document">

  <div class="wy-grid-for-nav">

    
    <nav data-toggle="wy-nav-shift" class="wy-nav-side">
      <div class="wy-side-scroll">
        <div class="wy-side-nav-search">
          

            <a href="../../index_cn.html" class="icon icon-home"> PaddlePaddle
          

          </a>

          
<div role="search">
  <form id="rtd-search-form" class="wy-form" action="../../search.html" method="get">
    <input type="text" name="q" placeholder="Search docs" />
    <input type="hidden" name="check_keywords" value="yes" />
    <input type="hidden" name="area" value="default" />
  </form>
</div>

          
        </div>

        <div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="main navigation">
          
<nav class="doc-menu-vertical" role="navigation">

<ul>
<li class="toctree-l1"><a class="reference internal" href="../../getstarted/index_cn.html">新手入门</a><ul>
<li class="toctree-l2"><a class="reference internal" href="../../getstarted/quickstart_cn.html">快速开始</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../getstarted/concepts/use_concepts_cn.html">基本使用概念</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="../../build_and_install/index_cn.html">安装与编译</a><ul>
<li class="toctree-l2"><a class="reference internal" href="../../build_and_install/pip_install_cn.html">使用pip安装</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../build_and_install/docker_install_cn.html">使用Docker安装运行</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../build_and_install/build_from_source_cn.html">从源码编译</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="../../howto/index_cn.html">进阶使用</a><ul>
<li class="toctree-l2"><a class="reference internal" href="../../howto/cmd_parameter/index_cn.html">命令行参数设置</a><ul>
<li class="toctree-l3"><a class="reference internal" href="../../howto/cmd_parameter/use_case_cn.html">使用案例</a></li>
<li class="toctree-l3"><a class="reference internal" href="../../howto/cmd_parameter/arguments_cn.html">参数概述</a></li>
<li class="toctree-l3"><a class="reference internal" href="../../howto/cmd_parameter/detail_introduction_cn.html">细节描述</a></li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="../../howto/cluster/index_cn.html">分布式训练</a><ul>
<li class="toctree-l3"><a class="reference internal" href="../../howto/cluster/preparations_cn.html">环境准备</a></li>
<li class="toctree-l3"><a class="reference internal" href="../../howto/cluster/cmd_argument_cn.html">启动参数说明</a></li>
<li class="toctree-l3"><a class="reference internal" href="../../howto/cluster/multi_cluster/index_cn.html">在不同集群中运行</a><ul>
<li class="toctree-l4"><a class="reference internal" href="../../howto/cluster/multi_cluster/k8s_cn.html">Kubernetes单机训练</a></li>
<li class="toctree-l4"><a class="reference internal" href="../../howto/cluster/multi_cluster/k8s_distributed_cn.html">Kubernetes分布式训练</a></li>
<li class="toctree-l4"><a class="reference internal" href="../../howto/cluster/multi_cluster/openmpi_cn.html">在OpenMPI集群中启动训练</a></li>
<li class="toctree-l4"><a class="reference internal" href="../../howto/cluster/multi_cluster/fabric_cn.html">使用fabric启动集群训练</a></li>
<li class="toctree-l4"><a class="reference internal" href="../../howto/cluster/multi_cluster/k8s_aws_cn.html">Kubernetes on AWS</a></li>
</ul>
</li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="../../howto/capi/index_cn.html">C-API预测库</a><ul>
<li class="toctree-l3"><a class="reference internal" href="../../howto/capi/compile_paddle_lib_cn.html">安装与编译C-API预测库</a></li>
<li class="toctree-l3"><a class="reference internal" href="../../howto/capi/organization_of_the_inputs_cn.html">输入/输出数据组织</a></li>
<li class="toctree-l3"><a class="reference internal" href="../../howto/capi/workflow_of_capi_cn.html">C-API使用流程</a></li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="../../howto/rnn/index_cn.html">RNN模型</a><ul>
<li class="toctree-l3"><a class="reference internal" href="../../howto/rnn/rnn_config_cn.html">RNN配置</a></li>
<li class="toctree-l3"><a class="reference internal" href="../../howto/rnn/recurrent_group_cn.html">Recurrent Group教程</a></li>
<li class="toctree-l3"><a class="reference internal" href="../../howto/rnn/hierarchical_layer_cn.html">支持双层序列作为输入的Layer</a></li>
<li class="toctree-l3"><a class="reference internal" href="../../howto/rnn/hrnn_rnn_api_compare_cn.html">单双层RNN API对比介绍</a></li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="../../howto/optimization/gpu_profiling_cn.html">GPU性能调优</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="../../dev/index_cn.html">开发标准</a><ul>
<li class="toctree-l2"><a class="reference internal" href="../../dev/contribute_to_paddle_cn.html">如何贡献代码</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../dev/write_docs_cn.html">如何贡献文档</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="../../faq/index_cn.html">FAQ</a><ul>
<li class="toctree-l2"><a class="reference internal" href="../../faq/build_and_install/index_cn.html">编译安装与单元测试</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../faq/model/index_cn.html">模型配置</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../faq/parameter/index_cn.html">参数设置</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../faq/local/index_cn.html">本地训练与预测</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../faq/cluster/index_cn.html">集群训练与预测</a></li>
</ul>
</li>
</ul>

</nav>

        </div>
      </div>
    </nav>

    <section data-toggle="wy-nav-shift" class="wy-nav-content-wrap">

      
      <nav class="wy-nav-top" role="navigation" aria-label="top navigation">
        <i data-toggle="wy-nav-top" class="fa fa-bars"></i>
        <a href="../../index_cn.html">PaddlePaddle</a>
      </nav>


      <div class="wy-nav-content">
        <div class="rst-content">
          

<div role="navigation" aria-label="breadcrumbs navigation">
  <ul class="wy-breadcrumbs">
    <li><a href="../../index_cn.html">Docs</a> &raquo;</li>
      
    <li>C-API 模型推断实现文档</li>
      <li class="wy-breadcrumbs-aside">
        
          
            <a href="../../_sources/design/multi_language_interface/01.inference_implementation.md.txt" rel="nofollow"> View page source</a>
          
        
      </li>
  </ul>
  <hr/>
</div>
          <div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article">
           <div itemprop="articleBody">
            
  <div class="section" id="c-api">
<span id="c-api"></span><h1>C-API 模型推断实现文档<a class="headerlink" href="#c-api" title="永久链接至标题">¶</a></h1>
<p>本文档描述Paddle C-API的实现细节。Paddle C-API是多语言API的基础部分。Paddle需要暴露的API很多。先实现模型推断的API，通过模型推断API的实现作为一个样例，来进行讨论。至于为什么需要C-API，请参考<a class="reference internal" href="00.why_plain_c.html"><span class="doc">Why Plain C</span></a>。</p>
<div class="section" id="table-of-contents">
<span id="table-of-contents"></span><h2>Table of Contents<a class="headerlink" href="#table-of-contents" title="永久链接至标题">¶</a></h2>
<ul class="simple">
<li><a class="reference external" href="#c-api-模型推断实现文档">C-API 模型推断实现文档</a><ul>
<li><a class="reference external" href="#暴露接口原则">暴露接口原则</a></li>
<li><a class="reference external" href="#目录结构">目录结构</a></li>
<li><a class="reference external" href="#实现方式">实现方式</a><ul>
<li><a class="reference external" href="#capih">capi.h</a></li>
<li><a class="reference external" href="#具体某种类型的头文件">具体某种类型的头文件</a></li>
<li><a class="reference external" href="#capi_privateh">capi_private.h</a></li>
<li><a class="reference external" href="#具体某种类型的实现文件">具体某种类型的实现文件</a></li>
<li><a class="reference external" href="#libpaddle_capi_sharedso-dylib">libpaddle_capi_shared.{so, dylib}</a></li>
<li><a class="reference external" href="#libpaddle_capi_wholea">libpaddle_capi_whole.a</a></li>
<li><a class="reference external" href="#examples">examples</a></li>
</ul>
</li>
<li><a class="reference external" href="#编译选项">编译选项</a></li>
</ul>
</li>
</ul>
</div>
<div class="section" id="">
<span id="id1"></span><h2>暴露接口原则<a class="headerlink" href="#" title="永久链接至标题">¶</a></h2>
<ol class="simple">
<li>所有的接口均为C接口。即使用<code class="docutils literal"><span class="pre">extern</span> <span class="pre">&quot;C&quot;</span></code></li>
<li>除构造某种类型的函数(<code class="docutils literal"><span class="pre">paddle_matrix_create</span></code>等)，其他函数均返回<code class="docutils literal"><span class="pre">paddle_error</span></code>。且调用时不能抛出异常或出现运行时错误。</li>
<li>所有类型名为<code class="docutils literal"><span class="pre">paddle_类型名</span></code>，所有与类型相关的函数，函数名为<code class="docutils literal"><span class="pre">paddle_类型名_函数名</span></code></li>
<li>如果某一个Paddle Core概念(GradientMachine/Matrix)需要被暴露到其他语言，那么<ul>
<li>为了暴露的接口尽量简单。只暴露概念的接口，而不暴露概念的实现。即暴露<code class="docutils literal"><span class="pre">GradientMachine</span></code>或者<code class="docutils literal"><span class="pre">Matrix</span></code>但不暴露<code class="docutils literal"><span class="pre">RecurrentGradientMachine</span></code>和<code class="docutils literal"><span class="pre">CpuSparseMatrix</span></code>。</li>
<li>暴露这个概念必要函数。<code class="docutils literal"><span class="pre">必要</span></code>是指，即完成某一个任务的最少函数。</li>
</ul>
</li>
<li>不在<code class="docutils literal"><span class="pre">capi</span></code>接口层做过多封装。<ul>
<li>如果某一个Paddle概念必须要暴露，但是又过于琐碎。不在<code class="docutils literal"><span class="pre">capi</span></code>这一层进行封装，而是直接修改Paddle Core。让Paddle核心中，这一概念不再琐碎。</li>
</ul>
</li>
</ol>
</div>
<div class="section" id="">
<span id="id2"></span><h2>目录结构<a class="headerlink" href="#" title="永久链接至标题">¶</a></h2>
<div class="highlight-text"><div class="highlight"><pre><span></span>Paddle
  `-- paddle
        `-- capi
              `-- examples  # The example project for C-API.
              `-- tests  # unittests for C-API
              `-- capi.h  # C-API header file.
              `-- capi_private.h  # The shared header file between implementation sources.
              `-- matrix.{h, cpp}
              `-- gradient_machine.{h, cpp}
              `-- ...
</pre></div>
</div>
<p>Paddle的C-API目录结构如上图表所示。这个目录中除了<code class="docutils literal"><span class="pre">capi_private.h</span></code>之外的所有头文件，均会被安装到include/paddle路径下。C-API生成的二进制文件会被安装到<code class="docutils literal"><span class="pre">lib</span></code>目录下。即，安装后的目录结构为</p>
<div class="highlight-text"><div class="highlight"><pre><span></span>`-- include
      `-- paddle
             `-- capi.h
             `-- matrix.h
             `-- gradient_machine.h
             `-- ...
`-- lib
     `-- libpaddle_capi_shared.{so, dylib}  # In mac, dynamic libary&#39;s file name extention is `dylib`
     `-- libpaddle_capi_whole.a  # static library for all symbols of Paddle.
</pre></div>
</div>
</div>
<div class="section" id="">
<span id="id3"></span><h2>实现方式<a class="headerlink" href="#" title="永久链接至标题">¶</a></h2>
<p>下面分别介绍某一类文件的实现方式。</p>
<div class="section" id="capi-h">
<span id="capi-h"></span><h3>capi.h<a class="headerlink" href="#capi-h" title="永久链接至标题">¶</a></h3>
<p><code class="docutils literal"><span class="pre">capi.h</span></code>是用户使用C-API时所唯一需要引入的头文件。在<code class="docutils literal"><span class="pre">capi.h</span></code>中，引入了类型的头文件，<code class="docutils literal"><span class="pre">matrix.h</span></code>, <code class="docutils literal"><span class="pre">gradient_machine.h</span></code>。在引入其他类型的头文件时，使用相对路径的引用方式。即<code class="docutils literal"><span class="pre">#include</span> <span class="pre">&quot;matrix.h&quot;</span></code></p>
</div>
<div class="section" id="">
<span id="id4"></span><h3>具体某种类型的头文件<a class="headerlink" href="#" title="永久链接至标题">¶</a></h3>
<p>具体某种类型的头文件，即例如<code class="docutils literal"><span class="pre">matrix.h</span></code>，<code class="docutils literal"><span class="pre">gradient_machine.h</span></code>等。在这些头文件中，包含了某种类型的类型定义和暴露的全部函数。</p>
<p>这个头文件不假设其他文件的引用顺序，即使用户直接引用某种类型的头文件，也不应该报错(虽然不鼓励这样)。如果某一个类型需要引用另一个类型，例如<code class="docutils literal"><span class="pre">gradient_machine</span></code>需要引用<code class="docutils literal"><span class="pre">matrix</span></code>，则直接引入另一种类型的头文件，即<code class="docutils literal"><span class="pre">#include</span> <span class="pre">&quot;matrix.h&quot;</span></code>。</p>
</div>
<div class="section" id="capi-private-h">
<span id="capi-private-h"></span><h3>capi_private.h<a class="headerlink" href="#capi-private-h" title="永久链接至标题">¶</a></h3>
<p><code class="docutils literal"><span class="pre">capi_prviate.h</span></code>是各个实现中共享的头文件，他主要包含了实际暴露的类型结构。在用户使用C-API时，Paddle的类型全部退化成<code class="docutils literal"><span class="pre">void</span> <span class="pre">*</span></code>，即<code class="docutils literal"><span class="pre">typedef</span> <span class="pre">paddle_matrix</span> <span class="pre">void*</span></code>。但，对于每种C-API暴露的类型，均是在<code class="docutils literal"><span class="pre">capi_private.h</span></code>中实现的结构体。</p>
<div class="highlight-cpp"><div class="highlight"><pre><span></span><span class="k">struct</span> <span class="n">CMatrix</span> <span class="p">{</span>
   <span class="kt">int</span> <span class="n">type</span> <span class="o">=</span> <span class="n">MatrixType</span><span class="p">;</span>
   <span class="n">std</span><span class="o">::</span><span class="n">shared_ptr</span><span class="o">&lt;</span><span class="n">paddle</span><span class="o">::</span><span class="n">Matrix</span><span class="o">&gt;</span> <span class="n">mat</span><span class="p">;</span>
<span class="p">};</span>
</pre></div>
</div>
<p>通常，这个结构体包含两个项目。</p>
<ul>
<li><p class="first"><code class="docutils literal"><span class="pre">type</span></code>是一个类型的标志。对于每种类型，type字段均不尽相同。这样，即使C-API接受的类型全是<code class="docutils literal"><span class="pre">void</span> <span class="pre">*</span></code>，我们也可以确定每一个参数的类型。</p>
<div class="highlight-cpp"><div class="highlight"><pre><span></span><span class="kt">void</span> <span class="nf">some_c_api_function</span><span class="p">(</span><span class="kt">void</span><span class="o">*</span> <span class="n">some_instance</span><span class="p">)</span> <span class="p">{</span>
   <span class="kt">int</span><span class="o">*</span> <span class="n">type</span> <span class="o">=</span> <span class="p">(</span><span class="kt">int</span> <span class="o">*</span><span class="p">)</span> <span class="n">some_instance</span><span class="p">;</span>
   <span class="k">switch</span> <span class="p">(</span><span class="o">*</span><span class="n">type</span><span class="p">)</span> <span class="p">{</span>
     <span class="k">case</span> <span class="nl">MatrixType</span><span class="p">:</span>
       <span class="n">CMatrix</span><span class="o">*</span> <span class="n">mat</span> <span class="o">=</span> <span class="p">(</span><span class="n">CMatrix</span> <span class="o">*</span><span class="p">)</span> <span class="n">some_instance</span><span class="p">;</span>
       <span class="p">...</span>
     <span class="p">...</span>
   <span class="p">}</span>
<span class="p">}</span>
</pre></div>
</div>
</li>
<li><p class="first">这个结构体中的另一个项目是，Paddle Core中这一类型接口的智能指针(shared_ptr)。</p>
<ul class="simple">
<li>使用智能指针的原因是: 用户可以安全的释放某个C-API的实例，而不必在意Paddle Core是否还在使用这个实例。</li>
<li>例如，用户通过C-API获得了神经网络的参数实例。当用户使用完这个参数后，直接删除这个参数即可。即便Paddle Core中的模型还在使用这个参数，这个参数也不会一并删除。</li>
</ul>
</li>
</ul>
</div>
<div class="section" id="">
<span id="id5"></span><h3>具体某种类型的实现文件<a class="headerlink" href="#" title="永久链接至标题">¶</a></h3>
<p>具体某种类型的实现文件，即<code class="docutils literal"><span class="pre">matrix.cpp</span></code>, <code class="docutils literal"><span class="pre">gradient_machine.cpp</span></code>等文件。在这些文件中，使用C++ 11实现了C-API的接口，并且使用<code class="docutils literal"><span class="pre">extern</span> <span class="pre">&quot;C&quot;</span></code>导出这些接口。在实现过程中，对输入参数的安全性进行了必要的判断，并将C-API接口的参数转发给<code class="docutils literal"><span class="pre">Paddle</span> <span class="pre">Core</span></code>。</p>
</div>
<div class="section" id="libpaddle-capi-shared-so-dylib">
<span id="libpaddle-capi-shared-so-dylib"></span><h3>libpaddle_capi_shared.{so, dylib}<a class="headerlink" href="#libpaddle-capi-shared-so-dylib" title="永久链接至标题">¶</a></h3>
<p><code class="docutils literal"><span class="pre">libpaddle_capi_shared</span></code>是C-API导出的动态库。这个动态库的连接参数与Paddle的其他二进制(例如<code class="docutils literal"><span class="pre">paddle_trainer</span></code>)类似。用户可以直接使用这个动态库来引入Paddle C-API。具体使用方法为<code class="docutils literal"><span class="pre">-lpaddle_capi_shared</span></code>。</p>
</div>
<div class="section" id="libpaddle-capi-whole-a">
<span id="libpaddle-capi-whole-a"></span><h3>libpaddle_capi_whole.a<a class="headerlink" href="#libpaddle-capi-whole-a" title="永久链接至标题">¶</a></h3>
<p><code class="docutils literal"><span class="pre">libpaddle_capi_whole</span></code>是C-API导出的静态库。这个静态库包含了Paddle的全部符号。他是将<code class="docutils literal"><span class="pre">libpaddle_gserver.a</span></code>, <code class="docutils literal"><span class="pre">libpaddle_math.a</span></code>, <code class="docutils literal"><span class="pre">libpaddle_capi.a</span></code>等全部静态库中的目标文件全部打包后产生的文件。具体使用方法为<code class="docutils literal"><span class="pre">--whole-archive</span> <span class="pre">-lpaddle_capi_whole</span> <span class="pre">--no-whole-archive</span></code>。</p>
</div>
<div class="section" id="examples">
<span id="examples"></span><h3>examples<a class="headerlink" href="#examples" title="永久链接至标题">¶</a></h3>
<p>在样例中，使用<code class="docutils literal"><span class="pre">C99</span></code>开发了模型预测的样例代码。具体请参考<a class="reference external" href="../paddle/capi/examples/README.md">example/README.md</a>。</p>
</div>
</div>
<div class="section" id="">
<span id="id6"></span><h2>编译选项<a class="headerlink" href="#" title="永久链接至标题">¶</a></h2>
<p>C-API的编译选项默认关闭，打开这个编译选项，需要在cmake的时候，设置</p>
<div class="highlight-bash"><div class="highlight"><pre><span></span>cmake <span class="si">${</span><span class="nv">YOUR_SOURCE_ROOT</span><span class="si">}</span> -DWITH_C_API<span class="o">=</span>ON -DWITH_PYTHON<span class="o">=</span>OFF -DWITH_SWIG_PY<span class="o">=</span>OFF
</pre></div>
</div>
<p>编译C-API的时候推荐Paddle不嵌入Python解释器，也不生成<code class="docutils literal"><span class="pre">SWIG</span></code>接口，具体原因参考<a class="reference internal" href="00.why_plain_c.html"><span class="doc">Why Plain C</span></a>。</p>
</div>
</div>


           </div>
          </div>
          <footer>
  

  <hr/>

  <div role="contentinfo">
    <p>
        &copy; Copyright 2016, PaddlePaddle developers.

    </p>
  </div>
  Built with <a href="http://sphinx-doc.org/">Sphinx</a> using a <a href="https://github.com/snide/sphinx_rtd_theme">theme</a> provided by <a href="https://readthedocs.org">Read the Docs</a>. 

</footer>

        </div>
      </div>

    </section>

  </div>
  

    <script type="text/javascript">
        var DOCUMENTATION_OPTIONS = {
            URL_ROOT:'../../',
            VERSION:'',
            COLLAPSE_INDEX:false,
            FILE_SUFFIX:'.html',
            HAS_SOURCE:  true
        };
    </script>
      <script type="text/javascript" src="../../_static/jquery.js"></script>
      <script type="text/javascript" src="../../_static/underscore.js"></script>
      <script type="text/javascript" src="../../_static/doctools.js"></script>
      <script type="text/javascript" src="../../_static/translations.js"></script>
      <script type="text/javascript" src="https://cdn.bootcss.com/mathjax/2.7.0/MathJax.js"></script>

  
    <script type="text/javascript" src="../../_static/js/theme.js"></script>
  

  <script type="text/javascript">
      jQuery(function () {
          SphinxRtdTheme.StickyNav.enable();
      });
  </script>
   

</body>
</html>