<p>In this <codeclass="code docutils literal"><spanclass="pre">trainer_config.py</span></code>, we just map each feature type to
a feature vector, following shows how to map each feature to a vector shows below.</p>
<ul>
<li><pclass="first"><codeclass="code docutils literal"><spanclass="pre">id</span></code>: Just simple embedding, and then add to fully connected layer.</p>
</li>
<ulclass="simple">
<li><codeclass="code docutils literal"><spanclass="pre">id</span></code>: Just simple embedding, and then add to fully connected layer.</li>
@@ -301,10 +301,10 @@ Fully connected layer takes a dense input vector with dimension <span class="mat
<h1>Write Gradient Check Unit Test<aclass="headerlink"href="#write-gradient-check-unit-test"title="Permalink to this headline">¶</a></h1>
<p>An easy way to verify the correctness of new layer’s implementation is to write a gradient check unit test. Gradient check unit test utilizes finite difference method to verify the gradient of a layer. It modifies the input with a small perturbation <spanclass="math">\(\Delta x\)</span> and observes the changes of output <spanclass="math">\(\Delta y\)</span>, the gradient can be computed as <spanclass="math">\(\frac{\Delta y}{\Delta x }\)</span>. This gradient can be compared with the gradient computed by the <codeclass="code docutils literal"><spanclass="pre">backward</span></code> function of the layer to ensure the correctness of the gradient computation. Notice that the gradient check only tests the correctness of the gradient computation, it does not necessarily guarantee the correctness of the implementation of the <codeclass="code docutils literal"><spanclass="pre">forward</span></code> and <codeclass="code docutils literal"><spanclass="pre">backward</span></code> function. You need to write more sophisticated unit tests to make sure your layer is implemented correctly.</p>
<p>All the gradient check unit tests are located in <codeclass="code docutils literal"><spanclass="pre">paddle/gserver/tests/test_LayerGrad.cpp</span></code>. You are recommended to put your test into a new test file if you are planning to write a new layer. The gradient test of the gradient check unit test of the fully connected layer is listed below. It has the following steps.</p>
<ul>
<ulclass="simple">
<li><dlclass="first docutils">
<dt>Create layer configuration. A layer configuration can include the following attributes:</dt>
<dd><ulclass="first last simple">
<dd><ulclass="first last">
<li>size of the bias parameter. (4096 in our example)</li>
<li>type of the layer. (fc in our example)</li>
<li>size of the layer. (4096 in our example)</li>
...
...
@@ -319,7 +319,7 @@ Fully connected layer takes a dense input vector with dimension <span class="mat
<dd><ulclass="first last">
<li><dlclass="first docutils">
<dt>type of the input (<codeclass="code docutils literal"><spanclass="pre">INPUT_DATA</span></code>) in our example. It can be one of the following types</dt>
<li><codeclass="code docutils literal"><spanclass="pre">INPUT_DATA_TARGET</span></code>: dense vector, but it does not used to compute gradient.</li>
...
...
@@ -332,23 +332,18 @@ Fully connected layer takes a dense input vector with dimension <span class="mat
</dd>
</dl>
</li>
<li><pclass="first">name of the input. (<codeclass="code docutils literal"><spanclass="pre">layer_0</span></code> in our example)</p>
</li>
<li><pclass="first">size of the input. (8192 in our example)</p>
</li>
<li><pclass="first">number of non-zeros, only useful for sparse inputs.</p>
</li>
<li><pclass="first">format of sparse data, only useful for sparse inputs.</p>
</li>
<li>name of the input. (<codeclass="code docutils literal"><spanclass="pre">layer_0</span></code> in our example)</li>
<li>size of the input. (8192 in our example)</li>
<li>number of non-zeros, only useful for sparse inputs.</li>
<li>format of sparse data, only useful for sparse inputs.</li>
</ul>
</dd>
</dl>
</li>
<li><pclass="first">each inputs needs to call <codeclass="code docutils literal"><spanclass="pre">config.layerConfig.add_inputs();</span></code> once.</p>
</li>
<li>each inputs needs to call <codeclass="code docutils literal"><spanclass="pre">config.layerConfig.add_inputs();</span></code> once.</li>
<li><dlclass="first docutils">
<dt>call <codeclass="code docutils literal"><spanclass="pre">testLayerGrad</span></code> to perform gradient checks. It has the following arguments.</dt>
<dd><ulclass="first last simple">
<dd><ulclass="first last">
<li>layer and input configurations. (<codeclass="code docutils literal"><spanclass="pre">config</span></code> in our example)</li>
<li>type of the input. (<codeclass="code docutils literal"><spanclass="pre">fc</span></code> in our example)</li>
<li>batch size of the gradient check. (100 in our example)</li>
<h1>Implement Python Wrapper<aclass="headerlink"href="#implement-python-wrapper"title="Permalink to this headline">¶</a></h1>
<p>Implementing Python wrapper allows us to use the added layer in configuration files. All the Python wrappers are in file <codeclass="code docutils literal"><spanclass="pre">python/paddle/trainer/config_parser.py</span></code>. An example of the Python wrapper for fully connected layer is listed below. It has the following steps:</p>
<ul>
<li><pclass="first">Use <codeclass="code docutils literal"><spanclass="pre">@config_layer('fc')</span></code> at the decorator for all the Python wrapper class. <codeclass="code docutils literal"><spanclass="pre">fc</span></code> is the identifier of the layer.</p>
</li>
<ulclass="simple">
<li>Use <codeclass="code docutils literal"><spanclass="pre">@config_layer('fc')</span></code> at the decorator for all the Python wrapper class. <codeclass="code docutils literal"><spanclass="pre">fc</span></code> is the identifier of the layer.</li>
<li>It first call <codeclass="code docutils literal"><spanclass="pre">super(FCLayer,</span><spanclass="pre">self).__init__(name,</span><spanclass="pre">'fc',</span><spanclass="pre">size,</span><spanclass="pre">inputs=inputs,</span><spanclass="pre">**xargs)</span></code> base constructor function. <codeclass="code docutils literal"><spanclass="pre">FCLayer</span></code> is the Python wrapper class name, and <codeclass="code docutils literal"><spanclass="pre">fc</span></code> is the layer identifier name. They must be correct in order for the wrapper to work.</li>
<li>Then it computes the size and format (whether sparse) of each transformation matrix as well as the size.</li>
<spanid="FloatArray"></span><spanclass="target"id="paddlestructFloatArray"></span><emclass="property">struct </em><codeclass="descname">FloatArray</code><aclass="headerlink"href="#_CPPv210FloatArray"title="Permalink to this definition">¶</a></dt>
<dd><em>#include <PaddleAPI.h></em><p>This type will map to python’s list of float. </p>
<spanid="IntArray"></span><spanclass="target"id="paddlestructIntArray"></span><emclass="property">struct </em><codeclass="descname">IntArray</code><aclass="headerlink"href="#_CPPv28IntArray"title="Permalink to this definition">¶</a></dt>
<dd><em>#include <PaddleAPI.h></em><p>This type will map to python’s list of int. </p>
<spanid="IntWithFloatArray"></span><spanclass="target"id="paddlestructIntWithFloatArray"></span><emclass="property">struct </em><codeclass="descname">IntWithFloatArray</code><aclass="headerlink"href="#_CPPv217IntWithFloatArray"title="Permalink to this definition">¶</a></dt>
<dd><em>#include <PaddleAPI.h></em><p>This type will map to python’s list of (int, float) </p>
<spanid="Matrix"></span><spanclass="target"id="paddleclassMatrix"></span><emclass="property">class </em><codeclass="descname">Matrix</code><aclass="headerlink"href="#_CPPv26Matrix"title="Permalink to this definition">¶</a></dt>
<spanid="Vector"></span><spanclass="target"id="paddleclassVector"></span><emclass="property">class </em><codeclass="descname">Vector</code><aclass="headerlink"href="#_CPPv26Vector"title="Permalink to this definition">¶</a></dt>
<spanid="IVector"></span><spanclass="target"id="paddleclassIVector"></span><emclass="property">class </em><codeclass="descname">IVector</code><aclass="headerlink"href="#_CPPv27IVector"title="Permalink to this definition">¶</a></dt>
<spanid="Arguments"></span><spanclass="target"id="paddleclassArguments"></span><emclass="property">class </em><codeclass="descname">Arguments</code><aclass="headerlink"href="#_CPPv29Arguments"title="Permalink to this definition">¶</a></dt>
<dd><em>#include <PaddleAPI.h></em><p>The <aclass="reference internal"href="#paddleclassArguments"><spanclass="std std-ref">Arguments</span></a> is actual a std::vector<paddle::Argument> in paddle. </p>
<spanid="ParameterConfig"></span><spanclass="target"id="paddleclassParameterConfig"></span><emclass="property">class </em><codeclass="descname">ParameterConfig</code><aclass="headerlink"href="#_CPPv215ParameterConfig"title="Permalink to this definition">¶</a></dt>
<spanid="OptimizationConfig"></span><spanclass="target"id="paddleclassOptimizationConfig"></span><emclass="property">class </em><codeclass="descname">OptimizationConfig</code><aclass="headerlink"href="#_CPPv218OptimizationConfig"title="Permalink to this definition">¶</a></dt>
<spanid="Parameter"></span><spanclass="target"id="paddleclassParameter"></span><emclass="property">class </em><codeclass="descname">Parameter</code><aclass="headerlink"href="#_CPPv29Parameter"title="Permalink to this definition">¶</a></dt>
<spanid="ModelConfig"></span><spanclass="target"id="paddleclassModelConfig"></span><emclass="property">class </em><codeclass="descname">ModelConfig</code><aclass="headerlink"href="#_CPPv211ModelConfig"title="Permalink to this definition">¶</a></dt>
<dd><em>#include <PaddleAPI.h></em><p>You can only get model config from <aclass="reference internal"href="#paddleclassTrainerConfig"><spanclass="std std-ref">TrainerConfig</span></a>.</p>
<p>It is used by <aclass="reference internal"href="#paddleclassGradientMachine"><spanclass="std std-ref">GradientMachine</span></a>. </p>
<spanid="TrainerConfig"></span><spanclass="target"id="paddleclassTrainerConfig"></span><emclass="property">class </em><codeclass="descname">TrainerConfig</code><aclass="headerlink"href="#_CPPv213TrainerConfig"title="Permalink to this definition">¶</a></dt>
<dd><em>#include <PaddleAPI.h></em><p>To get <aclass="reference internal"href="#paddleclassTrainerConfig"><spanclass="std std-ref">TrainerConfig</span></a> from file.</p>
<p>It is used by <aclass="reference internal"href="#paddleclassGradientMachine"><spanclass="std std-ref">GradientMachine</span></a>. </p>
<spanid="ParameterTraverseCallback"></span><spanclass="target"id="paddleclassParameterTraverseCallback"></span><emclass="property">class </em><codeclass="descname">ParameterTraverseCallback</code><aclass="headerlink"href="#_CPPv225ParameterTraverseCallback"title="Permalink to this definition">¶</a></dt>
<spanid="ParameterOptimizer"></span><spanclass="target"id="paddleclassParameterOptimizer"></span><emclass="property">class </em><codeclass="descname">ParameterOptimizer</code><aclass="headerlink"href="#_CPPv218ParameterOptimizer"title="Permalink to this definition">¶</a></dt>
<spanid="GradientMachine"></span><spanclass="target"id="paddleclassGradientMachine"></span><emclass="property">class </em><codeclass="descname">GradientMachine</code><aclass="headerlink"href="#_CPPv215GradientMachine"title="Permalink to this definition">¶</a></dt>
<spanid="Trainer"></span><spanclass="target"id="paddleclassTrainer"></span><emclass="property">class </em><codeclass="descname">Trainer</code><aclass="headerlink"href="#_CPPv27Trainer"title="Permalink to this definition">¶</a></dt>
<spanid="ISequenceResults"></span><spanclass="target"id="paddleclassISequenceResults"></span><emclass="property">class </em><codeclass="descname">ISequenceResults</code><aclass="headerlink"href="#_CPPv216ISequenceResults"title="Permalink to this definition">¶</a></dt>
<dd><em>#include <PaddleAPI.h></em><p>the N-Best results generated from one input sequence. </p>
<spanid="SequenceGenerator"></span><spanclass="target"id="paddleclassSequenceGenerator"></span><emclass="property">class </em><codeclass="descname">SequenceGenerator</code><aclass="headerlink"href="#_CPPv217SequenceGenerator"title="Permalink to this definition">¶</a></dt>
<spanid="_thread_device_resources"></span><spanclass="target"id="paddlestruct__thread__device__resources"></span><emclass="property">struct </em><codeclass="descname">_thread_device_resources</code><aclass="headerlink"href="#_CPPv224_thread_device_resources"title="Permalink to this definition">¶</a></dt>
<spanid="_hl_device_prop"></span><spanclass="target"id="paddlestruct__hl__device__prop"></span><emclass="property">struct </em><codeclass="descname">_hl_device_prop</code><aclass="headerlink"href="#_CPPv215_hl_device_prop"title="Permalink to this definition">¶</a></dt>
<spanid="_cudnn_tensor_descriptor"></span><spanclass="target"id="paddlestruct__cudnn__tensor__descriptor"></span><emclass="property">struct </em><codeclass="descname">_cudnn_tensor_descriptor</code><aclass="headerlink"href="#_CPPv224_cudnn_tensor_descriptor"title="Permalink to this definition">¶</a></dt>
<spanid="_cudnn_pooling_descriptor"></span><spanclass="target"id="paddlestruct__cudnn__pooling__descriptor"></span><emclass="property">struct </em><codeclass="descname">_cudnn_pooling_descriptor</code><aclass="headerlink"href="#_CPPv225_cudnn_pooling_descriptor"title="Permalink to this definition">¶</a></dt>
<spanid="_cudnn_filter_descriptor"></span><spanclass="target"id="paddlestruct__cudnn__filter__descriptor"></span><emclass="property">struct </em><codeclass="descname">_cudnn_filter_descriptor</code><aclass="headerlink"href="#_CPPv224_cudnn_filter_descriptor"title="Permalink to this definition">¶</a></dt>
<spanid="_cudnn_convolution_descriptor"></span><spanclass="target"id="paddlestruct__cudnn__convolution__descriptor"></span><emclass="property">struct </em><codeclass="descname">_cudnn_convolution_descriptor</code><aclass="headerlink"href="#_CPPv229_cudnn_convolution_descriptor"title="Permalink to this definition">¶</a></dt>
<spanid="BaseOp"></span><spanclass="target"id="paddleclassBaseOp"></span><emclass="property">class </em><codeclass="descname">BaseOp</code><aclass="headerlink"href="#_CPPv26BaseOp"title="Permalink to this definition">¶</a></dt>
<spanid="aggregate::sum"></span><spanclass="target"id="paddleclassaggregate_1_1sum"></span><emclass="property">class </em><codeclass="descname">sum</code><aclass="headerlink"href="#_CPPv2N9aggregate3sumE"title="Permalink to this definition">¶</a></dt>
<spanid="aggregate::max"></span><spanclass="target"id="paddleclassaggregate_1_1max"></span><emclass="property">class </em><codeclass="descname">max</code><aclass="headerlink"href="#_CPPv2N9aggregate3maxE"title="Permalink to this definition">¶</a></dt>
<spanid="aggregate::min"></span><spanclass="target"id="paddleclassaggregate_1_1min"></span><emclass="property">class </em><codeclass="descname">min</code><aclass="headerlink"href="#_CPPv2N9aggregate3minE"title="Permalink to this definition">¶</a></dt>
<spanid="base::binary::add"></span><spanclass="target"id="paddleclassbase_1_1binary_1_1add"></span><emclass="property">class </em><codeclass="descname">add</code><aclass="headerlink"href="#_CPPv2N4base6binary3addE"title="Permalink to this definition">¶</a></dt>
<spanid="base::binary::add2"></span><spanclass="target"id="paddleclassbase_1_1binary_1_1add2"></span><emclass="property">class </em><codeclass="descname">add2</code><aclass="headerlink"href="#_CPPv2N4base6binary4add2E"title="Permalink to this definition">¶</a></dt>
<spanid="base::binary::sub"></span><spanclass="target"id="paddleclassbase_1_1binary_1_1sub"></span><emclass="property">class </em><codeclass="descname">sub</code><aclass="headerlink"href="#_CPPv2N4base6binary3subE"title="Permalink to this definition">¶</a></dt>
<spanid="base::binary::mul"></span><spanclass="target"id="paddleclassbase_1_1binary_1_1mul"></span><emclass="property">class </em><codeclass="descname">mul</code><aclass="headerlink"href="#_CPPv2N4base6binary3mulE"title="Permalink to this definition">¶</a></dt>
<spanid="base::binary::div"></span><spanclass="target"id="paddleclassbase_1_1binary_1_1div"></span><emclass="property">class </em><codeclass="descname">div</code><aclass="headerlink"href="#_CPPv2N4base6binary3divE"title="Permalink to this definition">¶</a></dt>
<spanid="base::binary::squaredDiff"></span><spanclass="target"id="paddleclassbase_1_1binary_1_1squaredDiff"></span><emclass="property">class </em><codeclass="descname">squaredDiff</code><aclass="headerlink"href="#_CPPv2N4base6binary11squaredDiffE"title="Permalink to this definition">¶</a></dt>
<dd><p>Inherits from base::binary::SSESquaredDiff</p>
<spanid="base::binary::first"></span><spanclass="target"id="paddleclassbase_1_1binary_1_1first"></span><emclass="property">class </em><codeclass="descname">first</code><aclass="headerlink"href="#_CPPv2N4base6binary5firstE"title="Permalink to this definition">¶</a></dt>
<spanid="base::binary::second"></span><spanclass="target"id="paddleclassbase_1_1binary_1_1second"></span><emclass="property">class </em><codeclass="descname">second</code><aclass="headerlink"href="#_CPPv2N4base6binary6secondE"title="Permalink to this definition">¶</a></dt>
<spanid="base::binary::classificationError"></span><spanclass="target"id="paddleclassbase_1_1binary_1_1classificationError"></span><emclass="property">class </em><codeclass="descname">classificationError</code><aclass="headerlink"href="#_CPPv2N4base6binary19classificationErrorE"title="Permalink to this definition">¶</a></dt>
<dd><p>Inherits from base::binary::SSEClassificationError</p>
<spanid="base::unary::identity"></span><spanclass="target"id="paddleclassbase_1_1unary_1_1identity"></span><emclass="property">class </em><codeclass="descname">identity</code><aclass="headerlink"href="#_CPPv2N4base5unary8identityE"title="Permalink to this definition">¶</a></dt>
<spanid="hppl"></span><spanclass="target"id="paddlenamespacehppl"></span><emclass="property">namespace </em><codeclass="descname">hppl</code><aclass="headerlink"href="#_CPPv24hppl"title="Permalink to this definition">¶</a></dt>
<spanclass="target"id="paddlenamespacehppl"></span><emclass="property">namespace </em><codeclass="descname">hppl</code><aclass="headerlink"href="#_CPPv24hppl"title="Permalink to this definition">¶</a></dt>
<spanid="paddle"></span><spanclass="target"id="paddlenamespacepaddle"></span><emclass="property">namespace </em><codeclass="descname">paddle</code><aclass="headerlink"href="#_CPPv26paddle"title="Permalink to this definition">¶</a></dt>
<spanid="hppl::Active"></span><emclass="property">class </em><codeclass="descname">Active</code><aclass="headerlink"href="#_CPPv2N4hppl6ActiveE"title="Permalink to this definition">¶</a></dt>
<dd><em>#include <hl_activation_functions.h></em><p>Hppl supports sigmoid, relu, tanh, linear active functions for neural networks’ forward and backward activation. </p>
<spanid="hppl::cpu"></span><spanclass="target"id="paddlenamespacehppl_1_1cpu"></span><emclass="property">namespace </em><codeclass="descname">cpu</code><aclass="headerlink"href="#_CPPv2N4hppl3cpuE"title="Permalink to this definition">¶</a></dt>
<spanid="hppl::backward::lstm"></span><spanclass="target"id="paddleclasshppl_1_1backward_1_1lstm"></span><emclass="property">class </em><codeclass="descname">lstm</code><aclass="headerlink"href="#_CPPv2N4hppl8backward4lstmE"title="Permalink to this definition">¶</a></dt>
<spanid="hppl::forward::lstm"></span><spanclass="target"id="paddleclasshppl_1_1forward_1_1lstm"></span><emclass="property">class </em><codeclass="descname">lstm</code><aclass="headerlink"href="#_CPPv2N4hppl7forward4lstmE"title="Permalink to this definition">¶</a></dt>
<spanid="hppl::backward::gru_stateGrad"></span><spanclass="target"id="paddleclasshppl_1_1backward_1_1gru__stateGrad"></span><emclass="property">class </em><codeclass="descname">gru_stateGrad</code><aclass="headerlink"href="#_CPPv2N4hppl8backward13gru_stateGradE"title="Permalink to this definition">¶</a></dt>
<spanid="hppl::backward::gru_resetGrad"></span><spanclass="target"id="paddleclasshppl_1_1backward_1_1gru__resetGrad"></span><emclass="property">class </em><codeclass="descname">gru_resetGrad</code><aclass="headerlink"href="#_CPPv2N4hppl8backward13gru_resetGradE"title="Permalink to this definition">¶</a></dt>
<spanid="hppl::forward::gru_resetOutput"></span><spanclass="target"id="paddleclasshppl_1_1forward_1_1gru__resetOutput"></span><emclass="property">class </em><codeclass="descname">gru_resetOutput</code><aclass="headerlink"href="#_CPPv2N4hppl7forward15gru_resetOutputE"title="Permalink to this definition">¶</a></dt>
<spanid="hppl::forward::gru_finalOutput"></span><spanclass="target"id="paddleclasshppl_1_1forward_1_1gru__finalOutput"></span><emclass="property">class </em><codeclass="descname">gru_finalOutput</code><aclass="headerlink"href="#_CPPv2N4hppl7forward15gru_finalOutputE"title="Permalink to this definition">¶</a></dt>
<spanid="paddle::DataProviderGroup"></span><emclass="property">class </em><codeclass="descclassname">paddle::</code><codeclass="descname">DataProviderGroup</code><aclass="headerlink"href="#_CPPv2N6paddle17DataProviderGroupE"title="Permalink to this definition">¶</a></dt>
<dd><p>Inherits from <aclass="reference internal"href="#paddleclasspaddle_1_1DataProvider"><spanclass="std std-ref">paddle::DataProvider</span></a></p>
<spanid="paddle::MultiDataProvider"></span><spanclass="target"id="paddleclasspaddle_1_1MultiDataProvider"></span><emclass="property">class </em><codeclass="descclassname">paddle::</code><codeclass="descname">MultiDataProvider</code><aclass="headerlink"href="#_CPPv2N6paddle17MultiDataProviderE"title="Permalink to this definition">¶</a></dt>
<dd><p>Inherits from <aclass="reference internal"href="#paddleclasspaddle_1_1DataProvider"><spanclass="std std-ref">paddle::DataProvider</span></a></p>
<p>It will read python object, and fill to argument’s each slot. There are two steps, prepare and fill. Scanner will alloc memory during prepare step, fill data into argument during fill step. </p>
<spanid="paddle::DenseScanner"></span><spanclass="target"id="paddleclasspaddle_1_1DenseScanner"></span><emclass="property">class </em><codeclass="descclassname">paddle::</code><codeclass="descname">DenseScanner</code><aclass="headerlink"href="#_CPPv2N6paddle12DenseScannerE"title="Permalink to this definition">¶</a></dt>
<dd><p>Scanner for dense slot. </p>
<p>Inherits from <aclass="reference internal"href="#paddleclasspaddle_1_1IFieldScanner"><spanclass="std std-ref">paddle::IFieldScanner</span></a></p>
<spanid="paddle::IndexScanner"></span><spanclass="target"id="paddleclasspaddle_1_1IndexScanner"></span><emclass="property">class </em><codeclass="descclassname">paddle::</code><codeclass="descname">IndexScanner</code><aclass="headerlink"href="#_CPPv2N6paddle12IndexScannerE"title="Permalink to this definition">¶</a></dt>
<dd><p>Scanner for index slot </p>
<p>Inherits from <aclass="reference internal"href="#paddleclasspaddle_1_1IFieldScanner"><spanclass="std std-ref">paddle::IFieldScanner</span></a></p>
<spanid="paddle::SparseNonValueScanner"></span><spanclass="target"id="paddleclasspaddle_1_1SparseNonValueScanner"></span><emclass="property">class </em><codeclass="descclassname">paddle::</code><codeclass="descname">SparseNonValueScanner</code><aclass="headerlink"href="#_CPPv2N6paddle21SparseNonValueScannerE"title="Permalink to this definition">¶</a></dt>
<dd><p>Inherits from <aclass="reference internal"href="#paddleclasspaddle_1_1IFieldScanner"><spanclass="std std-ref">paddle::IFieldScanner</span></a></p>
<spanid="paddle::SparseValueScanner"></span><spanclass="target"id="paddleclasspaddle_1_1SparseValueScanner"></span><emclass="property">class </em><codeclass="descclassname">paddle::</code><codeclass="descname">SparseValueScanner</code><aclass="headerlink"href="#_CPPv2N6paddle18SparseValueScannerE"title="Permalink to this definition">¶</a></dt>
<dd><p>Inherits from <aclass="reference internal"href="#paddleclasspaddle_1_1SparseNonValueScanner"><spanclass="std std-ref">paddle::SparseNonValueScanner</span></a></p>
<spanclass="target"id="paddleclasspaddle_1_1SparseValueScanner"></span><emclass="property">class </em><codeclass="descclassname">paddle::</code><codeclass="descname">SparseValueScanner</code><aclass="headerlink"href="#_CPPv2N6paddle18SparseValueScannerE"title="Permalink to this definition">¶</a></dt>
<dd><p>Inherits from <aclass="reference internal"href="#paddleclasspaddle_1_1SparseNonValueScanner"><spanclass="std std-ref">paddle::SparseNonValueScanner</span></a></p>
<spanid="paddle::IPyDataProviderCache"></span><spanclass="target"id="paddleclasspaddle_1_1IPyDataProviderCache"></span><emclass="property">class </em><codeclass="descclassname">paddle::</code><codeclass="descname">IPyDataProviderCache</code><aclass="headerlink"href="#_CPPv2N6paddle20IPyDataProviderCacheE"title="Permalink to this definition">¶</a></dt>
<dd><p>Py Data Provider Cache Interface. </p>
<p>Subclassed by <aclass="reference internal"href="#paddleclasspaddle_1_1CacheOnePassInMemory"><spanclass="std std-ref">paddle::CacheOnePassInMemory</span></a>, <aclass="reference internal"href="#paddleclasspaddle_1_1NoCacheStrategy"><spanclass="std std-ref">paddle::NoCacheStrategy</span></a></p>
<spanid="paddle::NoCacheStrategy"></span><spanclass="target"id="paddleclasspaddle_1_1NoCacheStrategy"></span><emclass="property">class </em><codeclass="descclassname">paddle::</code><codeclass="descname">NoCacheStrategy</code><aclass="headerlink"href="#_CPPv2N6paddle15NoCacheStrategyE"title="Permalink to this definition">¶</a></dt>
<dd><p>No Cache Strategy. Will destruct old data immediately and load data from python every pass. </p>
<p>Inherits from <aclass="reference internal"href="#paddleclasspaddle_1_1IPyDataProviderCache"><spanclass="std std-ref">paddle::IPyDataProviderCache</span></a></p>
<p>In first pass, will load data from python and store them in memory. The rest passes, will load data from memory. </p>
<p>Inherits from <aclass="reference internal"href="#paddleclasspaddle_1_1IPyDataProviderCache"><spanclass="std std-ref">paddle::IPyDataProviderCache</span></a></p>
<p>Inherits from <aclass="reference internal"href="#paddleclasspaddle_1_1DataProvider"><spanclass="std std-ref">paddle::DataProvider</span></a></p>
<p>Subclassed by <aclass="reference internal"href="#paddleclasspaddle_1_1ProtoSequenceDataProvider"><spanclass="std std-ref">paddle::ProtoSequenceDataProvider</span></a></p>
<spanid="paddle::ProtoDataProvider::ProtoSlot"></span><spanclass="target"id="paddlestructpaddle_1_1ProtoDataProvider_1_1ProtoSlot"></span><emclass="property">struct </em><codeclass="descname">ProtoSlot</code><aclass="headerlink"href="#_CPPv2N6paddle17ProtoDataProvider9ProtoSlotE"title="Permalink to this definition">¶</a></dt>
<spanid="paddle::ProtoDataProvider::ProtoVarSlot"></span><spanclass="target"id="paddlestructpaddle_1_1ProtoDataProvider_1_1ProtoVarSlot"></span><emclass="property">struct </em><codeclass="descname">ProtoVarSlot</code><aclass="headerlink"href="#_CPPv2N6paddle17ProtoDataProvider12ProtoVarSlotE"title="Permalink to this definition">¶</a></dt>
<p>Inherits from <aclass="reference internal"href="#paddleclasspaddle_1_1ProtoDataProvider"><spanclass="std std-ref">paddle::ProtoDataProvider</span></a></p>
<spanid="paddle::Evaluator"></span><spanclass="target"id="paddleclasspaddle_1_1Evaluator"></span><emclass="property">class </em><codeclass="descclassname">paddle::</code><codeclass="descname">Evaluator</code><aclass="headerlink"href="#_CPPv2N6paddle9EvaluatorE"title="Permalink to this definition">¶</a></dt>
<dd><p>Base class for <aclass="reference internal"href="#paddleclasspaddle_1_1Evaluator"><spanclass="std std-ref">Evaluator</span></a> Evaluating the performance of a model is very important. It indicates how successful the scores(predictions) of a datasets has been by a trained model. </p>
<dd><p>sum <aclass="reference internal"href="#paddleclasspaddle_1_1Evaluator"><spanclass="std std-ref">Evaluator</span></a> Calculate the sum of output or label </p>
<p>The config file api is sum_evaluator. </p>
<p>Inherits from <aclass="reference internal"href="#paddleclasspaddle_1_1Evaluator"><spanclass="std std-ref">paddle::Evaluator</span></a></p>
@@ -384,7 +384,7 @@ The config file api is column_sum_evaluator. <dl class="docutils">
</dl>
</p>
<p>Inherits from <aclass="reference internal"href="#paddleclasspaddle_1_1ClassificationErrorEvaluator"><spanclass="std std-ref">paddle::ClassificationErrorEvaluator</span></a></p>
<spanid="paddle::PrecisionRecallEvaluator::StatsInfo"></span><spanclass="target"id="paddlestructpaddle_1_1PrecisionRecallEvaluator_1_1StatsInfo"></span><emclass="property">struct </em><codeclass="descname">StatsInfo</code><aclass="headerlink"href="#_CPPv2N6paddle24PrecisionRecallEvaluator9StatsInfoE"title="Permalink to this definition">¶</a></dt>
@@ -675,7 +675,7 @@ The config file api is column_sum_evaluator. <dl class="docutils">
<spanid="paddle::CTCErrorEvaluator"></span><spanclass="target"id="paddleclasspaddle_1_1CTCErrorEvaluator"></span><emclass="property">class </em><codeclass="descclassname">paddle::</code><codeclass="descname">CTCErrorEvaluator</code><aclass="headerlink"href="#_CPPv2N6paddle17CTCErrorEvaluatorE"title="Permalink to this definition">¶</a></dt>
@@ -743,7 +743,7 @@ The config file api is column_sum_evaluator. <dl class="docutils">
<dtid="_CPPv2N6paddle15PnpairEvaluatorE">
<spanid="paddle::PnpairEvaluator"></span><spanclass="target"id="paddleclasspaddle_1_1PnpairEvaluator"></span><emclass="property">class </em><codeclass="descclassname">paddle::</code><codeclass="descname">PnpairEvaluator</code><aclass="headerlink"href="#_CPPv2N6paddle15PnpairEvaluatorE"title="Permalink to this definition">¶</a></dt>
<dd><p>Inherits from <aclass="reference internal"href="#paddleclasspaddle_1_1Evaluator"><spanclass="std std-ref">paddle::Evaluator</span></a></p>
<spanid="paddle::PnpairEvaluator::PredictionResult"></span><spanclass="target"id="paddlestructpaddle_1_1PnpairEvaluator_1_1PredictionResult"></span><emclass="property">struct </em><codeclass="descname">PredictionResult</code><aclass="headerlink"href="#_CPPv2N6paddle15PnpairEvaluator16PredictionResultE"title="Permalink to this definition">¶</a></dt>
@@ -862,7 +862,7 @@ The config file api is column_sum_evaluator. <dl class="docutils">
<spanid="paddle::RankAucEvaluator"></span><spanclass="target"id="paddleclasspaddle_1_1RankAucEvaluator"></span><emclass="property">class </em><codeclass="descclassname">paddle::</code><codeclass="descname">RankAucEvaluator</code><aclass="headerlink"href="#_CPPv2N6paddle16RankAucEvaluatorE"title="Permalink to this definition">¶</a></dt>
<dd><p><aclass="reference internal"href="#paddleclasspaddle_1_1RankAucEvaluator"><spanclass="std std-ref">RankAucEvaluator</span></a> calculates the AUC of each list (i.e., titles under the same query), and averages them. Each list should be organized as a sequence. The inputs of this evaluator is [output, click, pv]. If pv is not provided, it will be set to 1. The types of click and pv are dense value. </p>
<p>Inherits from <aclass="reference internal"href="#paddleclasspaddle_1_1Evaluator"><spanclass="std std-ref">paddle::Evaluator</span></a></p>
@@ -1119,7 +1119,7 @@ The config file api is column_sum_evaluator. <dl class="docutils">
<p>Typically <aclass="reference internal"href="#paddleclasspaddle_1_1SequenceTextPrinter"><spanclass="std std-ref">SequenceTextPrinter</span></a> layer takes output of maxid or RecurrentGroup with maxid (when generating) as an input.</p>
<p>The config file api is seqtext_printer_evaluator. </p>
<p>Inherits from <aclass="reference internal"href="#paddleclasspaddle_1_1Evaluator"><spanclass="std std-ref">paddle::Evaluator</span></a></p>
@@ -1162,7 +1162,7 @@ The config file api is column_sum_evaluator. <dl class="docutils">
<dd><p>print classification error. </p>
<p>The config file api is classification_error_printer_evaluator. </p>
<p>Inherits from <aclass="reference internal"href="#paddleclasspaddle_1_1ClassificationErrorEvaluator"><spanclass="std std-ref">paddle::ClassificationErrorEvaluator</span></a></p>
<spanid="paddle::GradientMachine"></span><spanclass="target"id="paddleclasspaddle_1_1GradientMachine"></span><emclass="property">class </em><codeclass="descclassname">paddle::</code><codeclass="descname">GradientMachine</code><aclass="headerlink"href="#_CPPv2N6paddle15GradientMachineE"title="Permalink to this definition">¶</a></dt>
<dd><p>Subclassed by <aclass="reference internal"href="#paddleclasspaddle_1_1MultiGradientMachine"><spanclass="std std-ref">paddle::MultiGradientMachine</span></a>, <aclass="reference internal"href="#paddleclasspaddle_1_1NeuralNetwork"><spanclass="std std-ref">paddle::NeuralNetwork</span></a></p>
<spanid="paddle::IGradientMachineMode"></span><spanclass="target"id="paddleclasspaddle_1_1IGradientMachineMode"></span><emclass="property">class </em><codeclass="descclassname">paddle::</code><codeclass="descname">IGradientMachineMode</code><aclass="headerlink"href="#_CPPv2N6paddle20IGradientMachineModeE"title="Permalink to this definition">¶</a></dt>
<p>Inherits from <aclass="reference internal"href="#paddleclasspaddle_1_1GradientMachine"><spanclass="std std-ref">paddle::GradientMachine</span></a></p>
<spanid="paddle::TrainerThread"></span><spanclass="target"id="paddleclasspaddle_1_1TrainerThread"></span><emclass="property">class </em><codeclass="descclassname">paddle::</code><codeclass="descname">TrainerThread</code><aclass="headerlink"href="#_CPPv2N6paddle13TrainerThreadE"title="Permalink to this definition">¶</a></dt>
<spanid="paddle::RecurrentGradientMachine"></span><spanclass="target"id="paddleclasspaddle_1_1RecurrentGradientMachine"></span><emclass="property">class </em><codeclass="descclassname">paddle::</code><codeclass="descname">RecurrentGradientMachine</code><aclass="headerlink"href="#_CPPv2N6paddle24RecurrentGradientMachineE"title="Permalink to this definition">¶</a></dt>
<dd><p>Inherits from <aclass="reference internal"href="#paddleclasspaddle_1_1NeuralNetwork"><spanclass="std std-ref">paddle::NeuralNetwork</span></a></p>
<spanid="paddle::RecurrentGradientMachine::EosFrameLine"></span><spanclass="target"id="paddlestructpaddle_1_1RecurrentGradientMachine_1_1EosFrameLine"></span><emclass="property">struct </em><codeclass="descname">EosFrameLine</code><aclass="headerlink"href="#_CPPv2N6paddle24RecurrentGradientMachine12EosFrameLineE"title="Permalink to this definition">¶</a></dt>
<spanid="paddle::RecurrentGradientMachine::Generator"></span><spanclass="target"id="paddlestructpaddle_1_1RecurrentGradientMachine_1_1Generator"></span><emclass="property">struct </em><codeclass="descname">Generator</code><aclass="headerlink"href="#_CPPv2N6paddle24RecurrentGradientMachine9GeneratorE"title="Permalink to this definition">¶</a></dt>
<spanid="paddle::RecurrentGradientMachine::Info"></span><spanclass="target"id="paddlestructpaddle_1_1RecurrentGradientMachine_1_1Info"></span><emclass="property">struct </em><codeclass="descname">Info</code><aclass="headerlink"href="#_CPPv2N6paddle24RecurrentGradientMachine4InfoE"title="Permalink to this definition">¶</a></dt>
<spanid="paddle::RecurrentGradientMachine::InFrameLine"></span><spanclass="target"id="paddlestructpaddle_1_1RecurrentGradientMachine_1_1InFrameLine"></span><emclass="property">struct </em><codeclass="descname">InFrameLine</code><aclass="headerlink"href="#_CPPv2N6paddle24RecurrentGradientMachine11InFrameLineE"title="Permalink to this definition">¶</a></dt>
<spanid="paddle::RecurrentGradientMachine::MemoryFrameLine"></span><spanclass="target"id="paddlestructpaddle_1_1RecurrentGradientMachine_1_1MemoryFrameLine"></span><emclass="property">struct </em><codeclass="descname">MemoryFrameLine</code><aclass="headerlink"href="#_CPPv2N6paddle24RecurrentGradientMachine15MemoryFrameLineE"title="Permalink to this definition">¶</a></dt>
<spanid="paddle::RecurrentGradientMachine::OutFrameLine"></span><spanclass="target"id="paddlestructpaddle_1_1RecurrentGradientMachine_1_1OutFrameLine"></span><emclass="property">struct </em><codeclass="descname">OutFrameLine</code><aclass="headerlink"href="#_CPPv2N6paddle24RecurrentGradientMachine12OutFrameLineE"title="Permalink to this definition">¶</a></dt>
<spanid="paddle::RecurrentGradientMachine::Path"></span><spanclass="target"id="paddlestructpaddle_1_1RecurrentGradientMachine_1_1Path"></span><emclass="property">struct </em><codeclass="descname">Path</code><aclass="headerlink"href="#_CPPv2N6paddle24RecurrentGradientMachine4PathE"title="Permalink to this definition">¶</a></dt>
<spanid="paddle::NeuralNetwork"></span><spanclass="target"id="paddleclasspaddle_1_1NeuralNetwork"></span><emclass="property">class </em><codeclass="descclassname">paddle::</code><codeclass="descname">NeuralNetwork</code><aclass="headerlink"href="#_CPPv2N6paddle13NeuralNetworkE"title="Permalink to this definition">¶</a></dt>
<dd><p>Inherits from <aclass="reference internal"href="#paddleclasspaddle_1_1GradientMachine"><spanclass="std std-ref">paddle::GradientMachine</span></a></p>
<p>Subclassed by paddle::MultiNetwork, <aclass="reference internal"href="#paddleclasspaddle_1_1ParallelNeuralNetwork"><spanclass="std std-ref">paddle::ParallelNeuralNetwork</span></a>, <aclass="reference internal"href="#paddleclasspaddle_1_1RecurrentGradientMachine"><spanclass="std std-ref">paddle::RecurrentGradientMachine</span></a></p>
<spanid="paddle::ParallelNeuralNetwork"></span><spanclass="target"id="paddleclasspaddle_1_1ParallelNeuralNetwork"></span><emclass="property">class </em><codeclass="descclassname">paddle::</code><codeclass="descname">ParallelNeuralNetwork</code><aclass="headerlink"href="#_CPPv2N6paddle21ParallelNeuralNetworkE"title="Permalink to this definition">¶</a></dt>
<dd><p>A <aclass="reference internal"href="#paddleclasspaddle_1_1ParallelNeuralNetwork"><spanclass="std std-ref">ParallelNeuralNetwork</span></a> is capable of calculating a neural network through multiple threads in parallel. </p>
<p>Inherits from <aclass="reference internal"href="#paddleclasspaddle_1_1NeuralNetwork"><spanclass="std std-ref">paddle::NeuralNetwork</span></a></p>
<span id="paddle::Projection"></span><span class="target" id="paddleclasspaddle_1_1Projection"></span><em class="property">class </em><code class="descclassname">paddle::</code><code class="descname">Projection</code><a class="headerlink" href="#_CPPv2N6paddle10ProjectionE" title="Permalink to this definition">¶</a></dt>
<dd><p>A projection takes one <a class="reference internal" href="../../parameter/parameter/parameter.html#paddlestructpaddle_1_1Argument"><span class="std std-ref">Argument</span></a> as input, calculate the result and add it to output <a class="reference internal" href="../../parameter/parameter/parameter.html#paddlestructpaddle_1_1Argument"><span class="std std-ref">Argument</span></a>. </p>
<dd><p>A layer has full connections to all neurons in the previous layer. It computes an inner product with a set of learned weights, and (optionally) adds biases.</p>
<p>The config file api is fc_layer. </p>
<p>Inherits from <a class="reference internal" href="#paddleclasspaddle_1_1Layer"><span class="std std-ref">paddle::Layer</span></a></p>
<p><a class="reference internal" href="#paddleclasspaddle_1_1SelectiveFullyConnectedLayer"><span class="std std-ref">SelectiveFullyConnectedLayer</span></a> differs from <a class="reference internal" href="#paddleclasspaddle_1_1FullyConnectedLayer"><span class="std std-ref">FullyConnectedLayer</span></a> by that it requires an additional input to indicate several selected columns, and only compute the multiplications between the input matrices and the selected columns of the parameter matrices of this layer. If the selected columns is not specified, SelectiveFullyConnected layer acts exactly like <a class="reference internal" href="#paddleclasspaddle_1_1FullyConnectedLayer"><span class="std std-ref">FullyConnectedLayer</span></a>.</p>
<p>The config file api is selective_fc_layer. </p>
<p>Inherits from <a class="reference internal" href="#paddleclasspaddle_1_1Layer"><span class="std std-ref">paddle::Layer</span></a></p>
<dd><p>A Base Convolution <a class="reference internal" href="#paddleclasspaddle_1_1Layer"><span class="std std-ref">Layer</span></a>, which convolves the input image with learned filters and (optionally) adds biases. </p>
<p>Inherits from <a class="reference internal" href="#paddleclasspaddle_1_1Layer"><span class="std std-ref">paddle::Layer</span></a></p>
<p>Subclassed by <a class="reference internal" href="#paddleclasspaddle_1_1CudnnConvLayer"><span class="std std-ref">paddle::CudnnConvLayer</span></a>, paddle::ExpandConvBaseLayer</p>
<dd><p><a class="reference internal" href="#paddleclasspaddle_1_1ConvOperator"><span class="std std-ref">ConvOperator</span></a> takes two inputs to perform the convolution. The first input is the image, and the second input is the convolution kernel. The height of data for two inputs are the same. Each data of the first input is convolved with each data of the second input indepedently. </p>
<p>The config file api is conv_operator. </p>
<p>Inherits from <a class="reference internal" href="#paddleclasspaddle_1_1Operator"><span class="std std-ref">paddle::Operator</span></a></p>
<dd><p>A 2-dimension conv layer implemented by cuDNN. It only supports GPU mode. We automatic select <a class="reference internal" href="#paddleclasspaddle_1_1CudnnConvLayer"><span class="std std-ref">CudnnConvLayer</span></a> for GPU mode and <a class="reference internal" href="#paddleclasspaddle_1_1ExpandConvLayer"><span class="std std-ref">ExpandConvLayer</span></a> for CPU mode if you set type of “conv”. User also can specfiy type of “exconv” or “cudnn_conv” for particular type. </p>
<p>The config file api is img_conv_layer. </p>
<p>Inherits from <a class="reference internal" href="#paddleclasspaddle_1_1ConvBaseLayer"><span class="std std-ref">paddle::ConvBaseLayer</span></a></p>
<dd><p><a class="reference internal" href="#paddleclasspaddle_1_1CudnnPoolLayer"><span class="std std-ref">CudnnPoolLayer</span></a> is subclass of <a class="reference internal" href="#paddleclasspaddle_1_1PoolLayer"><span class="std std-ref">PoolLayer</span></a>, which is implemented by cudnn api and only supports GPU. </p>
<p>The config file api is img_pool_layer. </p>
<p>Inherits from <a class="reference internal" href="#paddleclasspaddle_1_1PoolLayer"><span class="std std-ref">paddle::PoolLayer</span></a></p>
<dd><p>response normalization within feature maps namely normalize in independent channel When code refactoring, we delete the original implementation. Need to implement in the futrue. </p>
<p>Inherits from <a class="reference internal" href="#paddleclasspaddle_1_1NormLayer"><span class="std std-ref">paddle::NormLayer</span></a></p>
<p>Subclassed by <a class="reference internal" href="#paddleclasspaddle_1_1CMRProjectionNormLayer"><span class="std std-ref">paddle::CMRProjectionNormLayer</span></a></p>
@@ -2458,7 +2458,7 @@ The config file api is batch_norm_layer. <dl class="docutils">
where <span class="math">\(in\)</span> is a (batchSize x dataDim) input vector, and <span class="math">\(out\)</span> is a (batchSize x dataDim) output vector.</p>
<p>The config file api is sum_to_one_norm_layer. </p>
<p>Inherits from <a class="reference internal" href="#paddleclasspaddle_1_1Layer"><span class="std std-ref">paddle::Layer</span></a></p>
@@ -2589,7 +2589,7 @@ The config file api is batch_norm_layer. <dl class="docutils">
\[\begin{split} out_{i} = act(in_{i}) \ \ \text{for} \ i = end \\ out_{i} = act(in_{i} + out_{i+1} * W) \ \ \text{for} \ start <= i < end \end{split}\]</div>
There are two methods to calculate rnn. One way is to compute rnn one sequence by one sequence. The other way is to reorganize the input into batches, then compute rnn one batch by one batch. Users can select them by rnn_use_batch flag. </p>
<p>Inherits from <a class="reference internal" href="#paddleclasspaddle_1_1Layer"><span class="std std-ref">paddle::Layer</span></a></p>
@@ -4011,7 +4011,7 @@ The config file api if gru_step_layer. <dl class="docutils">
<dd><p><a class="reference internal" href="#paddleclasspaddle_1_1AgentLayer"><span class="std std-ref">AgentLayer</span></a> use as a virtual input of another layer in config, before execute forward/backward, setRealLayer() should be called to set one and only one real layer </p>
<p>Inherits from <a class="reference internal" href="#paddleclasspaddle_1_1Layer"><span class="std std-ref">paddle::Layer</span></a></p>
<p>Subclassed by <a class="reference internal" href="#paddleclasspaddle_1_1SequenceAgentLayer"><span class="std std-ref">paddle::SequenceAgentLayer</span></a></p>
@@ -4070,7 +4070,7 @@ The config file api if gru_step_layer. <dl class="docutils">
<span id="paddle::SequenceAgentLayer"></span><span class="target" id="paddleclasspaddle_1_1SequenceAgentLayer"></span><em class="property">class </em><code class="descclassname">paddle::</code><code class="descname">SequenceAgentLayer</code><a class="headerlink" href="#_CPPv2N6paddle18SequenceAgentLayerE" title="Permalink to this definition">¶</a></dt>
<dd><p>like <a class="reference internal" href="#paddleclasspaddle_1_1AgentLayer"><span class="std std-ref">AgentLayer</span></a>, but use first <em>numSamples</em> sequences </p>
<p>Inherits from <a class="reference internal" href="#paddleclasspaddle_1_1AgentLayer"><span class="std std-ref">paddle::AgentLayer</span></a></p>
@@ -4106,7 +4106,7 @@ The config file api if gru_step_layer. <dl class="docutils">
<dd><p>Like <a class="reference internal" href="#paddleclasspaddle_1_1AgentLayer"><span class="std std-ref">AgentLayer</span></a>, but it can gather many real layers. Each real layer give a few rows of a sequence, after gather all real layers, <a class="reference internal" href="#paddleclasspaddle_1_1GatherAgentLayer"><span class="std std-ref">GatherAgentLayer</span></a> collect a complete sequence. </p>
<p>Inherits from <a class="reference internal" href="#paddleclasspaddle_1_1Layer"><span class="std std-ref">paddle::Layer</span></a></p>
<p>Subclassed by <a class="reference internal" href="#paddleclasspaddle_1_1SequenceGatherAgentLayer"><span class="std std-ref">paddle::SequenceGatherAgentLayer</span></a></p>
@@ -4180,7 +4180,7 @@ The config file api if gru_step_layer. <dl class="docutils">
<span id="paddle::SequenceGatherAgentLayer"></span><span class="target" id="paddleclasspaddle_1_1SequenceGatherAgentLayer"></span><em class="property">class </em><code class="descclassname">paddle::</code><code class="descname">SequenceGatherAgentLayer</code><a class="headerlink" href="#_CPPv2N6paddle24SequenceGatherAgentLayerE" title="Permalink to this definition">¶</a></dt>
<dd><p>Like <a class="reference internal" href="#paddleclasspaddle_1_1GatherAgentLayer"><span class="std std-ref">GatherAgentLayer</span></a>, but select a few sequence in real layer. <em>ids</em> in addRealLayer() are the ids of selected sequence. It’s used to reorder sequence output. </p>
<p>Inherits from <a class="reference internal" href="#paddleclasspaddle_1_1GatherAgentLayer"><span class="std std-ref">paddle::GatherAgentLayer</span></a></p>
@@ -4216,7 +4216,7 @@ The config file api if gru_step_layer. <dl class="docutils">
<dd><p>Like <a class="reference internal" href="#paddleclasspaddle_1_1AgentLayer"><span class="std std-ref">AgentLayer</span></a>, but only select a few rows in real layer. [idIndex, idIndex + idSize) of <em>ids</em> in setRealLayerAndOutput() are the selected row ids. It’s used to scatter one layer’s output to many small submodels. <a class="reference internal" href="#paddleclasspaddle_1_1ScatterAgentLayer"><span class="std std-ref">ScatterAgentLayer</span></a> can support ids real layer, if it is, the agent will select a few ids in real layer. </p>
<p>Inherits from <a class="reference internal" href="#paddleclasspaddle_1_1Layer"><span class="std std-ref">paddle::Layer</span></a></p>
<p>Subclassed by <a class="reference internal" href="#paddleclasspaddle_1_1SequenceScatterAgentLayer"><span class="std std-ref">paddle::SequenceScatterAgentLayer</span></a></p>
@@ -4326,7 +4326,7 @@ The config file api if gru_step_layer. <dl class="docutils">
<span id="paddle::SequenceScatterAgentLayer"></span><span class="target" id="paddleclasspaddle_1_1SequenceScatterAgentLayer"></span><em class="property">class </em><code class="descclassname">paddle::</code><code class="descname">SequenceScatterAgentLayer</code><a class="headerlink" href="#_CPPv2N6paddle25SequenceScatterAgentLayerE" title="Permalink to this definition">¶</a></dt>
<dd><p>Like <a class="reference internal" href="#paddleclasspaddle_1_1ScatterAgentLayer"><span class="std std-ref">ScatterAgentLayer</span></a>, but select a few sequence in real layer. <em>ids</em> in <a class="reference internal" href="#paddleclasspaddle_1_1ScatterAgentLayer_1abc1b52a1a9b0fa738300450626e24b80"><span class="std std-ref">setRealLayer()</span></a> or setRealLayerAndOutput() are the ids of selected sequence. It’s used to reorder sequence input. </p>
<p>Inherits from <a class="reference internal" href="#paddleclasspaddle_1_1ScatterAgentLayer"><span class="std std-ref">paddle::ScatterAgentLayer</span></a></p>
@@ -4411,7 +4411,7 @@ The config file api if gru_step_layer. <dl class="docutils">
<dd><p>A mixed layer has multiple input layers. Each input layer was processed by a <a class="reference internal" href="#paddleclasspaddle_1_1Projection"><span class="std std-ref">Projection</span></a> or <a class="reference internal" href="#paddleclasspaddle_1_1Operator"><span class="std std-ref">Operator</span></a>. The results of all projections or Operators are summed together with bias (if configured), and then go through an activation function and dropout (if configured).</p>
<p>The config file api is mixed_layer. </p>
<p>Inherits from <a class="reference internal" href="#paddleclasspaddle_1_1Layer"><span class="std std-ref">paddle::Layer</span></a></p>
@@ -4780,7 +4780,7 @@ The config file api if gru_step_layer. <dl class="docutils">
<dd><p>A layer for “internal average” for sequence input. Input: one or more sequences. Each sequence contains some instances. If SequenceLevel = kNonSeq: Output: output size is the number of input sequences (NOT input instances) output[i] = average_{for each instance in this sequence}{input[i]} If SequenceLevel = kSeq: Check input sequence must has sub-sequence Output: output size is the number of input sub-sequences output[i] = average_{for each instance in this sub-sequence}{input[i]}</p>
@@ -4864,7 +4864,7 @@ The config file api if gru_step_layer. <dl class="docutils">
<dd><p>A layer for “internal max” for sequence input. Input: one or more sequences. Each sequence contains some instances. If SequenceLevel = kNonSeq: Output: output size is the number of input sequences (NOT input instances) output[i] = max_{for each instance in this sequence}{input[i]} If SequenceLevel = kSeq: Check input sequence must has sub-sequence Output: output size is the number of input sub-sequences output[i] = max_{for each instance in this sub-sequence}{input[i]}</p>
@@ -4914,7 +4914,7 @@ The config file api if gru_step_layer. <dl class="docutils">
<dd><p>A layer for extracting the last instance of the input sequence. Input: a sequence If SequenceLevel = kNonseq: Output: a sequence containing only the last instance of the input sequence If SequenceLevel = kSeq: Check input sequence must has sub-sequence Output: a sequence containing only the last instance of each sub-sequence of the input sequence</p>
<p>The config file api is last_seq and first_seq. </p>
@@ -4971,7 +4971,7 @@ The config file api if gru_step_layer. <dl class="docutils">
<span id="paddle::ConcatenateLayer"></span><span class="target" id="paddleclasspaddle_1_1ConcatenateLayer"></span><em class="property">class </em><code class="descclassname">paddle::</code><code class="descname">ConcatenateLayer</code><a class="headerlink" href="#_CPPv2N6paddle16ConcatenateLayerE" title="Permalink to this definition">¶</a></dt>
<dd><p>A concatenate layer has multiple input layers. It concatenates rows of each input as one row for the output of this layer and apply activation. </p>
<p>Inherits from <a class="reference internal" href="#paddleclasspaddle_1_1Layer"><span class="std std-ref">paddle::Layer</span></a></p>
@@ -5012,7 +5012,7 @@ The config file api if gru_step_layer. <dl class="docutils">
<span id="paddle::ConcatenateLayer2"></span><span class="target" id="paddleclasspaddle_1_1ConcatenateLayer2"></span><em class="property">class </em><code class="descclassname">paddle::</code><code class="descname">ConcatenateLayer2</code><a class="headerlink" href="#_CPPv2N6paddle17ConcatenateLayer2E" title="Permalink to this definition">¶</a></dt>
<dd><p>concat2 layer is like concat layer, but each input layer was processed by a <a class="reference internal" href="#paddleclasspaddle_1_1Projection"><span class="std std-ref">Projection</span></a>. </p>
<p>Inherits from <a class="reference internal" href="#paddleclasspaddle_1_1Layer"><span class="std std-ref">paddle::Layer</span></a></p>
@@ -5081,7 +5081,7 @@ The config file api if gru_step_layer. <dl class="docutils">
<span id="paddle::SequenceConcatLayer"></span><span class="target" id="paddleclasspaddle_1_1SequenceConcatLayer"></span><em class="property">class </em><code class="descclassname">paddle::</code><code class="descname">SequenceConcatLayer</code><a class="headerlink" href="#_CPPv2N6paddle19SequenceConcatLayerE" title="Permalink to this definition">¶</a></dt>
<dd><p>A layer for concatenating the first sequence with the second sequence following the first Input: two sequences each containing some instances Output: a concatenated sequence of the two input sequences </p>
<p>Inherits from <a class="reference internal" href="#paddleclasspaddle_1_1Layer"><span class="std std-ref">paddle::Layer</span></a></p>
@@ -5208,7 +5208,7 @@ The config file api if gru_step_layer. <dl class="docutils">
<p>The expand method is the same with <a class="reference internal" href="#paddleclasspaddle_1_1ExpandConvLayer"><span class="std std-ref">ExpandConvLayer</span></a>, but saved the transposed value. After expanding, output_.sequenceStartPositions will store timeline. The number of time steps are outputH_ * outputW_ and the dimension of each time step is blockH_ * blockW_ * channels_. This layer can be used after convolution neural network, and before recurrent neural network.</p>
<p>The config file api is block_expand_layer. </p>
<p>Inherits from <a class="reference internal" href="#paddleclasspaddle_1_1Layer"><span class="std std-ref">paddle::Layer</span></a></p>
@@ -5578,7 +5578,7 @@ sequence is one) to sequence data.”</p>
where <span class="math">\(y\)</span> is output, <span class="math">\(x\)</span> is input, <span class="math">\(b\)</span> is bias, and <span class="math">\(f\)</span> is activation function.</p>
<p>The config file api is addto_layer. </p>
<p>Inherits from <a class="reference internal" href="#paddleclasspaddle_1_1Layer"><span class="std std-ref">paddle::Layer</span></a></p>
@@ -5704,7 +5704,7 @@ sequence is one) to sequence data.”</p>
where <span class="math">\(x_1\)</span> and <span class="math">\(x_2\)</span> are two (batchSize x dataDim) inputs, <span class="math">\(w\)</span> is (batchSize x 1) weight vector, and <span class="math">\(y\)</span> is (batchSize x dataDim) output.</p>
<p>The config file api is interpolation_layer. </p>
<p>Inherits from <a class="reference internal" href="#paddleclasspaddle_1_1Layer"><span class="std std-ref">paddle::Layer</span></a></p>
@@ -5932,7 +5932,7 @@ sequence is one) to sequence data.”</p>
where <span class="math">\(x\)</span> is a input vector, <span class="math">\(w\)</span> is scalar weight, and output <span class="math">\(y\)</span> is a vector.</p>
<p>The config file api is power_layer. </p>
<p>Inherits from <a class="reference internal" href="#paddleclasspaddle_1_1Layer"><span class="std std-ref">paddle::Layer</span></a></p>
@@ -5984,7 +5984,7 @@ sequence is one) to sequence data.”</p>
where <span class="math">\(x\)</span> is (batchSize x dataDim) input, <span class="math">\(w\)</span> is (batchSize x 1) weight vector, and <span class="math">\(y\)</span> is (batchSize x dataDim) output.</p>
<p>The config file api is scaling_layer. </p>
<p>Inherits from <a class="reference internal" href="#paddleclasspaddle_1_1Layer"><span class="std std-ref">paddle::Layer</span></a></p>
@@ -6320,7 +6320,7 @@ The space requirement is O(N)=O(N * sizeof(Interval)). The computational complex
<dd><p>A layer for sampling id from multinomial distribution from the input layer. Sampling one id for one sample. The result is stored in output_.ids. </p>
<p>The config file api is sampling_id_layer. </p>
<p>Inherits from <a class="reference internal" href="#paddleclasspaddle_1_1Layer"><span class="std std-ref">paddle::Layer</span></a></p>
@@ -6365,7 +6365,7 @@ The space requirement is O(N)=O(N * sizeof(Interval)). The computational complex
<dd><p>Base class for a particular type of cost layer. This type of cost should have one data layer, one label layer and an optional weight layer as input. The derived class should implemnt forwardImp() and backwardImp() which calculate the cost for data and label. The weight is automatically handled by the base class. </p>
<p>Inherits from <a class="reference internal" href="#paddleclasspaddle_1_1Layer"><span class="std std-ref">paddle::Layer</span></a></p>
@@ -6436,7 +6436,7 @@ The space requirement is O(N)=O(N * sizeof(Interval)). The computational complex
\[\begin{split} Loss = \left\{\begin{matrix} 4 * y * f & \textit{if} \ \ y* f < -1 \\ (1 - y * f)^2 & \textit{if} \ \ -1 < y * f < 1 \\ 0 & \textit{otherwise} \end{matrix}\right. \end{split}\]</div>
</p>
<p>Inherits from <a class="reference internal" href="#paddleclasspaddle_1_1CostLayer"><span class="std std-ref">paddle::CostLayer</span></a></p>
@@ -6632,7 +6632,7 @@ The space requirement is O(N)=O(N * sizeof(Interval)). The computational complex
<p>The <span class="math">\(Z(x)\)</span> is the softmax normalizer.</p>
<p>[1] Jacob Devlin, Rabih Zbib, Zhongqiang Huang, Thomas Lamar, Richard Schwartz, and John Makhoul. Fast and robust neural network joint models for statistical machine translation. In Proceedings of the ACL 2014 Conference. </p>
<p>Inherits from <a class="reference internal" href="#paddleclasspaddle_1_1CostLayer"><span class="std std-ref">paddle::CostLayer</span></a></p>
<span id="paddle::CRFDecodingLayer"></span><span class="target" id="paddleclasspaddle_1_1CRFDecodingLayer"></span><em class="property">class </em><code class="descclassname">paddle::</code><code class="descname">CRFDecodingLayer</code><a class="headerlink" href="#_CPPv2N6paddle16CRFDecodingLayerE" title="Permalink to this definition">¶</a></dt>
<dd><p>A layer for calculating the decoding sequence of sequential conditional random field model. The decoding sequence is stored in output_.ids It also calculate error, output_.value[i] is 1 for incorrect decoding or 0 for correct decoding) See LinearChainCRF.h for the detail of the CRF formulation. </p>
<p>Inherits from <a class="reference internal" href="#paddleclasspaddle_1_1CRFLayer"><span class="std std-ref">paddle::CRFLayer</span></a></p>
<dd><p>A layer for calculating the cost of sequential conditional random field model. See class <a class="reference internal" href="#paddleclasspaddle_1_1LinearChainCRF"><span class="std std-ref">LinearChainCRF</span></a> for the detail of the CRF formulation. </p>
<p>Inherits from <a class="reference internal" href="#paddleclasspaddle_1_1Layer"><span class="std std-ref">paddle::Layer</span></a></p>
<p>Subclassed by <a class="reference internal" href="#paddleclasspaddle_1_1CRFDecodingLayer"><span class="std std-ref">paddle::CRFDecodingLayer</span></a></p>
@@ -7486,7 +7486,7 @@ Hierarchical Probabilistic Neural Network Language Model.”</p>
<dd><p>Noise-contrastive estimation. Implements the method in the following paper: A fast and simple algorithm for training neural probabilistic language models.</p>
<p>The config file api is nce_layer. </p>
<p>Inherits from <a class="reference internal" href="#paddleclasspaddle_1_1Layer"><span class="std std-ref">paddle::Layer</span></a></p>
<dd><em>#include <Matrix.h></em><p>Copy or assignemnt constructor will share the data as opposed to making a copy of the original data. To make a copy of the orinal data, use copyFrom() instead. </p>
<p>Inherits from <a class="reference internal" href="#paddleclasspaddle_1_1BaseMatrixT"><span class="std std-ref">paddle::BaseMatrixT< real ></span></a></p>
@@ -4144,7 +4144,7 @@ where bit(i, j) = ((codes(i) + numClasses) & 2^j) ? 1 : 0
<dt id="_CPPv2N6paddle10GpuVectorTE">
<span id="paddle::GpuVectorT"></span><em class="property">class </em><code class="descname">GpuVectorT</code><a class="headerlink" href="#_CPPv2N6paddle10GpuVectorTE" title="Permalink to this definition">¶</a></dt>
<dd><p>Inherits from <a class="reference internal" href="#paddleclasspaddle_1_1VectorT"><span class="std std-ref">paddle::VectorT< T ></span></a></p>
@@ -4340,7 +4340,7 @@ where bit(i, j) = ((codes(i) + numClasses) & 2^j) ? 1 : 0
<span id="paddle::CpuVectorT"></span><em class="property">class </em><code class="descname">CpuVectorT</code><a class="headerlink" href="#_CPPv2N6paddle10CpuVectorTE" title="Permalink to this definition">¶</a></dt>
<dd><p>Inherits from <a class="reference internal" href="#paddleclasspaddle_1_1VectorT"><span class="std std-ref">paddle::VectorT< T ></span></a></p>
<p>Subclassed by <a class="reference internal" href="#paddleclasspaddle_1_1ParallelCpuVectorT"><span class="std std-ref">paddle::ParallelCpuVectorT< T ></span></a></p>
@@ -4540,7 +4540,7 @@ where bit(i, j) = ((codes(i) + numClasses) & 2^j) ? 1 : 0
<span id="paddle::BaseVector"></span><em class="property">class </em><code class="descname">BaseVector</code><a class="headerlink" href="#_CPPv2N6paddle10BaseVectorE" title="Permalink to this definition">¶</a></dt>
<dd><p>Inherits from <a class="reference internal" href="#paddleclasspaddle_1_1BaseMatrixT"><span class="std std-ref">paddle::BaseMatrixT< T ></span></a></p>
<p>Subclassed by <a class="reference internal" href="#paddleclasspaddle_1_1VectorT"><span class="std std-ref">paddle::VectorT< T ></span></a></p>
@@ -4571,7 +4571,7 @@ where bit(i, j) = ((codes(i) + numClasses) & 2^j) ? 1 : 0
<dd><em>#include <Vector.h></em><p>Copy or assignemnt constructor will share the data as opposed to making a copy of the original data. To make a copy of the orinal data, use <a class="reference internal" href="#paddleclasspaddle_1_1VectorT_1ad2afa0a39de13b7e43ee3f52ee96a188"><span class="std std-ref">copyFrom()</span></a> instead. </p>
<p>Inherits from <a class="reference internal" href="#paddleclasspaddle_1_1BaseVector"><span class="std std-ref">paddle::BaseVector< T ></span></a></p>
<p>Subclassed by <a class="reference internal" href="#paddleclasspaddle_1_1CpuVectorT"><span class="std std-ref">paddle::CpuVectorT< T ></span></a>, <a class="reference internal" href="#paddleclasspaddle_1_1GpuVectorT"><span class="std std-ref">paddle::GpuVectorT< T ></span></a></p>
@@ -4890,7 +4890,7 @@ where bit(i, j) = ((codes(i) + numClasses) & 2^j) ? 1 : 0
<dt id="_CPPv2N6paddle18ParallelCpuVectorTE">
<span id="paddle::ParallelCpuVectorT"></span><em class="property">class </em><code class="descname">ParallelCpuVectorT</code><a class="headerlink" href="#_CPPv2N6paddle18ParallelCpuVectorTE" title="Permalink to this definition">¶</a></dt>
<dd><p>Inherits from <a class="reference internal" href="#paddleclasspaddle_1_1CpuVectorT"><span class="std std-ref">paddle::CpuVectorT< T ></span></a></p>
<p>mostly same as <a class="reference internal" href="#paddleclasspaddle_1_1CpuMatrix"><span class="std std-ref">CpuMatrix</span></a>, but maintain sparse row ids occured, ids are hashed by worker thread id. </p>
<p>Inherits from <a class="reference internal" href="#paddleclasspaddle_1_1CpuMatrix"><span class="std std-ref">paddle::CpuMatrix</span></a></p>
<spanid="paddle"></span><spanclass="target"id="paddlenamespacepaddle"></span><emclass="property">namespace </em><codeclass="descname">paddle</code><aclass="headerlink"href="#_CPPv26paddle"title="Permalink to this definition">¶</a></dt>
<spanid="paddle::MemoryHandle"></span><spanclass="target"id="paddleclasspaddle_1_1MemoryHandle"></span><emclass="property">class </em><codeclass="descname">MemoryHandle</code><aclass="headerlink"href="#_CPPv2N6paddle12MemoryHandleE"title="Permalink to this definition">¶</a></dt>
<dd><p>Subclassed by <aclass="reference internal"href="#paddleclasspaddle_1_1CpuMemoryHandle"><spanclass="std std-ref">paddle::CpuMemoryHandle</span></a>, <aclass="reference internal"href="#paddleclasspaddle_1_1GpuMemoryHandle"><spanclass="std std-ref">paddle::GpuMemoryHandle</span></a></p>
<dd><em>#include <Allocator.h></em><p><aclass="reference internal"href="#paddleclasspaddle_1_1Allocator"><spanclass="std std-ref">Allocator</span></a> base class. </p>
<p>This is the base class of all <aclass="reference internal"href="#paddleclasspaddle_1_1Allocator"><spanclass="std std-ref">Allocator</span></a> class. </p>
<spanid="paddle::CpuAllocator"></span><spanclass="target"id="paddleclasspaddle_1_1CpuAllocator"></span><emclass="property">class </em><codeclass="descname">CpuAllocator</code><aclass="headerlink"href="#_CPPv2N6paddle12CpuAllocatorE"title="Permalink to this definition">¶</a></dt>
<spanid="paddle::GpuAllocator"></span><spanclass="target"id="paddleclasspaddle_1_1GpuAllocator"></span><emclass="property">class </em><codeclass="descname">GpuAllocator</code><aclass="headerlink"href="#_CPPv2N6paddle12GpuAllocatorE"title="Permalink to this definition">¶</a></dt>
<spanid="paddle::CudaHostAllocator"></span><spanclass="target"id="paddleclasspaddle_1_1CudaHostAllocator"></span><emclass="property">class </em><codeclass="descname">CudaHostAllocator</code><aclass="headerlink"href="#_CPPv2N6paddle17CudaHostAllocatorE"title="Permalink to this definition">¶</a></dt>
<spanid="paddle::PoolAllocator"></span><spanclass="target"id="paddleclasspaddle_1_1PoolAllocator"></span><emclass="property">class </em><codeclass="descname">PoolAllocator</code><aclass="headerlink"href="#_CPPv2N6paddle13PoolAllocatorE"title="Permalink to this definition">¶</a></dt>
<dd><em>#include <PoolAllocator.h></em><p>Memory pool allocator implementation. </p>
<spanid="paddle::StorageEngine"></span><spanclass="target"id="paddleclasspaddle_1_1StorageEngine"></span><emclass="property">class </em><codeclass="descname">StorageEngine</code><aclass="headerlink"href="#_CPPv2N6paddle13StorageEngineE"title="Permalink to this definition">¶</a></dt>
<dd><em>#include <Storage.h></em><p>Storage manager for multiple devices. </p>
<spanid="paddle::SgdOptimizer"></span><spanclass="target"id="paddleclasspaddle_1_1SgdOptimizer"></span><emclass="property">class </em><codeclass="descname">SgdOptimizer</code><aclass="headerlink"href="#_CPPv2N6paddle12SgdOptimizerE"title="Permalink to this definition">¶</a></dt>
<dd><p>Inherits from <aclass="reference internal"href="#paddleclasspaddle_1_1ParameterOptimizer"><spanclass="std std-ref">paddle::ParameterOptimizer</span></a></p>
<spanid="paddle::SparseMomentumParameterOptimizer"></span><spanclass="target"id="paddleclasspaddle_1_1SparseMomentumParameterOptimizer"></span><emclass="property">class </em><codeclass="descname">SparseMomentumParameterOptimizer</code><aclass="headerlink"href="#_CPPv2N6paddle32SparseMomentumParameterOptimizerE"title="Permalink to this definition">¶</a></dt>
<dd><p>Inherits from <aclass="reference internal"href="#paddleclasspaddle_1_1ParameterOptimizer"><spanclass="std std-ref">paddle::ParameterOptimizer</span></a></p>
<spanid="paddle::AdagradParameterOptimizer"></span><spanclass="target"id="paddleclasspaddle_1_1AdagradParameterOptimizer"></span><emclass="property">class </em><codeclass="descname">AdagradParameterOptimizer</code><aclass="headerlink"href="#_CPPv2N6paddle25AdagradParameterOptimizerE"title="Permalink to this definition">¶</a></dt>
<dd><p>Inherits from <aclass="reference internal"href="#paddleclasspaddle_1_1ParameterOptimizer"><spanclass="std std-ref">paddle::ParameterOptimizer</span></a></p>
<spanid="paddle::AdaDeltaParameterOptimizer"></span><spanclass="target"id="paddleclasspaddle_1_1AdaDeltaParameterOptimizer"></span><emclass="property">class </em><codeclass="descname">AdaDeltaParameterOptimizer</code><aclass="headerlink"href="#_CPPv2N6paddle26AdaDeltaParameterOptimizerE"title="Permalink to this definition">¶</a></dt>
<dd><p>Inherits from <aclass="reference internal"href="#paddleclasspaddle_1_1ParameterOptimizer"><spanclass="std std-ref">paddle::ParameterOptimizer</span></a></p>
<spanid="paddle::RMSPropParameterOptimizer"></span><spanclass="target"id="paddleclasspaddle_1_1RMSPropParameterOptimizer"></span><emclass="property">class </em><codeclass="descname">RMSPropParameterOptimizer</code><aclass="headerlink"href="#_CPPv2N6paddle25RMSPropParameterOptimizerE"title="Permalink to this definition">¶</a></dt>
<dd><p>Inherits from <aclass="reference internal"href="#paddleclasspaddle_1_1ParameterOptimizer"><spanclass="std std-ref">paddle::ParameterOptimizer</span></a></p>
<spanid="paddle::DecayedAdagradParameterOptimizer"></span><spanclass="target"id="paddleclasspaddle_1_1DecayedAdagradParameterOptimizer"></span><emclass="property">class </em><codeclass="descname">DecayedAdagradParameterOptimizer</code><aclass="headerlink"href="#_CPPv2N6paddle32DecayedAdagradParameterOptimizerE"title="Permalink to this definition">¶</a></dt>
<dd><p>Inherits from <aclass="reference internal"href="#paddleclasspaddle_1_1ParameterOptimizer"><spanclass="std std-ref">paddle::ParameterOptimizer</span></a></p>
<spanid="paddle::AdamParameterOptimizer"></span><spanclass="target"id="paddleclasspaddle_1_1AdamParameterOptimizer"></span><emclass="property">class </em><codeclass="descname">AdamParameterOptimizer</code><aclass="headerlink"href="#_CPPv2N6paddle22AdamParameterOptimizerE"title="Permalink to this definition">¶</a></dt>
<p>Inherits from <aclass="reference internal"href="#paddleclasspaddle_1_1ParameterOptimizer"><spanclass="std std-ref">paddle::ParameterOptimizer</span></a></p>
<spanid="paddle::AdamaxParameterOptimizer"></span><spanclass="target"id="paddleclasspaddle_1_1AdamaxParameterOptimizer"></span><emclass="property">class </em><codeclass="descname">AdamaxParameterOptimizer</code><aclass="headerlink"href="#_CPPv2N6paddle24AdamaxParameterOptimizerE"title="Permalink to this definition">¶</a></dt>
<p>Inherits from <aclass="reference internal"href="#paddleclasspaddle_1_1ParameterOptimizer"><spanclass="std std-ref">paddle::ParameterOptimizer</span></a></p>
<spanid="paddle::AddOptimizer"></span><spanclass="target"id="paddleclasspaddle_1_1AddOptimizer"></span><emclass="property">class </em><codeclass="descname">AddOptimizer</code><aclass="headerlink"href="#_CPPv2N6paddle12AddOptimizerE"title="Permalink to this definition">¶</a></dt>
<dd><p>Inherits from <aclass="reference internal"href="#paddleclasspaddle_1_1ParameterOptimizer"><spanclass="std std-ref">paddle::ParameterOptimizer</span></a></p>
<spanid="paddle::DummyOptimizer"></span><spanclass="target"id="paddleclasspaddle_1_1DummyOptimizer"></span><emclass="property">class </em><codeclass="descname">DummyOptimizer</code><aclass="headerlink"href="#_CPPv2N6paddle14DummyOptimizerE"title="Permalink to this definition">¶</a></dt>
<dd><p>Inherits from <aclass="reference internal"href="#paddleclasspaddle_1_1ParameterOptimizer"><spanclass="std std-ref">paddle::ParameterOptimizer</span></a></p>
<spanid="paddle::OptimizerWithGradientClipping"></span><spanclass="target"id="paddleclasspaddle_1_1OptimizerWithGradientClipping"></span><emclass="property">class </em><codeclass="descname">OptimizerWithGradientClipping</code><aclass="headerlink"href="#_CPPv2N6paddle29OptimizerWithGradientClippingE"title="Permalink to this definition">¶</a></dt>
<dd><p>Inherits from <aclass="reference internal"href="#paddleclasspaddle_1_1ParameterOptimizer"><spanclass="std std-ref">paddle::ParameterOptimizer</span></a></p>
<spanid="paddle::AverageOptimizer"></span><spanclass="target"id="paddleclasspaddle_1_1AverageOptimizer"></span><emclass="property">class </em><codeclass="descname">AverageOptimizer</code><aclass="headerlink"href="#_CPPv2N6paddle16AverageOptimizerE"title="Permalink to this definition">¶</a></dt>
<dd><p>Inherits from <aclass="reference internal"href="#paddleclasspaddle_1_1ParameterOptimizer"><spanclass="std std-ref">paddle::ParameterOptimizer</span></a></p>
<p>Subclassed by <aclass="reference internal"href="#paddleclasspaddle_1_1AverageSparseOptimizer"><spanclass="std std-ref">paddle::AverageSparseOptimizer</span></a></p>
<spanid="paddle::AverageSparseOptimizer"></span><spanclass="target"id="paddleclasspaddle_1_1AverageSparseOptimizer"></span><emclass="property">class </em><codeclass="descname">AverageSparseOptimizer</code><aclass="headerlink"href="#_CPPv2N6paddle22AverageSparseOptimizerE"title="Permalink to this definition">¶</a></dt>
<dd><p>Inherits from <aclass="reference internal"href="#paddleclasspaddle_1_1AverageOptimizer"><spanclass="std std-ref">paddle::AverageOptimizer</span></a></p>
<spanid="paddle::OptimizerWithRegularizer"></span><spanclass="target"id="paddleclasspaddle_1_1OptimizerWithRegularizer"></span><emclass="property">class </em><codeclass="descname">OptimizerWithRegularizer</code><aclass="headerlink"href="#_CPPv2N6paddle24OptimizerWithRegularizerE"title="Permalink to this definition">¶</a></dt>
<dd><p>Inherits from <aclass="reference internal"href="#paddleclasspaddle_1_1ParameterOptimizer"><spanclass="std std-ref">paddle::ParameterOptimizer</span></a></p>
<p>Subclassed by <aclass="reference internal"href="#paddleclasspaddle_1_1OptimizerWithRegularizerEveryNumBatches"><spanclass="std std-ref">paddle::OptimizerWithRegularizerEveryNumBatches</span></a>, <aclass="reference internal"href="#paddleclasspaddle_1_1OptimizerWithRegularizerSparse"><spanclass="std std-ref">paddle::OptimizerWithRegularizerSparse</span></a></p>
<spanid="paddle::OptimizerWithRegularizerEveryNumBatches"></span><spanclass="target"id="paddleclasspaddle_1_1OptimizerWithRegularizerEveryNumBatches"></span><emclass="property">class </em><codeclass="descname">OptimizerWithRegularizerEveryNumBatches</code><aclass="headerlink"href="#_CPPv2N6paddle39OptimizerWithRegularizerEveryNumBatchesE"title="Permalink to this definition">¶</a></dt>
<dd><p>Inherits from <aclass="reference internal"href="#paddleclasspaddle_1_1OptimizerWithRegularizer"><spanclass="std std-ref">paddle::OptimizerWithRegularizer</span></a></p>
<spanid="paddle::OptimizerWithRegularizerSparse"></span><spanclass="target"id="paddleclasspaddle_1_1OptimizerWithRegularizerSparse"></span><emclass="property">class </em><codeclass="descname">OptimizerWithRegularizerSparse</code><aclass="headerlink"href="#_CPPv2N6paddle30OptimizerWithRegularizerSparseE"title="Permalink to this definition">¶</a></dt>
<dd><p>Inherits from <aclass="reference internal"href="#paddleclasspaddle_1_1OptimizerWithRegularizer"><spanclass="std std-ref">paddle::OptimizerWithRegularizer</span></a></p>
<spanid="paddle"></span><spanclass="target"id="paddlenamespacepaddle"></span><emclass="property">namespace </em><codeclass="descname">paddle</code><aclass="headerlink"href="#_CPPv26paddle"title="Permalink to this definition">¶</a></dt>
<spanid="paddle::Weight"></span><spanclass="target"id="paddleclasspaddle_1_1Weight"></span><emclass="property">class </em><codeclass="descname">Weight</code><aclass="headerlink"href="#_CPPv2N6paddle6WeightE"title="Permalink to this definition">¶</a></dt>
<spanid="paddle::Regularizer"></span><spanclass="target"id="paddleclasspaddle_1_1Regularizer"></span><emclass="property">class </em><codeclass="descname">Regularizer</code><aclass="headerlink"href="#_CPPv2N6paddle11RegularizerE"title="Permalink to this definition">¶</a></dt>
<spanid="paddle::L1Regularizer"></span><spanclass="target"id="paddleclasspaddle_1_1L1Regularizer"></span><emclass="property">class </em><codeclass="descname">L1Regularizer</code><aclass="headerlink"href="#_CPPv2N6paddle13L1RegularizerE"title="Permalink to this definition">¶</a></dt>
<dd><p>Inherits from <aclass="reference internal"href="#paddleclasspaddle_1_1Regularizer"><spanclass="std std-ref">paddle::Regularizer</span></a></p>
<spanid="paddle::L1LrRegularizer"></span><spanclass="target"id="paddleclasspaddle_1_1L1LrRegularizer"></span><emclass="property">class </em><codeclass="descname">L1LrRegularizer</code><aclass="headerlink"href="#_CPPv2N6paddle15L1LrRegularizerE"title="Permalink to this definition">¶</a></dt>
<dd><p>Inherits from <aclass="reference internal"href="#paddleclasspaddle_1_1Regularizer"><spanclass="std std-ref">paddle::Regularizer</span></a></p>
<spanid="paddle::L2Regularizer"></span><spanclass="target"id="paddleclasspaddle_1_1L2Regularizer"></span><emclass="property">class </em><codeclass="descname">L2Regularizer</code><aclass="headerlink"href="#_CPPv2N6paddle13L2RegularizerE"title="Permalink to this definition">¶</a></dt>
<dd><p>Inherits from <aclass="reference internal"href="#paddleclasspaddle_1_1Regularizer"><spanclass="std std-ref">paddle::Regularizer</span></a></p>
<spanid="paddle::L2LrRegularizer"></span><spanclass="target"id="paddleclasspaddle_1_1L2LrRegularizer"></span><emclass="property">class </em><codeclass="descname">L2LrRegularizer</code><aclass="headerlink"href="#_CPPv2N6paddle15L2LrRegularizerE"title="Permalink to this definition">¶</a></dt>
<dd><p>Inherits from <aclass="reference internal"href="#paddleclasspaddle_1_1Regularizer"><spanclass="std std-ref">paddle::Regularizer</span></a></p>
<spanid="paddle::L1L2Regularizer"></span><spanclass="target"id="paddleclasspaddle_1_1L1L2Regularizer"></span><emclass="property">class </em><codeclass="descname">L1L2Regularizer</code><aclass="headerlink"href="#_CPPv2N6paddle15L1L2RegularizerE"title="Permalink to this definition">¶</a></dt>
<dd><p>Inherits from <aclass="reference internal"href="#paddleclasspaddle_1_1Regularizer"><spanclass="std std-ref">paddle::Regularizer</span></a></p>
<spanid="paddle::L1L2LrRegularizer"></span><spanclass="target"id="paddleclasspaddle_1_1L1L2LrRegularizer"></span><emclass="property">class </em><codeclass="descname">L1L2LrRegularizer</code><aclass="headerlink"href="#_CPPv2N6paddle17L1L2LrRegularizerE"title="Permalink to this definition">¶</a></dt>
<dd><p>Inherits from <aclass="reference internal"href="#paddleclasspaddle_1_1Regularizer"><spanclass="std std-ref">paddle::Regularizer</span></a></p>
<spanid="paddle::Argument"></span><spanclass="target"id="paddlestructpaddle_1_1Argument"></span><emclass="property">struct </em><codeclass="descname">Argument</code><aclass="headerlink"href="#_CPPv2N6paddle8ArgumentE"title="Permalink to this definition">¶</a></dt>
<spanid="paddle::Argument::SeqInfo"></span><spanclass="target"id="paddlestructpaddle_1_1Argument_1_1SeqInfo"></span><emclass="property">struct </em><codeclass="descname">SeqInfo</code><aclass="headerlink"href="#_CPPv2N6paddle8Argument7SeqInfoE"title="Permalink to this definition">¶</a></dt>
<spanid="paddle::Segment"></span><spanclass="target"id="paddlestructpaddle_1_1Segment"></span><emclass="property">struct </em><codeclass="descname">Segment</code><aclass="headerlink"href="#_CPPv2N6paddle7SegmentE"title="Permalink to this definition">¶</a></dt>
<spanid="paddle::Parameter"></span><spanclass="target"id="paddleclasspaddle_1_1Parameter"></span><emclass="property">class </em><codeclass="descname">Parameter</code><aclass="headerlink"href="#_CPPv2N6paddle9ParameterE"title="Permalink to this definition">¶</a></dt>
<spanid="paddle::Parameter::Header"></span><spanclass="target"id="paddlestructpaddle_1_1Parameter_1_1Header"></span><emclass="property">struct </em><codeclass="descname">Header</code><aclass="headerlink"href="#_CPPv2N6paddle9Parameter6HeaderE"title="Permalink to this definition">¶</a></dt>
<spanid="paddle::ParallelParameter"></span><spanclass="target"id="paddleclasspaddle_1_1ParallelParameter"></span><emclass="property">class </em><codeclass="descname">ParallelParameter</code><aclass="headerlink"href="#_CPPv2N6paddle17ParallelParameterE"title="Permalink to this definition">¶</a></dt>
<dd><p>Subclassed by <aclass="reference internal"href="#paddleclasspaddle_1_1AsyncParameter"><spanclass="std std-ref">paddle::AsyncParameter</span></a>, <aclass="reference internal"href="#paddleclasspaddle_1_1SyncParameter"><spanclass="std std-ref">paddle::SyncParameter</span></a></p>
<dd><em>#include <ParallelParameter.h></em><p>this class is designed for multi-threading training.</p>
<p>“Synchronous” means multiple GPUs calculate 1/4 mini-Batch, but will get only one gradient </p>
<p>Inherits from <aclass="reference internal"href="#paddleclasspaddle_1_1ParallelParameter"><spanclass="std std-ref">paddle::ParallelParameter</span></a></p>
<spanid="paddle::AsyncParameter"></span><spanclass="target"id="paddleclasspaddle_1_1AsyncParameter"></span><emclass="property">class </em><codeclass="descname">AsyncParameter</code><aclass="headerlink"href="#_CPPv2N6paddle14AsyncParameterE"title="Permalink to this definition">¶</a></dt>
<dd><p>Inherits from <aclass="reference internal"href="#paddleclasspaddle_1_1ParallelParameter"><spanclass="std std-ref">paddle::ParallelParameter</span></a></p>
<spanid="paddle::ParameterUpdater"></span><spanclass="target"id="paddleclasspaddle_1_1ParameterUpdater"></span><emclass="property">class </em><codeclass="descname">ParameterUpdater</code><aclass="headerlink"href="#_CPPv2N6paddle16ParameterUpdaterE"title="Permalink to this definition">¶</a></dt>
<spanid="paddle::ParameterUpdaterComposite"></span><spanclass="target"id="paddleclasspaddle_1_1ParameterUpdaterComposite"></span><emclass="property">class </em><codeclass="descname">ParameterUpdaterComposite</code><aclass="headerlink"href="#_CPPv2N6paddle25ParameterUpdaterCompositeE"title="Permalink to this definition">¶</a></dt>
<dd><p>Inherits from <aclass="reference internal"href="#paddleclasspaddle_1_1ParameterUpdater"><spanclass="std std-ref">paddle::ParameterUpdater</span></a></p>
<p>Subclassed by <aclass="reference internal"href="../../trainer/trainer.html#paddleclasspaddle_1_1SparseRemoteParameterUpdaterComposite"><spanclass="std std-ref">paddle::SparseRemoteParameterUpdaterComposite</span></a></p>
<spanclass="target"id="paddlenamespacepaddle"></span><emclass="property">namespace </em><codeclass="descname">paddle</code><aclass="headerlink"href="#_CPPv26paddle"title="Permalink to this definition">¶</a></dt>
<p>The <aclass="reference internal"href="../parameter/parameter.html#paddleclasspaddle_1_1Parameter"><spanclass="std std-ref">Parameter</span></a> Updater hooks is a group of methods invoke before ParameterUpdater::updateImpl. It can modify gradient/momentum/etc before parameter optimization. </p>
<spanid="paddle::BaseClient"></span><spanclass="target"id="paddleclasspaddle_1_1BaseClient"></span><emclass="property">class </em><codeclass="descclassname">paddle::</code><codeclass="descname">BaseClient</code><aclass="headerlink"href="#_CPPv2N6paddle10BaseClientE"title="Permalink to this definition">¶</a></dt>
<dd><p>it manages all connections to pservers. it exists two modes to manage connections to all pservers. Firstly, one connection owns two threads that separately manage to send and receive data. Secondly, each thread uses one connection for all activation in it. the first solution arms with sendThreads_/recvThreads_ and sendJobQueue_/ recvJobQueue_. the second solution use some shared thread pool to manage connections. In addition to pserver, metric learning also uses network to exchange features within multi-machines, so this class just abstracts some basic threads and queue buffer creation for them </p>
<p>Subclassed by <aclass="reference internal"href="#paddleclasspaddle_1_1ParameterClient2"><spanclass="std std-ref">paddle::ParameterClient2</span></a></p>
<spanid="paddle::BaseClient::SendJob"></span><spanclass="target"id="paddlestructpaddle_1_1BaseClient_1_1SendJob"></span><emclass="property">struct </em><codeclass="descname">SendJob</code><aclass="headerlink"href="#_CPPv2N6paddle10BaseClient7SendJobE"title="Permalink to this definition">¶</a></dt>
<spanid="paddle::ParameterClient2"></span><spanclass="target"id="paddleclasspaddle_1_1ParameterClient2"></span><emclass="property">class </em><codeclass="descclassname">paddle::</code><codeclass="descname">ParameterClient2</code><aclass="headerlink"href="#_CPPv2N6paddle16ParameterClient2E"title="Permalink to this definition">¶</a></dt>
<dd><p>The client interface for parameter server. <aclass="reference internal"href="#paddleclasspaddle_1_1ParameterClient2"><spanclass="std std-ref">ParameterClient2</span></a> supports 2 modes for managing connections to parameter servers, in the 1st mode one connection is shared by 2 threads that are separately responsible for sending and recieving activities, in the 2nd mode one connection is owned by only one thread, and all the sending and recieving activities run in that single thread. TODO(yanfei): Additional core idea to further optimizate pserver performance is to do sync-sgd based parameter level instead of pserver level. full-parallelization based parameter level for sync-sgd also can sense forwardbackward computation layer-by-layer for more deeper layer model. Firstly, pserver can do full-parallelization on all computation based parameter level instead of waiting for all gradients are finished and start to send back parameters value immediately if parameter is ready instead of waiting for all parameters value are ready Secondly, parameter client can write back parameters to GPU instead of waiting until all parameters are received to CPU host end. </p>
<p>Inherits from <aclass="reference internal"href="#paddleclasspaddle_1_1BaseClient"><spanclass="std std-ref">paddle::BaseClient</span></a></p>
<p>Inherits from <aclass="reference internal"href="../../utils/lock.html#paddleclasspaddle_1_1Thread"><spanclass="std std-ref">paddle::Thread</span></a></p>
<p>Subclassed by <aclass="reference internal"href="../server/server.html#paddleclasspaddle_1_1ProtoServer"><spanclass="std std-ref">paddle::ProtoServer</span></a></p>
<p>Inherits from <aclass="reference internal"href="../../utils/lock.html#paddleclasspaddle_1_1Thread"><spanclass="std std-ref">paddle::Thread</span></a></p>
<spanid="paddle::SocketChannel"></span><spanclass="target"id="paddleclasspaddle_1_1SocketChannel"></span><emclass="property">class </em><codeclass="descclassname">paddle::</code><codeclass="descname">SocketChannel</code><aclass="headerlink"href="#_CPPv2N6paddle13SocketChannelE"title="Permalink to this definition">¶</a></dt>
<dd><p>APIs for reading and writing byte stream data or naive iov data from the APIs both RDMA and TCP exhibits byte stream style </p>
<spanid="paddle::SocketChannel::MessageHeader"></span><spanclass="target"id="paddlestructpaddle_1_1SocketChannel_1_1MessageHeader"></span><emclass="property">struct </em><codeclass="descname">MessageHeader</code><aclass="headerlink"href="#_CPPv2N6paddle13SocketChannel13MessageHeaderE"title="Permalink to this definition">¶</a></dt>
@@ -537,7 +537,7 @@ rdma::readv and rdma::writev can take advantage of RDMA blocking offload transfe
<dtid="_CPPv2N6paddle9MsgReaderE">
<spanid="paddle::MsgReader"></span><spanclass="target"id="paddleclasspaddle_1_1MsgReader"></span><emclass="property">class </em><codeclass="descclassname">paddle::</code><codeclass="descname">MsgReader</code><aclass="headerlink"href="#_CPPv2N6paddle9MsgReaderE"title="Permalink to this definition">¶</a></dt>
<dd><p>reading a set of blocks of data from <aclass="reference internal"href="#paddleclasspaddle_1_1SocketChannel"><spanclass="std std-ref">SocketChannel</span></a>. </p>
<dd><p>It implements the rpc framework, which launchs one thread for each connection. Here define one parameter server as single TCP server binding on single port. All connections share single tcp <aclass="reference internal"href="#paddleclasspaddle_1_1ProtoServer"><spanclass="std std-ref">ProtoServer</span></a> object, each connection handles all requests from specified trainer within single worker thread. to accelerate bandwidth efficiency and harness multicore for pserver optimization to reduce pserver latency, you could launch more port for single NIC hardward with port=N(N>1) for small cluster job. </p>
<p>Inherits from <aclass="reference internal"href="../network/network.html#paddleclasspaddle_1_1SocketServer"><spanclass="std std-ref">paddle::SocketServer</span></a></p>
<p>Subclassed by <aclass="reference internal"href="#paddleclasspaddle_1_1ParameterServer2"><spanclass="std std-ref">paddle::ParameterServer2</span></a></p>
<dd><p>Client interface for the parameter server</p>
<p>it implements several rpc API for remote parameter client usage. for sync-sgd, client needs one controller thread to build connections to all pservers, these controller connections do barriers synchronization with these connections used for transfering data. each data connection uses block based fine grained synchronization to gain better scalability. Merging gradients from different trainers are concurrently executed with block units, so that some network overhead will be hidden in merging gradient. for async-sgd, the difference is that pserver will do optimization immediately if the gradients are ready, so that pserver needs to prepare separate buffer to store value for sending back to trainer to prevent from being polluted. </p>
<p>Inherits from <aclass="reference internal"href="#paddleclasspaddle_1_1ProtoServer"><spanclass="std std-ref">paddle::ProtoServer</span></a></p>
<spanid="paddle::ParameterServer2::BlockInfo"></span><spanclass="target"id="paddlestructpaddle_1_1ParameterServer2_1_1BlockInfo"></span><emclass="property">struct </em><codeclass="descname">BlockInfo</code><aclass="headerlink"href="#_CPPv2N6paddle16ParameterServer29BlockInfoE"title="Permalink to this definition">¶</a></dt>
<dd><p>to parallelize the multi-thread and multi-connnection computation at pserver, it use block unit to reduce the contention for computation, even further use block level optimizater control for each block for some special reason annotated below. </p>
<spanid="paddle::ParameterServer2::BlockKeyHash"></span><spanclass="target"id="paddlestructpaddle_1_1ParameterServer2_1_1BlockKeyHash"></span><emclass="property">struct </em><codeclass="descname">BlockKeyHash</code><aclass="headerlink"href="#_CPPv2N6paddle16ParameterServer212BlockKeyHashE"title="Permalink to this definition">¶</a></dt>
<spanid="paddle::ParameterServer2::Buffer"></span><spanclass="target"id="paddlestructpaddle_1_1ParameterServer2_1_1Buffer"></span><emclass="property">struct </em><codeclass="descname">Buffer</code><aclass="headerlink"href="#_CPPv2N6paddle16ParameterServer26BufferE"title="Permalink to this definition">¶</a></dt>
<spanid="paddle::TrainerStats"></span><spanclass="target"id="paddleclasspaddle_1_1TrainerStats"></span><emclass="property">class </em><codeclass="descclassname">paddle::</code><codeclass="descname">TrainerStats</code><aclass="headerlink"href="#_CPPv2N6paddle12TrainerStatsE"title="Permalink to this definition">¶</a></dt>
<dd><p><aclass="reference internal"href="#paddleclasspaddle_1_1TrainerStats"><spanclass="std std-ref">TrainerStats</span></a> object will statistics sample processed and total cost. </p>
<p>There are two stats in it, the ‘AvgCost’ and ‘CurrentAvgCost’. ‘AvgCost’ means cost through one pass(all mini-batches). ‘CurrentAvgCost’ means cost through one mini-batch. </p>
<p>This remote parameter updater does not use pipeline mechanism to hide copy latency from gpu to cpu buffer. In addition the overlapped between backward and communication is not supported. </p>
<p>Inherits from <aclass="reference internal"href="../parameter/update/update.html#paddleclasspaddle_1_1ParameterUpdater"><spanclass="std std-ref">paddle::ParameterUpdater</span></a></p>
<p>Subclassed by <aclass="reference internal"href="#paddleclasspaddle_1_1ConcurrentRemoteParameterUpdater"><spanclass="std std-ref">paddle::ConcurrentRemoteParameterUpdater</span></a></p>
<dd><p>This updater add additional optimization for overlapping synchronization from pservers with backward computation.</p>
<p><aclass="reference internal"href="../parameter/parameter/parameter.html#paddleclasspaddle_1_1Parameter"><spanclass="std std-ref">Parameter</span></a> can be sent to pservers when related backward stage is finished. This concurrent udpater does data copy from acceleration device to host memory aynchronously. In addition internal parameter client reads data in host memory and send them to all pservers in next stage. So this class help to pipeline device-to-host copy and host-to-network to hide network latency in backward stage. It contains separate send and recv thread for pipeline usage. </p>
<p>Inherits from <aclass="reference internal"href="#paddleclasspaddle_1_1RemoteParameterUpdater"><spanclass="std std-ref">paddle::RemoteParameterUpdater</span></a></p>
<p>For updating sparse parameters, all latest parameters are stored in pservers instead of keeping full copy at train end, so need to prefetch parameters weight value which can be changed in next-batch before doing next forwardbackward. Also, with above fact that the parameters can be stored in pserver instead of trainer, we can fetch specified parmeters if necessary, and can support huge parameters which is larger enough than the RAM size in single node.</p>
<p>Internally, this updater will direct internal parameter client to encapsulate sparse specified message for all pservers. </p>
<p>Inherits from <aclass="reference internal"href="../parameter/update/update.html#paddleclasspaddle_1_1ParameterUpdater"><spanclass="std std-ref">paddle::ParameterUpdater</span></a></p>
<p>Not all parts of one model are sparse, so it exists dense updater for normal layers while sparse updater is for sparse layers.</p>
<p>it directly call internal dense and sparse udpater individually. </p>
<p>Inherits from <aclass="reference internal"href="../parameter/update/update.html#paddleclasspaddle_1_1ParameterUpdaterComposite"><spanclass="std std-ref">paddle::ParameterUpdaterComposite</span></a></p>
<spanid="paddle::enumeration_wrapper"></span><spanclass="target"id="paddlenamespacepaddle_1_1enumeration__wrapper"></span><emclass="property">namespace </em><codeclass="descclassname">paddle::</code><codeclass="descname">enumeration_wrapper</code><aclass="headerlink"href="#_CPPv2N6paddle19enumeration_wrapperE"title="Permalink to this definition">¶</a></dt>
<spanid="paddle::Thread"></span><spanclass="target"id="paddleclasspaddle_1_1Thread"></span><emclass="property">class </em><codeclass="descclassname">paddle::</code><codeclass="descname">Thread</code><aclass="headerlink"href="#_CPPv2N6paddle6ThreadE"title="Permalink to this definition">¶</a></dt>
<dd><p><aclass="reference internal"href="#paddleclasspaddle_1_1ThreadWorker"><spanclass="std std-ref">ThreadWorker</span></a> maintains a job queue. It executes the jobs in the job queue sequentianlly in a separate thread.</p>
<p>Use <aclass="reference internal"href="#paddleclasspaddle_1_1ThreadWorker_1ac45663002b966b417524ccac3dd95b38"><spanclass="std std-ref">addJob()</span></a> to add a new job to the job queue. </p>
<p>Inherits from <aclass="reference internal"href="#paddleclasspaddle_1_1Thread"><spanclass="std std-ref">paddle::Thread</span></a></p>
<dd><p><aclass="reference internal"href="#paddleclasspaddle_1_1SyncThreadPool"><spanclass="std std-ref">SyncThreadPool</span></a> maintains a pool of threads. It executes the job use all workers in the pool.</p>
<p>Use <aclass="reference internal"href="#paddleclasspaddle_1_1SyncThreadPool_1aa477a8f126ba9433b7b7fe7e7fcd52e5"><spanclass="std std-ref">exec()</span></a> to run a new job, job complete when exec returned. Only one job can exec simultaneously.</p>
<p>Each worker has an tid whose range is [0, <aclass="reference internal"href="#paddleclasspaddle_1_1SyncThreadPool_1a137196ab4b87c59ba001b127ca197ed3"><spanclass="std std-ref">getNumThreads()</span></a>). JobFunc can use tid to divide input data. </p>
<p>Use <aclass="reference internal"href="#paddleclasspaddle_1_1MultiThreadWorker_1a9bee7e6b08a88a8e9596703796b384d2"><spanclass="std std-ref">waitResult()</span></a> to get the results until nullptr is returned. Use <aclass="reference internal"href="#paddleclasspaddle_1_1MultiThreadWorker_1a2a428608cfaf40492996951e6bf3ce15"><spanclass="std std-ref">stop()</span></a> to exit normally (<aclass="reference internal"href="#paddleclasspaddle_1_1MultiThreadWorker_1ac0993f26d31fc8b36494aa8dd18d3fb5"><spanclass="std std-ref">stopAddJob()</span></a> should be called first).</p>
<p>Force stop:</p>
<p>Use <aclass="reference internal"href="#paddleclasspaddle_1_1MultiThreadWorker_1a9eeba6482826c48450a5694c64164ac4"><spanclass="std std-ref">forceStop()</span></a> to exit forcibly even though there are remaining jobs in the job queue. </p>
<p>Use <aclass="reference internal"href="#paddleclasspaddle_1_1AsyncThreadPool_1a4848df09be835a672e9351ef5a544c81"><spanclass="std std-ref">addBatchJobs()</span></a> to add a batch of jobs. Unlike <aclass="reference internal"href="#paddleclasspaddle_1_1AsyncThreadPool_1a7389d5d38028ca8cd9cc07071f5d98b7"><spanclass="std std-ref">addJob()</span></a>‘s asynchronization, addBatchJobs will block caller’s thread until all jobs in the batch are finished.</p>
<p>Stop: Use <aclass="reference internal"href="#paddleclasspaddle_1_1AsyncThreadPool_1a7b5bbc8fd15e5f7ed52272a346f3f0d9"><spanclass="std std-ref">stop()</span></a> to stop the thread pool. Job can be added once stopped.</p>
<p>Process-wide Singleton: Use AsyncThreadPool::ProcessChannel(N) first to create N threads. Then call <aclass="reference internal"href="#paddleclasspaddle_1_1AsyncThreadPool_1a95be04dd6f37d66b4199cf2b1700444d"><spanclass="std std-ref">AsyncThreadPool::ProcessChannel()</span></a> to get the process-wide global thread pool. </p>
<spanid="paddle::BlockingQueue"></span><emclass="property">class </em><codeclass="descclassname">paddle::</code><codeclass="descname">BlockingQueue</code><aclass="headerlink"href="#_CPPv2N6paddle13BlockingQueueE"title="Permalink to this definition">¶</a></dt>
<p>Use <aclass="reference internal"href="#paddleclasspaddle_1_1RWLock_1acd26f9a9bb834456b0a92c63c7193d07"><spanclass="std std-ref">lock_shared()</span></a> to lock on read mode, other thread can get it by using the same method <aclass="reference internal"href="#paddleclasspaddle_1_1RWLock_1acd26f9a9bb834456b0a92c63c7193d07"><spanclass="std std-ref">lock_shared()</span></a>.</p>
<spanid="paddle::ReadLockGuard"></span><spanclass="target"id="paddleclasspaddle_1_1ReadLockGuard"></span><emclass="property">class </em><codeclass="descclassname">paddle::</code><codeclass="descname">ReadLockGuard</code><aclass="headerlink"href="#_CPPv2N6paddle13ReadLockGuardE"title="Permalink to this definition">¶</a></dt>
<dd><p>The <aclass="reference internal"href="#paddleclasspaddle_1_1ReadLockGuard"><spanclass="std std-ref">ReadLockGuard</span></a> is a read mode <aclass="reference internal"href="#paddleclasspaddle_1_1RWLock"><spanclass="std std-ref">RWLock</span></a> using RAII management mechanism. </p>
<spanid="paddle::SpinLock"></span><spanclass="target"id="paddleclasspaddle_1_1SpinLock"></span><emclass="property">class </em><codeclass="descclassname">paddle::</code><codeclass="descname">SpinLock</code><aclass="headerlink"href="#_CPPv2N6paddle8SpinLockE"title="Permalink to this definition">¶</a></dt>
<spanid="paddle::Semaphore"></span><spanclass="target"id="paddleclasspaddle_1_1Semaphore"></span><emclass="property">class </em><codeclass="descclassname">paddle::</code><codeclass="descname">Semaphore</code><aclass="headerlink"href="#_CPPv2N6paddle9SemaphoreE"title="Permalink to this definition">¶</a></dt>
<spanid="paddle::ThreadBarrier"></span><spanclass="target"id="paddleclasspaddle_1_1ThreadBarrier"></span><emclass="property">class </em><codeclass="descclassname">paddle::</code><codeclass="descname">ThreadBarrier</code><aclass="headerlink"href="#_CPPv2N6paddle13ThreadBarrierE"title="Permalink to this definition">¶</a></dt>
<spanid="paddle::LockedCondition"></span><spanclass="target"id="paddleclasspaddle_1_1LockedCondition"></span><emclass="property">class </em><codeclass="descclassname">paddle::</code><codeclass="descname">LockedCondition</code><aclass="headerlink"href="#_CPPv2N6paddle15LockedConditionE"title="Permalink to this definition">¶</a></dt>
<dd><p>A wrapper for condition variable with mutex. </p>
<li><codeclass="code docutils literal"><spanclass="pre">sparse_binary_vector</span></code>: sparse binary vector, most of the value is 0, and
the non zero elements are fixed to 1.</li>
<li><codeclass="code docutils literal"><spanclass="pre">sparse_float_vector</span></code>: sparse float vector, most of the value is 0, and some
<li><codeclass="code docutils literal"><spanclass="pre">sparse_vector</span></code>: sparse float vector, most of the value is 0, and some
non zero elements can be any float value. They are given by the user.</li>
<li><codeclass="code docutils literal"><spanclass="pre">integer</span></code>: an integer scalar, that is especially used for label or word index.</li>
@@ -106,24 +106,21 @@ MNIST classification problem. A complete sample code could be found at
<p>The module that does the most of the job is py_paddle.swig_paddle, it’s
generated by SWIG and has complete documents, for more details you can use
python’s <codeclass="code docutils literal"><spanclass="pre">help()</span></code> function. Let’s walk through the above python script:</p>
<ul>
<li><pclass="first">At the beginning, use <codeclass="code docutils literal"><spanclass="pre">swig_paddle.initPaddle()</span></code> to initialize
<ulclass="simple">
<li>At the beginning, use <codeclass="code docutils literal"><spanclass="pre">swig_paddle.initPaddle()</span></code> to initialize
PaddlePaddle with command line arguments, for more about command line arguments
see <aclass="reference external"href="../cmd_argument/detail_introduction.html">Command Line Arguments</a>.</p>
</li>
<li><pclass="first">Parse the configuration file that is used in training with <codeclass="code docutils literal"><spanclass="pre">parse_config()</span></code>.
see <aclass="reference external"href="../cmd_argument/detail_introduction.html">Command Line Arguments</a>.</li>
<li>Parse the configuration file that is used in training with <codeclass="code docutils literal"><spanclass="pre">parse_config()</span></code>.
Because data to predict with always have no label, and output of prediction work
normally is the output layer rather than the cost layer, so you should modify
the configuration file accordingly before using it in the prediction work.</p>
</li>
<li><pclass="first">Create a neural network with
the configuration file accordingly before using it in the prediction work.</li>
<li>Create a neural network with
<codeclass="code docutils literal"><spanclass="pre">swig_paddle.GradientMachine.createFromConfigproto()</span></code>, which takes the
parsed configuration <codeclass="code docutils literal"><spanclass="pre">conf.model_config</span></code> as argument. Then load the
trained parameters from the model with <codeclass="code docutils literal"><spanclass="pre">network.loadParameters()</span></code>.</p>
</li>
trained parameters from the model with <codeclass="code docutils literal"><spanclass="pre">network.loadParameters()</span></code>.</li>
<li><dlclass="first docutils">
<dt>Create a data converter object of utility class <codeclass="code docutils literal"><spanclass="pre">DataProviderConverter</span></code>.</dt>
<dd><ulclass="first last simple">
<dd><ulclass="first last">
<li>Note: As swig_paddle can only accept C++ matrices, we offer a utility
class DataProviderConverter that can accept the same input data with
PyDataProvider2, for more information please refer to document
...
...
@@ -132,9 +129,8 @@ of <a class="reference external" href="../data_provider/pydataprovider2.html">Py
</dd>
</dl>
</li>
<li><pclass="first">Do the prediction with <codeclass="code docutils literal"><spanclass="pre">forwardTest()</span></code>, which takes the converted
input data and outputs the activations of the output layer.</p>
</li>
<li>Do the prediction with <codeclass="code docutils literal"><spanclass="pre">forwardTest()</span></code>, which takes the converted
input data and outputs the activations of the output layer.</li>
