PaddlePaddle divides the description of neural network computation into two stages: compile time and runtime. At compile time, the neural network computation is described as a `ProgramDesc` whereas at runtime an `Executor` interprets the `ProgramDesc` to compute the operations.
PaddlePaddle use proto message to describe compile time program because
PaddlePaddle uses proto message to describe compile time program because :
1. The computation program description must be serializable and saved in a file.
1. During distributed training, the sreialized program will be sent to multiple workers. It should also be possible to break the program into different components, each of which can be executed on different workers.
1. During distributed training, the serialized program will be sent to multiple workers. It should also be possible to break the program into different components, each of which can be executed on a different worker.
The computation `Program` consists of nested `Blocks`. Each `Block` will consist of data(i.e. `Variable`) and `Operations`. The concept to represent them is in the table below.
...
...
@@ -14,28 +14,33 @@ The computation `Program` consists of nested `Blocks`. Each `Block` will consist
|Operation|OpDesc(proto)|Operator(cpp)|
## Definition of VarDesc
## Definition of VarType
A VarDesc should have a name, and value. The are two kinds of variable type in compile time, they are `LoDTensor` and `SelectedRows`.
A VarDesc should have a name, type and whether or not it is persistable. The are different kinds of variable types supported in PaddlePaddle, apart from the POD_Types like: `LOD_TENSOR`, `SELECTED_ROWS`, `FEED_MINIBATCH`, `FETCH_LIST`, `STEP_SCOPES`, `LOD_RANK_TABLE`, `LOD_TENSOR_ARRAY`, `PLACE_LIST`, `READER` and `CHANNEL`. These are declared inside `VarType`. A `VarDesc` then looks as the following:
<spanid="background"></span><h1>Background<aclass="headerlink"href="#background"title="Permalink to this headline">¶</a></h1>
<p>PaddlePaddle divides the description of neural network computation into two stages: compile time and runtime. At compile time, the neural network computation is described as a <codeclass="docutils literal"><spanclass="pre">ProgramDesc</span></code> whereas at runtime an <codeclass="docutils literal"><spanclass="pre">Executor</span></code> interprets the <codeclass="docutils literal"><spanclass="pre">ProgramDesc</span></code> to compute the operations.</p>
<p>PaddlePaddle use proto message to describe compile time program because</p>
<p>PaddlePaddle uses proto message to describe compile time program because :</p>
<olclass="simple">
<li>The computation program description must be serializable and saved in a file.</li>
<li>During distributed training, the sreialized program will be sent to multiple workers. It should also be possible to break the program into different components, each of which can be executed on different workers.</li>
<li>During distributed training, the serialized program will be sent to multiple workers. It should also be possible to break the program into different components, each of which can be executed on a different worker.</li>
</ol>
<p>The computation <codeclass="docutils literal"><spanclass="pre">Program</span></code> consists of nested <codeclass="docutils literal"><spanclass="pre">Blocks</span></code>. Each <codeclass="docutils literal"><spanclass="pre">Block</span></code> will consist of data(i.e. <codeclass="docutils literal"><spanclass="pre">Variable</span></code>) and <codeclass="docutils literal"><spanclass="pre">Operations</span></code>. The concept to represent them is in the table below.</p>
<p>| |compile time|runtime|
...
...
@@ -191,26 +191,29 @@
|Data|VarDesc(proto)|Variable(cpp)|
|Operation|OpDesc(proto)|Operator(cpp)|</p>
</div>
<divclass="section"id="definition-of-vardesc">
<spanid="definition-of-vardesc"></span><h1>Definition of VarDesc<aclass="headerlink"href="#definition-of-vardesc"title="Permalink to this headline">¶</a></h1>
<p>A VarDesc should have a name, and value. The are two kinds of variable type in compile time, they are <codeclass="docutils literal"><spanclass="pre">LoDTensor</span></code> and <codeclass="docutils literal"><spanclass="pre">SelectedRows</span></code>.</p>
<divclass="section"id="definition-of-vartype">
<spanid="definition-of-vartype"></span><h1>Definition of VarType<aclass="headerlink"href="#definition-of-vartype"title="Permalink to this headline">¶</a></h1>
<p>A VarDesc should have a name, type and whether or not it is persistable. The are different kinds of variable types supported in PaddlePaddle, apart from the POD_Types like: <codeclass="docutils literal"><spanclass="pre">LOD_TENSOR</span></code>, <codeclass="docutils literal"><spanclass="pre">SELECTED_ROWS</span></code>, <codeclass="docutils literal"><spanclass="pre">FEED_MINIBATCH</span></code>, <codeclass="docutils literal"><spanclass="pre">FETCH_LIST</span></code>, <codeclass="docutils literal"><spanclass="pre">STEP_SCOPES</span></code>, <codeclass="docutils literal"><spanclass="pre">LOD_RANK_TABLE</span></code>, <codeclass="docutils literal"><spanclass="pre">LOD_TENSOR_ARRAY</span></code>, <codeclass="docutils literal"><spanclass="pre">PLACE_LIST</span></code>, <codeclass="docutils literal"><spanclass="pre">READER</span></code> and <codeclass="docutils literal"><spanclass="pre">CHANNEL</span></code>. These are declared inside <codeclass="docutils literal"><spanclass="pre">VarType</span></code>. A <codeclass="docutils literal"><spanclass="pre">VarDesc</span></code> then looks as the following:</p>
<spanid="definition-of-tensordesc"></span><h1>Definition of TensorDesc<aclass="headerlink"href="#definition-of-tensordesc"title="Permalink to this headline">¶</a></h1>
<spanclass="k">repeated</span><spanclass="kt">int64</span><spanclass="na">dims</span><spanclass="o">=</span><spanclass="mi">2</span><spanclass="p">;</span><spanclass="c1">// [UNK, 640, 480] is saved as [-1, 640, 480]</span>
<spanclass="p">}</span>
</pre></div>
</div>
<p>The <codeclass="docutils literal"><spanclass="pre">Type</span></code> here comes from the enum defined inside of <codeclass="docutils literal"><spanclass="pre">VarType</span></code> :</p>
<spanclass="k">repeated</span><spanclass="kt">int64</span><spanclass="na">dims</span><spanclass="o">=</span><spanclass="mi">2</span><spanclass="p">;</span><spanclass="c1">// [UNK, 640, 480] is saved as [-1, 640, 480]</span>
<spanclass="c1">// Other types that may need additional descriptions</span>
<spanid="definition-of-lodtensordesc"></span><h1>Definition of LodTensorDesc<aclass="headerlink"href="#definition-of-lodtensordesc"title="Permalink to this headline">¶</a></h1>
PaddlePaddle divides the description of neural network computation into two stages: compile time and runtime. At compile time, the neural network computation is described as a `ProgramDesc` whereas at runtime an `Executor` interprets the `ProgramDesc` to compute the operations.
PaddlePaddle use proto message to describe compile time program because
PaddlePaddle uses proto message to describe compile time program because :
1. The computation program description must be serializable and saved in a file.
1. During distributed training, the sreialized program will be sent to multiple workers. It should also be possible to break the program into different components, each of which can be executed on different workers.
1. During distributed training, the serialized program will be sent to multiple workers. It should also be possible to break the program into different components, each of which can be executed on a different worker.
The computation `Program` consists of nested `Blocks`. Each `Block` will consist of data(i.e. `Variable`) and `Operations`. The concept to represent them is in the table below.
...
...
@@ -14,28 +14,33 @@ The computation `Program` consists of nested `Blocks`. Each `Block` will consist
|Operation|OpDesc(proto)|Operator(cpp)|
## Definition of VarDesc
## Definition of VarType
A VarDesc should have a name, and value. The are two kinds of variable type in compile time, they are `LoDTensor` and `SelectedRows`.
A VarDesc should have a name, type and whether or not it is persistable. The are different kinds of variable types supported in PaddlePaddle, apart from the POD_Types like: `LOD_TENSOR`, `SELECTED_ROWS`, `FEED_MINIBATCH`, `FETCH_LIST`, `STEP_SCOPES`, `LOD_RANK_TABLE`, `LOD_TENSOR_ARRAY`, `PLACE_LIST`, `READER` and `CHANNEL`. These are declared inside `VarType`. A `VarDesc` then looks as the following:
<p>PaddlePaddle divides the description of neural network computation into two stages: compile time and runtime. At compile time, the neural network computation is described as a <codeclass="docutils literal"><spanclass="pre">ProgramDesc</span></code> whereas at runtime an <codeclass="docutils literal"><spanclass="pre">Executor</span></code> interprets the <codeclass="docutils literal"><spanclass="pre">ProgramDesc</span></code> to compute the operations.</p>
<p>PaddlePaddle use proto message to describe compile time program because</p>
<p>PaddlePaddle uses proto message to describe compile time program because :</p>
<olclass="simple">
<li>The computation program description must be serializable and saved in a file.</li>
<li>During distributed training, the sreialized program will be sent to multiple workers. It should also be possible to break the program into different components, each of which can be executed on different workers.</li>
<li>During distributed training, the serialized program will be sent to multiple workers. It should also be possible to break the program into different components, each of which can be executed on a different worker.</li>
</ol>
<p>The computation <codeclass="docutils literal"><spanclass="pre">Program</span></code> consists of nested <codeclass="docutils literal"><spanclass="pre">Blocks</span></code>. Each <codeclass="docutils literal"><spanclass="pre">Block</span></code> will consist of data(i.e. <codeclass="docutils literal"><spanclass="pre">Variable</span></code>) and <codeclass="docutils literal"><spanclass="pre">Operations</span></code>. The concept to represent them is in the table below.</p>
<p>| |compile time|runtime|
...
...
@@ -210,26 +210,29 @@
|Data|VarDesc(proto)|Variable(cpp)|
|Operation|OpDesc(proto)|Operator(cpp)|</p>
</div>
<divclass="section"id="definition-of-vardesc">
<spanid="definition-of-vardesc"></span><h1>Definition of VarDesc<aclass="headerlink"href="#definition-of-vardesc"title="永久链接至标题">¶</a></h1>
<p>A VarDesc should have a name, and value. The are two kinds of variable type in compile time, they are <codeclass="docutils literal"><spanclass="pre">LoDTensor</span></code> and <codeclass="docutils literal"><spanclass="pre">SelectedRows</span></code>.</p>
<divclass="section"id="definition-of-vartype">
<spanid="definition-of-vartype"></span><h1>Definition of VarType<aclass="headerlink"href="#definition-of-vartype"title="永久链接至标题">¶</a></h1>
<p>A VarDesc should have a name, type and whether or not it is persistable. The are different kinds of variable types supported in PaddlePaddle, apart from the POD_Types like: <codeclass="docutils literal"><spanclass="pre">LOD_TENSOR</span></code>, <codeclass="docutils literal"><spanclass="pre">SELECTED_ROWS</span></code>, <codeclass="docutils literal"><spanclass="pre">FEED_MINIBATCH</span></code>, <codeclass="docutils literal"><spanclass="pre">FETCH_LIST</span></code>, <codeclass="docutils literal"><spanclass="pre">STEP_SCOPES</span></code>, <codeclass="docutils literal"><spanclass="pre">LOD_RANK_TABLE</span></code>, <codeclass="docutils literal"><spanclass="pre">LOD_TENSOR_ARRAY</span></code>, <codeclass="docutils literal"><spanclass="pre">PLACE_LIST</span></code>, <codeclass="docutils literal"><spanclass="pre">READER</span></code> and <codeclass="docutils literal"><spanclass="pre">CHANNEL</span></code>. These are declared inside <codeclass="docutils literal"><spanclass="pre">VarType</span></code>. A <codeclass="docutils literal"><spanclass="pre">VarDesc</span></code> then looks as the following:</p>
<spanclass="k">repeated</span><spanclass="kt">int64</span><spanclass="na">dims</span><spanclass="o">=</span><spanclass="mi">2</span><spanclass="p">;</span><spanclass="c1">// [UNK, 640, 480] is saved as [-1, 640, 480]</span>
<spanclass="p">}</span>
</pre></div>
</div>
<p>The <codeclass="docutils literal"><spanclass="pre">Type</span></code> here comes from the enum defined inside of <codeclass="docutils literal"><spanclass="pre">VarType</span></code> :</p>
<spanclass="k">repeated</span><spanclass="kt">int64</span><spanclass="na">dims</span><spanclass="o">=</span><spanclass="mi">2</span><spanclass="p">;</span><spanclass="c1">// [UNK, 640, 480] is saved as [-1, 640, 480]</span>
<spanclass="c1">// Other types that may need additional descriptions</span>
<spanid="definition-of-lodtensordesc"></span><h1>Definition of LodTensorDesc<aclass="headerlink"href="#definition-of-lodtensordesc"title="永久链接至标题">¶</a></h1>
