Skip to content

P
Paddle

PaddlePaddle / Paddle
大约 2 年 前同步成功

通知 2325
Star 20933
Fork 5424
P
Paddle
“a337e5b4779fd5e8821080dc67fea42c5018c83c”上不存在“develop/doc_cn/howto/cluster/multi_cluster/openmpi_cn.html”
提交 294a9139 编写于 作者: T Travis CI
浏览文件
操作

Deploy to GitHub Pages: d1479d93

上级 8f5718f5
隐藏空白更改
内联 并排
Showing with 94 addition and 90 deletion
develop/doc/_sources/design/refactorization.md.txt
浏览文件 @ 294a9139
...@@ -17,22 +17,22 @@ The goals of refactoring include: ...@@ -17,22 +17,22 @@ The goals of refactoring include:
1. A graph is composed of *variables* and *operators*. 1. A graph is composed of *variables* and *operators*.
1. The description of graphs must be capable of being serialized/deserialized, so that: 1. The description of graphs must be serializable/deserializable, so that:
1. It can to be sent to the cloud for distributed execution, and 1. It can be sent to the cloud for distributed execution, and
1. It can be sent to clients for mobile or enterprise deployment. 1. It can be sent to clients for mobile or enterprise deployment.
1. The Python program does the following steps 1. The Python program does two things
1. *compilation*: run a Python program to generate a protobuf message representation of the graph and send it to 1. *Compilation* runs a Python program to generate a protobuf message representation of the graph and send it to
1. the C++ library `libpaddle.so` for local execution, 1. the C++ library `libpaddle.so` for local execution,
1. the master process of a distributed training job for training, or 1. the master process of a distributed training job for training, or
1. the server process of a Kubernetes serving job for distributed serving. 1. the server process of a Kubernetes serving job for distributed serving.
1. *execution*: execute the graph by constructing instances of class [`Variable`](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/framework/variable.h#L24) and [`OperatorBase`](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/framework/operator.h#L70), according to the protobuf message. 1. *Execution* executes the graph by constructing instances of class [`Variable`](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/framework/variable.h#L24) and [`OperatorBase`](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/framework/operator.h#L70), according to the protobuf message.
## Description and Realization of Computation Graph ## Description and Realization of Computation Graph
At compile time, the Python program generates a protobuf message representation of the graph, or the description of the graph. At compile time, the Python program generates a protobuf message representation of the graph, or a description of the graph.
At runtime, the C++ program realizes the graph and runs it. At runtime, the C++ program realizes the graph and runs it.
...@@ -42,11 +42,11 @@ At runtime, the C++ program realizes the graph and runs it. ...@@ -42,11 +42,11 @@ At runtime, the C++ program realizes the graph and runs it.
|Operation|[OpDesc](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/framework/framework.proto#L35)|[Operator](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/framework/operator.h#L64)| |Operation|[OpDesc](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/framework/framework.proto#L35)|[Operator](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/framework/operator.h#L64)|
|Block|BlockDesc|Block| |Block|BlockDesc|Block|
The word *graph* is interchangeable with *block* in this document. A graph represents computation steps and local variables similar to a C++/Java program block, or a pair of parentheses(`{` and `}`). The word *graph* is interchangeable with *block* in this document. A graph consists of computation steps and local variables similar to a C++/Java program block, or a pair of parentheses(`{` and `}`).
## Compilation and Execution ## Compilation and Execution
1. Run an application Python program to describe the graph. In particular, the Python application program does the following: 1. Run a Python program to describe the graph. In particular, the Python application program does the following:
1. Create `VarDesc` to represent local/intermediate variables, 1. Create `VarDesc` to represent local/intermediate variables,
1. Create operators and set attributes, 1. Create operators and set attributes,
...@@ -54,10 +54,10 @@ The word *graph* is interchangeable with *block* in this document. A graph repr ...@@ -54,10 +54,10 @@ The word *graph* is interchangeable with *block* in this document. A graph repr
1. Infer the type and the shape of variables, 1. Infer the type and the shape of variables,
1. Plan memory-reuse for variables, 1. Plan memory-reuse for variables,
1. Generate the backward graph 1. Generate the backward graph
1. Optimize the computation graph. 1. Add optimization operators to the computation graph.
1. Potentially, split the graph for distributed training. 1. Optionally, split the graph for distributed training.
1. The invocation of `train` or [`infer`](https://github.com/PaddlePaddle/Paddle/blob/develop/python/paddle/v2/inference.py#L108) methods in the application Python program does the following: 1. The invocation of `train` or [`infer`](https://github.com/PaddlePaddle/Paddle/blob/develop/python/paddle/v2/inference.py#L108) methods in the Python program does the following:
1. Create a new Scope instance in the [scope hierarchy](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/scope.md) for each run of a block, 1. Create a new Scope instance in the [scope hierarchy](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/scope.md) for each run of a block,
1. realize local variables defined in the BlockDesc message in the new scope, 1. realize local variables defined in the BlockDesc message in the new scope,
...@@ -107,8 +107,8 @@ Compile Time -> IR -> Runtime ...@@ -107,8 +107,8 @@ Compile Time -> IR -> Runtime
![class_diagram](http://api.paddlepaddle.org/graphviz?dot=https://gist.githubusercontent.com/reyoung/53df507f6749762675dff3e7ce53372f/raw/dd598e8f1976f5759f58af5e5ef94738a6b2e661/op.dot) ![class_diagram](http://api.paddlepaddle.org/graphviz?dot=https://gist.githubusercontent.com/reyoung/53df507f6749762675dff3e7ce53372f/raw/dd598e8f1976f5759f58af5e5ef94738a6b2e661/op.dot)
* `Operator` is the fundamental building block of the user interface. * `Operator` is the fundamental building block of the user interface.
* Operator stores input/output variable names, and attributes. * Operator stores input/output variable names and attributes.
* The `InferShape` interface is used to infer the shape of the output variable shapes based on the shapes of the input variables. * The `InferShape` interface is used to infer the shape of the output variables based on the shapes of the input variables.
* Use `Run` to compute the `output` variables from the `input` variables. * Use `Run` to compute the `output` variables from the `input` variables.
--- ---
...@@ -139,7 +139,7 @@ Compile Time -> IR -> Runtime ...@@ -139,7 +139,7 @@ Compile Time -> IR -> Runtime
* Limit the number of `tensor.device(dev) = ` in your code. * Limit the number of `tensor.device(dev) = ` in your code.
* `thrust::transform` and `std::transform`. * `thrust::transform` and `std::transform`.
* `thrust` has the same API as C++ standard library. Using `transform`, one can quickly implement customized element-wise kernels. * `thrust` has the same API as C++ standard library. Using `transform`, one can quickly implement customized element-wise kernels.
* `thrust` also has more complex APIs, like `scan`, `reduce`, `reduce_by_key`. * `thrust`, in addition, supports more complex APIs, like `scan`, `reduce`, `reduce_by_key`.
* Hand-writing `GPUKernel` and `CPU` code * Hand-writing `GPUKernel` and `CPU` code
* Do not write in header (`.h`) files. CPU Kernel should be in cpp source (`.cc`) and GPU kernels should be in cuda (`.cu`) files. (GCC cannot compile GPU code.) * Do not write in header (`.h`) files. CPU Kernel should be in cpp source (`.cc`) and GPU kernels should be in cuda (`.cu`) files. (GCC cannot compile GPU code.)
--- ---
...@@ -185,10 +185,10 @@ Make sure the registration process is executed and linked. ...@@ -185,10 +185,10 @@ Make sure the registration process is executed and linked.
1. Write an Op class and its gradient Op class, if required. 1. Write an Op class and its gradient Op class, if required.
2. Write an Op maker class. In the constructor of this class, describe the inputs, outputs and attributes of the operator. 2. Write an Op maker class. In the constructor of this class, describe the inputs, outputs and attributes of the operator.
3. Invoke the macro `REGISTER_OP`. This macro will 3. Invoke the macro `REGISTER_OP`. This macro will
1. Call maker class to complete the `proto` and the `checker` 1. Call maker class to complete `proto` and `checker`
2. Using the completed `proto` and `checker`, it will add a new key-value pair to the `OpInfoMap` 2. Using the completed `proto` and `checker`, it will add a new key-value pair to the `OpInfoMap`
4. Invoke the `USE` macro in which the Op is used, to make sure that it is linked. 4. Invoke the `USE` macro in which the Op is used to make sure that it is linked.
--- ---
# Backward Module (1/2) # Backward Module (1/2)
...@@ -199,13 +199,14 @@ Make sure the registration process is executed and linked. ...@@ -199,13 +199,14 @@ Make sure the registration process is executed and linked.
--- ---
# Backward Module (2/2) # Backward Module (2/2)
### Build Backward Network ### Build Backward Network
- **Input**: graph of forward operators - **Input**: a graph of forward operators
- **Output**: graph of backward operators - **Output**: a graph of backward operators
- **Corner cases in construction** - **Corner cases in construction**
- Shared Variables => insert an `Add` operator to combine gradients - Shared Variables => insert an `Add` operator to combine gradients
- No Gradient => insert a `fill_zero_grad` operator - No Gradient => insert a `fill_zero_grad` operator
- Recursive NetOp => call `Backward` recursively - Recursive NetOp => call `Backward` recursively
- RNN Op => recursively call `Backward` on stepnet - RNN Op => recursively call `Backward` on stepnet
- RNN Op => recursively call `Backward` on stepnet
--- ---
...@@ -215,10 +216,10 @@ Make sure the registration process is executed and linked. ...@@ -215,10 +216,10 @@ Make sure the registration process is executed and linked.
* Only dims and data pointers are stored in `Tensor`. * Only dims and data pointers are stored in `Tensor`.
* All operations on `Tensor` are written in `Operator` or global functions. * All operations on `Tensor` are written in `Operator` or global functions.
* Variable length Tensor design [LoDTensor](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/framework/lod_tensor.md) * Variable length Tensor design [LoDTensor](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/framework/lod_tensor.md)
* `Variable` instances are the inputs and the outputs of an operator. Not just `Tensor`. * `Variable` instances are the inputs and the outputs of an operator, not just `Tensor`.
* `step_scopes` in RNN is a variable and not a tensor. * `step_scopes` in RNN is a variable and not a tensor.
* `Scope` is where variables are stores. * `Scope` is where variables are stored.
* map<string `variable_name`, Variable> * map<string `var name`, Variable>
* `Scope` has a hierarchical structure. The local scope can get variables from its parent scope. * `Scope` has a hierarchical structure. The local scope can get variables from its parent scope.
--- ---
...@@ -246,7 +247,7 @@ Make sure the registration process is executed and linked. ...@@ -246,7 +247,7 @@ Make sure the registration process is executed and linked.
--- ---
# Control the migration quality # Control the migration quality
- Compare the performance of migrated models with old ones. - Compare the performance of migrated models with old ones.
- Follow the google C++ style - Follow the google C++ style guide.
- Build the automatic workflow of generating Python/C++ documentations. - Build the automatic workflow of generating Python/C++ documentations.
- The documentation of layers and ops should be written inside the code. - The documentation of layers and ops should be written inside the code.
- Take the documentation quality into account when submitting pull requests. - Take the documentation quality into account when submitting pull requests.
......
develop/doc/design/refactorization.html
浏览文件 @ 294a9139
...@@ -193,49 +193,49 @@ ...@@ -193,49 +193,49 @@
<li>Please refer to <a class="reference external" href="https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/graph.md">computation graphs</a> for a concrete example.</li> <li>Please refer to <a class="reference external" href="https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/graph.md">computation graphs</a> for a concrete example.</li>
<li>Users write Python programs to describe the graphs and run them (locally or remotely).</li> <li>Users write Python programs to describe the graphs and run them (locally or remotely).</li>
<li>A graph is composed of <em>variables</em> and <em>operators</em>.</li> <li>A graph is composed of <em>variables</em> and <em>operators</em>.</li>
<li>The description of graphs must be capable of being serialized/deserialized, so that:<ol> <li>The description of graphs must be serializable/deserializable, so that:<ol>
<li>It can to be sent to the cloud for distributed execution, and</li> <li>It can be sent to the cloud for distributed execution, and</li>
<li>It can be sent to clients for mobile or enterprise deployment.</li> <li>It can be sent to clients for mobile or enterprise deployment.</li>
</ol> </ol>
</li> </li>
<li>The Python program does the following steps<ol> <li>The Python program does two things<ol>
<li><em>compilation</em>: run a Python program to generate a protobuf message representation of the graph and send it to<ol> <li><em>Compilation</em> runs a Python program to generate a protobuf message representation of the graph and send it to<ol>
<li>the C++ library <code class="docutils literal"><span class="pre">libpaddle.so</span></code> for local execution,</li> <li>the C++ library <code class="docutils literal"><span class="pre">libpaddle.so</span></code> for local execution,</li>
<li>the master process of a distributed training job for training, or</li> <li>the master process of a distributed training job for training, or</li>
<li>the server process of a Kubernetes serving job for distributed serving.</li> <li>the server process of a Kubernetes serving job for distributed serving.</li>
</ol> </ol>
</li> </li>
<li><em>execution</em>: execute the graph by constructing instances of class <a class="reference external" href="https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/framework/variable.h#L24"><code class="docutils literal"><span class="pre">Variable</span></code></a> and <a class="reference external" href="https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/framework/operator.h#L70"><code class="docutils literal"><span class="pre">OperatorBase</span></code></a>, according to the protobuf message.</li> <li><em>Execution</em> executes the graph by constructing instances of class <a class="reference external" href="https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/framework/variable.h#L24"><code class="docutils literal"><span class="pre">Variable</span></code></a> and <a class="reference external" href="https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/framework/operator.h#L70"><code class="docutils literal"><span class="pre">OperatorBase</span></code></a>, according to the protobuf message.</li>
</ol> </ol>
</li> </li>
</ol> </ol>
</div> </div>
<div class="section" id="description-and-realization-of-computation-graph"> <div class="section" id="description-and-realization-of-computation-graph">
<span id="description-and-realization-of-computation-graph"></span><h2>Description and Realization of Computation Graph<a class="headerlink" href="#description-and-realization-of-computation-graph" title="Permalink to this headline"></a></h2> <span id="description-and-realization-of-computation-graph"></span><h2>Description and Realization of Computation Graph<a class="headerlink" href="#description-and-realization-of-computation-graph" title="Permalink to this headline"></a></h2>
<p>At compile time, the Python program generates a protobuf message representation of the graph, or the description of the graph.</p> <p>At compile time, the Python program generates a protobuf message representation of the graph, or a description of the graph.</p>
<p>At runtime, the C++ program realizes the graph and runs it.</p> <p>At runtime, the C++ program realizes the graph and runs it.</p>
<p>| | Representation (protobuf messages) | Realization (C++ class objects) | <p>| | Representation (protobuf messages) | Realization (C++ class objects) |
|&#8212;|&#8212;|&#8212;| |&#8212;|&#8212;|&#8212;|
|Data|<a class="reference external" href="https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/framework/framework.proto#L107">VarDesc</a>|<a class="reference external" href="https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/framework/variable.h#L24">Variable</a>| |Data|<a class="reference external" href="https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/framework/framework.proto#L107">VarDesc</a>|<a class="reference external" href="https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/framework/variable.h#L24">Variable</a>|
|Operation|<a class="reference external" href="https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/framework/framework.proto#L35">OpDesc</a>|<a class="reference external" href="https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/framework/operator.h#L64">Operator</a>| |Operation|<a class="reference external" href="https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/framework/framework.proto#L35">OpDesc</a>|<a class="reference external" href="https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/framework/operator.h#L64">Operator</a>|
|Block|BlockDesc|Block|</p> |Block|BlockDesc|Block|</p>
<p>The word <em>graph</em> is interchangeable with <em>block</em> in this document. A graph represents computation steps and local variables similar to a C++/Java program block, or a pair of parentheses(<code class="docutils literal"><span class="pre">{</span></code> and <code class="docutils literal"><span class="pre">}</span></code>).</p> <p>The word <em>graph</em> is interchangeable with <em>block</em> in this document. A graph consists of computation steps and local variables similar to a C++/Java program block, or a pair of parentheses(<code class="docutils literal"><span class="pre">{</span></code> and <code class="docutils literal"><span class="pre">}</span></code>).</p>
</div> </div>
<div class="section" id="compilation-and-execution"> <div class="section" id="compilation-and-execution">
<span id="compilation-and-execution"></span><h2>Compilation and Execution<a class="headerlink" href="#compilation-and-execution" title="Permalink to this headline"></a></h2> <span id="compilation-and-execution"></span><h2>Compilation and Execution<a class="headerlink" href="#compilation-and-execution" title="Permalink to this headline"></a></h2>
<ol class="simple"> <ol class="simple">
<li>Run an application Python program to describe the graph. In particular, the Python application program does the following:<ol> <li>Run a Python program to describe the graph. In particular, the Python application program does the following:<ol>
<li>Create <code class="docutils literal"><span class="pre">VarDesc</span></code> to represent local/intermediate variables,</li> <li>Create <code class="docutils literal"><span class="pre">VarDesc</span></code> to represent local/intermediate variables,</li>
<li>Create operators and set attributes,</li> <li>Create operators and set attributes,</li>
<li>Validate attribute values,</li> <li>Validate attribute values,</li>
<li>Infer the type and the shape of variables,</li> <li>Infer the type and the shape of variables,</li>
<li>Plan memory-reuse for variables,</li> <li>Plan memory-reuse for variables,</li>
<li>Generate the backward graph</li> <li>Generate the backward graph</li>
<li>Optimize the computation graph.</li> <li>Add optimization operators to the computation graph.</li>
<li>Potentially, split the graph for distributed training.</li> <li>Optionally, split the graph for distributed training.</li>
</ol> </ol>
</li> </li>
<li>The invocation of <code class="docutils literal"><span class="pre">train</span></code> or <a class="reference external" href="https://github.com/PaddlePaddle/Paddle/blob/develop/python/paddle/v2/inference.py#L108"><code class="docutils literal"><span class="pre">infer</span></code></a> methods in the application Python program does the following:<ol> <li>The invocation of <code class="docutils literal"><span class="pre">train</span></code> or <a class="reference external" href="https://github.com/PaddlePaddle/Paddle/blob/develop/python/paddle/v2/inference.py#L108"><code class="docutils literal"><span class="pre">infer</span></code></a> methods in the Python program does the following:<ol>
<li>Create a new Scope instance in the <a class="reference external" href="https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/scope.md">scope hierarchy</a> for each run of a block,<ol> <li>Create a new Scope instance in the <a class="reference external" href="https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/scope.md">scope hierarchy</a> for each run of a block,<ol>
<li>realize local variables defined in the BlockDesc message in the new scope,</li> <li>realize local variables defined in the BlockDesc message in the new scope,</li>
<li>a scope is similar to the stack frame in programming languages,</li> <li>a scope is similar to the stack frame in programming languages,</li>
...@@ -298,8 +298,8 @@ ...@@ -298,8 +298,8 @@
<p><img alt="class_diagram" src="http://api.paddlepaddle.org/graphviz?dot=https://gist.githubusercontent.com/reyoung/53df507f6749762675dff3e7ce53372f/raw/dd598e8f1976f5759f58af5e5ef94738a6b2e661/op.dot" /></p> <p><img alt="class_diagram" src="http://api.paddlepaddle.org/graphviz?dot=https://gist.githubusercontent.com/reyoung/53df507f6749762675dff3e7ce53372f/raw/dd598e8f1976f5759f58af5e5ef94738a6b2e661/op.dot" /></p>
<ul class="simple"> <ul class="simple">
<li><code class="docutils literal"><span class="pre">Operator</span></code> is the fundamental building block of the user interface.<ul> <li><code class="docutils literal"><span class="pre">Operator</span></code> is the fundamental building block of the user interface.<ul>
<li>Operator stores input/output variable names, and attributes.</li> <li>Operator stores input/output variable names and attributes.</li>
<li>The <code class="docutils literal"><span class="pre">InferShape</span></code> interface is used to infer the shape of the output variable shapes based on the shapes of the input variables.</li> <li>The <code class="docutils literal"><span class="pre">InferShape</span></code> interface is used to infer the shape of the output variables based on the shapes of the input variables.</li>
<li>Use <code class="docutils literal"><span class="pre">Run</span></code> to compute the <code class="docutils literal"><span class="pre">output</span></code> variables from the <code class="docutils literal"><span class="pre">input</span></code> variables.</li> <li>Use <code class="docutils literal"><span class="pre">Run</span></code> to compute the <code class="docutils literal"><span class="pre">output</span></code> variables from the <code class="docutils literal"><span class="pre">input</span></code> variables.</li>
</ul> </ul>
</li> </li>
...@@ -343,7 +343,7 @@ ...@@ -343,7 +343,7 @@
</li> </li>
<li><code class="docutils literal"><span class="pre">thrust::transform</span></code> and <code class="docutils literal"><span class="pre">std::transform</span></code>.<ul> <li><code class="docutils literal"><span class="pre">thrust::transform</span></code> and <code class="docutils literal"><span class="pre">std::transform</span></code>.<ul>
<li><code class="docutils literal"><span class="pre">thrust</span></code> has the same API as C++ standard library. Using <code class="docutils literal"><span class="pre">transform</span></code>, one can quickly implement customized element-wise kernels.</li> <li><code class="docutils literal"><span class="pre">thrust</span></code> has the same API as C++ standard library. Using <code class="docutils literal"><span class="pre">transform</span></code>, one can quickly implement customized element-wise kernels.</li>
<li><code class="docutils literal"><span class="pre">thrust</span></code> also has more complex APIs, like <code class="docutils literal"><span class="pre">scan</span></code>, <code class="docutils literal"><span class="pre">reduce</span></code>, <code class="docutils literal"><span class="pre">reduce_by_key</span></code>.</li> <li><code class="docutils literal"><span class="pre">thrust</span></code>, in addition, supports more complex APIs, like <code class="docutils literal"><span class="pre">scan</span></code>, <code class="docutils literal"><span class="pre">reduce</span></code>, <code class="docutils literal"><span class="pre">reduce_by_key</span></code>.</li>
</ul> </ul>
</li> </li>
<li>Hand-writing <code class="docutils literal"><span class="pre">GPUKernel</span></code> and <code class="docutils literal"><span class="pre">CPU</span></code> code<ul> <li>Hand-writing <code class="docutils literal"><span class="pre">GPUKernel</span></code> and <code class="docutils literal"><span class="pre">CPU</span></code> code<ul>
...@@ -405,11 +405,11 @@ ...@@ -405,11 +405,11 @@
<li>Write an Op class and its gradient Op class, if required.</li> <li>Write an Op class and its gradient Op class, if required.</li>
<li>Write an Op maker class. In the constructor of this class, describe the inputs, outputs and attributes of the operator.</li> <li>Write an Op maker class. In the constructor of this class, describe the inputs, outputs and attributes of the operator.</li>
<li>Invoke the macro <code class="docutils literal"><span class="pre">REGISTER_OP</span></code>. This macro will<ol> <li>Invoke the macro <code class="docutils literal"><span class="pre">REGISTER_OP</span></code>. This macro will<ol>
<li>Call maker class to complete the <code class="docutils literal"><span class="pre">proto</span></code> and the <code class="docutils literal"><span class="pre">checker</span></code></li> <li>Call maker class to complete <code class="docutils literal"><span class="pre">proto</span></code> and <code class="docutils literal"><span class="pre">checker</span></code></li>
<li>Using the completed <code class="docutils literal"><span class="pre">proto</span></code> and <code class="docutils literal"><span class="pre">checker</span></code>, it will add a new key-value pair to the <code class="docutils literal"><span class="pre">OpInfoMap</span></code></li> <li>Using the completed <code class="docutils literal"><span class="pre">proto</span></code> and <code class="docutils literal"><span class="pre">checker</span></code>, it will add a new key-value pair to the <code class="docutils literal"><span class="pre">OpInfoMap</span></code></li>
</ol> </ol>
</li> </li>
<li>Invoke the <code class="docutils literal"><span class="pre">USE</span></code> macro in which the Op is used, to make sure that it is linked.</li> <li>Invoke the <code class="docutils literal"><span class="pre">USE</span></code> macro in which the Op is used to make sure that it is linked.</li>
</ol> </ol>
</div> </div>
<hr class="docutils" /> <hr class="docutils" />
...@@ -429,13 +429,14 @@ ...@@ -429,13 +429,14 @@
<div class="section" id="build-backward-network"> <div class="section" id="build-backward-network">
<span id="build-backward-network"></span><h2>Build Backward Network<a class="headerlink" href="#build-backward-network" title="Permalink to this headline"></a></h2> <span id="build-backward-network"></span><h2>Build Backward Network<a class="headerlink" href="#build-backward-network" title="Permalink to this headline"></a></h2>
<ul class="simple"> <ul class="simple">
<li><strong>Input</strong>: graph of forward operators</li> <li><strong>Input</strong>: a graph of forward operators</li>
<li><strong>Output</strong>: graph of backward operators</li> <li><strong>Output</strong>: a graph of backward operators</li>
<li><strong>Corner cases in construction</strong><ul> <li><strong>Corner cases in construction</strong><ul>
<li>Shared Variables =&gt; insert an <code class="docutils literal"><span class="pre">Add</span></code> operator to combine gradients</li> <li>Shared Variables =&gt; insert an <code class="docutils literal"><span class="pre">Add</span></code> operator to combine gradients</li>
<li>No Gradient =&gt; insert a <code class="docutils literal"><span class="pre">fill_zero_grad</span></code> operator</li> <li>No Gradient =&gt; insert a <code class="docutils literal"><span class="pre">fill_zero_grad</span></code> operator</li>
<li>Recursive NetOp =&gt; call <code class="docutils literal"><span class="pre">Backward</span></code> recursively</li> <li>Recursive NetOp =&gt; call <code class="docutils literal"><span class="pre">Backward</span></code> recursively</li>
<li>RNN Op =&gt; recursively call <code class="docutils literal"><span class="pre">Backward</span></code> on stepnet</li> <li>RNN Op =&gt; recursively call <code class="docutils literal"><span class="pre">Backward</span></code> on stepnet</li>
<li>RNN Op =&gt; recursively call <code class="docutils literal"><span class="pre">Backward</span></code> on stepnet</li>
</ul> </ul>
</li> </li>
</ul> </ul>
...@@ -451,12 +452,12 @@ ...@@ -451,12 +452,12 @@
<li>Variable length Tensor design <a class="reference external" href="https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/framework/lod_tensor.md">LoDTensor</a></li> <li>Variable length Tensor design <a class="reference external" href="https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/framework/lod_tensor.md">LoDTensor</a></li>
</ul> </ul>
</li> </li>
<li><code class="docutils literal"><span class="pre">Variable</span></code> instances are the inputs and the outputs of an operator. Not just <code class="docutils literal"><span class="pre">Tensor</span></code>.<ul> <li><code class="docutils literal"><span class="pre">Variable</span></code> instances are the inputs and the outputs of an operator, not just <code class="docutils literal"><span class="pre">Tensor</span></code>.<ul>
<li><code class="docutils literal"><span class="pre">step_scopes</span></code> in RNN is a variable and not a tensor.</li> <li><code class="docutils literal"><span class="pre">step_scopes</span></code> in RNN is a variable and not a tensor.</li>
</ul> </ul>
</li> </li>
<li><code class="docutils literal"><span class="pre">Scope</span></code> is where variables are stores.<ul> <li><code class="docutils literal"><span class="pre">Scope</span></code> is where variables are stored.<ul>
<li>map&lt;string <code class="docutils literal"><span class="pre">variable_name</span></code>, Variable&gt;</li> <li>map&lt;string <code class="docutils literal"><span class="pre">var</span> <span class="pre">name</span></code>, Variable&gt;</li>
<li><code class="docutils literal"><span class="pre">Scope</span></code> has a hierarchical structure. The local scope can get variables from its parent scope.</li> <li><code class="docutils literal"><span class="pre">Scope</span></code> has a hierarchical structure. The local scope can get variables from its parent scope.</li>
</ul> </ul>
</li> </li>
...@@ -503,7 +504,7 @@ ...@@ -503,7 +504,7 @@
<span id="control-the-migration-quality"></span><h1>Control the migration quality<a class="headerlink" href="#control-the-migration-quality" title="Permalink to this headline"></a></h1> <span id="control-the-migration-quality"></span><h1>Control the migration quality<a class="headerlink" href="#control-the-migration-quality" title="Permalink to this headline"></a></h1>
<ul class="simple"> <ul class="simple">
<li>Compare the performance of migrated models with old ones.</li> <li>Compare the performance of migrated models with old ones.</li>
<li>Follow the google C++ style</li> <li>Follow the google C++ style guide.</li>
<li>Build the automatic workflow of generating Python/C++ documentations.<ul> <li>Build the automatic workflow of generating Python/C++ documentations.<ul>
<li>The documentation of layers and ops should be written inside the code.</li> <li>The documentation of layers and ops should be written inside the code.</li>
<li>Take the documentation quality into account when submitting pull requests.</li> <li>Take the documentation quality into account when submitting pull requests.</li>
......
develop/doc/searchindex.js
浏览文件 @ 294a9139
因为 它太大了无法显示 source diff 。你可以改为 查看blob。
develop/doc_cn/_sources/design/refactorization.md.txt
浏览文件 @ 294a9139
...@@ -17,22 +17,22 @@ The goals of refactoring include: ...@@ -17,22 +17,22 @@ The goals of refactoring include:
1. A graph is composed of *variables* and *operators*. 1. A graph is composed of *variables* and *operators*.
1. The description of graphs must be capable of being serialized/deserialized, so that: 1. The description of graphs must be serializable/deserializable, so that:
1. It can to be sent to the cloud for distributed execution, and 1. It can be sent to the cloud for distributed execution, and
1. It can be sent to clients for mobile or enterprise deployment. 1. It can be sent to clients for mobile or enterprise deployment.
1. The Python program does the following steps 1. The Python program does two things
1. *compilation*: run a Python program to generate a protobuf message representation of the graph and send it to 1. *Compilation* runs a Python program to generate a protobuf message representation of the graph and send it to
1. the C++ library `libpaddle.so` for local execution, 1. the C++ library `libpaddle.so` for local execution,
1. the master process of a distributed training job for training, or 1. the master process of a distributed training job for training, or
1. the server process of a Kubernetes serving job for distributed serving. 1. the server process of a Kubernetes serving job for distributed serving.
1. *execution*: execute the graph by constructing instances of class [`Variable`](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/framework/variable.h#L24) and [`OperatorBase`](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/framework/operator.h#L70), according to the protobuf message. 1. *Execution* executes the graph by constructing instances of class [`Variable`](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/framework/variable.h#L24) and [`OperatorBase`](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/framework/operator.h#L70), according to the protobuf message.
## Description and Realization of Computation Graph ## Description and Realization of Computation Graph
At compile time, the Python program generates a protobuf message representation of the graph, or the description of the graph. At compile time, the Python program generates a protobuf message representation of the graph, or a description of the graph.
At runtime, the C++ program realizes the graph and runs it. At runtime, the C++ program realizes the graph and runs it.
...@@ -42,11 +42,11 @@ At runtime, the C++ program realizes the graph and runs it. ...@@ -42,11 +42,11 @@ At runtime, the C++ program realizes the graph and runs it.
|Operation|[OpDesc](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/framework/framework.proto#L35)|[Operator](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/framework/operator.h#L64)| |Operation|[OpDesc](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/framework/framework.proto#L35)|[Operator](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/framework/operator.h#L64)|
|Block|BlockDesc|Block| |Block|BlockDesc|Block|
The word *graph* is interchangeable with *block* in this document. A graph represents computation steps and local variables similar to a C++/Java program block, or a pair of parentheses(`{` and `}`). The word *graph* is interchangeable with *block* in this document. A graph consists of computation steps and local variables similar to a C++/Java program block, or a pair of parentheses(`{` and `}`).
## Compilation and Execution ## Compilation and Execution
1. Run an application Python program to describe the graph. In particular, the Python application program does the following: 1. Run a Python program to describe the graph. In particular, the Python application program does the following:
1. Create `VarDesc` to represent local/intermediate variables, 1. Create `VarDesc` to represent local/intermediate variables,
1. Create operators and set attributes, 1. Create operators and set attributes,
...@@ -54,10 +54,10 @@ The word *graph* is interchangeable with *block* in this document. A graph repr ...@@ -54,10 +54,10 @@ The word *graph* is interchangeable with *block* in this document. A graph repr
1. Infer the type and the shape of variables, 1. Infer the type and the shape of variables,
1. Plan memory-reuse for variables, 1. Plan memory-reuse for variables,
1. Generate the backward graph 1. Generate the backward graph
1. Optimize the computation graph. 1. Add optimization operators to the computation graph.
1. Potentially, split the graph for distributed training. 1. Optionally, split the graph for distributed training.
1. The invocation of `train` or [`infer`](https://github.com/PaddlePaddle/Paddle/blob/develop/python/paddle/v2/inference.py#L108) methods in the application Python program does the following: 1. The invocation of `train` or [`infer`](https://github.com/PaddlePaddle/Paddle/blob/develop/python/paddle/v2/inference.py#L108) methods in the Python program does the following:
1. Create a new Scope instance in the [scope hierarchy](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/scope.md) for each run of a block, 1. Create a new Scope instance in the [scope hierarchy](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/scope.md) for each run of a block,
1. realize local variables defined in the BlockDesc message in the new scope, 1. realize local variables defined in the BlockDesc message in the new scope,
...@@ -107,8 +107,8 @@ Compile Time -> IR -> Runtime ...@@ -107,8 +107,8 @@ Compile Time -> IR -> Runtime
![class_diagram](http://api.paddlepaddle.org/graphviz?dot=https://gist.githubusercontent.com/reyoung/53df507f6749762675dff3e7ce53372f/raw/dd598e8f1976f5759f58af5e5ef94738a6b2e661/op.dot) ![class_diagram](http://api.paddlepaddle.org/graphviz?dot=https://gist.githubusercontent.com/reyoung/53df507f6749762675dff3e7ce53372f/raw/dd598e8f1976f5759f58af5e5ef94738a6b2e661/op.dot)
* `Operator` is the fundamental building block of the user interface. * `Operator` is the fundamental building block of the user interface.
* Operator stores input/output variable names, and attributes. * Operator stores input/output variable names and attributes.
* The `InferShape` interface is used to infer the shape of the output variable shapes based on the shapes of the input variables. * The `InferShape` interface is used to infer the shape of the output variables based on the shapes of the input variables.
* Use `Run` to compute the `output` variables from the `input` variables. * Use `Run` to compute the `output` variables from the `input` variables.
--- ---
...@@ -139,7 +139,7 @@ Compile Time -> IR -> Runtime ...@@ -139,7 +139,7 @@ Compile Time -> IR -> Runtime
* Limit the number of `tensor.device(dev) = ` in your code. * Limit the number of `tensor.device(dev) = ` in your code.
* `thrust::transform` and `std::transform`. * `thrust::transform` and `std::transform`.
* `thrust` has the same API as C++ standard library. Using `transform`, one can quickly implement customized element-wise kernels. * `thrust` has the same API as C++ standard library. Using `transform`, one can quickly implement customized element-wise kernels.
* `thrust` also has more complex APIs, like `scan`, `reduce`, `reduce_by_key`. * `thrust`, in addition, supports more complex APIs, like `scan`, `reduce`, `reduce_by_key`.
* Hand-writing `GPUKernel` and `CPU` code * Hand-writing `GPUKernel` and `CPU` code
* Do not write in header (`.h`) files. CPU Kernel should be in cpp source (`.cc`) and GPU kernels should be in cuda (`.cu`) files. (GCC cannot compile GPU code.) * Do not write in header (`.h`) files. CPU Kernel should be in cpp source (`.cc`) and GPU kernels should be in cuda (`.cu`) files. (GCC cannot compile GPU code.)
--- ---
...@@ -185,10 +185,10 @@ Make sure the registration process is executed and linked. ...@@ -185,10 +185,10 @@ Make sure the registration process is executed and linked.
1. Write an Op class and its gradient Op class, if required. 1. Write an Op class and its gradient Op class, if required.
2. Write an Op maker class. In the constructor of this class, describe the inputs, outputs and attributes of the operator. 2. Write an Op maker class. In the constructor of this class, describe the inputs, outputs and attributes of the operator.
3. Invoke the macro `REGISTER_OP`. This macro will 3. Invoke the macro `REGISTER_OP`. This macro will
1. Call maker class to complete the `proto` and the `checker` 1. Call maker class to complete `proto` and `checker`
2. Using the completed `proto` and `checker`, it will add a new key-value pair to the `OpInfoMap` 2. Using the completed `proto` and `checker`, it will add a new key-value pair to the `OpInfoMap`
4. Invoke the `USE` macro in which the Op is used, to make sure that it is linked. 4. Invoke the `USE` macro in which the Op is used to make sure that it is linked.
--- ---
# Backward Module (1/2) # Backward Module (1/2)
...@@ -199,13 +199,14 @@ Make sure the registration process is executed and linked. ...@@ -199,13 +199,14 @@ Make sure the registration process is executed and linked.
--- ---
# Backward Module (2/2) # Backward Module (2/2)
### Build Backward Network ### Build Backward Network
- **Input**: graph of forward operators - **Input**: a graph of forward operators
- **Output**: graph of backward operators - **Output**: a graph of backward operators
- **Corner cases in construction** - **Corner cases in construction**
- Shared Variables => insert an `Add` operator to combine gradients - Shared Variables => insert an `Add` operator to combine gradients
- No Gradient => insert a `fill_zero_grad` operator - No Gradient => insert a `fill_zero_grad` operator
- Recursive NetOp => call `Backward` recursively - Recursive NetOp => call `Backward` recursively
- RNN Op => recursively call `Backward` on stepnet - RNN Op => recursively call `Backward` on stepnet
- RNN Op => recursively call `Backward` on stepnet
--- ---
...@@ -215,10 +216,10 @@ Make sure the registration process is executed and linked. ...@@ -215,10 +216,10 @@ Make sure the registration process is executed and linked.
* Only dims and data pointers are stored in `Tensor`. * Only dims and data pointers are stored in `Tensor`.
* All operations on `Tensor` are written in `Operator` or global functions. * All operations on `Tensor` are written in `Operator` or global functions.
* Variable length Tensor design [LoDTensor](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/framework/lod_tensor.md) * Variable length Tensor design [LoDTensor](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/framework/lod_tensor.md)
* `Variable` instances are the inputs and the outputs of an operator. Not just `Tensor`. * `Variable` instances are the inputs and the outputs of an operator, not just `Tensor`.
* `step_scopes` in RNN is a variable and not a tensor. * `step_scopes` in RNN is a variable and not a tensor.
* `Scope` is where variables are stores. * `Scope` is where variables are stored.
* map<string `variable_name`, Variable> * map<string `var name`, Variable>
* `Scope` has a hierarchical structure. The local scope can get variables from its parent scope. * `Scope` has a hierarchical structure. The local scope can get variables from its parent scope.
--- ---
...@@ -246,7 +247,7 @@ Make sure the registration process is executed and linked. ...@@ -246,7 +247,7 @@ Make sure the registration process is executed and linked.
--- ---
# Control the migration quality # Control the migration quality
- Compare the performance of migrated models with old ones. - Compare the performance of migrated models with old ones.
- Follow the google C++ style - Follow the google C++ style guide.
- Build the automatic workflow of generating Python/C++ documentations. - Build the automatic workflow of generating Python/C++ documentations.
- The documentation of layers and ops should be written inside the code. - The documentation of layers and ops should be written inside the code.
- Take the documentation quality into account when submitting pull requests. - Take the documentation quality into account when submitting pull requests.
......
develop/doc_cn/design/refactorization.html
浏览文件 @ 294a9139
...@@ -207,49 +207,49 @@ ...@@ -207,49 +207,49 @@
<li>Please refer to <a class="reference external" href="https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/graph.md">computation graphs</a> for a concrete example.</li> <li>Please refer to <a class="reference external" href="https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/graph.md">computation graphs</a> for a concrete example.</li>
<li>Users write Python programs to describe the graphs and run them (locally or remotely).</li> <li>Users write Python programs to describe the graphs and run them (locally or remotely).</li>
<li>A graph is composed of <em>variables</em> and <em>operators</em>.</li> <li>A graph is composed of <em>variables</em> and <em>operators</em>.</li>
<li>The description of graphs must be capable of being serialized/deserialized, so that:<ol> <li>The description of graphs must be serializable/deserializable, so that:<ol>
<li>It can to be sent to the cloud for distributed execution, and</li> <li>It can be sent to the cloud for distributed execution, and</li>
<li>It can be sent to clients for mobile or enterprise deployment.</li> <li>It can be sent to clients for mobile or enterprise deployment.</li>
</ol> </ol>
</li> </li>
<li>The Python program does the following steps<ol> <li>The Python program does two things<ol>
<li><em>compilation</em>: run a Python program to generate a protobuf message representation of the graph and send it to<ol> <li><em>Compilation</em> runs a Python program to generate a protobuf message representation of the graph and send it to<ol>
<li>the C++ library <code class="docutils literal"><span class="pre">libpaddle.so</span></code> for local execution,</li> <li>the C++ library <code class="docutils literal"><span class="pre">libpaddle.so</span></code> for local execution,</li>
<li>the master process of a distributed training job for training, or</li> <li>the master process of a distributed training job for training, or</li>
<li>the server process of a Kubernetes serving job for distributed serving.</li> <li>the server process of a Kubernetes serving job for distributed serving.</li>
</ol> </ol>
</li> </li>
<li><em>execution</em>: execute the graph by constructing instances of class <a class="reference external" href="https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/framework/variable.h#L24"><code class="docutils literal"><span class="pre">Variable</span></code></a> and <a class="reference external" href="https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/framework/operator.h#L70"><code class="docutils literal"><span class="pre">OperatorBase</span></code></a>, according to the protobuf message.</li> <li><em>Execution</em> executes the graph by constructing instances of class <a class="reference external" href="https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/framework/variable.h#L24"><code class="docutils literal"><span class="pre">Variable</span></code></a> and <a class="reference external" href="https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/framework/operator.h#L70"><code class="docutils literal"><span class="pre">OperatorBase</span></code></a>, according to the protobuf message.</li>
</ol> </ol>
</li> </li>
</ol> </ol>
</div> </div>
<div class="section" id="description-and-realization-of-computation-graph"> <div class="section" id="description-and-realization-of-computation-graph">
<span id="description-and-realization-of-computation-graph"></span><h2>Description and Realization of Computation Graph<a class="headerlink" href="#description-and-realization-of-computation-graph" title="永久链接至标题"></a></h2> <span id="description-and-realization-of-computation-graph"></span><h2>Description and Realization of Computation Graph<a class="headerlink" href="#description-and-realization-of-computation-graph" title="永久链接至标题"></a></h2>
<p>At compile time, the Python program generates a protobuf message representation of the graph, or the description of the graph.</p> <p>At compile time, the Python program generates a protobuf message representation of the graph, or a description of the graph.</p>
<p>At runtime, the C++ program realizes the graph and runs it.</p> <p>At runtime, the C++ program realizes the graph and runs it.</p>
<p>| | Representation (protobuf messages) | Realization (C++ class objects) | <p>| | Representation (protobuf messages) | Realization (C++ class objects) |
|&#8212;|&#8212;|&#8212;| |&#8212;|&#8212;|&#8212;|
|Data|<a class="reference external" href="https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/framework/framework.proto#L107">VarDesc</a>|<a class="reference external" href="https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/framework/variable.h#L24">Variable</a>| |Data|<a class="reference external" href="https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/framework/framework.proto#L107">VarDesc</a>|<a class="reference external" href="https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/framework/variable.h#L24">Variable</a>|
|Operation|<a class="reference external" href="https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/framework/framework.proto#L35">OpDesc</a>|<a class="reference external" href="https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/framework/operator.h#L64">Operator</a>| |Operation|<a class="reference external" href="https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/framework/framework.proto#L35">OpDesc</a>|<a class="reference external" href="https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/framework/operator.h#L64">Operator</a>|
|Block|BlockDesc|Block|</p> |Block|BlockDesc|Block|</p>
<p>The word <em>graph</em> is interchangeable with <em>block</em> in this document. A graph represents computation steps and local variables similar to a C++/Java program block, or a pair of parentheses(<code class="docutils literal"><span class="pre">{</span></code> and <code class="docutils literal"><span class="pre">}</span></code>).</p> <p>The word <em>graph</em> is interchangeable with <em>block</em> in this document. A graph consists of computation steps and local variables similar to a C++/Java program block, or a pair of parentheses(<code class="docutils literal"><span class="pre">{</span></code> and <code class="docutils literal"><span class="pre">}</span></code>).</p>
</div> </div>
<div class="section" id="compilation-and-execution"> <div class="section" id="compilation-and-execution">
<span id="compilation-and-execution"></span><h2>Compilation and Execution<a class="headerlink" href="#compilation-and-execution" title="永久链接至标题"></a></h2> <span id="compilation-and-execution"></span><h2>Compilation and Execution<a class="headerlink" href="#compilation-and-execution" title="永久链接至标题"></a></h2>
<ol class="simple"> <ol class="simple">
<li>Run an application Python program to describe the graph. In particular, the Python application program does the following:<ol> <li>Run a Python program to describe the graph. In particular, the Python application program does the following:<ol>
<li>Create <code class="docutils literal"><span class="pre">VarDesc</span></code> to represent local/intermediate variables,</li> <li>Create <code class="docutils literal"><span class="pre">VarDesc</span></code> to represent local/intermediate variables,</li>
<li>Create operators and set attributes,</li> <li>Create operators and set attributes,</li>
<li>Validate attribute values,</li> <li>Validate attribute values,</li>
<li>Infer the type and the shape of variables,</li> <li>Infer the type and the shape of variables,</li>
<li>Plan memory-reuse for variables,</li> <li>Plan memory-reuse for variables,</li>
<li>Generate the backward graph</li> <li>Generate the backward graph</li>
<li>Optimize the computation graph.</li> <li>Add optimization operators to the computation graph.</li>
<li>Potentially, split the graph for distributed training.</li> <li>Optionally, split the graph for distributed training.</li>
</ol> </ol>
</li> </li>
<li>The invocation of <code class="docutils literal"><span class="pre">train</span></code> or <a class="reference external" href="https://github.com/PaddlePaddle/Paddle/blob/develop/python/paddle/v2/inference.py#L108"><code class="docutils literal"><span class="pre">infer</span></code></a> methods in the application Python program does the following:<ol> <li>The invocation of <code class="docutils literal"><span class="pre">train</span></code> or <a class="reference external" href="https://github.com/PaddlePaddle/Paddle/blob/develop/python/paddle/v2/inference.py#L108"><code class="docutils literal"><span class="pre">infer</span></code></a> methods in the Python program does the following:<ol>
<li>Create a new Scope instance in the <a class="reference external" href="https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/scope.md">scope hierarchy</a> for each run of a block,<ol> <li>Create a new Scope instance in the <a class="reference external" href="https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/scope.md">scope hierarchy</a> for each run of a block,<ol>
<li>realize local variables defined in the BlockDesc message in the new scope,</li> <li>realize local variables defined in the BlockDesc message in the new scope,</li>
<li>a scope is similar to the stack frame in programming languages,</li> <li>a scope is similar to the stack frame in programming languages,</li>
...@@ -312,8 +312,8 @@ ...@@ -312,8 +312,8 @@
<p><img alt="class_diagram" src="http://api.paddlepaddle.org/graphviz?dot=https://gist.githubusercontent.com/reyoung/53df507f6749762675dff3e7ce53372f/raw/dd598e8f1976f5759f58af5e5ef94738a6b2e661/op.dot" /></p> <p><img alt="class_diagram" src="http://api.paddlepaddle.org/graphviz?dot=https://gist.githubusercontent.com/reyoung/53df507f6749762675dff3e7ce53372f/raw/dd598e8f1976f5759f58af5e5ef94738a6b2e661/op.dot" /></p>
<ul class="simple"> <ul class="simple">
<li><code class="docutils literal"><span class="pre">Operator</span></code> is the fundamental building block of the user interface.<ul> <li><code class="docutils literal"><span class="pre">Operator</span></code> is the fundamental building block of the user interface.<ul>
<li>Operator stores input/output variable names, and attributes.</li> <li>Operator stores input/output variable names and attributes.</li>
<li>The <code class="docutils literal"><span class="pre">InferShape</span></code> interface is used to infer the shape of the output variable shapes based on the shapes of the input variables.</li> <li>The <code class="docutils literal"><span class="pre">InferShape</span></code> interface is used to infer the shape of the output variables based on the shapes of the input variables.</li>
<li>Use <code class="docutils literal"><span class="pre">Run</span></code> to compute the <code class="docutils literal"><span class="pre">output</span></code> variables from the <code class="docutils literal"><span class="pre">input</span></code> variables.</li> <li>Use <code class="docutils literal"><span class="pre">Run</span></code> to compute the <code class="docutils literal"><span class="pre">output</span></code> variables from the <code class="docutils literal"><span class="pre">input</span></code> variables.</li>
</ul> </ul>
</li> </li>
...@@ -357,7 +357,7 @@ ...@@ -357,7 +357,7 @@
</li> </li>
<li><code class="docutils literal"><span class="pre">thrust::transform</span></code> and <code class="docutils literal"><span class="pre">std::transform</span></code>.<ul> <li><code class="docutils literal"><span class="pre">thrust::transform</span></code> and <code class="docutils literal"><span class="pre">std::transform</span></code>.<ul>
<li><code class="docutils literal"><span class="pre">thrust</span></code> has the same API as C++ standard library. Using <code class="docutils literal"><span class="pre">transform</span></code>, one can quickly implement customized element-wise kernels.</li> <li><code class="docutils literal"><span class="pre">thrust</span></code> has the same API as C++ standard library. Using <code class="docutils literal"><span class="pre">transform</span></code>, one can quickly implement customized element-wise kernels.</li>
<li><code class="docutils literal"><span class="pre">thrust</span></code> also has more complex APIs, like <code class="docutils literal"><span class="pre">scan</span></code>, <code class="docutils literal"><span class="pre">reduce</span></code>, <code class="docutils literal"><span class="pre">reduce_by_key</span></code>.</li> <li><code class="docutils literal"><span class="pre">thrust</span></code>, in addition, supports more complex APIs, like <code class="docutils literal"><span class="pre">scan</span></code>, <code class="docutils literal"><span class="pre">reduce</span></code>, <code class="docutils literal"><span class="pre">reduce_by_key</span></code>.</li>
</ul> </ul>
</li> </li>
<li>Hand-writing <code class="docutils literal"><span class="pre">GPUKernel</span></code> and <code class="docutils literal"><span class="pre">CPU</span></code> code<ul> <li>Hand-writing <code class="docutils literal"><span class="pre">GPUKernel</span></code> and <code class="docutils literal"><span class="pre">CPU</span></code> code<ul>
...@@ -419,11 +419,11 @@ ...@@ -419,11 +419,11 @@
<li>Write an Op class and its gradient Op class, if required.</li> <li>Write an Op class and its gradient Op class, if required.</li>
<li>Write an Op maker class. In the constructor of this class, describe the inputs, outputs and attributes of the operator.</li> <li>Write an Op maker class. In the constructor of this class, describe the inputs, outputs and attributes of the operator.</li>
<li>Invoke the macro <code class="docutils literal"><span class="pre">REGISTER_OP</span></code>. This macro will<ol> <li>Invoke the macro <code class="docutils literal"><span class="pre">REGISTER_OP</span></code>. This macro will<ol>
<li>Call maker class to complete the <code class="docutils literal"><span class="pre">proto</span></code> and the <code class="docutils literal"><span class="pre">checker</span></code></li> <li>Call maker class to complete <code class="docutils literal"><span class="pre">proto</span></code> and <code class="docutils literal"><span class="pre">checker</span></code></li>
<li>Using the completed <code class="docutils literal"><span class="pre">proto</span></code> and <code class="docutils literal"><span class="pre">checker</span></code>, it will add a new key-value pair to the <code class="docutils literal"><span class="pre">OpInfoMap</span></code></li> <li>Using the completed <code class="docutils literal"><span class="pre">proto</span></code> and <code class="docutils literal"><span class="pre">checker</span></code>, it will add a new key-value pair to the <code class="docutils literal"><span class="pre">OpInfoMap</span></code></li>
</ol> </ol>
</li> </li>
<li>Invoke the <code class="docutils literal"><span class="pre">USE</span></code> macro in which the Op is used, to make sure that it is linked.</li> <li>Invoke the <code class="docutils literal"><span class="pre">USE</span></code> macro in which the Op is used to make sure that it is linked.</li>
</ol> </ol>
</div> </div>
<hr class="docutils" /> <hr class="docutils" />
...@@ -443,13 +443,14 @@ ...@@ -443,13 +443,14 @@
<div class="section" id="build-backward-network"> <div class="section" id="build-backward-network">
<span id="build-backward-network"></span><h2>Build Backward Network<a class="headerlink" href="#build-backward-network" title="永久链接至标题"></a></h2> <span id="build-backward-network"></span><h2>Build Backward Network<a class="headerlink" href="#build-backward-network" title="永久链接至标题"></a></h2>
<ul class="simple"> <ul class="simple">
<li><strong>Input</strong>: graph of forward operators</li> <li><strong>Input</strong>: a graph of forward operators</li>
<li><strong>Output</strong>: graph of backward operators</li> <li><strong>Output</strong>: a graph of backward operators</li>
<li><strong>Corner cases in construction</strong><ul> <li><strong>Corner cases in construction</strong><ul>
<li>Shared Variables =&gt; insert an <code class="docutils literal"><span class="pre">Add</span></code> operator to combine gradients</li> <li>Shared Variables =&gt; insert an <code class="docutils literal"><span class="pre">Add</span></code> operator to combine gradients</li>
<li>No Gradient =&gt; insert a <code class="docutils literal"><span class="pre">fill_zero_grad</span></code> operator</li> <li>No Gradient =&gt; insert a <code class="docutils literal"><span class="pre">fill_zero_grad</span></code> operator</li>
<li>Recursive NetOp =&gt; call <code class="docutils literal"><span class="pre">Backward</span></code> recursively</li> <li>Recursive NetOp =&gt; call <code class="docutils literal"><span class="pre">Backward</span></code> recursively</li>
<li>RNN Op =&gt; recursively call <code class="docutils literal"><span class="pre">Backward</span></code> on stepnet</li> <li>RNN Op =&gt; recursively call <code class="docutils literal"><span class="pre">Backward</span></code> on stepnet</li>
<li>RNN Op =&gt; recursively call <code class="docutils literal"><span class="pre">Backward</span></code> on stepnet</li>
</ul> </ul>
</li> </li>
</ul> </ul>
...@@ -465,12 +466,12 @@ ...@@ -465,12 +466,12 @@
<li>Variable length Tensor design <a class="reference external" href="https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/framework/lod_tensor.md">LoDTensor</a></li> <li>Variable length Tensor design <a class="reference external" href="https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/framework/lod_tensor.md">LoDTensor</a></li>
</ul> </ul>
</li> </li>
<li><code class="docutils literal"><span class="pre">Variable</span></code> instances are the inputs and the outputs of an operator. Not just <code class="docutils literal"><span class="pre">Tensor</span></code>.<ul> <li><code class="docutils literal"><span class="pre">Variable</span></code> instances are the inputs and the outputs of an operator, not just <code class="docutils literal"><span class="pre">Tensor</span></code>.<ul>
<li><code class="docutils literal"><span class="pre">step_scopes</span></code> in RNN is a variable and not a tensor.</li> <li><code class="docutils literal"><span class="pre">step_scopes</span></code> in RNN is a variable and not a tensor.</li>
</ul> </ul>
</li> </li>
<li><code class="docutils literal"><span class="pre">Scope</span></code> is where variables are stores.<ul> <li><code class="docutils literal"><span class="pre">Scope</span></code> is where variables are stored.<ul>
<li>map&lt;string <code class="docutils literal"><span class="pre">variable_name</span></code>, Variable&gt;</li> <li>map&lt;string <code class="docutils literal"><span class="pre">var</span> <span class="pre">name</span></code>, Variable&gt;</li>
<li><code class="docutils literal"><span class="pre">Scope</span></code> has a hierarchical structure. The local scope can get variables from its parent scope.</li> <li><code class="docutils literal"><span class="pre">Scope</span></code> has a hierarchical structure. The local scope can get variables from its parent scope.</li>
</ul> </ul>
</li> </li>
...@@ -517,7 +518,7 @@ ...@@ -517,7 +518,7 @@
<span id="control-the-migration-quality"></span><h1>Control the migration quality<a class="headerlink" href="#control-the-migration-quality" title="永久链接至标题"></a></h1> <span id="control-the-migration-quality"></span><h1>Control the migration quality<a class="headerlink" href="#control-the-migration-quality" title="永久链接至标题"></a></h1>
<ul class="simple"> <ul class="simple">
<li>Compare the performance of migrated models with old ones.</li> <li>Compare the performance of migrated models with old ones.</li>
<li>Follow the google C++ style</li> <li>Follow the google C++ style guide.</li>
<li>Build the automatic workflow of generating Python/C++ documentations.<ul> <li>Build the automatic workflow of generating Python/C++ documentations.<ul>
<li>The documentation of layers and ops should be written inside the code.</li> <li>The documentation of layers and ops should be written inside the code.</li>
<li>Take the documentation quality into account when submitting pull requests.</li> <li>Take the documentation quality into account when submitting pull requests.</li>
......
develop/doc_cn/searchindex.js
浏览文件 @ 294a9139
因为 它太大了无法显示 source diff 。你可以改为 查看blob。
Markdown is supported
0% 或 .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑！
想要评论请 注册 或