Skip to content

P
Paddle

BaiXuePrincess / Paddle
与 Fork 源项目一致

Fork自 PaddlePaddle / Paddle

通知 1
Star 1
Fork 0
P
Paddle
提交 4377bc79 编写于 作者: T Travis CI
浏览文件
操作

Deploy to GitHub Pages: 3ef8ec37

上级 c283efe7
隐藏空白更改
内联 并排
Showing with 2028 addition and 2017 deletion
develop/doc/_sources/howto/dev/new_op_en.md.txt
浏览文件 @ 4377bc79
# How to write a new operator # How to write a new operator
- [Background](#background) - [Background](#background)
- [Implementing C++ Types](#implementing-c++-types) - [Implementing C++ Types](#implementing-c-types)
- [Defining ProtoMaker](#defining-protoMaker) - [Defining ProtoMaker](#defining-protomaker)
- [Defining Operator](#defining-operator) - [Defining Operator](#defining-operator)
- [Registering Operator](#registering-operator) - [Registering Operator](#registering-operator)
- [Compilation](#compilation) - [Compilation](#compilation)
...@@ -41,7 +41,7 @@ Let's take matrix multiplication operator, [MulOp](https://github.com/PaddlePadd ...@@ -41,7 +41,7 @@ Let's take matrix multiplication operator, [MulOp](https://github.com/PaddlePadd
## Implementing C++ Types ## Implementing C++ Types
### 1. Defining Class ProtoMaker ### Defining ProtoMaker
Matrix Multiplication can be written as $Out = X * Y$, meaning that the operation consists of two inputs and pne output. Matrix Multiplication can be written as $Out = X * Y$, meaning that the operation consists of two inputs and pne output.
...@@ -98,7 +98,7 @@ There are two changes in this example: ...@@ -98,7 +98,7 @@ There are two changes in this example:
- `AddAttr<AttrType>("scale", "...").SetDefault(1.0);` adds `scale`constant as an attribute, and sets the default value to 1.0. - `AddAttr<AttrType>("scale", "...").SetDefault(1.0);` adds `scale`constant as an attribute, and sets the default value to 1.0.
### 2. Defining Operator ### Defining Operator
The following code defines the interface for MulOp: The following code defines the interface for MulOp:
...@@ -147,7 +147,7 @@ MulOp(const std::string &type, const framework::VariableNameMap &inputs, ...@@ -147,7 +147,7 @@ MulOp(const std::string &type, const framework::VariableNameMap &inputs,
Usually `OpProtoMaker` and `Op`'s type definitions are written in `.cc` files, which also include the registration methods introduced later. Usually `OpProtoMaker` and `Op`'s type definitions are written in `.cc` files, which also include the registration methods introduced later.
### 3. Defining OpKernel ### Defining OpKernel
`MulKernel` inherits `framework::OpKernel`, which includes the following templates: `MulKernel` inherits `framework::OpKernel`, which includes the following templates:
...@@ -188,7 +188,7 @@ This concludes the forward implementation of an operator. Next its operation and ...@@ -188,7 +188,7 @@ This concludes the forward implementation of an operator. Next its operation and
The definition of its corresponding backward operator, if applicable, is similar to that of an forward operator. **Note that a backward operator does not include a `ProtoMaker`**. The definition of its corresponding backward operator, if applicable, is similar to that of an forward operator. **Note that a backward operator does not include a `ProtoMaker`**.
### 4. Registering Operator ### Registering Operator
- In `.cc` files, register forward and backward operator classes and the CPU kernel. - In `.cc` files, register forward and backward operator classes and the CPU kernel.
...@@ -220,7 +220,7 @@ The definition of its corresponding backward operator, if applicable, is similar ...@@ -220,7 +220,7 @@ The definition of its corresponding backward operator, if applicable, is similar
ops::MulGradKernel<paddle::platform::CUDADeviceContext, float>); ops::MulGradKernel<paddle::platform::CUDADeviceContext, float>);
``` ```
### 5. Compilation ### Compilation
Run the following commands to compile. Run the following commands to compile.
...@@ -284,8 +284,7 @@ A forward operator unit test inherits `unittest.TestCase` and defines metaclass ...@@ -284,8 +284,7 @@ A forward operator unit test inherits `unittest.TestCase` and defines metaclass
def test_check_grad_ingore_y(self): def test_check_grad_ingore_y(self):
self.check_grad( self.check_grad(
['X'], 'Out', max_relative_error=0.5, no_grad_set=set('Y')) ['X'], 'Out', max_relative_error=0.5, no_grad_set=set('Y'))
```
```
Get its output, and compare it with the forward operator's own output. Get its output, and compare it with the forward operator's own output.
The code above first loads required packages. In addition, we have The code above first loads required packages. In addition, we have
...@@ -294,6 +293,8 @@ The code above first loads required packages. In addition, we have ...@@ -294,6 +293,8 @@ The code above first loads required packages. In addition, we have
- `self.inputs` defines input, with type `numpy.array` and initializes it. - `self.inputs` defines input, with type `numpy.array` and initializes it.
- `self.outputs` defines output and completes the same operator computation in the Python script, and returns its result from the Python script. - `self.outputs` defines output and completes the same operator computation in the Python script, and returns its result from the Python script.
### Testing Backward Operators
Some key points in checking gradient above include: Some key points in checking gradient above include:
- `test_normal` calls `check_grad` to validate scaling tests' correctness and stability through numeric methods. - `test_normal` calls `check_grad` to validate scaling tests' correctness and stability through numeric methods.
......
develop/doc/howto/dev/new_op_en.html
浏览文件 @ 4377bc79
...@@ -207,8 +207,8 @@ ...@@ -207,8 +207,8 @@
<span id="how-to-write-a-new-operator"></span><h1>How to write a new operator<a class="headerlink" href="#how-to-write-a-new-operator" title="Permalink to this headline"></a></h1> <span id="how-to-write-a-new-operator"></span><h1>How to write a new operator<a class="headerlink" href="#how-to-write-a-new-operator" title="Permalink to this headline"></a></h1>
<ul class="simple"> <ul class="simple">
<li><a class="reference external" href="#background">Background</a></li> <li><a class="reference external" href="#background">Background</a></li>
<li><a class="reference external" href="#implementing-c++-types">Implementing C++ Types</a><ul> <li><a class="reference external" href="#implementing-c-types">Implementing C++ Types</a><ul>
<li><a class="reference external" href="#defining-protoMaker">Defining ProtoMaker</a></li> <li><a class="reference external" href="#defining-protomaker">Defining ProtoMaker</a></li>
<li><a class="reference external" href="#defining-operator">Defining Operator</a></li> <li><a class="reference external" href="#defining-operator">Defining Operator</a></li>
<li><a class="reference external" href="#registering-operator">Registering Operator</a></li> <li><a class="reference external" href="#registering-operator">Registering Operator</a></li>
<li><a class="reference external" href="#compilation">Compilation</a></li> <li><a class="reference external" href="#compilation">Compilation</a></li>
...@@ -244,8 +244,8 @@ Registering the Op | Ops are registered in <code class="docutils liter ...@@ -244,8 +244,8 @@ Registering the Op | Ops are registered in <code class="docutils liter
</div> </div>
<div class="section" id="implementing-c-types"> <div class="section" id="implementing-c-types">
<span id="implementing-c-types"></span><h2>Implementing C++ Types<a class="headerlink" href="#implementing-c-types" title="Permalink to this headline"></a></h2> <span id="implementing-c-types"></span><h2>Implementing C++ Types<a class="headerlink" href="#implementing-c-types" title="Permalink to this headline"></a></h2>
<div class="section" id="defining-class-protomaker"> <div class="section" id="defining-protomaker">
<span id="defining-class-protomaker"></span><h3>1. Defining Class ProtoMaker<a class="headerlink" href="#defining-class-protomaker" title="Permalink to this headline"></a></h3> <span id="defining-protomaker"></span><h3>Defining ProtoMaker<a class="headerlink" href="#defining-protomaker" title="Permalink to this headline"></a></h3>
<p>Matrix Multiplication can be written as $Out = X * Y$, meaning that the operation consists of two inputs and pne output.</p> <p>Matrix Multiplication can be written as $Out = X * Y$, meaning that the operation consists of two inputs and pne output.</p>
<p>First, define <code class="docutils literal"><span class="pre">ProtoMaker</span></code> to describe the Operator&#8217;s input, output, and additional comments:</p> <p>First, define <code class="docutils literal"><span class="pre">ProtoMaker</span></code> to describe the Operator&#8217;s input, output, and additional comments:</p>
<div class="highlight-cpp"><div class="highlight"><pre><span></span><span class="k">class</span> <span class="nc">MulOpMaker</span> <span class="o">:</span> <span class="k">public</span> <span class="n">framework</span><span class="o">::</span><span class="n">OpProtoAndCheckerMaker</span> <span class="p">{</span> <div class="highlight-cpp"><div class="highlight"><pre><span></span><span class="k">class</span> <span class="nc">MulOpMaker</span> <span class="o">:</span> <span class="k">public</span> <span class="n">framework</span><span class="o">::</span><span class="n">OpProtoAndCheckerMaker</span> <span class="p">{</span>
...@@ -293,7 +293,7 @@ Registering the Op | Ops are registered in <code class="docutils liter ...@@ -293,7 +293,7 @@ Registering the Op | Ops are registered in <code class="docutils liter
</ul> </ul>
</div> </div>
<div class="section" id="defining-operator"> <div class="section" id="defining-operator">
<span id="defining-operator"></span><h3>2. Defining Operator<a class="headerlink" href="#defining-operator" title="Permalink to this headline"></a></h3> <span id="defining-operator"></span><h3>Defining Operator<a class="headerlink" href="#defining-operator" title="Permalink to this headline"></a></h3>
<p>The following code defines the interface for MulOp:</p> <p>The following code defines the interface for MulOp:</p>
<div class="highlight-cpp"><div class="highlight"><pre><span></span><span class="k">class</span> <span class="nc">MulOp</span> <span class="o">:</span> <span class="k">public</span> <span class="n">framework</span><span class="o">::</span><span class="n">OperatorWithKernel</span> <span class="p">{</span> <div class="highlight-cpp"><div class="highlight"><pre><span></span><span class="k">class</span> <span class="nc">MulOp</span> <span class="o">:</span> <span class="k">public</span> <span class="n">framework</span><span class="o">::</span><span class="n">OperatorWithKernel</span> <span class="p">{</span>
<span class="k">public</span><span class="o">:</span> <span class="k">public</span><span class="o">:</span>
...@@ -336,7 +336,7 @@ Registering the Op | Ops are registered in <code class="docutils liter ...@@ -336,7 +336,7 @@ Registering the Op | Ops are registered in <code class="docutils liter
<p>Usually <code class="docutils literal"><span class="pre">OpProtoMaker</span></code> and <code class="docutils literal"><span class="pre">Op</span></code>&#8216;s type definitions are written in <code class="docutils literal"><span class="pre">.cc</span></code> files, which also include the registration methods introduced later.</p> <p>Usually <code class="docutils literal"><span class="pre">OpProtoMaker</span></code> and <code class="docutils literal"><span class="pre">Op</span></code>&#8216;s type definitions are written in <code class="docutils literal"><span class="pre">.cc</span></code> files, which also include the registration methods introduced later.</p>
</div> </div>
<div class="section" id="defining-opkernel"> <div class="section" id="defining-opkernel">
<span id="defining-opkernel"></span><h3>3. Defining OpKernel<a class="headerlink" href="#defining-opkernel" title="Permalink to this headline"></a></h3> <span id="defining-opkernel"></span><h3>Defining OpKernel<a class="headerlink" href="#defining-opkernel" title="Permalink to this headline"></a></h3>
<p><code class="docutils literal"><span class="pre">MulKernel</span></code> inherits <code class="docutils literal"><span class="pre">framework::OpKernel</span></code>, which includes the following templates:</p> <p><code class="docutils literal"><span class="pre">MulKernel</span></code> inherits <code class="docutils literal"><span class="pre">framework::OpKernel</span></code>, which includes the following templates:</p>
<ul class="simple"> <ul class="simple">
<li><code class="docutils literal"><span class="pre">typename</span> <span class="pre">DeviceContext</span></code> denotes device context type. When different devices, namely the CPUDeviceContext and the CUDADeviceContext, share the same kernel, this template needs to be added. If they don&#8217;t share kernels, this must not be added. An example of a non-sharing kernel is <a class="reference external" href="https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/operators/cross_entropy_op.h#L43"><code class="docutils literal"><span class="pre">OnehotCrossEntropyOpKernel</span></code></a>.</li> <li><code class="docutils literal"><span class="pre">typename</span> <span class="pre">DeviceContext</span></code> denotes device context type. When different devices, namely the CPUDeviceContext and the CUDADeviceContext, share the same kernel, this template needs to be added. If they don&#8217;t share kernels, this must not be added. An example of a non-sharing kernel is <a class="reference external" href="https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/operators/cross_entropy_op.h#L43"><code class="docutils literal"><span class="pre">OnehotCrossEntropyOpKernel</span></code></a>.</li>
...@@ -370,7 +370,7 @@ Registering the Op | Ops are registered in <code class="docutils liter ...@@ -370,7 +370,7 @@ Registering the Op | Ops are registered in <code class="docutils liter
<p>The definition of its corresponding backward operator, if applicable, is similar to that of an forward operator. <strong>Note that a backward operator does not include a <code class="docutils literal"><span class="pre">ProtoMaker</span></code></strong>.</p> <p>The definition of its corresponding backward operator, if applicable, is similar to that of an forward operator. <strong>Note that a backward operator does not include a <code class="docutils literal"><span class="pre">ProtoMaker</span></code></strong>.</p>
</div> </div>
<div class="section" id="registering-operator"> <div class="section" id="registering-operator">
<span id="registering-operator"></span><h3>4. Registering Operator<a class="headerlink" href="#registering-operator" title="Permalink to this headline"></a></h3> <span id="registering-operator"></span><h3>Registering Operator<a class="headerlink" href="#registering-operator" title="Permalink to this headline"></a></h3>
<ul> <ul>
<li><p class="first">In <code class="docutils literal"><span class="pre">.cc</span></code> files, register forward and backward operator classes and the CPU kernel.</p> <li><p class="first">In <code class="docutils literal"><span class="pre">.cc</span></code> files, register forward and backward operator classes and the CPU kernel.</p>
<div class="highlight-cpp"><div class="highlight"><pre><span></span><span class="k">namespace</span> <span class="n">ops</span> <span class="o">=</span> <span class="n">paddle</span><span class="o">::</span><span class="n">operators</span><span class="p">;</span> <div class="highlight-cpp"><div class="highlight"><pre><span></span><span class="k">namespace</span> <span class="n">ops</span> <span class="o">=</span> <span class="n">paddle</span><span class="o">::</span><span class="n">operators</span><span class="p">;</span>
...@@ -406,7 +406,7 @@ Registering the Op | Ops are registered in <code class="docutils liter ...@@ -406,7 +406,7 @@ Registering the Op | Ops are registered in <code class="docutils liter
</ul> </ul>
</div> </div>
<div class="section" id="compilation"> <div class="section" id="compilation">
<span id="compilation"></span><h3>5. Compilation<a class="headerlink" href="#compilation" title="Permalink to this headline"></a></h3> <span id="compilation"></span><h3>Compilation<a class="headerlink" href="#compilation" title="Permalink to this headline"></a></h3>
<p>Run the following commands to compile.</p> <p>Run the following commands to compile.</p>
<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">make</span> <span class="n">mul_op</span> <div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">make</span> <span class="n">mul_op</span>
</pre></div> </pre></div>
...@@ -462,7 +462,6 @@ Registering the Op | Ops are registered in <code class="docutils liter ...@@ -462,7 +462,6 @@ Registering the Op | Ops are registered in <code class="docutils liter
<span class="k">def</span> <span class="nf">test_check_grad_ingore_y</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span> <span class="k">def</span> <span class="nf">test_check_grad_ingore_y</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">check_grad</span><span class="p">(</span> <span class="bp">self</span><span class="o">.</span><span class="n">check_grad</span><span class="p">(</span>
<span class="p">[</span><span class="s1">&#39;X&#39;</span><span class="p">],</span> <span class="s1">&#39;Out&#39;</span><span class="p">,</span> <span class="n">max_relative_error</span><span class="o">=</span><span class="mf">0.5</span><span class="p">,</span> <span class="n">no_grad_set</span><span class="o">=</span><span class="nb">set</span><span class="p">(</span><span class="s1">&#39;Y&#39;</span><span class="p">))</span> <span class="p">[</span><span class="s1">&#39;X&#39;</span><span class="p">],</span> <span class="s1">&#39;Out&#39;</span><span class="p">,</span> <span class="n">max_relative_error</span><span class="o">=</span><span class="mf">0.5</span><span class="p">,</span> <span class="n">no_grad_set</span><span class="o">=</span><span class="nb">set</span><span class="p">(</span><span class="s1">&#39;Y&#39;</span><span class="p">))</span>
</pre></div> </pre></div>
</div> </div>
<p>Get its output, and compare it with the forward operator&#8217;s own output.</p> <p>Get its output, and compare it with the forward operator&#8217;s own output.</p>
...@@ -472,6 +471,9 @@ Registering the Op | Ops are registered in <code class="docutils liter ...@@ -472,6 +471,9 @@ Registering the Op | Ops are registered in <code class="docutils liter
<li><code class="docutils literal"><span class="pre">self.inputs</span></code> defines input, with type <code class="docutils literal"><span class="pre">numpy.array</span></code> and initializes it.</li> <li><code class="docutils literal"><span class="pre">self.inputs</span></code> defines input, with type <code class="docutils literal"><span class="pre">numpy.array</span></code> and initializes it.</li>
<li><code class="docutils literal"><span class="pre">self.outputs</span></code> defines output and completes the same operator computation in the Python script, and returns its result from the Python script.</li> <li><code class="docutils literal"><span class="pre">self.outputs</span></code> defines output and completes the same operator computation in the Python script, and returns its result from the Python script.</li>
</ul> </ul>
</div>
<div class="section" id="testing-backward-operators">
<span id="testing-backward-operators"></span><h3>Testing Backward Operators<a class="headerlink" href="#testing-backward-operators" title="Permalink to this headline"></a></h3>
<p>Some key points in checking gradient above include:</p> <p>Some key points in checking gradient above include:</p>
<ul class="simple"> <ul class="simple">
<li><code class="docutils literal"><span class="pre">test_normal</span></code> calls <code class="docutils literal"><span class="pre">check_grad</span></code> to validate scaling tests&#8217; correctness and stability through numeric methods.<ul> <li><code class="docutils literal"><span class="pre">test_normal</span></code> calls <code class="docutils literal"><span class="pre">check_grad</span></code> to validate scaling tests&#8217; correctness and stability through numeric methods.<ul>
......
develop/doc/operators.json
浏览文件 @ 4377bc79
因为 它太大了无法显示 source diff 。你可以改为 查看blob。
develop/doc/searchindex.js
浏览文件 @ 4377bc79
因为 它太大了无法显示 source diff 。你可以改为 查看blob。
develop/doc_cn/_sources/howto/dev/new_op_cn.md.txt
浏览文件 @ 4377bc79
# 如何写新的Operator # 如何写新的Operator
- [概念简介](#概念简介) - [概念简介](#概念简介)
- [实现C++类](#实现C++类) - [实现C++类](#实现c类)
- [定义ProtoMaker类](#定义ProtoMaker类) - [定义ProtoMaker类](#定义protomaker类)
- [定义Operator类](#定义Operator类) - [定义Operator类](#定义operator类)
- [定义OpKernel类](#定义OpKernel类) - [定义OpKernel类](#定义opkernel类)
- [注册Operator](#注册Operator) - [注册Operator](#注册operator)
- [编译](#编译) - [编译](#编译)
- [绑定Python](#绑定Python) - [绑定Python](#绑定python)
- [实现单元测试](#实现单元测试) - [实现单元测试](#实现单元测试)
- [前向Operator单测](#前向Operator单测) - [前向Operator单测](#前向operator单测)
- [反向Operator单测](#反向Operator单测) - [反向Operator单测](#反向operator单测)
- [编译和执行](#编译和执行) - [编译和执行](#编译和执行)
- [注意事项](#注意事项)
## 概念简介 ## 概念简介
...@@ -43,7 +44,7 @@ Kernel实现 | CPU、CUDA共享Kernel实现在`.h`文件中，否则，CPU ...@@ -43,7 +44,7 @@ Kernel实现 | CPU、CUDA共享Kernel实现在`.h`文件中，否则，CPU
## 实现C++类 ## 实现C++类
### 1. 定义ProtoMaker类 ### 定义ProtoMaker类
矩阵乘法的公式：$Out = X * Y$, 可见该计算由两个输入，一个输出组成。 矩阵乘法的公式：$Out = X * Y$, 可见该计算由两个输入，一个输出组成。
...@@ -100,7 +101,7 @@ The equation is: Out = scale*X ...@@ -100,7 +101,7 @@ The equation is: Out = scale*X
- `AddAttr<AttrType>("scale", "...").SetDefault(1.0);` : 增加`scale`系数，作为参数属性，并且设置默认值为1.0。 - `AddAttr<AttrType>("scale", "...").SetDefault(1.0);` : 增加`scale`系数，作为参数属性，并且设置默认值为1.0。
### 2. 定义Operator类 ### 定义Operator类
下面的点实现了MulOp的定义： 下面的点实现了MulOp的定义：
...@@ -149,7 +150,7 @@ MulOp(const std::string &type, const framework::VariableNameMap &inputs, ...@@ -149,7 +150,7 @@ MulOp(const std::string &type, const framework::VariableNameMap &inputs,
通常`OpProtoMaker`和`Op`类的定义写在`.cc`文件中，和下面将要介绍的注册函数一起放在`.cc`中 通常`OpProtoMaker`和`Op`类的定义写在`.cc`文件中，和下面将要介绍的注册函数一起放在`.cc`中
### 3. 定义OpKernel类 ### 定义OpKernel类
`MulKernel`继承自`framework::OpKernel`，带有下面两个模板参数: `MulKernel`继承自`framework::OpKernel`，带有下面两个模板参数:
...@@ -177,6 +178,7 @@ MulOp(const std::string &type, const framework::VariableNameMap &inputs, ...@@ -177,6 +178,7 @@ MulOp(const std::string &type, const framework::VariableNameMap &inputs,
math::matmul<DeviceContext, T>(*X, false, *Y, false, 1, Z, 0, device_context); math::matmul<DeviceContext, T>(*X, false, *Y, false, 1, Z, 0, device_context);
} }
}; };
```
需要注意：**不同设备(CPU、CUDA)共享一个Op定义，是否则共享同一个`OpKernel`，取决于`Compute`调用的函数是否支持不同设备。** 需要注意：**不同设备(CPU、CUDA)共享一个Op定义，是否则共享同一个`OpKernel`，取决于`Compute`调用的函数是否支持不同设备。**
...@@ -188,7 +190,7 @@ MulOp(const std::string &type, const framework::VariableNameMap &inputs, ...@@ -188,7 +190,7 @@ MulOp(const std::string &type, const framework::VariableNameMap &inputs,
到此，前向Op实现完成。接下来，需要在`.cc`文件中注册该op和kernel。 到此，前向Op实现完成。接下来，需要在`.cc`文件中注册该op和kernel。
反向Op类的定义，反向OpKernel的定义与前向Op类似，这里不再赘述。**但需注意反向Op没有`ProtoMaker`**。 反向Op类的定义，反向OpKernel的定义与前向Op类似，这里不再赘述。**但需注意反向Op没有`ProtoMaker`**。
### 4. 注册Operator ### 注册Operator
- 在`.cc`文件中注册前向、反向Op类，注册CPU Kernel。 - 在`.cc`文件中注册前向、反向Op类，注册CPU Kernel。
...@@ -220,7 +222,7 @@ MulOp(const std::string &type, const framework::VariableNameMap &inputs, ...@@ -220,7 +222,7 @@ MulOp(const std::string &type, const framework::VariableNameMap &inputs,
ops::MulGradKernel<paddle::platform::CUDADeviceContext, float>); ops::MulGradKernel<paddle::platform::CUDADeviceContext, float>);
``` ```
### 5. 编译 ### 编译
运行下面命令可以进行编译： 运行下面命令可以进行编译：
...@@ -236,6 +238,7 @@ make mul_op ...@@ -236,6 +238,7 @@ make mul_op
单测包括对比前向Op不同设备(CPU、CUDA)的实现、对比反向OP不同设备(CPU、CUDA)的实现、反向Op的梯度测试。下面介绍介绍[`MulOp`的单元测试](https://github.com/PaddlePaddle/Paddle/blob/develop/python/paddle/v2/framework/tests/test_mul_op.py)。 单测包括对比前向Op不同设备(CPU、CUDA)的实现、对比反向OP不同设备(CPU、CUDA)的实现、反向Op的梯度测试。下面介绍介绍[`MulOp`的单元测试](https://github.com/PaddlePaddle/Paddle/blob/develop/python/paddle/v2/framework/tests/test_mul_op.py)。
### 前向Operator单测
Op单元测试继承自`OpTest`。各项更加具体的单元测试在`TestMulOp`里完成。测试Operator，需要： Op单元测试继承自`OpTest`。各项更加具体的单元测试在`TestMulOp`里完成。测试Operator，需要：
...@@ -273,8 +276,7 @@ Op单元测试继承自`OpTest`。各项更加具体的单元测试在`TestMulOp ...@@ -273,8 +276,7 @@ Op单元测试继承自`OpTest`。各项更加具体的单元测试在`TestMulOp
def test_check_grad_ingore_y(self): def test_check_grad_ingore_y(self):
self.check_grad( self.check_grad(
['X'], 'Out', max_relative_error=0.5, no_grad_set=set('Y')) ['X'], 'Out', max_relative_error=0.5, no_grad_set=set('Y'))
```
```
上面的代码首先导入依赖的包，下面是对`setUp`函数中操作的重要变量的详细解释： 上面的代码首先导入依赖的包，下面是对`setUp`函数中操作的重要变量的详细解释：
...@@ -282,6 +284,8 @@ Op单元测试继承自`OpTest`。各项更加具体的单元测试在`TestMulOp ...@@ -282,6 +284,8 @@ Op单元测试继承自`OpTest`。各项更加具体的单元测试在`TestMulOp
- `self.inputs` : 定义输入，类型为`numpy.array`，并初始化。 - `self.inputs` : 定义输入，类型为`numpy.array`，并初始化。
- `self.outputs` : 定义输出，并在Python脚本中完成与operator同样的计算逻辑，返回Python端的计算结果。 - `self.outputs` : 定义输出，并在Python脚本中完成与operator同样的计算逻辑，返回Python端的计算结果。
### 反向operator单测
而反向测试中： 而反向测试中：
- `test_check_grad_normal`中调用`check_grad`使用数值法检测梯度正确性和稳定性。 - `test_check_grad_normal`中调用`check_grad`使用数值法检测梯度正确性和稳定性。
- 第一个参数`["X", "Y"]` : 指定对输入变量`X`、`Y`做梯度检测。 - 第一个参数`["X", "Y"]` : 指定对输入变量`X`、`Y`做梯度检测。
...@@ -290,7 +294,7 @@ Op单元测试继承自`OpTest`。各项更加具体的单元测试在`TestMulOp ...@@ -290,7 +294,7 @@ Op单元测试继承自`OpTest`。各项更加具体的单元测试在`TestMulOp
- `test_check_grad_ingore_x`和`test_check_grad_ingore_y`分支用来测试只需要计算一个输入梯度的情况。 - `test_check_grad_ingore_x`和`test_check_grad_ingore_y`分支用来测试只需要计算一个输入梯度的情况。
### 编译和执行单元测试 ### 编译和执行
`python/paddle/v2/framework/tests` 目录下新增的 `test_*.py` 单元测试会被自动加入工程进行编译。 `python/paddle/v2/framework/tests` 目录下新增的 `test_*.py` 单元测试会被自动加入工程进行编译。
......
develop/doc_cn/howto/dev/new_op_cn.html
浏览文件 @ 4377bc79
...@@ -208,21 +208,22 @@ ...@@ -208,21 +208,22 @@
<span id="operator"></span><h1>如何写新的Operator<a class="headerlink" href="#operator" title="永久链接至标题"></a></h1> <span id="operator"></span><h1>如何写新的Operator<a class="headerlink" href="#operator" title="永久链接至标题"></a></h1>
<ul class="simple"> <ul class="simple">
<li><a class="reference external" href="#概念简介">概念简介</a></li> <li><a class="reference external" href="#概念简介">概念简介</a></li>
<li><a class="reference external" href="#实现C++类">实现C++类</a><ul> <li><a class="reference external" href="#实现c类">实现C++类</a><ul>
<li><a class="reference external" href="#定义ProtoMaker类">定义ProtoMaker类</a></li> <li><a class="reference external" href="#定义protomaker类">定义ProtoMaker类</a></li>
<li><a class="reference external" href="#定义Operator类">定义Operator类</a></li> <li><a class="reference external" href="#定义operator类">定义Operator类</a></li>
<li><a class="reference external" href="#定义OpKernel类">定义OpKernel类</a></li> <li><a class="reference external" href="#定义opkernel类">定义OpKernel类</a></li>
<li><a class="reference external" href="#注册Operator">注册Operator</a></li> <li><a class="reference external" href="#注册operator">注册Operator</a></li>
<li><a class="reference external" href="#编译">编译</a></li> <li><a class="reference external" href="#编译">编译</a></li>
</ul> </ul>
</li> </li>
<li><a class="reference external" href="#绑定Python">绑定Python</a></li> <li><a class="reference external" href="#绑定python">绑定Python</a></li>
<li><a class="reference external" href="#实现单元测试">实现单元测试</a><ul> <li><a class="reference external" href="#实现单元测试">实现单元测试</a><ul>
<li><a class="reference external" href="#前向Operator单测">前向Operator单测</a></li> <li><a class="reference external" href="#前向operator单测">前向Operator单测</a></li>
<li><a class="reference external" href="#反向Operator单测">反向Operator单测</a></li> <li><a class="reference external" href="#反向operator单测">反向Operator单测</a></li>
<li><a class="reference external" href="#编译和执行">编译和执行</a></li> <li><a class="reference external" href="#编译和执行">编译和执行</a></li>
</ul> </ul>
</li> </li>
<li><a class="reference external" href="#注意事项">注意事项</a></li>
</ul> </ul>
<div class="section" id=""> <div class="section" id="">
<span id="id1"></span><h2>概念简介<a class="headerlink" href="#" title="永久链接至标题"></a></h2> <span id="id1"></span><h2>概念简介<a class="headerlink" href="#" title="永久链接至标题"></a></h2>
...@@ -246,7 +247,7 @@ Kernel实现 | CPU、CUDA共享Kernel实现在<code class="docutils litera ...@@ -246,7 +247,7 @@ Kernel实现 | CPU、CUDA共享Kernel实现在<code class="docutils litera
<div class="section" id="c"> <div class="section" id="c">
<span id="c"></span><h2>实现C++类<a class="headerlink" href="#c" title="永久链接至标题"></a></h2> <span id="c"></span><h2>实现C++类<a class="headerlink" href="#c" title="永久链接至标题"></a></h2>
<div class="section" id="protomaker"> <div class="section" id="protomaker">
<span id="protomaker"></span><h3>1. 定义ProtoMaker类<a class="headerlink" href="#protomaker" title="永久链接至标题"></a></h3> <span id="protomaker"></span><h3>定义ProtoMaker类<a class="headerlink" href="#protomaker" title="永久链接至标题"></a></h3>
<p>矩阵乘法的公式：$Out = X * Y$, 可见该计算由两个输入，一个输出组成。</p> <p>矩阵乘法的公式：$Out = X * Y$, 可见该计算由两个输入，一个输出组成。</p>
<p>首先定义<code class="docutils literal"><span class="pre">ProtoMaker</span></code>来描述该Op的输入、输出，并添加注释：</p> <p>首先定义<code class="docutils literal"><span class="pre">ProtoMaker</span></code>来描述该Op的输入、输出，并添加注释：</p>
<div class="highlight-cpp"><div class="highlight"><pre><span></span><span class="k">class</span> <span class="nc">MulOpMaker</span> <span class="o">:</span> <span class="k">public</span> <span class="n">framework</span><span class="o">::</span><span class="n">OpProtoAndCheckerMaker</span> <span class="p">{</span> <div class="highlight-cpp"><div class="highlight"><pre><span></span><span class="k">class</span> <span class="nc">MulOpMaker</span> <span class="o">:</span> <span class="k">public</span> <span class="n">framework</span><span class="o">::</span><span class="n">OpProtoAndCheckerMaker</span> <span class="p">{</span>
...@@ -294,7 +295,7 @@ Kernel实现 | CPU、CUDA共享Kernel实现在<code class="docutils litera ...@@ -294,7 +295,7 @@ Kernel实现 | CPU、CUDA共享Kernel实现在<code class="docutils litera
</ul> </ul>
</div> </div>
<div class="section" id="operator"> <div class="section" id="operator">
<span id="id2"></span><h3>2. 定义Operator类<a class="headerlink" href="#operator" title="永久链接至标题"></a></h3> <span id="id2"></span><h3>定义Operator类<a class="headerlink" href="#operator" title="永久链接至标题"></a></h3>
<p>下面的点实现了MulOp的定义：</p> <p>下面的点实现了MulOp的定义：</p>
<div class="highlight-cpp"><div class="highlight"><pre><span></span><span class="k">class</span> <span class="nc">MulOp</span> <span class="o">:</span> <span class="k">public</span> <span class="n">framework</span><span class="o">::</span><span class="n">OperatorWithKernel</span> <span class="p">{</span> <div class="highlight-cpp"><div class="highlight"><pre><span></span><span class="k">class</span> <span class="nc">MulOp</span> <span class="o">:</span> <span class="k">public</span> <span class="n">framework</span><span class="o">::</span><span class="n">OperatorWithKernel</span> <span class="p">{</span>
<span class="k">public</span><span class="o">:</span> <span class="k">public</span><span class="o">:</span>
...@@ -337,7 +338,7 @@ Kernel实现 | CPU、CUDA共享Kernel实现在<code class="docutils litera ...@@ -337,7 +338,7 @@ Kernel实现 | CPU、CUDA共享Kernel实现在<code class="docutils litera
<p>通常<code class="docutils literal"><span class="pre">OpProtoMaker</span></code>和<code class="docutils literal"><span class="pre">Op</span></code>类的定义写在<code class="docutils literal"><span class="pre">.cc</span></code>文件中，和下面将要介绍的注册函数一起放在<code class="docutils literal"><span class="pre">.cc</span></code>中</p> <p>通常<code class="docutils literal"><span class="pre">OpProtoMaker</span></code>和<code class="docutils literal"><span class="pre">Op</span></code>类的定义写在<code class="docutils literal"><span class="pre">.cc</span></code>文件中，和下面将要介绍的注册函数一起放在<code class="docutils literal"><span class="pre">.cc</span></code>中</p>
</div> </div>
<div class="section" id="opkernel"> <div class="section" id="opkernel">
<span id="opkernel"></span><h3>3. 定义OpKernel类<a class="headerlink" href="#opkernel" title="永久链接至标题"></a></h3> <span id="opkernel"></span><h3>定义OpKernel类<a class="headerlink" href="#opkernel" title="永久链接至标题"></a></h3>
<p><code class="docutils literal"><span class="pre">MulKernel</span></code>继承自<code class="docutils literal"><span class="pre">framework::OpKernel</span></code>，带有下面两个模板参数:</p> <p><code class="docutils literal"><span class="pre">MulKernel</span></code>继承自<code class="docutils literal"><span class="pre">framework::OpKernel</span></code>，带有下面两个模板参数:</p>
<ul class="simple"> <ul class="simple">
<li><code class="docutils literal"><span class="pre">typename</span> <span class="pre">DeviceContext</span></code>: 表示设备类型，不同设备(CPU、CUDA)共享同一个Kernel时，需加该模板参数，不共享则不加，一个不共享的例子是<a class="reference external" href="https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/operators/cross_entropy_op.h#L43"><code class="docutils literal"><span class="pre">OnehotCrossEntropyOpKernel</span></code></a>。</li> <li><code class="docutils literal"><span class="pre">typename</span> <span class="pre">DeviceContext</span></code>: 表示设备类型，不同设备(CPU、CUDA)共享同一个Kernel时，需加该模板参数，不共享则不加，一个不共享的例子是<a class="reference external" href="https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/operators/cross_entropy_op.h#L43"><code class="docutils literal"><span class="pre">OnehotCrossEntropyOpKernel</span></code></a>。</li>
...@@ -350,47 +351,45 @@ Kernel实现 | CPU、CUDA共享Kernel实现在<code class="docutils litera ...@@ -350,47 +351,45 @@ Kernel实现 | CPU、CUDA共享Kernel实现在<code class="docutils litera
<li><code class="docutils literal"><span class="pre">Compute</span></code>函数里实现<code class="docutils literal"><span class="pre">OpKernel</span></code>的具体计算逻辑。</li> <li><code class="docutils literal"><span class="pre">Compute</span></code>函数里实现<code class="docutils literal"><span class="pre">OpKernel</span></code>的具体计算逻辑。</li>
</ul> </ul>
<p>下面是 <code class="docutils literal"><span class="pre">MulKernel</span></code> <code class="docutils literal"><span class="pre">Compute</span></code>的实现：</p> <p>下面是 <code class="docutils literal"><span class="pre">MulKernel</span></code> <code class="docutils literal"><span class="pre">Compute</span></code>的实现：</p>
<div class="highlight-cpp"><div class="highlight"><pre><span></span>template &lt;typename DeviceContext, typename T&gt; <div class="highlight-cpp"><div class="highlight"><pre><span></span><span class="k">template</span> <span class="o">&lt;</span><span class="k">typename</span> <span class="n">DeviceContext</span><span class="p">,</span> <span class="k">typename</span> <span class="n">T</span><span class="o">&gt;</span>
class MulKernel : public framework::OpKernel { <span class="k">class</span> <span class="nc">MulKernel</span> <span class="o">:</span> <span class="k">public</span> <span class="n">framework</span><span class="o">::</span><span class="n">OpKernel</span> <span class="p">{</span>
public: <span class="k">public</span><span class="o">:</span>
void Compute(const framework::ExecutionContext&amp; context) const override { <span class="kt">void</span> <span class="n">Compute</span><span class="p">(</span><span class="k">const</span> <span class="n">framework</span><span class="o">::</span><span class="n">ExecutionContext</span><span class="o">&amp;</span> <span class="n">context</span><span class="p">)</span> <span class="k">const</span> <span class="k">override</span> <span class="p">{</span>
auto* X = context.Input&lt;Tensor&gt;(&quot;X&quot;); <span class="k">auto</span><span class="o">*</span> <span class="n">X</span> <span class="o">=</span> <span class="n">context</span><span class="p">.</span><span class="n">Input</span><span class="o">&lt;</span><span class="n">Tensor</span><span class="o">&gt;</span><span class="p">(</span><span class="s">&quot;X&quot;</span><span class="p">);</span>
auto* Y = context.Input&lt;Tensor&gt;(&quot;Y&quot;); <span class="k">auto</span><span class="o">*</span> <span class="n">Y</span> <span class="o">=</span> <span class="n">context</span><span class="p">.</span><span class="n">Input</span><span class="o">&lt;</span><span class="n">Tensor</span><span class="o">&gt;</span><span class="p">(</span><span class="s">&quot;Y&quot;</span><span class="p">);</span>
auto* Z = context.Output&lt;Tensor&gt;(&quot;Out&quot;); <span class="k">auto</span><span class="o">*</span> <span class="n">Z</span> <span class="o">=</span> <span class="n">context</span><span class="p">.</span><span class="n">Output</span><span class="o">&lt;</span><span class="n">Tensor</span><span class="o">&gt;</span><span class="p">(</span><span class="s">&quot;Out&quot;</span><span class="p">);</span>
Z-&gt;mutable_data&lt;T&gt;(context.GetPlace()); <span class="n">Z</span><span class="o">-&gt;</span><span class="n">mutable_data</span><span class="o">&lt;</span><span class="n">T</span><span class="o">&gt;</span><span class="p">(</span><span class="n">context</span><span class="p">.</span><span class="n">GetPlace</span><span class="p">());</span>
auto&amp; device_context = context.template device_context&lt;DeviceContext&gt;(); <span class="k">auto</span><span class="o">&amp;</span> <span class="n">device_context</span> <span class="o">=</span> <span class="n">context</span><span class="p">.</span><span class="k">template</span> <span class="n">device_context</span><span class="o">&lt;</span><span class="n">DeviceContext</span><span class="o">&gt;</span><span class="p">();</span>
math::matmul&lt;DeviceContext, T&gt;(*X, false, *Y, false, 1, Z, 0, device_context); <span class="n">math</span><span class="o">::</span><span class="n">matmul</span><span class="o">&lt;</span><span class="n">DeviceContext</span><span class="p">,</span> <span class="n">T</span><span class="o">&gt;</span><span class="p">(</span><span class="o">*</span><span class="n">X</span><span class="p">,</span> <span class="nb">false</span><span class="p">,</span> <span class="o">*</span><span class="n">Y</span><span class="p">,</span> <span class="nb">false</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="n">Z</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="n">device_context</span><span class="p">);</span>
} <span class="p">}</span>
}; <span class="p">};</span>
需要注意：**不同设备(CPU、CUDA)共享一个Op定义，是否则共享同一个`OpKernel`，取决于`Compute`调用的函数是否支持不同设备。**
`MulOp`的CPU、CUDA实现共享同一个`Kernel`。`OpKernel`不共享的例子可以参考：[`OnehotCrossEntropyOpKernel`](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/operators/cross_entropy_op.h#L43)。
为了使`OpKernel`的计算过程书写更加简单，并且CPU、CUDA的代码可以复用，我们通常借助 Eigen unsupported Tensor模块来实现`Compute`接口。关于在PaddlePaddle中如何使用Eigen库，请参考[使用文档](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/howto/dev/use_eigen_cn.md)。
到此，前向Op实现完成。接下来，需要在`.cc`文件中注册该op和kernel。
反向Op类的定义，反向OpKernel的定义与前向Op类似，这里不再赘述。**但需注意反向Op没有`ProtoMaker`**。
### 4. 注册Operator
- 在`.cc`文件中注册前向、反向Op类，注册CPU Kernel。
```cpp
namespace ops = paddle::operators;
REGISTER_OP(mul, ops::MulOp, ops::MulOpMaker, mul_grad, ops::MulOpGrad);
REGISTER_OP_CPU_KERNEL(mul, ops::MulKernel&lt;paddle::platform::CPUDeviceContext, float&gt;);
REGISTER_OP_CPU_KERNEL(mul_grad,
ops::MulGradKernel&lt;paddle::platform::CPUDeviceContext, float&gt;);
</pre></div> </pre></div>
</div> </div>
<p>在上面的代码中：</p> <p>需要注意：<strong>不同设备(CPU、CUDA)共享一个Op定义，是否则共享同一个<code class="docutils literal"><span class="pre">OpKernel</span></code>，取决于<code class="docutils literal"><span class="pre">Compute</span></code>调用的函数是否支持不同设备。</strong></p>
<div class="highlight-default"><div class="highlight"><pre><span></span>- `REGISTER_OP` ： 注册`ops::MulOp`类，类型名为`mul`，该类的`ProtoMaker`为`ops::MulOpMaker`，注册`ops::MulOpGrad`，类型名为`mul_grad`。 <p><code class="docutils literal"><span class="pre">MulOp</span></code>的CPU、CUDA实现共享同一个<code class="docutils literal"><span class="pre">Kernel</span></code>。<code class="docutils literal"><span class="pre">OpKernel</span></code>不共享的例子可以参考：<a class="reference external" href="https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/operators/cross_entropy_op.h#L43"><code class="docutils literal"><span class="pre">OnehotCrossEntropyOpKernel</span></code></a>。</p>
- `REGISTER_OP_WITHOUT_GRADIENT` ： 用于注册没有反向的Op。 <p>为了使<code class="docutils literal"><span class="pre">OpKernel</span></code>的计算过程书写更加简单，并且CPU、CUDA的代码可以复用，我们通常借助 Eigen unsupported Tensor模块来实现<code class="docutils literal"><span class="pre">Compute</span></code>接口。关于在PaddlePaddle中如何使用Eigen库，请参考<a class="reference external" href="https://github.com/PaddlePaddle/Paddle/blob/develop/doc/howto/dev/use_eigen_cn.md">使用文档</a>。</p>
- `REGISTER_OP_CPU_KERNEL` ：注册`ops::MulKernel`类，并特化模板参数为`paddle::platform::CPUPlace`和`float`类型，同理，注册`ops::MulGradKernel`类。 <p>到此，前向Op实现完成。接下来，需要在<code class="docutils literal"><span class="pre">.cc</span></code>文件中注册该op和kernel。
反向Op类的定义，反向OpKernel的定义与前向Op类似，这里不再赘述。<strong>但需注意反向Op没有<code class="docutils literal"><span class="pre">ProtoMaker</span></code></strong>。</p>
</div>
<div class="section" id="operator">
<span id="id3"></span><h3>注册Operator<a class="headerlink" href="#operator" title="永久链接至标题"></a></h3>
<ul>
<li><p class="first">在<code class="docutils literal"><span class="pre">.cc</span></code>文件中注册前向、反向Op类，注册CPU Kernel。</p>
<div class="highlight-cpp"><div class="highlight"><pre><span></span><span class="k">namespace</span> <span class="n">ops</span> <span class="o">=</span> <span class="n">paddle</span><span class="o">::</span><span class="n">operators</span><span class="p">;</span>
<span class="n">REGISTER_OP</span><span class="p">(</span><span class="n">mul</span><span class="p">,</span> <span class="n">ops</span><span class="o">::</span><span class="n">MulOp</span><span class="p">,</span> <span class="n">ops</span><span class="o">::</span><span class="n">MulOpMaker</span><span class="p">,</span> <span class="n">mul_grad</span><span class="p">,</span> <span class="n">ops</span><span class="o">::</span><span class="n">MulOpGrad</span><span class="p">);</span>
<span class="n">REGISTER_OP_CPU_KERNEL</span><span class="p">(</span><span class="n">mul</span><span class="p">,</span> <span class="n">ops</span><span class="o">::</span><span class="n">MulKernel</span><span class="o">&lt;</span><span class="n">paddle</span><span class="o">::</span><span class="n">platform</span><span class="o">::</span><span class="n">CPUDeviceContext</span><span class="p">,</span> <span class="kt">float</span><span class="o">&gt;</span><span class="p">);</span>
<span class="n">REGISTER_OP_CPU_KERNEL</span><span class="p">(</span><span class="n">mul_grad</span><span class="p">,</span>
<span class="n">ops</span><span class="o">::</span><span class="n">MulGradKernel</span><span class="o">&lt;</span><span class="n">paddle</span><span class="o">::</span><span class="n">platform</span><span class="o">::</span><span class="n">CPUDeviceContext</span><span class="p">,</span> <span class="kt">float</span><span class="o">&gt;</span><span class="p">);</span>
</pre></div> </pre></div>
</div> </div>
<p>在上面的代码中：</p>
<ul class="simple">
<li><code class="docutils literal"><span class="pre">REGISTER_OP</span></code> ： 注册<code class="docutils literal"><span class="pre">ops::MulOp</span></code>类，类型名为<code class="docutils literal"><span class="pre">mul</span></code>，该类的<code class="docutils literal"><span class="pre">ProtoMaker</span></code>为<code class="docutils literal"><span class="pre">ops::MulOpMaker</span></code>，注册<code class="docutils literal"><span class="pre">ops::MulOpGrad</span></code>，类型名为<code class="docutils literal"><span class="pre">mul_grad</span></code>。</li>
<li><code class="docutils literal"><span class="pre">REGISTER_OP_WITHOUT_GRADIENT</span></code> ： 用于注册没有反向的Op。</li>
<li><code class="docutils literal"><span class="pre">REGISTER_OP_CPU_KERNEL</span></code> ：注册<code class="docutils literal"><span class="pre">ops::MulKernel</span></code>类，并特化模板参数为<code class="docutils literal"><span class="pre">paddle::platform::CPUPlace</span></code>和<code class="docutils literal"><span class="pre">float</span></code>类型，同理，注册<code class="docutils literal"><span class="pre">ops::MulGradKernel</span></code>类。</li>
</ul>
</li>
</ul>
<ul> <ul>
<li><p class="first">在 <code class="docutils literal"><span class="pre">.cu</span></code>文件中注册CUDA Kernel。</p> <li><p class="first">在 <code class="docutils literal"><span class="pre">.cu</span></code>文件中注册CUDA Kernel。</p>
<ul class="simple"> <ul class="simple">
...@@ -409,7 +408,7 @@ void Compute(const framework::ExecutionContext&amp; context) const override { ...@@ -409,7 +408,7 @@ void Compute(const framework::ExecutionContext&amp; context) const override {
</ul> </ul>
</div> </div>
<div class="section" id=""> <div class="section" id="">
<span id="id3"></span><h3>5. 编译<a class="headerlink" href="#" title="永久链接至标题"></a></h3> <span id="id4"></span><h3>编译<a class="headerlink" href="#" title="永久链接至标题"></a></h3>
<p>运行下面命令可以进行编译：</p> <p>运行下面命令可以进行编译：</p>
<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">make</span> <span class="n">mul_op</span> <div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">make</span> <span class="n">mul_op</span>
</pre></div> </pre></div>
...@@ -421,8 +420,10 @@ void Compute(const framework::ExecutionContext&amp; context) const override { ...@@ -421,8 +420,10 @@ void Compute(const framework::ExecutionContext&amp; context) const override {
<p>系统会对新增的op自动绑定Python，并链接到生成的lib库中。</p> <p>系统会对新增的op自动绑定Python，并链接到生成的lib库中。</p>
</div> </div>
<div class="section" id=""> <div class="section" id="">
<span id="id4"></span><h2>实现单元测试<a class="headerlink" href="#" title="永久链接至标题"></a></h2> <span id="id5"></span><h2>实现单元测试<a class="headerlink" href="#" title="永久链接至标题"></a></h2>
<p>单测包括对比前向Op不同设备(CPU、CUDA)的实现、对比反向OP不同设备(CPU、CUDA)的实现、反向Op的梯度测试。下面介绍介绍<a class="reference external" href="https://github.com/PaddlePaddle/Paddle/blob/develop/python/paddle/v2/framework/tests/test_mul_op.py"><code class="docutils literal"><span class="pre">MulOp</span></code>的单元测试</a>。</p> <p>单测包括对比前向Op不同设备(CPU、CUDA)的实现、对比反向OP不同设备(CPU、CUDA)的实现、反向Op的梯度测试。下面介绍介绍<a class="reference external" href="https://github.com/PaddlePaddle/Paddle/blob/develop/python/paddle/v2/framework/tests/test_mul_op.py"><code class="docutils literal"><span class="pre">MulOp</span></code>的单元测试</a>。</p>
<div class="section" id="operator">
<span id="id6"></span><h3>前向Operator单测<a class="headerlink" href="#operator" title="永久链接至标题"></a></h3>
<p>Op单元测试继承自<code class="docutils literal"><span class="pre">OpTest</span></code>。各项更加具体的单元测试在<code class="docutils literal"><span class="pre">TestMulOp</span></code>里完成。测试Operator，需要：</p> <p>Op单元测试继承自<code class="docutils literal"><span class="pre">OpTest</span></code>。各项更加具体的单元测试在<code class="docutils literal"><span class="pre">TestMulOp</span></code>里完成。测试Operator，需要：</p>
<ol class="simple"> <ol class="simple">
<li>在<code class="docutils literal"><span class="pre">setUp</span></code>函数定义输入、输出，以及相关的属性参数。</li> <li>在<code class="docutils literal"><span class="pre">setUp</span></code>函数定义输入、输出，以及相关的属性参数。</li>
...@@ -457,7 +458,6 @@ void Compute(const framework::ExecutionContext&amp; context) const override { ...@@ -457,7 +458,6 @@ void Compute(const framework::ExecutionContext&amp; context) const override {
<span class="k">def</span> <span class="nf">test_check_grad_ingore_y</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span> <span class="k">def</span> <span class="nf">test_check_grad_ingore_y</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">check_grad</span><span class="p">(</span> <span class="bp">self</span><span class="o">.</span><span class="n">check_grad</span><span class="p">(</span>
<span class="p">[</span><span class="s1">&#39;X&#39;</span><span class="p">],</span> <span class="s1">&#39;Out&#39;</span><span class="p">,</span> <span class="n">max_relative_error</span><span class="o">=</span><span class="mf">0.5</span><span class="p">,</span> <span class="n">no_grad_set</span><span class="o">=</span><span class="nb">set</span><span class="p">(</span><span class="s1">&#39;Y&#39;</span><span class="p">))</span> <span class="p">[</span><span class="s1">&#39;X&#39;</span><span class="p">],</span> <span class="s1">&#39;Out&#39;</span><span class="p">,</span> <span class="n">max_relative_error</span><span class="o">=</span><span class="mf">0.5</span><span class="p">,</span> <span class="n">no_grad_set</span><span class="o">=</span><span class="nb">set</span><span class="p">(</span><span class="s1">&#39;Y&#39;</span><span class="p">))</span>
</pre></div> </pre></div>
</div> </div>
<p>上面的代码首先导入依赖的包，下面是对<code class="docutils literal"><span class="pre">setUp</span></code>函数中操作的重要变量的详细解释：</p> <p>上面的代码首先导入依赖的包，下面是对<code class="docutils literal"><span class="pre">setUp</span></code>函数中操作的重要变量的详细解释：</p>
...@@ -466,6 +466,9 @@ void Compute(const framework::ExecutionContext&amp; context) const override { ...@@ -466,6 +466,9 @@ void Compute(const framework::ExecutionContext&amp; context) const override {
<li><code class="docutils literal"><span class="pre">self.inputs</span></code> : 定义输入，类型为<code class="docutils literal"><span class="pre">numpy.array</span></code>，并初始化。</li> <li><code class="docutils literal"><span class="pre">self.inputs</span></code> : 定义输入，类型为<code class="docutils literal"><span class="pre">numpy.array</span></code>，并初始化。</li>
<li><code class="docutils literal"><span class="pre">self.outputs</span></code> : 定义输出，并在Python脚本中完成与operator同样的计算逻辑，返回Python端的计算结果。</li> <li><code class="docutils literal"><span class="pre">self.outputs</span></code> : 定义输出，并在Python脚本中完成与operator同样的计算逻辑，返回Python端的计算结果。</li>
</ul> </ul>
</div>
<div class="section" id="operator">
<span id="id7"></span><h3>反向operator单测<a class="headerlink" href="#operator" title="永久链接至标题"></a></h3>
<p>而反向测试中：</p> <p>而反向测试中：</p>
<ul class="simple"> <ul class="simple">
<li><code class="docutils literal"><span class="pre">test_check_grad_normal</span></code>中调用<code class="docutils literal"><span class="pre">check_grad</span></code>使用数值法检测梯度正确性和稳定性。<ul> <li><code class="docutils literal"><span class="pre">test_check_grad_normal</span></code>中调用<code class="docutils literal"><span class="pre">check_grad</span></code>使用数值法检测梯度正确性和稳定性。<ul>
...@@ -476,8 +479,9 @@ void Compute(const framework::ExecutionContext&amp; context) const override { ...@@ -476,8 +479,9 @@ void Compute(const framework::ExecutionContext&amp; context) const override {
</li> </li>
<li><code class="docutils literal"><span class="pre">test_check_grad_ingore_x</span></code>和<code class="docutils literal"><span class="pre">test_check_grad_ingore_y</span></code>分支用来测试只需要计算一个输入梯度的情况。</li> <li><code class="docutils literal"><span class="pre">test_check_grad_ingore_x</span></code>和<code class="docutils literal"><span class="pre">test_check_grad_ingore_y</span></code>分支用来测试只需要计算一个输入梯度的情况。</li>
</ul> </ul>
</div>
<div class="section" id=""> <div class="section" id="">
<span id="id5"></span><h3>编译和执行单元测试<a class="headerlink" href="#" title="永久链接至标题"></a></h3> <span id="id8"></span><h3>编译和执行<a class="headerlink" href="#" title="永久链接至标题"></a></h3>
<p><code class="docutils literal"><span class="pre">python/paddle/v2/framework/tests</span></code> 目录下新增的 <code class="docutils literal"><span class="pre">test_*.py</span></code> 单元测试会被自动加入工程进行编译。</p> <p><code class="docutils literal"><span class="pre">python/paddle/v2/framework/tests</span></code> 目录下新增的 <code class="docutils literal"><span class="pre">test_*.py</span></code> 单元测试会被自动加入工程进行编译。</p>
<p>请注意，<strong>不同于Op的编译测试，运行单元测试测时需要编译整个工程</strong>，并且编译时需要打开<code class="docutils literal"><span class="pre">WITH_TESTING</span></code>, 即<code class="docutils literal"><span class="pre">cmake</span> <span class="pre">paddle_dir</span> <span class="pre">-DWITH_TESTING=ON</span></code>。编译成功后，执行下面的命令来运行单元测试：</p> <p>请注意，<strong>不同于Op的编译测试，运行单元测试测时需要编译整个工程</strong>，并且编译时需要打开<code class="docutils literal"><span class="pre">WITH_TESTING</span></code>, 即<code class="docutils literal"><span class="pre">cmake</span> <span class="pre">paddle_dir</span> <span class="pre">-DWITH_TESTING=ON</span></code>。编译成功后，执行下面的命令来运行单元测试：</p>
<div class="highlight-bash"><div class="highlight"><pre><span></span>make <span class="nb">test</span> <span class="nv">ARGS</span><span class="o">=</span><span class="s2">&quot;-R test_mul_op -V&quot;</span> <div class="highlight-bash"><div class="highlight"><pre><span></span>make <span class="nb">test</span> <span class="nv">ARGS</span><span class="o">=</span><span class="s2">&quot;-R test_mul_op -V&quot;</span>
...@@ -490,7 +494,7 @@ void Compute(const framework::ExecutionContext&amp; context) const override { ...@@ -490,7 +494,7 @@ void Compute(const framework::ExecutionContext&amp; context) const override {
</div> </div>
</div> </div>
<div class="section" id=""> <div class="section" id="">
<span id="id6"></span><h2>注意事项<a class="headerlink" href="#" title="永久链接至标题"></a></h2> <span id="id9"></span><h2>注意事项<a class="headerlink" href="#" title="永久链接至标题"></a></h2>
<ul class="simple"> <ul class="simple">
<li>为每个Op创建单独的<code class="docutils literal"><span class="pre">*_op.h</span></code>（如有）、<code class="docutils literal"><span class="pre">*_op.cc</span></code>和<code class="docutils literal"><span class="pre">*_op.cu</span></code>（如有）。不允许一个文件中包含多个Op，这将会导致编译出错。</li> <li>为每个Op创建单独的<code class="docutils literal"><span class="pre">*_op.h</span></code>（如有）、<code class="docutils literal"><span class="pre">*_op.cc</span></code>和<code class="docutils literal"><span class="pre">*_op.cu</span></code>（如有）。不允许一个文件中包含多个Op，这将会导致编译出错。</li>
<li>注册Op时的类型名，需要和该Op的名字一样。即不允许在<code class="docutils literal"><span class="pre">A_op.cc</span></code>里面，注册<code class="docutils literal"><span class="pre">REGISTER_OP(B,</span> <span class="pre">...)</span></code>等，这将会导致单元测试出错。</li> <li>注册Op时的类型名，需要和该Op的名字一样。即不允许在<code class="docutils literal"><span class="pre">A_op.cc</span></code>里面，注册<code class="docutils literal"><span class="pre">REGISTER_OP(B,</span> <span class="pre">...)</span></code>等，这将会导致单元测试出错。</li>
......
develop/doc_cn/searchindex.js
浏览文件 @ 4377bc79
因为 它太大了无法显示 source diff 。你可以改为 查看blob。
Markdown is supported
0% 或 .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑！
想要评论请 注册 或