提交 4377bc79 编写于 作者: T Travis CI

Deploy to GitHub Pages: 3ef8ec37

上级 c283efe7
# How to write a new operator
- [Background](#background)
- [Implementing C++ Types](#implementing-c++-types)
- [Defining ProtoMaker](#defining-protoMaker)
- [Implementing C++ Types](#implementing-c-types)
- [Defining ProtoMaker](#defining-protomaker)
- [Defining Operator](#defining-operator)
- [Registering Operator](#registering-operator)
- [Compilation](#compilation)
......@@ -41,7 +41,7 @@ Let's take matrix multiplication operator, [MulOp](https://github.com/PaddlePadd
## Implementing C++ Types
### 1. Defining Class ProtoMaker
### Defining ProtoMaker
Matrix Multiplication can be written as $Out = X * Y$, meaning that the operation consists of two inputs and pne output.
......@@ -98,7 +98,7 @@ There are two changes in this example:
- `AddAttr<AttrType>("scale", "...").SetDefault(1.0);` adds `scale`constant as an attribute, and sets the default value to 1.0.
### 2. Defining Operator
### Defining Operator
The following code defines the interface for MulOp:
......@@ -147,7 +147,7 @@ MulOp(const std::string &type, const framework::VariableNameMap &inputs,
Usually `OpProtoMaker` and `Op`'s type definitions are written in `.cc` files, which also include the registration methods introduced later.
### 3. Defining OpKernel
### Defining OpKernel
`MulKernel` inherits `framework::OpKernel`, which includes the following templates:
......@@ -188,7 +188,7 @@ This concludes the forward implementation of an operator. Next its operation and
The definition of its corresponding backward operator, if applicable, is similar to that of an forward operator. **Note that a backward operator does not include a `ProtoMaker`**.
### 4. Registering Operator
### Registering Operator
- In `.cc` files, register forward and backward operator classes and the CPU kernel.
......@@ -220,7 +220,7 @@ The definition of its corresponding backward operator, if applicable, is similar
ops::MulGradKernel<paddle::platform::CUDADeviceContext, float>);
```
### 5. Compilation
### Compilation
Run the following commands to compile.
......@@ -284,8 +284,7 @@ A forward operator unit test inherits `unittest.TestCase` and defines metaclass
def test_check_grad_ingore_y(self):
self.check_grad(
['X'], 'Out', max_relative_error=0.5, no_grad_set=set('Y'))
```
```
Get its output, and compare it with the forward operator's own output.
The code above first loads required packages. In addition, we have
......@@ -294,6 +293,8 @@ The code above first loads required packages. In addition, we have
- `self.inputs` defines input, with type `numpy.array` and initializes it.
- `self.outputs` defines output and completes the same operator computation in the Python script, and returns its result from the Python script.
### Testing Backward Operators
Some key points in checking gradient above include:
- `test_normal` calls `check_grad` to validate scaling tests' correctness and stability through numeric methods.
......
......@@ -207,8 +207,8 @@
<span id="how-to-write-a-new-operator"></span><h1>How to write a new operator<a class="headerlink" href="#how-to-write-a-new-operator" title="Permalink to this headline"></a></h1>
<ul class="simple">
<li><a class="reference external" href="#background">Background</a></li>
<li><a class="reference external" href="#implementing-c++-types">Implementing C++ Types</a><ul>
<li><a class="reference external" href="#defining-protoMaker">Defining ProtoMaker</a></li>
<li><a class="reference external" href="#implementing-c-types">Implementing C++ Types</a><ul>
<li><a class="reference external" href="#defining-protomaker">Defining ProtoMaker</a></li>
<li><a class="reference external" href="#defining-operator">Defining Operator</a></li>
<li><a class="reference external" href="#registering-operator">Registering Operator</a></li>
<li><a class="reference external" href="#compilation">Compilation</a></li>
......@@ -244,8 +244,8 @@ Registering the Op | Ops are registered in <code class="docutils liter
</div>
<div class="section" id="implementing-c-types">
<span id="implementing-c-types"></span><h2>Implementing C++ Types<a class="headerlink" href="#implementing-c-types" title="Permalink to this headline"></a></h2>
<div class="section" id="defining-class-protomaker">
<span id="defining-class-protomaker"></span><h3>1. Defining Class ProtoMaker<a class="headerlink" href="#defining-class-protomaker" title="Permalink to this headline"></a></h3>
<div class="section" id="defining-protomaker">
<span id="defining-protomaker"></span><h3>Defining ProtoMaker<a class="headerlink" href="#defining-protomaker" title="Permalink to this headline"></a></h3>
<p>Matrix Multiplication can be written as $Out = X * Y$, meaning that the operation consists of two inputs and pne output.</p>
<p>First, define <code class="docutils literal"><span class="pre">ProtoMaker</span></code> to describe the Operator&#8217;s input, output, and additional comments:</p>
<div class="highlight-cpp"><div class="highlight"><pre><span></span><span class="k">class</span> <span class="nc">MulOpMaker</span> <span class="o">:</span> <span class="k">public</span> <span class="n">framework</span><span class="o">::</span><span class="n">OpProtoAndCheckerMaker</span> <span class="p">{</span>
......@@ -293,7 +293,7 @@ Registering the Op | Ops are registered in <code class="docutils liter
</ul>
</div>
<div class="section" id="defining-operator">
<span id="defining-operator"></span><h3>2. Defining Operator<a class="headerlink" href="#defining-operator" title="Permalink to this headline"></a></h3>
<span id="defining-operator"></span><h3>Defining Operator<a class="headerlink" href="#defining-operator" title="Permalink to this headline"></a></h3>
<p>The following code defines the interface for MulOp:</p>
<div class="highlight-cpp"><div class="highlight"><pre><span></span><span class="k">class</span> <span class="nc">MulOp</span> <span class="o">:</span> <span class="k">public</span> <span class="n">framework</span><span class="o">::</span><span class="n">OperatorWithKernel</span> <span class="p">{</span>
<span class="k">public</span><span class="o">:</span>
......@@ -336,7 +336,7 @@ Registering the Op | Ops are registered in <code class="docutils liter
<p>Usually <code class="docutils literal"><span class="pre">OpProtoMaker</span></code> and <code class="docutils literal"><span class="pre">Op</span></code>&#8216;s type definitions are written in <code class="docutils literal"><span class="pre">.cc</span></code> files, which also include the registration methods introduced later.</p>
</div>
<div class="section" id="defining-opkernel">
<span id="defining-opkernel"></span><h3>3. Defining OpKernel<a class="headerlink" href="#defining-opkernel" title="Permalink to this headline"></a></h3>
<span id="defining-opkernel"></span><h3>Defining OpKernel<a class="headerlink" href="#defining-opkernel" title="Permalink to this headline"></a></h3>
<p><code class="docutils literal"><span class="pre">MulKernel</span></code> inherits <code class="docutils literal"><span class="pre">framework::OpKernel</span></code>, which includes the following templates:</p>
<ul class="simple">
<li><code class="docutils literal"><span class="pre">typename</span> <span class="pre">DeviceContext</span></code> denotes device context type. When different devices, namely the CPUDeviceContext and the CUDADeviceContext, share the same kernel, this template needs to be added. If they don&#8217;t share kernels, this must not be added. An example of a non-sharing kernel is <a class="reference external" href="https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/operators/cross_entropy_op.h#L43"><code class="docutils literal"><span class="pre">OnehotCrossEntropyOpKernel</span></code></a>.</li>
......@@ -370,7 +370,7 @@ Registering the Op | Ops are registered in <code class="docutils liter
<p>The definition of its corresponding backward operator, if applicable, is similar to that of an forward operator. <strong>Note that a backward operator does not include a <code class="docutils literal"><span class="pre">ProtoMaker</span></code></strong>.</p>
</div>
<div class="section" id="registering-operator">
<span id="registering-operator"></span><h3>4. Registering Operator<a class="headerlink" href="#registering-operator" title="Permalink to this headline"></a></h3>
<span id="registering-operator"></span><h3>Registering Operator<a class="headerlink" href="#registering-operator" title="Permalink to this headline"></a></h3>
<ul>
<li><p class="first">In <code class="docutils literal"><span class="pre">.cc</span></code> files, register forward and backward operator classes and the CPU kernel.</p>
<div class="highlight-cpp"><div class="highlight"><pre><span></span><span class="k">namespace</span> <span class="n">ops</span> <span class="o">=</span> <span class="n">paddle</span><span class="o">::</span><span class="n">operators</span><span class="p">;</span>
......@@ -406,7 +406,7 @@ Registering the Op | Ops are registered in <code class="docutils liter
</ul>
</div>
<div class="section" id="compilation">
<span id="compilation"></span><h3>5. Compilation<a class="headerlink" href="#compilation" title="Permalink to this headline"></a></h3>
<span id="compilation"></span><h3>Compilation<a class="headerlink" href="#compilation" title="Permalink to this headline"></a></h3>
<p>Run the following commands to compile.</p>
<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">make</span> <span class="n">mul_op</span>
</pre></div>
......@@ -462,7 +462,6 @@ Registering the Op | Ops are registered in <code class="docutils liter
<span class="k">def</span> <span class="nf">test_check_grad_ingore_y</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">check_grad</span><span class="p">(</span>
<span class="p">[</span><span class="s1">&#39;X&#39;</span><span class="p">],</span> <span class="s1">&#39;Out&#39;</span><span class="p">,</span> <span class="n">max_relative_error</span><span class="o">=</span><span class="mf">0.5</span><span class="p">,</span> <span class="n">no_grad_set</span><span class="o">=</span><span class="nb">set</span><span class="p">(</span><span class="s1">&#39;Y&#39;</span><span class="p">))</span>
</pre></div>
</div>
<p>Get its output, and compare it with the forward operator&#8217;s own output.</p>
......@@ -472,6 +471,9 @@ Registering the Op | Ops are registered in <code class="docutils liter
<li><code class="docutils literal"><span class="pre">self.inputs</span></code> defines input, with type <code class="docutils literal"><span class="pre">numpy.array</span></code> and initializes it.</li>
<li><code class="docutils literal"><span class="pre">self.outputs</span></code> defines output and completes the same operator computation in the Python script, and returns its result from the Python script.</li>
</ul>
</div>
<div class="section" id="testing-backward-operators">
<span id="testing-backward-operators"></span><h3>Testing Backward Operators<a class="headerlink" href="#testing-backward-operators" title="Permalink to this headline"></a></h3>
<p>Some key points in checking gradient above include:</p>
<ul class="simple">
<li><code class="docutils literal"><span class="pre">test_normal</span></code> calls <code class="docutils literal"><span class="pre">check_grad</span></code> to validate scaling tests&#8217; correctness and stability through numeric methods.<ul>
......
此差异已折叠。
因为 它太大了无法显示 source diff 。你可以改为 查看blob
# 如何写新的Operator
- [概念简介](#概念简介)
- [实现C++类](#实现C++类)
- [定义ProtoMaker类](#定义ProtoMaker类)
- [定义Operator类](#定义Operator类)
- [定义OpKernel类](#定义OpKernel类)
- [注册Operator](#注册Operator)
- [实现C++类](#实现c类)
- [定义ProtoMaker类](#定义protomaker类)
- [定义Operator类](#定义operator类)
- [定义OpKernel类](#定义opkernel类)
- [注册Operator](#注册operator)
- [编译](#编译)
- [绑定Python](#绑定Python)
- [绑定Python](#绑定python)
- [实现单元测试](#实现单元测试)
- [前向Operator单测](#前向Operator单测)
- [反向Operator单测](#反向Operator单测)
- [前向Operator单测](#前向operator单测)
- [反向Operator单测](#反向operator单测)
- [编译和执行](#编译和执行)
- [注意事项](#注意事项)
## 概念简介
......@@ -43,7 +44,7 @@ Kernel实现 | CPU、CUDA共享Kernel实现在`.h`文件中,否则,CPU
## 实现C++类
### 1. 定义ProtoMaker类
### 定义ProtoMaker类
矩阵乘法的公式:$Out = X * Y$, 可见该计算由两个输入,一个输出组成。
......@@ -100,7 +101,7 @@ The equation is: Out = scale*X
- `AddAttr<AttrType>("scale", "...").SetDefault(1.0);` : 增加`scale`系数,作为参数属性,并且设置默认值为1.0。
### 2. 定义Operator类
### 定义Operator类
下面的点实现了MulOp的定义:
......@@ -149,7 +150,7 @@ MulOp(const std::string &type, const framework::VariableNameMap &inputs,
通常`OpProtoMaker`和`Op`类的定义写在`.cc`文件中,和下面将要介绍的注册函数一起放在`.cc`中
### 3. 定义OpKernel类
### 定义OpKernel类
`MulKernel`继承自`framework::OpKernel`,带有下面两个模板参数:
......@@ -177,6 +178,7 @@ MulOp(const std::string &type, const framework::VariableNameMap &inputs,
math::matmul<DeviceContext, T>(*X, false, *Y, false, 1, Z, 0, device_context);
}
};
```
需要注意:**不同设备(CPU、CUDA)共享一个Op定义,是否则共享同一个`OpKernel`,取决于`Compute`调用的函数是否支持不同设备。**
......@@ -188,7 +190,7 @@ MulOp(const std::string &type, const framework::VariableNameMap &inputs,
到此,前向Op实现完成。接下来,需要在`.cc`文件中注册该op和kernel。
反向Op类的定义,反向OpKernel的定义与前向Op类似,这里不再赘述。**但需注意反向Op没有`ProtoMaker`**。
### 4. 注册Operator
### 注册Operator
- 在`.cc`文件中注册前向、反向Op类,注册CPU Kernel。
......@@ -220,7 +222,7 @@ MulOp(const std::string &type, const framework::VariableNameMap &inputs,
ops::MulGradKernel<paddle::platform::CUDADeviceContext, float>);
```
### 5. 编译
### 编译
运行下面命令可以进行编译:
......@@ -236,6 +238,7 @@ make mul_op
单测包括对比前向Op不同设备(CPU、CUDA)的实现、对比反向OP不同设备(CPU、CUDA)的实现、反向Op的梯度测试。下面介绍介绍[`MulOp`的单元测试](https://github.com/PaddlePaddle/Paddle/blob/develop/python/paddle/v2/framework/tests/test_mul_op.py)。
### 前向Operator单测
Op单元测试继承自`OpTest`。各项更加具体的单元测试在`TestMulOp`里完成。测试Operator,需要:
......@@ -273,8 +276,7 @@ Op单元测试继承自`OpTest`。各项更加具体的单元测试在`TestMulOp
def test_check_grad_ingore_y(self):
self.check_grad(
['X'], 'Out', max_relative_error=0.5, no_grad_set=set('Y'))
```
```
上面的代码首先导入依赖的包,下面是对`setUp`函数中操作的重要变量的详细解释:
......@@ -282,6 +284,8 @@ Op单元测试继承自`OpTest`。各项更加具体的单元测试在`TestMulOp
- `self.inputs` : 定义输入,类型为`numpy.array`,并初始化。
- `self.outputs` : 定义输出,并在Python脚本中完成与operator同样的计算逻辑,返回Python端的计算结果。
### 反向operator单测
而反向测试中:
- `test_check_grad_normal`中调用`check_grad`使用数值法检测梯度正确性和稳定性。
- 第一个参数`["X", "Y"]` : 指定对输入变量`X`、`Y`做梯度检测。
......@@ -290,7 +294,7 @@ Op单元测试继承自`OpTest`。各项更加具体的单元测试在`TestMulOp
- `test_check_grad_ingore_x`和`test_check_grad_ingore_y`分支用来测试只需要计算一个输入梯度的情况。
### 编译和执行单元测试
### 编译和执行
`python/paddle/v2/framework/tests` 目录下新增的 `test_*.py` 单元测试会被自动加入工程进行编译。
......
此差异已折叠。
Markdown is supported
0% .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑!
想要评论请 注册