Usually `OpProtoMaker` and `Op`'s type definitions are written in `.cc` files, which also include the registration methods introduced later.
### 3. Defining OpKernel
### Defining OpKernel
`MulKernel` inherits `framework::OpKernel`, which includes the following templates:
...
...
@@ -188,7 +188,7 @@ This concludes the forward implementation of an operator. Next its operation and
The definition of its corresponding backward operator, if applicable, is similar to that of an forward operator. **Note that a backward operator does not include a `ProtoMaker`**.
### 4. Registering Operator
### Registering Operator
- In `.cc` files, register forward and backward operator classes and the CPU kernel.
...
...
@@ -220,7 +220,7 @@ The definition of its corresponding backward operator, if applicable, is similar
<spanid="how-to-write-a-new-operator"></span><h1>How to write a new operator<aclass="headerlink"href="#how-to-write-a-new-operator"title="Permalink to this headline">¶</a></h1>
@@ -244,8 +244,8 @@ Registering the Op | Ops are registered in <code class="docutils liter
</div>
<divclass="section"id="implementing-c-types">
<spanid="implementing-c-types"></span><h2>Implementing C++ Types<aclass="headerlink"href="#implementing-c-types"title="Permalink to this headline">¶</a></h2>
<spanid="defining-class-protomaker"></span><h3>1. Defining Class ProtoMaker<aclass="headerlink"href="#defining-class-protomaker"title="Permalink to this headline">¶</a></h3>
<divclass="section"id="defining-protomaker">
<spanid="defining-protomaker"></span><h3>Defining ProtoMaker<aclass="headerlink"href="#defining-protomaker"title="Permalink to this headline">¶</a></h3>
<p>Matrix Multiplication can be written as $Out = X * Y$, meaning that the operation consists of two inputs and pne output.</p>
<p>First, define <codeclass="docutils literal"><spanclass="pre">ProtoMaker</span></code> to describe the Operator’s input, output, and additional comments:</p>
@@ -336,7 +336,7 @@ Registering the Op | Ops are registered in <code class="docutils liter
<p>Usually <codeclass="docutils literal"><spanclass="pre">OpProtoMaker</span></code> and <codeclass="docutils literal"><spanclass="pre">Op</span></code>‘s type definitions are written in <codeclass="docutils literal"><spanclass="pre">.cc</span></code> files, which also include the registration methods introduced later.</p>
</div>
<divclass="section"id="defining-opkernel">
<spanid="defining-opkernel"></span><h3>3. Defining OpKernel<aclass="headerlink"href="#defining-opkernel"title="Permalink to this headline">¶</a></h3>
<spanid="defining-opkernel"></span><h3>Defining OpKernel<aclass="headerlink"href="#defining-opkernel"title="Permalink to this headline">¶</a></h3>
<p><codeclass="docutils literal"><spanclass="pre">MulKernel</span></code> inherits <codeclass="docutils literal"><spanclass="pre">framework::OpKernel</span></code>, which includes the following templates:</p>
<ulclass="simple">
<li><codeclass="docutils literal"><spanclass="pre">typename</span><spanclass="pre">DeviceContext</span></code> denotes device context type. When different devices, namely the CPUDeviceContext and the CUDADeviceContext, share the same kernel, this template needs to be added. If they don’t share kernels, this must not be added. An example of a non-sharing kernel is <aclass="reference external"href="https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/operators/cross_entropy_op.h#L43"><codeclass="docutils literal"><spanclass="pre">OnehotCrossEntropyOpKernel</span></code></a>.</li>
...
...
@@ -370,7 +370,7 @@ Registering the Op | Ops are registered in <code class="docutils liter
<p>The definition of its corresponding backward operator, if applicable, is similar to that of an forward operator. <strong>Note that a backward operator does not include a <codeclass="docutils literal"><spanclass="pre">ProtoMaker</span></code></strong>.</p>
</div>
<divclass="section"id="registering-operator">
<spanid="registering-operator"></span><h3>4. Registering Operator<aclass="headerlink"href="#registering-operator"title="Permalink to this headline">¶</a></h3>
<spanid="registering-operator"></span><h3>Registering Operator<aclass="headerlink"href="#registering-operator"title="Permalink to this headline">¶</a></h3>
<ul>
<li><pclass="first">In <codeclass="docutils literal"><spanclass="pre">.cc</span></code> files, register forward and backward operator classes and the CPU kernel.</p>
<p>Get its output, and compare it with the forward operator’s own output.</p>
...
...
@@ -472,6 +471,9 @@ Registering the Op | Ops are registered in <code class="docutils liter
<li><codeclass="docutils literal"><spanclass="pre">self.inputs</span></code> defines input, with type <codeclass="docutils literal"><spanclass="pre">numpy.array</span></code> and initializes it.</li>
<li><codeclass="docutils literal"><spanclass="pre">self.outputs</span></code> defines output and completes the same operator computation in the Python script, and returns its result from the Python script.</li>
<spanid="testing-backward-operators"></span><h3>Testing Backward Operators<aclass="headerlink"href="#testing-backward-operators"title="Permalink to this headline">¶</a></h3>
<p>Some key points in checking gradient above include:</p>
<ulclass="simple">
<li><codeclass="docutils literal"><spanclass="pre">test_normal</span></code> calls <codeclass="docutils literal"><spanclass="pre">check_grad</span></code> to validate scaling tests’ correctness and stability through numeric methods.<ul>
为了使`OpKernel`的计算过程书写更加简单，并且CPU、CUDA的代码可以复用，我们通常借助 Eigen unsupported Tensor模块来实现`Compute`接口。关于在PaddlePaddle中如何使用Eigen库，请参考[使用文档](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/howto/dev/use_eigen_cn.md)。
