- `framework::OpKernel`: Base class for Op computation.
- `framework::OperatorWithKernel`: Inherited from OperatorBase, describing an operator with computation.
- `class OpProtoAndCheckerMaker`: Describes an Operator's input, output, attributes and description, mainly used to interface with Python API.
An operator can be differentiated by whether in has kernel methods. An operator with kernel inherits from `OperatorWithKernel` while the ones without inherit from `OperatorBase`. This tutorial focuses on implementing operators with kernels. In short, an operator includes the following information:
Information | Where is it defined
-------------- | :----------------------
OpProtoMake definition | `.cc`files, Backward Op does not need an OpProtoMake interface.
Op definition | `.cc` files
Kernel implementation | The kernel methods shared between CPU and GPU are defined in `.h` files. CPU-specific kernels live in `.cc` files, while GPU-specific kernels are implemented in `.cu`files.
Registering the Op | Ops are registered in `.cc` files; For Kernel registration, `.cc` files contain the CPU implementation, while `.cu` files contain the GPU implementation.
New Operator implementations are added to the list [paddle/operators](https://github.com/PaddlePaddle/Paddle/tree/develop/paddle/operators), with file names in the format `*_op.h` (if applicable), `*_op.cc`, `*_op.cu` (if applicable).** The system will use the naming scheme to automatically build operators and their corresponding Python extensions. **
Let's take matrix multiplication operator, [MulOp](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/operators/mul_op.cc), as an example to introduce the writing of an Operator with Kernel.
## Implementing C++ Types
### 1. Defining Class ProtoMaker
Matrix Multiplication can be written as $Out = X * Y$, meaning that the operation consists of two inputs and pne output.
First, define `ProtoMaker` to describe the Operator's input, output, and additional comments:
```cpp
class MulOpMaker : public framework::OpProtoAndCheckerMaker {
AddInput("X", "(Tensor), 2D tensor of size (M x K)");
AddInput("Y", "(Tensor), 2D tensor of size (K x N)");
AddOutput("Out", "(Tensor), 2D tensor of size (M x N)");
AddComment(R"DOC(
Two Element Mul Operator.
The equation is: Out = X * Y
)DOC");
}
};
```
[`MulOpMaker`](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/operators/mul_op.cc#L43)is inherited from`framework::OpProtoAndCheckerMaker`, consisting of 2 variables in the constructor：
- `framework::OpProto` stores Operator input and variable attribute, used for generating Python API interfaces.
- `framework::OpAttrChecker` is used to validate variable attributes.
The constructor utilizes `AddInput`, `AddOutput`, and `AddComment`, so that the corresponding information will be added to `OpProto`.
The code above adds two inputs `X` and `Y` to `MulOp`, an output `Out`, and their corresponding descriptions, in accordance to Paddle's [naming convention](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/operators/name_convention.md).
An additional example [`ScaleOp`](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/operators/scale_op.cc#L37) is implemented as follows:
```cpp
template <typename AttrType>
class ScaleOpMaker : public framework::OpProtoAndCheckerMaker {
AddInput("X", "The input tensor of scale operator.").NotInGradient();
AddOutput("Out", "The output tensor of scale operator.").NotInGradient();
AddComment(R"DOC(Scale operator
The equation is: Out = scale*X
)DOC");
AddAttr<AttrType>("scale", "scale of scale operator.").SetDefault(1.0);
}
};
```
There are two changes in this example:
- `AddInput("X","...").NotInGradient()` expresses that input `X` is not involved in `ScaleOp`'s corresponding computation. If an input to an operator is not participating in back-propagation, please explicitly set `.NotInGradient()`.
- `AddAttr<AttrType>("scale", "...").SetDefault(1.0);` adds `scale`constant as an attribute, and sets the default value to 1.0.
### 2. Defining Operator
The following code defines the interface for MulOp:
```cpp
class MulOp : public framework::OperatorWithKernel {
public:
using framework::OperatorWithKernel::OperatorWithKernel;
[`MulOp`](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/operators/mul_op.cc#L22) is inherited from `OperatorWithKernel`. Its `public` member
```cpp
using framework::OperatorWithKernel::OperatorWithKernel;
```
expresses an operator constructor using base class `OperatorWithKernel`, alternatively written as
`InferShape` interface needs to be re-written.`InferShape` is a constant method and cannot modify Op's member variables, its constant member `const framework::InferShapeContext &ctx` can be used to extract input, output, and attributes. It functions to
- 1). validate and error out early: it checks input data dimensions and types.
- 2). configures the tensor shape in the output.
Usually `OpProtoMaker` and `Op`'s type definitions are written in `.cc` files, which also include the registration methods introduced later.
### 3. Defining OpKernel
`MulKernel` inherits `framework::OpKernel`, which includes the following templates:
- `typename Place` denotes device type. When different devices, namely the CPU and the GPU, share the same kernel, this template needs to be added. If they don't share kernels, this must not be added. An example of a non-sharing kernel is [`OnehotCrossEntropyOpKernel`](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/operators/cross_entropy_op.h#L43).
- `typename T` denotes data type, such as `float` or `double`.
`MulKernel` types need to rewrite the interface for `Compute`.
- `Compute` takes one input variable `const framework::ExecutionContext& context`.
- Compared with `InferShapeContext`, `ExecutionContext` includes device types, and can similarly extract input, output, and attribute variables.
- `Compute` implements the computation logics of an `OpKernel`.
`MulKernel`'s implementation of `Compute` is as follows:
Note that **different devices (CPU, GPU)share an Op definition; whether or not they share the same `OpKernel` depends on whether `Compute` calls functions that support both devices.**
`MulOp`'s CPU and GPU share the same `Kernel`. A non-sharing `OpKernel` example can be seen in [`OnehotCrossEntropyOpKernel`](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/operators/cross_entropy_op.h#L43).
To ease the writing of `OpKernel` compute, and for reusing code cross-device, `Eigen unsupported Tensor` module is used to implement `Compute` interface. To learn about how the Eigen library is used in PaddlePaddle, please see [usage document](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/howto/dev/use_eigen_cn.md).
This concludes the forward implementation of an operator. Next its operation and kernel need to be registered in a `.cc` file.
The definition of its corresponding backward operator, if applicable, is similar to that of an forward operator. **Note that a backward operator does not include a `ProtoMaker`**.
### 4. Registering Operator
- In `.cc` files, register forward and backward operator classes and the CPU kernel.
- `REGISTER_OP` registers the `ops::MulOp` class, type named `mul`, its type `ProtoMaker` is `ops::MulOpMaker`, registering `ops::MulOpGrad` as `mul_grad`.
- `REGISTER_OP_WITHOUT_GRADIENT` registers an operator without gradient.
- `REGISTER_OP_CPU_KERNEL` registers `ops::MulKernel` class and specialized template types `paddle::platform::CPUPlace` and `float`, which also registers `ops::MulKernel`.
- Registering GPU Kernel in `.cu` files
- Note that if GPU Kernel is implemented using the `Eigen unsupported` module, then on top of `.cu`, a macro definition `#define EIGEN_USE_GPU` is needed, such as
```cpp
// if use Eigen unsupported module before include head files
The system will automatically bind to Python and link it to a generated library.
## Unit Tests
Unit tests include comparing a forward operator's implementations on different devices, comparing a backward operator's implementation on different devices, and a scaling test for the backward operator. Here, we introduce the [unit tests for `MulOp`](https://github.com/PaddlePaddle/Paddle/blob/develop/python/paddle/v2/framework/tests/test_mul_op.py).
<liclass="toctree-l2"><aclass="reference internal"href="../../getstarted/build_and_install/index_en.html">Install and Build</a><ul>
<liclass="toctree-l3"><aclass="reference internal"href="../../getstarted/build_and_install/docker_install_en.html">PaddlePaddle in Docker Containers</a></li>
<liclass="toctree-l3"><aclass="reference internal"href="../../getstarted/build_and_install/build_from_source_en.html">Installing from Sources</a></li>
<liclass="toctree-l2"><aclass="reference internal"href="../usage/k8s/k8s_en.html">Paddle On Kubernetes</a></li>
<liclass="toctree-l2"><aclass="reference internal"href="../usage/k8s/k8s_aws_en.html">Distributed PaddlePaddle Training on AWS with Kubernetes</a></li>
<liclass="toctree-l2"><aclass="reference internal"href="build_en.html">Build PaddlePaddle from Source Code and Run Unit Test</a></li>
<liclass="toctree-l2"><aclass="reference internal"href="new_layer_en.html">Write New Layers</a></li>
<spanid="how-to-write-a-new-operator"></span><h1>How to write a new operator<aclass="headerlink"href="#how-to-write-a-new-operator"title="Permalink to this headline">¶</a></h1>
<li><codeclass="docutils literal"><spanclass="pre">framework::OpKernel</span></code>: Base class for Op computation.</li>
<li><codeclass="docutils literal"><spanclass="pre">framework::OperatorWithKernel</span></code>: Inherited from OperatorBase, describing an operator with computation.</li>
<li><codeclass="docutils literal"><spanclass="pre">class</span><spanclass="pre">OpProtoAndCheckerMaker</span></code>: Describes an Operator’s input, output, attributes and description, mainly used to interface with Python API.</li>
</ul>
<p>An operator can be differentiated by whether in has kernel methods. An operator with kernel inherits from <codeclass="docutils literal"><spanclass="pre">OperatorWithKernel</span></code> while the ones without inherit from <codeclass="docutils literal"><spanclass="pre">OperatorBase</span></code>. This tutorial focuses on implementing operators with kernels. In short, an operator includes the following information:</p>
OpProtoMake definition | <codeclass="docutils literal"><spanclass="pre">.cc</span></code>files, Backward Op does not need an OpProtoMake interface.
Op definition | <codeclass="docutils literal"><spanclass="pre">.cc</span></code> files
Kernel implementation | The kernel methods shared between CPU and GPU are defined in <codeclass="docutils literal"><spanclass="pre">.h</span></code> files. CPU-specific kernels live in <codeclass="docutils literal"><spanclass="pre">.cc</span></code> files, while GPU-specific kernels are implemented in <codeclass="docutils literal"><spanclass="pre">.cu</span></code>files.
Registering the Op | Ops are registered in <codeclass="docutils literal"><spanclass="pre">.cc</span></code> files; For Kernel registration, <codeclass="docutils literal"><spanclass="pre">.cc</span></code> files contain the CPU implementation, while <codeclass="docutils literal"><spanclass="pre">.cu</span></code> files contain the GPU implementation.</p>
<p>New Operator implementations are added to the list <aclass="reference external"href="https://github.com/PaddlePaddle/Paddle/tree/develop/paddle/operators">paddle/operators</a>, with file names in the format <codeclass="docutils literal"><spanclass="pre">*_op.h</span></code> (if applicable), <codeclass="docutils literal"><spanclass="pre">*_op.cc</span></code>, <codeclass="docutils literal"><spanclass="pre">*_op.cu</span></code> (if applicable).** The system will use the naming scheme to automatically build operators and their corresponding Python extensions. **</p>
<p>Let’s take matrix multiplication operator, <aclass="reference external"href="https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/operators/mul_op.cc">MulOp</a>, as an example to introduce the writing of an Operator with Kernel.</p>
</div>
<divclass="section"id="implementing-c-types">
<spanid="implementing-c-types"></span><h2>Implementing C++ Types<aclass="headerlink"href="#implementing-c-types"title="Permalink to this headline">¶</a></h2>
<spanid="defining-class-protomaker"></span><h3>1. Defining Class ProtoMaker<aclass="headerlink"href="#defining-class-protomaker"title="Permalink to this headline">¶</a></h3>
<p>Matrix Multiplication can be written as $Out = X * Y$, meaning that the operation consists of two inputs and pne output.</p>
<p>First, define <codeclass="docutils literal"><spanclass="pre">ProtoMaker</span></code> to describe the Operator’s input, output, and additional comments:</p>
<spanclass="n">AddInput</span><spanclass="p">(</span><spanclass="s">"X"</span><spanclass="p">,</span><spanclass="s">"(Tensor), 2D tensor of size (M x K)"</span><spanclass="p">);</span>
<spanclass="n">AddInput</span><spanclass="p">(</span><spanclass="s">"Y"</span><spanclass="p">,</span><spanclass="s">"(Tensor), 2D tensor of size (K x N)"</span><spanclass="p">);</span>
<spanclass="n">AddOutput</span><spanclass="p">(</span><spanclass="s">"Out"</span><spanclass="p">,</span><spanclass="s">"(Tensor), 2D tensor of size (M x N)"</span><spanclass="p">);</span>
<p><aclass="reference external"href="https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/operators/mul_op.cc#L43"><codeclass="docutils literal"><spanclass="pre">MulOpMaker</span></code></a>is inherited from<codeclass="docutils literal"><spanclass="pre">framework::OpProtoAndCheckerMaker</span></code>, consisting of 2 variables in the constructor：</p>
<ulclass="simple">
<li><codeclass="docutils literal"><spanclass="pre">framework::OpProto</span></code> stores Operator input and variable attribute, used for generating Python API interfaces.</li>
<li><codeclass="docutils literal"><spanclass="pre">framework::OpAttrChecker</span></code> is used to validate variable attributes.</li>
</ul>
<p>The constructor utilizes <codeclass="docutils literal"><spanclass="pre">AddInput</span></code>, <codeclass="docutils literal"><spanclass="pre">AddOutput</span></code>, and <codeclass="docutils literal"><spanclass="pre">AddComment</span></code>, so that the corresponding information will be added to <codeclass="docutils literal"><spanclass="pre">OpProto</span></code>.</p>
<p>The code above adds two inputs <codeclass="docutils literal"><spanclass="pre">X</span></code> and <codeclass="docutils literal"><spanclass="pre">Y</span></code> to <codeclass="docutils literal"><spanclass="pre">MulOp</span></code>, an output <codeclass="docutils literal"><spanclass="pre">Out</span></code>, and their corresponding descriptions, in accordance to Paddle’s <aclass="reference external"href="https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/operators/name_convention.md">naming convention</a>.</p>
<p>An additional example <aclass="reference external"href="https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/operators/scale_op.cc#L37"><codeclass="docutils literal"><spanclass="pre">ScaleOp</span></code></a> is implemented as follows:</p>
<spanclass="n">AddAttr</span><spanclass="o"><</span><spanclass="n">AttrType</span><spanclass="o">></span><spanclass="p">(</span><spanclass="s">"scale"</span><spanclass="p">,</span><spanclass="s">"scale of scale operator."</span><spanclass="p">).</span><spanclass="n">SetDefault</span><spanclass="p">(</span><spanclass="mf">1.0</span><spanclass="p">);</span>
<spanclass="p">}</span>
<spanclass="p">};</span>
</pre></div>
</div>
<p>There are two changes in this example:</p>
<ulclass="simple">
<li><codeclass="docutils literal"><spanclass="pre">AddInput("X","...").NotInGradient()</span></code> expresses that input <codeclass="docutils literal"><spanclass="pre">X</span></code> is not involved in <codeclass="docutils literal"><spanclass="pre">ScaleOp</span></code>‘s corresponding computation. If an input to an operator is not participating in back-propagation, please explicitly set <codeclass="docutils literal"><spanclass="pre">.NotInGradient()</span></code>.</li>
<li><codeclass="docutils literal"><spanclass="pre">AddAttr<AttrType>("scale",</span><spanclass="pre">"...").SetDefault(1.0);</span></code> adds <codeclass="docutils literal"><spanclass="pre">scale</span></code>constant as an attribute, and sets the default value to 1.0.</li>
</ul>
</div>
<divclass="section"id="defining-operator">
<spanid="defining-operator"></span><h3>2. Defining Operator<aclass="headerlink"href="#defining-operator"title="Permalink to this headline">¶</a></h3>
<p>The following code defines the interface for MulOp:</p>
<p><aclass="reference external"href="https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/operators/mul_op.cc#L22"><codeclass="docutils literal"><spanclass="pre">MulOp</span></code></a> is inherited from <codeclass="docutils literal"><spanclass="pre">OperatorWithKernel</span></code>. Its <codeclass="docutils literal"><spanclass="pre">public</span></code> member</p>
<p>expresses an operator constructor using base class <codeclass="docutils literal"><spanclass="pre">OperatorWithKernel</span></code>, alternatively written as</p>
<p><codeclass="docutils literal"><spanclass="pre">InferShape</span></code> interface needs to be re-written.<codeclass="docutils literal"><spanclass="pre">InferShape</span></code> is a constant method and cannot modify Op’s member variables, its constant member <codeclass="docutils literal"><spanclass="pre">const</span><spanclass="pre">framework::InferShapeContext</span><spanclass="pre">&ctx</span></code> can be used to extract input, output, and attributes. It functions to</p>
<ulclass="simple">
<li>1). validate and error out early: it checks input data dimensions and types.</li>
<li>2). configures the tensor shape in the output.</li>
</ul>
<p>Usually <codeclass="docutils literal"><spanclass="pre">OpProtoMaker</span></code> and <codeclass="docutils literal"><spanclass="pre">Op</span></code>‘s type definitions are written in <codeclass="docutils literal"><spanclass="pre">.cc</span></code> files, which also include the registration methods introduced later.</p>
</div>
<divclass="section"id="defining-opkernel">
<spanid="defining-opkernel"></span><h3>3. Defining OpKernel<aclass="headerlink"href="#defining-opkernel"title="Permalink to this headline">¶</a></h3>
<p><codeclass="docutils literal"><spanclass="pre">MulKernel</span></code> inherits <codeclass="docutils literal"><spanclass="pre">framework::OpKernel</span></code>, which includes the following templates:</p>
<ulclass="simple">
<li><codeclass="docutils literal"><spanclass="pre">typename</span><spanclass="pre">Place</span></code> denotes device type. When different devices, namely the CPU and the GPU, share the same kernel, this template needs to be added. If they don’t share kernels, this must not be added. An example of a non-sharing kernel is <aclass="reference external"href="https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/operators/cross_entropy_op.h#L43"><codeclass="docutils literal"><spanclass="pre">OnehotCrossEntropyOpKernel</span></code></a>.</li>
<li><codeclass="docutils literal"><spanclass="pre">typename</span><spanclass="pre">T</span></code> denotes data type, such as <codeclass="docutils literal"><spanclass="pre">float</span></code> or <codeclass="docutils literal"><spanclass="pre">double</span></code>.</li>
</ul>
<p><codeclass="docutils literal"><spanclass="pre">MulKernel</span></code> types need to rewrite the interface for <codeclass="docutils literal"><spanclass="pre">Compute</span></code>.</p>
<ulclass="simple">
<li><codeclass="docutils literal"><spanclass="pre">Compute</span></code> takes one input variable <codeclass="docutils literal"><spanclass="pre">const</span><spanclass="pre">framework::ExecutionContext&</span><spanclass="pre">context</span></code>.</li>
<li>Compared with <codeclass="docutils literal"><spanclass="pre">InferShapeContext</span></code>, <codeclass="docutils literal"><spanclass="pre">ExecutionContext</span></code> includes device types, and can similarly extract input, output, and attribute variables.</li>
<li><codeclass="docutils literal"><spanclass="pre">Compute</span></code> implements the computation logics of an <codeclass="docutils literal"><spanclass="pre">OpKernel</span></code>.</li>
</ul>
<p><codeclass="docutils literal"><spanclass="pre">MulKernel</span></code>‘s implementation of <codeclass="docutils literal"><spanclass="pre">Compute</span></code> is as follows:</p>
<p>Note that <strong>different devices (CPU, GPU)share an Op definition; whether or not they share the same <codeclass="docutils literal"><spanclass="pre">OpKernel</span></code> depends on whether <codeclass="docutils literal"><spanclass="pre">Compute</span></code> calls functions that support both devices.</strong></p>
<p><codeclass="docutils literal"><spanclass="pre">MulOp</span></code>‘s CPU and GPU share the same <codeclass="docutils literal"><spanclass="pre">Kernel</span></code>. A non-sharing <codeclass="docutils literal"><spanclass="pre">OpKernel</span></code> example can be seen in <aclass="reference external"href="https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/operators/cross_entropy_op.h#L43"><codeclass="docutils literal"><spanclass="pre">OnehotCrossEntropyOpKernel</span></code></a>.</p>
<p>To ease the writing of <codeclass="docutils literal"><spanclass="pre">OpKernel</span></code> compute, and for reusing code cross-device, <codeclass="docutils literal"><spanclass="pre">Eigen</span><spanclass="pre">unsupported</span><spanclass="pre">Tensor</span></code> module is used to implement <codeclass="docutils literal"><spanclass="pre">Compute</span></code> interface. To learn about how the Eigen library is used in PaddlePaddle, please see <aclass="reference external"href="https://github.com/PaddlePaddle/Paddle/blob/develop/doc/howto/dev/use_eigen_cn.md">usage document</a>.</p>
<p>This concludes the forward implementation of an operator. Next its operation and kernel need to be registered in a <codeclass="docutils literal"><spanclass="pre">.cc</span></code> file.</p>
<p>The definition of its corresponding backward operator, if applicable, is similar to that of an forward operator. <strong>Note that a backward operator does not include a <codeclass="docutils literal"><spanclass="pre">ProtoMaker</span></code></strong>.</p>
</div>
<divclass="section"id="registering-operator">
<spanid="registering-operator"></span><h3>4. Registering Operator<aclass="headerlink"href="#registering-operator"title="Permalink to this headline">¶</a></h3>
<ul>
<li><pclass="first">In <codeclass="docutils literal"><spanclass="pre">.cc</span></code> files, register forward and backward operator classes and the CPU kernel.</p>
<li><codeclass="docutils literal"><spanclass="pre">REGISTER_OP</span></code> registers the <codeclass="docutils literal"><spanclass="pre">ops::MulOp</span></code> class, type named <codeclass="docutils literal"><spanclass="pre">mul</span></code>, its type <codeclass="docutils literal"><spanclass="pre">ProtoMaker</span></code> is <codeclass="docutils literal"><spanclass="pre">ops::MulOpMaker</span></code>, registering <codeclass="docutils literal"><spanclass="pre">ops::MulOpGrad</span></code> as <codeclass="docutils literal"><spanclass="pre">mul_grad</span></code>.</li>
<li><codeclass="docutils literal"><spanclass="pre">REGISTER_OP_WITHOUT_GRADIENT</span></code> registers an operator without gradient.</li>
<li><codeclass="docutils literal"><spanclass="pre">REGISTER_OP_CPU_KERNEL</span></code> registers <codeclass="docutils literal"><spanclass="pre">ops::MulKernel</span></code> class and specialized template types <codeclass="docutils literal"><spanclass="pre">paddle::platform::CPUPlace</span></code> and <codeclass="docutils literal"><spanclass="pre">float</span></code>, which also registers <codeclass="docutils literal"><spanclass="pre">ops::MulKernel</span></code>.</li>
</ul>
</li>
</ul>
<ul>
<li><pclass="first">Registering GPU Kernel in <codeclass="docutils literal"><spanclass="pre">.cu</span></code> files</p>
<ulclass="simple">
<li>Note that if GPU Kernel is implemented using the <codeclass="docutils literal"><spanclass="pre">Eigen</span><spanclass="pre">unsupported</span></code> module, then on top of <codeclass="docutils literal"><spanclass="pre">.cu</span></code>, a macro definition <codeclass="docutils literal"><spanclass="pre">#define</span><spanclass="pre">EIGEN_USE_GPU</span></code> is needed, such as</li>
</ul>
<divclass="highlight-cpp"><divclass="highlight"><pre><span></span><spanclass="c1">// if use Eigen unsupported module before include head files</span>
<spanid="python-binding"></span><h2>Python Binding<aclass="headerlink"href="#python-binding"title="Permalink to this headline">¶</a></h2>
<p>The system will automatically bind to Python and link it to a generated library.</p>
</div>
<divclass="section"id="unit-tests">
<spanid="unit-tests"></span><h2>Unit Tests<aclass="headerlink"href="#unit-tests"title="Permalink to this headline">¶</a></h2>
<p>Unit tests include comparing a forward operator’s implementations on different devices, comparing a backward operator’s implementation on different devices, and a scaling test for the backward operator. Here, we introduce the <aclass="reference external"href="https://github.com/PaddlePaddle/Paddle/blob/develop/python/paddle/v2/framework/tests/test_mul_op.py">unit tests for <codeclass="docutils literal"><spanclass="pre">MulOp</span></code></a>.</p>
Built with <ahref="http://sphinx-doc.org/">Sphinx</a> using a <ahref="https://github.com/snide/sphinx_rtd_theme">theme</a> provided by <ahref="https://readthedocs.org">Read the Docs</a>.
