Skip to content

F
FluidDoc

PaddlePaddle / FluidDoc

通知 10
Star 2
Fork 0
F
FluidDoc
提交 062b5d48 编写于 作者: M Mr.Lee 提交者： Hao Wang
浏览文件
操作

upload new_op_en.md (#568) 

* upload new_op_en.md

* Review

* Review2
上级 96527e81
隐藏空白更改
内联 并排
Showing with 501 addition and 0 deletion
doc/fluid/dev/new_op_en.md → doc/fluid/advanced_usage/development/new_op/new_op_en.md
浏览文件 @ 062b5d48
# How to write a new operator
- [Background](#background)
- [Implementing C++ Types](#implementing-c-types)
- [Defining ProtoMaker](#defining-protomaker)
- [Defining Operator](#defining-operator)
- [Defining OpKernel](#defining-opkernel)
- [Registering Operator and OpKernel](#registering-operator-and-opkernel)
- [Compilation](#compilation)
- [Python Binding](#python-binding)
- [Unit Tests](#unit-tests)
- [Testing Forward Operators](#testing-forward-operators)
- [Testing Backward Operators](#testing-backward-operators)
- [Compiling and Running](#compiling-and-running)
- [Remarks](#remarks)
<!--## Table of Contents
- [How to write a new operator](#How to write a new operator)
- [Background](#Background)
- [Implementing C++ Types](#Implementing C++ Types)
- [Defining ProtoMaker](#Defining ProtoMaker)
- [Defining the GradProtoMaker class](#Defining the GradProtoMaker class)
- [Defining Operator](#Defining Operator)
- [Defining OpKernel](#Defining OpKernel)
- [Registering Operator and OpKernel](#Registering Operator and OpKernel)
- [Compilation](#Compilation)
- [Python Binding](#Python Binding)
- [Unit Tests](#Unit Tests)
- [Unit Test for Forward Operators](#Unit Test for Forward Operators)
- [Unit test for backward operators](#Unit test for backward operators)
- [Compiling and Running](#Compiling and Running)
- [Remarks](#Remarks)
- [PADDLE_ENFORCE Usage Note](#PADDLE_ENFORCE Usage Note)
- [General Principles](#General Principles)
- [Error Message Standard](#Error Message Standard)
- [Typical Problems](#Typical Problems)
- [OP InferShape check message special instructions](#OP InferShape check message special instructions)
-->
<a name="Background"></a>
## Background
Here are the base types needed. For details, please refer to the design docs.
......@@ -59,13 +70,13 @@ New Operator implementations are added to the list [paddle/operators](https://gi
Let's take matrix multiplication operator, [MulOp](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/fluid/operators/mul_op.cc), as an example to introduce the writing of an Operator with Kernel.
<a name="Implementing C++ Types"></a>
## Implementing C++ Types
<a name="Defining ProtoMaker"></a>
### Defining ProtoMaker
Matrix Multiplication can be written as $Out = X * Y$, meaning that the operation consists of two inputs and pne output.
Matrix Multiplication can be written as $Out = X * Y$, meaning that the operation consists of two inputs and one output.
First, define `ProtoMaker` to describe the Operator's input, output, and additional comments:
......@@ -90,32 +101,58 @@ The equation is: Out = X * Y
- `framework::OpProto` stores Operator input and variable attribute, used for generating Python API interfaces.
- `framework::OpAttrChecker` is used to validate variable attributes.
The constructor utilizes `AddInput`, `AddOutput`, and `AddComment`, so that the corresponding information will be added to `OpProto`.
The constructor utilizes `AddInput` to add input parameter, `AddOutput` to add output parameter, and `AddComment` to add comments for the Op, so that the corresponding information will be added to `OpProto`.
The code above adds two inputs `X` and `Y` to `MulOp`, an output `Out`, and their corresponding descriptions, in accordance to Paddle's [naming convention](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/fluid/dev/name_convention.md).
The code above adds two inputs `X` and `Y` to `MulOp`, an output `Out`, and their corresponding descriptions. Names are given in accordance to Paddle's [naming convention](https://github.com/PaddlePaddle/FluidDoc/blob/develop/doc/fluid/dev/name_convention.md).
An additional example [`ScaleOp`](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/fluid/operators/scale_op.cc#L38-L55) is implemented as follows:
```cpp
template <typename AttrType>
class ScaleOpMaker : public framework::OpProtoAndCheckerMaker {
public:
ScaleOpMaker(OpProto *proto, OpAttrChecker *op_checker)
: OpProtoAndCheckerMaker(proto, op_checker) {
AddInput("X", "The input tensor of scale operator.").NotInGradient();
AddOutput("Out", "The output tensor of scale operator.").NotInGradient();
AddComment(R"DOC(Scale operator
The equation is: Out = scale*X
AddInput("X", "(Tensor) Input tensor of scale operator.");
AddOutput("Out", "(Tensor) Output tensor of scale operator.");
AddComment(R"DOC(
Scale operator
$$Out = scale*X$$
)DOC");
AddAttr<AttrType>("scale", "scale of scale operator.").SetDefault(1.0);
AddAttr<AttrType>("scale",
"(float, default 1.0)"
"The scaling factor of the scale operator.")
.SetDefault(1.0);
}
};
```
Note `AddAttr<AttrType>("scale", "...").SetDefault(1.0);` adds `scale`constant as an attribute, and sets the default value to 1.0.
<a name="Defining the GradProtoMaker class"></a>
### Defining the GradProtoMaker class
Each Op must have a corresponding GraProtoMaker. If GradProtoMaker corresponding to the forward Op is not customized, Fluid provides DefaultGradProtoMaker. The default registration will use all input and output, including Input, Output, Output@Grad and so on. Using unnecessary variables will cause waste of memory.
The following example defines ScaleOp's GradProtoMaker.
```cpp
class ScaleGradMaker : public framework::SingleGradOpDescMaker {
public:
using framework::SingleGradOpDescMaker::SingleGradOpDescMaker;
std::unique_ptr<framework::OpDesc> Apply() const override {
auto *grad_op = new framework::OpDesc();
grad_op->SetType("scale");
grad_op->SetInput("X", OutputGrad("Out"));
grad_op->SetOutput("Out", InputGrad("X"));
grad_op->SetAttr("scale", GetAttr("scale"));
return std::unique_ptr<framework::OpDesc>(grad_op);
}
};
```
<a name="Defining Operator"></a>
### Defining Operator
The following code defines the interface for MulOp:
......@@ -127,8 +164,9 @@ class MulOp : public framework::OperatorWithKernel {
protected:
void InferShape(const framework::InferShapeContext &ctx) const override {
auto dim0 = ctx.Input<Tensor>("X")->dims();
auto dim1 = ctx.Input<Tensor>("Y")->dims();
//never use Input<Tensor> or Output<Tensor> if you want a to get a LoDTensor.
auto dim0 = ctx.Input<LoDTensor>("X")->dims();
auto dim1 = ctx.Input<LoDTensor>("Y")->dims();
PADDLE_ENFORCE_EQ(dim0.size(), 2,
"input X(%s) should be a tensor with 2 dims, a matrix",
ctx.op_.Input("X"));
......@@ -138,7 +176,7 @@ class MulOp : public framework::OperatorWithKernel {
PADDLE_ENFORCE_EQ(
dim0[1], dim1[0],
"First matrix's width must be equal with second matrix's height.");
ctx.Output<Tensor>("Out")->Resize({dim0[0], dim1[1]});
ctx.Output<LoDTensor>("Out")->Resize({dim0[0], dim1[1]});
}
};
```
......@@ -158,18 +196,19 @@ MulOp(const std::string &type, const framework::VariableNameMap &inputs,
: OperatorWithKernel(type, inputs, outputs, attrs) {}
```
`InferShape` interface needs to be re-written.`InferShape` is a constant method and cannot modify Op's member variables, its constant member `const framework::InferShapeContext &ctx` can be used to extract input, output, and attributes. It functions to
`InferShape` interface needs to be re-written.`InferShape` is a const method and cannot modify Op's member variables. Its constant member `const framework::InferShapeContext &ctx` can be used to extract input, output, and attributes. Its functions are
- 1). validate and error out early: it checks input data dimensions and types.
- 2). configures the tensor shape in the output.
Usually `OpProtoMaker` and `Op`'s type definitions are written in `.cc` files, which also include the registration methods introduced later.
Usually `OpProtoMaker` and `Op` definitions are written in `.cc` files, which also include the registration methods introduced later.
<a name="Defining OpKernel"></a>
### Defining OpKernel
`MulKernel` inherits `framework::OpKernel`, which includes the following templates:
`MulKernel` is derived from `framework::OpKernel`, which includes the following templates:
- `typename DeviceContext` denotes device context type. When different devices, namely the CPUDeviceContext and the CUDADeviceContext, share the same kernel, this template needs to be added. If they don't share kernels, this must not be added. An example of a non-sharing kernel is [`OnehotCrossEntropyOpKernel`](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/fluid/operators/cross_entropy_op.h#L43).
- `typename DeviceContext` denotes device context type. When different devices, namely the CPU and the CUDA, share the same kernel, this template needs to be added. If they don't share kernels, this must not be added. An example of a non-sharing kernel is [`OnehotCrossEntropyOpKernel`](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/fluid/operators/cross_entropy_op.h#L43).
- `typename T` denotes data type, such as `float` or `double`.
......@@ -177,36 +216,43 @@ Usually `OpProtoMaker` and `Op`'s type definitions are written in `.cc` files, w
- `Compute` takes one input parameter: `const framework::ExecutionContext& context`.
- Compared with `InferShapeContext`, `ExecutionContext` includes device types, and can similarly extract input, output, and attribute variables.
- `Compute` implements the computation logics of an `OpKernel`.
- `Compute` function implements the computation logics of an `OpKernel`.
The input and output of Op can be obtained by `ExecutionContext::Input<T>()` and `ExecutionContext::Output<T>()` respectively.
**Note:** If the input/output variable type of op is `LoDTensor` (In Fluid, all Tensors are LoDTensor type by default), please write `ExecutionContext::Input<LoDTensor>()` and `ExecutionContext:: Output<LoDTensor>()`, do not write `ExecutionContext::Input<Tensor>()` and `ExecutionContext::Output<Tensor>()`. Because if the actual variable type is `SelectedRows`, the `Input<Tensor>()` and `Output<Tensor>()` methods will specialize the `SelectedRows` type to `Tensor`, causing a potential error.
`MulKernel`'s implementation of `Compute` is as follows:
```cpp
template <typename DeviceContext, typename T>
class MulKernel : public framework::OpKernel {
public:
void Compute(const framework::ExecutionContext& context) const override {
auto* X = context.Input<Tensor>("X");
auto* Y = context.Input<Tensor>("Y");
auto* Z = context.Output<Tensor>("Out");
Z->mutable_data<T>(context.GetPlace());
auto& device_context = context.template device_context<DeviceContext>();
math::matmul<DeviceContext, T>(*X, false, *Y, false, 1, Z, 0, device_context);
}
};
```
```cpp
template <typename DeviceContext, typename T>
class MulKernel : public framework::OpKernel {
public:
void Compute(const framework::ExecutionContext& context) const override {
auto* X = context.Input<LoDTensor>("X");
auto* Y = context.Input<LoDTensor>("Y");
auto* Z = context.Output<LoDTensor>("Out");
Z->mutable_data<T>(context.GetPlace());
auto& device_context = context.template device_context<DeviceContext>();
math::matmul<DeviceContext, T>(*X, false, *Y, false, 1, Z, 0, device_context);
}
};
```
Note that **different devices (CPU, CUDA)share one Op definition; whether or not they share the same `OpKernel` depends on whether `Compute` calls functions can support both devices.**
Note that **different devices (CPU, CUDA)share one Op definition; whether or not they share the same `OpKernel` depends on whether functions called by `Compute`can support both devices.**
`MulOp`'s CPU and CUDA share the same `Kernel`. A non-sharing `OpKernel` example can be seen in [`OnehotCrossEntropyOpKernel`](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/fluid/operators/cross_entropy_op.cc).
To ease the writing of `OpKernel` compute, and for reusing code cross-device, [`Eigen-unsupported Tensor`](https://bitbucket.org/eigen/eigen/src/default/unsupported/Eigen/CXX11/src/Tensor/README.md?fileviewer=file-view-default) module is used to implement `Compute` interface. To learn about how the Eigen library is used in PaddlePaddle, please see [usage document](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/fluid/dev/use_eigen_en.md).
To ease the writing of `OpKernel` compute, and for reusing code cross-device, [`Eigen-unsupported Tensor`](https://bitbucket.org/eigen/eigen/src/default/unsupported/Eigen/CXX11/src/Tensor/README.md?fileviewer=file-view-default) module is used to implement `Compute` interface. To learn about how the Eigen library is used in PaddlePaddle, please see [usage document](https://github.com/PaddlePaddle/FluidDoc/blob/develop/doc/fluid/dev/use_eigen_en.md).
This concludes the forward implementation of an operator. Next its operation and kernel need to be registered in a `.cc` file.
The definition of its corresponding backward operator, if applicable, is similar to that of an forward operator. **Note that a backward operator does not include a `ProtoMaker`**.
<a name="Registering Operator and OpKernel"></a>
### Registering Operator and OpKernel
- In `.cc` files, register forward and backward operator classes and the CPU kernel.
......@@ -216,17 +262,15 @@ The definition of its corresponding backward operator, if applicable, is similar
REGISTER_OPERATOR(mul, ops::MulOp, ops::MulOpMaker,
paddle::framework::DefaultGradOpDescMaker<true>)
REGISTER_OPERATOR(mul_grad, ops::MulGradOp)
REGISTER_OP_CPU_KERNEL(mul, ops::MulKernel<paddle::platform::CPUDeviceContext, float>);
REGISTER_OP_CPU_KERNEL(mul_grad,
ops::MulGradKernel<paddle::platform::CPUDeviceContext, float>);
```
In that code block,
In that code block,
- `REGISTER_OPERATOR` registers the `ops::MulOp` class, type named `mul`, its type `ProtoMaker` is `ops::MulOpMaker`, registering `ops::MulOpGrad` as `mul_grad`.
- `REGISTER_OP_WITHOUT_GRADIENT` registers an operator without gradient.
- `REGISTER_OP_CPU_KERNEL` registers `ops::MulKernel` class and specialized template types `paddle::platform::CPUPlace` and `float`, which also registers `ops::MulGradKernel`.
- `REGISTER_OPERATOR` registers the `ops::MulOp` class, with the type named `mul`. Its `ProtoMaker` is `ops::MulOpMaker`. Register `ops::MulOpGrad` as type named `mul_grad`.
- `REGISTER_OP_CPU_KERNEL` registers `ops::MulKernel` class and specializes template parameters as type `paddle::platform::CPUPlace` and `float`, and also registers `ops::MulGradKernel`.
- Registering CUDA Kernel in `.cu` files
......@@ -240,44 +284,46 @@ The definition of its corresponding backward operator, if applicable, is similar
REGISTER_OP_CUDA_KERNEL(mul, ops::MulKernel<paddle::platform::CUDADeviceContext, float>);
REGISTER_OP_CUDA_KERNEL(mul_grad,
ops::MulGradKernel<paddle::platform::CUDADeviceContext, float>);
```
<a name="Compilation"></a>
### Compilation
Run the following commands to compile.
```
# maybe you need to rerun cmake
make mul_op
```
<a name="Python Binding"></a>
## Python Binding
The system will automatically bind to Python and link it to a generated library.
The system will automatically bind the new op to Python and link it to a generated library.
<a name="Unit Tests"></a>
## Unit Tests
Unit tests for an operator include
1. comparing a forward operator's implementations on different devices,
1. comparing a forward operator's implementations on different devices (CPU, CUDA)
2. comparing a backward operator's implementation on different devices, and
2. comparing a backward operator's implementation on different devices (CPU, CUDA)
3. a scaling test for the backward operator.
3. a gradient test for the backward operator.
Here, we introduce the [unit tests for `MulOp`](https://github.com/PaddlePaddle/Paddle/blob/develop/python/paddle/fluid/tests/unittests/test_mul_op.py).
### Testing Forward Operators
A forward operator unit test inherits `unittest.TestCase` and defines metaclass `__metaclass__ = OpTestMeta`. More concrete tests are performed in `OpTestMeta`. Testing a forward operator requires the following:
1. Defining input, output and relevant attributes in `setUp` method.
<a name="Unit Test for Forward Operators"></a>
### Unit Test for Forward Operators
2. Generating random input data.
The Op unit test is inherited from `OpTest`. More specific unit tests are done in `TestMulOp`. To test the Operator, you need to:
3. Implementing the same computation logic in a Python script.
4. Call check gradient function to check the backward operator.
1. Define input, output, and related property parameters in the `setUp` function.
2. Generate random input data.
3. Implement the same calculation logic as the forward operator in the Python script to get the output, which is to be compared with the output of the forward operator calculation.
4. The backward calculation has been automatically integrated into the test framework and the corresponding interface can be called directly.
```python
import unittest
......@@ -308,7 +354,7 @@ A forward operator unit test inherits `unittest.TestCase` and defines metaclass
self.check_grad(
['X'], 'Out', max_relative_error=0.5, no_grad_set=set('Y'))
```
Get its output, and compare it with the forward operator's own output.
The code above first loads required packages. In addition, we have
......@@ -316,20 +362,23 @@ The code above first loads required packages. In addition, we have
- `self.inputs` defines input, with type `numpy.array` and initializes it.
- `self.outputs` defines output and completes the same operator computation in the Python script, and returns its result from the Python script.
### Testing Backward Operators
<a name="Unit test for backward operators"></a>
### Unit Test for Backward Operators
Some key points in checking gradient above include:
In the backward operator test:
- `test_normal` calls `check_grad` to validate scaling tests' correctness and stability through numeric methods.
- The first variable `["X", "Y"]` appoints `X` and `Y` to be scale tested.
- The second variable `"Out"` points to the network's final output target `Out`.
- The third variable `max_relative_error` points to the maximum relative tolerance error during scaling tests.
- `test_check_grad_ingore_x` and `test_check_grad_ingore_y`branches test the cases where there is only one scaling input.
- `check_grad` is called in `test_check_grad_normal` to use numerical methods to detect gradient correctness and stability.
- The first parameter `["X", "Y"]` : specifies gradient check for the input variables `X`, `Y`.
- The second parameter `"Out"` : specifies the final output target variable `Out` of the forward network.
- The third parameter `max_relative_error`: specifies the maximum error value that can be tolerated when checking gradients.
- The `test_check_grad_ingore_x` and `test_check_grad_ingore_y` branches are used to test cases where only one input gradient needs to be calculated.
<a name="Compiling and Running"></a>
### Compiling and Running
Any new unit testing file of the format `test_*.py` added to the director `python/paddle/fluid/tests/unittests/` is automatically added to the project to compile.
Any new unit testing file of the format `test_*.py` added to the directory `python/paddle/fluid/tests/unittests/` is automatically added to the project to compile.
Note that **unlike the compile test for Ops, running unit tests requires compiling the entire project** and requires compiling with flag `WITH_TESTING` on i.e. `cmake paddle_dir -DWITH_TESTING=ON`.
......@@ -345,8 +394,108 @@ Or,
ctest -R test_mul_op
```
<a name="Remarks"></a>
## Remarks
- The type with which an operator is registered needs to be identical to the Op's name. Registering `REGISTER_OPERATOR(B, ...)` in `A_op.cc` will cause unit testing failures.
- If the operator does not implement a CUDA kernel, please refrain from creating an empty `*_op.cu` file, or else unit tests will fail.
- If multiple operators rely on some shared methods, a file NOT named `*_op.*` can be created to store them, such as `gather.h`.
<a name="PADDLE_ENFORCE Usage Note"></a>
### PADDLE_ENFORCE Usage Note
To check the validity of data when implementing Op, you need to use macro definitions such as PADDLE_ENFORCE and PADDLE_ENFORCE_EQ. The basic format is as follows:
```
PADDLE_ENFORCE (expression, error message)
PADDLE_ENFORCE_EQ (comparison object A, comparison object B, error message)
```
If the expression is true, or the comparison object A=B, the check will be passed, otherwise the program will be terminated and the corresponding error message will be fed back to the user.
In order to ensure that the feedbacks are user-friendly and easy to understand, developers need to pay attention to how to use them.
<a name="General Principles"></a>
#### General Principles
Any place where PADDLE_ENFORCE and PADDLE_ENFORCE_EQ are used must have a properly detailed explanation of the comments! **Error message** can't be empty!
<a name="Error Message Standard"></a>
#### Error Message Standard
1. [required] Where does it go wrong? Why is it wrong?
- For example: `ValueError: Mismatched label shape`
2. [optional] What is the expected input? What is the actual input?
- For example: `Expected labels dimension=1. Received 4.`
3. [optional] Can you come up with a suggestion?
- For example: `Suggested Fix: If your classifier expects one-hot encoding label, check your n_classes argument to the estimatorand/or the shape of your label.Otherwise, check the shape of your label.`
If it is not necessary or concise description is enough to clearly express the above points, just write based on actual needs.
<a name="Typical Problems"></a>
#### Typical Problems
1.No error message exists or error message is too short to provide effective notification to the user.
Problem example 1: Absent message
```
PADDLE_ENFORCE(ctx->HasInput("X"), "");
```
Problem example 2: The prompt message is too short
```
PADDLE_ENFORCE(i != nullptr, "i must be set"); // What is i?
```
2.Using developer-defined variable abbreviations in error messages is not easy to understand.
Example of the problem:
```
PADDLE_ENFORCE(forward_pd != nullptr,
"Fail to find eltwise_fwd_pd in device context"); //eltwise_fwd_pduser may not be understood
```
3.The OP internally calls the illegal interface: If Op appears inside Output = ShareDataWith(Input)
Example of the problem:
```cpp
auto *out = ctx.Output<framework::LoDTensor>("Out");
auto *in = ctx.Input<framework::LoDTensor>("X");
out->ShareDataWith(*in);
```
If there is Output = ShareDataWith(Input) inside Op, it will equivalently indicate a hidden edge in the operator graph, which connects Input and Output. This edge cannot be expressed in graph analysis, causing error based on graph optimization.
4.Performance of OP implementation. It called eigen's broadcast, chop and other operations, the performance will be over several times worse than the handwritten cuda kernel. At this point, the implementation of cpu can reuse eigen, and the gpu implementation can implement cuda kernel.
<a name="Special instructions for OP InferShape check message"></a>
#### Special Instructions for OP InferShape Check Message
- Check input and output variables, please follow the following format
`Input(variable name) of OP name operator should not be null.`
The correct example:
```
PADDLE_ENFORCE(ctx->HasInput("Input"),
"Input(Input) of LSTMP operator should not be null.");
```
- Backward Op input and output check, to write the name of the backward Op
The correct example:
```
PADDLE_ENFORCE(ctx->HasInput("X"),
"Input(X) of LoDResetGrad opreator should not be null.");
```
Markdown is supported
0% 或 .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑！
想要评论请 注册 或