In a DL system, we can compose one or more fine grained operators into a coarse grained one. For example, the FC layer can be composed of a multiplication operator and an add operator.
Historically, some fine grained operations are known as operators, and some coarse level ones are known as layers. But we need a well-defined separation.
In general, operators are those very fine grained operations, e.g., mul and add. In the implementation, we can write them as C++ functions:
```c++
template <typename T> T add(T x, T y) { return x + y; }
template <typename T> T mul(T x, T y) { return x * y; }
```
Then we can wrap them into operators which are C++ classes and can be created from Python bindings by name. A C macro can do this. For example, the following macro invocation
```c++
#define MAKE_FUNCTION_OPERATOR(mul);
```
generates
```c++
template <typename T> class mulOp : public OperatorBase {...};
We need to support such composition in Python as well. To do so, we need a higher level Python wrapping of operator creation than `paddle.cpp.create_operator`. This higher level operator API should be compatible with the layer API.
Let's explain using an example. Suppose that we are going to compose the FC using mul and add in Python, we'd like to have Python functions `mul` and `add` defined in module `operator`:
We'd like to have Python bindings to operators in package `paddle.operator`, and Python compositions of operators in package `paddle.layer`. So we have the following concepts in above illustrative example:
<liclass="toctree-l2"><aclass="reference internal"href="../getstarted/build_and_install/index_en.html">Install and Build</a><ul>
<liclass="toctree-l3"><aclass="reference internal"href="../getstarted/build_and_install/docker_install_en.html">PaddlePaddle in Docker Containers</a></li>
<liclass="toctree-l3"><aclass="reference internal"href="../getstarted/build_and_install/build_from_source_en.html">Installing from Sources</a></li>
<liclass="toctree-l2"><aclass="reference internal"href="../howto/usage/k8s/k8s_en.html">Paddle On Kubernetes</a></li>
<liclass="toctree-l2"><aclass="reference internal"href="../howto/usage/k8s/k8s_aws_en.html">Distributed PaddlePaddle Training on AWS with Kubernetes</a></li>
<liclass="toctree-l2"><aclass="reference internal"href="../howto/dev/build_en.html">Build PaddlePaddle from Source Code and Run Unit Test</a></li>
<liclass="toctree-l2"><aclass="reference internal"href="../howto/dev/new_layer_en.html">Write New Layers</a></li>
<spanid="design-doc-functions-operators-and-layers"></span><h1>Design Doc: Functions, Operators, and Layers<aclass="headerlink"href="#design-doc-functions-operators-and-layers"title="Permalink to this headline">¶</a></h1>
<p>In a DL system, we can compose one or more fine grained operators into a coarse grained one. For example, the FC layer can be composed of a multiplication operator and an add operator.</p>
<p>Historically, some fine grained operations are known as operators, and some coarse level ones are known as layers. But we need a well-defined separation.</p>
<p>In general, operators are those very fine grained operations, e.g., mul and add. In the implementation, we can write them as C++ functions:</p>
<p>Then we can wrap them into operators which are C++ classes and can be created from Python bindings by name. A C macro can do this. For example, the following macro invocation</p>
<p>Also, at the same time, we can compose a coarse level C++ operator class by composing functions <codeclass="docutils literal"><spanclass="pre">mul</span></code> and <codeclass="docutils literal"><spanclass="pre">add</span></code>:</p>
<p>We need to support such composition in Python as well. To do so, we need a higher level Python wrapping of operator creation than <codeclass="docutils literal"><spanclass="pre">paddle.cpp.create_operator</span></code>. This higher level operator API should be compatible with the layer API.</p>
<p>Let’s explain using an example. Suppose that we are going to compose the FC using mul and add in Python, we’d like to have Python functions <codeclass="docutils literal"><spanclass="pre">mul</span></code> and <codeclass="docutils literal"><spanclass="pre">add</span></code> defined in module <codeclass="docutils literal"><spanclass="pre">operator</span></code>:</p>
<p>If we don’t have <codeclass="docutils literal"><spanclass="pre">operator.mul</span></code> and <codeclass="docutils literal"><spanclass="pre">operator.add</span></code>, the definiton of <codeclass="docutils literal"><spanclass="pre">layer.fc</span></code> would be complicated:</p>
<p>We’d like to have Python bindings to operators in package <codeclass="docutils literal"><spanclass="pre">paddle.operator</span></code>, and Python compositions of operators in package <codeclass="docutils literal"><spanclass="pre">paddle.layer</span></code>. So we have the following concepts in above illustrative example:</p>
<divclass="highlight-default"><divclass="highlight"><pre><span></span>| C++ functions/functors | mul | add | | |
<p>This is how we differentiate layer and operators in PaddlePaddle:</p>
<ulclass="simple">
<li>those defined in C++ and have a lightweighted Python wrapper in module <codeclass="docutils literal"><spanclass="pre">operators</span></code> are operators; whereas</li>
<li>those who don’t have C++ implementations but a Python implementation that compose C++ operators are known as layers.</li>
Built with <ahref="http://sphinx-doc.org/">Sphinx</a> using a <ahref="https://github.com/snide/sphinx_rtd_theme">theme</a> provided by <ahref="https://readthedocs.org">Read the Docs</a>.
In a DL system, we can compose one or more fine grained operators into a coarse grained one. For example, the FC layer can be composed of a multiplication operator and an add operator.
Historically, some fine grained operations are known as operators, and some coarse level ones are known as layers. But we need a well-defined separation.
In general, operators are those very fine grained operations, e.g., mul and add. In the implementation, we can write them as C++ functions:
```c++
template <typename T> T add(T x, T y) { return x + y; }
template <typename T> T mul(T x, T y) { return x * y; }
```
Then we can wrap them into operators which are C++ classes and can be created from Python bindings by name. A C macro can do this. For example, the following macro invocation
```c++
#define MAKE_FUNCTION_OPERATOR(mul);
```
generates
```c++
template <typename T> class mulOp : public OperatorBase {...};
We need to support such composition in Python as well. To do so, we need a higher level Python wrapping of operator creation than `paddle.cpp.create_operator`. This higher level operator API should be compatible with the layer API.
Let's explain using an example. Suppose that we are going to compose the FC using mul and add in Python, we'd like to have Python functions `mul` and `add` defined in module `operator`:
We'd like to have Python bindings to operators in package `paddle.operator`, and Python compositions of operators in package `paddle.layer`. So we have the following concepts in above illustrative example:
<spanid="design-doc-functions-operators-and-layers"></span><h1>Design Doc: Functions, Operators, and Layers<aclass="headerlink"href="#design-doc-functions-operators-and-layers"title="永久链接至标题">¶</a></h1>
<p>In a DL system, we can compose one or more fine grained operators into a coarse grained one. For example, the FC layer can be composed of a multiplication operator and an add operator.</p>
<p>Historically, some fine grained operations are known as operators, and some coarse level ones are known as layers. But we need a well-defined separation.</p>
<p>In general, operators are those very fine grained operations, e.g., mul and add. In the implementation, we can write them as C++ functions:</p>
<p>Then we can wrap them into operators which are C++ classes and can be created from Python bindings by name. A C macro can do this. For example, the following macro invocation</p>
<p>Also, at the same time, we can compose a coarse level C++ operator class by composing functions <codeclass="docutils literal"><spanclass="pre">mul</span></code> and <codeclass="docutils literal"><spanclass="pre">add</span></code>:</p>
<p>We need to support such composition in Python as well. To do so, we need a higher level Python wrapping of operator creation than <codeclass="docutils literal"><spanclass="pre">paddle.cpp.create_operator</span></code>. This higher level operator API should be compatible with the layer API.</p>
<p>Let’s explain using an example. Suppose that we are going to compose the FC using mul and add in Python, we’d like to have Python functions <codeclass="docutils literal"><spanclass="pre">mul</span></code> and <codeclass="docutils literal"><spanclass="pre">add</span></code> defined in module <codeclass="docutils literal"><spanclass="pre">operator</span></code>:</p>
<p>If we don’t have <codeclass="docutils literal"><spanclass="pre">operator.mul</span></code> and <codeclass="docutils literal"><spanclass="pre">operator.add</span></code>, the definiton of <codeclass="docutils literal"><spanclass="pre">layer.fc</span></code> would be complicated:</p>
<p>We’d like to have Python bindings to operators in package <codeclass="docutils literal"><spanclass="pre">paddle.operator</span></code>, and Python compositions of operators in package <codeclass="docutils literal"><spanclass="pre">paddle.layer</span></code>. So we have the following concepts in above illustrative example:</p>
<divclass="highlight-default"><divclass="highlight"><pre><span></span>| C++ functions/functors | mul | add | | |
<p>This is how we differentiate layer and operators in PaddlePaddle:</p>
<ulclass="simple">
<li>those defined in C++ and have a lightweighted Python wrapper in module <codeclass="docutils literal"><spanclass="pre">operators</span></code> are operators; whereas</li>
<li>those who don’t have C++ implementations but a Python implementation that compose C++ operators are known as layers.</li>
Built with <ahref="http://sphinx-doc.org/">Sphinx</a> using a <ahref="https://github.com/snide/sphinx_rtd_theme">theme</a> provided by <ahref="https://readthedocs.org">Read the Docs</a>.
