Due to the refactorization of the PaddlePaddle core, we need Python classes to construct corresponding protobuf messages that describe a DL program.
| Python classes | Protobuf messages |
| --- | --- |
| Program | ProgramDesc |
| Block | BlockDesc |
| Operator | OpDesc |
| Variable | VarDesc |
Please be aware that these Python classes need to maintain some construction-time information, which are not part of the protobuf messages.
## Core Concepts
### Program
A `ProgramDesc` describes a [DL program](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/program.md), which is composed of an array of `BlockDesc`s. A `BlockDesc` refers to its parent block by its index in the array. For example, operators in the step block of an RNN operator needs to be able to access variables in its ancessor blocks.
Whenever we create a block, we need set its parent block to the current block, so the Python class `Program` needs to maintain a data member `current_block`.
```python
class Program(objects):
def __init__(self):
self.proto = core.NewProgram() # a C++ ProgramDesc pointer.
self.blocks = vector<Block>()
self.blocks.append(Block(self, -1)) # the global block
self.current_block = 0 # initialized to the global block
`Program` is an accessor to the protobuf message `ProgramDesc`, which is created in C++ space, because the InferShape function is in C++, which manipulates `VarDesc` messages, which are in turn members of `BlockDesc`, which is a member of `ProgramDesc`.
`Program` creates the first block as the global block in its constructor. All parameters and their initializer operators are in the global block.
### Block
A [Block](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/block.md) includes
1. a map from variable names to an instance of the Python `Variable` class, and
1. a list of `Operator` instances.
```python
class Block(objects):
def __init__(self, program, parent_idx):
self.proto = core.NewBlock(program.proto)
self.program = program
self.vars = map<string, Variable>()
self.ops = vector<Operator>()
self.parent_idx = parent_idx
def create_var(self, ...):
return Variable(self, ...)
def _create_global_var(self, ...):
program.global_block().create_var(...)
def create_parameter(self, name, ...):
# Parameter is a subclass of variable. See Parameter section for details.
`create_parameter` is necessary because parameters are global variables, those defined in the global block, but can be created in some sub-blocks, e.g., an FC layer in the step block of an RNN operator.
`prepand_operator` is necessary because the constructor of `Parameter` needs to create the initialize (or load) operator of the parameter, and would like to put it in the *preamble* of the global block.
### Operator
The `Operator` class fills in the `OpDesc` message and calls the C++ function `InferShape` to infer output shape from input shape.
Please be aware of `self.writer`, that tracks operator who creates the variable. It possible that there are more than one operators who write a variable, but in Python space, each writes to a variable is represented by a Variable class. This is guaranteed by the fact that **`core.NewVarDesc` must NOT create a new `VarDesc` message if its name already exists in the specified block**.
### Parameter
A parameter is a global variable with an initializer (or load) operator.
In above example, `init_attr.type` names an initialize operator. It can also name the load operator
```python
init_attr={
type: "load",
filename: "something.numpy",
}
```
`optimize_op_attrs` is not in the `VarDesc` message, but kept in the Python instance, as it will be used in the Python space when creating the optimize operator's `OpDesc`, and will be in the `OpDesc` message.
## Layer Functions
A layer is a Python function that creates some operators and variables. Layers simplify the work of application programmers.
### Data Layer
```python
def data_layer(name, type, column_name):
block = the_current_program.glolal_block()
var = block.create_global_var(
name=name,
shape=[None] + type.dims(),
dtype=type.dtype)
block.prepend_operator(block,
type="Feed",
inputs = None,
outputs = [var],
{column_name: column_name})
return var
```
The input to the feed operator is a special variable in the global scope, which is the output of [Python readers](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/reader/README.md).
### FC Layer
```python
def fc_layer(input, size, ...):
block = program.current_block()
w = block.create_parameter(...)
b = block.create_parameter(...)
out = block.create_var()
op = block.append_operator("FC", X=input, W=w, b=b, out=out)
<liclass="toctree-l2"><aclass="reference internal"href="../getstarted/build_and_install/index_en.html">Install and Build</a><ul>
<liclass="toctree-l3"><aclass="reference internal"href="../getstarted/build_and_install/docker_install_en.html">PaddlePaddle in Docker Containers</a></li>
<liclass="toctree-l3"><aclass="reference internal"href="../getstarted/build_and_install/build_from_source_en.html">Installing from Sources</a></li>
<liclass="toctree-l2"><aclass="reference internal"href="../howto/usage/k8s/k8s_en.html">Paddle On Kubernetes</a></li>
<liclass="toctree-l2"><aclass="reference internal"href="../howto/usage/k8s/k8s_aws_en.html">Distributed PaddlePaddle Training on AWS with Kubernetes</a></li>
<liclass="toctree-l2"><aclass="reference internal"href="../howto/dev/build_en.html">Build PaddlePaddle from Source Code and Run Unit Test</a></li>
<liclass="toctree-l2"><aclass="reference internal"href="../howto/dev/new_layer_en.html">Write New Layers</a></li>
<spanid="design-doc-python-api"></span><h1>Design Doc: Python API<aclass="headerlink"href="#design-doc-python-api"title="Permalink to this headline">¶</a></h1>
<p>Due to the refactorization of the PaddlePaddle core, we need Python classes to construct corresponding protobuf messages that describe a DL program.</p>
<p>| Python classes | Protobuf messages |
| — | — |
| Program | ProgramDesc |
| Block | BlockDesc |
| Operator | OpDesc |
| Variable | VarDesc |</p>
<p>Please be aware that these Python classes need to maintain some construction-time information, which are not part of the protobuf messages.</p>
<divclass="section"id="core-concepts">
<spanid="core-concepts"></span><h2>Core Concepts<aclass="headerlink"href="#core-concepts"title="Permalink to this headline">¶</a></h2>
<divclass="section"id="program">
<spanid="program"></span><h3>Program<aclass="headerlink"href="#program"title="Permalink to this headline">¶</a></h3>
<p>A <codeclass="docutils literal"><spanclass="pre">ProgramDesc</span></code> describes a <aclass="reference external"href="https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/program.md">DL program</a>, which is composed of an array of <codeclass="docutils literal"><spanclass="pre">BlockDesc</span></code>s. A <codeclass="docutils literal"><spanclass="pre">BlockDesc</span></code> refers to its parent block by its index in the array. For example, operators in the step block of an RNN operator needs to be able to access variables in its ancessor blocks.</p>
<p>Whenever we create a block, we need set its parent block to the current block, so the Python class <codeclass="docutils literal"><spanclass="pre">Program</span></code> needs to maintain a data member <codeclass="docutils literal"><spanclass="pre">current_block</span></code>.</p>
<spanclass="bp">self</span><spanclass="o">.</span><spanclass="n">proto</span><spanclass="o">=</span><spanclass="n">core</span><spanclass="o">.</span><spanclass="n">NewProgram</span><spanclass="p">()</span><spanclass="c1"># a C++ ProgramDesc pointer.</span>
<spanclass="bp">self</span><spanclass="o">.</span><spanclass="n">blocks</span><spanclass="o">.</span><spanclass="n">append</span><spanclass="p">(</span><spanclass="n">Block</span><spanclass="p">(</span><spanclass="bp">self</span><spanclass="p">,</span><spanclass="o">-</span><spanclass="mi">1</span><spanclass="p">))</span><spanclass="c1"># the global block</span>
<spanclass="bp">self</span><spanclass="o">.</span><spanclass="n">current_block</span><spanclass="o">=</span><spanclass="mi">0</span><spanclass="c1"># initialized to the global block</span>
<p><codeclass="docutils literal"><spanclass="pre">Program</span></code> is an accessor to the protobuf message <codeclass="docutils literal"><spanclass="pre">ProgramDesc</span></code>, which is created in C++ space, because the InferShape function is in C++, which manipulates <codeclass="docutils literal"><spanclass="pre">VarDesc</span></code> messages, which are in turn members of <codeclass="docutils literal"><spanclass="pre">BlockDesc</span></code>, which is a member of <codeclass="docutils literal"><spanclass="pre">ProgramDesc</span></code>.</p>
<p><codeclass="docutils literal"><spanclass="pre">Program</span></code> creates the first block as the global block in its constructor. All parameters and their initializer operators are in the global block.</p>
</div>
<divclass="section"id="block">
<spanid="block"></span><h3>Block<aclass="headerlink"href="#block"title="Permalink to this headline">¶</a></h3>
<p><codeclass="docutils literal"><spanclass="pre">create_parameter</span></code> is necessary because parameters are global variables, those defined in the global block, but can be created in some sub-blocks, e.g., an FC layer in the step block of an RNN operator.</p>
<p><codeclass="docutils literal"><spanclass="pre">prepand_operator</span></code> is necessary because the constructor of <codeclass="docutils literal"><spanclass="pre">Parameter</span></code> needs to create the initialize (or load) operator of the parameter, and would like to put it in the <em>preamble</em> of the global block.</p>
</div>
<divclass="section"id="operator">
<spanid="operator"></span><h3>Operator<aclass="headerlink"href="#operator"title="Permalink to this headline">¶</a></h3>
<p>The <codeclass="docutils literal"><spanclass="pre">Operator</span></code> class fills in the <codeclass="docutils literal"><spanclass="pre">OpDesc</span></code> message and calls the C++ function <codeclass="docutils literal"><spanclass="pre">InferShape</span></code> to infer output shape from input shape.</p>
<p><codeclass="docutils literal"><spanclass="pre">Operator</span></code> creates the <codeclass="docutils literal"><spanclass="pre">OpDesc</span></code> message in C++ space, so could it call the <codeclass="docutils literal"><spanclass="pre">InferShape</span></code> function, which is in C++.</p>
</div>
<divclass="section"id="variable">
<spanid="variable"></span><h3>Variable<aclass="headerlink"href="#variable"title="Permalink to this headline">¶</a></h3>
<p>Operators take Variables as its inputs and outputs.</p>
<p>Please be aware of <codeclass="docutils literal"><spanclass="pre">self.writer</span></code>, that tracks operator who creates the variable. It possible that there are more than one operators who write a variable, but in Python space, each writes to a variable is represented by a Variable class. This is guaranteed by the fact that <strong><codeclass="docutils literal"><spanclass="pre">core.NewVarDesc</span></code> must NOT create a new <codeclass="docutils literal"><spanclass="pre">VarDesc</span></code> message if its name already exists in the specified block</strong>.</p>
</div>
<divclass="section"id="parameter">
<spanid="parameter"></span><h3>Parameter<aclass="headerlink"href="#parameter"title="Permalink to this headline">¶</a></h3>
<p>A parameter is a global variable with an initializer (or load) operator.</p>
<p>In above example, <codeclass="docutils literal"><spanclass="pre">init_attr.type</span></code> names an initialize operator. It can also name the load operator</p>
<p><codeclass="docutils literal"><spanclass="pre">optimize_op_attrs</span></code> is not in the <codeclass="docutils literal"><spanclass="pre">VarDesc</span></code> message, but kept in the Python instance, as it will be used in the Python space when creating the optimize operator’s <codeclass="docutils literal"><spanclass="pre">OpDesc</span></code>, and will be in the <codeclass="docutils literal"><spanclass="pre">OpDesc</span></code> message.</p>
</div>
</div>
<divclass="section"id="layer-functions">
<spanid="layer-functions"></span><h2>Layer Functions<aclass="headerlink"href="#layer-functions"title="Permalink to this headline">¶</a></h2>
<p>A layer is a Python function that creates some operators and variables. Layers simplify the work of application programmers.</p>
<divclass="section"id="data-layer">
<spanid="data-layer"></span><h3>Data Layer<aclass="headerlink"href="#data-layer"title="Permalink to this headline">¶</a></h3>
<p>The input to the feed operator is a special variable in the global scope, which is the output of <aclass="reference external"href="https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/reader/README.md">Python readers</a>.</p>
</div>
<divclass="section"id="fc-layer">
<spanid="fc-layer"></span><h3>FC Layer<aclass="headerlink"href="#fc-layer"title="Permalink to this headline">¶</a></h3>
Built with <ahref="http://sphinx-doc.org/">Sphinx</a> using a <ahref="https://github.com/snide/sphinx_rtd_theme">theme</a> provided by <ahref="https://readthedocs.org">Read the Docs</a>.
Due to the refactorization of the PaddlePaddle core, we need Python classes to construct corresponding protobuf messages that describe a DL program.
| Python classes | Protobuf messages |
| --- | --- |
| Program | ProgramDesc |
| Block | BlockDesc |
| Operator | OpDesc |
| Variable | VarDesc |
Please be aware that these Python classes need to maintain some construction-time information, which are not part of the protobuf messages.
## Core Concepts
### Program
A `ProgramDesc` describes a [DL program](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/program.md), which is composed of an array of `BlockDesc`s. A `BlockDesc` refers to its parent block by its index in the array. For example, operators in the step block of an RNN operator needs to be able to access variables in its ancessor blocks.
Whenever we create a block, we need set its parent block to the current block, so the Python class `Program` needs to maintain a data member `current_block`.
```python
class Program(objects):
def __init__(self):
self.proto = core.NewProgram() # a C++ ProgramDesc pointer.
self.blocks = vector<Block>()
self.blocks.append(Block(self, -1)) # the global block
self.current_block = 0 # initialized to the global block
`Program` is an accessor to the protobuf message `ProgramDesc`, which is created in C++ space, because the InferShape function is in C++, which manipulates `VarDesc` messages, which are in turn members of `BlockDesc`, which is a member of `ProgramDesc`.
`Program` creates the first block as the global block in its constructor. All parameters and their initializer operators are in the global block.
### Block
A [Block](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/block.md) includes
1. a map from variable names to an instance of the Python `Variable` class, and
1. a list of `Operator` instances.
```python
class Block(objects):
def __init__(self, program, parent_idx):
self.proto = core.NewBlock(program.proto)
self.program = program
self.vars = map<string, Variable>()
self.ops = vector<Operator>()
self.parent_idx = parent_idx
def create_var(self, ...):
return Variable(self, ...)
def _create_global_var(self, ...):
program.global_block().create_var(...)
def create_parameter(self, name, ...):
# Parameter is a subclass of variable. See Parameter section for details.
`create_parameter` is necessary because parameters are global variables, those defined in the global block, but can be created in some sub-blocks, e.g., an FC layer in the step block of an RNN operator.
`prepand_operator` is necessary because the constructor of `Parameter` needs to create the initialize (or load) operator of the parameter, and would like to put it in the *preamble* of the global block.
### Operator
The `Operator` class fills in the `OpDesc` message and calls the C++ function `InferShape` to infer output shape from input shape.
Please be aware of `self.writer`, that tracks operator who creates the variable. It possible that there are more than one operators who write a variable, but in Python space, each writes to a variable is represented by a Variable class. This is guaranteed by the fact that **`core.NewVarDesc` must NOT create a new `VarDesc` message if its name already exists in the specified block**.
### Parameter
A parameter is a global variable with an initializer (or load) operator.
In above example, `init_attr.type` names an initialize operator. It can also name the load operator
```python
init_attr={
type: "load",
filename: "something.numpy",
}
```
`optimize_op_attrs` is not in the `VarDesc` message, but kept in the Python instance, as it will be used in the Python space when creating the optimize operator's `OpDesc`, and will be in the `OpDesc` message.
## Layer Functions
A layer is a Python function that creates some operators and variables. Layers simplify the work of application programmers.
### Data Layer
```python
def data_layer(name, type, column_name):
block = the_current_program.glolal_block()
var = block.create_global_var(
name=name,
shape=[None] + type.dims(),
dtype=type.dtype)
block.prepend_operator(block,
type="Feed",
inputs = None,
outputs = [var],
{column_name: column_name})
return var
```
The input to the feed operator is a special variable in the global scope, which is the output of [Python readers](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/reader/README.md).
### FC Layer
```python
def fc_layer(input, size, ...):
block = program.current_block()
w = block.create_parameter(...)
b = block.create_parameter(...)
out = block.create_var()
op = block.append_operator("FC", X=input, W=w, b=b, out=out)
<p>Due to the refactorization of the PaddlePaddle core, we need Python classes to construct corresponding protobuf messages that describe a DL program.</p>
<p>| Python classes | Protobuf messages |
| — | — |
| Program | ProgramDesc |
| Block | BlockDesc |
| Operator | OpDesc |
| Variable | VarDesc |</p>
<p>Please be aware that these Python classes need to maintain some construction-time information, which are not part of the protobuf messages.</p>
<p>A <codeclass="docutils literal"><spanclass="pre">ProgramDesc</span></code> describes a <aclass="reference external"href="https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/program.md">DL program</a>, which is composed of an array of <codeclass="docutils literal"><spanclass="pre">BlockDesc</span></code>s. A <codeclass="docutils literal"><spanclass="pre">BlockDesc</span></code> refers to its parent block by its index in the array. For example, operators in the step block of an RNN operator needs to be able to access variables in its ancessor blocks.</p>
<p>Whenever we create a block, we need set its parent block to the current block, so the Python class <codeclass="docutils literal"><spanclass="pre">Program</span></code> needs to maintain a data member <codeclass="docutils literal"><spanclass="pre">current_block</span></code>.</p>
<spanclass="bp">self</span><spanclass="o">.</span><spanclass="n">proto</span><spanclass="o">=</span><spanclass="n">core</span><spanclass="o">.</span><spanclass="n">NewProgram</span><spanclass="p">()</span><spanclass="c1"># a C++ ProgramDesc pointer.</span>
<spanclass="bp">self</span><spanclass="o">.</span><spanclass="n">blocks</span><spanclass="o">.</span><spanclass="n">append</span><spanclass="p">(</span><spanclass="n">Block</span><spanclass="p">(</span><spanclass="bp">self</span><spanclass="p">,</span><spanclass="o">-</span><spanclass="mi">1</span><spanclass="p">))</span><spanclass="c1"># the global block</span>
<spanclass="bp">self</span><spanclass="o">.</span><spanclass="n">current_block</span><spanclass="o">=</span><spanclass="mi">0</span><spanclass="c1"># initialized to the global block</span>
<p><codeclass="docutils literal"><spanclass="pre">Program</span></code> is an accessor to the protobuf message <codeclass="docutils literal"><spanclass="pre">ProgramDesc</span></code>, which is created in C++ space, because the InferShape function is in C++, which manipulates <codeclass="docutils literal"><spanclass="pre">VarDesc</span></code> messages, which are in turn members of <codeclass="docutils literal"><spanclass="pre">BlockDesc</span></code>, which is a member of <codeclass="docutils literal"><spanclass="pre">ProgramDesc</span></code>.</p>
<p><codeclass="docutils literal"><spanclass="pre">Program</span></code> creates the first block as the global block in its constructor. All parameters and their initializer operators are in the global block.</p>
<p><codeclass="docutils literal"><spanclass="pre">create_parameter</span></code> is necessary because parameters are global variables, those defined in the global block, but can be created in some sub-blocks, e.g., an FC layer in the step block of an RNN operator.</p>
<p><codeclass="docutils literal"><spanclass="pre">prepand_operator</span></code> is necessary because the constructor of <codeclass="docutils literal"><spanclass="pre">Parameter</span></code> needs to create the initialize (or load) operator of the parameter, and would like to put it in the <em>preamble</em> of the global block.</p>
<p>The <codeclass="docutils literal"><spanclass="pre">Operator</span></code> class fills in the <codeclass="docutils literal"><spanclass="pre">OpDesc</span></code> message and calls the C++ function <codeclass="docutils literal"><spanclass="pre">InferShape</span></code> to infer output shape from input shape.</p>
<p><codeclass="docutils literal"><spanclass="pre">Operator</span></code> creates the <codeclass="docutils literal"><spanclass="pre">OpDesc</span></code> message in C++ space, so could it call the <codeclass="docutils literal"><spanclass="pre">InferShape</span></code> function, which is in C++.</p>
<p>Please be aware of <codeclass="docutils literal"><spanclass="pre">self.writer</span></code>, that tracks operator who creates the variable. It possible that there are more than one operators who write a variable, but in Python space, each writes to a variable is represented by a Variable class. This is guaranteed by the fact that <strong><codeclass="docutils literal"><spanclass="pre">core.NewVarDesc</span></code> must NOT create a new <codeclass="docutils literal"><spanclass="pre">VarDesc</span></code> message if its name already exists in the specified block</strong>.</p>
<p>In above example, <codeclass="docutils literal"><spanclass="pre">init_attr.type</span></code> names an initialize operator. It can also name the load operator</p>
<p><codeclass="docutils literal"><spanclass="pre">optimize_op_attrs</span></code> is not in the <codeclass="docutils literal"><spanclass="pre">VarDesc</span></code> message, but kept in the Python instance, as it will be used in the Python space when creating the optimize operator’s <codeclass="docutils literal"><spanclass="pre">OpDesc</span></code>, and will be in the <codeclass="docutils literal"><spanclass="pre">OpDesc</span></code> message.</p>
<p>The input to the feed operator is a special variable in the global scope, which is the output of <aclass="reference external"href="https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/reader/README.md">Python readers</a>.</p>
Built with <ahref="http://sphinx-doc.org/">Sphinx</a> using a <ahref="https://github.com/snide/sphinx_rtd_theme">theme</a> provided by <ahref="https://readthedocs.org">Read the Docs</a>.
