The basic structure of a PaddlePaddle program is some nested blocks, as a C++ or Java program.
## Compile and Execution
A PaddlePaddle program consists of two parts -- the first generates a `ProgramDesc` protobuf message that describes the program, and the second runs this message using a C++ class `Executor`.
As described in [graph.md](./graph.md), the first five lines of the following PaddlePaddle program
A simple example PaddlePaddle program can be found in [graph.md](./graph.md):
```python
```python
x = layer.data("images")
x = layer.data("images")
...
@@ -13,36 +15,112 @@ optimize(cost)
...
@@ -13,36 +15,112 @@ optimize(cost)
train(cost, reader=mnist.train())
train(cost, reader=mnist.train())
```
```
generates, or compiles, a PaddelPaddle program, which is represented by the following protobuf message:
The first five lines of the following PaddlePaddle program generates, or, compiles, the `ProgramDesc` message. The last line runs it.
```protobuf
## Programs and Blocks
message ProgramDesc {
repeated BlockDesc blocks = 1;
The basic structure of a PaddlePaddle program is some nested blocks, as a C++ or Java program.
- program: some nested blocks
- [block](./block.md):
- some local variable definitions, and
- a sequence of operators
The concept of block comes from usual programs. For example, the following C++ program has three blocks:
```c++
int main() { // block 0
int i = 0;
if (i < 10) { // block 1
for (int j = 0; j < 10; j++) { // block 2
}
}
return 0;
}
}
```
The following PaddlePaddle program has three blocks:
```python
import paddle as pd // block 0
x = minibatch([10, 20, 30]) # shape=[None, 1]
y = var(1) # shape=[1], value=1
z = minibatch([10, 20, 30]) # shape=[None, 1]
cond = larger_than(x, 15) # [false, true, true]
ie = pd.ifelse()
with ie.true_block(): // block 1
d = pd.layer.add_scalar(x, y)
ie.output(d, pd.layer.softmax(d))
with ie.false_block(): // block 2
d = pd.layer.fc(z)
ie.output(d, d+1)
o1, o2 = ie(cond)
```
## `BlockDesc` and `ProgramDesc`
All protobuf messages are defined in `framework.proto`.
`BlockDesc` is straight-forward -- it includes local variable definitions, `vars`, and a sequence of operators, `ops`.
```protobuf
message BlockDesc {
message BlockDesc {
required int32 parent = 1;
required int32 parent = 1;
repeated VarDesc vars = 2;
repeated VarDesc vars = 2;
repeated OpDesc ops = 3;
repeated OpDesc ops = 3;
}
}
```
The parent ID indicates the parent block so that operators in a block can refer to variables defined locally and also those defined in their ancestor blocks.
All hierarchical blocks in a program are flattened and stored in an array. The block ID is the index of the block in this array.
```protobuf
message ProgramDesc {
repeated BlockDesc blocks = 1;
}
```
### Global Block
The global block is the first one in the above array.
## Operators that Use Blocks
In the above example, the operator `IfElseOp` has two blocks -- the true branch and the false branch.
The definition of `OpDesc` shows that an operator could have some attributes:
```protobuf
message OpDesc {
message OpDesc {
AttrDesc attrs = 1;
AttrDesc attrs = 1;
...
...
}
}
```
and an attribute could be of type block, which is, in fact, a block ID as described above:
```
message AttrDesc {
message AttrDesc {
required AttrType type = 1;
required string name = 1;
// index into ProgramDesc::blocks when type==BLOCK
enum AttrType {
optional int32 block = 2;
INT = 1,
STRING = 2,
...
BLOCK = ...
}
required AttrType type = 2;
optional int32 block = 10; // when type == BLOCK
...
...
}
}
```
```
When each of the first five lines runs, related Python function, e.g., `layer.fc`, calls C++ InferShape functions. This InferShape function needs to access the properties of VarDesc's accessed by the current OpDesc. These VarDesc's might not be defined in the current block, but in some ancestor blocks. This requires that we can trace the parent of a block.
## InferShape
A nested block is often an attribute of an operator, most likely, an IfElseOp or a WhileOp. In above solution, all blocks are in `ProgramDesc::blocks`, this implicitly assigns a zero-based ID to each block -- the index of the block in `ProgramDesc::blocks`. So that `AttrDesc::block` could be an integer block ID.
With this design, the InferShape function should take the following parameters:
With this design, the InferShape function should take the following parameters:
<spanid="design-doc-programdesc"></span><h1>Design Doc: ProgramDesc<aclass="headerlink"href="#design-doc-programdesc"title="Permalink to this headline">¶</a></h1>
<spanid="design-doc-paddlepaddle-programs"></span><h1>Design Doc: PaddlePaddle Programs<aclass="headerlink"href="#design-doc-paddlepaddle-programs"title="Permalink to this headline">¶</a></h1>
<p>The basic structure of a PaddlePaddle program is some nested blocks, as a C++ or Java program.</p>
<divclass="section"id="compile-and-execution">
<p>As described in <aclass="reference internal"href="graph.html"><spanclass="doc">graph.md</span></a>, the first five lines of the following PaddlePaddle program</p>
<spanid="compile-and-execution"></span><h2>Compile and Execution<aclass="headerlink"href="#compile-and-execution"title="Permalink to this headline">¶</a></h2>
<p>A PaddlePaddle program consists of two parts – the first generates a <codeclass="docutils literal"><spanclass="pre">ProgramDesc</span></code> protobuf message that describes the program, and the second runs this message using a C++ class <codeclass="docutils literal"><spanclass="pre">Executor</span></code>.</p>
<p>A simple example PaddlePaddle program can be found in <aclass="reference internal"href="graph.html"><spanclass="doc">graph.md</span></a>:</p>
<p>generates, or compiles, a PaddelPaddle program, which is represented by the following protobuf message:</p>
<p>The first five lines of the following PaddlePaddle program generates, or, compiles, the <codeclass="docutils literal"><spanclass="pre">ProgramDesc</span></code> message. The last line runs it.</p>
<spanid="programs-and-blocks"></span><h2>Programs and Blocks<aclass="headerlink"href="#programs-and-blocks"title="Permalink to this headline">¶</a></h2>
<p>The basic structure of a PaddlePaddle program is some nested blocks, as a C++ or Java program.</p>
<spanid="blockdesc-and-programdesc"></span><h2><codeclass="docutils literal"><spanclass="pre">BlockDesc</span></code> and <codeclass="docutils literal"><spanclass="pre">ProgramDesc</span></code><aclass="headerlink"href="#blockdesc-and-programdesc"title="Permalink to this headline">¶</a></h2>
<p>All protobuf messages are defined in <codeclass="docutils literal"><spanclass="pre">framework.proto</span></code>.</p>
<p><codeclass="docutils literal"><spanclass="pre">BlockDesc</span></code> is straight-forward – it includes local variable definitions, <codeclass="docutils literal"><spanclass="pre">vars</span></code>, and a sequence of operators, <codeclass="docutils literal"><spanclass="pre">ops</span></code>.</p>
<p>The parent ID indicates the parent block so that operators in a block can refer to variables defined locally and also those defined in their ancestor blocks.</p>
<p>All hierarchical blocks in a program are flattened and stored in an array. The block ID is the index of the block in this array.</p>
<spanid="operators-that-use-blocks"></span><h2>Operators that Use Blocks<aclass="headerlink"href="#operators-that-use-blocks"title="Permalink to this headline">¶</a></h2>
<p>In the above example, the operator <codeclass="docutils literal"><spanclass="pre">IfElseOp</span></code> has two blocks – the true branch and the false branch.</p>
<p>The definition of <codeclass="docutils literal"><spanclass="pre">OpDesc</span></code> shows that an operator could have some attributes:</p>
<p>When each of the first five lines runs, related Python function, e.g., <codeclass="docutils literal"><spanclass="pre">layer.fc</span></code>, calls C++ InferShape functions. This InferShape function needs to access the properties of VarDesc’s accessed by the current OpDesc. These VarDesc’s might not be defined in the current block, but in some ancestor blocks. This requires that we can trace the parent of a block.</p>
</div>
<p>A nested block is often an attribute of an operator, most likely, an IfElseOp or a WhileOp. In above solution, all blocks are in <codeclass="docutils literal"><spanclass="pre">ProgramDesc::blocks</span></code>, this implicitly assigns a zero-based ID to each block – the index of the block in <codeclass="docutils literal"><spanclass="pre">ProgramDesc::blocks</span></code>. So that <codeclass="docutils literal"><spanclass="pre">AttrDesc::block</span></code> could be an integer block ID.</p>
<divclass="section"id="infershape">
<spanid="infershape"></span><h2>InferShape<aclass="headerlink"href="#infershape"title="Permalink to this headline">¶</a></h2>
<p>With this design, the InferShape function should take the following parameters:</p>
<p>With this design, the InferShape function should take the following parameters:</p>
<li><codeclass="docutils literal"><spanclass="pre">current_block</span></code> indices into <codeclass="docutils literal"><spanclass="pre">ProgramDesc::blocks</span></code>,</li>
<li><codeclass="docutils literal"><spanclass="pre">current_block</span></code> indices into <codeclass="docutils literal"><spanclass="pre">ProgramDesc::blocks</span></code>,</li>
<li><codeclass="docutils literal"><spanclass="pre">current_operator</span></code> indices into <codeclass="docutils literal"><spanclass="pre">BlockDesc::ops</span></code>.</li>
<li><codeclass="docutils literal"><spanclass="pre">current_operator</span></code> indices into <codeclass="docutils literal"><spanclass="pre">BlockDesc::ops</span></code>.</li>
The basic structure of a PaddlePaddle program is some nested blocks, as a C++ or Java program.
## Compile and Execution
A PaddlePaddle program consists of two parts -- the first generates a `ProgramDesc` protobuf message that describes the program, and the second runs this message using a C++ class `Executor`.
As described in [graph.md](./graph.md), the first five lines of the following PaddlePaddle program
A simple example PaddlePaddle program can be found in [graph.md](./graph.md):
```python
```python
x = layer.data("images")
x = layer.data("images")
...
@@ -13,36 +15,112 @@ optimize(cost)
...
@@ -13,36 +15,112 @@ optimize(cost)
train(cost, reader=mnist.train())
train(cost, reader=mnist.train())
```
```
generates, or compiles, a PaddelPaddle program, which is represented by the following protobuf message:
The first five lines of the following PaddlePaddle program generates, or, compiles, the `ProgramDesc` message. The last line runs it.
```protobuf
## Programs and Blocks
message ProgramDesc {
repeated BlockDesc blocks = 1;
The basic structure of a PaddlePaddle program is some nested blocks, as a C++ or Java program.
- program: some nested blocks
- [block](./block.md):
- some local variable definitions, and
- a sequence of operators
The concept of block comes from usual programs. For example, the following C++ program has three blocks:
```c++
int main() { // block 0
int i = 0;
if (i < 10) { // block 1
for (int j = 0; j < 10; j++) { // block 2
}
}
return 0;
}
}
```
The following PaddlePaddle program has three blocks:
```python
import paddle as pd // block 0
x = minibatch([10, 20, 30]) # shape=[None, 1]
y = var(1) # shape=[1], value=1
z = minibatch([10, 20, 30]) # shape=[None, 1]
cond = larger_than(x, 15) # [false, true, true]
ie = pd.ifelse()
with ie.true_block(): // block 1
d = pd.layer.add_scalar(x, y)
ie.output(d, pd.layer.softmax(d))
with ie.false_block(): // block 2
d = pd.layer.fc(z)
ie.output(d, d+1)
o1, o2 = ie(cond)
```
## `BlockDesc` and `ProgramDesc`
All protobuf messages are defined in `framework.proto`.
`BlockDesc` is straight-forward -- it includes local variable definitions, `vars`, and a sequence of operators, `ops`.
```protobuf
message BlockDesc {
message BlockDesc {
required int32 parent = 1;
required int32 parent = 1;
repeated VarDesc vars = 2;
repeated VarDesc vars = 2;
repeated OpDesc ops = 3;
repeated OpDesc ops = 3;
}
}
```
The parent ID indicates the parent block so that operators in a block can refer to variables defined locally and also those defined in their ancestor blocks.
All hierarchical blocks in a program are flattened and stored in an array. The block ID is the index of the block in this array.
```protobuf
message ProgramDesc {
repeated BlockDesc blocks = 1;
}
```
### Global Block
The global block is the first one in the above array.
## Operators that Use Blocks
In the above example, the operator `IfElseOp` has two blocks -- the true branch and the false branch.
The definition of `OpDesc` shows that an operator could have some attributes:
```protobuf
message OpDesc {
message OpDesc {
AttrDesc attrs = 1;
AttrDesc attrs = 1;
...
...
}
}
```
and an attribute could be of type block, which is, in fact, a block ID as described above:
```
message AttrDesc {
message AttrDesc {
required AttrType type = 1;
required string name = 1;
// index into ProgramDesc::blocks when type==BLOCK
enum AttrType {
optional int32 block = 2;
INT = 1,
STRING = 2,
...
BLOCK = ...
}
required AttrType type = 2;
optional int32 block = 10; // when type == BLOCK
...
...
}
}
```
```
When each of the first five lines runs, related Python function, e.g., `layer.fc`, calls C++ InferShape functions. This InferShape function needs to access the properties of VarDesc's accessed by the current OpDesc. These VarDesc's might not be defined in the current block, but in some ancestor blocks. This requires that we can trace the parent of a block.
## InferShape
A nested block is often an attribute of an operator, most likely, an IfElseOp or a WhileOp. In above solution, all blocks are in `ProgramDesc::blocks`, this implicitly assigns a zero-based ID to each block -- the index of the block in `ProgramDesc::blocks`. So that `AttrDesc::block` could be an integer block ID.
With this design, the InferShape function should take the following parameters:
With this design, the InferShape function should take the following parameters:
<p>The basic structure of a PaddlePaddle program is some nested blocks, as a C++ or Java program.</p>
<divclass="section"id="compile-and-execution">
<p>As described in <aclass="reference internal"href="graph.html"><spanclass="doc">graph.md</span></a>, the first five lines of the following PaddlePaddle program</p>
<spanid="compile-and-execution"></span><h2>Compile and Execution<aclass="headerlink"href="#compile-and-execution"title="永久链接至标题">¶</a></h2>
<p>A PaddlePaddle program consists of two parts – the first generates a <codeclass="docutils literal"><spanclass="pre">ProgramDesc</span></code> protobuf message that describes the program, and the second runs this message using a C++ class <codeclass="docutils literal"><spanclass="pre">Executor</span></code>.</p>
<p>A simple example PaddlePaddle program can be found in <aclass="reference internal"href="graph.html"><spanclass="doc">graph.md</span></a>:</p>
<p>generates, or compiles, a PaddelPaddle program, which is represented by the following protobuf message:</p>
<p>The first five lines of the following PaddlePaddle program generates, or, compiles, the <codeclass="docutils literal"><spanclass="pre">ProgramDesc</span></code> message. The last line runs it.</p>
<spanid="blockdesc-and-programdesc"></span><h2><codeclass="docutils literal"><spanclass="pre">BlockDesc</span></code> and <codeclass="docutils literal"><spanclass="pre">ProgramDesc</span></code><aclass="headerlink"href="#blockdesc-and-programdesc"title="永久链接至标题">¶</a></h2>
<p>All protobuf messages are defined in <codeclass="docutils literal"><spanclass="pre">framework.proto</span></code>.</p>
<p><codeclass="docutils literal"><spanclass="pre">BlockDesc</span></code> is straight-forward – it includes local variable definitions, <codeclass="docutils literal"><spanclass="pre">vars</span></code>, and a sequence of operators, <codeclass="docutils literal"><spanclass="pre">ops</span></code>.</p>
<p>The parent ID indicates the parent block so that operators in a block can refer to variables defined locally and also those defined in their ancestor blocks.</p>
<p>All hierarchical blocks in a program are flattened and stored in an array. The block ID is the index of the block in this array.</p>
<spanid="operators-that-use-blocks"></span><h2>Operators that Use Blocks<aclass="headerlink"href="#operators-that-use-blocks"title="永久链接至标题">¶</a></h2>
<p>In the above example, the operator <codeclass="docutils literal"><spanclass="pre">IfElseOp</span></code> has two blocks – the true branch and the false branch.</p>
<p>The definition of <codeclass="docutils literal"><spanclass="pre">OpDesc</span></code> shows that an operator could have some attributes:</p>
<p>When each of the first five lines runs, related Python function, e.g., <codeclass="docutils literal"><spanclass="pre">layer.fc</span></code>, calls C++ InferShape functions. This InferShape function needs to access the properties of VarDesc’s accessed by the current OpDesc. These VarDesc’s might not be defined in the current block, but in some ancestor blocks. This requires that we can trace the parent of a block.</p>
</div>
<p>A nested block is often an attribute of an operator, most likely, an IfElseOp or a WhileOp. In above solution, all blocks are in <codeclass="docutils literal"><spanclass="pre">ProgramDesc::blocks</span></code>, this implicitly assigns a zero-based ID to each block – the index of the block in <codeclass="docutils literal"><spanclass="pre">ProgramDesc::blocks</span></code>. So that <codeclass="docutils literal"><spanclass="pre">AttrDesc::block</span></code> could be an integer block ID.</p>
<li><codeclass="docutils literal"><spanclass="pre">current_block</span></code> indices into <codeclass="docutils literal"><spanclass="pre">ProgramDesc::blocks</span></code>,</li>
<li><codeclass="docutils literal"><spanclass="pre">current_block</span></code> indices into <codeclass="docutils literal"><spanclass="pre">ProgramDesc::blocks</span></code>,</li>
<li><codeclass="docutils literal"><spanclass="pre">current_operator</span></code> indices into <codeclass="docutils literal"><spanclass="pre">BlockDesc::ops</span></code>.</li>
<li><codeclass="docutils literal"><spanclass="pre">current_operator</span></code> indices into <codeclass="docutils literal"><spanclass="pre">BlockDesc::ops</span></code>.</li>
