@@ -55,17 +55,23 @@ Let us consolidate the discussion by presenting some examples.
...
@@ -55,17 +55,23 @@ Let us consolidate the discussion by presenting some examples.
The following C++ programs shows how blocks are used with the `if-else` structure:
The following C++ programs shows how blocks are used with the `if-else` structure:
```c++
```c++
namespace pd = paddle;
int x = 10;
int x = 10;
int y = 20;
int y = 1;
int out;
int z = 10;
bool cond = false;
bool cond = false;
int o1, o2;
if (cond) {
if (cond) {
int z = x + y;
int z = x + y;
out = softmax(z);
o1 = z;
o2 = pd::layer::softmax(z);
} else {
} else {
int z = fc(x);
int d = pd::layer::fc(z);
out = z;
o1 = d;
o2 = d+1;
}
}
```
```
An equivalent PaddlePaddle program from the design doc of the [IfElseOp operator](./if_else_op.md) is as follows:
An equivalent PaddlePaddle program from the design doc of the [IfElseOp operator](./if_else_op.md) is as follows:
...
@@ -73,57 +79,55 @@ An equivalent PaddlePaddle program from the design doc of the [IfElseOp operator
...
@@ -73,57 +79,55 @@ An equivalent PaddlePaddle program from the design doc of the [IfElseOp operator
```python
```python
import paddle as pd
import paddle as pd
x = var(10)
x = minibatch([10, 20, 30]) # shape=[None, 1]
y = var(20)
y = var(1) # shape=[1], value=1
cond = var(false)
z = minibatch([10, 20, 30]) # shape=[None, 1]
ie = pd.create_ifelseop(inputs=[x], output_num=1)
cond = larger_than(x, 15) # [false, true, true]
ie = pd.ifelse()
with ie.true_block():
with ie.true_block():
x = ie.inputs(true, 0)
d = pd.layer.add_scalar(x, y)
z = operator.add(x, y)
ie.output(d, pd.layer.softmax(d))
ie.set_output(true, 0, operator.softmax(z))
with ie.false_block():
with ie.false_block():
x = ie.inputs(false, 0)
d = pd.layer.fc(z)
z = layer.fc(x)
ie.output(d, d+1)
ie.set_output(true, 0, operator.softmax(z))
o1, o2 = ie(cond)
out = b(cond)
```
```
In both examples, the left branch computes `softmax(x+y)` and the right branch computes `fc(x)`.
In both examples, the left branch computes `x+y` and `softmax(x+y)`, the right branch computes `x+1` and `fc(x)`.
A difference is that variables in the C++ program contain scalar values, whereas those in the PaddlePaddle programs are mini-batches of instances. The `ie.input(true, 0)` invocation returns instances in the 0-th input, `x`, that corresponds to true values in `cond` as the local variable `x`, where `ie.input(false, 0)` returns instances corresponding to false values.
A difference is that variables in the C++ program contain scalar values, whereas those in the PaddlePaddle programs are mini-batches of instances. The `ie.input(true, 0)` invocation returns instances in the 0-th input, `x`, that corresponds to true values in `cond` as the local variable `x`, where `ie.input(false, 0)` returns instances corresponding to false values.
### Blocks with `for` and `RNNOp`
### Blocks with `for` and `RNNOp`
The following RNN model from the [RNN design doc](./rnn.md)
The following RNN model from the [RNN design doc](./rnn.md)
```python
```python
x = sequence([10, 20, 30])
x = sequence([10, 20, 30]) # shape=[None, 1]
m = var(0)
m = var(0) # shape=[1]
W = tensor()
W = var(0.314, param=true) # shape=[1]
U = tensor()
U = var(0.375, param=true) # shape=[1]
rnn = create_rnn(inputs=[input])
rnn = pd.rnn()
with rnn.stepnet() as net:
with rnn.step():
x = net.set_inputs(0)
h = rnn.memory(init = m)
h = net.add_memory(init=m)
hh = rnn.previous_memory(h)
fc_out = pd.matmul(W, x)
a = layer.fc(W, x)
hidden_out = pd.matmul(U, h.pre(n=1))
b = layer.fc(U, hh)
sum = pd.add_two(fc_out, hidden_out)
s = pd.add(a, b)
act = pd.sigmoid(sum)
act = pd.sigmoid(s)
h.update(act) # update memory with act
rnn.update_memory(h, act)
net.set_outputs(0, act, hidden_out) # two outputs
rnn.output(a, b)
o1, o2 = rnn()
o1, o2 = rnn()
print o1, o2
```
```
has its equivalent C++ program as follows
has its equivalent C++ program as follows
```c++
```c++
int* x = {10, 20, 30};
int* x = {10, 20, 30};
int m = 0;
int* m = {0};
int W = some_value();
int* W = {0.314};
int U = some_other_value();
int* U = {0.375};
int mem[sizeof(x) / sizeof(x[0]) + 1];
int mem[sizeof(x) / sizeof(x[0]) + 1];
int o1[sizeof(x) / sizeof(x[0]) + 1];
int o1[sizeof(x) / sizeof(x[0]) + 1];
...
@@ -131,20 +135,16 @@ int o2[sizeof(x) / sizeof(x[0]) + 1];
...
@@ -131,20 +135,16 @@ int o2[sizeof(x) / sizeof(x[0]) + 1];
for (int i = 1; i <= sizeof(x)/sizeof(x[0]); ++i) {
for (int i = 1; i <= sizeof(x)/sizeof(x[0]); ++i) {
int x = x[i-1];
int x = x[i-1];
if (i == 1) mem[0] = m;
if (i == 1) mem[0] = m;
int fc_out = W * x;
int a = W * x;
int hidden_out = Y * mem[i-1];
int b = Y * mem[i-1];
int sum = fc_out + hidden_out;
int s = fc_out + hidden_out;
int act = sigmoid(sum);
int act = sigmoid(sum);
mem[i] = act;
mem[i] = act;
o1[i] = act;
o1[i] = act;
o2[i] = hidden_out;
o2[i] = hidden_out;
}
}
print_array(o1);
print_array(o2);
```
```
## Compilation and Execution
## Compilation and Execution
Like TensorFlow programs, a PaddlePaddle program is written in Python. The first part describes a neural network as a protobuf message, and the rest part executes the message for training or inference.
Like TensorFlow programs, a PaddlePaddle program is written in Python. The first part describes a neural network as a protobuf message, and the rest part executes the message for training or inference.
...
@@ -210,11 +210,11 @@ a = pd.Varaible(shape=[20, 20])
...
@@ -210,11 +210,11 @@ a = pd.Varaible(shape=[20, 20])
<spanid="blocks-with-if-else-and-ifelseop"></span><h3>Blocks with <codeclass="docutils literal"><spanclass="pre">if-else</span></code> and <codeclass="docutils literal"><spanclass="pre">IfElseOp</span></code><aclass="headerlink"href="#blocks-with-if-else-and-ifelseop"title="Permalink to this headline">¶</a></h3>
<spanid="blocks-with-if-else-and-ifelseop"></span><h3>Blocks with <codeclass="docutils literal"><spanclass="pre">if-else</span></code> and <codeclass="docutils literal"><spanclass="pre">IfElseOp</span></code><aclass="headerlink"href="#blocks-with-if-else-and-ifelseop"title="Permalink to this headline">¶</a></h3>
<p>The following C++ programs shows how blocks are used with the <codeclass="docutils literal"><spanclass="pre">if-else</span></code> structure:</p>
<p>The following C++ programs shows how blocks are used with the <codeclass="docutils literal"><spanclass="pre">if-else</span></code> structure:</p>
<p>An equivalent PaddlePaddle program from the design doc of the <aclass="reference internal"href="if_else_op.html"><spanclass="doc">IfElseOp operator</span></a> is as follows:</p>
<p>An equivalent PaddlePaddle program from the design doc of the <aclass="reference internal"href="if_else_op.html"><spanclass="doc">IfElseOp operator</span></a> is as follows:</p>
<p>In both examples, the left branch computes <codeclass="docutils literal"><spanclass="pre">softmax(x+y)</span></code> and the right branch computes<codeclass="docutils literal"><spanclass="pre">fc(x)</span></code>.</p>
<p>In both examples, the left branch computes <codeclass="docutils literal"><spanclass="pre">x+y</span></code> and <codeclass="docutils literal"><spanclass="pre">softmax(x+y)</span></code>, the right branch computes <codeclass="docutils literal"><spanclass="pre">x+1</span></code> and<codeclass="docutils literal"><spanclass="pre">fc(x)</span></code>.</p>
<p>A difference is that variables in the C++ program contain scalar values, whereas those in the PaddlePaddle programs are mini-batches of instances. The <codeclass="docutils literal"><spanclass="pre">ie.input(true,</span><spanclass="pre">0)</span></code> invocation returns instances in the 0-th input, <codeclass="docutils literal"><spanclass="pre">x</span></code>, that corresponds to true values in <codeclass="docutils literal"><spanclass="pre">cond</span></code> as the local variable <codeclass="docutils literal"><spanclass="pre">x</span></code>, where <codeclass="docutils literal"><spanclass="pre">ie.input(false,</span><spanclass="pre">0)</span></code> returns instances corresponding to false values.</p>
<p>A difference is that variables in the C++ program contain scalar values, whereas those in the PaddlePaddle programs are mini-batches of instances. The <codeclass="docutils literal"><spanclass="pre">ie.input(true,</span><spanclass="pre">0)</span></code> invocation returns instances in the 0-th input, <codeclass="docutils literal"><spanclass="pre">x</span></code>, that corresponds to true values in <codeclass="docutils literal"><spanclass="pre">cond</span></code> as the local variable <codeclass="docutils literal"><spanclass="pre">x</span></code>, where <codeclass="docutils literal"><spanclass="pre">ie.input(false,</span><spanclass="pre">0)</span></code> returns instances corresponding to false values.</p>
<spanid="blocks-with-for-and-rnnop"></span><h3>Blocks with <codeclass="docutils literal"><spanclass="pre">for</span></code> and <codeclass="docutils literal"><spanclass="pre">RNNOp</span></code><aclass="headerlink"href="#blocks-with-for-and-rnnop"title="Permalink to this headline">¶</a></h3>
<spanid="blocks-with-for-and-rnnop"></span><h3>Blocks with <codeclass="docutils literal"><spanclass="pre">for</span></code> and <codeclass="docutils literal"><spanclass="pre">RNNOp</span></code><aclass="headerlink"href="#blocks-with-for-and-rnnop"title="Permalink to this headline">¶</a></h3>
<p>The following RNN model from the <aclass="reference external"href="design/rnn.md">RNN design doc</a></p>
<p>The following RNN model from the <aclass="reference external"href="design/rnn.md">RNN design doc</a></p>
<spanclass="n">h</span><spanclass="o">.</span><spanclass="n">update</span><spanclass="p">(</span><spanclass="n">act</span><spanclass="p">)</span><spanclass="c1"># update memory with act</span>
<spanclass="n">net</span><spanclass="o">.</span><spanclass="n">set_outputs</span><spanclass="p">(</span><spanclass="mi">0</span><spanclass="p">,</span><spanclass="n">act</span><spanclass="p">,</span><spanclass="n">hidden_out</span><spanclass="p">)</span><spanclass="c1"># two outputs</span>
@@ -55,17 +55,23 @@ Let us consolidate the discussion by presenting some examples.
...
@@ -55,17 +55,23 @@ Let us consolidate the discussion by presenting some examples.
The following C++ programs shows how blocks are used with the `if-else` structure:
The following C++ programs shows how blocks are used with the `if-else` structure:
```c++
```c++
namespace pd = paddle;
int x = 10;
int x = 10;
int y = 20;
int y = 1;
int out;
int z = 10;
bool cond = false;
bool cond = false;
int o1, o2;
if (cond) {
if (cond) {
int z = x + y;
int z = x + y;
out = softmax(z);
o1 = z;
o2 = pd::layer::softmax(z);
} else {
} else {
int z = fc(x);
int d = pd::layer::fc(z);
out = z;
o1 = d;
o2 = d+1;
}
}
```
```
An equivalent PaddlePaddle program from the design doc of the [IfElseOp operator](./if_else_op.md) is as follows:
An equivalent PaddlePaddle program from the design doc of the [IfElseOp operator](./if_else_op.md) is as follows:
...
@@ -73,57 +79,55 @@ An equivalent PaddlePaddle program from the design doc of the [IfElseOp operator
...
@@ -73,57 +79,55 @@ An equivalent PaddlePaddle program from the design doc of the [IfElseOp operator
```python
```python
import paddle as pd
import paddle as pd
x = var(10)
x = minibatch([10, 20, 30]) # shape=[None, 1]
y = var(20)
y = var(1) # shape=[1], value=1
cond = var(false)
z = minibatch([10, 20, 30]) # shape=[None, 1]
ie = pd.create_ifelseop(inputs=[x], output_num=1)
cond = larger_than(x, 15) # [false, true, true]
ie = pd.ifelse()
with ie.true_block():
with ie.true_block():
x = ie.inputs(true, 0)
d = pd.layer.add_scalar(x, y)
z = operator.add(x, y)
ie.output(d, pd.layer.softmax(d))
ie.set_output(true, 0, operator.softmax(z))
with ie.false_block():
with ie.false_block():
x = ie.inputs(false, 0)
d = pd.layer.fc(z)
z = layer.fc(x)
ie.output(d, d+1)
ie.set_output(true, 0, operator.softmax(z))
o1, o2 = ie(cond)
out = b(cond)
```
```
In both examples, the left branch computes `softmax(x+y)` and the right branch computes `fc(x)`.
In both examples, the left branch computes `x+y` and `softmax(x+y)`, the right branch computes `x+1` and `fc(x)`.
A difference is that variables in the C++ program contain scalar values, whereas those in the PaddlePaddle programs are mini-batches of instances. The `ie.input(true, 0)` invocation returns instances in the 0-th input, `x`, that corresponds to true values in `cond` as the local variable `x`, where `ie.input(false, 0)` returns instances corresponding to false values.
A difference is that variables in the C++ program contain scalar values, whereas those in the PaddlePaddle programs are mini-batches of instances. The `ie.input(true, 0)` invocation returns instances in the 0-th input, `x`, that corresponds to true values in `cond` as the local variable `x`, where `ie.input(false, 0)` returns instances corresponding to false values.
### Blocks with `for` and `RNNOp`
### Blocks with `for` and `RNNOp`
The following RNN model from the [RNN design doc](./rnn.md)
The following RNN model from the [RNN design doc](./rnn.md)
```python
```python
x = sequence([10, 20, 30])
x = sequence([10, 20, 30]) # shape=[None, 1]
m = var(0)
m = var(0) # shape=[1]
W = tensor()
W = var(0.314, param=true) # shape=[1]
U = tensor()
U = var(0.375, param=true) # shape=[1]
rnn = create_rnn(inputs=[input])
rnn = pd.rnn()
with rnn.stepnet() as net:
with rnn.step():
x = net.set_inputs(0)
h = rnn.memory(init = m)
h = net.add_memory(init=m)
hh = rnn.previous_memory(h)
fc_out = pd.matmul(W, x)
a = layer.fc(W, x)
hidden_out = pd.matmul(U, h.pre(n=1))
b = layer.fc(U, hh)
sum = pd.add_two(fc_out, hidden_out)
s = pd.add(a, b)
act = pd.sigmoid(sum)
act = pd.sigmoid(s)
h.update(act) # update memory with act
rnn.update_memory(h, act)
net.set_outputs(0, act, hidden_out) # two outputs
rnn.output(a, b)
o1, o2 = rnn()
o1, o2 = rnn()
print o1, o2
```
```
has its equivalent C++ program as follows
has its equivalent C++ program as follows
```c++
```c++
int* x = {10, 20, 30};
int* x = {10, 20, 30};
int m = 0;
int* m = {0};
int W = some_value();
int* W = {0.314};
int U = some_other_value();
int* U = {0.375};
int mem[sizeof(x) / sizeof(x[0]) + 1];
int mem[sizeof(x) / sizeof(x[0]) + 1];
int o1[sizeof(x) / sizeof(x[0]) + 1];
int o1[sizeof(x) / sizeof(x[0]) + 1];
...
@@ -131,20 +135,16 @@ int o2[sizeof(x) / sizeof(x[0]) + 1];
...
@@ -131,20 +135,16 @@ int o2[sizeof(x) / sizeof(x[0]) + 1];
for (int i = 1; i <= sizeof(x)/sizeof(x[0]); ++i) {
for (int i = 1; i <= sizeof(x)/sizeof(x[0]); ++i) {
int x = x[i-1];
int x = x[i-1];
if (i == 1) mem[0] = m;
if (i == 1) mem[0] = m;
int fc_out = W * x;
int a = W * x;
int hidden_out = Y * mem[i-1];
int b = Y * mem[i-1];
int sum = fc_out + hidden_out;
int s = fc_out + hidden_out;
int act = sigmoid(sum);
int act = sigmoid(sum);
mem[i] = act;
mem[i] = act;
o1[i] = act;
o1[i] = act;
o2[i] = hidden_out;
o2[i] = hidden_out;
}
}
print_array(o1);
print_array(o2);
```
```
## Compilation and Execution
## Compilation and Execution
Like TensorFlow programs, a PaddlePaddle program is written in Python. The first part describes a neural network as a protobuf message, and the rest part executes the message for training or inference.
Like TensorFlow programs, a PaddlePaddle program is written in Python. The first part describes a neural network as a protobuf message, and the rest part executes the message for training or inference.
...
@@ -210,11 +210,11 @@ a = pd.Varaible(shape=[20, 20])
...
@@ -210,11 +210,11 @@ a = pd.Varaible(shape=[20, 20])
<spanid="blocks-with-if-else-and-ifelseop"></span><h3>Blocks with <codeclass="docutils literal"><spanclass="pre">if-else</span></code> and <codeclass="docutils literal"><spanclass="pre">IfElseOp</span></code><aclass="headerlink"href="#blocks-with-if-else-and-ifelseop"title="永久链接至标题">¶</a></h3>
<spanid="blocks-with-if-else-and-ifelseop"></span><h3>Blocks with <codeclass="docutils literal"><spanclass="pre">if-else</span></code> and <codeclass="docutils literal"><spanclass="pre">IfElseOp</span></code><aclass="headerlink"href="#blocks-with-if-else-and-ifelseop"title="永久链接至标题">¶</a></h3>
<p>The following C++ programs shows how blocks are used with the <codeclass="docutils literal"><spanclass="pre">if-else</span></code> structure:</p>
<p>The following C++ programs shows how blocks are used with the <codeclass="docutils literal"><spanclass="pre">if-else</span></code> structure:</p>
<p>An equivalent PaddlePaddle program from the design doc of the <aclass="reference internal"href="if_else_op.html"><spanclass="doc">IfElseOp operator</span></a> is as follows:</p>
<p>An equivalent PaddlePaddle program from the design doc of the <aclass="reference internal"href="if_else_op.html"><spanclass="doc">IfElseOp operator</span></a> is as follows:</p>
<p>In both examples, the left branch computes <codeclass="docutils literal"><spanclass="pre">softmax(x+y)</span></code> and the right branch computes<codeclass="docutils literal"><spanclass="pre">fc(x)</span></code>.</p>
<p>In both examples, the left branch computes <codeclass="docutils literal"><spanclass="pre">x+y</span></code> and <codeclass="docutils literal"><spanclass="pre">softmax(x+y)</span></code>, the right branch computes <codeclass="docutils literal"><spanclass="pre">x+1</span></code> and<codeclass="docutils literal"><spanclass="pre">fc(x)</span></code>.</p>
<p>A difference is that variables in the C++ program contain scalar values, whereas those in the PaddlePaddle programs are mini-batches of instances. The <codeclass="docutils literal"><spanclass="pre">ie.input(true,</span><spanclass="pre">0)</span></code> invocation returns instances in the 0-th input, <codeclass="docutils literal"><spanclass="pre">x</span></code>, that corresponds to true values in <codeclass="docutils literal"><spanclass="pre">cond</span></code> as the local variable <codeclass="docutils literal"><spanclass="pre">x</span></code>, where <codeclass="docutils literal"><spanclass="pre">ie.input(false,</span><spanclass="pre">0)</span></code> returns instances corresponding to false values.</p>
<p>A difference is that variables in the C++ program contain scalar values, whereas those in the PaddlePaddle programs are mini-batches of instances. The <codeclass="docutils literal"><spanclass="pre">ie.input(true,</span><spanclass="pre">0)</span></code> invocation returns instances in the 0-th input, <codeclass="docutils literal"><spanclass="pre">x</span></code>, that corresponds to true values in <codeclass="docutils literal"><spanclass="pre">cond</span></code> as the local variable <codeclass="docutils literal"><spanclass="pre">x</span></code>, where <codeclass="docutils literal"><spanclass="pre">ie.input(false,</span><spanclass="pre">0)</span></code> returns instances corresponding to false values.</p>
<spanid="blocks-with-for-and-rnnop"></span><h3>Blocks with <codeclass="docutils literal"><spanclass="pre">for</span></code> and <codeclass="docutils literal"><spanclass="pre">RNNOp</span></code><aclass="headerlink"href="#blocks-with-for-and-rnnop"title="永久链接至标题">¶</a></h3>
<spanid="blocks-with-for-and-rnnop"></span><h3>Blocks with <codeclass="docutils literal"><spanclass="pre">for</span></code> and <codeclass="docutils literal"><spanclass="pre">RNNOp</span></code><aclass="headerlink"href="#blocks-with-for-and-rnnop"title="永久链接至标题">¶</a></h3>
<p>The following RNN model from the <aclass="reference external"href="design/rnn.md">RNN design doc</a></p>
<p>The following RNN model from the <aclass="reference external"href="design/rnn.md">RNN design doc</a></p>
<spanclass="n">h</span><spanclass="o">.</span><spanclass="n">update</span><spanclass="p">(</span><spanclass="n">act</span><spanclass="p">)</span><spanclass="c1"># update memory with act</span>
<spanclass="n">net</span><spanclass="o">.</span><spanclass="n">set_outputs</span><spanclass="p">(</span><spanclass="mi">0</span><spanclass="p">,</span><spanclass="n">act</span><spanclass="p">,</span><spanclass="n">hidden_out</span><spanclass="p">)</span><spanclass="c1"># two outputs</span>
