未验证 提交 2edeb639 编写于 作者: Y Yang Yang(Tony) 提交者: GitHub

Update parallel_do.md (#8665)

上级 718642e9
...@@ -39,15 +39,16 @@ In the backward pass ...@@ -39,15 +39,16 @@ In the backward pass
This implementation allows to write mixed device program like this This implementation allows to write mixed device program like this
```python ```python
# get embedding feature on CPU W1 = fluid.tensor(size=[100,20], parameter=true)
feature = some_cpu_only_op(data) W2 = fluid.tensor(size=[20,15], parameter=true)
gpu_places = get_place(use_gpu=True) data = layers.data()
gpu_places = layers.get_place(use_gpu=True)
# parallel processing on multiple GPUs # parallel processing on multiple GPUs
pd = ParallelDo(gpu_places) pd = ParallelDo(gpu_places)
with pd.do(): with pd.do(input=data):
read_input(feature) prediction = softmax(fc(fc(data, W1), W2))
prediction = my_net(feature)
write_output(prediction) write_output(prediction)
prediction = pd() prediction = pd()
loss = cross_entropy(prediction, label) loss = cross_entropy(prediction, label)
...@@ -66,20 +67,20 @@ start_program ...@@ -66,20 +67,20 @@ start_program
main_program main_program
{ {
block0 { block0 {
vars: data, places, w1, w2 vars: data, places, w1, w2, w1_grad, w2_grad,
ops: data, get_place, parallel_do(block1), ops: data, get_place, parallel_do(block1),
parallel_do_grad(block2), parallel_do_grad(block2),
sgd(w2, w2_grad), sgd(w2, w2_grad),
sgd(w1, w1_grad) sgd(w1, w1_grad)
} }
block1 { block1 { # the forward pass
parent_block: 0 parent_block: 0
vars: data, h1, h2, loss vars: data, h1, h2, loss
ops: fc, fc, softmax ops: fc, fc, softmax
} }
block2 { block2 { # the backward pass
parent_block: 1 parent_block: 1
vars: data_grad, h1_grad, h2_grad, loss_gard, w1_grad, w2_grad vars: data_grad, h1_grad, h2_grad, loss_gard, local_w1_grad, local_w2_grad
ops: softmax_grad, ops: softmax_grad,
fc_grad fc_grad
fc_grad fc_grad
......
Markdown is supported
0% .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑!
想要评论请 注册