提交 2a058b10 编写于 作者: Y yangyaming

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into fix-6581

# Executor Design Doc
## Motivation
In the [fluid](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/fluid.md), we encourage user use deep learning programming paradigms to describe training process. When the user-written Python program is executed, it will create a protobuf message
In [fluid](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/fluid.md), we encourage the user to use deep learning programming paradigms to describe the training process. When the user-written Python program is executed, it will first create a protobuf message
[`ProgramDesc`](https://github.com/PaddlePaddle/Paddle/blob/a91efdde6910ce92a78e3aa7157412c4c88d9ee8/paddle/framework/framework.proto#L145) that describes the process and is conceptually like an [abstract syntax tree](https://en.wikipedia.org/wiki/Abstract_syntax_tree).
The executor runs the `ProgramDesc` like an interpreter. `ProgramDesc` contains intrinsics/operators and variables which will be used, executor explicitly execute the stored precompiled code.
The executor runs the `ProgramDesc` like an interpreter. `ProgramDesc` contains the intrinsics (operators in this case) and variables which will be used, executor explicitly executes the stored precompiled code.
## Overview
An executor takes a `ProgramDesc`, a `block_id` and a `Scope`. The `ProgramDesc` is a list of blocks and each block contains the protobuf definition of all the parameters and operators. The `block_id` specifies the entrance block. And the `Scope` is the container of all the variable instance, which is persistent throughout different runs.
An executor takes a `ProgramDesc`, a `block_id` and a `Scope`. The `ProgramDesc` is a list of blocks and each block contains the protobuf definition of all the parameters and operators in the block. The `block_id` specifies the entrance block. And the `Scope` is the container of all the variable instances, which is persistent throughout different runs.
## Executor
`Executor` explicitly executes all the intrinsics/operators in the `block_id`th block of a `ProgramDesc`. Essentially, it instantiates Variables and Operators, then runs all the operators in sequence. It is very similar to push stack frame when entering the block, it will destroy the temporary variables when mini-batch is finished, but it does not have stack frame pop process.
The `Executor` explicitly executes all the intrinsics (operators here) in the `block_id`th block of a `ProgramDesc`. Essentially, it instantiates Variables and Operators, then runs all the operators in sequence one-by-one.
It is very similar to how a push stack frame works when entering a block, following which it cleans up all the temporary variables when a mini-batch is finished. It does not however, have the stack frame pop process.
### Interface
### The interface
```c++
Executor(places);
```
A executor does not own any computing resources, user can only construct an executor with specified places.
A executor does not own any computing resources, a user can only construct an executor using the specified places.
### Running an Executor
```
void Run(ProgramDesc, Scope, block_id, create_local_scope);
```
A executor only provides an unified way to execute `ProgramDesc`. `ProgramDesc` is the target will be executed, scope specifies the variable container. `block_id` indicates the entrance block, `create_local_scope` means if it will destroy the temporary variables after execution finished.
An `Executor` only provides a unified way to execute `ProgramDesc`. `ProgramDesc` is the target that will be executed, the `Scope` specifies the variable container, the `block_id` indicates the entrance block and `create_local_scope` is a boolean that states whether it will destroy the temporary variables after the execution is finished.
import numpy as np
import layers
from framework import Program, unique_name, Variable
from framework import Program, unique_name, Variable, program_guard
from layer_helper import LayerHelper
__all__ = ['Accuracy', 'ChunkEvaluator']
......@@ -49,15 +49,12 @@ class Evaluator(object):
if reset_program is None:
reset_program = Program()
for var in self.states:
assert isinstance(var, Variable)
g_var = _clone_var_(reset_program.current_block(), var)
layers.fill_constant(
shape=g_var.shape,
value=0.0,
dtype=g_var.dtype,
out=g_var,
main_program=reset_program)
with program_guard(main_program=reset_program):
for var in self.states:
assert isinstance(var, Variable)
g_var = _clone_var_(reset_program.current_block(), var)
layers.fill_constant(
shape=g_var.shape, value=0.0, dtype=g_var.dtype, out=g_var)
executor.run(reset_program)
......@@ -104,20 +101,14 @@ class Accuracy(Evaluator):
self.total = self.create_state(dtype='int64', shape=[1], suffix='total')
self.correct = self.create_state(
dtype='int64', shape=[1], suffix='correct')
kwargs = {'main_program': main_program}
total = self.helper.create_tmp_variable(dtype='int')
correct = self.helper.create_tmp_variable(dtype='int')
acc = layers.accuracy(
input=input,
label=label,
k=k,
total=total,
correct=correct,
**kwargs)
total = layers.cast(x=total, dtype='int64', **kwargs)
correct = layers.cast(x=correct, dtype='int64', **kwargs)
layers.sums(input=[self.total, total], out=self.total, **kwargs)
layers.sums(input=[self.correct, correct], out=self.correct, **kwargs)
input=input, label=label, k=k, total=total, correct=correct)
total = layers.cast(x=total, dtype='int64')
correct = layers.cast(x=correct, dtype='int64')
layers.sums(input=[self.total, total], out=self.total)
layers.sums(input=[self.correct, correct], out=self.correct)
self.metrics.append(acc)
......@@ -125,12 +116,12 @@ class Accuracy(Evaluator):
if eval_program is None:
eval_program = Program()
block = eval_program.current_block()
kwargs = {'main_program': eval_program}
total = _clone_var_(block, self.total)
correct = _clone_var_(block, self.correct)
total = layers.cast(total, dtype='float32', **kwargs)
correct = layers.cast(correct, dtype='float32', **kwargs)
out = layers.elementwise_div(x=correct, y=total, **kwargs)
with program_guard(main_program=eval_program):
total = _clone_var_(block, self.total)
correct = _clone_var_(block, self.correct)
total = layers.cast(total, dtype='float32')
correct = layers.cast(correct, dtype='float32')
out = layers.elementwise_div(x=correct, y=total)
return np.array(executor.run(eval_program, fetch_list=[out])[0])
......@@ -141,14 +132,14 @@ class ChunkEvaluator(Evaluator):
numbers.
"""
def __init__(self,
input,
label,
chunk_scheme,
num_chunk_types,
excluded_chunk_types=None,
**kwargs):
super(ChunkEvaluator, self).__init__("chunk_eval", **kwargs)
def __init__(
self,
input,
label,
chunk_scheme,
num_chunk_types,
excluded_chunk_types=None, ):
super(ChunkEvaluator, self).__init__("chunk_eval")
main_program = self.helper.main_program
if main_program.current_block().idx != 0:
raise ValueError("You can only invoke Evaluator in root block")
......@@ -159,26 +150,21 @@ class ChunkEvaluator(Evaluator):
dtype='int64', shape=[1], suffix='num_label_chunks')
self.num_correct_chunks = self.create_state(
dtype='int64', shape=[1], suffix='num_correct_chunks')
kwargs = {'main_program': main_program}
precision, recall, f1_score, num_infer_chunks, num_label_chunks, num_correct_chunks = layers.chunk_eval(
input=input,
label=label,
chunk_scheme=chunk_scheme,
num_chunk_types=num_chunk_types,
excluded_chunk_types=excluded_chunk_types,
**kwargs)
excluded_chunk_types=excluded_chunk_types, )
layers.sums(
input=[self.num_infer_chunks, num_infer_chunks],
out=self.num_infer_chunks,
**kwargs)
out=self.num_infer_chunks)
layers.sums(
input=[self.num_label_chunks, num_label_chunks],
out=self.num_label_chunks,
**kwargs)
out=self.num_label_chunks)
layers.sums(
input=[self.num_correct_chunks, num_correct_chunks],
out=self.num_correct_chunks,
**kwargs)
out=self.num_correct_chunks)
self.metrics.extend([precision, recall, f1_score])
......@@ -186,7 +172,6 @@ class ChunkEvaluator(Evaluator):
if eval_program is None:
eval_program = Program()
block = eval_program.current_block()
kwargs = {'main_program': eval_program}
num_infer_chunks, num_label_chunks, num_correct_chunks = executor.run(
eval_program,
fetch_list=[_clone_var_(block, state) for state in self.states])
......
......@@ -21,19 +21,11 @@ class LayerHelper(object):
@property
def main_program(self):
prog = self.kwargs.get('main_program', None)
if prog is None:
return default_main_program()
else:
return prog
return default_main_program()
@property
def startup_program(self):
prog = self.kwargs.get('startup_program', None)
if prog is None:
return default_startup_program()
else:
return prog
return default_startup_program()
def append_op(self, *args, **kwargs):
return self.main_program.current_block().append_op(*args, **kwargs)
......@@ -151,13 +143,6 @@ class LayerHelper(object):
persistable=True,
initializer=initializer)
@property
def to_kwargs(self):
return {
'main_program': self.main_program,
'startup_program': self.startup_program
}
def append_bias_op(self, input_var, dim_start=1, dim_end=None):
"""
Append bias operator and return its output. If the user does not set
......
......@@ -14,11 +14,7 @@ __all__ = [
]
def split_lod_tensor(input,
mask,
level=0,
main_program=None,
startup_program=None):
def split_lod_tensor(input, mask, level=0):
helper = LayerHelper('split_lod_tensor', **locals())
out_true = helper.create_tmp_variable(dtype=input.dtype)
out_false = helper.create_tmp_variable(dtype=input.dtype)
......@@ -34,13 +30,7 @@ def split_lod_tensor(input,
return out_true, out_false
def merge_lod_tensor(in_true,
in_false,
x,
mask,
level=0,
main_program=None,
startup_program=None):
def merge_lod_tensor(in_true, in_false, x, mask, level=0):
helper = LayerHelper('merge_lod_tensor', **locals())
out = helper.create_tmp_variable(dtype=in_true.dtype)
helper.append_op(
......@@ -135,9 +125,8 @@ class StaticRNN(object):
IN_RNN_BLOCK = 1
AFTER_RNN_BLOCK = 2
def __init__(self, name=None, main_program=None):
self.helper = LayerHelper(
"static_rnn", name=name, main_program=main_program)
def __init__(self, name=None):
self.helper = LayerHelper("static_rnn", name=name)
self.memories = {} # memory map, from pre_mem.name --> MemoryLink
self.inputs = [] # input variable list in current block
self.outputs = [] # output variable list in parent block
......@@ -354,8 +343,8 @@ class While(object):
IN_WHILE_BLOCK = 1
AFTER_WHILE_BLOCK = 2
def __init__(self, cond, name=None, main_program=None):
self.helper = LayerHelper("while", name=name, main_program=main_program)
def __init__(self, cond, name=None):
self.helper = LayerHelper("while", name=name)
self.status = While.BEFORE_WHILE_BLOCK
if not isinstance(cond, Variable):
raise TypeError("condition should be a variable")
......@@ -406,7 +395,7 @@ class While(object):
attrs={'sub_block': while_block})
def lod_rank_table(x, level=0, main_program=None):
def lod_rank_table(x, level=0):
"""
This function creates an operator for creating a LOD_RANK_TABLE
using the input x.
......@@ -423,7 +412,7 @@ def lod_rank_table(x, level=0, main_program=None):
return table
def max_sequence_len(rank_table, main_program=None):
def max_sequence_len(rank_table):
"""
This function creates an operator to calculate the length of
max seqence through input rank_table(should be a lod_rank_table)
......@@ -437,7 +426,7 @@ def max_sequence_len(rank_table, main_program=None):
return res
def topk(input, k, main_program=None, startup_program=None):
def topk(input, k):
helper = LayerHelper('topk', **locals())
topk_out = helper.create_tmp_variable(dtype=input.data_type)
topk_indices = helper.create_tmp_variable(dtype='int64')
......@@ -450,7 +439,7 @@ def topk(input, k, main_program=None, startup_program=None):
return topk_out, topk_indices
def lod_tensor_to_array(x, table, main_program=None):
def lod_tensor_to_array(x, table):
"""
This function creates an operator to convert an LOD_Tensor to
an array.
......@@ -468,7 +457,7 @@ def lod_tensor_to_array(x, table, main_program=None):
return array
def array_to_lod_tensor(x, table, main_program=None, startup_program=None):
def array_to_lod_tensor(x, table):
"""
This function creates an operator to convert an array to a
LOD_Tensor.
......@@ -483,11 +472,7 @@ def array_to_lod_tensor(x, table, main_program=None, startup_program=None):
return tmp
def increment(x,
value=1.0,
in_place=True,
main_program=None,
startup_program=None):
def increment(x, value=1.0, in_place=True):
"""
This function creates an operator to increment each value in the input
`x` by an amount: `value` as mentioned in the input parameter. This
......@@ -506,7 +491,7 @@ def increment(x,
return out
def array_write(x, i, array=None, main_program=None, startup_program=None):
def array_write(x, i, array=None):
"""
This function creates an operator to write the data out as a
LOD_TENSOR_ARRAY.
......@@ -525,7 +510,7 @@ def array_write(x, i, array=None, main_program=None, startup_program=None):
return array
def create_array(dtype, main_program=None):
def create_array(dtype):
helper = LayerHelper("array", **locals())
return helper.create_variable(
name="{0}.out".format(helper.name),
......@@ -533,7 +518,7 @@ def create_array(dtype, main_program=None):
dtype=dtype)
def less_than(x, y, cond=None, main_program=None, **ignored):
def less_than(x, y, cond=None, **ignored):
helper = LayerHelper("less_than", **locals())
if cond is None:
cond = helper.create_tmp_variable(dtype='bool')
......@@ -545,7 +530,7 @@ def less_than(x, y, cond=None, main_program=None, **ignored):
return cond
def array_read(array, i, main_program=None, startup_program=None):
def array_read(array, i):
"""
This function creates an operator to read the data in as a
LOD_TENSOR_ARRAY.
......@@ -564,7 +549,7 @@ def array_read(array, i, main_program=None, startup_program=None):
return out
def shrink_memory(x, i, table, main_program=None, startup_program=None):
def shrink_memory(x, i, table):
"""
This function creates an operator to shrink_rnn_memory using the RankTable
as mentioned in the input parameter.
......@@ -581,7 +566,7 @@ def shrink_memory(x, i, table, main_program=None, startup_program=None):
return out
def array_length(array, main_program=None):
def array_length(array):
"""
This function creates an operator to find the length of the
LOD_TENSOR_ARRAY.
......@@ -611,20 +596,12 @@ class ConditionalBlockGuard(BlockGuard):
class ConditionalBlock(object):
def __init__(self,
inputs,
name=None,
main_program=None,
startup_program=None):
def __init__(self, inputs, name=None):
for each_input in inputs:
if not isinstance(each_input, Variable):
raise TypeError("Each input should be variable")
self.inputs = inputs
self.helper = LayerHelper(
'conditional_block',
name=name,
main_program=main_program,
startup_program=startup_program)
self.helper = LayerHelper('conditional_block', name=name)
def block(self):
return ConditionalBlockGuard(self)
......@@ -709,15 +686,10 @@ class IfElse(object):
IN_IF_ELSE_TRUE_BLOCKS = 1
IN_IF_ELSE_FALSE_BLOCKS = 2
def __init__(self, cond, name=None, main_program=None,
startup_program=None):
def __init__(self, cond, name=None):
if not isinstance(cond, Variable):
raise TypeError("cond must be a Variable")
self.helper = LayerHelper(
'ifelse',
name=name,
main_program=main_program,
startup_program=startup_program)
self.helper = LayerHelper('ifelse', name=name)
self.cond = cond
self.input_table = {}
self.status = IfElse.OUT_IF_ELSE_BLOCKS
......@@ -782,11 +754,7 @@ class IfElse(object):
out_table.append(outside_out)
# assign local var to outside
assign(
input=each_out,
output=outside_out,
main_program=self.helper.main_program,
startup_program=self.helper.startup_program)
assign(input=each_out, output=outside_out)
def __call__(self):
if self.status != self.OUT_IF_ELSE_BLOCKS:
......@@ -810,9 +778,7 @@ class IfElse(object):
in_false=false_var,
mask=self.cond,
x=self.cond,
level=0,
main_program=self.helper.main_program,
startup_program=self.helper.startup_program))
level=0))
return rlist
......@@ -821,12 +787,8 @@ class DynamicRNN(object):
IN_RNN = 1
AFTER_RNN = 2
def __init__(self, name=None, main_program=None, startup_program=None):
self.helper = LayerHelper(
'dynamic_rnn',
name=name,
main_program=main_program,
startup_program=startup_program)
def __init__(self, name=None):
self.helper = LayerHelper('dynamic_rnn', name=name)
self.status = DynamicRNN.BEFORE_RNN
self.lod_rank_table = None
self.max_seq_len = None
......@@ -880,8 +842,7 @@ class DynamicRNN(object):
inputs={'X': x,
'RankTable': self.lod_rank_table},
outputs={'Out': input_array})
return array_read(
array=input_array, i=self.step_idx, **self.helper.to_kwargs)
return array_read(array=input_array, i=self.step_idx)
@contextlib.contextmanager
def block(self):
......@@ -892,32 +853,18 @@ class DynamicRNN(object):
self.status = DynamicRNN.IN_RNN
with self.while_op.block():
yield
increment(
x=self.step_idx,
value=1.0,
in_place=True,
**self.helper.to_kwargs)
increment(x=self.step_idx, value=1.0, in_place=True)
for new_mem, mem_array in self.mem_link:
array_write(
x=new_mem,
i=self.step_idx,
array=mem_array,
**self.helper.to_kwargs)
less_than(
x=self.step_idx,
y=self.max_seq_len,
cond=self.cond,
**self.helper.to_kwargs)
array_write(x=new_mem, i=self.step_idx, array=mem_array)
less_than(x=self.step_idx, y=self.max_seq_len, cond=self.cond)
self.status = DynamicRNN.AFTER_RNN
for each_array in self.output_array:
self.outputs.append(
array_to_lod_tensor(
x=each_array,
table=self.lod_rank_table,
**self.helper.to_kwargs))
x=each_array, table=self.lod_rank_table))
def __call__(self, *args, **kwargs):
if self.status != DynamicRNN.AFTER_RNN:
......@@ -944,13 +891,9 @@ class DynamicRNN(object):
inputs={'X': init,
'I': self.zero_idx},
outputs={'Out': mem_array})
retv = array_read(
array=mem_array, i=self.step_idx, **self.helper.to_kwargs)
retv = array_read(array=mem_array, i=self.step_idx)
retv = shrink_memory(
x=retv,
i=self.step_idx,
table=self.lod_rank_table,
**self.helper.to_kwargs)
x=retv, i=self.step_idx, table=self.lod_rank_table)
self.mem_dict[retv.name] = mem_array
return retv
else:
......
......@@ -10,8 +10,6 @@ def data(name,
dtype='float32',
lod_level=0,
type=core.VarDesc.VarType.LOD_TENSOR,
main_program=None,
startup_program=None,
stop_gradient=True):
"""
Data Layer.
......
......@@ -23,9 +23,7 @@ def fc(input,
param_attr=None,
bias_attr=None,
act=None,
name=None,
main_program=None,
startup_program=None):
name=None):
"""
Fully Connected Layer.
......@@ -91,13 +89,7 @@ def fc(input,
return helper.append_activation(pre_activation)
def embedding(input,
size,
is_sparse=False,
param_attr=None,
dtype='float32',
main_program=None,
startup_program=None):
def embedding(input, size, is_sparse=False, param_attr=None, dtype='float32'):
"""
Embedding Layer.
......@@ -143,9 +135,7 @@ def dynamic_lstm(input,
gate_activation='sigmoid',
cell_activation='tanh',
candidate_activation='tanh',
dtype='float32',
main_program=None,
startup_program=None):
dtype='float32'):
helper = LayerHelper('lstm', **locals())
size = size / 4
weight = helper.create_parameter(
......@@ -188,9 +178,7 @@ def gru_unit(input,
weight=None,
bias=None,
activation='tanh',
gate_activation='sigmoid',
main_program=None,
startup_program=None):
gate_activation='sigmoid'):
"""
GRUUnit Operator implements partial calculations of the GRU unit as following:
......@@ -253,11 +241,7 @@ def gru_unit(input,
return updated_hidden, reset_hidden_pre, gate
def linear_chain_crf(input,
label,
param_attr=None,
main_program=None,
startup_program=None):
def linear_chain_crf(input, label, param_attr=None):
helper = LayerHelper('linear_chain_crf', **locals())
size = input.shape[1]
transition = helper.create_parameter(
......@@ -283,11 +267,7 @@ def linear_chain_crf(input,
return log_likelihood
def crf_decoding(input,
param_attr,
label=None,
main_program=None,
startup_program=None):
def crf_decoding(input, param_attr, label=None):
helper = LayerHelper('crf_decoding', **locals())
transition = helper.get_parameter(param_attr.name)
viterbi_path = helper.create_tmp_variable(dtype=helper.input_dtype())
......@@ -435,9 +415,7 @@ def sequence_conv(input,
padding=None,
bias_attr=None,
param_attr=None,
act=None,
main_program=None,
startup_program=None):
act=None):
"""
This function creates the op for sequence_conv, using the inputs and
other convolutional configurations for the filters and stride as given
......@@ -480,9 +458,7 @@ def conv2d(input,
param_attr=None,
bias_attr=None,
act=None,
name=None,
main_program=None,
startup_program=None):
name=None):
"""
This function creates the op for a 2-dimensional Convolution.
This is performed using the parameters of filters(size, dimensionality etc)
......@@ -568,9 +544,7 @@ def pool2d(input,
pool_type,
pool_stride=None,
pool_padding=None,
global_pooling=False,
main_program=None,
startup_program=None):
global_pooling=False):
"""
This function adds the operator for pooling in 2 dimensions, using the
pooling configurations mentioned in input parameters.
......@@ -616,9 +590,7 @@ def batch_norm(input,
epsilon=1e-05,
param_attr=None,
bias_attr=None,
data_layout='NCHW',
main_program=None,
startup_program=None):
data_layout='NCHW'):
"""
This function helps create an operator to implement
the BatchNorm layer using the configurations from the input parameters.
......@@ -688,7 +660,7 @@ def batch_norm(input,
return helper.append_activation(batch_norm_out)
def beam_search_decode(ids, scores, main_program=None, startup_program=None):
def beam_search_decode(ids, scores):
helper = LayerHelper('beam_search_decode', **locals())
sentence_ids = helper.create_tmp_variable(dtype=ids.dtype)
sentence_scores = helper.create_tmp_variable(dtype=ids.dtype)
......@@ -711,9 +683,7 @@ def conv2d_transpose(input,
filter_size=None,
padding=None,
stride=None,
param_attr=None,
main_program=None,
startup_program=None):
param_attr=None):
"""
The transpose of conv2d layer.
......
......@@ -6,12 +6,12 @@ __all__ = [
]
def create_tensor(dtype, name=None, main_program=None, startup_program=None):
def create_tensor(dtype, name=None):
helper = LayerHelper("create_tensor", **locals())
return helper.create_variable(name=helper.name, dtype=dtype)
def cast(x, dtype, main_program=None):
def cast(x, dtype):
"""
This function takes in the input with input_dtype
and casts it to the output_dtype as the output.
......@@ -27,7 +27,7 @@ def cast(x, dtype, main_program=None):
return out
def concat(input, axis, main_program=None, startup_program=None):
def concat(input, axis):
"""
This function concats the input along the axis mentioned
and returns that as the output.
......@@ -42,7 +42,7 @@ def concat(input, axis, main_program=None, startup_program=None):
return out
def sums(input, out=None, main_program=None, startup_program=None):
def sums(input, out=None):
"""
This function takes in the input and performs the sum operation on it
and returns that as the output.
......@@ -54,7 +54,7 @@ def sums(input, out=None, main_program=None, startup_program=None):
return out
def assign(input, output, main_program=None, startup_program=None):
def assign(input, output):
helper = LayerHelper('assign', **locals())
helper.append_op(
type='scale',
......@@ -64,12 +64,7 @@ def assign(input, output, main_program=None, startup_program=None):
return output
def fill_constant(shape,
dtype,
value,
out=None,
main_program=None,
startup_program=None):
def fill_constant(shape, dtype, value, out=None):
"""
This function creates a tensor , with shape as mentioned in the input and
specified dtype and fills this up with a constant value that
......@@ -94,9 +89,7 @@ def fill_constant_batch_size_like(input,
dtype,
value,
input_dim_idx=0,
output_dim_idx=0,
main_program=None,
startup_program=None):
output_dim_idx=0):
helper = LayerHelper("fill_constant_batch_size_like", **locals())
out = helper.create_tmp_variable(dtype=dtype)
helper.append_op(
......@@ -114,7 +107,7 @@ def fill_constant_batch_size_like(input,
return out
def ones(shape, dtype, main_program=None):
def ones(shape, dtype):
"""
This function performs the same function as fill_constant() declared above
with the constant value being 1.0.
......@@ -122,7 +115,7 @@ def ones(shape, dtype, main_program=None):
return fill_constant(value=1.0, **locals())
def zeros(shape, dtype, main_program=None):
def zeros(shape, dtype):
"""
This function performs the same function as fill_constant() declared above
with the constant value being 0.0.
......
......@@ -10,25 +10,19 @@ def simple_img_conv_pool(input,
pool_stride,
act,
param_attr=None,
pool_type='max',
main_program=None,
startup_program=None):
pool_type='max'):
conv_out = layers.conv2d(
input=input,
num_filters=num_filters,
filter_size=filter_size,
param_attr=param_attr,
act=act,
main_program=main_program,
startup_program=startup_program)
act=act)
pool_out = layers.pool2d(
input=conv_out,
pool_size=pool_size,
pool_type=pool_type,
pool_stride=pool_stride,
main_program=main_program,
startup_program=startup_program)
pool_stride=pool_stride)
return pool_out
......@@ -42,9 +36,7 @@ def img_conv_group(input,
conv_with_batchnorm=False,
conv_batchnorm_drop_rate=None,
pool_stride=1,
pool_type=None,
main_program=None,
startup_program=None):
pool_type=None):
"""
Image Convolution Group, Used for vgg net.
"""
......@@ -75,31 +67,19 @@ def img_conv_group(input,
filter_size=conv_filter_size[i],
padding=conv_padding[i],
param_attr=param_attr[i],
act=local_conv_act,
main_program=main_program,
startup_program=startup_program)
act=local_conv_act)
if conv_with_batchnorm[i]:
tmp = layers.batch_norm(
input=tmp,
act=conv_act,
main_program=main_program,
startup_program=startup_program)
tmp = layers.batch_norm(input=tmp, act=conv_act)
drop_rate = conv_batchnorm_drop_rate[i]
if abs(drop_rate) > 1e-5:
tmp = layers.dropout(
x=tmp,
dropout_prob=drop_rate,
main_program=main_program,
startup_program=startup_program)
tmp = layers.dropout(x=tmp, dropout_prob=drop_rate)
pool_out = layers.pool2d(
input=tmp,
pool_size=pool_size,
pool_type=pool_type,
pool_stride=pool_stride,
main_program=main_program,
startup_program=startup_program)
pool_stride=pool_stride)
return pool_out
......@@ -108,21 +88,13 @@ def sequence_conv_pool(input,
filter_size,
param_attr=None,
act="sigmoid",
pool_type="max",
main_program=None,
startup_program=None):
pool_type="max"):
conv_out = layers.sequence_conv(
input=input,
num_filters=num_filters,
filter_size=filter_size,
param_attr=param_attr,
act=act,
main_program=main_program,
startup_program=startup_program)
act=act)
pool_out = layers.sequence_pool(
input=conv_out,
pool_type=pool_type,
main_program=main_program,
startup_program=startup_program)
pool_out = layers.sequence_pool(input=conv_out, pool_type=pool_type)
return pool_out
......@@ -2,7 +2,7 @@ from collections import defaultdict
import framework
from backward import append_backward_ops
from framework import unique_name
from framework import unique_name, program_guard
from initializer import Constant
from layer_helper import LayerHelper
from regularizer import append_regularization_ops
......@@ -159,34 +159,32 @@ class Optimizer(object):
# Create any accumulators
program = loss.block.program
self.helper = LayerHelper(
self.__class__.__name__,
main_program=program,
startup_program=startup_program)
self._create_accumulators(loss.block,
[p[0] for p in parameters_and_grads])
optimize_ops = []
for param_and_grad in parameters_and_grads:
if param_and_grad[0].trainable is True and param_and_grad[
1] is not None:
optimize_op = self._append_optimize_op(loss.block,
param_and_grad)
optimize_ops.append(optimize_op)
# Returned list of ops can include more ops in addition
# to optimization ops
return_ops = optimize_ops
# Get custom finish ops for subclasses
# FIXME: Need to fix this once we figure out how to handle dependencies
finish_ops = self._finish_update(loss.block)
if finish_ops is not None:
return_ops += finish_ops
if self._global_step is not None:
return_ops.append(self._increment_global_step(loss.block))
return return_ops
with program_guard(program, startup_program):
self.helper = LayerHelper(self.__class__.__name__)
self._create_accumulators(loss.block,
[p[0] for p in parameters_and_grads])
optimize_ops = []
for param_and_grad in parameters_and_grads:
if param_and_grad[0].trainable is True and param_and_grad[
1] is not None:
optimize_op = self._append_optimize_op(loss.block,
param_and_grad)
optimize_ops.append(optimize_op)
# Returned list of ops can include more ops in addition
# to optimization ops
return_ops = optimize_ops
# Get custom finish ops for subclasses
# FIXME: Need to fix this once we figure out how to handle dependencies
finish_ops = self._finish_update(loss.block)
if finish_ops is not None:
return_ops += finish_ops
if self._global_step is not None:
return_ops.append(self._increment_global_step(loss.block))
return return_ops
def minimize(self,
loss,
......
image/
fit_a_line.model/
tmp
cuda_profiler.txt
......@@ -33,11 +33,10 @@ opts = optimizer.minimize(avg_cost)
accuracy = fluid.evaluator.Accuracy(input=predict, label=label)
inference_program = fluid.default_main_program().clone()
test_accuracy = fluid.evaluator.Accuracy(
input=predict, label=label, main_program=inference_program)
test_target = [avg_cost] + test_accuracy.metrics + test_accuracy.states
inference_program = fluid.io.get_inference_program(
test_target, main_program=inference_program)
with fluid.program_guard(inference_program):
test_accuracy = fluid.evaluator.Accuracy(input=predict, label=label)
test_target = [avg_cost] + test_accuracy.metrics + test_accuracy.states
inference_program = fluid.io.get_inference_program(test_target)
train_reader = paddle.batch(
paddle.reader.shuffle(
......
......@@ -4,12 +4,7 @@ import paddle.v2.fluid as fluid
from paddle.v2.fluid.layer_helper import LayerHelper
def lstm(x,
c_pre_init,
hidden_dim,
forget_bias=None,
main_program=None,
startup_program=None):
def lstm(x, c_pre_init, hidden_dim, forget_bias=None):
"""
This function helps create an operator for the LSTM (Long Short Term
Memory) cell that can be used inside an RNN.
......@@ -20,15 +15,8 @@ def lstm(x,
c_pre = rnn.memory(init=c_pre_init)
x_t = rnn.step_input(x)
before_fc = fluid.layers.concat(
input=[x_t, c_pre],
axis=1,
main_program=main_program,
startup_program=startup_program)
after_fc = fluid.layers.fc(input=before_fc,
size=hidden_dim * 4,
main_program=main_program,
startup_program=startup_program)
before_fc = fluid.layers.concat(input=[x_t, c_pre], axis=1)
after_fc = fluid.layers.fc(input=before_fc, size=hidden_dim * 4)
dtype = x.dtype
c = helper.create_tmp_variable(dtype)
......
......@@ -5,12 +5,7 @@ import paddle.v2.fluid.nets as nets
from paddle.v2.fluid.framework import Program
def conv_block(input,
num_filter,
groups,
dropouts,
main_program=None,
startup_program=None):
def conv_block(input, num_filter, groups, dropouts):
return nets.img_conv_group(
input=input,
pool_size=2,
......@@ -20,90 +15,54 @@ def conv_block(input,
conv_act='relu',
conv_with_batchnorm=True,
conv_batchnorm_drop_rate=dropouts,
pool_type='max',
main_program=main_program,
startup_program=startup_program)
pool_type='max')
class TestLayer(unittest.TestCase):
def test_batch_norm_layer(self):
main_program = Program()
startup_program = Program()
images = fluid.layers.data(
name='pixel',
shape=[3, 48, 48],
dtype='float32',
main_program=main_program)
hidden1 = fluid.layers.batch_norm(
input=images,
main_program=main_program,
startup_program=startup_program)
hidden2 = fluid.layers.fc(input=hidden1,
size=128,
act='relu',
main_program=main_program)
hidden3 = fluid.layers.batch_norm(
input=hidden2,
main_program=main_program,
startup_program=startup_program)
with fluid.program_guard(main_program, startup_program):
images = fluid.layers.data(
name='pixel', shape=[3, 48, 48], dtype='float32')
hidden1 = fluid.layers.batch_norm(input=images)
hidden2 = fluid.layers.fc(input=hidden1, size=128, act='relu')
fluid.layers.batch_norm(input=hidden2)
print str(main_program)
def test_dropout_layer(self):
main_program = Program()
startup_program = Program()
images = fluid.layers.data(
name='pixel',
shape=[3, 48, 48],
dtype='float32',
main_program=main_program)
fluid.layers.dropout(
x=images,
dropout_prob=0.5,
main_program=main_program,
startup_program=startup_program)
with fluid.program_guard(main_program, startup_program):
images = fluid.layers.data(
name='pixel', shape=[3, 48, 48], dtype='float32')
fluid.layers.dropout(x=images, dropout_prob=0.5)
# print str(main_program)
print str(main_program)
def test_img_conv_group(self):
main_program = Program()
startup_program = Program()
images = fluid.layers.data(
name='pixel',
shape=[3, 48, 48],
dtype='float32',
main_program=main_program,
startup_program=startup_program)
conv1 = conv_block(images, 64, 2, [0.3, 0], main_program,
startup_program)
conv2 = conv_block(conv1, 256, 3, [0.4, 0.4, 0], main_program,
startup_program)
with fluid.program_guard(main_program, startup_program):
images = fluid.layers.data(
name='pixel', shape=[3, 48, 48], dtype='float32')
conv1 = conv_block(images, 64, 2, [0.3, 0])
conv_block(conv1, 256, 3, [0.4, 0.4, 0])
# print str(main_program)
print str(main_program)
def test_elementwise_add_with_act(self):
main_program = Program()
startup_program = Program()
image1 = fluid.layers.data(
name='pixel1',
shape=[3, 48, 48],
dtype='float32',
main_program=main_program,
startup_program=startup_program)
image2 = fluid.layers.data(
name='pixel2',
shape=[3, 48, 48],
dtype='float32',
main_program=main_program,
startup_program=startup_program)
out = fluid.layers.elementwise_add(
x=image1,
y=image2,
act='relu',
main_program=main_program,
startup_program=startup_program)
# print(main_program)
with fluid.program_guard(main_program, startup_program):
image1 = fluid.layers.data(
name='pixel1', shape=[3, 48, 48], dtype='float32')
image2 = fluid.layers.data(
name='pixel2', shape=[3, 48, 48], dtype='float32')
fluid.layers.elementwise_add(x=image1, y=image2, act='relu')
print(main_program)
if __name__ == '__main__':
......
......@@ -6,7 +6,7 @@ import paddle.v2.fluid.core as core
import paddle.v2.fluid.executor as executor
import paddle.v2.fluid.layers as layers
import paddle.v2.fluid.optimizer as optimizer
from paddle.v2.fluid.framework import Program
from paddle.v2.fluid.framework import Program, program_guard
from paddle.v2.fluid.io import save_inference_model, load_inference_model
......@@ -16,35 +16,18 @@ class TestBook(unittest.TestCase):
init_program = Program()
program = Program()
x = layers.data(
name='x',
shape=[2],
dtype='float32',
main_program=program,
startup_program=init_program)
y = layers.data(
name='y',
shape=[1],
dtype='float32',
main_program=program,
startup_program=init_program)
y_predict = layers.fc(input=x,
size=1,
act=None,
main_program=program,
startup_program=init_program)
cost = layers.square_error_cost(
input=y_predict,
label=y,
main_program=program,
startup_program=init_program)
avg_cost = layers.mean(
x=cost, main_program=program, startup_program=init_program)
sgd_optimizer = optimizer.SGDOptimizer(learning_rate=0.001)
sgd_optimizer.minimize(avg_cost, init_program)
with program_guard(program, init_program):
x = layers.data(name='x', shape=[2], dtype='float32')
y = layers.data(name='y', shape=[1], dtype='float32')
y_predict = layers.fc(input=x, size=1, act=None)
cost = layers.square_error_cost(input=y_predict, label=y)
avg_cost = layers.mean(x=cost)
sgd_optimizer = optimizer.SGDOptimizer(learning_rate=0.001)
sgd_optimizer.minimize(avg_cost, init_program)
place = core.CPUPlace()
exe = executor.Executor(place)
......
......@@ -2,7 +2,7 @@ import unittest
import paddle.v2.fluid.core as core
import numpy
import paddle.v2.fluid.layers as layers
from paddle.v2.fluid.framework import Program
from paddle.v2.fluid.framework import Program, program_guard
from paddle.v2.fluid.executor import Executor
from paddle.v2.fluid.backward import append_backward_ops
......@@ -118,16 +118,17 @@ class TestCPULoDTensorArrayOps(unittest.TestCase):
def main(self, tensor, expect_array, expect_lod, expect_max_len, level=0):
place = self.place()
program = Program()
x = layers.data(name='x', shape=[10], main_program=program)
x.persistable = True
table = layers.lod_rank_table(x, level=level, main_program=program)
max_len = layers.max_sequence_len(table, main_program=program)
max_len.persistable = True
array = layers.lod_tensor_to_array(x, table, main_program=program)
array.persistable = True
result = layers.array_to_lod_tensor(array, table, main_program=program)
result.persistable = True
with program_guard(program):
x = layers.data(name='x', shape=[10])
x.persistable = True
table = layers.lod_rank_table(x, level=level)
max_len = layers.max_sequence_len(table)
max_len.persistable = True
array = layers.lod_tensor_to_array(x, table)
array.persistable = True
result = layers.array_to_lod_tensor(array, table)
result.persistable = True
exe = Executor(place)
scope = core.Scope()
exe.run(program, feed={'x': tensor}, scope=scope)
......@@ -160,19 +161,16 @@ class TestCPULoDTensorArrayOpGrad(unittest.TestCase):
place = core.CPUPlace()
program = Program()
x = layers.data(
name='x',
shape=[1],
dtype='float32',
main_program=program,
stop_gradient=False)
table = layers.lod_rank_table(x, level=0, main_program=program)
array = layers.lod_tensor_to_array(x, table, main_program=program)
result = layers.array_to_lod_tensor(array, table, main_program=program)
with program_guard(program):
x = layers.data(
name='x', shape=[1], dtype='float32', stop_gradient=False)
table = layers.lod_rank_table(x, level=0)
array = layers.lod_tensor_to_array(x, table)
result = layers.array_to_lod_tensor(array, table)
mean = layers.mean(x=result, main_program=program)
mean = layers.mean(x=result)
append_backward_ops(mean)
append_backward_ops(mean)
tensor = core.LoDTensor()
tensor.set(numpy.arange(10).reshape(10, 1).astype('float32'), place)
......
import paddle.v2.fluid.layers as layers
from paddle.v2.fluid.framework import Program
from paddle.v2.fluid.framework import Program, program_guard, default_main_program, default_startup_program
from paddle.v2.fluid.executor import Executor
from paddle.v2.fluid.optimizer import MomentumOptimizer
import paddle.v2.fluid.core as core
......@@ -10,44 +10,42 @@ import numpy as np
class TestMNISTIfElseOp(unittest.TestCase):
def test_raw_api(self):
kwargs = {'startup_program': Program(), 'main_program': Program()}
image = layers.data(name='x', shape=[784], dtype='float32', **kwargs)
prog = Program()
startup_prog = Program()
with program_guard(prog, startup_prog):
image = layers.data(name='x', shape=[784], dtype='float32')
label = layers.data(name='y', shape=[1], dtype='int64', **kwargs)
label = layers.data(name='y', shape=[1], dtype='int64')
limit = layers.fill_constant_batch_size_like(
input=label, dtype='int64', shape=[1], value=5.0, **kwargs)
limit = layers.fill_constant_batch_size_like(
input=label, dtype='int64', shape=[1], value=5.0)
cond = layers.less_than(x=label, y=limit)
true_image, false_image = layers.split_lod_tensor(
input=image, mask=cond)
cond = layers.less_than(x=label, y=limit, **kwargs)
true_image, false_image = layers.split_lod_tensor(
input=image, mask=cond, **kwargs)
true_out = layers.create_tensor(dtype='float32')
true_cond = layers.ConditionalBlock([true_image])
true_out = layers.create_tensor(dtype='float32', **kwargs)
true_cond = layers.ConditionalBlock([true_image], **kwargs)
with true_cond.block():
hidden = layers.fc(input=true_image, size=100, act='tanh')
prob = layers.fc(input=hidden, size=10, act='softmax')
layers.assign(input=prob, output=true_out)
with true_cond.block():
hidden = layers.fc(input=true_image, size=100, act='tanh', **kwargs)
prob = layers.fc(input=hidden, size=10, act='softmax', **kwargs)
layers.assign(input=prob, output=true_out, **kwargs)
false_out = layers.create_tensor(dtype='float32')
false_cond = layers.ConditionalBlock([false_image])
false_out = layers.create_tensor(dtype='float32', **kwargs)
false_cond = layers.ConditionalBlock([false_image], **kwargs)
with false_cond.block():
hidden = layers.fc(input=false_image, size=200, act='tanh')
prob = layers.fc(input=hidden, size=10, act='softmax')
layers.assign(input=prob, output=false_out)
with false_cond.block():
hidden = layers.fc(input=false_image,
size=200,
act='tanh',
**kwargs)
prob = layers.fc(input=hidden, size=10, act='softmax', **kwargs)
layers.assign(input=prob, output=false_out, **kwargs)
prob = layers.merge_lod_tensor(
in_true=true_out, in_false=false_out, mask=cond, x=image)
loss = layers.cross_entropy(input=prob, label=label)
avg_loss = layers.mean(x=loss)
prob = layers.merge_lod_tensor(
in_true=true_out, in_false=false_out, mask=cond, x=image, **kwargs)
loss = layers.cross_entropy(input=prob, label=label, **kwargs)
avg_loss = layers.mean(x=loss, **kwargs)
optimizer = MomentumOptimizer(learning_rate=0.001, momentum=0.9)
optimizer.minimize(avg_loss, kwargs['startup_program'])
optimizer = MomentumOptimizer(learning_rate=0.001, momentum=0.9)
optimizer.minimize(avg_loss, startup_prog)
train_reader = paddle.batch(
paddle.reader.shuffle(
......@@ -57,7 +55,7 @@ class TestMNISTIfElseOp(unittest.TestCase):
place = core.CPUPlace()
exe = Executor(place)
exe.run(kwargs['startup_program'])
exe.run(startup_prog)
PASS_NUM = 100
for pass_id in range(PASS_NUM):
for data in train_reader():
......@@ -65,7 +63,7 @@ class TestMNISTIfElseOp(unittest.TestCase):
y_data = np.array(map(lambda x: x[1], data)).astype("int64")
y_data = np.expand_dims(y_data, axis=1)
outs = exe.run(kwargs['main_program'],
outs = exe.run(prog,
feed={'x': x_data,
'y': y_data},
fetch_list=[avg_loss])
......@@ -75,39 +73,36 @@ class TestMNISTIfElseOp(unittest.TestCase):
self.assertFalse(True)
def test_ifelse(self):
kwargs = {'startup_program': Program(), 'main_program': Program()}
image = layers.data(name='x', shape=[784], dtype='float32', **kwargs)
label = layers.data(name='y', shape=[1], dtype='int64', **kwargs)
limit = layers.fill_constant_batch_size_like(
input=label, dtype='int64', shape=[1], value=5.0, **kwargs)
cond = layers.less_than(x=label, y=limit, **kwargs)
ie = layers.IfElse(cond, **kwargs)
with ie.true_block():
true_image = ie.input(image)
hidden = layers.fc(input=true_image, size=100, act='tanh', **kwargs)
prob = layers.fc(input=hidden, size=10, act='softmax', **kwargs)
ie.output(prob)
with ie.false_block():
false_image = ie.input(image)
hidden = layers.fc(input=false_image,
size=200,
act='tanh',
**kwargs)
prob = layers.fc(input=hidden, size=10, act='softmax', **kwargs)
ie.output(prob)
prob = ie()
loss = layers.cross_entropy(input=prob[0], label=label, **kwargs)
avg_loss = layers.mean(x=loss, **kwargs)
optimizer = MomentumOptimizer(learning_rate=0.001, momentum=0.9)
optimizer.minimize(avg_loss, kwargs['startup_program'])
prog = Program()
startup_prog = Program()
with program_guard(prog, startup_prog):
image = layers.data(name='x', shape=[784], dtype='float32')
label = layers.data(name='y', shape=[1], dtype='int64')
limit = layers.fill_constant_batch_size_like(
input=label, dtype='int64', shape=[1], value=5.0)
cond = layers.less_than(x=label, y=limit)
ie = layers.IfElse(cond)
with ie.true_block():
true_image = ie.input(image)
hidden = layers.fc(input=true_image, size=100, act='tanh')
prob = layers.fc(input=hidden, size=10, act='softmax')
ie.output(prob)
with ie.false_block():
false_image = ie.input(image)
hidden = layers.fc(input=false_image, size=200, act='tanh')
prob = layers.fc(input=hidden, size=10, act='softmax')
ie.output(prob)
prob = ie()
loss = layers.cross_entropy(input=prob[0], label=label)
avg_loss = layers.mean(x=loss)
optimizer = MomentumOptimizer(learning_rate=0.001, momentum=0.9)
optimizer.minimize(avg_loss, startup_prog)
train_reader = paddle.batch(
paddle.reader.shuffle(
paddle.dataset.mnist.train(), buf_size=8192),
......@@ -135,4 +130,5 @@ class TestMNISTIfElseOp(unittest.TestCase):
if __name__ == '__main__':
unittest.main()
# temp disable if else unittest since it could be buggy.
exit(0)
from __future__ import print_function
import unittest
from paddle.v2.fluid.framework import Program, default_main_program
from paddle.v2.fluid.framework import Program, default_main_program, program_guard
import paddle.v2.fluid.layers as layers
main_program = default_main_program()
......@@ -129,13 +129,10 @@ class TestProgram(unittest.TestCase):
def test_program_clone_with_parameter(self):
main_program = Program()
startup_program = Program()
kwargs = {
'main_program': main_program,
'startup_program': startup_program
}
d = layers.data(name='x', shape=[784], dtype='float32', **kwargs)
hidden = layers.fc(input=d, size=100, **kwargs)
layers.fc(input=hidden, size=100, **kwargs)
with program_guard(main_program, startup_program):
d = layers.data(name='x', shape=[784], dtype='float32')
hidden = layers.fc(input=d, size=100)
layers.fc(input=hidden, size=100)
new_program = main_program.clone()
self.assertNotEqual(0, len(new_program.blocks[0].all_parameters()))
......
......@@ -2,7 +2,7 @@ import unittest
import paddle.v2.fluid.core as core
import numpy as np
import paddle.v2.fluid.layers as layers
from paddle.v2.fluid.framework import Program
from paddle.v2.fluid.framework import Program, program_guard
from paddle.v2.fluid.executor import Executor
from paddle.v2.fluid.backward import append_backward_ops
......@@ -75,26 +75,22 @@ class TestCPULoDTensorArrayOps(unittest.TestCase):
level=0):
place = self.place()
program = Program()
x = layers.data(name='x', shape=[1], main_program=program)
x.persistable = True
with program_guard(program):
x = layers.data(name='x', shape=[1])
x.persistable = True
y = layers.data(name='y', shape=[1], main_program=program)
y.persistable = True
y = layers.data(name='y', shape=[1])
y.persistable = True
out_true, out_false = layers.split_lod_tensor(
input=x, mask=y, level=level, main_program=program)
out_true.persistable = True
out_false.persistable = True
out_true, out_false = layers.split_lod_tensor(
input=x, mask=y, level=level)
out_true.persistable = True
out_false.persistable = True
out = layers.merge_lod_tensor(
in_true=out_true,
in_false=out_false,
mask=y,
x=x,
level=level,
main_program=program)
out = layers.merge_lod_tensor(
in_true=out_true, in_false=out_false, mask=y, x=x, level=level)
out.persistable = True
out.persistable = True
exe = Executor(place)
scope = core.Scope()
......@@ -123,34 +119,21 @@ class TestCPUSplitMergeLoDTensorGrad(unittest.TestCase):
def test_grad(self):
place = core.CPUPlace()
program = Program()
with program_guard(program):
x = layers.data(
name='x', shape=[1], dtype='float32', stop_gradient=False)
y = layers.data(
name='y', shape=[1], dtype='bool', stop_gradient=False)
x = layers.data(
name='x',
shape=[1],
dtype='float32',
main_program=program,
stop_gradient=False)
y = layers.data(
name='y',
shape=[1],
dtype='bool',
main_program=program,
stop_gradient=False)
level = 0
out_true, out_false = layers.split_lod_tensor(
input=x, mask=y, level=level, main_program=program)
out = layers.merge_lod_tensor(
in_true=out_true,
in_false=out_false,
mask=y,
x=x,
level=level,
main_program=program)
mean = layers.mean(x=out, main_program=program)
append_backward_ops(mean)
level = 0
out_true, out_false = layers.split_lod_tensor(
input=x, mask=y, level=level)
out = layers.merge_lod_tensor(
in_true=out_true, in_false=out_false, mask=y, x=x, level=level)
mean = layers.mean(x=out)
append_backward_ops(mean)
tensor = core.LoDTensor()
tensor.set(np.arange(10).reshape(10, 1).astype('float32'), place)
......
Markdown is supported
0% .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑!
想要评论请 注册