提交 dfb2261d 编写于 作者: Y yangyaming

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into fix-6712

# Executor Design Doc # Executor Design Doc
## Motivation ## Motivation
In the [fluid](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/fluid.md), we encourage user use deep learning programming paradigms to describe training process. When the user-written Python program is executed, it will create a protobuf message In [fluid](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/fluid.md), we encourage the user to use deep learning programming paradigms to describe the training process. When the user-written Python program is executed, it will first create a protobuf message
[`ProgramDesc`](https://github.com/PaddlePaddle/Paddle/blob/a91efdde6910ce92a78e3aa7157412c4c88d9ee8/paddle/framework/framework.proto#L145) that describes the process and is conceptually like an [abstract syntax tree](https://en.wikipedia.org/wiki/Abstract_syntax_tree). [`ProgramDesc`](https://github.com/PaddlePaddle/Paddle/blob/a91efdde6910ce92a78e3aa7157412c4c88d9ee8/paddle/framework/framework.proto#L145) that describes the process and is conceptually like an [abstract syntax tree](https://en.wikipedia.org/wiki/Abstract_syntax_tree).
The executor runs the `ProgramDesc` like an interpreter. `ProgramDesc` contains intrinsics/operators and variables which will be used, executor explicitly execute the stored precompiled code. The executor runs the `ProgramDesc` like an interpreter. `ProgramDesc` contains the intrinsics (operators in this case) and variables which will be used, executor explicitly executes the stored precompiled code.
## Overview ## Overview
An executor takes a `ProgramDesc`, a `block_id` and a `Scope`. The `ProgramDesc` is a list of blocks and each block contains the protobuf definition of all the parameters and operators. The `block_id` specifies the entrance block. And the `Scope` is the container of all the variable instance, which is persistent throughout different runs. An executor takes a `ProgramDesc`, a `block_id` and a `Scope`. The `ProgramDesc` is a list of blocks and each block contains the protobuf definition of all the parameters and operators in the block. The `block_id` specifies the entrance block. And the `Scope` is the container of all the variable instances, which is persistent throughout different runs.
## Executor ## Executor
`Executor` explicitly executes all the intrinsics/operators in the `block_id`th block of a `ProgramDesc`. Essentially, it instantiates Variables and Operators, then runs all the operators in sequence. It is very similar to push stack frame when entering the block, it will destroy the temporary variables when mini-batch is finished, but it does not have stack frame pop process. The `Executor` explicitly executes all the intrinsics (operators here) in the `block_id`th block of a `ProgramDesc`. Essentially, it instantiates Variables and Operators, then runs all the operators in sequence one-by-one.
It is very similar to how a push stack frame works when entering a block, following which it cleans up all the temporary variables when a mini-batch is finished. It does not however, have the stack frame pop process.
### Interface ### The interface
```c++ ```c++
Executor(places); Executor(places);
``` ```
A executor does not own any computing resources, user can only construct an executor with specified places. A executor does not own any computing resources, a user can only construct an executor using the specified places.
### Running an Executor
``` ```
void Run(ProgramDesc, Scope, block_id, create_local_scope); void Run(ProgramDesc, Scope, block_id, create_local_scope);
``` ```
A executor only provides an unified way to execute `ProgramDesc`. `ProgramDesc` is the target will be executed, scope specifies the variable container. `block_id` indicates the entrance block, `create_local_scope` means if it will destroy the temporary variables after execution finished. An `Executor` only provides a unified way to execute `ProgramDesc`. `ProgramDesc` is the target that will be executed, the `Scope` specifies the variable container, the `block_id` indicates the entrance block and `create_local_scope` is a boolean that states whether it will destroy the temporary variables after the execution is finished.
import numpy as np import numpy as np
import layers import layers
from framework import Program, unique_name, Variable from framework import Program, unique_name, Variable, program_guard
from layer_helper import LayerHelper from layer_helper import LayerHelper
__all__ = ['Accuracy', 'ChunkEvaluator'] __all__ = ['Accuracy', 'ChunkEvaluator']
...@@ -49,15 +49,12 @@ class Evaluator(object): ...@@ -49,15 +49,12 @@ class Evaluator(object):
if reset_program is None: if reset_program is None:
reset_program = Program() reset_program = Program()
for var in self.states: with program_guard(main_program=reset_program):
assert isinstance(var, Variable) for var in self.states:
g_var = _clone_var_(reset_program.current_block(), var) assert isinstance(var, Variable)
layers.fill_constant( g_var = _clone_var_(reset_program.current_block(), var)
shape=g_var.shape, layers.fill_constant(
value=0.0, shape=g_var.shape, value=0.0, dtype=g_var.dtype, out=g_var)
dtype=g_var.dtype,
out=g_var,
main_program=reset_program)
executor.run(reset_program) executor.run(reset_program)
...@@ -104,20 +101,14 @@ class Accuracy(Evaluator): ...@@ -104,20 +101,14 @@ class Accuracy(Evaluator):
self.total = self.create_state(dtype='int64', shape=[1], suffix='total') self.total = self.create_state(dtype='int64', shape=[1], suffix='total')
self.correct = self.create_state( self.correct = self.create_state(
dtype='int64', shape=[1], suffix='correct') dtype='int64', shape=[1], suffix='correct')
kwargs = {'main_program': main_program}
total = self.helper.create_tmp_variable(dtype='int') total = self.helper.create_tmp_variable(dtype='int')
correct = self.helper.create_tmp_variable(dtype='int') correct = self.helper.create_tmp_variable(dtype='int')
acc = layers.accuracy( acc = layers.accuracy(
input=input, input=input, label=label, k=k, total=total, correct=correct)
label=label, total = layers.cast(x=total, dtype='int64')
k=k, correct = layers.cast(x=correct, dtype='int64')
total=total, layers.sums(input=[self.total, total], out=self.total)
correct=correct, layers.sums(input=[self.correct, correct], out=self.correct)
**kwargs)
total = layers.cast(x=total, dtype='int64', **kwargs)
correct = layers.cast(x=correct, dtype='int64', **kwargs)
layers.sums(input=[self.total, total], out=self.total, **kwargs)
layers.sums(input=[self.correct, correct], out=self.correct, **kwargs)
self.metrics.append(acc) self.metrics.append(acc)
...@@ -125,12 +116,12 @@ class Accuracy(Evaluator): ...@@ -125,12 +116,12 @@ class Accuracy(Evaluator):
if eval_program is None: if eval_program is None:
eval_program = Program() eval_program = Program()
block = eval_program.current_block() block = eval_program.current_block()
kwargs = {'main_program': eval_program} with program_guard(main_program=eval_program):
total = _clone_var_(block, self.total) total = _clone_var_(block, self.total)
correct = _clone_var_(block, self.correct) correct = _clone_var_(block, self.correct)
total = layers.cast(total, dtype='float32', **kwargs) total = layers.cast(total, dtype='float32')
correct = layers.cast(correct, dtype='float32', **kwargs) correct = layers.cast(correct, dtype='float32')
out = layers.elementwise_div(x=correct, y=total, **kwargs) out = layers.elementwise_div(x=correct, y=total)
return np.array(executor.run(eval_program, fetch_list=[out])[0]) return np.array(executor.run(eval_program, fetch_list=[out])[0])
...@@ -141,14 +132,14 @@ class ChunkEvaluator(Evaluator): ...@@ -141,14 +132,14 @@ class ChunkEvaluator(Evaluator):
numbers. numbers.
""" """
def __init__(self, def __init__(
input, self,
label, input,
chunk_scheme, label,
num_chunk_types, chunk_scheme,
excluded_chunk_types=None, num_chunk_types,
**kwargs): excluded_chunk_types=None, ):
super(ChunkEvaluator, self).__init__("chunk_eval", **kwargs) super(ChunkEvaluator, self).__init__("chunk_eval")
main_program = self.helper.main_program main_program = self.helper.main_program
if main_program.current_block().idx != 0: if main_program.current_block().idx != 0:
raise ValueError("You can only invoke Evaluator in root block") raise ValueError("You can only invoke Evaluator in root block")
...@@ -159,26 +150,21 @@ class ChunkEvaluator(Evaluator): ...@@ -159,26 +150,21 @@ class ChunkEvaluator(Evaluator):
dtype='int64', shape=[1], suffix='num_label_chunks') dtype='int64', shape=[1], suffix='num_label_chunks')
self.num_correct_chunks = self.create_state( self.num_correct_chunks = self.create_state(
dtype='int64', shape=[1], suffix='num_correct_chunks') dtype='int64', shape=[1], suffix='num_correct_chunks')
kwargs = {'main_program': main_program}
precision, recall, f1_score, num_infer_chunks, num_label_chunks, num_correct_chunks = layers.chunk_eval( precision, recall, f1_score, num_infer_chunks, num_label_chunks, num_correct_chunks = layers.chunk_eval(
input=input, input=input,
label=label, label=label,
chunk_scheme=chunk_scheme, chunk_scheme=chunk_scheme,
num_chunk_types=num_chunk_types, num_chunk_types=num_chunk_types,
excluded_chunk_types=excluded_chunk_types, excluded_chunk_types=excluded_chunk_types, )
**kwargs)
layers.sums( layers.sums(
input=[self.num_infer_chunks, num_infer_chunks], input=[self.num_infer_chunks, num_infer_chunks],
out=self.num_infer_chunks, out=self.num_infer_chunks)
**kwargs)
layers.sums( layers.sums(
input=[self.num_label_chunks, num_label_chunks], input=[self.num_label_chunks, num_label_chunks],
out=self.num_label_chunks, out=self.num_label_chunks)
**kwargs)
layers.sums( layers.sums(
input=[self.num_correct_chunks, num_correct_chunks], input=[self.num_correct_chunks, num_correct_chunks],
out=self.num_correct_chunks, out=self.num_correct_chunks)
**kwargs)
self.metrics.extend([precision, recall, f1_score]) self.metrics.extend([precision, recall, f1_score])
...@@ -186,7 +172,6 @@ class ChunkEvaluator(Evaluator): ...@@ -186,7 +172,6 @@ class ChunkEvaluator(Evaluator):
if eval_program is None: if eval_program is None:
eval_program = Program() eval_program = Program()
block = eval_program.current_block() block = eval_program.current_block()
kwargs = {'main_program': eval_program}
num_infer_chunks, num_label_chunks, num_correct_chunks = executor.run( num_infer_chunks, num_label_chunks, num_correct_chunks = executor.run(
eval_program, eval_program,
fetch_list=[_clone_var_(block, state) for state in self.states]) fetch_list=[_clone_var_(block, state) for state in self.states])
......
...@@ -21,19 +21,11 @@ class LayerHelper(object): ...@@ -21,19 +21,11 @@ class LayerHelper(object):
@property @property
def main_program(self): def main_program(self):
prog = self.kwargs.get('main_program', None) return default_main_program()
if prog is None:
return default_main_program()
else:
return prog
@property @property
def startup_program(self): def startup_program(self):
prog = self.kwargs.get('startup_program', None) return default_startup_program()
if prog is None:
return default_startup_program()
else:
return prog
def append_op(self, *args, **kwargs): def append_op(self, *args, **kwargs):
return self.main_program.current_block().append_op(*args, **kwargs) return self.main_program.current_block().append_op(*args, **kwargs)
...@@ -151,13 +143,6 @@ class LayerHelper(object): ...@@ -151,13 +143,6 @@ class LayerHelper(object):
persistable=True, persistable=True,
initializer=initializer) initializer=initializer)
@property
def to_kwargs(self):
return {
'main_program': self.main_program,
'startup_program': self.startup_program
}
def append_bias_op(self, input_var, dim_start=1, dim_end=None): def append_bias_op(self, input_var, dim_start=1, dim_end=None):
""" """
Append bias operator and return its output. If the user does not set Append bias operator and return its output. If the user does not set
......
...@@ -14,11 +14,7 @@ __all__ = [ ...@@ -14,11 +14,7 @@ __all__ = [
] ]
def split_lod_tensor(input, def split_lod_tensor(input, mask, level=0):
mask,
level=0,
main_program=None,
startup_program=None):
helper = LayerHelper('split_lod_tensor', **locals()) helper = LayerHelper('split_lod_tensor', **locals())
out_true = helper.create_tmp_variable(dtype=input.dtype) out_true = helper.create_tmp_variable(dtype=input.dtype)
out_false = helper.create_tmp_variable(dtype=input.dtype) out_false = helper.create_tmp_variable(dtype=input.dtype)
...@@ -34,13 +30,7 @@ def split_lod_tensor(input, ...@@ -34,13 +30,7 @@ def split_lod_tensor(input,
return out_true, out_false return out_true, out_false
def merge_lod_tensor(in_true, def merge_lod_tensor(in_true, in_false, x, mask, level=0):
in_false,
x,
mask,
level=0,
main_program=None,
startup_program=None):
helper = LayerHelper('merge_lod_tensor', **locals()) helper = LayerHelper('merge_lod_tensor', **locals())
out = helper.create_tmp_variable(dtype=in_true.dtype) out = helper.create_tmp_variable(dtype=in_true.dtype)
helper.append_op( helper.append_op(
...@@ -135,9 +125,8 @@ class StaticRNN(object): ...@@ -135,9 +125,8 @@ class StaticRNN(object):
IN_RNN_BLOCK = 1 IN_RNN_BLOCK = 1
AFTER_RNN_BLOCK = 2 AFTER_RNN_BLOCK = 2
def __init__(self, name=None, main_program=None): def __init__(self, name=None):
self.helper = LayerHelper( self.helper = LayerHelper("static_rnn", name=name)
"static_rnn", name=name, main_program=main_program)
self.memories = {} # memory map, from pre_mem.name --> MemoryLink self.memories = {} # memory map, from pre_mem.name --> MemoryLink
self.inputs = [] # input variable list in current block self.inputs = [] # input variable list in current block
self.outputs = [] # output variable list in parent block self.outputs = [] # output variable list in parent block
...@@ -354,8 +343,8 @@ class While(object): ...@@ -354,8 +343,8 @@ class While(object):
IN_WHILE_BLOCK = 1 IN_WHILE_BLOCK = 1
AFTER_WHILE_BLOCK = 2 AFTER_WHILE_BLOCK = 2
def __init__(self, cond, name=None, main_program=None): def __init__(self, cond, name=None):
self.helper = LayerHelper("while", name=name, main_program=main_program) self.helper = LayerHelper("while", name=name)
self.status = While.BEFORE_WHILE_BLOCK self.status = While.BEFORE_WHILE_BLOCK
if not isinstance(cond, Variable): if not isinstance(cond, Variable):
raise TypeError("condition should be a variable") raise TypeError("condition should be a variable")
...@@ -406,7 +395,7 @@ class While(object): ...@@ -406,7 +395,7 @@ class While(object):
attrs={'sub_block': while_block}) attrs={'sub_block': while_block})
def lod_rank_table(x, level=0, main_program=None): def lod_rank_table(x, level=0):
""" """
This function creates an operator for creating a LOD_RANK_TABLE This function creates an operator for creating a LOD_RANK_TABLE
using the input x. using the input x.
...@@ -423,7 +412,7 @@ def lod_rank_table(x, level=0, main_program=None): ...@@ -423,7 +412,7 @@ def lod_rank_table(x, level=0, main_program=None):
return table return table
def max_sequence_len(rank_table, main_program=None): def max_sequence_len(rank_table):
""" """
This function creates an operator to calculate the length of This function creates an operator to calculate the length of
max seqence through input rank_table(should be a lod_rank_table) max seqence through input rank_table(should be a lod_rank_table)
...@@ -437,7 +426,7 @@ def max_sequence_len(rank_table, main_program=None): ...@@ -437,7 +426,7 @@ def max_sequence_len(rank_table, main_program=None):
return res return res
def topk(input, k, main_program=None, startup_program=None): def topk(input, k):
helper = LayerHelper('topk', **locals()) helper = LayerHelper('topk', **locals())
topk_out = helper.create_tmp_variable(dtype=input.data_type) topk_out = helper.create_tmp_variable(dtype=input.data_type)
topk_indices = helper.create_tmp_variable(dtype='int64') topk_indices = helper.create_tmp_variable(dtype='int64')
...@@ -450,7 +439,7 @@ def topk(input, k, main_program=None, startup_program=None): ...@@ -450,7 +439,7 @@ def topk(input, k, main_program=None, startup_program=None):
return topk_out, topk_indices return topk_out, topk_indices
def lod_tensor_to_array(x, table, main_program=None): def lod_tensor_to_array(x, table):
""" """
This function creates an operator to convert an LOD_Tensor to This function creates an operator to convert an LOD_Tensor to
an array. an array.
...@@ -468,7 +457,7 @@ def lod_tensor_to_array(x, table, main_program=None): ...@@ -468,7 +457,7 @@ def lod_tensor_to_array(x, table, main_program=None):
return array return array
def array_to_lod_tensor(x, table, main_program=None, startup_program=None): def array_to_lod_tensor(x, table):
""" """
This function creates an operator to convert an array to a This function creates an operator to convert an array to a
LOD_Tensor. LOD_Tensor.
...@@ -483,11 +472,7 @@ def array_to_lod_tensor(x, table, main_program=None, startup_program=None): ...@@ -483,11 +472,7 @@ def array_to_lod_tensor(x, table, main_program=None, startup_program=None):
return tmp return tmp
def increment(x, def increment(x, value=1.0, in_place=True):
value=1.0,
in_place=True,
main_program=None,
startup_program=None):
""" """
This function creates an operator to increment each value in the input This function creates an operator to increment each value in the input
`x` by an amount: `value` as mentioned in the input parameter. This `x` by an amount: `value` as mentioned in the input parameter. This
...@@ -506,7 +491,7 @@ def increment(x, ...@@ -506,7 +491,7 @@ def increment(x,
return out return out
def array_write(x, i, array=None, main_program=None, startup_program=None): def array_write(x, i, array=None):
""" """
This function creates an operator to write the data out as a This function creates an operator to write the data out as a
LOD_TENSOR_ARRAY. LOD_TENSOR_ARRAY.
...@@ -525,7 +510,7 @@ def array_write(x, i, array=None, main_program=None, startup_program=None): ...@@ -525,7 +510,7 @@ def array_write(x, i, array=None, main_program=None, startup_program=None):
return array return array
def create_array(dtype, main_program=None): def create_array(dtype):
helper = LayerHelper("array", **locals()) helper = LayerHelper("array", **locals())
return helper.create_variable( return helper.create_variable(
name="{0}.out".format(helper.name), name="{0}.out".format(helper.name),
...@@ -533,7 +518,7 @@ def create_array(dtype, main_program=None): ...@@ -533,7 +518,7 @@ def create_array(dtype, main_program=None):
dtype=dtype) dtype=dtype)
def less_than(x, y, cond=None, main_program=None, **ignored): def less_than(x, y, cond=None, **ignored):
helper = LayerHelper("less_than", **locals()) helper = LayerHelper("less_than", **locals())
if cond is None: if cond is None:
cond = helper.create_tmp_variable(dtype='bool') cond = helper.create_tmp_variable(dtype='bool')
...@@ -545,7 +530,7 @@ def less_than(x, y, cond=None, main_program=None, **ignored): ...@@ -545,7 +530,7 @@ def less_than(x, y, cond=None, main_program=None, **ignored):
return cond return cond
def array_read(array, i, main_program=None, startup_program=None): def array_read(array, i):
""" """
This function creates an operator to read the data in as a This function creates an operator to read the data in as a
LOD_TENSOR_ARRAY. LOD_TENSOR_ARRAY.
...@@ -564,7 +549,7 @@ def array_read(array, i, main_program=None, startup_program=None): ...@@ -564,7 +549,7 @@ def array_read(array, i, main_program=None, startup_program=None):
return out return out
def shrink_memory(x, i, table, main_program=None, startup_program=None): def shrink_memory(x, i, table):
""" """
This function creates an operator to shrink_rnn_memory using the RankTable This function creates an operator to shrink_rnn_memory using the RankTable
as mentioned in the input parameter. as mentioned in the input parameter.
...@@ -581,7 +566,7 @@ def shrink_memory(x, i, table, main_program=None, startup_program=None): ...@@ -581,7 +566,7 @@ def shrink_memory(x, i, table, main_program=None, startup_program=None):
return out return out
def array_length(array, main_program=None): def array_length(array):
""" """
This function creates an operator to find the length of the This function creates an operator to find the length of the
LOD_TENSOR_ARRAY. LOD_TENSOR_ARRAY.
...@@ -611,20 +596,12 @@ class ConditionalBlockGuard(BlockGuard): ...@@ -611,20 +596,12 @@ class ConditionalBlockGuard(BlockGuard):
class ConditionalBlock(object): class ConditionalBlock(object):
def __init__(self, def __init__(self, inputs, name=None):
inputs,
name=None,
main_program=None,
startup_program=None):
for each_input in inputs: for each_input in inputs:
if not isinstance(each_input, Variable): if not isinstance(each_input, Variable):
raise TypeError("Each input should be variable") raise TypeError("Each input should be variable")
self.inputs = inputs self.inputs = inputs
self.helper = LayerHelper( self.helper = LayerHelper('conditional_block', name=name)
'conditional_block',
name=name,
main_program=main_program,
startup_program=startup_program)
def block(self): def block(self):
return ConditionalBlockGuard(self) return ConditionalBlockGuard(self)
...@@ -709,15 +686,10 @@ class IfElse(object): ...@@ -709,15 +686,10 @@ class IfElse(object):
IN_IF_ELSE_TRUE_BLOCKS = 1 IN_IF_ELSE_TRUE_BLOCKS = 1
IN_IF_ELSE_FALSE_BLOCKS = 2 IN_IF_ELSE_FALSE_BLOCKS = 2
def __init__(self, cond, name=None, main_program=None, def __init__(self, cond, name=None):
startup_program=None):
if not isinstance(cond, Variable): if not isinstance(cond, Variable):
raise TypeError("cond must be a Variable") raise TypeError("cond must be a Variable")
self.helper = LayerHelper( self.helper = LayerHelper('ifelse', name=name)
'ifelse',
name=name,
main_program=main_program,
startup_program=startup_program)
self.cond = cond self.cond = cond
self.input_table = {} self.input_table = {}
self.status = IfElse.OUT_IF_ELSE_BLOCKS self.status = IfElse.OUT_IF_ELSE_BLOCKS
...@@ -782,11 +754,7 @@ class IfElse(object): ...@@ -782,11 +754,7 @@ class IfElse(object):
out_table.append(outside_out) out_table.append(outside_out)
# assign local var to outside # assign local var to outside
assign( assign(input=each_out, output=outside_out)
input=each_out,
output=outside_out,
main_program=self.helper.main_program,
startup_program=self.helper.startup_program)
def __call__(self): def __call__(self):
if self.status != self.OUT_IF_ELSE_BLOCKS: if self.status != self.OUT_IF_ELSE_BLOCKS:
...@@ -810,9 +778,7 @@ class IfElse(object): ...@@ -810,9 +778,7 @@ class IfElse(object):
in_false=false_var, in_false=false_var,
mask=self.cond, mask=self.cond,
x=self.cond, x=self.cond,
level=0, level=0))
main_program=self.helper.main_program,
startup_program=self.helper.startup_program))
return rlist return rlist
...@@ -821,12 +787,8 @@ class DynamicRNN(object): ...@@ -821,12 +787,8 @@ class DynamicRNN(object):
IN_RNN = 1 IN_RNN = 1
AFTER_RNN = 2 AFTER_RNN = 2
def __init__(self, name=None, main_program=None, startup_program=None): def __init__(self, name=None):
self.helper = LayerHelper( self.helper = LayerHelper('dynamic_rnn', name=name)
'dynamic_rnn',
name=name,
main_program=main_program,
startup_program=startup_program)
self.status = DynamicRNN.BEFORE_RNN self.status = DynamicRNN.BEFORE_RNN
self.lod_rank_table = None self.lod_rank_table = None
self.max_seq_len = None self.max_seq_len = None
...@@ -880,8 +842,7 @@ class DynamicRNN(object): ...@@ -880,8 +842,7 @@ class DynamicRNN(object):
inputs={'X': x, inputs={'X': x,
'RankTable': self.lod_rank_table}, 'RankTable': self.lod_rank_table},
outputs={'Out': input_array}) outputs={'Out': input_array})
return array_read( return array_read(array=input_array, i=self.step_idx)
array=input_array, i=self.step_idx, **self.helper.to_kwargs)
@contextlib.contextmanager @contextlib.contextmanager
def block(self): def block(self):
...@@ -892,32 +853,18 @@ class DynamicRNN(object): ...@@ -892,32 +853,18 @@ class DynamicRNN(object):
self.status = DynamicRNN.IN_RNN self.status = DynamicRNN.IN_RNN
with self.while_op.block(): with self.while_op.block():
yield yield
increment( increment(x=self.step_idx, value=1.0, in_place=True)
x=self.step_idx,
value=1.0,
in_place=True,
**self.helper.to_kwargs)
for new_mem, mem_array in self.mem_link: for new_mem, mem_array in self.mem_link:
array_write( array_write(x=new_mem, i=self.step_idx, array=mem_array)
x=new_mem,
i=self.step_idx, less_than(x=self.step_idx, y=self.max_seq_len, cond=self.cond)
array=mem_array,
**self.helper.to_kwargs)
less_than(
x=self.step_idx,
y=self.max_seq_len,
cond=self.cond,
**self.helper.to_kwargs)
self.status = DynamicRNN.AFTER_RNN self.status = DynamicRNN.AFTER_RNN
for each_array in self.output_array: for each_array in self.output_array:
self.outputs.append( self.outputs.append(
array_to_lod_tensor( array_to_lod_tensor(
x=each_array, x=each_array, table=self.lod_rank_table))
table=self.lod_rank_table,
**self.helper.to_kwargs))
def __call__(self, *args, **kwargs): def __call__(self, *args, **kwargs):
if self.status != DynamicRNN.AFTER_RNN: if self.status != DynamicRNN.AFTER_RNN:
...@@ -944,13 +891,9 @@ class DynamicRNN(object): ...@@ -944,13 +891,9 @@ class DynamicRNN(object):
inputs={'X': init, inputs={'X': init,
'I': self.zero_idx}, 'I': self.zero_idx},
outputs={'Out': mem_array}) outputs={'Out': mem_array})
retv = array_read( retv = array_read(array=mem_array, i=self.step_idx)
array=mem_array, i=self.step_idx, **self.helper.to_kwargs)
retv = shrink_memory( retv = shrink_memory(
x=retv, x=retv, i=self.step_idx, table=self.lod_rank_table)
i=self.step_idx,
table=self.lod_rank_table,
**self.helper.to_kwargs)
self.mem_dict[retv.name] = mem_array self.mem_dict[retv.name] = mem_array
return retv return retv
else: else:
......
...@@ -10,8 +10,6 @@ def data(name, ...@@ -10,8 +10,6 @@ def data(name,
dtype='float32', dtype='float32',
lod_level=0, lod_level=0,
type=core.VarDesc.VarType.LOD_TENSOR, type=core.VarDesc.VarType.LOD_TENSOR,
main_program=None,
startup_program=None,
stop_gradient=True): stop_gradient=True):
""" """
Data Layer. Data Layer.
......
...@@ -20,9 +20,7 @@ def fc(input, ...@@ -20,9 +20,7 @@ def fc(input,
param_attr=None, param_attr=None,
bias_attr=None, bias_attr=None,
act=None, act=None,
name=None, name=None):
main_program=None,
startup_program=None):
""" """
Fully Connected Layer. Fully Connected Layer.
...@@ -88,13 +86,7 @@ def fc(input, ...@@ -88,13 +86,7 @@ def fc(input,
return helper.append_activation(pre_activation) return helper.append_activation(pre_activation)
def embedding(input, def embedding(input, size, is_sparse=False, param_attr=None, dtype='float32'):
size,
is_sparse=False,
param_attr=None,
dtype='float32',
main_program=None,
startup_program=None):
""" """
Embedding Layer. Embedding Layer.
...@@ -140,9 +132,7 @@ def dynamic_lstm(input, ...@@ -140,9 +132,7 @@ def dynamic_lstm(input,
gate_activation='sigmoid', gate_activation='sigmoid',
cell_activation='tanh', cell_activation='tanh',
candidate_activation='tanh', candidate_activation='tanh',
dtype='float32', dtype='float32'):
main_program=None,
startup_program=None):
helper = LayerHelper('lstm', **locals()) helper = LayerHelper('lstm', **locals())
size = size / 4 size = size / 4
weight = helper.create_parameter( weight = helper.create_parameter(
...@@ -185,9 +175,7 @@ def gru_unit(input, ...@@ -185,9 +175,7 @@ def gru_unit(input,
weight=None, weight=None,
bias=None, bias=None,
activation='tanh', activation='tanh',
gate_activation='sigmoid', gate_activation='sigmoid'):
main_program=None,
startup_program=None):
""" """
GRUUnit Operator implements partial calculations of the GRU unit as following: GRUUnit Operator implements partial calculations of the GRU unit as following:
...@@ -250,11 +238,7 @@ def gru_unit(input, ...@@ -250,11 +238,7 @@ def gru_unit(input,
return updated_hidden, reset_hidden_pre, gate return updated_hidden, reset_hidden_pre, gate
def linear_chain_crf(input, def linear_chain_crf(input, label, param_attr=None):
label,
param_attr=None,
main_program=None,
startup_program=None):
helper = LayerHelper('linear_chain_crf', **locals()) helper = LayerHelper('linear_chain_crf', **locals())
size = input.shape[1] size = input.shape[1]
transition = helper.create_parameter( transition = helper.create_parameter(
...@@ -280,11 +264,7 @@ def linear_chain_crf(input, ...@@ -280,11 +264,7 @@ def linear_chain_crf(input,
return log_likelihood return log_likelihood
def crf_decoding(input, def crf_decoding(input, param_attr, label=None):
param_attr,
label=None,
main_program=None,
startup_program=None):
helper = LayerHelper('crf_decoding', **locals()) helper = LayerHelper('crf_decoding', **locals())
transition = helper.get_parameter(param_attr.name) transition = helper.get_parameter(param_attr.name)
viterbi_path = helper.create_tmp_variable(dtype=helper.input_dtype()) viterbi_path = helper.create_tmp_variable(dtype=helper.input_dtype())
...@@ -432,9 +412,7 @@ def sequence_conv(input, ...@@ -432,9 +412,7 @@ def sequence_conv(input,
padding=None, padding=None,
bias_attr=None, bias_attr=None,
param_attr=None, param_attr=None,
act=None, act=None):
main_program=None,
startup_program=None):
""" """
This function creates the op for sequence_conv, using the inputs and This function creates the op for sequence_conv, using the inputs and
other convolutional configurations for the filters and stride as given other convolutional configurations for the filters and stride as given
...@@ -477,9 +455,7 @@ def conv2d(input, ...@@ -477,9 +455,7 @@ def conv2d(input,
param_attr=None, param_attr=None,
bias_attr=None, bias_attr=None,
act=None, act=None,
name=None, name=None):
main_program=None,
startup_program=None):
""" """
This function creates the op for a 2-dimensional Convolution. This function creates the op for a 2-dimensional Convolution.
This is performed using the parameters of filters(size, dimensionality etc) This is performed using the parameters of filters(size, dimensionality etc)
...@@ -565,9 +541,7 @@ def pool2d(input, ...@@ -565,9 +541,7 @@ def pool2d(input,
pool_type, pool_type,
pool_stride=None, pool_stride=None,
pool_padding=None, pool_padding=None,
global_pooling=False, global_pooling=False):
main_program=None,
startup_program=None):
""" """
This function adds the operator for pooling in 2 dimensions, using the This function adds the operator for pooling in 2 dimensions, using the
pooling configurations mentioned in input parameters. pooling configurations mentioned in input parameters.
...@@ -613,9 +587,7 @@ def batch_norm(input, ...@@ -613,9 +587,7 @@ def batch_norm(input,
epsilon=1e-05, epsilon=1e-05,
param_attr=None, param_attr=None,
bias_attr=None, bias_attr=None,
data_layout='NCHW', data_layout='NCHW'):
main_program=None,
startup_program=None):
""" """
This function helps create an operator to implement This function helps create an operator to implement
the BatchNorm layer using the configurations from the input parameters. the BatchNorm layer using the configurations from the input parameters.
...@@ -685,7 +657,7 @@ def batch_norm(input, ...@@ -685,7 +657,7 @@ def batch_norm(input,
return helper.append_activation(batch_norm_out) return helper.append_activation(batch_norm_out)
def beam_search_decode(ids, scores, main_program=None, startup_program=None): def beam_search_decode(ids, scores):
helper = LayerHelper('beam_search_decode', **locals()) helper = LayerHelper('beam_search_decode', **locals())
sentence_ids = helper.create_tmp_variable(dtype=ids.dtype) sentence_ids = helper.create_tmp_variable(dtype=ids.dtype)
sentence_scores = helper.create_tmp_variable(dtype=ids.dtype) sentence_scores = helper.create_tmp_variable(dtype=ids.dtype)
...@@ -708,9 +680,7 @@ def conv2d_transpose(input, ...@@ -708,9 +680,7 @@ def conv2d_transpose(input,
filter_size=None, filter_size=None,
padding=None, padding=None,
stride=None, stride=None,
param_attr=None, param_attr=None):
main_program=None,
startup_program=None):
""" """
The transpose of conv2d layer. The transpose of conv2d layer.
......
...@@ -6,12 +6,12 @@ __all__ = [ ...@@ -6,12 +6,12 @@ __all__ = [
] ]
def create_tensor(dtype, name=None, main_program=None, startup_program=None): def create_tensor(dtype, name=None):
helper = LayerHelper("create_tensor", **locals()) helper = LayerHelper("create_tensor", **locals())
return helper.create_variable(name=helper.name, dtype=dtype) return helper.create_variable(name=helper.name, dtype=dtype)
def cast(x, dtype, main_program=None): def cast(x, dtype):
""" """
This function takes in the input with input_dtype This function takes in the input with input_dtype
and casts it to the output_dtype as the output. and casts it to the output_dtype as the output.
...@@ -27,7 +27,7 @@ def cast(x, dtype, main_program=None): ...@@ -27,7 +27,7 @@ def cast(x, dtype, main_program=None):
return out return out
def concat(input, axis, main_program=None, startup_program=None): def concat(input, axis):
""" """
This function concats the input along the axis mentioned This function concats the input along the axis mentioned
and returns that as the output. and returns that as the output.
...@@ -42,7 +42,7 @@ def concat(input, axis, main_program=None, startup_program=None): ...@@ -42,7 +42,7 @@ def concat(input, axis, main_program=None, startup_program=None):
return out return out
def sums(input, out=None, main_program=None, startup_program=None): def sums(input, out=None):
""" """
This function takes in the input and performs the sum operation on it This function takes in the input and performs the sum operation on it
and returns that as the output. and returns that as the output.
...@@ -54,7 +54,7 @@ def sums(input, out=None, main_program=None, startup_program=None): ...@@ -54,7 +54,7 @@ def sums(input, out=None, main_program=None, startup_program=None):
return out return out
def assign(input, output, main_program=None, startup_program=None): def assign(input, output):
helper = LayerHelper('assign', **locals()) helper = LayerHelper('assign', **locals())
helper.append_op( helper.append_op(
type='scale', type='scale',
...@@ -64,12 +64,7 @@ def assign(input, output, main_program=None, startup_program=None): ...@@ -64,12 +64,7 @@ def assign(input, output, main_program=None, startup_program=None):
return output return output
def fill_constant(shape, def fill_constant(shape, dtype, value, out=None):
dtype,
value,
out=None,
main_program=None,
startup_program=None):
""" """
This function creates a tensor , with shape as mentioned in the input and This function creates a tensor , with shape as mentioned in the input and
specified dtype and fills this up with a constant value that specified dtype and fills this up with a constant value that
...@@ -94,9 +89,7 @@ def fill_constant_batch_size_like(input, ...@@ -94,9 +89,7 @@ def fill_constant_batch_size_like(input,
dtype, dtype,
value, value,
input_dim_idx=0, input_dim_idx=0,
output_dim_idx=0, output_dim_idx=0):
main_program=None,
startup_program=None):
helper = LayerHelper("fill_constant_batch_size_like", **locals()) helper = LayerHelper("fill_constant_batch_size_like", **locals())
out = helper.create_tmp_variable(dtype=dtype) out = helper.create_tmp_variable(dtype=dtype)
helper.append_op( helper.append_op(
...@@ -114,7 +107,7 @@ def fill_constant_batch_size_like(input, ...@@ -114,7 +107,7 @@ def fill_constant_batch_size_like(input,
return out return out
def ones(shape, dtype, main_program=None): def ones(shape, dtype):
""" """
This function performs the same function as fill_constant() declared above This function performs the same function as fill_constant() declared above
with the constant value being 1.0. with the constant value being 1.0.
...@@ -122,7 +115,7 @@ def ones(shape, dtype, main_program=None): ...@@ -122,7 +115,7 @@ def ones(shape, dtype, main_program=None):
return fill_constant(value=1.0, **locals()) return fill_constant(value=1.0, **locals())
def zeros(shape, dtype, main_program=None): def zeros(shape, dtype):
""" """
This function performs the same function as fill_constant() declared above This function performs the same function as fill_constant() declared above
with the constant value being 0.0. with the constant value being 0.0.
......
...@@ -10,25 +10,19 @@ def simple_img_conv_pool(input, ...@@ -10,25 +10,19 @@ def simple_img_conv_pool(input,
pool_stride, pool_stride,
act, act,
param_attr=None, param_attr=None,
pool_type='max', pool_type='max'):
main_program=None,
startup_program=None):
conv_out = layers.conv2d( conv_out = layers.conv2d(
input=input, input=input,
num_filters=num_filters, num_filters=num_filters,
filter_size=filter_size, filter_size=filter_size,
param_attr=param_attr, param_attr=param_attr,
act=act, act=act)
main_program=main_program,
startup_program=startup_program)
pool_out = layers.pool2d( pool_out = layers.pool2d(
input=conv_out, input=conv_out,
pool_size=pool_size, pool_size=pool_size,
pool_type=pool_type, pool_type=pool_type,
pool_stride=pool_stride, pool_stride=pool_stride)
main_program=main_program,
startup_program=startup_program)
return pool_out return pool_out
...@@ -42,9 +36,7 @@ def img_conv_group(input, ...@@ -42,9 +36,7 @@ def img_conv_group(input,
conv_with_batchnorm=False, conv_with_batchnorm=False,
conv_batchnorm_drop_rate=None, conv_batchnorm_drop_rate=None,
pool_stride=1, pool_stride=1,
pool_type=None, pool_type=None):
main_program=None,
startup_program=None):
""" """
Image Convolution Group, Used for vgg net. Image Convolution Group, Used for vgg net.
""" """
...@@ -75,31 +67,19 @@ def img_conv_group(input, ...@@ -75,31 +67,19 @@ def img_conv_group(input,
filter_size=conv_filter_size[i], filter_size=conv_filter_size[i],
padding=conv_padding[i], padding=conv_padding[i],
param_attr=param_attr[i], param_attr=param_attr[i],
act=local_conv_act, act=local_conv_act)
main_program=main_program,
startup_program=startup_program)
if conv_with_batchnorm[i]: if conv_with_batchnorm[i]:
tmp = layers.batch_norm( tmp = layers.batch_norm(input=tmp, act=conv_act)
input=tmp,
act=conv_act,
main_program=main_program,
startup_program=startup_program)
drop_rate = conv_batchnorm_drop_rate[i] drop_rate = conv_batchnorm_drop_rate[i]
if abs(drop_rate) > 1e-5: if abs(drop_rate) > 1e-5:
tmp = layers.dropout( tmp = layers.dropout(x=tmp, dropout_prob=drop_rate)
x=tmp,
dropout_prob=drop_rate,
main_program=main_program,
startup_program=startup_program)
pool_out = layers.pool2d( pool_out = layers.pool2d(
input=tmp, input=tmp,
pool_size=pool_size, pool_size=pool_size,
pool_type=pool_type, pool_type=pool_type,
pool_stride=pool_stride, pool_stride=pool_stride)
main_program=main_program,
startup_program=startup_program)
return pool_out return pool_out
...@@ -108,21 +88,13 @@ def sequence_conv_pool(input, ...@@ -108,21 +88,13 @@ def sequence_conv_pool(input,
filter_size, filter_size,
param_attr=None, param_attr=None,
act="sigmoid", act="sigmoid",
pool_type="max", pool_type="max"):
main_program=None,
startup_program=None):
conv_out = layers.sequence_conv( conv_out = layers.sequence_conv(
input=input, input=input,
num_filters=num_filters, num_filters=num_filters,
filter_size=filter_size, filter_size=filter_size,
param_attr=param_attr, param_attr=param_attr,
act=act, act=act)
main_program=main_program,
startup_program=startup_program)
pool_out = layers.sequence_pool( pool_out = layers.sequence_pool(input=conv_out, pool_type=pool_type)
input=conv_out,
pool_type=pool_type,
main_program=main_program,
startup_program=startup_program)
return pool_out return pool_out
...@@ -2,7 +2,7 @@ from collections import defaultdict ...@@ -2,7 +2,7 @@ from collections import defaultdict
import framework import framework
from backward import append_backward_ops from backward import append_backward_ops
from framework import unique_name from framework import unique_name, program_guard
from initializer import Constant from initializer import Constant
from layer_helper import LayerHelper from layer_helper import LayerHelper
from regularizer import append_regularization_ops from regularizer import append_regularization_ops
...@@ -159,34 +159,32 @@ class Optimizer(object): ...@@ -159,34 +159,32 @@ class Optimizer(object):
# Create any accumulators # Create any accumulators
program = loss.block.program program = loss.block.program
self.helper = LayerHelper( with program_guard(program, startup_program):
self.__class__.__name__, self.helper = LayerHelper(self.__class__.__name__)
main_program=program, self._create_accumulators(loss.block,
startup_program=startup_program) [p[0] for p in parameters_and_grads])
self._create_accumulators(loss.block,
[p[0] for p in parameters_and_grads]) optimize_ops = []
for param_and_grad in parameters_and_grads:
optimize_ops = [] if param_and_grad[0].trainable is True and param_and_grad[
for param_and_grad in parameters_and_grads: 1] is not None:
if param_and_grad[0].trainable is True and param_and_grad[ optimize_op = self._append_optimize_op(loss.block,
1] is not None: param_and_grad)
optimize_op = self._append_optimize_op(loss.block, optimize_ops.append(optimize_op)
param_and_grad)
optimize_ops.append(optimize_op) # Returned list of ops can include more ops in addition
# to optimization ops
# Returned list of ops can include more ops in addition return_ops = optimize_ops
# to optimization ops
return_ops = optimize_ops # Get custom finish ops for subclasses
# FIXME: Need to fix this once we figure out how to handle dependencies
# Get custom finish ops for subclasses finish_ops = self._finish_update(loss.block)
# FIXME: Need to fix this once we figure out how to handle dependencies if finish_ops is not None:
finish_ops = self._finish_update(loss.block) return_ops += finish_ops
if finish_ops is not None:
return_ops += finish_ops if self._global_step is not None:
return_ops.append(self._increment_global_step(loss.block))
if self._global_step is not None: return return_ops
return_ops.append(self._increment_global_step(loss.block))
return return_ops
def minimize(self, def minimize(self,
loss, loss,
......
image/ image/
fit_a_line.model/ fit_a_line.model/
tmp tmp
cuda_profiler.txt
...@@ -33,11 +33,10 @@ opts = optimizer.minimize(avg_cost) ...@@ -33,11 +33,10 @@ opts = optimizer.minimize(avg_cost)
accuracy = fluid.evaluator.Accuracy(input=predict, label=label) accuracy = fluid.evaluator.Accuracy(input=predict, label=label)
inference_program = fluid.default_main_program().clone() inference_program = fluid.default_main_program().clone()
test_accuracy = fluid.evaluator.Accuracy( with fluid.program_guard(inference_program):
input=predict, label=label, main_program=inference_program) test_accuracy = fluid.evaluator.Accuracy(input=predict, label=label)
test_target = [avg_cost] + test_accuracy.metrics + test_accuracy.states test_target = [avg_cost] + test_accuracy.metrics + test_accuracy.states
inference_program = fluid.io.get_inference_program( inference_program = fluid.io.get_inference_program(test_target)
test_target, main_program=inference_program)
train_reader = paddle.batch( train_reader = paddle.batch(
paddle.reader.shuffle( paddle.reader.shuffle(
......
...@@ -4,12 +4,7 @@ import paddle.v2.fluid as fluid ...@@ -4,12 +4,7 @@ import paddle.v2.fluid as fluid
from paddle.v2.fluid.layer_helper import LayerHelper from paddle.v2.fluid.layer_helper import LayerHelper
def lstm(x, def lstm(x, c_pre_init, hidden_dim, forget_bias=None):
c_pre_init,
hidden_dim,
forget_bias=None,
main_program=None,
startup_program=None):
""" """
This function helps create an operator for the LSTM (Long Short Term This function helps create an operator for the LSTM (Long Short Term
Memory) cell that can be used inside an RNN. Memory) cell that can be used inside an RNN.
...@@ -20,15 +15,8 @@ def lstm(x, ...@@ -20,15 +15,8 @@ def lstm(x,
c_pre = rnn.memory(init=c_pre_init) c_pre = rnn.memory(init=c_pre_init)
x_t = rnn.step_input(x) x_t = rnn.step_input(x)
before_fc = fluid.layers.concat( before_fc = fluid.layers.concat(input=[x_t, c_pre], axis=1)
input=[x_t, c_pre], after_fc = fluid.layers.fc(input=before_fc, size=hidden_dim * 4)
axis=1,
main_program=main_program,
startup_program=startup_program)
after_fc = fluid.layers.fc(input=before_fc,
size=hidden_dim * 4,
main_program=main_program,
startup_program=startup_program)
dtype = x.dtype dtype = x.dtype
c = helper.create_tmp_variable(dtype) c = helper.create_tmp_variable(dtype)
......
...@@ -5,12 +5,7 @@ import paddle.v2.fluid.nets as nets ...@@ -5,12 +5,7 @@ import paddle.v2.fluid.nets as nets
from paddle.v2.fluid.framework import Program from paddle.v2.fluid.framework import Program
def conv_block(input, def conv_block(input, num_filter, groups, dropouts):
num_filter,
groups,
dropouts,
main_program=None,
startup_program=None):
return nets.img_conv_group( return nets.img_conv_group(
input=input, input=input,
pool_size=2, pool_size=2,
...@@ -20,90 +15,54 @@ def conv_block(input, ...@@ -20,90 +15,54 @@ def conv_block(input,
conv_act='relu', conv_act='relu',
conv_with_batchnorm=True, conv_with_batchnorm=True,
conv_batchnorm_drop_rate=dropouts, conv_batchnorm_drop_rate=dropouts,
pool_type='max', pool_type='max')
main_program=main_program,
startup_program=startup_program)
class TestLayer(unittest.TestCase): class TestLayer(unittest.TestCase):
def test_batch_norm_layer(self): def test_batch_norm_layer(self):
main_program = Program() main_program = Program()
startup_program = Program() startup_program = Program()
images = fluid.layers.data( with fluid.program_guard(main_program, startup_program):
name='pixel', images = fluid.layers.data(
shape=[3, 48, 48], name='pixel', shape=[3, 48, 48], dtype='float32')
dtype='float32', hidden1 = fluid.layers.batch_norm(input=images)
main_program=main_program) hidden2 = fluid.layers.fc(input=hidden1, size=128, act='relu')
hidden1 = fluid.layers.batch_norm( fluid.layers.batch_norm(input=hidden2)
input=images,
main_program=main_program,
startup_program=startup_program)
hidden2 = fluid.layers.fc(input=hidden1,
size=128,
act='relu',
main_program=main_program)
hidden3 = fluid.layers.batch_norm(
input=hidden2,
main_program=main_program,
startup_program=startup_program)
print str(main_program) print str(main_program)
def test_dropout_layer(self): def test_dropout_layer(self):
main_program = Program() main_program = Program()
startup_program = Program() startup_program = Program()
images = fluid.layers.data( with fluid.program_guard(main_program, startup_program):
name='pixel', images = fluid.layers.data(
shape=[3, 48, 48], name='pixel', shape=[3, 48, 48], dtype='float32')
dtype='float32', fluid.layers.dropout(x=images, dropout_prob=0.5)
main_program=main_program)
fluid.layers.dropout(
x=images,
dropout_prob=0.5,
main_program=main_program,
startup_program=startup_program)
# print str(main_program) print str(main_program)
def test_img_conv_group(self): def test_img_conv_group(self):
main_program = Program() main_program = Program()
startup_program = Program() startup_program = Program()
images = fluid.layers.data( with fluid.program_guard(main_program, startup_program):
name='pixel', images = fluid.layers.data(
shape=[3, 48, 48], name='pixel', shape=[3, 48, 48], dtype='float32')
dtype='float32', conv1 = conv_block(images, 64, 2, [0.3, 0])
main_program=main_program, conv_block(conv1, 256, 3, [0.4, 0.4, 0])
startup_program=startup_program)
conv1 = conv_block(images, 64, 2, [0.3, 0], main_program,
startup_program)
conv2 = conv_block(conv1, 256, 3, [0.4, 0.4, 0], main_program,
startup_program)
# print str(main_program) print str(main_program)
def test_elementwise_add_with_act(self): def test_elementwise_add_with_act(self):
main_program = Program() main_program = Program()
startup_program = Program() startup_program = Program()
image1 = fluid.layers.data( with fluid.program_guard(main_program, startup_program):
name='pixel1', image1 = fluid.layers.data(
shape=[3, 48, 48], name='pixel1', shape=[3, 48, 48], dtype='float32')
dtype='float32', image2 = fluid.layers.data(
main_program=main_program, name='pixel2', shape=[3, 48, 48], dtype='float32')
startup_program=startup_program) fluid.layers.elementwise_add(x=image1, y=image2, act='relu')
image2 = fluid.layers.data( print(main_program)
name='pixel2',
shape=[3, 48, 48],
dtype='float32',
main_program=main_program,
startup_program=startup_program)
out = fluid.layers.elementwise_add(
x=image1,
y=image2,
act='relu',
main_program=main_program,
startup_program=startup_program)
# print(main_program)
if __name__ == '__main__': if __name__ == '__main__':
......
...@@ -6,7 +6,7 @@ import paddle.v2.fluid.core as core ...@@ -6,7 +6,7 @@ import paddle.v2.fluid.core as core
import paddle.v2.fluid.executor as executor import paddle.v2.fluid.executor as executor
import paddle.v2.fluid.layers as layers import paddle.v2.fluid.layers as layers
import paddle.v2.fluid.optimizer as optimizer import paddle.v2.fluid.optimizer as optimizer
from paddle.v2.fluid.framework import Program from paddle.v2.fluid.framework import Program, program_guard
from paddle.v2.fluid.io import save_inference_model, load_inference_model from paddle.v2.fluid.io import save_inference_model, load_inference_model
...@@ -16,35 +16,18 @@ class TestBook(unittest.TestCase): ...@@ -16,35 +16,18 @@ class TestBook(unittest.TestCase):
init_program = Program() init_program = Program()
program = Program() program = Program()
x = layers.data(
name='x', with program_guard(program, init_program):
shape=[2], x = layers.data(name='x', shape=[2], dtype='float32')
dtype='float32', y = layers.data(name='y', shape=[1], dtype='float32')
main_program=program,
startup_program=init_program) y_predict = layers.fc(input=x, size=1, act=None)
y = layers.data(
name='y', cost = layers.square_error_cost(input=y_predict, label=y)
shape=[1], avg_cost = layers.mean(x=cost)
dtype='float32',
main_program=program, sgd_optimizer = optimizer.SGDOptimizer(learning_rate=0.001)
startup_program=init_program) sgd_optimizer.minimize(avg_cost, init_program)
y_predict = layers.fc(input=x,
size=1,
act=None,
main_program=program,
startup_program=init_program)
cost = layers.square_error_cost(
input=y_predict,
label=y,
main_program=program,
startup_program=init_program)
avg_cost = layers.mean(
x=cost, main_program=program, startup_program=init_program)
sgd_optimizer = optimizer.SGDOptimizer(learning_rate=0.001)
sgd_optimizer.minimize(avg_cost, init_program)
place = core.CPUPlace() place = core.CPUPlace()
exe = executor.Executor(place) exe = executor.Executor(place)
......
...@@ -2,7 +2,7 @@ import unittest ...@@ -2,7 +2,7 @@ import unittest
import paddle.v2.fluid.core as core import paddle.v2.fluid.core as core
import numpy import numpy
import paddle.v2.fluid.layers as layers import paddle.v2.fluid.layers as layers
from paddle.v2.fluid.framework import Program from paddle.v2.fluid.framework import Program, program_guard
from paddle.v2.fluid.executor import Executor from paddle.v2.fluid.executor import Executor
from paddle.v2.fluid.backward import append_backward_ops from paddle.v2.fluid.backward import append_backward_ops
...@@ -118,16 +118,17 @@ class TestCPULoDTensorArrayOps(unittest.TestCase): ...@@ -118,16 +118,17 @@ class TestCPULoDTensorArrayOps(unittest.TestCase):
def main(self, tensor, expect_array, expect_lod, expect_max_len, level=0): def main(self, tensor, expect_array, expect_lod, expect_max_len, level=0):
place = self.place() place = self.place()
program = Program() program = Program()
x = layers.data(name='x', shape=[10], main_program=program) with program_guard(program):
x.persistable = True x = layers.data(name='x', shape=[10])
table = layers.lod_rank_table(x, level=level, main_program=program) x.persistable = True
max_len = layers.max_sequence_len(table, main_program=program) table = layers.lod_rank_table(x, level=level)
max_len.persistable = True max_len = layers.max_sequence_len(table)
array = layers.lod_tensor_to_array(x, table, main_program=program) max_len.persistable = True
array.persistable = True array = layers.lod_tensor_to_array(x, table)
array.persistable = True
result = layers.array_to_lod_tensor(array, table, main_program=program)
result.persistable = True result = layers.array_to_lod_tensor(array, table)
result.persistable = True
exe = Executor(place) exe = Executor(place)
scope = core.Scope() scope = core.Scope()
exe.run(program, feed={'x': tensor}, scope=scope) exe.run(program, feed={'x': tensor}, scope=scope)
...@@ -160,19 +161,16 @@ class TestCPULoDTensorArrayOpGrad(unittest.TestCase): ...@@ -160,19 +161,16 @@ class TestCPULoDTensorArrayOpGrad(unittest.TestCase):
place = core.CPUPlace() place = core.CPUPlace()
program = Program() program = Program()
x = layers.data( with program_guard(program):
name='x', x = layers.data(
shape=[1], name='x', shape=[1], dtype='float32', stop_gradient=False)
dtype='float32', table = layers.lod_rank_table(x, level=0)
main_program=program, array = layers.lod_tensor_to_array(x, table)
stop_gradient=False) result = layers.array_to_lod_tensor(array, table)
table = layers.lod_rank_table(x, level=0, main_program=program)
array = layers.lod_tensor_to_array(x, table, main_program=program)
result = layers.array_to_lod_tensor(array, table, main_program=program)
mean = layers.mean(x=result, main_program=program) mean = layers.mean(x=result)
append_backward_ops(mean) append_backward_ops(mean)
tensor = core.LoDTensor() tensor = core.LoDTensor()
tensor.set(numpy.arange(10).reshape(10, 1).astype('float32'), place) tensor.set(numpy.arange(10).reshape(10, 1).astype('float32'), place)
......
import paddle.v2.fluid.layers as layers import paddle.v2.fluid.layers as layers
from paddle.v2.fluid.framework import Program from paddle.v2.fluid.framework import Program, program_guard, default_main_program, default_startup_program
from paddle.v2.fluid.executor import Executor from paddle.v2.fluid.executor import Executor
from paddle.v2.fluid.optimizer import MomentumOptimizer from paddle.v2.fluid.optimizer import MomentumOptimizer
import paddle.v2.fluid.core as core import paddle.v2.fluid.core as core
...@@ -10,44 +10,42 @@ import numpy as np ...@@ -10,44 +10,42 @@ import numpy as np
class TestMNISTIfElseOp(unittest.TestCase): class TestMNISTIfElseOp(unittest.TestCase):
def test_raw_api(self): def test_raw_api(self):
kwargs = {'startup_program': Program(), 'main_program': Program()} prog = Program()
image = layers.data(name='x', shape=[784], dtype='float32', **kwargs) startup_prog = Program()
with program_guard(prog, startup_prog):
image = layers.data(name='x', shape=[784], dtype='float32')
label = layers.data(name='y', shape=[1], dtype='int64', **kwargs) label = layers.data(name='y', shape=[1], dtype='int64')
limit = layers.fill_constant_batch_size_like( limit = layers.fill_constant_batch_size_like(
input=label, dtype='int64', shape=[1], value=5.0, **kwargs) input=label, dtype='int64', shape=[1], value=5.0)
cond = layers.less_than(x=label, y=limit)
true_image, false_image = layers.split_lod_tensor(
input=image, mask=cond)
cond = layers.less_than(x=label, y=limit, **kwargs) true_out = layers.create_tensor(dtype='float32')
true_image, false_image = layers.split_lod_tensor( true_cond = layers.ConditionalBlock([true_image])
input=image, mask=cond, **kwargs)
true_out = layers.create_tensor(dtype='float32', **kwargs) with true_cond.block():
true_cond = layers.ConditionalBlock([true_image], **kwargs) hidden = layers.fc(input=true_image, size=100, act='tanh')
prob = layers.fc(input=hidden, size=10, act='softmax')
layers.assign(input=prob, output=true_out)
with true_cond.block(): false_out = layers.create_tensor(dtype='float32')
hidden = layers.fc(input=true_image, size=100, act='tanh', **kwargs) false_cond = layers.ConditionalBlock([false_image])
prob = layers.fc(input=hidden, size=10, act='softmax', **kwargs)
layers.assign(input=prob, output=true_out, **kwargs)
false_out = layers.create_tensor(dtype='float32', **kwargs) with false_cond.block():
false_cond = layers.ConditionalBlock([false_image], **kwargs) hidden = layers.fc(input=false_image, size=200, act='tanh')
prob = layers.fc(input=hidden, size=10, act='softmax')
layers.assign(input=prob, output=false_out)
with false_cond.block(): prob = layers.merge_lod_tensor(
hidden = layers.fc(input=false_image, in_true=true_out, in_false=false_out, mask=cond, x=image)
size=200, loss = layers.cross_entropy(input=prob, label=label)
act='tanh', avg_loss = layers.mean(x=loss)
**kwargs)
prob = layers.fc(input=hidden, size=10, act='softmax', **kwargs)
layers.assign(input=prob, output=false_out, **kwargs)
prob = layers.merge_lod_tensor( optimizer = MomentumOptimizer(learning_rate=0.001, momentum=0.9)
in_true=true_out, in_false=false_out, mask=cond, x=image, **kwargs) optimizer.minimize(avg_loss, startup_prog)
loss = layers.cross_entropy(input=prob, label=label, **kwargs)
avg_loss = layers.mean(x=loss, **kwargs)
optimizer = MomentumOptimizer(learning_rate=0.001, momentum=0.9)
optimizer.minimize(avg_loss, kwargs['startup_program'])
train_reader = paddle.batch( train_reader = paddle.batch(
paddle.reader.shuffle( paddle.reader.shuffle(
...@@ -57,7 +55,7 @@ class TestMNISTIfElseOp(unittest.TestCase): ...@@ -57,7 +55,7 @@ class TestMNISTIfElseOp(unittest.TestCase):
place = core.CPUPlace() place = core.CPUPlace()
exe = Executor(place) exe = Executor(place)
exe.run(kwargs['startup_program']) exe.run(startup_prog)
PASS_NUM = 100 PASS_NUM = 100
for pass_id in range(PASS_NUM): for pass_id in range(PASS_NUM):
for data in train_reader(): for data in train_reader():
...@@ -65,7 +63,7 @@ class TestMNISTIfElseOp(unittest.TestCase): ...@@ -65,7 +63,7 @@ class TestMNISTIfElseOp(unittest.TestCase):
y_data = np.array(map(lambda x: x[1], data)).astype("int64") y_data = np.array(map(lambda x: x[1], data)).astype("int64")
y_data = np.expand_dims(y_data, axis=1) y_data = np.expand_dims(y_data, axis=1)
outs = exe.run(kwargs['main_program'], outs = exe.run(prog,
feed={'x': x_data, feed={'x': x_data,
'y': y_data}, 'y': y_data},
fetch_list=[avg_loss]) fetch_list=[avg_loss])
...@@ -75,39 +73,36 @@ class TestMNISTIfElseOp(unittest.TestCase): ...@@ -75,39 +73,36 @@ class TestMNISTIfElseOp(unittest.TestCase):
self.assertFalse(True) self.assertFalse(True)
def test_ifelse(self): def test_ifelse(self):
kwargs = {'startup_program': Program(), 'main_program': Program()} prog = Program()
image = layers.data(name='x', shape=[784], dtype='float32', **kwargs) startup_prog = Program()
with program_guard(prog, startup_prog):
label = layers.data(name='y', shape=[1], dtype='int64', **kwargs) image = layers.data(name='x', shape=[784], dtype='float32')
limit = layers.fill_constant_batch_size_like( label = layers.data(name='y', shape=[1], dtype='int64')
input=label, dtype='int64', shape=[1], value=5.0, **kwargs)
limit = layers.fill_constant_batch_size_like(
cond = layers.less_than(x=label, y=limit, **kwargs) input=label, dtype='int64', shape=[1], value=5.0)
cond = layers.less_than(x=label, y=limit)
ie = layers.IfElse(cond, **kwargs) ie = layers.IfElse(cond)
with ie.true_block(): with ie.true_block():
true_image = ie.input(image) true_image = ie.input(image)
hidden = layers.fc(input=true_image, size=100, act='tanh', **kwargs) hidden = layers.fc(input=true_image, size=100, act='tanh')
prob = layers.fc(input=hidden, size=10, act='softmax', **kwargs) prob = layers.fc(input=hidden, size=10, act='softmax')
ie.output(prob) ie.output(prob)
with ie.false_block(): with ie.false_block():
false_image = ie.input(image) false_image = ie.input(image)
hidden = layers.fc(input=false_image, hidden = layers.fc(input=false_image, size=200, act='tanh')
size=200, prob = layers.fc(input=hidden, size=10, act='softmax')
act='tanh', ie.output(prob)
**kwargs)
prob = layers.fc(input=hidden, size=10, act='softmax', **kwargs) prob = ie()
ie.output(prob) loss = layers.cross_entropy(input=prob[0], label=label)
avg_loss = layers.mean(x=loss)
prob = ie()
loss = layers.cross_entropy(input=prob[0], label=label, **kwargs) optimizer = MomentumOptimizer(learning_rate=0.001, momentum=0.9)
avg_loss = layers.mean(x=loss, **kwargs) optimizer.minimize(avg_loss, startup_prog)
optimizer = MomentumOptimizer(learning_rate=0.001, momentum=0.9)
optimizer.minimize(avg_loss, kwargs['startup_program'])
train_reader = paddle.batch( train_reader = paddle.batch(
paddle.reader.shuffle( paddle.reader.shuffle(
paddle.dataset.mnist.train(), buf_size=8192), paddle.dataset.mnist.train(), buf_size=8192),
...@@ -135,4 +130,5 @@ class TestMNISTIfElseOp(unittest.TestCase): ...@@ -135,4 +130,5 @@ class TestMNISTIfElseOp(unittest.TestCase):
if __name__ == '__main__': if __name__ == '__main__':
unittest.main() # temp disable if else unittest since it could be buggy.
exit(0)
from __future__ import print_function from __future__ import print_function
import unittest import unittest
from paddle.v2.fluid.framework import Program, default_main_program from paddle.v2.fluid.framework import Program, default_main_program, program_guard
import paddle.v2.fluid.layers as layers import paddle.v2.fluid.layers as layers
main_program = default_main_program() main_program = default_main_program()
...@@ -129,13 +129,10 @@ class TestProgram(unittest.TestCase): ...@@ -129,13 +129,10 @@ class TestProgram(unittest.TestCase):
def test_program_clone_with_parameter(self): def test_program_clone_with_parameter(self):
main_program = Program() main_program = Program()
startup_program = Program() startup_program = Program()
kwargs = { with program_guard(main_program, startup_program):
'main_program': main_program, d = layers.data(name='x', shape=[784], dtype='float32')
'startup_program': startup_program hidden = layers.fc(input=d, size=100)
} layers.fc(input=hidden, size=100)
d = layers.data(name='x', shape=[784], dtype='float32', **kwargs)
hidden = layers.fc(input=d, size=100, **kwargs)
layers.fc(input=hidden, size=100, **kwargs)
new_program = main_program.clone() new_program = main_program.clone()
self.assertNotEqual(0, len(new_program.blocks[0].all_parameters())) self.assertNotEqual(0, len(new_program.blocks[0].all_parameters()))
......
...@@ -2,7 +2,7 @@ import unittest ...@@ -2,7 +2,7 @@ import unittest
import paddle.v2.fluid.core as core import paddle.v2.fluid.core as core
import numpy as np import numpy as np
import paddle.v2.fluid.layers as layers import paddle.v2.fluid.layers as layers
from paddle.v2.fluid.framework import Program from paddle.v2.fluid.framework import Program, program_guard
from paddle.v2.fluid.executor import Executor from paddle.v2.fluid.executor import Executor
from paddle.v2.fluid.backward import append_backward_ops from paddle.v2.fluid.backward import append_backward_ops
...@@ -75,26 +75,22 @@ class TestCPULoDTensorArrayOps(unittest.TestCase): ...@@ -75,26 +75,22 @@ class TestCPULoDTensorArrayOps(unittest.TestCase):
level=0): level=0):
place = self.place() place = self.place()
program = Program() program = Program()
x = layers.data(name='x', shape=[1], main_program=program) with program_guard(program):
x.persistable = True x = layers.data(name='x', shape=[1])
x.persistable = True
y = layers.data(name='y', shape=[1], main_program=program) y = layers.data(name='y', shape=[1])
y.persistable = True y.persistable = True
out_true, out_false = layers.split_lod_tensor( out_true, out_false = layers.split_lod_tensor(
input=x, mask=y, level=level, main_program=program) input=x, mask=y, level=level)
out_true.persistable = True out_true.persistable = True
out_false.persistable = True out_false.persistable = True
out = layers.merge_lod_tensor( out = layers.merge_lod_tensor(
in_true=out_true, in_true=out_true, in_false=out_false, mask=y, x=x, level=level)
in_false=out_false,
mask=y,
x=x,
level=level,
main_program=program)
out.persistable = True out.persistable = True
exe = Executor(place) exe = Executor(place)
scope = core.Scope() scope = core.Scope()
...@@ -123,34 +119,21 @@ class TestCPUSplitMergeLoDTensorGrad(unittest.TestCase): ...@@ -123,34 +119,21 @@ class TestCPUSplitMergeLoDTensorGrad(unittest.TestCase):
def test_grad(self): def test_grad(self):
place = core.CPUPlace() place = core.CPUPlace()
program = Program() program = Program()
with program_guard(program):
x = layers.data(
name='x', shape=[1], dtype='float32', stop_gradient=False)
y = layers.data(
name='y', shape=[1], dtype='bool', stop_gradient=False)
x = layers.data( level = 0
name='x',
shape=[1], out_true, out_false = layers.split_lod_tensor(
dtype='float32', input=x, mask=y, level=level)
main_program=program, out = layers.merge_lod_tensor(
stop_gradient=False) in_true=out_true, in_false=out_false, mask=y, x=x, level=level)
y = layers.data( mean = layers.mean(x=out)
name='y',
shape=[1], append_backward_ops(mean)
dtype='bool',
main_program=program,
stop_gradient=False)
level = 0
out_true, out_false = layers.split_lod_tensor(
input=x, mask=y, level=level, main_program=program)
out = layers.merge_lod_tensor(
in_true=out_true,
in_false=out_false,
mask=y,
x=x,
level=level,
main_program=program)
mean = layers.mean(x=out, main_program=program)
append_backward_ops(mean)
tensor = core.LoDTensor() tensor = core.LoDTensor()
tensor.set(np.arange(10).reshape(10, 1).astype('float32'), place) tensor.set(np.arange(10).reshape(10, 1).astype('float32'), place)
......
Markdown is supported
0% .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑!
想要评论请 注册