* add more settings for distributed strategy
Basically, DistributedStrategy has several parts of configurations:
- BuildStrategy: the same as paddle.fluid.BuildStrategy, but the distributed arguments are moved out of BuildStrategy
- ExecutionStrategy: the same as paddle.fluid.ExecutionStrategy
- collective communication configs: nccl_comm_num, hierarchical allreduce and so on
- distributed algorithms: async_update(mainly used in PS), lars, lamb and so on

* Refine Model

1. Take the network (instance of Layer) as the input of Model.
2. Refine set_dict/load_dict of Layer.
3. Refine Input interface, so update code sample about Input

* clean __str__ of VarBase and ParamBase, test=develop

* clean to_string, test=develop

* update unittest, test=develop

* connect stop_gradient with trainable; test=develop

* add value error message; test=develop

* add user defined func test=develop

* update test=develop

* update test=develop

* fix name conflicts test=develop

* add unittest test=develop

* change 2018 to 2020 test=develop

* add comment test=develop

* add comment for function test=develop

* fix details test=develop

* fix details test=develop

* add the support of device index for device_guard.

* add queue_generator_op, dequeue_op, enqueue_op and ut, test=develop

* Compatible int32 and int64 for attr in op slice/strided_slice. test=develop

* Polish code in nn.py  test=develop

* Fix bug: set the same dtype for the inputs of elementwise_add. test=develop

* Convert int32 to int64 in slice op to avoid data overflow. test=develop

* Convert int32 to int64 in strided_slice_op to avoid data overflow. test=develop

* test=develop, test=document_fix

* test=develop, test=document_fix
Co-authored-by: Nswtkiwi <1208425345@qq.com>

* support to train in static

* support to independent decorator

* remove in_dygraph_mode condition in ProgramTranslator

* fix import param_guard and add train/eval test=develop

* Modify into ShareVarsFromScope and rm __all__ in partial_program test=develop

* Add complex layer for transpose & matmul, test=develop

* Tiny fixes in doc, test=develop

* Fix docs, test=develop

* add print transformer & unify print format, test=develop

* remove using of dygraph_to_static_func, test=develop

* remove python stdout capture, test=develop

* fix compatibility problems for PY2, test=develop

* fix detail error, test=develop

* fix type analysis bug, test=develop

* fix print tuple compatible error in PY2, test=develop

* replace get_func to declarative, test=develop

* fix detail bug, test=develop

* fix some detail problems, test=develop

* change visit_call in print transformer, test=develop

* add to_readable_code method, test=develop

* polish doc details, test=develop

* polish doc note, test=develop

* fix unittest error, test=develop

* fix coverage, test=develop

* add print test, test=develop

* add print param, test=develop

* hidden to_readable_code api, test=develop

* remove original tool methods, test=develop

* remove old api using code, test=develop

* add decorator, test=develop

* add fake_interface_only, test=develop

* remove some dygraph_not_support, test=develop

* change dygraph to imperative, test=develop

* Add trainable_statistics in attr for batch_norm
* Unifying behavior of dynamic graph and static graph

* Init complex number neural network, test=develop

* Improve doc writing, test=develop

* Fix elementwise add & sub, test=develop

* Fix elementwise mul act, test=develop

* a) add ut for complex variable; b) remove arg act in elementwise_ops. test=develop

* test=develop
Co-authored-by: Nwuxing03 <wuxing03@baidu.com>

* fix program sample code error, test=develop, test=document_fix

* fix detail error, test=develop, test=document_fix

* fix typo, test=develop, test=document_fix

* API(program_guard) error message enhancement test=develop

* fix Cycle import test=develop

* polish code according comment test=develop

* polish api program error message, test=develop

* fix condition error, test=develop

* fix test prune error, test=develop

* fix coverage problem, test=develop

* Add a function to update FLAGS

test=develop

* Add a function to update FLAGS

test=develop

* expr flags

* Add a function to update FLAGS

test=develop

* distinguish public/private vars, test=develop

* fix windows issues, test=develop

* expr flag

* Add functions to get and set FLAGS

test=develop

* Add functions to get and set FLAGS

test=develop

* Add functions to get and set FLAGS

test=develop

* Add functions to get and set flags

test=develop

* Add functions to get and set FLAGS

test=develop

* Add a function to update FLAGS

test=develop

* Add a function to update FLAGS

test=develop

* Add functions to get and set flags in Paddle

test=develop
Co-authored-by: Nsneaxiy <sneaxiy@126.com>

* prune train program by fetch_list, test=develop

* add unittest for prune, test=develop

* fix pruned feed, test=develop

* support ParallelExecutor and feed prune, test=develop

* add comments, test=develop

* update unittest, test=develop

* update unittests, test=develop

* remove debug code, test=develop

* support cond in clone, test=develop

* support cond in prune, test=develop

* support multiple minimize, test=develop

* support cache, test=develop

* fix _copy_param_info_from, test=develop

* support python2 str, test=develop

* remove debug code, test=develop

* fix bug of caching CompiledProgram, test=develop

* fix multi_device issue, test=develop

* tmp

* support tuple in fetch_list and overriding use_prune, test=develop

* dont use nonlocal in python2, test=develop

* remove nonlocal, test=develop

* code clean, test=develop

* code clean, test=develop

* feed list, test=develop

* test adam, test=develop

* follow comments, test=develop

* reduce duplicate code, test=develop

* update comments, test=develop

* refine grad maker, test=develop

* refactor tracer stage 1, test=develop

* merge develop to solve conflict third times, test=develop

* add fluid.device_guard to specify the device type for Op

* functional dygraph enable/disable
test=develop

* use context manager instead
test=develop

* refine sample code
test=develop

* rename api & expose to fluid
test=develop

* fix sample code
test=develop

* support cond in clone, test=develop

* refine code, test=develop

* refine code, test=develop

* follow comments, test=develop

* refine code, test=develop

* refine code, test=develop

* reuse contain_var, test=develop

* add all_parameters for Program, test=develop

* refine unittest, test=develop

* add example print result, test=develop

* modify fc to linear in sample code, test=develop

* remove FC, test=develop

* remove warnings, test=develop

* drop fluid/imperative/README.md , test=develop

* change fc to linear, test=develop

* polish code style, test=develop

* fix Variable's gradient api in framework.py, test=develop

* remove namescope, test=develop

* append optimize op in the grad block of current block if current block is in control flow. test=develop

* add conditional grad op when optimizer used in control flow. test=develop

* add comment and modify typo. test=develop

* fix append_backward to support control flow. test=develop

* add test. test=develop

* fix copy_var_to_parent_block and conditional_block_grad. test=develop

* fix bug: revert to append conditional_block_grad vars to sub grad block. test=develop

* fix bug: revert to assign var to parent block even if var already is in parent block

* fix bug: consider outputs is empty. test=develop

* move _rename_grad_ out. test=develop

* modify code according to reviews from Huihuang. test=develop

* modify code according to reviews from Jinle. test=develop

* remove params in Tracer object, test=develop

* Repair failed optest, test=develop

* remove build_once & name_scope (Conv2D)
test=develop

* fix unittest
test=develop

* Conv2DTranspose

* Conv3D & Conv3DTranspose
test=develop

* Pool2D & BatchNorm

* Embedding

* LayerNorm

* GRUUnit & NCE

* PRelu

* BilinearTensorProduct

* GroupNorm & SpectralNorm

* TreeConv
test=develop

* fix LayerNorm in transformer unnittest
test=develop

* disable LayerNorm or BatchNorm in multicard
test=develop

* refine Layer.create_parameter api
test=develop

* refine LayerNorm, remove begin_norm_axis param, add normed shape check
test=develop

* LayerNorm bug fix
test=develop

* fix optest,test=develop

* fix optest, test=develop

* fix optest for pass parameter_list when constructing an Optimizer class instance, test=develop

* polish code for better code style, test=develop

* fix se_resnext optest, test=develop

* polish code for better code style, test=develop
Co-authored-by: Nsongyouwei <youwei0314@gmail.com>