* add new save load interface; test=develop

* add new save interface; test=develop

* add save load interface ;

* fix save load error;

* fix dygraph set dict bug;

* add save load unit test; test=develop

* fix test_imperative_optimizer bug; test=develop

* fix unitest optimizer bug; test=develop

* fix code coverage; test=develop

* fix converage; test=develop

* add document for apis; test=develop

* fix unitest error; test=develop

* fix save load unit test error; test=develop

* fix error message; test=develop

* change set_parameter set_optimizer to save_dygraph; test=develop

* add load_graph check; test=develop

* fix api spec; test=develop

* opt FC

* opt rest of dygraph.nn

* new param shape check and unittest

* add kwargs for Layer

* add new set_value api

* use property decorator

* update API.spec, test=develop

* use UserList, separate gettersetters, test=develop

* update test_custom_layer_with_kwargs, test=develop

* fix UserList compatibility, test=develop

* fix UserList compatibility, test=develop

* keep FC._w, test=develop

* add unittests, Conv3D bug fix, test=develop

* clean code, test=develop

* fix dygraph guard in unittest, test=develop

* add property setters, remove unused param in tracer, test=develop

* tracer none check, test=develop

* merge, test=develop

* refine, test=develop

* bug fix in  prelu and conv3d_transpose, test=develop

* rm __set__, test=develop

* set tensor value instead of assign op

* fix property setter call, test=develop

* fix api.spec, test=develop

* fix doc sample, test=develop

* test=develop, fix docker with paddle nccl problem

* test=develop, Add Variable api and refine dygraph related API

* test=develop, Add Variable api and refine dygraph related API

* test=develop, refine test for new api and error info

* test=develop, refine error info and test_layers

* test=develop, add API.spec

* test=devleop, fix to_string python2 and python3 compat error and refien doc

* test=devleop, add API spec

* test=devleop, update API spec

* test=devleop, update API spec

* test=develop, invoke ci

* test=develop, fix example code

* test=develop, update API spec

* test=develop, add compat test and fix inplace campat dict error

* Add fp16 support for dygraph
test=develop

* Add unit test
test=develop

* update dygraph api-doc and backward api-doc, test=develop

* update dygraph api-doc and backward api-doc, update api.spec, test=develop

* update dygraph api-doc and backward api-doc, update api.spec, test=develop

* update API.spec, test=develop

* delete useless raw variables in Conv2D, test=develop

* adjust the vars number in test_graph_wrapper to pass unittest, test=develop

* refactor dygraph,test=develop

* fix failed unittest,test=develop

* polish code,test=develop

* check windows ci error,test=develop
try to fix windows ci error by np.allclose,test=develop

* polish vlog and profiler, test=develop

* try to fix preceding ops order,test=develop

* test transformer in windows ci, test=develop

* use python c-api to speed up tracer.trace,test=develop

* test=develop, fix docker with paddle nccl problem

* test=develop, add ut for debug string and gradient_accumulator

* test=develop, add tests for layer/gradient_accumulator/prepared_op

* test=develop, fix complie error for test_prepared_op

* test=develop, add more ut for dygraph

* test=develop, create API.spec for dygraph api change

* test=develop, refoctor name to make it easier to understand

* test=develop, refoctor name to make it easier to understand

* test=develop, fix multi-gpu failed problem , add Tracer tests, change PADDLEENFORCE to PADDLEENFORCE_EQ

* test=develop, fix ut failed on parallel se-resnext

* test=develop, change one more PADDLE_ENFORCE

* fix batchnorm api param: data_layout

* fix batchnorm data_layout param; test=develop

* fix con2d transpose bias by create and init it in build_onee

* fix API spec

* test=develop, invoke ci

* fix bias_attr and act has no effect error on layer norm, conv2dTranpose, billinearTensorProduct, sequece_conv. fix original_mode not used error on GRUunit. fix sample_weight not set error on NCE. Add ut for all thoese layer

* test=develop, change success standard for conv2dTranspose

* test=develop, fix test_layers to invoke some error branch

* test=develop, fix sample code

* test=develop, fix BilinearTensorProduct failed in dygraph mode

* test=develop, fix test_layers segment fault error

This PR fix problem with issue #18096 , which raise an error for user to specify the error about load dir is wrong

* add default getItem

* test=develop, fix has_attr disabled error in Layer

* test=develop, fix GroupNorm and deepcf bug on attrs

* update parallel.py
test=develop

* update parallel.py
test=develop

remove dygraph.enable from __all__
hidden dygraph. profiler
add doc to dygraph. no_grad

* test=develop, fix docker with paddle nccl problem

* test=develop, hide no_support api and add ut for it

Fix dygraph show style for FluidDoc.

* fix dygraph mem leak, test=develop

* polish msg, test=develop

fix doc: conv3d, layer norm, bilineartensorproduct, conv2dtranspose, conv3dtranspose,rowconv, groupnorm, spectralnorm, treeconv

hidden api: sequenceConv

* add api commet; test=develop

* fix fc dtype bug; test=develop

* remove float32 in default parameter; test=develop

* fix exmpale bug; test=develop

* fix build once; test=develop

* fix num_chanels bug; test=develop

* fix install check failed bug; test=develop

* WIP

* WIP

* test=develop, add api doc and example code for dygraph

* test=develop, refine api

* test=develop, fix bug when error occured on save_persistable with no optimizer

* test=develop, refine waring

* test=develop, refine example code and comments

* test=develop, add dygraph_not_support and refine ocr

* test=develop, shrink name of dygraph_not_support

* add import, test=develop

* fix fill_constant

* fix deconv

* add trainable_statist for bn in dygraph

* save optimizer related vars in dygraph

* test=develop, add optimizer save and load

* test=develop, add optimizer save and load

* test=develop, merge code and add multi-optimizer save and load

* test=develop, fix test_imperative_checkpoint

* test=develop, fix include error

* test=develop, fix include error

* test=develop, renew api spec

* test=develop, refine code

* test=develop, set default value for checkpoint

* test=develop, fix ci error

* test=develop, change API.spec and make api more readable

* test=develop, refine version and time stamp

* test=develop, add example code and refine code

* test=develop, refine doc

* test=develop, change version

* use python api to reduce python time cost, test=develop

* fix travis ci, test=develop

* fix Py_None error,test=develop

* for debug

* test=develop, memory optimize for dygraph using shared_ptr

* test=develop, fix travis ci showed error

* test=develop, fix bug for recurrent usage of varbase

* test=develop, init varbase when it need to be Add

* add no_grad decorator to dygraph, test=develop

* add unittest,test=develop

 use numpy assignment to restore parameters at build-once

test=develop

* This PR adds broadcast for multi-process. And it could be used in dynamic graph to broadcast parameters.

* test=develop, fix con2d with no bias

* test=develop, fix conv_2d with no bias will have null shape

* add var grad hook test=develop