* for test_mnist.py

* remove comments

* using type() replace isinstance()

* valid vars for run program OP in io.py

* open test_mnist in eager_gurad for coverage

* Update comment

* [Eager] Support test_imperative_hook_for_layer with _test_eager_guard()

* Polish code name style

* Fix a error name

* Polish code, make it clear and simple

* check amp.decorate and DataParallel

* refine coverage

* fix layer dtype

* refine code

* fix error when set buffer in forward

* add unittest

* refine class name

* refine not framework.in_dygraph_mode() in if

* fix UT error

* add comment

* refine code

* remove useless import

* fix bug of dp in pfp16

* fix topo

* fix bug

* refine code

* refine code

* refine code

* fix error when calling sublayer's non-forward func in dy2stat

* fix circular import using an inelegant way

* deal with parameters

* remove param_guard in __call__

* remove comment

* fix error when jit.load

* rename block var

* remove wrong code

* add unit test

* layer.to api support numpy.dtype and paddle.dtype

* skip the layer to eaxmple code executing for env not support

* optimize dygraph probl

* refine code

* fix convert dtype error

* fix import datafeeder error

* optimizer __call__ to make dygraph faster

* fix return type

* remove cpp Layer

* optimizer __call__ to make dygraph faster

* fix return type

* refine layer to

* delete comment

* refine logic

* refine code

* refine pure_fp16_init

* refine comment

* Layer.to reutrn self

* add device required

* remove additional warnning in layer.to

* remove additional warnning in layer.to

* remove additional warnning in layer.to

* remove additional warnning in layer.to

* remove additional warnning in layer.to

Add Tokenizer related functionalities for Transformer model in order that the process of training and predicting is consistent.

* support the text string as an input Tensor
* support the "VOCAB"unordered_map<wstring, int> as an input Tensor to lookup tokens
* Tokenizer used for BERT. This tokenizer applies an end-to-end, text string to wordpiece tokenization.
* It first applies basic tokenization, followed by wordpiece tokenization.

* add attr is_distributed

* refine code

* refine black/white list for pure fp16

* add pure fp16 major function in auto_cast & tracer

* support master weight in dygraph for pure fp16

* check mix dtype of fp16&fp32 for check_finite_and_unscale op

* change pure fp16 funtion name

* refine some bug in auto_cast

* refine auto_cast interface logic

* add param _casted_by_pure_fp16 for class Layer

* support state_dict hook for save model by user appointed dtype in pure_fp16_decorator

* refine pure_fp16_decorator as decorator

* add unittest

* add comment

* add comment

* support recompute

* add comment for auto_cast and decorator

* support to_static_state_dict for paddle.jit.save

* unlimite models num and optimizers num

* add lookup_table in black_list

* fix momentum and layer state_dict

* fix bug in layer state_dict

* fix bug in layer state_dict_helper

* refine unittest

* refine test_momentun_op

* refine interface and some code

* refine amp_decorator interface

* refine pure fp16 interface

* refine master weight interface

* Support append customize attributes into op_desc in nn.Layer

* fix code style

* support override

* add unittest

* Add param_guard in ParameterList to support @to_static

* Refactor param_guard of @to_static

* fix unittest failed

* add more unittest

* add varbasecopy func to fix the paraBase type bug in layers.to API

* overload _copy_to func for ParamBase

* add xpuplace

* add waiting varbsecopy completion when not blocking

* fix dst_device bug

* modify varbase to shared_ptr

* fix sublayer error with include_sublayers=False

* add ut

* refactor include_sublayers related api

* fix ut

* fix ut of transformer

* fix ut of transformer

* remove useless code

* change sublayer api

* polish code

* add test for include_self=True

* add layer.to api

* add layer.to api

* add layer.to api

* add the doc for Layer.to

* add input type checking

* modify assert and import bug

* format code style

* format code style

* make place support str type

* add SetGradVarBase method to set the gradient after conversion

* modify argument palce to device

* modify argument palce to device

* modify doc of layers.to API

* add xpuplace to device argument

* support save/load single tensor

* compatibility modification according to unnittest

* Some python2.7 don't have 'copyreg' modules

* Handle a syntax error.

* Dealing with compatibility problems on Mac.

* Dealing with compatibility problems on Mac.

* edit unittest to improve coverage.

* Modify the code according to the review comments

* Reduce redundant code.

* support for static graph loading dygraph state_dict

* edit code according to CI

* edit unittest

* edit unnittest

* delete redundant file

* edit code according to Comments

* edit english doc

* edit english doc

* edit English DOC.

* get/set_tensor->get/set_value; return_numpy=False

* get/set_tensor->get/set_value; return_numpy=False

* edit unnittest

* edit unnittest

* polish code.

* support control flow

* supoort sync_parameters_buffers

* fix the bug of sparse embedding

* delete empty line of pybing.cc, test=develop

* fix layers train eval bug, test=develop

* Polish and Optimize the print/repr message of all layer

* fix some code format

fix the state_dict bug for the xpu

* support deepcopy for Layer/Tensor/Paramerbase

* fix some code

This PR fixes several problems in dy2stat for Deoldify model in PaddleGan.

In model, software engineer wrote if x.shape == y.shape, the Tenser shape is a tuple in dygraph so the == returns True/False, but in static graph the == becomes element-wise comparison, which is a different behavior. In this PR we reduce the element-wise comparison result.

If software engineer write computations which uses parameters in hooks, the static graph can loss the parameter variable because we put param_guard at forward of a Layer. In this PR we made param_guard cover pre-hook and post-hook.

In PaddleGan, software engineer calculated some parameter values in __init__ by running some dygraph code. Those code also run during dy2stat. So some variables may be assign as a VarBase (Tensor) first and then Variable, which raised an error. We fixed the bug in this PR by handling the case.

TODO: We just added testcase for the 1. shape comparison. Should add test case for 2. and 3. But since we are chasing 2.0RC, I will do it in the near future PR

* fix eng doc, test=develop

* add import deprecated for layers, test=develop

* add block line for doc generate, test=develop

* remove todo for create_variable, test=develop

* add blank line for doc generate, test=develop

* add blank line for doc generate, test=develop

* [Dy2stat] Support to modify value of buffer tensor

* remove "defaultTest"

* fix name confliction

* rename manual_seed to seed

* rename xxx1d-->xxx1D, xxx2d-->xxx2D, xxx3d-->xxx3D

* rename manual_seed --> seed

* do not rename .cc, .cu and .h file

* rename manual_seed --> seed

* rename manual_seed --> seed

* rename manual_seed --> seed

* rename manual_seed --> seed

* disable_static on doc example code

* donot change manual_seed on generator

* add enable_static on sample code

* convert python/paddle/fluid/layers/nn.py to bak

* fix typo

* fix code style

* fix seed to manual_seed when call functions of Generator()

* fix bug

* modify doc for Layer, test=develop

* modify doc for Layer2, test=develop

* set dtype default value to float32, test=develop

* add example code for paddle.nn.Layer, test=develop

* set create_parameter and create_variable dtype default value to None, test=develop

* modify some example code, tet=develop

* refine, test=develop

* del no ues code, test=develop

* modify doc, example code, args, test=develop

* modify doc, test=develop

* init commit of new save/load

* fix failed unittests

* fix save_load_v2 unittest failed

* fix failed unittest & polish doc

* add tests for coverage

* add more tests & move static apis

* fix example code error

* polish emample code

* fix detail example code problem

* refine error mesg

* update set_dict method name & add aliases

* fix var name error

* fix alias formats

* use set_state_dict in unittest

* add decorator solve compatible problem

* polish decorator

* replace layer set_state_dict by patched method

* remove import monkey path layer

* fix import function error

* add unittest for coverage