* remove additional warnning in layer.to

* remove additional warnning in layer.to

* remove additional warnning in layer.to

* remove additional warnning in layer.to

* remove additional warnning in layer.to

* [hapi] support dygrapg amp O2

* fix problem of static pure fp16 in hapi

* fix bug

* fix format

* fix ut

* follow comments

* update ut

* update amp save/load

* fix ut

* refine code format

Add Tokenizer related functionalities for Transformer model in order that the process of training and predicting is consistent.

* support the text string as an input Tensor
* support the "VOCAB"unordered_map<wstring, int> as an input Tensor to lookup tokens
* Tokenizer used for BERT. This tokenizer applies an end-to-end, text string to wordpiece tokenization.
* It first applies basic tokenization, followed by wordpiece tokenization.

* remove no_value using var.name

* fix unit test for CI

* fix unit test

* add test case

* fix test case

* add more test case

* catch the generatorfunction and intercept it.

* add test generator

* add test case

* refine the testcase

* fix no_grad context error in train mode when using save/load

* change net to train mode in test case

* add no_sync for parameters sync

* add pipeline for moe

* add attr is_distributed

* refine code

* refine black/white list for pure fp16

* refine amp level

* fix typo

* update tracer._amp_level

* modify format

* modify format

* change time to remove static tempfile

* delete remove_static_file() function

* init functional jacobian api

* finish test with dtype float32

* add float64 test case

* polish code

* use atol=1e-5 with dtype float64

* fix for ci

* set timeout for test_jacobian

* polish API docstring

* modify docstring

* split minimize() to step() + update()

* add unscale and step for grad_scaler

* add unittest

* refine code in minimize

* delete step in loss_scaler

* fix example bug

* refine comment

* refine unittest

* add unittest

* add pure fp16 major function in auto_cast & tracer

* support master weight in dygraph for pure fp16

* check mix dtype of fp16&fp32 for check_finite_and_unscale op

* change pure fp16 funtion name

* refine some bug in auto_cast

* refine auto_cast interface logic

* add param _casted_by_pure_fp16 for class Layer

* support state_dict hook for save model by user appointed dtype in pure_fp16_decorator

* refine pure_fp16_decorator as decorator

* add unittest

* add comment

* add comment

* support recompute

* add comment for auto_cast and decorator

* support to_static_state_dict for paddle.jit.save

* unlimite models num and optimizers num

* add lookup_table in black_list

* fix momentum and layer state_dict

* fix bug in layer state_dict

* fix bug in layer state_dict_helper

* refine unittest

* refine test_momentun_op

* refine interface and some code

* refine amp_decorator interface

* refine pure fp16 interface

* refine master weight interface

* fix no_grad context error in dy2stat

* remove useless comments

* fix error by drop_kids in python

* add test and fix review

* clip op extra information when export model,test=ocr

* rename clip_extra parameter to kwargs in save_inference_model, test=ocr

* support numpy.ndarray index.

* polish code.

* [NPU] fix depthwise_conv2d_grad, test=develop

* remove debug files, test=develop

* Add solutions to PyLayer which is unsupported in DataParallel

* modify note format for parallel.py

* modify docs of dataparallel

* add docs of dp with pylayer

* modify docs format

* modify example format

* change example of dp with pylayer

* add unittest for dp with pylayer

* modify ut

* merge latest codes

* update

* modify for CI-Coverage

* modify text-indent

* dy2stat_error: add revise suggestion for two error cases

* fix test_error

* fix review

This reverts commit ae93d9c2.

* support numpy dtype and polish code of list index.

* polish code.

* modify dy2stat error message in compile time

* fix variable name

* Support append method and initialized value for List in ControlFlow

* polish error msg and en doc

* fix code style

* Support getitem by Bool index

* delete some debug info of bool index

* support the case that the shape of bool index is different from indexed tensor

* support setitem by bool index

* add the unittest for throwing exception

* merge conflict

* add check for int tensor when index is bool

* polish code

* polish code.

* polish code.

* polish code.

* polish code.

* Add no_sync in data parallel for dynamic graph

* modify UT of no_sync

* delete test_parallel_dygraph_dataparallel_no_sync.py

* add test_parallel_dygraph_no_sync.py

* modify run_trainer_with_spawn in UTs

* Add UT of complex control flow in no_sync

* add specific descriptions and notes for no_sync

* check code style

* modify UT's TIMEOUT in CMakeLists.txt

* dygraph amp support param_group

* remove unused code

* fix doc

* add state_dict and load_state_dict and unittest for class GradScaler

* refine unittest for coverage of load_state_dict

* refine comments of code-block

* refine some comments

* refine state_dict code and unittest

* add #require gpu, xpu for GradScaler get/set example code

* add #require gpu, xpu for GradScaler get/set example code

* refine example code

* refine unittest for state_dict

* refine unittest for state_dict

* fix bug of DataLoader in TestGradScalerStateDict

* add flag FLAGS_cudnn_deterministic

* fix paddle.optimizer test=document_fix

* fix paddle.optimizer test=document_fix

* fix bugs in paddle.nn.functional document test=document_fix

* fix bugs in paddle.nn.functional document test=document_fix

* fix bugs in paddle.nn.functional document test=document_fix

* fix bugs in paddle.nn.functional document test=document_fix

* fix nn formula bugs test=document_fix

* fix nn formula bugs test=document_fix

* fix nn formula bugs test=document_fix

* integrated gast library

* integrated gast library

* fix unittest and remove ast2.py

* remove 'gast' from __all__ in __init__.py

* add copyright in other files

* fix copyright

* Support Mixed Precision training in @to_static

* fix block.vars logic

* fix GPU training loss diff

* remove unused code

* Add build_strategy in @to_static to support open pass

* fix os.environ

* add timeout

* disable test_build_strategy on openblas