* Reimplement the check_nan_inf function as check_numerics kernel.

* Remove the cpu implemention to phi.

* Add ifdef for the including of omp.h.

* Move the use of FLAGS_check_nan_inf_level out of header file.

* Implement a common PrintAndThrowError function.

* Fix the error using of __NVCC__, which should be instead with __CUDA_ARCH__.

* Add dependency of phi.

* Polish codes and unittest.

* [dygraph]unify _non_static_mode() in_dygraph_mode() and in_dynamic_mode()

* [dygraph]unify _non_static_mode() in_dygraph_mode() and in_dynamic_mode()

* [dygraph]unify _non_static_mode() in_dygraph_mode() and in_dynamic_mode()

* [dygraph]unify _non_static_mode() in_dygraph_mode() and in_dynamic_mode()

* [dygraph]unify _non_static_mode() in_dygraph_mode() and in_dynamic_mode()

* [dygraph]unify _non_static_mode() in_dygraph_mode() and in_dynamic_mode()

* fixed cyclic reference that caused patial import

* fixed bad change

* fix bad import

* fix bad import

* fix bad import

* fix ut failed caused by change in_dynamic_mode

* fix ut failed caused by change in_dynamic_mode

* fixed usage of in_dynamic_mode() or in_dygraph_mode()

* revert python3 to python in .pre-commit-config.yaml

* fix merge conflicts

* add master gradients on static graph

* add unit test for bf16 master grad static graph

* use float16 as v100 test dtype

* only skip GPU which do not support bf16

* use linear layer to test master grad

* 1.push master grad creation before all optimizer ops; 2.remove useless unittest; 3.use a function to create master grad states

* Allow to switch whether to use promote strategy to choose kernel for O2 training.

* Fix comparing error and add unittest.

* fix embedding model weight type mismatch error

* Update fp16_utils.py

---------
Co-authored-by: NZhang Ting <zhangting_2017@163.com>

* support OD level and skip dynamic loss scaling for bf16

[AMP] Allow to enable multi_precision through paddle.static.amp.decorate and add documents for some apis. (#53012)

* Add document for some apis. test=docs_preview

* Allow to set master_weight in paddle.static.amp.decorate.

* Polish codes and add unittest.

* Refine docs.

* Remove the repetitive function.

* support promote dtype for static amp training

* unify o1 and o2

* update for unittest

* fix op_role

* add use_promote arg

* fix doc

* add promote unittest

* polish unittests

* fix controflow and test

* Implement a common AmpTestBase.

* Support overload of decorate.

* Change the ignore list of flake and fix an error.

* Unify the static amp codes of fp16 and bf16.

* Polish apis and add unittest.

* Add operator stats collecting tools for program.

* Add the check of number of bloat16 operators in unittest.

* Add warning for operator not supported for amp.

* Add testing of BF16 O1 and O2.

* support set master_grad

* move register_hook to auto_cast

* update unittest

* fix fp16 test

* update for review comments

* [AMP] Add python API for collecting operator stats.

* Fix import and polish codes.

* Add more unittest.

* Add doc for the new APIs.