* Add casting initializers for bf16 training

* Changes after review

* Correct test and add comment
Co-authored-by: Njoanna.wozna.intel <joanna.wozna@intel.com>

This is cherry-pick of #32281

* refactor_check_finite_and_scale_npu_kernel

* fix compile

* add alloc_float_status op

* add alloc_float_status op

* add FloatStatus for check_finite_and_unscale

* refine code

* remove unneccessary logic

* refine for fleet

* support ernie trt-int8 for inference

* fix reshape

fix test sync_with_cpp (#32212)

* Support converting the model from fp32 to fp16

* Use the runtime to create the unsupported_fp16_list using in AMP.

* Add more infos about supported ops.

* Add some comments for the function of OpSupportedInfos.

* Fix the unit test of test_multi_precision_fp16_train.

* Use layer to calculate output scale
* add backward for moving_average_abs_max_scale and save output scales to op's attr

* update, test=develop

* Refine saving output scale to infer program

* Fix skip_quant in QAT

* Refine calculating output scale of dygraph qat, test=develop

* Support loading parameters from checkpoint to save quantized model

* Fix the unittest test_moving_average_abs_max_scale_op

* Add unittest of save_quantized_model from checkpoint

* Add comments to explain the function

* fix entry

* fix distributed lookup table fuse case

* fix entry bug at first time

* move entry from paddle.fluid -> paddle.distributed

* fix ut with paddle.enable_static()
Co-authored-by: Nmalin10 <malin10@baidu.com>

* Use correct master weights in AdamW.

* Just modify the master weight.

* Update for CI Coverage.

post quantize support insert fake_quantize_dequantize node before the OPs that will be used in VIS's faceid models (#30659)

test=develop

* Fix the error of save_quantized_model

* add fleet amp.init()

* add unittest for fleet_amp_init

* Collect weight threshold of lstm, test=develop

* skip quantizing ops in cpu inference, test=develop

* Implemented AddQuantDequantPass in imperative quantization.

* Supported LeakyReLU Quantization

* For meeting coverage rate.

* Changed the file name of test of AddQuantDequant

* Implemented more Quantized NoWeightLayers.

* Fix the loss cannot align problem between static and dynamic model quantization, add swish as supported quantized layer in imperative quantization.

* remove noweight_list

* support 2.0 API such as Pool2D and ReLu

* add sparse embedding & load vars for 2.0

Change-Id: I36b59ed5f015189dc9d9d2e34a9357722d369f1b

* fix hdfs gloo

Change-Id: Ia84d579053720ad804183e54c9a04b4f031c79c6

* fix gloo hdfs

Change-Id: I5ab982fd483cddc10adcdef0b8aa83aca976cb9e

* move loadvar/sparse embedding from incubute to static

Change-Id: I57081d3545ad2efab78c72420d2162c0eacaf3a0

* add cast ops before and after unsupported fp16 ops.

* Keep partial net in FP32 pattern.

* Support check_finite_and_unscale and update_loss_scaling for FP16 calculation mode.

* Add fp16 support for adam op.

* add multi precision attr for adam.

* Fix the bug of test_multi_precision_fp16_train UT.

* Code format for CI.

* Fix the redefine error about MPTypeTrait on windows.

* fix bugs of the _create_accumulators func in Momentum.

* fix bug when inserting post cast op.

* Add the update_loss_scaling op in allow_set of UnusedVarCheck.

* Update for ci coverage.

* Add some doc for OptimizerWithMixedPrecision.

* Fix the code style.

* Imporve the doc of `amp_init`.

* Change for fp16 testing if users have the infer program defined in separate way.

* Quantization supports 2.0 APIs

* Fix the error of save_quantized_model