* distribute best cfg

* adapt to multi args transmission

* update metric extracting

* fix bugs of prune and reading log

* fix time default value

* remove time record

* adjust the order of searching dim

* fix prune bugs

* fix adding cfg bug

* fix multi nodes bug

* reset status

* remove alarm and set logdir

* deepcopy ctx

* change alarm

* fix restart bug

* add exit

* best no need alarm

* add warmup time

* Support nvprof for auto parallel

* Fix CI errors

* Fix CI errors

* [prim]support fp16 for instance_norm and instance_norm_grad

* support fp16 and bfp16 dtype for instance_norm prim rules

* fix new ir test

---------
Co-authored-by: Ncxxly <chenxx_id@163.com>

* add phi operator c_concat and ut

* update create_var use

* update copyright

* Support AMP program for onnx QAT API

* Integrate QAT into distributed optimizer

* Reduce the size of test data and increase time limit

* Use logger and reduce time limit of unittests

* Rename and move unittest into fleet test

* Test qat_init API

* fix protobuf problem

* fix protobuf problem

* support sharding parallel

* fix name

* fix

* update

* test amp for sharding

---------

Co-authored-by: pangengzheng <pangengzheng.baidu.com>

* DOCS: Adding imformation about datatype in math.py

* replaced uint16 with bfloat16.

* format correction

* variable names adjustment

* variable names adjustment, name-->type, value-->sub_program

replace the AdagradOptimizer 、adamaxOptimizer、AdadeltaOptimizer、RMSPropOptimizer、LambOptimizer and Momentum (#54152)

* replace the AdadeltaOptimizer with Adadelta

* replace the RMSPropOptimizer with RMSProp

* replace the LambOptimizer with lamb

* replace the momentum in contrib/optimizer.py with Momentum in python/paddle/optimizer/momentum.py

* fix bug

* fix bug

* fix bug

* fix bug of Lamp

* fix bug of Lamp

* fix bug of import

* replace the AdamaxOptimizer with Admax and change the optimizer base for AdagradOptimizer

* fix bug

* fix bug

* Update optimizer.py

* fix bug

* fix bug

* add rmsnorm kernel
* add static graph test
* fix round type
* use alignas to avoid msvc compile error
* remove redundant headerfile to avoid rocm compile error
* fix rocm compile not found cub
* Add document

* rename weight_only/llm.int8

* [尝试] 给tensor增加一个属性, 这个属性是一个定值 1

* 暴露gradnode 并构建gradnode新的方法(用来测试)进行暴露给python python端可以访问

* 开发grad_fn、next_functions两个API 并暴露到python端- 做一些规范化处理

* 增加一个单元测试

* 优化 code-style

* [jit] add copy-from; test=document_fix

* [jit] add copy-from; test=document_fix

* fix TracedLayer

* remove the extend_optimizer_with_weight_decay function

* Update __init__.py

* fix bug

* fix bug

* [autograd] add copy-from; test=document_fix

* [autograd] add copy-from; test=document_fix

* fix

* add clip_grad_value_ api

* add test for ClipGradByValue

* typo fix

* refine and modify clip_grad_norm_

* no_grad

* clip_

* remove g=p.grad

* bug: AssertionError: When Variable is used as the condition of if/while , Variable can only contain one element.

* fix prim add fill_any_like bug

* polish code

* remove allreduce before c_allgather

* update reshard insert_fill_constant_op func

* insert_fill_constant_op add shape arg

This reverts commit 15c87528.

* [Fix]fix cleandoc with a first blank line

* [Fix]fix metrics.py code-block

* [Fix]fix metrics.py code-block indent

* add lerp bf16 support

* fix bug

* Update test_lerp_op.py

modify the input dtype

* modify the test_lerp_op.py

* Update test_lerp_op.py

* fix bug of import

* add user_defined_grads

* Update test_lerp_op.py

* fix bug of grad

* fix bug of grad

* fix bug of grad

* add the check for bfloat16 dtype

* add linear_compress API

replace the PolynomialDecay、NoamDecay、LinearLrWarmup、ReduceLROnPlateau in fluid with 2.0 version (#54806)

* remove the ReduceLROnPlateau in fluid

* fix bug

* remove the PolynomialDecay in fluid

* remove the LinearLrWarmup in fluid

* fix bug

* remove the NoamDecay in fluid

* fix bug

* fix bug

* fix bug