* support sharding parallel

* fix name

* fix

* update

* test amp for sharding

---------

Co-authored-by: pangengzheng <pangengzheng.baidu.com>

* DOCS: Adding imformation about datatype in math.py

* replaced uint16 with bfloat16.

* format correction

* variable names adjustment

* variable names adjustment, name-->type, value-->sub_program

replace the AdagradOptimizer 、adamaxOptimizer、AdadeltaOptimizer、RMSPropOptimizer、LambOptimizer and Momentum (#54152)

* replace the AdadeltaOptimizer with Adadelta

* replace the RMSPropOptimizer with RMSProp

* replace the LambOptimizer with lamb

* replace the momentum in contrib/optimizer.py with Momentum in python/paddle/optimizer/momentum.py

* fix bug

* fix bug

* fix bug

* fix bug of Lamp

* fix bug of Lamp

* fix bug of import

* replace the AdamaxOptimizer with Admax and change the optimizer base for AdagradOptimizer

* fix bug

* fix bug

* Update optimizer.py

* fix bug

* fix bug

* add rmsnorm kernel
* add static graph test
* fix round type
* use alignas to avoid msvc compile error
* remove redundant headerfile to avoid rocm compile error
* fix rocm compile not found cub
* Add document

* rename weight_only/llm.int8

* [尝试] 给tensor增加一个属性, 这个属性是一个定值 1

* 暴露gradnode 并构建gradnode新的方法(用来测试)进行暴露给python python端可以访问

* 开发grad_fn、next_functions两个API 并暴露到python端- 做一些规范化处理

* 增加一个单元测试

* 优化 code-style

Co-authored-by: NSigureMo <sigure.qaq@gmail.com>

* [jit] add copy-from; test=document_fix

* [jit] add copy-from; test=document_fix

* fix TracedLayer

* remove the extend_optimizer_with_weight_decay function

* Update __init__.py

* fix bug

* fix bug

* remove some pybind interface in cinn to fix compile problem

* modify cmake

* fix cmake

* add log for build cinn whl

* fix ninja for cinn

* fix conflict

* [autograd] add copy-from; test=document_fix

* [autograd] add copy-from; test=document_fix

* fix

* add clip_grad_value_ api

* add test for ClipGradByValue

* typo fix

* refine and modify clip_grad_norm_

* no_grad

* clip_

* remove g=p.grad

* bug: AssertionError: When Variable is used as the condition of if/while , Variable can only contain one element.

* fix prim add fill_any_like bug

* polish code

* remove allreduce before c_allgather

* update reshard insert_fill_constant_op func

* insert_fill_constant_op add shape arg

This reverts commit 15c87528.

* [Fix]fix cleandoc with a first blank line

* [Fix]fix metrics.py code-block

* [Fix]fix metrics.py code-block indent

* add lerp bf16 support

* fix bug

* Update test_lerp_op.py

modify the input dtype

* modify the test_lerp_op.py

* Update test_lerp_op.py

* fix bug of import

* add user_defined_grads

* Update test_lerp_op.py

* fix bug of grad

* fix bug of grad

* fix bug of grad

* add the check for bfloat16 dtype

* add linear_compress API

replace the PolynomialDecay、NoamDecay、LinearLrWarmup、ReduceLROnPlateau in fluid with 2.0 version (#54806)

* remove the ReduceLROnPlateau in fluid

* fix bug

* remove the PolynomialDecay in fluid

* remove the LinearLrWarmup in fluid

* fix bug

* remove the NoamDecay in fluid

* fix bug

* fix bug

* fix bug

[prim] modify multiply_double_grad composite rules to solve paddlescience poiseuille_flow.py problem (#54908)

* modify eular_beam

* modify matmul infermeta

* add test

* modify timeout

* modify mutiply_double nullptr

* modify tanh_triple_gradnode create

* COPY FROM issue of paddle nn

* 移除额外添加的示例，并修改Examplex->Examples

* remove example name of transformer

Co-authored-by: NSigureMo <sigure.qaq@gmail.com>

* fix vpp mp init prob

* triger ci