* [Kunlun] Mofify some legacy code on distributed training

There were limitations on XPUs before, such as concat/split is not
supported, and c_broadcast only support fp32. These limitations are
lifted recently.

Multi-device profiling on XPU will also be supported by this PR.
Without this PR, a hanging broadcast will be issued by devices that
enables profiling, eventually lead to kernel timeout error.

* fix typo

Signed-off-by: Njjyaoao <jjyaoao@126.com>

Signed-off-by: Njjyaoao <jjyaoao@126.com>

* add paddle.static.setitem

* add some help doc

* support setitem

* support machanism

* add more unittest

* remove usless code

* raise error in static setitem

* fix d2s UT

* remove static only for both-used code

* fix bool set_value in static, fix set_value_op UT

* fix unittests

* [May case some error]: remove inplace-version check

* add two test case for dy2st

* fix function in vision

* fix dy2st setitem support, refine UT case

* fix slice in static_mode

* add ParametersMap

* remove pop

* modify place

* [fix]: variable is also a tensor

* rewrite some ut & remove slicetransformer in dy2st

* solve error in static-mode

* fix ut

* return a result for set_array_write

* fix test_set_value_op_xpu

* code is different in dynamic / static mode

---------
Co-authored-by: NAurelius84 <zhangliujie@baidu.com>
Co-authored-by: NNotHaozi <zhangmenghao@baidu.com>

* add_example_for_lazygurd

* fix

* batch add inpalce api

* fix inplace fn generate

* add test for  new inpalce api

* fix typro

* fix typro

* fix typro

* fix test error

* fix atan2

* remove atan2

* auto genereate inpalce api

* fix inplace generate fn error

* fix windows error

* fix test error

* fix test error

* fix windows ci error

* fix test error

* fix test_error

* fix test error

* fix eigen aliasing error in inplace

* remove elementwise_pow inplace

* fix doc error

* fix test error

* fix new ir concat op bug

* fix bug

* using add_n_with_kernel instead of add_n impl

* fix pd_op yaml bug

* fix bug

* [Add]Add Xdoctester

* [Fix]fix beta docstring

* [Doctest]change dirichlet docstring

* [Doctest]change gumbel docstring

* [Doctest]change bernoulli docstring

* [Doctest]change categorical docstring

* [Doctest]change ops.py docstring

* [Doctest]change conv docstring

* [Doctest]change distance docstring, test=docs_preview

* [Change]add ref

* [Change]patch xdoctest debug

* add more data type for all/any.

* remove xpu fix.

* add test unit.

* fix typename name.

* fix output data type.

* distribute best cfg

* adapt to multi args transmission

* update metric extracting

* fix bugs of prune and reading log

* fix time default value

* remove time record

* adjust the order of searching dim

* fix prune bugs

* fix adding cfg bug

* fix multi nodes bug

* reset status

* remove alarm and set logdir

* deepcopy ctx

* change alarm

* fix restart bug

* add exit

* best no need alarm

* add warmup time

* Support nvprof for auto parallel

* Fix CI errors

* Fix CI errors

* [prim]support fp16 for instance_norm and instance_norm_grad

* support fp16 and bfp16 dtype for instance_norm prim rules

* fix new ir test

---------
Co-authored-by: Ncxxly <chenxx_id@163.com>

* add phi operator c_concat and ut

* update create_var use

* update copyright

* Support AMP program for onnx QAT API

* Integrate QAT into distributed optimizer

* Reduce the size of test data and increase time limit

* Use logger and reduce time limit of unittests

* Rename and move unittest into fleet test

* Test qat_init API

* fix protobuf problem

* fix protobuf problem

* support sharding parallel

* fix name

* fix

* update

* test amp for sharding

---------

Co-authored-by: pangengzheng <pangengzheng.baidu.com>

* DOCS: Adding imformation about datatype in math.py

* replaced uint16 with bfloat16.

* format correction

* variable names adjustment

* variable names adjustment, name-->type, value-->sub_program

replace the AdagradOptimizer 、adamaxOptimizer、AdadeltaOptimizer、RMSPropOptimizer、LambOptimizer and Momentum (#54152)

* replace the AdadeltaOptimizer with Adadelta

* replace the RMSPropOptimizer with RMSProp

* replace the LambOptimizer with lamb

* replace the momentum in contrib/optimizer.py with Momentum in python/paddle/optimizer/momentum.py

* fix bug

* fix bug

* fix bug

* fix bug of Lamp

* fix bug of Lamp

* fix bug of import

* replace the AdamaxOptimizer with Admax and change the optimizer base for AdagradOptimizer

* fix bug

* fix bug

* Update optimizer.py

* fix bug

* fix bug

* add rmsnorm kernel
* add static graph test
* fix round type
* use alignas to avoid msvc compile error
* remove redundant headerfile to avoid rocm compile error
* fix rocm compile not found cub
* Add document

* rename weight_only/llm.int8

* [尝试] 给tensor增加一个属性, 这个属性是一个定值 1

* 暴露gradnode 并构建gradnode新的方法(用来测试)进行暴露给python python端可以访问

* 开发grad_fn、next_functions两个API 并暴露到python端- 做一些规范化处理

* 增加一个单元测试

* 优化 code-style

* [jit] add copy-from; test=document_fix

* [jit] add copy-from; test=document_fix

* fix TracedLayer

* remove the extend_optimizer_with_weight_decay function

* Update __init__.py

* fix bug

* fix bug

* [autograd] add copy-from; test=document_fix

* [autograd] add copy-from; test=document_fix

* fix