* move fluid apis

* fix type error

* remove static exponential_decay

* fix some import error

* remove nn.py

* fix some error

* fix type error

remove the AdamOptimizer、SGDOptimizer、MomentumOptimizer、ModelAverage、LookaheadOptimizer、FtrlOptimizer、DecayedAdagradOptimizer、DpsgdOptimizer in fluid and relocate the ExponentialMovingAverage、PipelineOptimizer、GradientMergeOptimizer and change optimizer base for LarsMomentumOptimizer and RecomputeOptimizer (#55970)

* change the optimizer base for SGDOptimizer

* change the optimizer base for SGDOptimizer

* replace the SGDOptimizer with SGD

* fix bug of sgd

* change the optimizer base for MomentumOptimizer

* fix the remaining tests

* remove the Momentum in fluid/optimizer.py

* fix bug

* fix bug

* fix bug

* fix bug

* Update test_resnet_cinn.py

* Update test_resnet_prim_cinn.py

* fix bug

* fix bug

* fix bug

* remove the ModelAverage in fluid

* remove the LookaheadOptimizer in fluid

* fix bug

* remove AdamOptimizer in fluid

* Update test_image_classification_fp16.py

* fix bug

* relocate the ExponentialMovingAverage in fluid

* restore the static api

* remove the FtrlOptimizer in fluid

* remove the DecayedAdagradOptimizer in fluid

* remove the DpsgdOptimizer in fluid

* fix bug

* fix codestyle

* fix bug

* fix bug

* relocate the PipelineOptimizer

* relocate the GradientMergeOptimizer

* fix bug

* fix bug

* fix bug

* fix doc

* Update __init__.py

* Update test_fleet_qat_meta_optimizer.py

* change optimizer base for LarsMomentumOptimizer

* fix bug

* fix conflict

* fix code-style

* fix sample codes

* fix bug

* fix bug

* fix cinn bug

* fix bug

* fix bug

* Update qat_optimizer.py

* Update __init__.py

* fix bug

* change optimizer base for RecomputeOptimizer

* fix bug

* fix bug

* Update test_imperative_optimizer_v2.py

* Improve GC for pipeline parallel

* Delete print

* open

* update

* make tcp store a global instance

* fix windows compile error

* Update autoparallel DistributedDataLoader

* add places for engine.dataloder()

* use string as key for comm_context_manager

* remove device_id from comm_context

* simple reaplce

* for debug

* fix bugs

* fix some bugs

* del fill_constant_batch_size_like

* Call multiply_ instead of scale_ to avoid multiple DtoH copy.

* Call _squared_l2_norm to calculate grad_clip.

* Fix import error.

Signed-off-by: Njjyaoao <jjyaoao@126.com>

* add shard tensor api

* add DistAttr api

* add unittest for coverage

* fix process mesh sample code

* fix checking error

* fix new launch

* fix ps uit

* [Kunlun] Mofify some legacy code on distributed training

There were limitations on XPUs before, such as concat/split is not
supported, and c_broadcast only support fp32. These limitations are
lifted recently.

Multi-device profiling on XPU will also be supported by this PR.
Without this PR, a hanging broadcast will be issued by devices that
enables profiling, eventually lead to kernel timeout error.

* fix typo

* distribute best cfg

* adapt to multi args transmission

* update metric extracting

* fix bugs of prune and reading log

* fix time default value

* remove time record

* adjust the order of searching dim

* fix prune bugs

* fix adding cfg bug

* fix multi nodes bug

* reset status

* remove alarm and set logdir

* deepcopy ctx

* change alarm

* fix restart bug

* add exit

* best no need alarm

* add warmup time

* Support nvprof for auto parallel

* Fix CI errors

* Fix CI errors

* add phi operator c_concat and ut

* update create_var use

* update copyright

* Support AMP program for onnx QAT API

* Integrate QAT into distributed optimizer

* Reduce the size of test data and increase time limit

* Use logger and reduce time limit of unittests

* Rename and move unittest into fleet test

* Test qat_init API

* fix protobuf problem

* fix protobuf problem

* support sharding parallel

* fix name

* fix

* update

* test amp for sharding

---------

Co-authored-by: pangengzheng <pangengzheng.baidu.com>

* format correction

* variable names adjustment

* variable names adjustment, name-->type, value-->sub_program

replace the AdagradOptimizer 、adamaxOptimizer、AdadeltaOptimizer、RMSPropOptimizer、LambOptimizer and Momentum (#54152)

* replace the AdadeltaOptimizer with Adadelta

* replace the RMSPropOptimizer with RMSProp

* replace the LambOptimizer with lamb

* replace the momentum in contrib/optimizer.py with Momentum in python/paddle/optimizer/momentum.py

* fix bug

* fix bug

* fix bug

* fix bug of Lamp

* fix bug of Lamp

* fix bug of import

* replace the AdamaxOptimizer with Admax and change the optimizer base for AdagradOptimizer

* fix bug

* fix bug

* Update optimizer.py

* fix bug

* fix bug

* remove allreduce before c_allgather

* update reshard insert_fill_constant_op func

* insert_fill_constant_op add shape arg