* [NewIR]Update send recv infermeta and add unittest

* rm new ir flag

* rm fluid api

* skip runing startup prog

* update flag name

* update recv_v2 yaml

* fix conflict

* unittest only for pp

* fix cmakelist

* unittest check precision

* control random

* fix cmakelist

* sep group

* add test

* test ok

* polish

* test cmake script generated

* add sep group

* format

* polish

* polish

* fix static_build for pp

* add mannual_event to support streams across progs

* revert static_build.sh

* fix coverage-ci

* modify the method to name events

* change code according to review

* replace amp.use_pure_fp16 with amp.dtype and amp.level

* old api still use use_pure_fp16

* test_fuse_adamw_pass still use use_pure_fp16

* add test case tuning recompute with amp(float16,o2)

* reset new test case properties TIMEOUT 60

* set smaller value of batch_size and batch_num

* deepcopy dist_context fix _rename_input problem

* fix loss name after cast

* set tuning.enable=True and use engine._tune()

* restore some changes in _rename_input()/_rename_output()

* add self.amp_dtype for _cast_loss() in auto_parallel_amp.py

* fix insert op index in _cast_loss()

[xdoctest][task 184-185] reformat example code with google style in `distributed/auto_parallel/static/*` (#56666)

* [Doctest]fix No.184,185, test=docs_preview

* add env skip

* fix @staticmethod

* fix

* add xdoctest for v2

* fix

[xdoctest][task 224-225] reformat example code with google style in `python/paddle/distributed/fleet` (#56815)

* [Doctest]fix No.224-225, test=docs_preview

* fix the AttributeError

* input.py

* Update python/paddle/nn/functional/input.py

* Update input.py

* Update all_gather.py

* Update all_gather.py

* xdoc

* Apply suggestions from code review

* Update python/paddle/distributed/models/moe/utils.py

* Apply suggestions from code review
Co-authored-by: NNyakku Shigure <sigure.qaq@gmail.com>

* Apply suggestions from code review

* Apply suggestions from code review

* Apply suggestions from code review

---------
Co-authored-by: NNyakku Shigure <sigure.qaq@gmail.com>

* for verify

fluid operator support new comm library

* u

* u

* u

* compatiable new comm library upgrade for c_allgather, c_reduce, c_reduce_scatter and c_scatter.

* Remove useless comments in process_group.py

* Polish code style.

* Fix some problems.

* Remove use fluid api in phi comm_context_manager.

* Add PPADDLE_WITH_CUDA and PADDLE_WITH_NCCL micro judgement.

* Fix bug of HIP architecture.

* Fix some problems.
1. remove useless loggings.
2. Fix conditional compilation for HIP.
3. Fix problems of test_pass_generation_pipeline.py. It calls paddle.distributed.init_parallel_env() at first,
then auto.Engine calls _init_comm(), which will calls process_group.instantiate(). However, init_parallel_env() will call
paddle.distributed.barrier(), it will call CreateNCCLEnvCache and create corresponding NCCLCommContext. But dev_id is not
set, as a result, NCCLCommContext's dev_ctx is not initialized.

* Fix some problems.

* Polish code.

* Polish code.

* Revert compatiable upgrade for communication operators. Their upgrades
will be submitted in another PR.

* Remove StaticTCPStore.

* Remove useless modification.

* Remove useless set_cuda_device_id.

* Polish code.

* Remove fluid header files in phi files.

* Remove useless comments.

* Fix problems of hip arch.

* Fix some problems.

* Polish code.

* Polish code style.

---------
Co-authored-by: hitywt <yuwentao126@126.com>

* weight_norm_hook

* Update weight_norm_hook.py

* Update weight_norm_hook.py

* Update python/paddle/nn/utils/weight_norm_hook.py

* Update python/paddle/nn/utils/weight_norm_hook.py

* Update python/paddle/nn/utils/weight_norm_hook.py
Co-authored-by: NNyakku Shigure <sigure.qaq@gmail.com>

* xdoc

* Apply suggestions from code review

* Apply suggestions from code review

---------
Co-authored-by: NNyakku Shigure <sigure.qaq@gmail.com>

[xdoctest][task 181-183] reformat example code with google style in `sparse/multiary.py`,`distributed/auto_parallel/*` (#56665)

* [Doctest]fix No.181-183, test=docs_preview

* add env skip

[xdoctest][task 213,215-217] reformat example code with google style in `python/paddle/distributed/fleet/base` (#56651)

* [xdoctest][task 213,215-217] reformat example code with google style in python/paddle/distributed/fleet/base

* fix output as comments

* fix eval_loss bug in parallelizer_v2.py

* fix fetch problem in pass_utils.py

* fix pylayer py39 mem leak

* Update input.py

* Update input.py

* Update gather.py

* Update broadcast.py

* Update batch_isend_irecv.py

* Update all_to_all.py

* Update all_reduce.py

* Update all_gather.py

* rollback

* Apply suggestions from code review

* Apply suggestions from code review

* Apply suggestions from code review

* modify feed_data for dataloader in pipline parallel mode

* add pre-commit

* remove read op, use feed op

* fix validate batch_size

* tiny fix

* support catch EOFException

* fix conflict

* fix conflict

* fix executor if cond

---------
Co-authored-by: Frida-a <2624653516@qq.com>

* shard_tensor support static graph

* add comments

* add dy2static ut

* use property in c++ side

* 203

* 204

* 205

* 206

* 207

* 208

* 209

* 210

* 211

* Update all_to_all.py

* Apply suggestions from code review

* fix: updated code examples.

* fix: added paddle.seed

* fix: updated code style

* Apply suggestions from code review

* refactor: refine detail of code examples

* Update python/paddle/distributed/auto_parallel/static/process_mesh_v2.py

* fix: refine detail

* fix: refine detail

* Update python/paddle/distributed/auto_parallel/static/process_mesh_v2.py
Co-authored-by: NNyakku Shigure <sigure.qaq@gmail.com>

* refactor: refine detail

* refactor: refine detail

* fix: refine doc

---------
Co-authored-by: NNyakku Shigure <sigure.qaq@gmail.com>

* optimize the memory

* fix bug in static_build.cc

* fix bug when using logging

* change the static_build

* fix bug in windows

* fix code accordding to review

* remove empty block program

* update implementation

* [Custom Dice]add run_check support for custom device

* fix error msg

* fix typo

* update for all custom device

* fix

* add warning msg

* [WIP] Add mp_all_reduce asynchronize overlap.

* Fix some problems.

* Fix dw compute bug, and use a temporary solution to achieve overlap.

* Use fused_linear_param_grad_add to compute dw.

* Reformat ColumnParallel _overlap_linear. Use environment flags to
control following behaviors:
1. export Flags_mp_aysnc_allreduce=True to turn on mp async all_reduce
2. export Flags_skip_mp_c_identity=True to skip two c_identity operators
   in dygraph mode.
3. export Flags_fused_linear_param_grad_add to enable fused_linear_param_grad_add
   in ColumnParallel backward with mp async all_reduce.

* Polish code.

* Remove useless communication API.

* Fix some problems in mp_async_all_reduce and skip_c_identity.

* Add test cases.

* Remove environment variable Flags_fused_linear_param_grad_add in test case.

* Reset error threshold.

* Reset threshold in test case.

* Add useful log. Remove useless test cases.

* make params_grads order same bewteen dynamic and static mode

* revert inplace clip

* use sorted attribute to control

* tiny fix

* fix find loss_grad_op

* Improve GC for pipeline parallel

* Delete print

* fix bug of nop_op and sharding

---------
Co-authored-by: Nchenruibiao <chenruibiao@baidu.com>