* Optimization grad merge performance (#29784)

* [fleet] combine amp and gradient merge, test=develop (#30086)

* fix assign_op_xpu concat_op_xpu warining (#30120)
Co-authored-by: Nliuyuhui <liuyuhui@baidu.com>

* gen nccl id use socket (#29431)

* fix gen_nccl_id_op_helper compile failed, test=develop (#29614)

* update, test=develop

* remove data parallel scale loss & apply collective_grads

* move apply in minimize

* fix failed unittests

* add gloo initializer, test=develop

* add gloo initializer, test=develop

paddle.distributed.fleet supports dynamic graph execution.

* add dygraph parallel run interface

* polish implement & unified env property name

* add print config arg

* refactor init_parallel_env function

* Compatible with multiprocessing and launch modes

* set default trainer start port

* support run in python 2

* polish python2 support code

* remove python2 support

* refine launch import

* polish dome design details

* refactor api implemention & path

* use new method _set_expected_place

* add spawn unittest framework & mnist test

* add more unittests & doc

* fix unittest failed

* polish english doc

* self review and polish details

* refactor code by reviewer's comments

* fix unittest failed

* fix parallel_env unittest

* fix several typos

* fix error introduced when fixing typos

* add unpublic note for start_processes

* polish details by xiaoguang's comment

* verify correctly when spawn nprocs=-1

* refactor spawn & init_parallel_env design

* polish doc details

* open spawn unittests

* try to fix doc compile error

* try to fix unknown doc format error

* add skip unittest when not gpu

* add retry for prefetch

* Revert "Fix/sync barrier (#25016)"

This reverts commit be6a315f.

* reopen dist UT, test=develop

* remove fl UT, test=develop

* disable distributed UT temporary，enable it soon, test=develop

use 2 cards test=develop

modify the implementation of save_persistables and save_inference_model for fleet collective mode (#20802)

* modify the implementation of  save_persistables and save_inference_model functions for fleet collective, test=develop

* add ut, test=develop

 Add detail logs on resnet unit test

* add a base class for the Communicator
* add AsyncCommunicator Impl for async distributed training

test=develop

* update parallel.py
test=develop

add python coverage launch when WITH_COVERAGE=ON

Polish fleet API to support cuda collective mode and nccl2 mode

* deprecate python memory optimize, test=develop

* remove memory_optimize in unittests, test=develop

* add unittests to deprecated interfaces, test=develop

* Enhance fuse optimization op pass
test=develop

* add more print function for timeout issue, make timeout value larger

* add class name and timeline for test_dist_base.py

* add print log for unittest of distributed training
test=develop

* add var grad hook test=develop

implement dygraph.parallel.DataParallel to hook reduce op.