* [Kunlun] Mofify some legacy code on distributed training

There were limitations on XPUs before, such as concat/split is not
supported, and c_broadcast only support fp32. These limitations are
lifted recently.

Multi-device profiling on XPU will also be supported by this PR.
Without this PR, a hanging broadcast will be issued by devices that
enables profiling, eventually lead to kernel timeout error.

* fix typo

* no endpoints in dy mode

* fix fleet api inconsistent

* bump ruff to 0.0.271 and update config

* exclude third_party

* bump ruff to 0.0.272

* refine config

* [dygraph]unify _non_static_mode() in_dygraph_mode() and in_dynamic_mode()

* [dygraph]unify _non_static_mode() in_dygraph_mode() and in_dynamic_mode()

* [dygraph]unify _non_static_mode() in_dygraph_mode() and in_dynamic_mode()

* [dygraph]unify _non_static_mode() in_dygraph_mode() and in_dynamic_mode()

* [dygraph]unify _non_static_mode() in_dygraph_mode() and in_dynamic_mode()

* [dygraph]unify _non_static_mode() in_dygraph_mode() and in_dynamic_mode()

* fixed cyclic reference that caused patial import

* fixed bad change

* fix bad import

* fix bad import

* fix bad import

* fix ut failed caused by change in_dynamic_mode

* fix ut failed caused by change in_dynamic_mode

* fixed usage of in_dynamic_mode() or in_dygraph_mode()

* revert python3 to python in .pre-commit-config.yaml

* fix merge conflicts

* [XPU] remove scale_loss in parallel.py

* [XPU] throw Unimplemented when using Reducer

* print modifed flags

* fix ref, opt print

* fix default getter

* fix ut

* rename _varbase_creator as create_tensor

* rename _varbase_creator as create_tensor

* rem is_compiled_with_npu

* rem nup related code

* make lint happy

* rem test

* remove some tests

* Update grad_scaler.py

* fix an error

* rem is_compiled_with_mlu

* fix some mlu_place and mlu_device_coount

* make lint happy

* delete old dygraph op test

* remove no used fluid beam_search_decoder

* move Layer and related helper to paddle.nn.common

* modify Layer references from dygraph.layers.Layer to paddle.nn.common.layers

* stash changge

* remove fluid layer_object_helper, layers.py

* remove fluid layers init

* add setip

* fix unitest

* delete layers in fluid.dygraph

* merge paddle.tensor.stat,py

* fix circle import

* fix curcle import

* remove redundant in_dygraph_mode import

* revoce paddle.nn.common.* in fluid.__init__

* recovery nn.rnn

* paddle.frame use lazy import import paddle.jit to avoid circle import

* remove left dygraph.layers ref

* merge develop

* fix import error

* fix test error

* fxi merge error

* fix test fluid.Layer

* fix test error

* fix test error

* fix import error

* fix import error

* fix comments

* fix circle import

* fix rnn import error

* fix circle import

* fluid clean: remove parallel and parallel_helper api

* fix: fix the import path.

* fix DataParallel imports issue

* remove dygraph.parallel.ParallelEnv

* logger.py error: AttributeError: module 'paddle' has no attribute 'distributed'

* move the implenmentation to the root folder

* logger.py import ParallelEnv from paddle.parallel to avoid circular import

* add the comment of why import ParallelEnv from paddle.parallel in logger.py and remove the api interface in the paddle/parallel.py

* outdated Env and note removed

* decouple the logger.py and ParallelEnv

* remove another ref of parallel in init.py

* comm_context and static init

* refactor: move to phi/core/distributed

* refactor: avoid mutable_data usage

* fix: windows sock

* fix: device without nccl
Co-authored-by: Wen Sun <syl1887415157@126.com>

* Fix bug of reduce_sum op. When input.numel() > INT32_MAX, its result
is wrong.

* Remove climits.

* Clean fluid API in paddle/distributed and paddle/fleetx folders.
Include following files:
python/paddle/distributed/__init__.py
python/paddle/distributed/collective.py
python/paddle/distributed/fleet/utils/fs.py
python/paddle/distributed/fleet/utils/hybrid_parallel_inference.py
python/paddle/distributed/fleet/utils/hybrid_parallel_util.py
python/paddle/distributed/fleet/utils/internal_storage.py
python/paddle/distributed/launch/context/device.py
python/paddle/distributed/parallel.py
python/paddle/distributed/parallel_with_gloo.py
python/paddle/distributed/spawn.py
python/paddle/framework/__init__.py
To be mentioned, 'paddle.fluid.dygraph.parallel.ParallelEnv'
 and 'fluid.framework.core' keeps unchanged in those files.
ParallelEnv is used by paddle.fluid.dygraph.parallel.DataParallel.
However, APIs in paddle.fluid.dygraph.parallel can't be
migrated to paddle.distributed, as there exists cyclic import
dependencies in modules like paddle.static, paddle.tensor. And
'fluid.framework.core' will be changed to import framework.core
after fluid.core is transmitted.

* Change TODO authors.

* isort all files

* revert conflicting files

* revert conflicting files

* revert conflicting files

* fix_docx_stanh

* fix einsum api en docs issue

* fix model api en docs issue

* for codestyle

* fix_einsum.py_einsum, test=document_fix

* fix_model.py_Model, test=ducument_fix

* fix_creation.py_meshgrid, test=document_fix

* fix_linalg.py_slogdet, test=document_fix

* fix_loss.py_SoftMarginLoss_CrossEntropyLoss_NLLLoss_BCELoss, test=document_fix

* norm.py_SyncBatchNorm, test=document-fix

* norm.py_SyncBatchNorm, test=document_fix

* norm.py_SyncBatchNorm, test=document_fix

* list18-30, test=document_fix

* refix_list1-15, test=document_fix

* deletefiles, test=document_fix

* fixedapi_pre-commit, test=document_fix

* fix_list31-45, test=document_fix

* list111, test=document_fix

* some_fix, test=document_fix

* some_fix, test=document_fix

* somefix, test=document_fix

* somefix, test=document_fix

* refix, test=document_fix

* refix, test=document_fix

* refix, test=document_fix

* refix, test=document_fix

* rerfix, test=document_fix
Co-authored-by: Ligoml <limengliu@tiaozhan.com>

* [CodeStyle][py2][U008] remove unnecessary args in `super()`

* remove remained args

* revert changes in test_pylayer_op

* Revert "revert changes in test_pylayer_op"

This reverts commit ff185a9ae738afac3b0264f61bde6c6b7f72e7c4.

* revert some changes in example code

* update config

* re-blacken python code

* temporarily disable date and diff_py_file

* skip a format

* [CodeStyle][F401] remove unused import in python/paddle/distributed

* remove pass

* empty commit

* Fix ValueError: list.remove(x): x not in list for meta_optimizer_names.

Fix ValueError: list.remove(x): x not in list for meta_optimizer_names.

* Fix split import.

Fix split import.

* add noqa after meta_optimizers in factory

* restort collective ops

* expand `import *`

* add noqa after required imports

* try to fix APIs without core.ops

* Revert "try to fix APIs without core.ops"

This reverts commit 6172beaf601e84bf61f2490c12c4739f0edaa5eb.

* fix an increment

* empty commit

* add noqa after required imports

* expand `import *`, fix ci error
Co-authored-by: NShuangchi He <34329208+Yulv-git@users.noreply.github.com>

* Update README.md

* Update README.md

* Fix F632: replace 'is', 'is not' with ==, != respectively

* 1. Delete some expression like 'This Op'
2. remove import numpy as np

* test=document_fix

* fix eg; test=document_fix

* fix 'import numpy' cases; test=document_fix

* fix 'import numpy' cases; test=document_fix

* fix some docs; test=document_fix

* delete raise; test=document_fix

* add some introduction; test=document_fix

* add some introduction; test=document_fix

* test=document_fix

* Fix ’note‘ format; test=document_fix

* Fix Returns of cholesky; test=document_fix

* Fix Example format; test=document_fix

* Fix det; test=document_fix

* Fix eig; test=document_fix

* Fix eigh; test=document_fix

* Fix eigh; test=document_fix

* Apply suggestions from code review;test = document_fix
Co-authored-by: NNyakku Shigure <sigure.qaq@gmail.com>

* Apply suggestions from code review;test = document_fix
Co-authored-by: NNyakku Shigure <sigure.qaq@gmail.com>

* Apply suggestions from code review;test = document_fix
Co-authored-by: NNyakku Shigure <sigure.qaq@gmail.com>

* test=document_fix

* test=document_fix

* KLDiv;test=document_fix

* norm example code; test=document_fix

* revert python/paddle/fluid/**/*

* revert python/paddle/distributed/spawn.py

* revert python/paddle/fluid/*

* fix a `Note` format

* Fix inv; test=document_fix

* Fix lu; test=document_fix

* Fix lu_unpack; test=document_fix

* Fix matrix_power; test=document_fix

* Fix multi_dot; test=document_fix

* Fix solve; test=document_fix
Co-authored-by: NNyakku Shigure <sigure.qaq@gmail.com>

* trim trailing whitespace

* fix `.cmake-format.py`

* revert npu ut changes, avoid npu ci error

* [CustomDevice] add custom ccl 2/2

* update

* update

* update launch

* fix conflict

* new pg apis

* add docs of new apis

* update

* fix coverage

* update

* fix bug

* fix reduce scatter

* fix api

* update
Co-authored-by: NForFishes <2282912238@qq.com>

* tmp fix

* init

* compile ok

* compile ok

* add vlogs

* add test

* fix termination error

* add testfile

* add

* fix window compile

* fix window compile

* fix windows compile

* fix windows compile

* fix windows compile

* fix windows compile

* fix windows compile

* fix windows compile

* fix kunlun compile

* fix compilation

* fix compilation

* fix compilation

* tmp fix

* add windows

* add windows

* add more logs

* change timeout to protected

* SB

* add

* add

* fix timeout

* add

* fix test

* fix test

* fix test

* fix ut

* fix ut

* fix ut

* use yapf to format all python file

* yapf exclude two unittests file for they rely on writing and reading file, and format will break them

* disable diff_py_file because too many diff files cause command following failed

* update