* Fix bug of reduce_sum op. When input.numel() > INT32_MAX, its result
is wrong.

* Support allow_partial switch, which can be configure in
pipeline_configs. If sent tensor are not the same from
different hosts, they shouldn't been sent partially and
then concated as a whole tensor.

* Change name allow_partial to enable_partial_send_recv.

* Add global variable _enable_partial_send_recv

* Fix bug of reduce_sum op. When input.numel() > INT32_MAX, its result
is wrong.

* support pure bfloat16

* support bf16 linear

* update PR to pass CI

* tiny fix where_grad_kernel.cu

* Support bfloat16 type for reducer and sharding.

* Fix some bug.

* Polish code.

* Polise code.

* Add bfloat16 datatype in fill_grad kernels.
Co-authored-by: Nsneaxiy <sneaxiy@126.com>

* combine dp and stage2 hybrid parallel.

* update condition.

* [Zero-Dim] support input 0D Tensor for unary api

* fix CI

* [CodeStyle][F401] remove unused import in python/paddle/distributed

* remove pass

* empty commit

* Fix ValueError: list.remove(x): x not in list for meta_optimizer_names.

Fix ValueError: list.remove(x): x not in list for meta_optimizer_names.

* Fix split import.

Fix split import.

* add noqa after meta_optimizers in factory

* restort collective ops

* expand `import *`

* add noqa after required imports

* try to fix APIs without core.ops

* Revert "try to fix APIs without core.ops"

This reverts commit 6172beaf601e84bf61f2490c12c4739f0edaa5eb.

* fix an increment

* empty commit

* add noqa after required imports

* expand `import *`, fix ci error
Co-authored-by: NShuangchi He <34329208+Yulv-git@users.noreply.github.com>

* Update README.md

* Update README.md

* Fix E713: convert 'not ... in' to 'not in'

* Update README.md

* Update README.md

* Fix F541 by converting f-string to normal strings

* [CodeStyle] remove all future import

* revert test_error.py

* restore future import in example code

* 1. Delete some expression like 'This Op'
2. remove import numpy as np

* test=document_fix

* fix eg; test=document_fix

* fix 'import numpy' cases; test=document_fix

* fix 'import numpy' cases; test=document_fix

* fix some docs; test=document_fix

* delete raise; test=document_fix

* add some introduction; test=document_fix

* add some introduction; test=document_fix

* test=document_fix

* Fix ’note‘ format; test=document_fix

* Fix Returns of cholesky; test=document_fix

* Fix Example format; test=document_fix

* Fix det; test=document_fix

* Fix eig; test=document_fix

* Fix eigh; test=document_fix

* Fix eigh; test=document_fix

* Apply suggestions from code review;test = document_fix
Co-authored-by: NNyakku Shigure <sigure.qaq@gmail.com>

* Apply suggestions from code review;test = document_fix
Co-authored-by: NNyakku Shigure <sigure.qaq@gmail.com>

* Apply suggestions from code review;test = document_fix
Co-authored-by: NNyakku Shigure <sigure.qaq@gmail.com>

* test=document_fix

* test=document_fix

* KLDiv;test=document_fix

* norm example code; test=document_fix

* revert python/paddle/fluid/**/*

* revert python/paddle/distributed/spawn.py

* revert python/paddle/fluid/*

* fix a `Note` format

* Fix inv; test=document_fix

* Fix lu; test=document_fix

* Fix lu_unpack; test=document_fix

* Fix matrix_power; test=document_fix

* Fix multi_dot; test=document_fix

* Fix solve; test=document_fix
Co-authored-by: NNyakku Shigure <sigure.qaq@gmail.com>

* fix bugs of mp

* fix bugs of mp

* update

* update

* fix bug

uniform logger manager in FleetAPI.
hidde API under distributed/utils which users don't need.

* polish code comments

* polish data_device_transform.cc

* refactor mp.

* update setup.py.

* update mp_layers.py for compatibility.

* add documents for mp_layers.py

* update init.py

* update collective.py.

* update.

* update mp_ops.py

* update.

* update code style.

* update code style.

* trim trailing whitespace

* fix `.cmake-format.py`

* revert npu ut changes, avoid npu ci error

* support iterable dataset for auto parallel

* add split_data proto

* fix unittest bug

* fix recompute bug

* update cmake

* config

* fix unittest

* zero init & cache & patch config

* add barrier to save and load

* add unittest