* Fix bug of reduce_sum op. When input.numel() > INT32_MAX, its result
is wrong.

* support pure bfloat16

* support bf16 linear

* update PR to pass CI

* tiny fix where_grad_kernel.cu

* Support bfloat16 type for reducer and sharding.

* Fix some bug.

* Polish code.

* Polise code.

* Add bfloat16 datatype in fill_grad kernels.
Co-authored-by: Nsneaxiy <sneaxiy@126.com>
Co-authored-by: Nsneaxiy <sneaxiy@126.com>

* [dygraph sharding] Overlap the reduce and the caculation for sharding stage 2. (#46495)

* [dygraph sharding stage 2] sharding broadcast overlap (#46656)

* Multi groups for broadcast of sharding stage 2 (#46894)

uniform logger manager in FleetAPI.
hidde API under distributed/utils which users don't need.

* fix sharding in eager

* support eager sharding

* rename eagerpylayer

* Replace np.bool/np.bool8 with np.bool_

* Replace np.object with np.object_

* Replace np.complex with np.complex128

* Replace np.float with np.float64

* Replace np.int with np.int_

* Rerun pre-commit for newer pre-commit configuration

* Use builtin bool instead of np.bool_ based on the context

* use yapf to format all python file

* yapf exclude two unittests file for they rely on writing and reading file, and format will break them

* disable diff_py_file because too many diff files cause command following failed

* fix_sharding3_offload

* fix_fp16dtype_bug

* fix_sharding_group

* fix_sharding_group

* support param groups in grad_clip

* update

* modify for review

* update

* fix bugs

* modify code style

* fix bugs of _get_global_group

* merge latest develop branch

* fix bugs

* update

* fix bugs for unittest

* modify for less use of gpu mem

* fix bugs of using _reset_grad_inplace_version

* update

* update

* modify for CI-Coverage

* retrick all CIs

* Debug

* Fixed issue with reset_grad_inplace_version when used with clear_gradient & cross-batch accumulation

* Rearranged interfaces

* Fixed ci issues