* Fix bug of reduce_sum op. When input.numel() > INT32_MAX, its result
is wrong.

* support pure bfloat16

* support bf16 linear

* update PR to pass CI

* tiny fix where_grad_kernel.cu

* Support bfloat16 type for reducer and sharding.

* Fix some bug.

* Polish code.

* Polise code.

* Add bfloat16 datatype in fill_grad kernels.
Co-authored-by: Nsneaxiy <sneaxiy@126.com>

* add rpc module in cpp side

* add rpc module in python side

* support win32 and mac for rpc

* 代码优化

* 优化代码

* update rpc

* update rpc launch

* rpc remove rank and world_size api

* fix logger import bug

* remove support for win and mac

* remove support for xpu, npu, cinn and rocm

* remove support for xpu, npu, cinn and rocm

* fix shutdown barrier timeout bug

* update:python_rpc_handler to shared ptr

* fix master shutodwn first bug

* tests support for cpu

* update log to vlog

* update get service info api

* add single process test case

* remove process group

* remove some useless dependencies

* update rpc api comments

* update rpc comments: Example to Examples

* update rpc api comments

* update rpc api comments

* update launch api comments

* update init_rpc comments

* update rpc sync and async comments

* fix bug: init_rpc cant be called repeatly in a process

* update rpc api comment: make master endpoint unique

* update rpc api:service to worker, timeout_ms to timeout

* rename ServiceInfo to WorkerInfo

* refactor: rename server to worker, log to vlog

* add launch test

* remove unused codes

* refine

* remove using lodtensor part1

* polish history code format

* remove needless using tensor

* remove needless using tensor

* resolve conflict

* replace tensor using

* fix format error

* revert needless changing

* fix rocm and npu compile error

* fix cinn compile error

* fix format error

* fix mkldnn format error

* fix mkldnn format error

* fix cinn compile error

* fix cinn compile error

* fix cinn compile error

* resolve conflict

* [CodeStyle] trim trailing whitespace in .md and .rst

* empty commit, test=document_fix

* make dense_contents_ lazy init

* update legacy dygraph

* fix legacy dygraph bug

* config

* fix unittest

* zero init & cache & patch config

* add barrier to save and load

* add unittest

* update brpc version;test=develop

* [XPU] add some collective ops. test=kunlun

* use XPUOpTestWrapper. test=kunlun

* skip kl1 for collective ops. fix typo: deivce -> device. test=kunlun

* back fl

* delete ssl cert

* .

* make warning

* .

* unittest paral degree

* solve unittest

* heter & multi cloud commm ready

* .

* .

* fl-ps v1.0

* .

* support N + N mode

* .

* .

* .

* .

* delete print

* .

* .

* .

* .

* fix bug

* .

* .

* fl-ps with coordinator ready

* merge dev

* update message parse only

* update fl client scheduler

* fix bug

* update multithreads sync

* fix ci errors

* update role_maker.py

* update role_maker.py

* fix ci error: windows py import error

* fix ci error: windows py import error

* fix windows ci pylib import error

* add dump fields & params

* try to fix windows import fleet error

* fix ps FLAGS error

* fix logging risk

* fix logging possible risk

* write trainer_desc file

* support split sparse params in local & remote

* fix import paddle.fluid.core.PSGPU

* fix import paddle.fluid.core.PSGPU

* add remote_sparse & local_sparse config

* fix unittest

* fix test_dist_fleet_geo table error

* fix PADDLE_ENFORCE error

* fix other's pr conflict

* [Auto Parallel] Pybind11 ProcessMesh and DeviceMesh

* [Auto Parallel] Fix the unittest problem

* [Auto Parallel] Explicitly add the src file for auto_parallel target

* [Auto Parallel] Add the proto depedency explicitly

* [Auto Parallel] Fix the cmake bug on windows and mac

* [Auto Parallel] Remove the pybind11 header file in process_mesh.h

* first refine

* second refine

* remove some code unuseful

* [Auto Parallel] Add the c++ dist attrs

* [Auto Parallel] Remove some codes to be less than 1000 lines

* save load

* save load

* add unittest

* first commit

* second commit

* third commit

* remove SaveLocalFS in memory sparse table

* save dense param

* update

* push slot

* fix push show clk: int -> float

* add unittest

* fix sample

* unittest

* add AsExtra for op

* unittest

* modify fs.py

* modify fs.py

* fix some bugs

* add dataset hdfs config

* local change

* dataset use differenct hadoop ugi/fs_name

* add

* fix conflict

* fix

* remove logs

* code style

* fix

* code style

* code style

* fix

* code style

* save_dense_param

* fix

* fix

* fix

* fix

* change momentum in dense optimzer

* fix

* fix

* change fluid => paddle.static

* remove some unuseful code
Co-authored-by: Nesythan <esythan@126.com>

* [Auto Parallel] Add ProcessMesh, DeviceMesh and DistributedMapper

* [Auto Parallel] Remove unecessary codes

* [Auto Parallel] Comment out unnecessary cmake statements

* [XPU] fleet dist_model support xpu. test=kunlun

* [XPU] fleet dist_model support xpu. test=kunlun

* move unittest file location. test=kunlun

* [CustomDevice] add custom ccl 2/2

* update

* update

* update launch