* add async save for sparse table
* add load profiler for sparse table
* add load info for sparse table

* fix pipeline

* fix mp pp dp

* fix utest of hybrid parallel

* add utest for tuple

* fix precision of mp

* fix bug of seed

* fix dp

* print group

* fix cpp lint
* fix save/load with unexpected value
* fix save and user interface

* fix find_unused_parameters default value

* add trainprofiler for heterps in oneps; test=develop

* add set_use_ps_gpu; test=develop

* add raw program, test=develop

* update, test=develop

avoid polluting logging's root logger

* Revert "Revert "[PsCore] optimize performance of large kv (#32535)" (#32599)"

This reverts commit 809ac036.

* brpc dep

* add alltoall api, test=develop

* fixed docs.

* Fixed docs. test=document_fix

code bak.

fixed docs. test=document_fix

* Revert to previous version of python/paddle/fluid/backward.py

* fixed bugs.

* test=document_fix. Fixed examples.

* Support list and tuple for parameters of layer_norm, multiprocess_reader, DatasetFolder and ImageFolder.

* add unittest for layer_norm.

* add require gpu for example.

This reverts commit 4b7242b0.

* fix amp bug

* fix name of wordsize

* add sendrecv, test=develop

* optimize pull sparse

* optimize pull sparse

* change macro

* format

* fix model parallel

* rm parallel_help.py

* add embedding

* add pp dygraph, test=develop

* Dygraph reocmpute

* unitest for Dygraph reocmpute

* dy recompute remove unitest for win and mac

* add pipeline layer

* update

solve hccl communicate conflict (#32447)

* add clip/check

* add amp & clip grad in dygraph

* add logging

* add allreduce and broadcast without test (#31024)

add allreduce and broadcast without test

* Refactor HCCLCommContext to be compatible with Paddle (#31359)

Refactor HCCLCommContext to be compatible with Paddle (#31359)

* [NPU] add npu kernel for communication op (#31437)

* add allreduce and broadcast without test

* add c_broadcast_test case

* build c_comm_init and c_create_group operators

* make the whole thing compile

* add broadcast and init op test case but run failed

* make unit test compile

* fix broadcast test bug and change into hcom for ccl

* change c_comm_init and c_create_group ops accordingly

* make tests compile

* transfer code to 27

* compiled successfully in 28, but run failed

* test broadcast in 28, but failed

* make hcom primitives work

* change hccl data type for base.h

* fix broadcast bug

* make attributes work

* fix group name bug

* add allreduce but test failed

* allreduce bug for qiuliang

* allreduce finished

* add allgather and reducescatter

* merge all op code

* add allgather test

* finish run all ccl op test exclude send/recv

* all all op and test exclude send/recv

* send_v2_npu.cc recv_v2_npiu.cc compiled

* fix ccl core dump bug and test allgather, reducescatter, broadcast op

* fix allreduce bug just for test

* hcom send&recv test pass, without hcom_destroy

* for qiuliang test

* Ascend Send&Recv Test Pass

* all op (ex send/recv) ok

* fix bug

* merge all ccl op

* style merge to PaddlePaddle

* merge style

* new merge style

* merge style 2

* insert an empty at the end

* disable ctest for hcom to pass ci
Co-authored-by: Nvoid-main <voidmain1313113@gmail.com>
Co-authored-by: Nf2hkop <f2huestc@outlook.com>

* Add auto-increasing tag id for Hcom OPs (#31702)

* add c_reduce_sum op (#31793)

add c_reduce_sum op

* update Ascendrc hccl to 20.3 (#32126)

update Ascendrc hccl to 20.3 (#32126)

* fix merge code

* change cmake.txt1

* [NPU] Support npu kernel for c sync stream op (#31386)

* sync stream npu op

* add with_ascend_acl

* update c++ unittest

* compile all failed

* try to pre commit

* after pre commit

* merge&compile&test hccl successfully!

* fix code style

* fix code style

* fix bugs about hccl

* fix some bugs

* fix code style

* fix style

* fix style

* fix

* fixed

* merge develop
Co-authored-by: Nlw921014 <liuwei921014@yeah.net>
Co-authored-by: NVoid Main <voidmain1313113@gmail.com>
Co-authored-by: Nf2hkop <f2huestc@outlook.com>
Co-authored-by: Nxiayanming <41795079@qq.com>

* sharding: update config DOC

* update pipeline config

* sharding update doc