* fix no sync bugs

* update

* update task chain

fix: update wait chain

feat: add `GetDeviceContext` for gloo

* fix oom

* fix dev

* update

* update
Co-authored-by: NLiYuRio <liyuruijx@163.com>
Co-authored-by: NForFishes <2282912238@qq.com>

* update config

* re-blacken python code

* temporarily disable date and diff_py_file

* skip a format

* trim trailing whitespace

* fix `.cmake-format.py`

* revert npu ut changes, avoid npu ci error

* [CustomDevice] add custom ccl 2/2

* update

* update

* update launch

* use yapf to format all python file

* yapf exclude two unittests file for they rely on writing and reading file, and format will break them

* disable diff_py_file because too many diff files cause command following failed

* refactor mp in eager mode

* update

* update

* add uts

* remove unrequired ut cases

* update

* fix bugs

* update

* support process group in dp with fleet api

* update

* fix uts

* update

* add uts for EagerReducer

* add more uts

* fix bugs

* fix bugs

* modify

* modify uts

* fix bugs

* update

* update

* update

* solve conflicts and merge

* add some other uts

* modify time of uts

* update

* update

* update

* remove uts of resnet

* refactor eager flags

* fix flags error when we switch from eager to dygraph

* fix ci problem

* fix ci

* fix ci

* merge develop and fix code style

* merge develop and fix code style

* fix op test error

* fix op test error

* fix op test error

* fix op test error

* fix op test error

* merge develop

* refactor reducer

* modify cmakelists

* solve conflicts

* rename group and update process_group

* fix bugs of ProcessGroupNCCL

* modify for CIs

* refactoring reducer

* [MLU] add fleet init api and collective api pytest for mlu

* fix no value for argument 'data_type' in method call

* add reducer

* fix bug of dp in pfp16

* fix topo

* fix data parallel when VOCAB var in program

* Added HCCL backend support in dynamic graph mode

* fix segmentation fault

* add ut

* add no_sync for parameters sync

* add pipeline for moe

* Add solutions to PyLayer which is unsupported in DataParallel

* modify note format for parallel.py

* modify docs of dataparallel

* add docs of dp with pylayer

* modify docs format

* modify example format

* change example of dp with pylayer

* add unittest for dp with pylayer

* modify ut

* merge latest codes

* update

* modify for CI-Coverage

* modify text-indent

* Add no_sync in data parallel for dynamic graph

* modify UT of no_sync

* delete test_parallel_dygraph_dataparallel_no_sync.py

* add test_parallel_dygraph_no_sync.py

* modify run_trainer_with_spawn in UTs

* Add UT of complex control flow in no_sync

* add specific descriptions and notes for no_sync

* check code style

* modify UT's TIMEOUT in CMakeLists.txt

* fix find_unused_parameters default value

* add clip/check

* add amp & clip grad in dygraph

* add logging

* support dp & mp

* fix sublayer error with include_sublayers=False

* add ut

* refactor include_sublayers related api

* fix ut

* fix ut of transformer

* fix ut of transformer

* remove useless code

* change sublayer api

* polish code

* add test for include_self=True

* support control flow

* supoort sync_parameters_buffers

* fix the bug of sparse embedding

* fix fleet for multi-stream

* fix memcpy for ncclid

* use sync to solve move operation

* add tensor_indices in AssignGroupBySize

* add rebuild group in reducer

* fix the bug of reducer in embedding

* add comment

* add reducer

* refine envent for memorycopy

* add concat&split for allreduce

* apply concat & split for fuse tensor

* fix nccl dep

* fix the untest, compile problem and ddp initialize problem

* fix untest for mac & add some comments & solve the repeated param in sublayers

* fix untest for windows & fix document