* remove deps and prior comm

* grad comm fuse

* add deps for amp&global norm

* stage2 broadcast prior deps

* stage2 grad overlap

* stream_analyzer bugfix

* overlap enable

* dep op namescope

* depend support multiple inputs

* check finite deps

* stage2 param comm overlap

* Set kD2HStream

* grad comm hierarchical

* grad comm hierarchical

* new unitest
Co-authored-by: Nchenruibiao <chenruibiao@baidu.com>

* 1219

* temporarily change the num_diff_files limit, test=document_fix

* Revert "temporarily change the num_diff_files limit, test=document_fix"

This reverts commit 8e70f00ef468d2dad0e38b3da06295ed62990d20.

* for codestyle

* remove duplicate license

* `static mode` -> `static graph mode`

* Update hybrid_parallel_inference.py

* Update layer_function_generator.py

* Update manipulation.py

* reset
Co-authored-by: NLigoml <39876205+Ligoml@users.noreply.github.com>
Co-authored-by: NSigureMo <sigure.qaq@gmail.com>

* auto parallel bf16

* [AutoParallel] quantization pass support export

* support subgraph

* move_presist_var_to_global_block

* update unittest

* fix ci-coverage

* fix codestyle

* fix fake_dequantize_op

* remove unused var

* fix ci error and aprroval error

* add unittest for fp16 in test_dequant_linear

* replace mutable data

* fix unittest in non-cuda-core

* fix unittest
Co-authored-by: Ncarryyu <569782149@qq.com>
Co-authored-by: Nwufeisheng <wfs1997@163.com>

* [Auto Parallel] Rename methods of ProcessMesh

* [Auto Parallel] Impl the python process_mesh by the c++ one

* [Auto Parallel] Add some minor modifications

* [Auto Parallel] Rename some methods

* [Auto Parallel] Remove unnecessary codes

* [Auto Parallel] Add back some removed files

* [Auto Parallel] Fix bugs

* [Auto Parallel] Fix a bug

* Update process_mesh.cc

* [Auto Parallel] Fix a bug

* memcpy overlap

* memcpy newexe

* [AutoParallel] fix update complete and add_to_collection

* fix annotation

* fix amp fill_constant dist_attr

* [AutoParallel] recompute tuning

* fix conflict

* update comment

* bug fix

* update rc algo

* tiny fix

* fix clear process_group

* remove comment

* update segment print

* fix import OpRole

* adapt amp pass and grad_clip pass for opt_tuner

* update tuning config

* fix import

* annotate recompute info on ops and upgrade recompute pass

* add op_namescope for seed op

* record reserved vars

* fix recompute var's dist_attr

* fix strategy unittest

* adapt for fp16

* update unittest

* revert copy opt

* update unittest

* rename set_recompute_segments

* fix unittest

* Fix fuse_gemm_epilogue

* update tests

* Update CMakeLists.txt

* Update CMakeLists.txt

* Update CMakeLists.txt

* fix random seed

* use assert_allclose

* Update test_dist_fuse_gemm_epilogue_pass.py

* Update cpp_pass.py

* Update test_dist_fuse_gemm_epilogue_pass.py

* fix codestyle

* update seed and atol

* isort all files

* revert conflicting files

* revert conflicting files

* revert conflicting files

* add depend

* add origin amp files

* fp16 distinguish None & False

* engine log

* dp add deps for graph exe

* add dep for grad clip

* dep ops in comm stream

* unitest

* remove fluid

* update public

* core

* public

* public1

* ci

* add depend

* fp16 pass distinguish None & False

* engine log

* clear fluid apis in fleet and passes

* fix model.py

* fix model.py

* fix cpp_pass.py

* [AutoParallel] selective recompute

* add cmakelist

[CodeStyle][F821] fix undefined variables due to missing imports, misspelled variable names (#47899)

* `hann` -> `_hann`

* `false` -> `False`

* a missing passed argument `reduce_all`

* some missing imports

* `device_type` -> `heter_device_type`

* `PKVClient` -> `KVClient`

* fix some typos and missing imports

* fixed cond state

* fixed cond state

* fix insert concat op

* fix fp16 assert

* fix flake8 CodeStyle E266

* fix comments

[Auto Parallel] Sharding Optimization：Partition Algorithm & Stage2 Parameter Bucket communication  (#47180)

* partition param by order

* add logging

* reorder opt

* config

* stage2 bucket

* update unitest

* [CodeStyle][py2][U004] unecessary explicit `object` inheritance in class definition

* fix an increment

* fp16 pass support assign op

* choose assign op exec mode

* add unittest

* add cmakelist

* [CodeStyle][E262][E265] make comments start with `# `

* flake8 config

* [CodeStyle][py2][U008] remove unnecessary args in `super()`

* remove remained args

* revert changes in test_pylayer_op

* Revert "revert changes in test_pylayer_op"

This reverts commit ff185a9ae738afac3b0264f61bde6c6b7f72e7c4.

* revert some changes in example code

* [CodeStyle][E712] use `if cond`/`if cond is True` for comparison with `True`

* revert changes in fluid

* revert unrelated file

* revert changes in norm

* revert changes in auto_parallel_amp

* fix norm and auto_parallel_amp

* revert a typo fix due to fixed at #47477

* update config

* re-blacken python code

* temporarily disable date and diff_py_file

* skip a format

* [AutoParallel] fix fp16 for subblock

* fix engine

* fix comment

* [CodeStyle][py2] remove `compat` module (to_text)

* remove some unnecessary decode

* remove to_text definition and unittest

* Revert "remove to_text definition and unittest"

This reverts commit a6b69cb8dca8b9b031ce10ea32d1040e7e0dd267.

* remove an assertion

* empty commit

* [AutoParallel] add callbacks

* fix unittest

* fix dist_context

* fix engine

* fix cmakelist

* fix unittest's returns

* fix cmakelist

* [CodeStyle][F401] remove unused import in python/paddle/distributed

* remove pass

* empty commit

* Fix ValueError: list.remove(x): x not in list for meta_optimizer_names.

Fix ValueError: list.remove(x): x not in list for meta_optimizer_names.

* Fix split import.

Fix split import.

* add noqa after meta_optimizers in factory

* restort collective ops

* expand `import *`

* add noqa after required imports

* try to fix APIs without core.ops

* Revert "try to fix APIs without core.ops"

This reverts commit 6172beaf601e84bf61f2490c12c4739f0edaa5eb.

* fix an increment

* empty commit

* add noqa after required imports

* expand `import *`, fix ci error
Co-authored-by: NShuangchi He <34329208+Yulv-git@users.noreply.github.com>

* [Auto Parallel] Unify the logger and outputs of Engine API

* [Auto Parallel] Fix the bugs of to_static

* [Auto Parallel] Adjust the test_to_static.py