* modify numel in lbfgs and add a new test case. test=develop

* change param 'lr' to 'learning_rate' in lbfgs and its test

* add opt LBFGS and change test

* support 0-D output and 0-D as indice in __getitem__

* fix tests

* fix inference and UT

* add unittest for setitem

* fix xpu test

* fix xpu 0-d

* autofix
Co-authored-by: NLiyulingyue <83450930+Liyulingyue@users.noreply.github.com>

* revert changes in python/paddle/distributed/fleet/utils/hybrid_parallel_util.py

* empty commit, trigger ci

* fix test_slice

---------
Co-authored-by: NSigureMo <sigure.qaq@gmail.com>

* remove super call with parameters

* fix bug

* refine _found_inf

* Revert "fixoptminizer _set_auxiliary_var bug (#50335)"

This reverts commit c44005f0.

* Revert "refine optimizer create accumulators (#50188)"

This reverts commit 244e7546.

* Revert "fix found_inf bug for custom optimizer (#50158)"

This reverts commit 64573f9f.

* Revert "refine amp scaler found_inf (#49864)"

This reverts commit 382e9a06.

* fix code format

* fix conflict

* support xpu multi-card infer

* add ut

* clean code

* clean code

* fix

* fix

* fix

* fix

* 1219

* temporarily change the num_diff_files limit, test=document_fix

* Revert "temporarily change the num_diff_files limit, test=document_fix"

This reverts commit 8e70f00ef468d2dad0e38b3da06295ed62990d20.

* for codestyle

* remove duplicate license

* `static mode` -> `static graph mode`

* Update hybrid_parallel_inference.py

* Update layer_function_generator.py

* Update manipulation.py

* reset
Co-authored-by: NLigoml <39876205+Ligoml@users.noreply.github.com>
Co-authored-by: NSigureMo <sigure.qaq@gmail.com>

* lbfgs right version

* delete log

* code_style

* add test

* delete sparse grad

* add test for converage

* modify example in dygraph

* revert lbfgs

* add test for linesearch

* code_style

* add test

* modify test

* modify test

* modify set

* modify set

* modify test

* modiy code_block; test=docs_preview

* modiy code_block; test=docs_preview

* isort all files

* revert conflicting files

* revert conflicting files

* revert conflicting files

* remove apis in fluid.ops

* fix test_activation_nn_grad

* fix circle import error

* fix ops

* fix cos

* fix divide not inplace

* remove lazy-import part

* fix flake8 CodeStyle E266

* fix comments

* [CodeStyle][py2][U008] remove unnecessary args in `super()`

* remove remained args

* revert changes in test_pylayer_op

* Revert "revert changes in test_pylayer_op"

This reverts commit ff185a9ae738afac3b0264f61bde6c6b7f72e7c4.

* revert some changes in example code

* Remove reduntant numpy input in Example code

* Remove reduntant numpy input in Example code, test=document_fix

* update config

* re-blacken python code

* temporarily disable date and diff_py_file

* skip a format

[CodeStyle][F401] remove unused imports in python_paddle/inference_device_profiler_text_metric_incubate_quantization_libs_audio_amp_jit. (#46762)

* update for py3.6.

* update for py_3.6

* trim trailing whitespace

* fix `.cmake-format.py`

* revert npu ut changes, avoid npu ci error

* add use_hierarchical_allreduce

* support hierarchical allreduce for more cases

* move average_accumulates op to phi kernel

* add nproc_per_node for DistributedFusedLamb

* fix nproc_per_node communicator bug

* fix ring_id = 1 init bug

* fix ci

* fix test_parallel_executor_mnist.py

* add use_master_acc_grad

* add ut

* use yapf to format all python file

* yapf exclude two unittests file for they rely on writing and reading file, and format will break them

* disable diff_py_file because too many diff files cause command following failed

* improve introduction of bfgs args; test=document_fix

* modify according to zhouwei's comment; test=document_fix

* add gradient merge for DistributedFusedLamb

* use master acc gradient

* fix CI ut

* polish

* remove math_function_impl.h change

* fix test_update_loss_scaling_op.py

* try to fix XPU/NPU CI

* add gm ut

* fix bfgs_doc; test=document_fix

* add parameter name; test=document_fix

* modify according to chenlong's comments;test=document_fix

* fix image cv2 import

* fix lamb

* add Output(Step) to distributed fused lamb op

* add _set_step

* cut off relation between xk and initial_position's graph

* fix_bug

* add detach to cut off with original graph