Co-authored-by: Nronnywang <524019753@qq.com>

[cherry-pick][hybrid performance] optim the grad fuse for pipeline mode by sorting the grad by dtype (#35070) (#35300)

[cherry-pick][hybrid performance] Grad fuse for gradient merge under pipeline mode (#35004) (#35299)

Co-authored-by: Ngongweibao <weibao.gong@gmail.com>

Co-authored-by: NWangXi <wangxi16@baidu.com>

Co-authored-by: NWangXi <wangxi16@baidu.com>

Co-authored-by: NWangXi <wangxi16@baidu.com>

[cherry-pick][Hybrid Performance] Move the cast op of AMP which cast fp32 param to fp16 param to the optimizer (#34965) (#35296)
Co-authored-by: NWangXi <wangxi16@baidu.com>

Co-authored-by: NWangXi <wangxi16@baidu.com>

* add retry for HcclGetRootInfo

* refine code

* reduce retry interval

* support class center sample of PartialFC

* unitest_quant_dequant

* fix

* fix

* deleted: test_trt_quant_conv2d_dequant_fuse_pass.py

* fix

* fix op-benchmark

* test=document_fix

* copy boost optional.hpp to paddle

* copy boost optional.hpp to paddle

* move directions

* del fluid/utils

* modify .hpp to .h

* move directions

* modify to paddle::optional

* add modification description

* format code stype for the files in paddle/utils

* format code stype

* - disabled caching of layer norm

- fix in compilation

- compilation fix

- transpose caching disabled

- compilation fix

- more compilation fixes

- sum caching disabled

- compilation fix

* - LRN with disabled cache

* lint fixes

* add exclude rules of pre-commit to paddle/utils and third_party

* remove exclude direction distributed/third_party

* remove exclude of paddle/utils for format cpplint check

* polish unittest.

* polish code

* polish code

* [bug fix] fix unfold negative_size_param

* add mkl multi-thread test cases

* fix codestyle

* fix codestyle & enable ernie mkl test

* dygraph support more ctc grad scale

* scale for 1.x

* fix unitest

* fix unitest

* format code

* fix unittest

* fix log info

* unittest cov

* fix format;notest,test=cpu,coverage

* skip ctc_loss egs;test=cpu

* warpctc grad cov;test=coverage

* add dygraph test;test=coverage

* format;test=cpu,coverage

* format;test=cpu

* add api compat;test=cpu

* add cpu test

* rename

* rename

* fix

* fix test

* format

* eigen cpu

* eigen gpu grad pass

* cuda gpu pass

* format

* fix ci

* add inplace passes and tests

* update

* fix use_cuda undefined
fix compile error of op compat

* add more ut

* fix CPU CI error

* check adam unique

* fix mac/windows ci, improve coverage

* fix ci error

* follow weihang's comment

* fix BlockDesc::MoveFrom

* follow qiuliang's comment

* update

* follow huihuang's comments

* fix drop_last not work in IterableDataset. test=develop

fix a bug in nlp: text_matching/sentence_transformers when last dim is 1 and reduce mid dim (#34941)

* Fix typos in english docs for diag and diagflat.

* [NPU] Support npu op:(1)arg_min (2)arg_max

* Modify and add unit test cases

* Modify unit test cases

* change bilinear thread for nano and tx2

* change bilinear thread for nano and tx2

* add size npu op

* modify support data type

* no longer use NPU size OP

* remove useless comments, add test case

* fix copyright, remove useless include

Change the invoking method of settiem by Ellipsis and None index from numpy to set_value op (#34911)

* Change invoking mathod of the settiem by Ellipsis and None index from numpy to set_value op

* add none_axes into attr of set_value_op in dygraph mode

* Remove force_fp32_output from elementwise_add quantization

* Fix cpu_quantize_placement test

* Review related changes

* - Added softmax without caching

* - Binary is no longer manually cached

* - Activation onednn caching removed

* - Removed manual caching of activation

* - modified UT

* - fix

* - fix

* - fixes to building

* - fix

* - fix

* - fix to UT

* - Faulty UT workaround

* - approval workaround

* - Fixes after review

* - compilation fixes

* - more lint fixes

* - more fixes after review

* - fixes after another round of review

* - hopefully compilation fix

- compilation fix

* Add NPU kernel for nearest_interp op

* Add grad op

* Modify codes according to the review comments

* Modify codes according to the review comments