[Hybrid Performance] Move the cast op of AMP which cast fp32 param to fp16 param to the optimizer (#34965)

* fix ext_tensor.cast failed bug

* remove useless deps

* fix windows cmake failed

* try to fix windows make failed

* fix make error on windwos

* Add function to disable paddle signal handler

Paddle used google::InstallFaultSignalHandler to handle selected system signals,
mainly for debugging and bug report purposes.

However, this can be conflicted with other python packages whoever captures similar signals.
Such python package involves tvm and more

To resolve this issue, we support a function to disable signal handler

* Remove signal test from WIN32 platform

* Remove redundant return from disable_signal_handler() function

* Add detailed messages to en_doc

* add retry for HcclGetRootInfo

* refine code

* reduce retry interval

* support class center sample of PartialFC

* copy boost optional.hpp to paddle

* copy boost optional.hpp to paddle

* move directions

* del fluid/utils

* modify .hpp to .h

* move directions

* modify to paddle::optional

* add modification description

* format code stype for the files in paddle/utils

* format code stype

* - disabled caching of layer norm

- fix in compilation

- compilation fix

- transpose caching disabled

- compilation fix

- more compilation fixes

- sum caching disabled

- compilation fix

* - LRN with disabled cache

* lint fixes

* [bug fix] fix unfold negative_size_param

* add mkl multi-thread test cases

* fix codestyle

* fix codestyle & enable ernie mkl test

* dygraph support more ctc grad scale

* scale for 1.x

* fix unitest

* fix unitest

* format code

* fix unittest

* fix log info

* unittest cov

* fix format;notest,test=cpu,coverage

* skip ctc_loss egs;test=cpu

* warpctc grad cov;test=coverage

* add dygraph test;test=coverage

* format;test=cpu,coverage

* format;test=cpu

* add api compat;test=cpu

* add cpu test

* rename

* rename

* fix

* fix test

* format

* eigen cpu

* eigen gpu grad pass

* cuda gpu pass

* format

* fix ci

* add inplace passes and tests

* update

* fix use_cuda undefined
fix compile error of op compat

* add more ut

* fix CPU CI error

* check adam unique

* fix mac/windows ci, improve coverage

* fix ci error

* follow weihang's comment

* fix BlockDesc::MoveFrom

* follow qiuliang's comment

* update

* follow huihuang's comments

fix a bug in nlp: text_matching/sentence_transformers when last dim is 1 and reduce mid dim (#34941)

* Fix typos in english docs for diag and diagflat.

* [NPU] Support npu op:(1)arg_min (2)arg_max

* Modify and add unit test cases

* Modify unit test cases

* change bilinear thread for nano and tx2

* change bilinear thread for nano and tx2

* add size npu op

* modify support data type

* no longer use NPU size OP

* remove useless comments, add test case

* fix copyright, remove useless include

Change the invoking method of settiem by Ellipsis and None index from numpy to set_value op (#34911)

* Change invoking mathod of the settiem by Ellipsis and None index from numpy to set_value op

* add none_axes into attr of set_value_op in dygraph mode

* Remove force_fp32_output from elementwise_add quantization

* Fix cpu_quantize_placement test

* Review related changes

* - Added softmax without caching

* - Binary is no longer manually cached

* - Activation onednn caching removed

* - Removed manual caching of activation

* - modified UT

* - fix

* - fix

* - fixes to building

* - fix

* - fix

* - fix to UT

* - Faulty UT workaround

* - approval workaround

* - Fixes after review

* - compilation fixes

* - more lint fixes

* - more fixes after review

* - fixes after another round of review

* - hopefully compilation fix

- compilation fix

* Add NPU kernel for nearest_interp op

* Add grad op

* Modify codes according to the review comments

* Modify codes according to the review comments

* add unique_consecutive_op

* add unique_consecutive_op

* add unique_consecutive_op

* add unique_consecutive_op

* add unique_consecutive_op

* add unique_consecutive_op

* add unique_consecutive_op

* add unique_consecutive_op

* remove unity build

* add unique_consecutive op

* add unique_consecutive op

* add enable static

* add noqa

* add space line

* add default case.

* add comma

* add space line

* modify unique_consecutive unittest

* optimize ut coverage

* rebase develop

* improve coverage

* update en docs

* update en docs

* update en docs

* update en docs

* update en docs

* update en doc

* support margin loss (arcface, cosface, sphereface)

* Support NPU OP hard_swish and hard_swish_grad

* Support NPU OP hard_swish and hard_swish_grad

* add the unittest to compare the result between npu ans cpu

* format the prompt of exception

* replace Min and Max op by ClipByValue op

* fix the precision problem for fp16

* Using HardtanhGrad to improve performace

* add p_norm_op_npu

* remove p_norm_grad op

* update

* OP dot: refactor CPU kernels and get better loop performance.

* Minor fix on code format.

* Fixed minor errors.

* Add new API: einsum

* Update the Einsum unit test.

One case failed with matmul_v2, where the dtype is int64:

a = np.arange(2 * 3 * 1).reshape(2, 3, 1)
b = np.arange(1)
paddle.einsum("...i, ...i", a, b)

* Test cases in test_einsum test floating point dtypes only.

As of now Paddle only supports float/double dtypes in matmul, which is
one of building blocks of this Einsum implementation. We decide not to
test einsum against other dtypes.

* Polish format.

* More formatting.

* Format...

* Einsum: improve test coverage.

* Einsum: bug fixes and more testcases for testing error messages

* Einsum: fix format..

* Einsum: fixed typo and format.

* Einsum: format again...

* Einsum: applied suggested changes.

* Einsum API: improve API documentation.

* Einsum API: apply suggested changes.

* Einsum API: Add dygraph only note.

* Einsum API: Add dygraph only note.

* Einsum API: fixed unittest.

* Fix a bug : can't load more than one custom op module

* Fix a bug : can't load more than one custom op module

* add test for load multiple modules of custom c++ op

* add config for Coverage CI