* cherry pick from #43397

* fix code

* [EinsumOp] Polish forward logic and backward logic for optimize (#42603)

* change logic for optimize

* modifty

* merge

* change einsum_v2 as default and add new flags: FLAG_einsum_opt=1|0 (#43010)

* [EinsumOp] Make EinsumOp support bfloat16. (#43085)

* change einsum_v2 as default and add new flags: FLAG_einsum_opt=1|0

* make EInsumOP support bf16

* add unittest for BF16

* add condition for test_BF16

* fix bugs

* fix

* change the backward api to fit einsum op

* Extend python einsum interface to make einsum_v2 support multi-operands and switch it to default.

* add opt_einsum dependence

* add yaml and support eager model

* fix by code review

* full api fix

* when out is None, go old dygraph mode

* by static check

* first version: support 2-inputs forwards. TODO: 1. backward  2. BroadCast  3. MultiVariable

* time out -> 120

* cherry-pick python/paddle/utils/code_gen/backward.yaml

* remove unsupported yaml
Co-authored-by: NZhanlue Yang <jim19930609@gmail.com>

* add rsqrt yaml and unittest (#41443)

* Add expand equal all yaml (#41540)

* add expand, poisson

* add poison grad

* add expand equal_all poisson triangular solve yaml
Co-authored-by: Nhong <43953930+phlrain@users.noreply.github.com>

Cherry-pick #40338 #41741 #41313

* split reduce_kernel

* rm reduce_kernel in cmake

* split reduce_grad kernels

* fix cmake build error

* format code

* fix standalone_executor_test error

* add multi_dot,maxout,multiplex yaml

* add code converage

* update

* add conv yaml

* add backward

* remove useless code

* fix bug

* fix bug

* revert fluid dygraph conv2d

* remove useless infermeta function

* fix meta fn deluplicat error

* conv using custom impl

* remove amp include

* fix bug

* use cudnn = true

* fix test mkldnn caching bug

* [Refactor] refactored eager_gen.py PR #2

* [DoubleGrad PR #1] Decoupled code generation logics for Dygraph ForwardFunctions and GradNodes

* Fixed minor issue

* Adjusted logics of GenerateNodeCreationCodes and GenerateForwardDefinition

* Fixed issues

* Supported higher-order grad node generation

* [DoubleGrad PR #4] Supported higher-order GradNode generation

* [DoubleGrad #4] Bug Fixes to Double Grad Node Generation

* Fixed yaml typo

* Fixed yaml typo

* fixed minor issues

* [DoubleGrad PR #5] Enabled gradient computations for grad_tensors passed to paddle.grad()

* Fixed minor issue

* Fixed CI-Inference issue

* Fixed CI-inference issues

* [DoubleGrad PR #7] paddle.grad() to copy backward graph before backward run

* Fixed minor issues

* Fixed issue with backward graph construction logic

* Fixed implementation issues with backward graph reconstruction

* Fixed unittest issue

* Fixed issues

* [DoubleGrad PR #8] Enabled triple grads for sigmoid and matmul

* Fixed issues with phi kernel

* Added triple grad test case

* Fixed minor issue

* add concat_grad kernel

* fix error

* remove comment code

* fix outs nullptr error

* change to phi header

* add concat_grad declare for standalone_executor_test

* add yaml for ele_max ele_min

* add yaml for: mish / logexpsum / expm1 / elemenwise_pow / elementwise_floordiv

* move clip op to phi

* fix as review

* update hierarchical_sigmoid_kernel.cc

* update selected_rows

* update clip_kernel.cu

* fix as review

* 1. add the python api grad 2. add final and intermediate state vlog 3. change the python_api error logic

* add python api or close the check_eager=True

* fix the compatibility

* add several yaml

* polish strided slice kernel & add yaml

* reorder yaml

* add several yaml

* revert yaml config change

* resolve conflict

* Update test_strided_slice_op.py

* rename scalar_array to int_array

* update cmake

* fix conflict

* remove useless log

Revert "Revert "[Phi] Move elementwise_floordiv and elementwise_pow to phi (#40993)" (#41065)" (#41110)

This reverts commit 3a6f1135.

This reverts commit 054fc997.

This reverts commit 05f3d48e.

This reverts commit 9c0eaada.

This reverts commit b532315d.

This reverts commit e77a947e.

* [Phi] trans logsumexp op

* fix bugs

* fix bugs

* fix bugs

* fix bugs

* fix bugs

* fix bugs

* add sig

* fix sig bugs

* fix sig bugs

* fix xpu bugs

* fix review bugs

* test=develop

* mv floordiv to phi

* mv elementwise_pow to phi

* fix as review

* move momentum, rmsprop to phi; test=develop

* update

* update

* update

* update

* udpate; test=develop

* fix xpu npu bugs; test=develop

* fix npu bug; test=develop

* fix windows compile error; test=develop

* fix windows compile error; test=develop

* polish code; test=develop

* fix conflict; test=develop

* add meshgrid;

* update

* polish code

* polish code;

* fix bug

* format; remove useless code

* fix npu bug

* fix bug

* update

* add forward case

* update

* update; test=develop

* add some grad kernel; test=develop

* move gpu kernel; test=develop

* update

* update;

* update test;

* fix selected rows bug;

* add mix vector include ;

* add mixed vector depen; test=develop

* add logit grad signature;

* polish code

* fix bug;

* add namespace for abs

* revert code

* not move softsign

* revmove duplate register;

* fix softsign bug

* polish code

* format

* format

* fix bug

* remove cmake dep

* add square sqrt selected rows support

* update

* remove clip norm

* add standalone executor sqrt dep

* standalone exec denp sqrt

* remove sqrt op in cmkaelist

* open some case

* remove in_dtype, out_dtype in redcue grad

* set the dtype and layout in noneedbufferInputs func

* moving OP

* move forward

* move grad and infershape

* code format

* format code

* fix code

* fix code

* fix CMakerLists.txt

* fix comments

* Refine CMakeLists for rocm ci

* move slice to pten

* merge develop; test=develop

* fix slice bug;

* update

* update

* fix error

* update

* fix bug

* polish code

* polish code

* polish code

* try to fix windows bug

* add gpu compile flag;

* try to fix

* remov template;

* polish code;

* fix npu bug;

* fix npu bug

* fix npu bug; test=develop

* fix slice bug;

* remove no need dep

* [Phi] Migrate strided_slice into Phi

* [Phi] Migrate strided_slice into Phi

* fix compilation problem

* [Phi] Migrate Adam and Adamw into Phi

* fix compile error and unittest ok

* fix compile error and unittest ok

* fix undefined reference to fLI::FLAGS

* test depend on operator

* fix cmake

* fix xpu compile

* fix infrt

* fix amp_type_traits

* fix amp_type_traits

* modify according reviewer

* modify according reviewer

* fix dtype float16

* fix typo

* fix Cmake

* fix code style

* add mul phi kernel

* remove mul op kernel

* remove original mul grad op

* fix cinn test

* fix dygraph test failed

* move deformable_conv_grad to phi

* move infershape of deformable_conv to phi

* adjust some code format

* move deformable_conv_v1 to phi