* add multi tensor for adam

* add merged_adam op

* refine code

* refine adam compute logic

* Add fp16 support for scale/bias for fused_layernnorm_residual_dropout_bias op.

* move mid api and rename kernel

* use empty kernel

This reverts commit c0e2b98e.

* fix the wrong filename

* first commit

* adjust the full kernel

* remove creation.h

* use Empty to create tensor in full

* move gpu_impl of elementwise kernel

* change copyright to 2022

* added exp activation and use_dst_for_bwd kernels

* CI RERUN

* minor change

* init commit: new elem_mul_grad

* add template speciallization for complex in multiply

* reply review comments

* correct dx and dy computation when T is complex

* reply review comments

* update to new ReduceRunctor

* mul-output broadcast

* call functions

* call functions with comments

* remove comments

* change 'math' to 'math_kernel'

* fix compile bugs

* merge develop

* fix compile bugs

* fix compile bugs

* move reduce files by new rule

* add set header

* format code style

* merge develop and fix conflict

* merge develop and fix conflict
Co-authored-by: NYuanRisheng <yuanrisheng@baidu.com>

* fix for matmul_v2 broadcasting

* fix for output shape not broadcasted

* add huber_loss for kunlun

* update xpu.cmake

* update unitests

* update unitests

* update elementwise_add

* update elementwise_add

* update elementwise_add

* add elementwise div

* move mul and div grad functor

* Combine multiple CUDA kernels

* Update the reduce interface call

* add multi-output

* add multi-output div

* add branch judge

* Package branch

* Combine the x and y functions into one

Add OpFunctor and replace cast, scale, clip, bce_loss and abs_grad with elementwise_no_broadcast (#38500)

* Fix memcpyD2H sync behavior with other stream

* add wait

* add wait

* add wait

* change 'math' to 'math_kernel'

* fix compile bugs

* merge develop

* fix compile bugs

* move cpu_impl of elementwise kernel to new directory

[NPU] add pad and pad_grad

* move inner cast api to cast_kernel.h

* resolve conflit

* [XPU]add split op for kunlun2,*test=kunlun

* [XPU]add split op for kunlun2,*test=kunlun

* [XPU]add split op for kunlun,*test=kunlun
Co-authored-by: NQingshuChen <chenqingshu@baidu.com>

* add new api/op kthvalue

* kthvalue cuda kernel to cub sorting

* fix example code error

* throw errors instead of LOG in cuda sort

* throw errors by Paddle_ENFORCE

* change 'math' to 'math_kernel'

* fix compile bugs

* merge develop

* fix compile bugs

* fix compile bugs

* add fold opereators, test=develop

* add fold opereators, test=develop

* add fold opereators, test=develop

* update fold op error test, test=develop

* fix unitext, test=develop

* fix unitext, test=develop

Paddle new APIs: put_along_axis.

Xu Huang is on holiday so we created this PR to work on it. It is based on his PR: https://github.com/PaddlePaddle/Paddle/pull/37921

* unify data layout

* fix test_transfer_layout error

* change 'math' to 'math_kernel'

* fix compile bugs

* merge develop

* fix compile bugs

LGTM

* add sigmoid cross entropy with logits to kl1. test=kunlun

* add sigmoid cross entropy with logits to kl1. test=kunlun

Add exp, abs_grad, reciprocal, reciprocal_grad operator for XPU and update xpu2_op_list.h,test=kunlun (#38570)

* add new OP mode

* rename trans-variable name and fix UT

* add cpu kernel of lstsq

* update

* modify code style

* modify unittest

* remove support for complex

    将cuSparse的handle与DeviceContext进行绑定，避免op中进行创建和销毁
    添加对cuSparse中dense和sparse转换的API进行封装
    添加对封装的API的单测

* dynamic shape clone supported