* optimize logsumexp in small data scale

* fix

* fix

* add #pragma once

* swith to use aligned_vector and support arbitrarily shape

* fix store

* fix store

* refine for special cases

* try

* fix

* update

* fix

* fix all_reduce

* try

* fix rocm bug

* fix rocm bug

* fix rocm bug

* fix rocm bug

* fix rocm bug

* fix rocm bug

* fix rocm bug

* fix rocm bug

* add fp&bf16 bernoulli

* add check_dtype & fix error

* fix rocm error

* support master_grad for adam and momentum

Co-authored-by: zhangting_2017@163.com <zhangting2020>

* Reimplement the check_nan_inf function as check_numerics kernel.

* Remove the cpu implemention to phi.

* Add ifdef for the including of omp.h.

* Move the use of FLAGS_check_nan_inf_level out of header file.

* Implement a common PrintAndThrowError function.

* Fix the error using of __NVCC__, which should be instead with __CUDA_ARCH__.

* Add dependency of phi.

* Polish codes and unittest.

* create phi so

* fix ci bugs

* fix py3 bugs

* add file

* fix py3 bugs

* fix windows bugs

* perfect so

* fix py3 bugs

* delete all static target in phi

* fix windows bugs

* fix py3 bugs

* fix ci bugs

* fix windows bugs

* fix bugs: gflags can't be linked by dynamic and static lib

* fix bugs that can not load 3rd party

* fix ci bugs

* fix compile bugs

* fix py3 bugs

* fix conflict

* fix xpu bugs

* fix mac compile bugs

* fix psgpu bugs

* fix inference failed

* deal with conflict

* fix LIBRARY_PATH bug

* fix windows bugs

* fix onednn error

* fix windows compile bugs

* fix windows compile bugs

* fix test_cuda_graph_static_mode_error aborted

* fix windows bugs

* fix mac-python3 error

* fix hip compile bugs

* change mode to static

* change to static mode

* fix ci bugs

* fix py3 bugs

* fix windows bugs

* fix bugs

* add static flag

* add PADDLE_API

* change position of PADDLE_API

* fix windows bugs

* change mode to dynamic lib

* fix windows static bugs

* deal with conflict

* fix windows unit bug

* fix coverage

* deal with conflict

* fix windows-inference

* fix py3 bugs

* fix bugs when compile type_info

* fix compile bugs

* fix py3 bugs

* fix windows bugs

* fix windows openblas

* fix xpu bugs

* fix enforce_test in windows

* update code according comment

* fix windows cmake bug

* fix windows bugs

* fix windows bugs

* delete cinn unittest

* fix cinn bugs

---------
Co-authored-by: lzydev <1528794076@qq.com>

Removed unnecessary header files introduced

* [AMP OP&Test] Support float16 in selu

* fix

* Add GPU kernel for multiclass_nms3 op

* Make multiclass_nms3 gpu kernel output consistent with cpu kernel

* Fix API incompatibility

* Fix unittests on builds without CUDA

* Fix ROCM build

* Remove fluid headers; Use default atol for unittest

* Change function and variable naming

* Add comments; Reduce redundant code

* Use paddle test framework

* delete bf16 of cross entropy

* delete bf16 of cross entropy

* add meshgrid,expand_as, prod and grad bf16 kernel

* fix bf16 for optest

* modify code style

* fix amp test

* Add i1 and i1e op

* resolve merge conflicts

* add bf16 for mode

* remove random seed 666

* try to fix op_type error

* test for me

* try to fix op_type

* fix redundancy code

* add fp,bf for lastdim

* fix some error

* simplify code

* fix shape error

* optype error

* fix skipif bf16

* modify lerp_kernel.cu

* pre-commit

* fix some CI issues

* fix some CI issues

* fix some CI issues

* fix some CI issues

* fix some CI issues

* fix some CI issues

* fix some CI issues

* fix some CI issues

* Add files via upload

fix some CI issues

* update

* fix bug

* test

* test

* update

* update mutable_data

* fix bug

* update

* fix bug

* update output type reg

* update

* update

* [phi]mv StftKernel to phi

* [phi] fix KernelSignature

* [phi]fix arr error

* [phi] Disable check_dygraph

* [phi]fix include

* [phi] rewrite mutable_data, add output register

* [phi] fix  Alloc

* [phi] fix Alloc again

* [phi] fix mutable_data

* [phi] fix onesided_out Resize

* update

* Update backward.h

* Update composite_backward_api.h

* Update tensor_utils.cc

* Update backward.cc

* update

* stype

* update

* add ctest

* code stype

* move OneHotRawKernel to legacy

* fix

* added base code for i0 and i0e

* added grad base code for i0 and i0e

* added i0 and i0e python code

* added ops and backward yaml config

* added i0 and i0e cpu kernel, but not test.

* added i0 and i0e code and unitest files

* added test files

* added i0/i0e gpu implementation code

* updated code style

* updated code style

* fixed unitests code

* updated i0 with eigen3

* fixed bug and added more test cases

* refactor: fixed static graph bug

* refactor: removed i0 and i0e from op_compat

* refactor: updated code style

* refactor: updated op_compat.yaml

* refactor: updated op_compat.yaml

* refactor: fixed op name mapping and optimize unittest case

* refactor: manually implement i0 / i0e

* refactor: added grad kernel for i0 / i0e,didn't finish

* Update math.py

* refactor: added equation to doc in English and added comments for computing i0 / i0e gradient

* refactor: removed eigen implementation

* refactor: finished i0 / i0e cpu and gpu op

* refactor: updated code style

* fix: find  a bug but not fix

* fix: incorrect unittest cases

* update: updated code style and remove my file

* update: updated unittest case

* fix: fixed sign error

* fix: fixed mistakes when merging

* refactor: updated code style

* refactor: remove unused code

* refactor: updated code style

* add rules

* modify silu_grad input

* modify kernel signature

* modify kernel signature

* code style

* review

* sequence_mask functionalization

* fix sequence_mask test

* update

* update

* add index_put api

* fix value broadcast in backward and add test case in static

* add timeout=120s for index_put

* add op_compat for index_put

* add inplace index_put test

* add test case when index tensor in indices is int32 when indices.size less than x.dims

* add index_put api backward in cpu place

* add backward test case

* refactor code to delete some duplicated code

* replace reshape with resize for decrease extra memcpy

* add datatype flag in backward yaml

* fix bug in documentation

* Update python/paddle/tensor/manipulation.py

---------
Co-authored-by: NLigoml <39876205+Ligoml@users.noreply.github.com>

This reverts commit 9e9b705a.

* add complex support for  optest

* add complex grad test

* append one

* move some debug info

* move some debug info

* move some debug info

* move some debug info

* add more complex test

* Fix naming ambiguity

* Revert "add more complex test"

This reverts commit dbcb0516b8e53ba42e2d6089878a39b395345969.

* change backward gradient, add TODO