* preln_residual 2 fused_bias_residual

* skip layernorm fix and ut

* code refine

* code style refine

* fix ut

* fix output

* add trt layer fall back info

* refine op teller and ut

* DropoutMaskOut output fix

* jit layer support multi thread

* fix bug

* clone prediector not do graph optimizer

* format

* fix comment and format

* fix override and fromat

* fix

* fix

* add dimension check for deformable_conv

* add unittest

* [Divide by 0 Error] add lu check

* [Divide by 0 Error] lu check migrate to c++

* [Divide by 0 Error] add eig check

* [Divide by 0 Error] eig check migrate to c++

* [Divide by 0 Error] Fix class name error

* [Divide by 0 Error] add norm check

* [Divide by 0 Error] fix x AttributeError

* [Divide by 0 Error] norm check migrate to c++

* A leap of try for cudaLaunchCooperativeKernel

* fix bugs

* Totally replace the lar cuda kernel

* Fix bugs

* fix code according to comments

* fix codes according to  review comments

* adding some function overload

* relocate the power operation.

* add bf16 support for index select relevant ops

* revert bf16 type change.

* add changes for more op

* fix code writting bugs

* add clip_grad_norm_ api.

* fix docs and some details according to the comments.

* fix code style.

* fix no_grad problem, and fix doc.

* fix code style.

* fix doc and remove type information

* add axis check for unbind

* add axis range check for unbind

* update unittest and axis validation for unbind

* add unittest invalid axis for unbind

* restore axis extract for unbind

* [PrimCinn]Fix some vars are wrongly gc in CINN+InterpreterCore

* fix baseline unittest config

* fix code style

* profile reduce kernel for fp16 and reduceHigherdim

* use reinterpret_cast

* fix for CI on ROCm

* add Macro for ROCm

* ROCm CI config

* ROCm CI config

* unit test repair

* pull

* add common_funcs.h

* reduceType

* Update reduce_function.h

* not higher

* rename

* implement of matmul using cublasLt instead of cublas

* cublasLt bugfix

* Update matmul_kernel_impl.h

* Update matmul_kernel_impl_via_blasLt.h

* for-loop-algo

* PR comments changes

* add macro

* ci unused variable isCublasLt

* ci unused variable isCublasLt macro

* split matmul to autotune

* rewrite the split kernel with segmented_array

* rewrite the split kernel with segmented_array

* rewrite the split kernel with segmented_array

* add some method for cuda_graph

* fix bugs for rocm

* change for ci-error

* i dont know why ci-model-benchmark gives a shit error, so i recover codes with original one to see if original codes work.

* add some changes for passing mode_benchmark and coverage ci

* fix ci error

* fix ci-rocm error

* add some changes for header

---------
Co-authored-by: Nzhangbopd <1299246947@qq.com>
Co-authored-by: NBo Zhang <105368690+zhangbopd@users.noreply.github.com>

* gn_silu

* add ut

* set TIMEOUT

* correct comments

* comments

* disable windows ut

* rename parameter

* check dygraph on for op tests

* reset eigh and modify prelu&sign

* update eager_op_test

* lint

* add more ops

* fix reduce

* modify reduce test

* reset reduce_op

* modify matmul test

* revert prelu

* update elementwise ops tests

* add more ops

* modify sum&split

* lint

* rm check_dygraph

* revert pow

* modify add for cpu test

* revert reshape

* modify min

* more ops

* revert some ops

* reset some ops

* disable integer

* disable integer

* add cast layer

* polish static grad op maker gen

* fix some bugs

* fix static code gen

* solve conflict

* modify composite grad maker name

* integrate phi and fluid info in static code gen

* rename some composite maker

* modify static code gen format

* add FP16 dtype for CastNumpy2Scalar

* fix throw message

* add test

* fix SyntaxWarning

* test skip for float16

* fix dtype mistakes

* fix div 0 error of NoamDecay

* add unittest

* Update lr.py

* [CustomDevice] add custom device api

* update

* update

* test=document_fix

* update

* update

* add  examples

* [KUNLUN] rename test_pool_max_op.py

* update xpu toolchain

* fix incorrect output shape of broadcast

* add unittest