* support isfinite for phi

* mark v2

* fixed bugs

* fixed include bugs

* deal with comments

* decoupling selected_rows

* rm bfloat16

* fixed infermeta

* fixed code style

* rm useless code

* replace pt by pd

* [custom kernel]Upgade support for multi libs

* upgrade phi_custom_kernel deps

* [MLU] add fleet init api and collective api pytest for mlu

* fix no value for argument 'data_type' in method call

* [Phi] transfer old kernel to pten kernel for the graph_send_recv op

* update the code for the define of graph_send_recv

* fix the gradient problem for graph_send_recv

* fix the compile problem

* update the enfore message for the windows

* update the code for the compiler

* update compiler problem for the windows

* udpate the code for windows

* fix some format problem

* [Phi] move InferShape for truncated_gaussian_random and gaussian_random

* [Phi] delete useless codes

* move infershapes to phi

* update code format

* update code format

* add profiler helper

* fix unittest

* improve test coverage rate

* add python profiler package

* update according to review

* fix bug

* fix bug

* fix bug

* add unit test

* Revert "add unit test"

This reverts commit 4e69ff71b0645e069afe5dd8fea0d07717852c48.

* reduce for pr

* add unit test

* modify for pr

* fix unittest

* update for ci coverage

* modify according to review

* fix bug

* improve coverage

* add python api test in TestOp

* test_python_api if self.python_api is set

* fix code by CR

* refactor unittest for nearest_interp_v2_op_xpu. test=kunlun

* fix code style. test=kunlun

* fix code style. test=kunlun

* move bincount OP to phi

* fix dtype

* set_dtype by weights or x

* fix conflicts

* Added cuBlasLtHandle_t to device context.

* Added fused_gemm_epilogue op.

1. Added fused_gemm_epilogue op to leverage cuBlastLt Epilogue.
2. Support fusion Act(X*Y + bias), X'dims >=2 and Y'dims shoule be 2.
2. Act currently only be supported ReLU. (Will add GeLU in the future).

* Added UT to fused_gemm_epilogue op.

* Added LinearAct Pattern

1. Added LinearAct into graph_pattern_detector.* to define (2.)'s
pattern.
2. LinearAct is used to detect act(element_add(matmul_v2(x, w), bias)).
3. act currently only support ReLU (Will support GeLU in the future).

* Added FuseGemmEpiloguePass

1, Added FuseGemmEpiloguePass to handle nn.Linear + Act{ReLU}
fusion (GeLU will be supported in the future).
2. Only support matmul_v2 from nn.Linear.

* Added pybind to BuildStrageter.fuse_gemm_epilogue_.

* Added UT for fuse_gemm_epilogue_pass.

* GeLU support and EpilogueSingleton

1. Added GeLU support to fused_gemm_epilogue op.
2. Added EpilogueSingleton to cache auxiliary pointer.
3. Added related UTs.

* Rename cublaslt_epilogue_opto gemm_epilogue_op.*.

* Added both train and infer pattern to LinearAct.

1. Added support of fwd graph with grap_ops linking to LinearAct.
2. Added related changes to fuse_gemm_epilogue_pass for above
modification.

* Changed CUDA requirement from 11.4 to 11.6 for fuse_gemm_epilogue_pass.

* Added identity activation support to gemm_epilogue_op.

* Added Linear Fusion (matmul_v2 + ele_add)

1. Added matmul_v2 + ele_add pattern to LinearActPattern.
2. Added matmul_v2 + ele_add support to fuse_gemm_epilogue_pass.

* Rename gemm_epilogue_op.* to fused_gemm_epilogue_op.*

* Add fused_gemm_epilogue_grad op.

1. Added fused_gemm_epilogue_grad to support backward epilogue fusion.

* Add UTs to fused_gemm_epilogue_grad_op.

* Change attribute name in fused_gemm_epilogue_grad_op for clearing.

* Allow DX and DBias be dispensable to fused_gemm_epilogue_grad op.

* Added ElementwiseAdd+Matmul+Act graph pattern detection.

* Fuse backward of Linear( Act(x))

1. Added backward fusion pass to Linear( Act(x)).
2. Added backward fusion pass to Linear(x).

* Added UTs to backward fusion of Linear(Act(x)).

* Complete document of arguments to fused_gemm_epilogue_op.

* Made arguments of some functions pass by reference.

* Modify code with review comments.

1. Made arguments of some function pass by reference.
2. Removed redundant code.
3. Followed Google code style to change code.

* Made 'const' code style be consistent

* Fixed random seed of python UTs.

* Set Compiling constrains to cuBlasLt

1. Require CUDA 11.6+
2. Remove fuse_gemm_epilogue related tests when CUDA < 11.6.

* Code Reivew from Paddle

1. Changed arguments name is_first_gemm to without_x_gradient for
clearing.
2. Applied PADDLE_THROW in fused_gemm_epilogue_op.

* Remove EpilogueSingleton

1. Applied ReserveSpace to replace Epilogue for passing auxiliary
pointers between FWD and BWD.

* Fix a logical error and enhance UTs.

1. Added act op count checking in UTs.
2. Fix issue to fuse backward or ReLU(Linear(X)).
3. TODO: solve GELU fusion issues.

* Fix Linear and GeLU fusion issues.

1. Modified graph_detech_pattern to fit with both linear wiht gelu or
relu.
2. Modified data range in Uts to allow negative values.

* Removed fused_gemm_epilogue_op.h.

* Rename namespace pten to phi.

* Rename name of arguments in fused_gemm_epilogue_op

1. bias -> Bias.
2. out -> Out.
3. reserve_space -> ReserveSpace.

* Change EpiloguePassActivationCache as local variable.

1. Removed singleton in EpiloguePassActivationCache.
2. Made EpiloguePassActivationCache as an argument to each pass
functions.

* Add is_empty

* fixed for CI

* fixed code style

* resolve conflict

* deal with comments

* replace pt by pd

* infrt add trt engine

* update engine name

* move elementwise_div grad

* change mutable_data to alloc

* fix compile bugs

* dbg pool infer shapes

* dbg

* fix format

* fix_conv2d_trt_convert_test_case

* fix_conv2d_trt_convert_test_case

* fix_conv2d_trt_convert_test_case

* fix_conv2d_trt_convert_test_case

* remove storage deps of empty

* remove invalid empty method

* remove error empty using

* fix test_sparse_utils_dev_api

* revert some sparse change

* add memset for conv grad

* resolve conflict

* resolve conflict

* resolve conflict

* add gaussian random

* add full

* refine reduce

* refine code

* refine gaussian_random unittest

* add unittest for fill_any_like fill_constant

* [phi] move multi_dot OP

* fix the segment bug

* fix bug

* delete useless comment

* fix CI bug

* engine support pp

* fix format

* avoid multi print

* fix convert

* bug fix

* add pp unittest

* add activ

* refine unittest

* refine unittest

* refine unittest

* refine unittest

* refine code

* [MLU]support reduce tensors on mlu

* [MLU]fix compiler options

* [Phi]Migrate Adamax into phi

* Add adadelta kernel

* refine amp.decorate code example

* refine code

* replace prefix pt by pd

* replace added kernel

* revert util change

* pd kernel to phi

* resolve conflict

* resolve conflict

* move dist op to phi

* fix

* fix

* fix as reviews

* Migrate triangular_solve op into phi

* fix CI

* move MatrixReduceSum to phi funcs

* move MatrixReduceSum to phi funcs

* fix comment

* fic CI

* remove eig dep for svd helper

* fix win failed

* [Phi] move infershape for mv

* [Phi] delete extra codes for mv

* fix benchmark and communicator config

* fix bugs of the_one_ps

* multi program and fix bug in optimizer

* multi program in the_one_ps

* public commcontext

* ps optimizer multi programs

* the one ps merge

* fix bug in test