* add inferface of get registered phi kernels

* change KernelType to KernelKey

* add test

* refactor code

* [XPU] add fp16 support for shape op.

* [XPU] add fp16 support for lookup_table_v2 op.

* update approval list: add qingshu's id.

* remove utils

* remove utils

* remove utils

* remove utils

* Update get_data_from_tensor.h

* Update rnn_functor.h

* Update rnn_grad_kernel.cu.cc

* Update rnn_kernel.cu.cc

* Update rnn_kernel.cc

* Update rnn_grad_kernel.cu.cc

* Update rnn_functor.h

* Update rnn_kernel.cu.cc

* Update rnn_kernel.cc

* remove utils

* Update rnn_functor.h

* remove utils

* remove utils

* remove utils

* remove utils

* remove utils

* Update rnn_functor.h

* Update unsqueeze_op.h

* Update utils.h

* roll back

* Update tensor_utils.h

* Update tensor_utils.h

* Update tensor_utils.h

* Update tensor_utils.h

* Update tensor_utils.h

* use TensorToVector

* use TensorToVector

* use TensorToVector

* use TensorToVector

* use TensorToVector

* Update rnn_kernel.cc

* Update rnn_grad_kernel.cc

* Update rnn_functor.h

* Update rnn_grad_kernel.cu.cc

* Update rnn_kernel.cu.cc

* Update rnn_functor.h

* Update rnn_grad_kernel.cu.cc

* Update rnn_kernel.cu.cc

* Update rnn_functor.h

* Update rnn_grad_kernel.cu.cc

* Update rnn_kernel.cu.cc

* add TensorToVector

* roll back

* Update tensor_utils.h

* Update rnn_functor.h

* Update rnn_grad_kernel.cu.cc

* Update tensor_utils.h

* Update rnn_kernel.cu.cc

* Update rnn_grad_kernel.cc

* Update rnn_kernel.cc

* Update rnn_grad_kernel.cu.cc

* Update rnn_kernel.cu.cc

* Update rnn_grad_kernel.cc

* Update rnn_kernel.cc

* TensorCopySync to phi::Copy

* fix codestyle

* rnn_kernel.cc: add ;

* replace all GetDataFromTensor with phi::GetVectorFromTensor

* delete include of util.h

* Fix fp16 dtype checking for AvgPool1D op

* Update code style for PR-CI-Static-Check

* add sm version check

* use GetGPUComputeCapability

* [Tensor Operants & Prim] Tensor pow API uses elementwise_pow

* unittest change to fill_constant+elementwise_pow

* support fp16 on AvgPool3D

* Apply suggestions from code review

* fix fp16 dtype checking for clip op

* modify the name

* fix type error

* fix check error

* Update test_clip_op.py

fix test error

* Update test_clip_op.py

fix code style

---------
Co-authored-by: NZhang Ting <Douyaer2020@qq.com>

* [Error Msg] Polish error message when GPU kernel not found

* Only test in GPU environment

* fix fp16 dtype checking for argmax op

* run fp16 test when place is gpu

* Update search.py

fix doc

* add float16 in python/paddle/math

* add unittest for float16

* add float16 support in python.paddle.tensor.search.where

* remove fp16 error cases

* Add NotImplementedError unittest

* fix codestyle

* fluid to paddle.static; add cases with GPU

* Add float16 in English docs

* conflict

* add UpdateSliceAttrs

* [kunlun] support reduce_scatter

* uncomment unittest

* update xccl to 1.0.10

* add TypeUniquer and IrContext

* refine include code

* add Type, TypeBase

* add built-in type

* add bulit-in Float32Type

* refine ut

* refine code

* refine code

* delete type_base

* rename ImplType to StorageType

* rename ImplType to StorageType

* add macros util for register type

* add macros util for register type

* refine name

* refine name

* change storage manager

* add multi_thread for ir_ctx

* rwlock_2_spinlock, add REGISTER_TYPE_2_IRCONTEXT

* DECLARE_TYPE_UTILITY_FUNCTOR

* refine ircontext singleton

* del destructor for ParametricStorageManager

* refine code

* Add necessary logs for debugging

* refine ir_context instance

* refine type get interface

* refine code by comment

* [XPU] bind op scatter_nd_add

* [XPU] add more data type for op: clip, transpose2 & assign_value

* fix dist_attr in data_parallel in optimization

* fix grad_clip pass when pp2

* fix dist_attr

* register bfloat16 datatype for squared l2 norm

* register bfloat16 datatype for softmax with upper triangular mask

* register bfloat16 for tril triu cuda kernel

* [mv fleet] mv fleet to distributed

* [mv fleet] for ci

* [mv fleet] for ci

* [mv fleet] solve ci of version

* fix set_grad_var_shape

* recover modify

* implement of matmul using cublasLt instead of cublas

* Update matmul_kernel_impl_via_blasLt.h

---------
Co-authored-by: Nzhangbopd <1299246947@qq.com>
Co-authored-by: NBo Zhang <105368690+zhangbopd@users.noreply.github.com>
Co-authored-by: NLiu Yiqun <liuyiqun01@baidu.com>

* Enable matmul + bias fusion in fused_gat_attention.

* Add a variable to control whether using fused matmul + bias.

* change outputs and grads from fp16-fp16-comparision and fp16-fp32
comparision

* support grad comparision fp16-fp32

* the change of reference dtype only occured from np.float16 to np.float32

* fix the list type can not infer the dtype by attribute dtype by transfer
the list to array

* adjust the default atol and rtol of float16 to 1e-3

* Polish code

* fix error

* fix

* Polish code

* fix the _is_cal_ref and np.float16

* fix the combination of is_calc_ref and np.float16

* remove unuseful codes in op_test.py

* fix ci

* fix the rtol set in the dygraph checker and eager checker

---------
Co-authored-by: NZzSean <18818272991@163.com>

* rename elementwise_heaviside to heaviside

* delete __init__.py

* fix bug