* remove distribute

* fix py3 bugs

* fix gpu-ps bugs

* fix compile bugs

* fix unittest bugs

* add FP16 for multinomial

* fix input data

* update code

* fix FP16

* fix code

* stride slice fp16 and bf16 unitest

* fix code style

* add self.dtype

* regist output type for GraphSampleNeighbors and GroupNorm

* Update return type

* fix return type

* update

* fix detail

* fix int8 support for full kernel

* fix ut.

* add unittest for conv2d/depthwise_conv2d/conv2d_transpose

* add bf16 for DWConv and ConvTranspose

* fix unitest of conv2d_transpose

* modify DWConv2d op and unittest

* fix unittest of conv2d_transpose_bf16

* modify unittest name according to review

* modify atol of DWConv2D unittest

* unbind support bool dtype

* replace np.array_equal

* add all_reduce, reduce kernel and api

* fix all_reduce reduce ut

fix reduce op maker conflict

fix merge conflicts

* fix conflicts, rename ReduceOp->ReduceBaseOp in reduce_ops

rename allreduce op, to remove

* fix code format

fix comments

* modify test_collective_reduce_api ut timeout

* fix PR-CI-Build

fix comments: format phi operator

* fix momentum dtype infer

* fix momentum datatype

* fix on cpu

* add momentum

* untracked files

* kthvalue perf

* remove unused files

* fix isnan

* fix isnan2

* fix bug

* try to fix rocm error

* Add bf16 support for elementwise_pow

* Update ut

* add fp16 and bfp16 for temporalshift

* add fp16 and bfp16 for complex

* fix bug

* fix bug

* add fp16 and bf16 for conj

* fix bug

* fix bug

* Update complex_kernel.h

fix bug

* Update temporal_shift_grad_kernel.h

fix bug

* Update temporal_shift_kernel.h

fix bug

* add nanmedian output defs

* remove the multiclass_nms3 momentum

* test_logit_op

* add cudaKernel to replace eigen impl

* bf16 unit test CI

This reverts commit 3b2cd23a.

* test_accuracy_op

* add create_test_fp/bf16_class

* cast after calculation

* change convert_uint16_to_float_ifneed

* delete TestAccuracyOpFp32 according to PR comment

* fix the rtol setting rules in bfloat16 forward

* move DataType from paddle::experimental to phi

* convert namespace

* convert namespace

* convert namespace

* clarify namespace

* convert more datatype

* Revert "convert more datatype"

This reverts commit 083b462959e6a22d4d8767707b628b95b396642e.

* convert more in auto_code_generator

* fix conflicts for XPU

* fix namespace conflicts

* fix errors

* Revert "fix errors"

This reverts commit f9d9958b54ee32141112274c8a5c3c381ab0f876.

* fix errors

* fix formatting

* fix flash_attention

* Update mp_layers.py

* add fp16 unittest

* support bf16 and add unittest

* fix according to review

* with printf

* add DropOutNdForwardKernel

* PR comment

* remove init

* delete fluid in context pool

* fix custom op bugs

* fix profiler bugs

* fix ci bugs

* fix window compile bugs

* fix windows bugs

* fix window bugs

* resgister for ftt_r2c, ftt_c2_r

* fix clang-format

* Add output defs for argsort kernel

* Update argsort_kernel.cc

* Update argsort_kernel.cu

* Update argsort_kernel.cc

* finish task

* fix some question.

* fix error

* change unittest:zeroDim.

* add multinomial output defs

* fix register on gpu

* [AMP OP&Test] Support float & bfloat16 when using cub

* fix compile error

* fix

* fix rocm compile error

* rename flash_attn_raw to flash_attn_unpadded

* fix static api

* fix static return

* add fp32 grad plus fp16 param in adamw

* add python UT

* fix test case

* in test_adamw_op py file, force the moment2 value LE 0

* add a compare option

* remove bf16 fused adam kernel case