* add auto parallel tuner options in launch

* add ut for launch in auto_parallel tuner

fix code format

* fix ci-converage

* adjust test_elementwise_sub's tolerance, max_relative_error of grad and
atol/rtol of output to 1e-3

* fix the dtype in setUp

* fix the elementwise_sub optest

* modify elementwise_sub optest

* fix and add bf16/fp16 to elementwise_sub

* fix elementwise_sub bugs

* fix bugs

* fix elementwise_sub op

* fix the data type

* fix elementwise_sub

* fix elementwise

* fix elementwise_sub

* fix bugs

* fix elementwise sub

* fix elementwise_sub

* remove scalar and vector

* Fix _param_grad_names when like 'param@GRAD@GRAD'

* [CINN] optest add cinn check test

* replace set self.check_cinn to pass check_cinn by function parameter

* fix ci bug

* add cinn atol/rtol

* fix bug in cross_entropy in static mode

* fix ci-coverage

* rem cncl in new test

* rem cncl in build sh

* rem cncl in old test

---------
Co-authored-by: NZhang Jun <ewalker@live.cn>

* [Auto Parallel] Speedup the completion process

* [Auto Parallel] Skip the property of dist_context when deepcopying

* [Auto Parallel] Remove the unnecessary print

* [Auto Parallel] Move some changes from 2.4 branch to develop

* Update engine.py

* [Auto Parallel] Fix a bug

* remove c_comm_init_hccl_op.cc and c_gen_hccl_id_op.cc

* remove gen_hccl_id_op.cc

* fix dtype cast in amp.

* add test case and update docs.

* remove set_prim.

* add bf16 support and bf16/fp16 unittest for pool2d

* add include files

* dbg

* reformat

* reformat

* modify code according to review comment

* remove duplicate code

* remove dup code

* remove useless include

* dbg

* move python/paddle/fluid/tests/unittests/xpu to test/xpu

* update CMakeLists.txt

* remove xpu in fluid/tests/unittests/

* add path to op_test_xpu

* fix incorrect path

* update test script

* fix test_adadelta_op_xpu error

* [AMP OP&Test] support bf16 for batchnorm

* codestyle

* Update batch_norm_grad_kernel.cu

* Update batch_norm_kernel.cu

* fix codestyle

* fix

* fix

* fix

* fix

* fix

* Update batch_norm_kernel.cc

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* isamp

* gpu

* cpu

* noamp

* fix instance_norm

* fix

* fix unit test

* fix unit test

* add unit test

* fix

* add big data tests

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* add test case

* fix

* fix

* fix

* fix

* fix

* remove amp test

---------
Co-authored-by: Nheyanru01 <429520051@qq.com>

* 【Hackathon 4th No.13】为 Paddle 新增 Bernoulli API

* [Change]change unittest_py scipy version

* [Change]修改BernoulliNumpy的类型参数;优化静态图测试流程

* [Change]优化类的初始化及逻辑;增加0D相关测试用例

* [Test MV] fft

* Update test_spectral_op.py

* stack fp16 & bf16 support

* unstack fp16 support

* unstack bf16 support

* append stack fp16 ut

* add unstack

* recover unstack cpu kernel

* fix some issue for unstack ut

* delete unuse var

* add check_place

* fix inference err

* add bfp16 test for isfinite

* fixed for ci

* deal with comments

* fixed test

* skip test in cpu

* deal with comments

* fixed for ci

* fixed testcase

* fixed for ci

* fixed for testcase

* Add backend for to_static API

* [CINN] Reopen some prim+cinn unittest

* revert scatter for ci

* change cinn test timeout from 120 to 150

* add softmax_with_cross_entropybf16 test

* correct defalut value in testBF16/FP16 op

* fix test checkout/grad, add skipif

* remove legacy profiler

* rm test_parallel_executor_profiler

* add_fp_bf_for_flip_gaussian_random

* forget convert uint

* fix some error

* fix some error

* unique id for mesh

* rng ctrl

* support dropout

* register op

* adopt for recompute

* update unitest

* support pp

* add PoissonNLLLoss API

* update unittests

* Fix poisson_nll_loss init and update data type support

* remove type comment

* Update doc string

* Fix doc string erro

* Fix doc string math equation format

* Add float16 and bfloat16 support

* Support two inputs of multihead attention named qk_multihead