* trans fused attention to phi

* add optional parm

* trans fused_attention_grad to phi

* add fused attention grad register info

* fix include

* test=kunlun

* add fused attention to static build list

* add remove

* update remove

* mv sequence tests

* mv sequence tests

* fix test error

* fix

* mv ps distributed dir

* fix

* add del auto_parallel

* add auto_parallel

* fix ps

* fix bug

* fix test bug

* fix test bug

* merge develop fix error

* merge develop fix error

* merge develop fix error

* [Test Mv] remove rnn

* Update test_rnn_cell_api.py

* add register info for eigh and eig_gard

* add sync_batch_norm_op.cu register info

* add lamb output register info

* add unique register info

* change type name

* change type name

* add output register info for check_finite_and_unscale

* update cmake and config file

* add register info for adagrad

* fix build error

* add sync to run_unittests.sh

* add register info for unique_consecutive

* fix build error

* add eigh to STATIC_BUILD_TESTS

* update eig_kernel.cc

* update eig_kernel.cc

* fix infer mate error

* fix unique register error

* fix lamb register info error

* fix lamb register info

* update lamb register info

* fix lamb

* remove one Output Register

* update static build file

* add eigh op to disable_wingpu_test

* update run_unittests

* [CINN] optest add cinn check test

* replace set self.check_cinn to pass check_cinn by function parameter

* fix ci bug

* add cinn atol/rtol

* move python/paddle/fluid/tests/unittests/xpu to test/xpu

* update CMakeLists.txt

* remove xpu in fluid/tests/unittests/

* add path to op_test_xpu

* fix incorrect path

* update test script

* fix test_adadelta_op_xpu error

* [Test MV] fft

* Update test_spectral_op.py

* [CINN] Reopen some prim+cinn unittest

* revert scatter for ci

* change cinn test timeout from 120 to 150

* remove legacy profiler

* rm test_parallel_executor_profiler

* mv WITH_ASCEND_CL

* mv WITH_ASCEND

* rollback

* remove WITH_ASCEND

* remove WITH_ASCEND

* mv WITH_ASCEND_CL

* mv WITH_ASCEND

* rollback

* remove run_program

* remove FLAGS_USE_STANDALONE_EXECUTOR

* [Test MV] standalone_executor

* update

as

* update

as

* update codestyle

* feat: add relu composite rule

* feat: add relu composite rule, maximum op

* feat: add relu composite rule, maximum op

* feat: add relu composite rule, polish comments

* feat: add relu composite rule, polish comments

* feat: add relu composite rule, add python api of relu

* feat: add relu composite rule, commit hook

* fix: maximum type error & ban cinn test

* fix: maximum input sequence bugs

* resolve conflicts

* fix: code style bugs

* add: relu fp16 test

* feat: add rsqrt composite rule

* feat: add rsqrt composite rule

* resolve conflicts of composite rule

* fix: delete check eager

* feat: add roll grad composite rule

* fix minus shift

* fix test roll op

* Improve new executor static build

* Skip GC for static build

* Skip infershape for static build

* Handle read_op

* Add fused_attention to OpsWithFluidKernelNeedMoveToPhi

* Fix argsort typos

* Add sequence_pool to OpsWithFluidKernelNeedMoveToPhi

* Fix skip share lod errors

* Fix errors for adam

* Fix errors for eigvals, memcpy and fake_quantize

* Add static_build.cc

* Add black list

* Fix CI errors

* Fix CI errors

* Fix CI errors

* Fix TensorArray

* Fix TensorArray

* Add update_loss_scaling to OpsNeedSetOutputDtypeWhenRegisterPhiKernel

* Fix copy

* Fix errors

* Fix momentum

* Skip mkldnn

* Fix CI errors

* Fix c_sync_calc_stream_op

* Fix CINN

* Fix while op

* All CI pass, disable FLAGS to merge code, enable it after more tests in future

* Add UTs

* Fix typos

* Fix typos

* Add mkldnn UT

* Remove mkldnn test

* Fix typos

* Fix dist test

* Fix typos

* Fix CI errors

* Fix CI errors

* Add UTs

* Fix typos

* Fix typos

* Add sparse tests

* ToComplexType -> ToComplex

* Add test_matmul_op_static_build to disable_win_inference_test

* move mkldnn unittests to test dir

* add py_test_modules funcs

* move py_test_modules to test/mkldnn

* fix import error

* Revert "move py_test_modules to test/mkldnn"

This reverts commit 0277e0c29c22e5c29b7a3b843103f3d0b711c442.

* fix import error by conflicts

* fix errors

* fix mistake on solving conflicts

* support layer_norm prim and cinn test

* enable cinn test

* fix merge conflict

* polish input for check_output_with_place

* fix merge conflict

* add more test case

* fix merge conflict

* polish test case

* polish op_test

* change ln_g rules

* modify scale is none case

* modify scale is none case

* add public_python_api for check prim

* modify setoutputgrad and fp64bug

* add todo & delete log

* recover

* fix some errors

* recover

* recover

* recover

* recover

* fix merge conflicts

---------
Co-authored-by: Nwangruting <wangruting@baidu.com>

* update

* update

* add scatter composite rule.

* add public_python_api

* add python unit16 support.

* fix code style.

* add cinn to makelist

* cinn unsupport uint16, forbidden cinn when dtype==uint16.

* [Test Mv] ipu_test

* [Test Mv] cmake add py_test_modules

* [Move Test] rm py_test_modules

* rm asp

* add group_norm composite rule

* add test for scale_grad and bias_grad

* resolve conflicts

* remove amp in composite_rule.py

* add float16 test

* deal with NHWC format

* keep the composite rule in float16 identical as original kernel

* resolve conflicts

* [Test Mv] asp_test

* [Test Mv] cmake add py_test_modules

* [Move Test] rm py_test_modules

* [Test Mv] autograd_test

* [Move Test] rm py_test_modules

* update

* update

* add tanh and cast prim test

* fix tanh test

* fix 0-d test

* add sqrt fp16 prim test

* add public_python_api in prim test

* fix test_squeeze2_op

* add tanh prim test

* add dropout prim test

* [Dy2St]Fix clone for test state problem

* clean code

* modify test_cumsum_op

* modify test_cumsum_op

* fix dropout test

* add dropout in cmake

* fix dropout test

---------
Co-authored-by: NAurelius84 <zhangliujie@baidu.com>

* first version, notest

* return final rst, notest

* use infinity() instead of max

* ut structure

* start up of ut

* generate lse

* update

* add depense

* reconstruct cmake

* move file

* add memory efficient attention and fix blasimpl

* update

* update cmake

* add namespace

* update cmake

* use .cu

* update for pad3d

* bug fix

* bug fix

* update

* bug fix

* update enforce

* add test case

* merge the lse pad

* fix kernel_fn of backward

* fix PADDLE_ENFORCE_EQ and phi_api

* fix PADDLE_ENFORCE

* fix PADDLE_ENFORCE

* rerun coverage

* fix memory efficient attention test

* rerun ci

* add cuda version condition

* add cuda version condition

* delete WIP test

* replace PADDLE_ENFORCE

* edit the namespace of datatype in multiple.cc

* rerun

* rerun

---------
Co-authored-by: Nliuyuang <liuyuang@baidu.com>

* add meshgrid composite rule

* add meshgrid composite rule

* update

* add into CMakeLists

* fix

* update

* update

* optimize code

* fix meshgrid op

* update test

* Add fused_feed_forward pass for semi-automatic static graph training.

* Add fused_feedforward property in parallel_executor.cc

* Polish code.

* Polish fused feed_forward pass code. Support use_dropout1 and
use_dropout2 option.

* Support model parallel in fused_feedforward pass.

* add composite rule for squeeze

* fix pre commit

* fix pre commit

* simplify rules

* arrange code

* fix int axis

* simplify squeeze axis rules

* bugfix

* fix pre commit

* remove fluid.layers.collective and related unittests.

* fix layers.init import

* fix all_gather method

* remove unused args in all_gather method