* expand mix_precision to custom_device

* fix bug

* fix bug

* fix comment

* fix DEFINE bug

* process unit test matched test_p*

* fix ci bug

* fix codestyle

* remove all tests about pe and restore some irrelated tests

* delete test_parallel_executor_test_while_train.py

* [Dy2St]Enhance @not_to_static API

* del breakpoint()

* add gelu composite rule

* use full replace fill_constant

* change the form of calculation

* remove float16 test for composite gelu

* reformate code

* remove float16 test case

* add forwad with prim and backward without prim test

* add float16 test for composite gelu and add high dims test

* add float16 test case and high dims test

* shield float16 and cpu test case

* increase train step to 10 in test cinn prim gelu

* replace pow to multiply

* add cast setvalue op

* add set_value to op teller

* renew test and add description

* add setAxis and add complex test

* change test

* update trt workspace size for inference predictor ut

* add xpu adagrad and where_grad kernels, test=kunlun

* add xpu pool3d kernels, test=kunlun

* support size 0 dot input

* prevent div 0 in grad

* add unittest

* remove unnecessary vlog

* add unittests

* upgrade protobuf to 3.19.0 in cmake

* recover protobuf python version

* fix distribute compile

* fix

* fix framework.data_feed_pb2

* fix macos ifdef

* fix lite

* test

* update protoc from 3.19.0 t0 3.20.0

* test

* debug

* test

* test

* debug

* debug

* debug

* debug

* test

* debug

* update protocol from 3.20.0 to 4.21.12

* modify graph_brpc_client.h

* modify graph_brpc_client.h

* test

* test

* test

* fix third_party cache problem on build ci

* updata proto

* test

* test

* test

* test

* test

* test

* fix coverage failed test

* try to fix test_exe_fleet_model_run

* fix cinn bug

* fix windows compile problem

* fix python/requirements

---------
Co-authored-by: Npangyoki <pangyoki@126.com>

* init commit

* fix tensor operator*

* fix compile bug

* bug reproduce

* update commit

* polish codes

* fix compile bug

* test begin

* test begin

* compile finish

* restore origin composite_backward_api

* pass local CI

* fix merge error

* fix merge error

* change py_test from GPU->CPU, test custom op

* polish codes, modify prim unittest

* modify prim unittest

* determine phi_tensor_operants location

* polish codes

* add header file

* solve windows unresolved symbol

* fix some CI error

* add overload defination

* fix CI inference and Windows

* polish codes according to reviewers' opinion

* polish codes according to reviewers' opinion

* eleadd_trans first version

log fix

* refine code for linear format, add pass check

* linear format refine and ut fix

* fix ut

* windows ut

* windows ut 2

* move tensorMeta and alloc to configure

d_bias are nullptr. Modify the code style of full_kernel.cc. Add new data
type for concat, elementwise_add, gather, scale, scatter ops. test=kunlun

* fluid clean

* fix optimizer

* fix distributed_transpiler

* fix fluid.__init__

* remove from fluid.init

* add dim check in scatter

* add check in scatter.cu

* add unittest

* remove unnecessary log and comment

---------

Co-authored-by: RedContritio <>

* 为split增加取值范围维度的判断

* 为glu的axis进行取值判断并添加单测

* 完善glu的单测

* fix glu

* add test_std

* add test_var

* fix std/var assertequal

* fix std/var assertequal

* fix std/var assertequal

* -madd api name to reduce_api

* fix

* fix var

* fix

* fix

* fix stat

* fix unitest

* fix stat/var

* fix stat/var, unittest

* fix stat/std, unittest

* add unittest of var,std, fix stat/var,std

* fix stat/var, unittest

* fix

* fix unittest

* fix

* fix

* fix

* fix unittest

* update

* support int64 shape tensor as engine input

* add inference_predictor ut

* remove dygraph.parallel.ParallelEnv

* logger.py error: AttributeError: module 'paddle' has no attribute 'distributed'

* move the implenmentation to the root folder

* logger.py import ParallelEnv from paddle.parallel to avoid circular import

* add the comment of why import ParallelEnv from paddle.parallel in logger.py and remove the api interface in the paddle/parallel.py

* outdated Env and note removed

* decouple the logger.py and ParallelEnv

* remove another ref of parallel in init.py

* fix the processing order of passes in pass_base.py

* fix processing order

* add _PASS_PROCESS_ORDER_LIST

* delete some pass in _PASS_PROCESS_ORDER_LIST

* add assert in pass_base.py

* remove fuse_optimizer

* add _fusion_opt_list_rule

* add test_pass_base_list.py

* fix some bug

* add fused_attention

* add some passes to list

* fix ci bug

* fix ci bug

* trans_layernorm

* QuantTranpose pattern is being found by pass

* quant + transpose fuse

* code style changes

* UT written, reorder fixed

* Dequantize + transpose2 fuse  added

* pass name changed

* UT added & shift corrected

* got rid of redundancy

* review changes

* AsIntermediate corrected

* compat added

* update

* update

* format code

* update

* Update test_trt_convert_nearest_interp_v2.py

* hidden unzip

* fix

* fix