* remove with_data_parallel in custom op test

* finish TestCustomOpReluModelStaticMultiDevice

* fix typo

* add checks for relu output

* fix ci

* fix ci

* fix compile checks

* fix coverage ci

* [AMP OP&Test] Modify the logic of comparing grad in bfloat16

* AMP arange & Test

* fix arange bfloat16 dtype

* update for review

* update for review2

* fix tile

* update

* fix ci

* r

* f

* fix windows ci

* update bfloat data

* fix bloat16 input

* add print

* Update test_where_op.py

* update kernel

* del repeat

* update review

* where op test

* update bfloat16

* fix

* fix windows ci

* update bfloat16 data

* fix bloat16 x

* reset

* fix randint

* add print

* add delta

* cancel print

* code style

* update revirew

* move fluid.utils to paddle.utils.layers_utils

* fix error

* delete original fluid layers utils

* remove import and old utils

* remove more old utils import

* change import path of fill_constant in the layers_utils.py

* fix mistake

* fix error

* expose in __init__.py

* for comment

* when change the ref of func is_sequence, it should change to the root of is_sequence instead

* for codecheck

* add prim erf grad

* add yaml config for prim erf grad

* add math.h

* add cmath

* add math  defines

* use define math

* use define math

* define M_2_SQRTPI

* M_2_SQRTPI math

* try math.h

* fix typro

* remove pow in erf grad

* use new optest

* add fp16 fp32 test

* remove fp16 test

* support run haokanctr model in heterps-models

* polish setup.py

* polish JVM_LIB in evn_dict

* add softplus double grad

* use constant method

* support elementwise_pow bfloat16

* add only_check_prim parameters in check_grad

* modify unit test

* fix floor test

* fix sigmoid bfloat16 test

* norm some ops prim test

* add unit16 for sqrt

* fix accuracy and activation

* add adadelta

* support pad2d

* support pad

* modify exponential and linear_interp_v2

* modify meshgrid test

* add group_norm

* support some ops

* modify activation&group norm

* modify act

* reset group_norm

* modify acti

* modify pow test

* modify acti

* lint

* modify mkldnn test

* fix acti

* modify adadelta

* lint

* fix act mkldnn

* reset acti

* [clear ps] del some all list

* [clear ps] for ci

* [clear ps] rm fluid emb

* split_list

* support reshape test on prim and cinn

* fix mkldnn test

* polish test case

* [Dy2St]Enhance @to_static auto-skip paddle inner API

* fix comment

* fix class method

* tile_op

* fix bfloat16 x

* update review

* del out

* add abs composite backward op

* add missing changes during merge

* modify according to new rules

* local UT OK

* fix typo

* codestyle

* register composite operator

* add fp16 test for abs

* replace experimenta::tensor

* Fix bug of reduce_sum op. When input.numel() > INT32_MAX, its result
is wrong.

* Remove climits.

* Fix bug of hybrid parallel strategy with recompute using bf16.

* Fix bug of recompute_hybrid ctx.amp_dtype

* Fix bug of amp_dtype.

* Fix bug of auto_cast.

* * add comm context for device context

* add broadcast phi operator kernel and api

* add broadcast support dtype, update ut

* fix broadcast bfloat16 type

* fix ut

* update test_collective_broadcast_api timeout to 300

* [cinn] add elementwise_pow backward

* [cinn] update unnitest

* [cinn] update by comments

* [cinn] for ci

* [cinn] for ci

* [cinn] for ci

* [cinn] for ci

* [cinn] for ci

* remove with_data_parallel in collective_optimizer

* add comm op

* fix collective optimizer

* remove check_err_log=True

* mean fp16

* fp16

* [AMP OP&Test] mean append bf/fp16

* means append more bf16 uts

* format class name

* fix ci

* fix for windows

* fix issue

* fix redundancy

* fix redund

* fix elewise_max ut bf16 numeric delta

* remove use func

* move entry_attr

* remove evaluator

* fix evaluator

* modify optimizer tests

* lint

* tile_op

* fix bfloat16 x

* feat: add relu composite rule

* feat: add relu composite rule, maximum op

* feat: add relu composite rule, maximum op

* feat: add relu composite rule, polish comments

* feat: add relu composite rule, polish comments

* feat: add relu composite rule, add python api of relu

* feat: add relu composite rule, commit hook

* fix: maximum type error & ban cinn test

* fix: maximum input sequence bugs

* resolve conflicts

* fix: code style bugs

* add: relu fp16 test

* modify the 'import' position

* rollback

* move 'from paddle.fluid.io import save_inference_model' to head of api.py

* support transpose test on prim and cinn

* fix transpose logic and polish eager_op_test

* move io

* fix ci bugs

* fix ci bugs

* fix py3 bugs

* fix example code

* fix example code

* fix text

* fix text

* deal with ci bugs

* perfect code according comment

* delete import batch