d_bias are nullptr. Modify the code style of full_kernel.cc. Add new data
type for concat, elementwise_add, gather, scale, scatter ops. test=kunlun

* Fix inferMefer in transpose2_grad

* fix infershape

* fix unittest

* fluid clean

* fix optimizer

* fix distributed_transpiler

* fix fluid.__init__

* remove from fluid.init

* add dim check in scatter

* add check in scatter.cu

* add unittest

* remove unnecessary log and comment

---------

Co-authored-by: RedContritio <>

* fix NLP-Bert model performance loss

* fix windows compile error

* fix test_fleet_exe_dist_model_run

* test

* 为split增加取值范围维度的判断

* 为glu的axis进行取值判断并添加单测

* 完善glu的单测

* fix glu

* add test_std

* add test_var

* fix std/var assertequal

* fix std/var assertequal

* fix std/var assertequal

* -madd api name to reduce_api

* fix

* fix var

* fix

* fix

* fix stat

* fix unitest

* fix stat/var

* fix stat/var, unittest

* fix stat/std, unittest

* add unittest of var,std, fix stat/var,std

* fix stat/var, unittest

* fix

* fix unittest

* fix

* fix

* fix

* fix unittest

* remove AllocatorFacade in phi

* fix include

* fix bugs

* rm gradient_accumulator in phi

* update

* update

* support int64 shape tensor as engine input

* add inference_predictor ut

Co-authored-by: Nzhangxiaoci <zhangxiaoci@baidu.com>

* Adjust mkldnn_placement_pass to check library type and data type

* Check if var has inputs

* Remove unrelated test

* Refactor

* remove dygraph.parallel.ParallelEnv

* logger.py error: AttributeError: module 'paddle' has no attribute 'distributed'

* move the implenmentation to the root folder

* logger.py import ParallelEnv from paddle.parallel to avoid circular import

* add the comment of why import ParallelEnv from paddle.parallel in logger.py and remove the api interface in the paddle/parallel.py

* outdated Env and note removed

* decouple the logger.py and ParallelEnv

* remove another ref of parallel in init.py

* decouple strided_memcpy

* move strided_memcpy

* move strided_memcpy to phi

* fix namespace

* update

* fix gpu compile bugs

* add multi_tenosr_adam

* update multi_tensor_base.py, test_multi_tensor_adam.py, adamw.py

* fix adam.py optimizer.py

* fix adamw.py

* fix test_multi_tensor_adam.py

* fix CI bug

* fix CI coverage

* fix ci bug

* fix betapow

* fix some bugs

* fix test_adamw_op.py

* fix CI coverage

* fix multi_tensor_adam_kernel.cc

* fix CI bug

* fix multi_tensor_adam_op.cc and test_multi_tensor_adam.py

* fix code style

* update C++ parts

* remove python parts modification temporarily

* add C++ ut

* update betapow copy code logic

* fix ci ut

* fix windows ci

* fix coverage ci

* improve coverage rate

---------
Co-authored-by: Nsneaxiy <sneaxiy@126.com>

* fix typo

* add sox_io in audio test

* fix

* fix

* fix the processing order of passes in pass_base.py

* fix processing order

* add _PASS_PROCESS_ORDER_LIST

* delete some pass in _PASS_PROCESS_ORDER_LIST

* add assert in pass_base.py

* remove fuse_optimizer

* add _fusion_opt_list_rule

* add test_pass_base_list.py

* fix some bug

* add fused_attention

* add some passes to list

* fix ci bug

* fix ci bug

* add fmha_flashattention oss plugin

* add fmhca

* add oss fmhca

* code reconstruct and add ut

* code style refine

* fix ut and enforce check

* refine trt version check

refine compile

fix compile

* fix cross ut

* code refine

* use runtime trt version check

* bug fix and code refine

* compile fix

* merge develop

* add GN QDQ kernel

* support GN int8 fake kernel

* add with_int8

* add GN int8 fake kernel

* add GN int8 fake kernel

* add GN int8 fake kernel

* add GN int8 fake kernel

* add GN int8 fake kernel

* add GN int8 fake kernel

* add GN int8 fake kernel

* add GN int8  UT

* add verison > 8000  in GN int8  UT

* add some check in .cu

* add stdlib.h in UT

* little change  in .cu

* remove rand_r use rand

* remove use rand

* setAxis(1)

* when int8 is on allow fall back to fp16

---------
Co-authored-by: Nwwbitejotunn <wang_bojun@outlook.com>

* fix judgement about scope validation

* fix ci bug: same address is not enough for data consistency

* remove useless check

* add TypeID

* Specification comment code

* refine code

* add AbstractType

* add TypeStorage

* fix unittest bug

* change dir

* change dir

* refine code

* fix bug

* Refine code by comment

* delete unused code

* normative naming rules

* refine code by comment

* refine doc

* refine codestyle