* fix copysign compile error on Windows

* fix more files' macro

* Tensor support void* data() function

* add unittest

* add selectedRows unittest

* polish unittest

* polish unittest

* polish unittest

* polish unittest

* add xpu adagrad and where_grad kernels, test=kunlun

* add xpu pool3d kernels, test=kunlun

* support size 0 dot input

* prevent div 0 in grad

* add unittest

* remove unnecessary vlog

* add unittests

* fix py::array_t calling bug

* polish code

* upgrade protobuf to 3.19.0 in cmake

* recover protobuf python version

* fix distribute compile

* fix

* fix framework.data_feed_pb2

* fix macos ifdef

* fix lite

* test

* update protoc from 3.19.0 t0 3.20.0

* test

* debug

* test

* test

* debug

* debug

* debug

* debug

* test

* debug

* update protocol from 3.20.0 to 4.21.12

* modify graph_brpc_client.h

* modify graph_brpc_client.h

* test

* test

* test

* fix third_party cache problem on build ci

* updata proto

* test

* test

* test

* test

* test

* test

* fix coverage failed test

* try to fix test_exe_fleet_model_run

* fix cinn bug

* fix windows compile problem

* fix python/requirements

---------
Co-authored-by: Npangyoki <pangyoki@126.com>

* init commit

* fix tensor operator*

* fix compile bug

* bug reproduce

* update commit

* polish codes

* fix compile bug

* test begin

* test begin

* compile finish

* restore origin composite_backward_api

* pass local CI

* fix merge error

* fix merge error

* change py_test from GPU->CPU, test custom op

* polish codes, modify prim unittest

* modify prim unittest

* determine phi_tensor_operants location

* polish codes

* add header file

* solve windows unresolved symbol

* fix some CI error

* add overload defination

* fix CI inference and Windows

* polish codes according to reviewers' opinion

* polish codes according to reviewers' opinion

* eleadd_trans first version

log fix

* refine code for linear format, add pass check

* linear format refine and ut fix

* fix ut

* windows ut

* windows ut 2

* move tensorMeta and alloc to configure

d_bias are nullptr. Modify the code style of full_kernel.cc. Add new data
type for concat, elementwise_add, gather, scale, scatter ops. test=kunlun

* Fix inferMefer in transpose2_grad

* fix infershape

* fix unittest

* add dim check in scatter

* add check in scatter.cu

* add unittest

* remove unnecessary log and comment

---------

Co-authored-by: RedContritio <>

* fix NLP-Bert model performance loss

* fix windows compile error

* fix test_fleet_exe_dist_model_run

* test

* remove AllocatorFacade in phi

* fix include

* fix bugs

* rm gradient_accumulator in phi

* update

* update

* support int64 shape tensor as engine input

* add inference_predictor ut

Co-authored-by: Nzhangxiaoci <zhangxiaoci@baidu.com>

* Adjust mkldnn_placement_pass to check library type and data type

* Check if var has inputs

* Remove unrelated test

* Refactor

* decouple strided_memcpy

* move strided_memcpy

* move strided_memcpy to phi

* fix namespace

* update

* fix gpu compile bugs

* add multi_tenosr_adam

* update multi_tensor_base.py, test_multi_tensor_adam.py, adamw.py

* fix adam.py optimizer.py

* fix adamw.py

* fix test_multi_tensor_adam.py

* fix CI bug

* fix CI coverage

* fix ci bug

* fix betapow

* fix some bugs

* fix test_adamw_op.py

* fix CI coverage

* fix multi_tensor_adam_kernel.cc

* fix CI bug

* fix multi_tensor_adam_op.cc and test_multi_tensor_adam.py

* fix code style

* update C++ parts

* remove python parts modification temporarily

* add C++ ut

* update betapow copy code logic

* fix ci ut

* fix windows ci

* fix coverage ci

* improve coverage rate

---------
Co-authored-by: Nsneaxiy <sneaxiy@126.com>

* add fmha_flashattention oss plugin

* add fmhca

* add oss fmhca

* code reconstruct and add ut

* code style refine

* fix ut and enforce check

* refine trt version check

refine compile

fix compile

* fix cross ut

* code refine

* use runtime trt version check

* bug fix and code refine

* compile fix

* merge develop

* add GN QDQ kernel

* support GN int8 fake kernel

* add with_int8

* add GN int8 fake kernel

* add GN int8 fake kernel

* add GN int8 fake kernel

* add GN int8 fake kernel

* add GN int8 fake kernel

* add GN int8 fake kernel

* add GN int8 fake kernel

* add GN int8  UT

* add verison > 8000  in GN int8  UT

* add some check in .cu

* add stdlib.h in UT

* little change  in .cu

* remove rand_r use rand

* remove use rand

* setAxis(1)

* when int8 is on allow fall back to fp16

---------
Co-authored-by: Nwwbitejotunn <wang_bojun@outlook.com>

* fix judgement about scope validation

* fix ci bug: same address is not enough for data consistency

* remove useless check

* add TypeID

* Specification comment code

* refine code

* add AbstractType

* add TypeStorage

* fix unittest bug

* change dir

* change dir

* refine code

* fix bug

* Refine code by comment

* delete unused code

* normative naming rules

* refine code by comment

* refine doc

* refine codestyle

* trans_layernorm

* QuantTranpose pattern is being found by pass

* quant + transpose fuse

* code style changes

* UT written, reorder fixed

* Dequantize + transpose2 fuse  added

* pass name changed

* UT added & shift corrected

* got rid of redundancy

* review changes

* AsIntermediate corrected

* compat added

* add support for bf16 fused_ops

* fused_matmul only