* add n-d input support for trt scale converter (#31316)

* add n-d input support for trt scale converter

* add flatten for ut

* fix dims

* fix batchnorm when inpu dims < 3 (#31933)

* fix batchnorm when inpu dims < 3

* add unittest for batchnorm dims = 2

* fix unittest
Co-authored-by: NPei Yang <peiyang@baidu.com>

* fix conflict

* fix compile error

* cherry-pick #31316

* Refine cudnn softmax (#25757)

* refine cudnn softmax

* Trt elementwise plugin serialize (#31587)

* add serialize unittest

* fix element_op trt plugin serialize bug

* remove PassVersionChecker.IsCompatible

* fix unittest
Co-authored-by: NPei Yang <peiyang@baidu.com>
Co-authored-by: NGaoWei8 <53294385+GaoWei8@users.noreply.github.com>

* cherry-pick PR #27933

* fix: cuda version is in varibale CUDA_VERSION in 1.8 cuda.cmake

* close unittest failed temporarily

* cherry-pick PR #27544, fix layer_norm and softmax bug in tensorRT

* fix tensorrt output shape error

* fix unittest tensorrt_engine_op_test

* fix code style for unitest

* cherry-pick a22ea652

* fix analysis_config bug. (#29304)

* fix code format
Co-authored-by: NWilber <jiweibo@baidu.com>

* update.

* fix trt error in ssd model.

* 裁剪transformer模型trt支持；修复tensorRT不支持DeletePass的bug (#28517)

* skip_layernorm_op done

* add unittest

* slice op convertor support trt < 6

* skip_layernorm only work in ernie

* fix unittest

* fix unittest

* Fix TRT plugin registry without TRT lib (#25982)

* fix trt plugin registry without trt lib

* support trt4

* refine code style

* pick ea851796 from develop

* cherry-pick develop PR  #26273 && #27796

* fix unittest error

* fix unittest error

* remove const_cast
Co-authored-by: NPei Yang <peiyang@baidu.com>

* optimize slice TRT plugin

This patch removes unnecessary barrier for data transfer of needed offset,
so data transfer can be overlap with GPU kernel execution.

This patch also fixes incorrect name of slice plugin. That is, replaces
"layernorm" with "slice"

test=develop

* add serialize/deserialize to slice plugin

* add static shape slice trt plugin

* fix slice trt op convertor dynamic shape bug

* fix format by clang-format

* fix pylint format error

* fix problems commented by peiyang
Co-authored-by: NRyan Jeng <rjeng@nvidia.com>
Co-authored-by: NShang Zhizhou <shangzhizhou@baidu.com>
Co-authored-by: NRyan Jeng <rjeng@nvidia.com>

* [Paddle-TRT] Stack op plugin (#25605)

* add stack_op to CMakeLists

* add dim=3 support for scale op

* add trt stack op, test=develop

* remove debug message

* add stack plugin serialize

* remove slice, scale op, will add later

* enhence error message

* revise trt ernie test to conver the stack op CI testi, test=develop

* add stack op serialization

* fix test shape after adding stack op

* remove slice op, will add after implementing serialization

* roll back to min_graph=5 to avoid using slice op

* fix scale op output layer

* implement stack op createPlugin

* use workspace and move the defination to .cu

* move stack plugin creator definition to .cu, test=develop

* sync ut with develop
Co-authored-by: Nzlsh80826 <zlsh80826@gmail.com>

This commit fixs the compiling bug regarding unique_ptr of IOptimizationProfile.

IOptimizationProfile has protected dtor and is controlled by TensorRT
internally. Application shouldn't delete the pointer of IOptimizationProfile.
See TensorRT document: https://docs.nvidia.com/deeplearning/sdk/tensorrt-api/c_api/classnvinfer1_1_1_i_builder.html#a9ac47e100454151d8206ac91d543299a
test=develop
Co-authored-by: NJeng Bai-Cheng <jeng1220@users.noreply.github.com>

* add macro check for using TRT api dynamicRangeIsSet() (#25694)

* adjust minimum trt version for hard_sigmoid converter to 5130. test=develop (#24746)

Co-authored-by: NTao Luo <luotao02@baidu.com>

Co-authored-by: NWilber <jiweibo@baidu.com>

Co-authored-by: NWilber <jiweibo@baidu.com>

* solve conflict

* fix crash when trt not found in python; update unittest model path

* fix multhead matmul's instable
test=develop

* fix multihead matmul bug
test=develop

* fix converage problem
test=develop
Co-authored-by: NZhaolong Xing <nhzlx.dragon@gmail.com>

* fix conflict

* fix conflict

* fix code

cherry pick encryption api
Co-authored-by: NYanghello <915769235@qq.com>
Co-authored-by: NYanghello <yangqingyou@baidu.com>

* ignore warnings of external libraries, test=develop (#24193)

* fix repeat definitions in liengine.cc, test=develop (#25020)

* remove paddle_use_kernel and paddle_use_op. test=develop (#25189)

* fix compile for lite subgraph. test=develop (#25285)

* [CI] [Lite-Subgraph] CI add lite subgraph check. (#25346)

* supports xpu runtime, test=develop (#25554)

* fix cmake of lite, test=develop (#25680)

* change commit files, test=release/1.8
Co-authored-by: NWilber <jiweibo@baidu.com>

test=release/1.8

* fix bug of prelu when rank not equal 4, test=develop

* fix prelu inference, test=develop

* fix api, test=develop

* fix shape when mode is chennel, test=develop

* remove debug code, test=develop

* add unittest, test=develop

test=develop
Co-authored-by: Nnhzlx <nhzlx.dragon@gmail.com>

* update local data preprocess doc

* update for 1.8 QAT

* update benchmark data
Co-authored-by: NWojciech Uss <wojciech.uss@intel.com>
test=release/1.8
test=document_fix

test=release/2.0

* Chery_pick:[Eernie TRT]: add slice op and add emb eltwise layernorm fp16 support (#23723)

* refine ernie trt dynamic shape support
1. add slice op converter
2. add emb eltwise layernorm fp16 support
test=develop

* fix dynamic shape test ut
test=develop

* fix comments.
test=develop

* fix comments
test=develop

* cherry-pick [BUG]: Head number can only be > 1 on multihead op (#23974)

* support the head number == 1
test=develop

* fix slice op error.
test=develop

* cherry-pick :disable trt test, test=develop (#23984)

test=release/2.0-beta

* cherry-pick,Optimize the error messages of paddle CUDA API

* fix the error messages of paddle CUDA API

* Refactoring PADDLE_ENFORCE_CUDA_SUCCESS, and apply to curand/cudnn/cublas/NCCL

* remove build_ex_string

cherry-pick #23645

* OP clip, merge_lod_tensor, convert/elementwise error message enhancement. test=develop