* add auto_cast, test=develop

* add loss scaler, test=develop

* add comments, test=develop

* refine code, test=develop

* refine code, test=develop

* do not set flags automatically, test=develop

* fix custom op bug, test=develop

* add more test, test=develop

* refine enable logic, test=develop

* enable amp test with GPU, test=develop

* add unittest

* add test for found_inf

* follow comments

* follow comments

* remove global variable, use singleton

* add some notes

* update comments

* update comments

* update comments

* add use_dynamic_loss_scaling argument

* refine found_inf

* refine found_inf

* fix loaded var suffix repeat error

* use new dygraph name for loaded param

* add nll loss API, update demo code of the comment

* Add the max, min, maximum, minimum api for the API 2.0, test=develop

add lamb to fleet meta optimizer

* Add laxical GRU unit test

performance works

* Get model accuracy

* model and data name to be confirmed
test=develop

* update model name and output format
test=develop

* update according to reviews
test=develop

* add accuracy check

* accuracy check between native and analysis
test=develop

* fix a reading bug, fix gru passes sequence
test=develop

* fix passes sequence
test=develop

* Fix inference bug

* fix inference lib

* add encrypt for inference lib

* fix code;test=develop

* fix test; test=develop

* bug fix; test=develop

* add MakeCipher;test=develop

* fix bug;test=develop

* move MakeCipher to paddle space; test=develop

* fix include dir ;test=develop

* add include dir; test=develop

* move include; test=develop

* move include; test=develop

* fix for windows ci

* fix cmake; test=develop

* fix bug

bug fix

* add the support for multi-node training

update cumsum api and fix up the cumsum op

* Add oneDNN relu6 op

* Lint fixes

test=develop

* Fixed compile warning about incorrect compile options,fix paddle_build.bat

* fix paddle_build.bat to more safe

* add paddle.fleet.AsyncOptimizer
Co-authored-by: Ndongdaxiang <dongdaxiang@baidu.com>

* add pin memory control

* fix buffered reader init problem

* fix unittest error

* add unittest for coverage

* add cuda pinned place branch

* add unittest

* add skip when not gpu

1. update document template for unary elementwise layers(a.k.a. activation layer);
2. remove generate_op_noattr and use generate_activation instead; remove redundant function copies;
3. minor update for docstring to fix rst format errors.
4. fix doc for Rsqrt OP
5. add sample code for each activation separately;
6. remove the unused deprecated decorator.

* Change use_quantizer attribute name and data type

* Fix problem with setting attribute

* Add changes due to review

* Small change in function

* Restore use_quantizer attr for compatibility

* refine unsqueeze, test=develop

* update unsqueeze, test=develop

* refine unsqueeze, test=develop

* refine unsqueeze, test=develop

* update

* remove None, test=develop

* follow comments

* support bool

* update doc

* follow comments

* merge develop

* fix large scale KV 
* fix single training using async ssa graph

* add flags_call_stack_level

* update

* refine code

test=develop

* remove global variable

* refine code

* fix trt plugin registry without trt lib

* support trt4

* refine code style

* test=develop,test=document_fix, remove the out args

* fleet_util move to paddle.fleet
Co-authored-by: NWuHaobo <wuhaobo1994@gmail.com>
Co-authored-by: Ntangwei12 <tangwei12@baidu.com>

* Add oneDNN fusion_gru kernel and fix fc+gru pass
test=develop

* Formatting changes
test=develop

* Lint fixes
test=develop

* Add memory::format_tag::any to GRU weights
test=develop

* Fix build with CUDA

* Fix build with CUDA v2

* add heter ps mode

* code style
test=develop

* add with_pslib
test=develop

* unitest
test=develop

* code style
test=develop

* code style
test=develop

* code style
test=develop

* code style
test=develop

* code style
test=develop

* code style
test=develop

* code style
test=develop

* code style
test=develop

* test monitor
test=develop

* prepare trainer
test=develop

* code style
test=develop

fix invalid read of pnorm gradient function and delete the unused code

Add dgc to fleet meta optimizer, rm dgc from optimizer all

* cunn8 support
test=develop

* fix ci error
test=develop

test=develop

* Add some error messages for the op without double grads.

* fix the test_imperative_double_grad UT.

* add fleetrun command for distributed running; test=develop

* develop dynamic shape serilization

* add test param for gelu

* fix bugs

* delete redundant comments

* debug

* fix conflict. test=develop

* fix bug. test=develop

* add trt dynamic shape serialized support

* fix ernie serialized bug
test=develop

* fix codestyle
test=develop

* fix bug
test=develop

* fix bug.test=develop

* modify cmakelist test=develop

* fix bug
test=develop

* fix error message.  test=develop

* fix trt register plugin based on pr#25003

* add trt dynload

* fix deserialization bug of not finding plugin registration

* refine code style

* recover engine key in tensorrt_subgraph_pass

* for ci coverage

* add unittest for deserialization
Co-authored-by: Nhaozech <chenhaoze94@gmail.com>

Update the code for the compare ops for the broadcast function