* remove -DSUPPORTS_CUDA_FP16 in cuda.cmake

* comile with cuda9

* add some unittest

* notest;test=coverage

* add unittest for trt plugin swish && split

* update ernie unittest

* fix some error message

* remove repeated judgement of CUDA version in mbEltwiseLayerNormOpConverter

* fix comile errror when CUDA_ARCH_NAME < Pascal"

* fix comile error

* update unittest timeout

* compile with cuda9

* update error msg

* fix code style

* add some comments

* add define IF_CUDA_ARCH_SUPPORT_FP16

* rename IF_CUDA_ARCH_SUPPORT_FP16 to CUDA_ARCH_FP16_SUPPORTED

Fix ops doc for some ops 

* split train_mode and has_grad

* fix format

* fix ci problems

* fix sample code

add API serialize_program, serialize_persistables, save_to_file, deserialize_program, deserialize_persistables, load_from_file. (#29034)

* Add bf16 pool2d and unify bf16 unit tests

* Add change default ops test

* Fix cpu_bfloat16_pass

* Add output_format

* Fix incorrect SetOutput

* Change fromating

* fix win ci failure, test=develop

* add ci test, test=develop

* default not show cpp statck & add hint

* fix failed unittest

* fix failed unittests

add uint8 for reshape operator

* fix tensor detach to zero copy

* fix tensor detach to zero copy

Simple code clean up

remove eigen threadpool for the speed up

* Add multi_gru_fuse_pass and tests

* fix date

* cleaned up headers

* open unittests on windows

* open GPU unittest on windows

* upgrade comment string to raw string

* fix string in

* fix string with ' '

* revert update on comments

* upgrade only necessary

* fix sample code checker

* fix comments with '''

* Add multi_gru_seq_fuse_pass and tests

* fix date

* removed unused functions

* test=develop, optimize global_step

* update, test=develop

* ps gpu transpile

* ps gpu

* remove op

* gps trainer

* local ps

* add macro

* HeterBox

* def cuda

* tab

* code style

* style

Co-authored-by: Thunderbrook <a754913769#163.com>

* refactor momentum op to combine weight_decay (scale op and sum op)

* add musl docker build script

* rm space test=document_fix

* fix some docs and types errors test=document_fix

* move install of python requirement to docker build

* add copyright to docker file.

* add extr opts

* format docs

* change avg pooling and global pooling to trt layer

* add support for static shape global pooling

* modify trt errmsg