* add reducer

* refine envent for memorycopy

* add concat&split for allreduce

* apply concat & split for fuse tensor

* fix nccl dep

* fix the untest, compile problem and ddp initialize problem

* fix untest for mac & add some comments & solve the repeated param in sublayers

* fix untest for windows & fix document

update expand as op to use the shape of the target tensor instead of the target tensor itself. (#29020)

* update, test=develop

* update conda_build script with removing opencv

* modified filepath

* modified some content

* Delete Commented-Out Code

* delete prettytable in conda_build
Co-authored-by: NXieYunshen <1084314248@qq.com>

Add eigen gru and fix the dropout bug in the rnn 

* remove -DSUPPORTS_CUDA_FP16 in cuda.cmake

* comile with cuda9

* add some unittest

* notest;test=coverage

* add unittest for trt plugin swish && split

* update ernie unittest

* fix some error message

* remove repeated judgement of CUDA version in mbEltwiseLayerNormOpConverter

* fix comile errror when CUDA_ARCH_NAME < Pascal"

* fix comile error

* update unittest timeout

* compile with cuda9

* update error msg

* fix code style

* add some comments

* add define IF_CUDA_ARCH_SUPPORT_FP16

* rename IF_CUDA_ARCH_SUPPORT_FP16 to CUDA_ARCH_FP16_SUPPORTED

Fix ops doc for some ops 

* split train_mode and has_grad

* fix format

* fix ci problems

* fix sample code

add API serialize_program, serialize_persistables, save_to_file, deserialize_program, deserialize_persistables, load_from_file. (#29034)

* Add bf16 pool2d and unify bf16 unit tests

* Add change default ops test

* Fix cpu_bfloat16_pass

* Add output_format

* Fix incorrect SetOutput

* Change fromating

* fix win ci failure, test=develop

* add ci test, test=develop

* default not show cpp statck & add hint

* fix failed unittest

* fix failed unittests

add uint8 for reshape operator

* fix tensor detach to zero copy

* fix tensor detach to zero copy

Simple code clean up

remove eigen threadpool for the speed up

* Add multi_gru_fuse_pass and tests

* fix date

* cleaned up headers

* open unittests on windows

* open GPU unittest on windows

* upgrade comment string to raw string

* fix string in

* fix string with ' '

* revert update on comments

* upgrade only necessary

* fix sample code checker

* fix comments with '''

* Add multi_gru_seq_fuse_pass and tests

* fix date

* removed unused functions

* test=develop, optimize global_step

* update, test=develop

* ps gpu transpile

* ps gpu

* remove op

* gps trainer

* local ps

* add macro

* HeterBox

* def cuda

* tab

* code style

* style

Co-authored-by: Thunderbrook <a754913769#163.com>

* refactor momentum op to combine weight_decay (scale op and sum op)