* layernorm fw opt

* layernorm bw opt

* fix typo, test=develop

* remove const dim3 for windows CI compatibility

* merge develop
Co-authored-by: Nzlsh80826 <zlsh80826@gmail.com>

* Add ReserveSpace in dygraph batch_norm.

* Add unittest for reservespace

* Flags assert condition more general, print output if pattern not found

* removed test_flags_use_mkldnn form skip list regarding #29080 descr

* add ps table

Change-Id: I468a04bd071d21ff52654926fcf4d5f3da19e178

* remove tensor copy in the update_loss_scaling op

* not use thrust.

* fix some cuda memory access error.

* support deepcopy for Layer/Tensor/Paramerbase

* fix some code

As the title 

As the title

* vec softmax fw

* vec softmax bw

* add a message argument for compiler compatibility

fix increamental coverage script bug, WITH_INCREMENTAL_COVERAGE to DWITH_INCREMENTAL_COVERAGE, test=develop (#29509)

* remove addcmul

* remove unittest and other related code of addcmul

* fix bug

* fix merge conflict

fix abs double grad unittest & define the data range for the abs double grad

* add tensor_indices in AssignGroupBySize

* add rebuild group in reducer

Add Sleep Time for CUDA Retry, which is similar to our GPU retry logic. This is a try to avoid init GPU allocation random failure in unit test.

* test ccache hit statistics, test=develop

* test ccache hit statistics, test=develop

* add cache hit statistics, test=develop

* fix no percent symbol erro on windows, test=develop

* remove switch, test=develop

* fix the bug of reducer in embedding

* add comment

This reverts commit befd6d53.

* added external reorder to profiler

* added external and internal reorders to profiler

* added internal and external reorder to profiler

* added formatting to int/ext reorder commit

* removed unnecessary comment

* change hard_swish from plugin to layer

* add ut when threshold != scale

* update, test=develop

Co-authored-by: Nroot <root@bjhw-sys-rpm0223.bjhw.baidu.com>

* fix the ut of matmulv2 for broadcast

* update reduce_sum op on xpu

* update reduce_sum op on xpu

* support running on xpu