* save/load in ps runtime(the_one_ps) (#36097)

* add trainer desc config to distributed strategy

* code style modified

* data_feed set lod

* fix bug

* code style

* fix bug

* save load

* save load

* save unittest

* add unittest of the_one_ps

* unittest

* add todo in communicator sendsparse

* fix bug in save_inference_model (#37362)

* enhance scatter err msg check

* fix ci error

[Dy2stat]Allow users to switch eval/train mode when using @to_static to decorate a function (#37383) (#37432)

本PR之前使用@to_static装饰一个单独的function时，对于生成的Program无法切换train/eval模式，只能运行在train模式下。这也就导致动转静后用户多次调用function显存会一直增长。
本PR之后，使用@to_static装饰一个单独的function时，可以通过function.train()或者function.eval()的方式来切换train/eval模式。

* fix a quantization bug
Co-authored-by: NXGZhang <46363693+XGZhang11@users.noreply.github.com>

* Add paddle.incubate.graph_send_recv API

* fix bug in CudaAtomicMin and CudaAtomicMax

* add empty line

fix bug to support dropout eval grad computing. cherry-pick #37305.

该PR使得动转静模块能够正确转换如下的for i in [1, 2, 3]语句。

set net.forward to original forward function in flops when net is a dy2stat model.

* fix cusparse compile bug in CUDA11.2, test=develop

* modify sparse_attention docs, test=document_fix (#36554)

* modify sparse_attention docs, test=develop

* add warning

* add warning ,test=document_fix

* fix_qkv_plugin: half_scale

* [Paddle-Inference] fix_qkv_plugin: fix half scale

修复了fused_transformer_encoder_layer fine-tune过程发现的一些问题：

    fused_attention_op添加attn_mask=None的支持：PR
    pre_layer_norm处理问题：PR
    参数处理，计算错误的问题：PR
    add_bias计算错误问题：PR
    添加pure fp16的支持：PR

修复了一维Tensor在使用省略号(...)索引时维度检测异常的问题。

Co-authored-by: NPei Yang <peiyang@baidu.com>

* add mlperf optimization PRs

* update

* fix rnn grad bug when num_layers is set 2 and dropout_prob is set 0

* add more test for rnn

    Renamed the variable and function
    Removed the original template function
    Removed the tests_properties in CMakeLists.txt

att,Fix issue:36902

* fix cusparse compile bug in CUDA11.2, test=develop

* fix bug

Cherry-pick #36525

2. add complex data type support for paddle.shape at graph assembly.

Cherry-pick PR #36511

* fix device docs;test=document_fix

* update __init__.py

* add paddle.version.cuda and paddle.version.cudnn API

* fix little bug

* fix bug

* add doc string

* fix mkdir error

* fix windows path

* fix new paddle/version path

* fix unittest

* fix format

* [cherry-pick 2.2]support quantization of bert

support quantization for maumul_v2

* Update quantization_pass.py

* Fix fused_attention english doc test=document_fix

* change api for support trt8

* Revert "Align CTC grad scale same with ESPNet (#34729)"

This reverts commit 10f9644c.

* ctc grad compute on gpu

* Fix bug when pre_layer_norm is false.