cherry pick pr46536 

* bug fix for virtual pipeline parallel (#45922)

* dont wait for send op under dygraph pp (#46209)

* [interleave pp] sync recv for 1f1b (#46399)

* [dygraph pp] all sync for allgather partial (#46483)

* fix gather op convert to only support int32 index as input.
* add ut

* 1. refactor the return transformer.
2. fix some bugs in return transformer.

* support raise error while return stmt's father is For or while

* fix ci error.

* fix ci error and add some unittest

* code format

* fix ci error

Add FP16 support for uniform in dygraph mode on Nvidia GPU
Dev PR link PR46212

* set flag of clip_extra in save_inference_model to true (#46151)

* open the clip_extra flag in paddle.static.save_inference_model, test=allcase (#46456)

* Open the clip_extra flag in TracedLayer.save_inference_model (#46473)

* open the clip_extra flag in paddle.static.save_inference_model, test=allcase

* set the defalut value of clip_extra in TracedLayer from False to True, test=allcase

* update english doc of paddle.static.save_inference_model, test=document_fix (#46484)

* Fix clip_extra logic in remove_training_info (#46534)

* fix clip_extra code in remove_training_info

* revert rnn opmaker clear

* fix libpaddle soname mismatch error

* fix windows failed

* polish linux and windows make impl

* unify winddows lib name

* fix windows error

* revert copy dst change

* revert naming change

* revert windows change

* fix gpups compile failed

* Clear extra attrs of elementwise op in OpMaker (#45845)

* clear extra attrs of elementwise op in opmaker

* fix op_debug_string_test

* fix bug of grad_add

* fix sort of runtime attrs

* Clear extra attrs of scale in OpMaker (#45984)

* clear extra attr of scale in opmaker

* fix sum bug

* fix merge conflict

* fix minus

* Clear extra attributes of some Op in OpMaker (Part4) (#46060)

* clear extra attr of some ops in opmaker

* revert clear use_cudnn for pool

* fix test_operator_desc

* fix Attr interface of OperatorBase

* fix code stype

* back fl

* delete ssl cert

* .

* make warning

* .

* unittest paral degree

* solve unittest

* heter & multi cloud commm ready

* .

* .

* fix gloo compile warning

* adapt for nn fl-ps

* flps del fake-init op

* add learning_rate_0 intializer op

* bug fix

* .

* .

* fix sub sign reverse for mkldnn

* refactor code as comment

* remove useless

* unexpose audio ParameterError

* clean audio utils api

* use re replace judge by case

* simplify re

* [BugFix]Fix reduce_mean/min/sum/prod, cumsum grad_op infershape bug

* fix typo

* fix typo

uniform logger manager in FleetAPI.
hidde API under distributed/utils which users don't need.

* fix bugs of mp

* fix bugs of mp

* update

* update

* fix bug

* [Check]Enhance pooling output_size type check

* add unittest

* back fl

* delete ssl cert

* .

* make warning

* .

* unittest paral degree

* solve unittest

* heter & multi cloud commm ready

* .

* .

* fix gloo compile warning

* adapt for nn fl-ps

* flps del fake-init op

* add learning_rate_0 intializer op

* fix cast bug

* polish code comments

* polish data_device_transform.cc

* [Auto Parallel] Change the import way of Auto Parallel (#46115)

* fix strategy (#46256)

* [Auto Parallel] performance improvement for Sharding-DP hybrid parallelism (#46180)

* remove no need grad allreduce communication when sharding-dp

* remove no need grad allreduce communication when sharding-dp

* bugfix

* bugfix

* bugfix
Co-authored-by: NYulong Ao <aoyulong@baidu.com>
Co-authored-by: NJZ-LIANG <jianzhongliang10@gmail.com>

* Support matmul_v2 in Paddle-TensorRT converter.

* Fix TransDataBackend Error when call unsqueeze using MKL Tensor

* Add UT

* Refine UT

cherry-pick : #46016, #46021, #45974

* [Sparse]Sparse add support gpu (#45974)

* [Sparse]Remove unused code (#46021)

* [Sparse] Add infer meta (#46016)

* fix linspace error in amp

* fix log

* fix amp error

* support cast op backward refuse forward and fix some bugs (#46173)

* support cast op backward refuse forward

* Fix the bug of high order unit test framework

* support sign op backward refuse forward (#46002)

* add scope cache & reuse

* add gc scope for end of each train step

* del scope reuse for jit

* refine code

* test

cherry-pick from #45826
LayoutAutotune 支持 inplace 类型的OP
 根据 Add eager layout autotune #45409 修改意见调整UseAutotune
将LayoutAutotune判断放到controller中，与AMP 判断保持一致