* [dy2static]implement tensor.cuda() in static graph

* [dy2static]implement tensor.cuda() in static graph - change the patch place

* [dy2static]implement tensor.cuda() in static graph - fix code-block in comment

* [dy2static]implement tensor.cuda() in static graph - add ut for warning branch

* remove the optimizer base and learning rate base

* fix bug

* fix bug

* perfect code

* delete __all__

[xdoctest][task 105]reformat example code with google style in python/paddle/optimizer/adamax.py (#56167)

* [xdoctest][task 105]reformat example code with google style in python/paddle/optimizer/adamax.py

* fix word typo test=docs_preview

* fix comment test=docs_preview

* [prim][newir] add basic framework for primitive

* support desctensor in new ir

* add vjp interface

* support vjp in new ir

* support vjp in new ir

* polish vjp interface

* fix stop_gradients set

* fix vjp dispatch

* add comment

* add vjp test for new ir

* add test for tanh vjp

* [prim][newir] add basic framework for primitive

* support desctensor in new ir

* support vjp in new ir

* support vjp in new ir

* polish vjp interface

* fix stop_gradients set

* fix vjp dispatch

* add comment

* add vjp test for new ir

* add test for tanh vjp

* add eager and static backend for warp lower level api

* support call_vjp pybind

* polish code and add test for vjp

* remove useless code

* polish code

* remove useless code

* support mean vjp

* add test for mean vjp and support has_vjp function

* fix call_vjp

* polish code

* add primitive ops set for backend

* add vjp test for tanh_

* fix inference CI

* fix inference ci

* modify fluid cmake

* remove useless deps

* add cmake

* fix comment

* fix test

* polish code

* modify backward stop_gradients

* modify static_backend.cc

* remove useless code

---------
Co-authored-by: Ncxxly <chenxx_id@163.com>
Co-authored-by: Nzhangbo9674 <zhangbo54@baidu.com>

repacle fluid.io.load_inference_model, fluid.io.save_inference_model in fluid with 2.0 version  (#55345)

* repacle fluid.io.load_inference_model

* replace fluid.io.save_inference_model

* fix some bug

* fix some bugs of load & save model

* fix some bug

* fix test_inference_model_io bug

* fix word2vec_inference_model bug

* fix some bug

* fix valueError bug

* fix some bug

* fix a warning error

* for debug

* for debug

* fix io error

* fix test_wordvec_book error

* remove debug print

* fix load_var bug

* for debug cinn test

* revert cinn & fix inference_pass_test in windows

* fix some bugs

* revert cinn & fix inference_pass_test in windows

* for debug vars

* for debug

* fix quant_dequant_test

* fix some path errors

* remove fluid save/load

* fix incubate-fleet save

* move some from fluid.io to static.io

* move fluid apis

* fix type error

* remove static exponential_decay

* fix some import error

* remove nn.py

* fix some error

* fix type error

* fix get var in prim

* fix stack test case

* add variable_length_memory_efficient_attention
* update variable_length_memory_efficient_attention unittest
* update variable_length_mem_eff_attn's docs and unittest
* update variable_length_mem_eff_attn's docs
* Update test_variable_length_memory_efficient_attention.py
* Update variable_length_memory_efficient_attention.cu
* fix codestyle
* fix variable_length_fmha's docs and unittest
* fix variable_length_fmha's docs

* [prim][newir] add basic framework for primitive

* support desctensor in new ir

* add vjp interface

* support vjp in new ir

* support vjp in new ir

* polish vjp interface

* fix stop_gradients set

* fix vjp dispatch

* add comment

* add vjp test for new ir

* add test for tanh vjp

* [prim][newir] add basic framework for primitive

* support desctensor in new ir

* support vjp in new ir

* support vjp in new ir

* polish vjp interface

* fix stop_gradients set

* fix vjp dispatch

* add comment

* add vjp test for new ir

* add test for tanh vjp

* add eager and static backend for warp lower level api

* support call_vjp pybind

* polish code and add test for vjp

* remove useless code

* polish code

* remove useless code

* support mean vjp

* backward origin code

* add test for mean vjp and support has_vjp function

* fix call_vjp

* polish code

* add attrs and dtype interface

* add primitive ops set for backend

* fix compile bugs

* fix some bugs

* fix windows bugs

* add vjp test for tanh_

* fix inference CI

* fix inference ci

* modify fluid cmake

* origin test of tanh and mean passed

* fix conflict

* modify stop_gradient

* modify block.ops

* modify test

* fix conflict

* reply review comments

* reply review comments

* pulish code

---------
Co-authored-by: Ncxxly <chenxx_id@163.com>
Co-authored-by: NCharles-hit <wanghao107@baidu.com>
Co-authored-by: Nzhangbo9674 <zhangbo54@baidu.com>
Co-authored-by: NYuanRisheng <yuanrisheng@baidu.com>

* refine program translator

* fix warning: not override

* fix bug

* merge new modifications

* modify by reviews

* resolve conflicts

* resolve conflicts

* fix

* fix

* fix conflicts

* pseudocode of backward

* modify test

* modify register op

* clear other code

* modify ci build bug

* reply review comments

* reply review comments

---------
Co-authored-by: Nkangguangli <kangguangli@hotmail.com>

* input.py

* Update python/paddle/nn/functional/input.py

* Update input.py

remove the AdamOptimizer、SGDOptimizer、MomentumOptimizer、ModelAverage、LookaheadOptimizer、FtrlOptimizer、DecayedAdagradOptimizer、DpsgdOptimizer in fluid and relocate the ExponentialMovingAverage、PipelineOptimizer、GradientMergeOptimizer and change optimizer base for LarsMomentumOptimizer and RecomputeOptimizer (#55970)

* change the optimizer base for SGDOptimizer

* change the optimizer base for SGDOptimizer

* replace the SGDOptimizer with SGD

* fix bug of sgd

* change the optimizer base for MomentumOptimizer

* fix the remaining tests

* remove the Momentum in fluid/optimizer.py

* fix bug

* fix bug

* fix bug

* fix bug

* Update test_resnet_cinn.py

* Update test_resnet_prim_cinn.py

* fix bug

* fix bug

* fix bug

* remove the ModelAverage in fluid

* remove the LookaheadOptimizer in fluid

* fix bug

* remove AdamOptimizer in fluid

* Update test_image_classification_fp16.py

* fix bug

* relocate the ExponentialMovingAverage in fluid

* restore the static api

* remove the FtrlOptimizer in fluid

* remove the DecayedAdagradOptimizer in fluid

* remove the DpsgdOptimizer in fluid

* fix bug

* fix codestyle

* fix bug

* fix bug

* relocate the PipelineOptimizer

* relocate the GradientMergeOptimizer

* fix bug

* fix bug

* fix bug

* fix doc

* Update __init__.py

* Update test_fleet_qat_meta_optimizer.py

* change optimizer base for LarsMomentumOptimizer

* fix bug

* fix conflict

* fix code-style

* fix sample codes

* fix bug

* fix bug

* fix cinn bug

* fix bug

* fix bug

* Update qat_optimizer.py

* Update __init__.py

* fix bug

* change optimizer base for RecomputeOptimizer

* fix bug

* fix bug

* Update test_imperative_optimizer_v2.py

Flatten function docstring case 1 displayed incorrect output

* Fatal bug fix for hapi.eval

When multiple metrics are specified in hapi for eval, the output samples of the model are incorrectly truncated, resulting in inaccurate metric calculations

* fix codestyle

* [enhancement] Implement tensor.cpu() in static graph

* [enhancement] Implement tensor.cpu() in static graph

* [dy2static] add unitest cases for tensor.cpu()

* [dy2static] add unitest cases for tensor.cpu() - run only on paddle compiled with cuda mode

* [dy2static]Refine the format of the api-doc for Variable.cpu().

* Improve GC for pipeline parallel

* Delete print

* open

* update

* Support Prim Forward in New IR

* Fix test case

* polish code

* fix code

* polish format

* format code

* add mask

* add backword

* add enforce info

* update scale

* integrate code

* update enforce

* add enforce eq

* add error type

* update enforce

* add test_flash_attention

* Polish codes and fix compiling errors.

* Set num_splits to 0 for flash-attn with tensor mask.

* Fix the compiling error for non flash-attn case.

---------
Co-authored-by: NLiu Yiqun <liuyiqun01@baidu.com>

* make tcp store a global instance

* fix windows compile error

* add attrs and dtype interface

* fix compile bugs

* fix some bugs

* fix windows bugs

* [xdoctest] reformat example code with google style in `paddle/tensor/math`

* preview, test=docs_preview

* scale, test=docs_preview

* stanh, test=docs_preview

* multiplex, test=docs_preview

* pow, test=docs_preview

* max, test=docs_preview

* others..., test=docs_preview

* update trunc example code, test=docs_preview

* add paddle-symbolic-trace to paddle

* add symoblic trace

* delete swp

* support Layer in symbolic trace

* fix test-symbolic-trace, make symbolic trace return a StaticFunction

* template the error message

* fix some unittest

* Modify the execution mode of test

* Modify the module name

* add dy2static unittest decorator

* change some unittest files by @ast_only_test

* fix unittest.

* test-symbolic-trace

* update test_write_python_container.py

* update

* fix test_param_parse.py

* add submodule and ln -sf in cmakefile

* update

* update

* fix some ast only errors

* update

* Polish ut

* fix unittests

* update

* update

* fix unittests

* update

* test warning ast only

* update

* Ast only some uts

* Fix unitests

* test_error ast only

* update

* update

* Support build_strategy for sot

* update

* import sot as a third party module

* update

* update

* Polish code

* update

* update

* update

* update

* update

* remove old fluid api and use paddle.nn.relu instead

* fix

* comment the print of ast code

* add try-finally block

* fix dy2static stop-gradient bugs

* fix code

* remove unused submodule and minor codestyle fix

* fix

* fix cast error

* fix interpolate meets int64 in static model

* add evalframe support for py311

* fix

* fix err

* switch ENABLE_FALL_BACK=False

* fix

* Fix CI for some unittest

* add ENABLE_SOT

* remove setup.py dependences

---------
Co-authored-by: NNotHaozi <zhangmenghao@baidu.com>
Co-authored-by: Nfeifei-111 <2364819892@qq.com>
Co-authored-by: N0x45f <wangzhen45@baidu.com>
Co-authored-by: NSigureMo <sigure.qaq@gmail.com>