* Add Support for Tuple in for Loop (#30998)

Dy2stat didn't support tuple as iteration variable in the past. This PR added there main cases:

       1). Non-enumerate case: for var1, var2 in var|var.numpy() will be re-written as:
          for FOR_ITER_TUPLE_PREFIX_x in var | var.numpy():
            var1 = FOR_ITER_TUPLE_PREFIX_x[0]
            var2 = FOR_ITER_TUPLE_PREFIX_x[1]
        2). Enumerate out tuple case: for t in enumerate(var|var.numpy) will be rewritten as:
          for FOR_ITER_TUPLE_INDEX_PREFIX_x, FOR_ITER_TUPLE_PREFIX_x in enumerate(var|var.numpy):
            t = (FOR_ITER_TUPLE_INDEX_PREFIX_x, FOR_ITER_TUPLE_PREFIX_x)
        3). Enumerate inner tuple case: for i, (var1, (var2, va3)) in enumerate(var|var.numpy()) will
        be re-written as:
          for i, FOR_ITER_TUPLE_PREFIX_x in var | var.numpy():
            var1 = FOR_ITER_TUPLE_PREFIX_x[0]
            var2 = FOR_ITER_TUPLE_PREFIX_x[1][0]
            var3 = FOR_ITER_TUPLE_PREFIX_x[1][1]

* Refine fake_interface Error Message (#30981)

Refine fake_interface Error Message

Cherry-pick：Split unittest(#30727) to solve the problem of 'test_static_save_load' timeout.(#30727) (#30916)

单测test_static_save_load耗时较长，将主要耗时部分单独拆分出来，以免增加其他单测对其造成影响。
原始PR：#30727

cherry-pick #29998
* Polish and Optimize the print/repr message of all layer
* fix some code format

make abs support complex types

cherry-pick:
#30375
#30637

Co-authored-by: Ntianshuo78520a <707759223@qq.com>

* upgrade gather_tree to core.ops (#30697)

* upgrade gather_tree to core.ops

* update gather_tree unittests

* update gather_tree doc (#30693)

* update gather_tree doc, test=document_fix

* update sample code, test=document_fix

* remove tensor type, test=document_fix

* Add cache for Transformer encoder.

* Bug fixed.

* add unittests for transformer encoder.

* Alias from  paddle.fluid.layers.auc to paddle.static.auc (#30206)

* add alias from  fluid.layers.auc to static.auc

* Update __init__.py

* add auc into all list

* alias acc, expose to users

* add auc into 'all' list (#30310)

* add auc into 'all' list

* alias acc, expose to users

* update sample code

* fix paddle.static.acc and auc sample code bug, test=document_fix

* Add tf32 support for A100 tensor core acceleration for cuBLAS (#28732)

* Fixed an error

* Fixed an error

动态图中Conv2D保存成预测模型时，对应的Op可能是conv2d，也可能是depthwise_conv2d，但目前的save_quantized_model接口并未考虑depthwise_conv2d情况，可能会致使out_scale的值保存错误，该PR主要是修复这个问题。

* Fix the bug in fleet amp_init.

* Fix the amp_init unit test.

This PR is cherry-picked from PR: #29192
Function: Added TF32 switch for cuDNN. Turned on as default, turned off when users set the switch as False

[Dy2static]Fix paddle prefix in is_paddle_api (#30569)
cherry-pick #30569

* add fleet amp.init()

* add unittest for fleet_amp_init

* Implemented AddQuantDequantPass in imperative quantization.

* support 2.0 API such as Pool2D and ReLU

原始PR：#30485，#30507

[cherry-pick] support layer_norm fp16 in dygraph amp (#30430)

完善static.load的var_list参数。
当加载的是多个小文件时，Tensor列表可以是所有加载文件中Tensor的子集。
原始PR：#30457

cherry-pick #30536

* add 2.0 API: device_guard

* support transformer v2.0

* fix range op crash in dygraph xpu place

 cherry-pick #30527 

* Modify the calculation logic of LambOptimizer (#29313)

* Modify the calculation logic of LambOptimizer

* Modify the calculation logic of LambOptimizer

* Modify the calculation logic of LambOptimizer

* add pad and concat double grad

* resolve conflict

* add fp16 support for tril_triu op (#30186)

* add VecCastCUDAKernel (#30296)
Co-authored-by: Nfurnace <34057289+windstamp@users.noreply.github.com>

* test=develop, fix fleet.metrics(mse, rmse, mae)

Cherry-pick PR 30103. Add Inplace strategy (Output reuse Input Varbase) in dygraph (#30103) (#30496)

* add view strategy on squeeze,unsqueeze,reshape,flatten

* add squeeze unittest

* add unittests

* use View strategy as name rather than Reuse Allacation

* fix view api doc

* fix format

* use core.ops when input of reshape2 is Tensor

* fix test_cross_entropy_loss error because of reshape2

* fix test_cross_entropy_loss error because of reshape2

* add inplace strategy

* add elementwise_add sub

* let backward op not use inplace

* grad op do not use inplace

* fix memory increase error and add leaf error message

* delete selected_rows

* change op_function

* little change

* solve HandleViewBetweenInputAndOutput

* add unittest and leaf error message

* merge view error

* optimize op_function_generator format and support sum inplace op

* fix format of basic_engine

* fix format for framework

* little change of variable wrapper

* add reshape, squeeze, unsqueeze, scatter api

* add relu elu tanh softmax inplace api

* fix test_squeeze_op unittest

* fix test_relu_op unittest

* fix comment problems

* delete sample code of inplace api

* add reference of grad_pending_nodes in basic_engine

* fix unittest name

* add inplace apis into wlist

* fix error message

* add PADDLE_ENFORCE for set grad op twice

* fix head file error

* Cherry-pick 30072, add dispenable input for core.ops.reshape2/expand/slice (#30072)

* add dispenable input 'shape' for core.ops.reshape2

* add dispenable inputs for core.ops.reshape2/expand/slice

* add ut

* save reshape update in pr 30180

* save reshape update v2 in pr 30180
Co-authored-by: NLeo Chen <chenqiuliang@baidu.com>

* add transpose double grad test=develop (#29600)

* add transpose double grad test=develop

* cherry-pick test=develop

* Alias from  paddle.fluid.layers.auc to paddle.static.auc (#30206)

* add alias from  fluid.layers.auc to static.auc

* Update __init__.py

* add auc into all list

* alias acc, expose to users

* add auc into 'all' list (#30310)

* add auc into 'all' list

* alias acc, expose to users

* update sample code

* test=develop, add distributed_infer (#30300)

* test=develop, add distributed_infer

* test=develop, fix unittest cmakefile conflict

* test=develop, fix test_dist_fleet_base

 [cherry-pick2.0]Enhance installation error message after separating AVX and NO_AVX compilation #30442 

cherry-pick #30413
1. 30架构对应很早期的显卡，在2.0及之后移除该架构编译
2. 分离avx与core_avx编译，并优化了安装报错信息。