* add distributed.split, test=develop

fix the state_dict bug for the xpu

* fix fleet for multi-stream

* fix memcpy for ncclid

* use sync to solve move operation

[Dy2Stat] Fix bug for loop: a variable is used and created in loop, but used before created (#29769)

Enable jit.save to Save Without Running.

Support to transformfor ele in var stms in which var is a slice of Tensor.

[Dy2Stat] 1. Fix bug of for-range stmts. 2. Support that step value is negative in for-range stmts (#29519)

1. Fix error in _build_cond_stmt of for-range stmts. 

2. Support that step value is negative in for-range stmts

3. Fix code because of the diff between Py2 and Py3

* add fast path for p == 0 in dropout

* add ut

* Add ReserveSpace in dygraph batch_norm.

* Add unittest for reservespace

* support deepcopy for Layer/Tensor/Paramerbase

* fix some code

* add tensor_indices in AssignGroupBySize

* add rebuild group in reducer

* fix the bug of reducer in embedding

* add comment

* rm complexvariable

* modify test_var_base unittest

* remove duplicated codes

Reduce exception type so that if covert_to_static failed, it reports right error message.

[Dy2Stat] Fix bug: Do not use gast.Subscript to replace gast.Name in when transforming for_enumerate_loop (#29310)

* basic impl of type promote

* add comment & another testcase

* fix complex bugs & support python op promote type

* fix failed unittests & polish code

* add unittest for coverage

* change to only promote complex type

* polish code details

* polish several comments

* use has_grad instead of train_mode

* add vlog for debug

* fix ut

* fix ut

[Dy2stat] Add a decorator paddle.jit.not_to_static to support that not to convert a function in Dynamic-to-Static. (#29253)

Usage scenarios：A function could have run successfully in static mode,  you can use it to decorate a function in the following cases:
  1. An unknown error occurs in the dynamic-to-static conversion process of the function;
  2. In the internal implementation of the function, it has two branches: dynamic branch and static branch;
  3. Users don't want to convert the function in the process of dynamic to static.

This PR fixes several problems in dy2stat for Deoldify model in PaddleGan.

In model, software engineer wrote if x.shape == y.shape, the Tenser shape is a tuple in dygraph so the == returns True/False, but in static graph the == becomes element-wise comparison, which is a different behavior. In this PR we reduce the element-wise comparison result.

If software engineer write computations which uses parameters in hooks, the static graph can loss the parameter variable because we put param_guard at forward of a Layer. In this PR we made param_guard cover pre-hook and post-hook.

In PaddleGan, software engineer calculated some parameter values in __init__ by running some dygraph code. Those code also run during dy2stat. So some variables may be assign as a VarBase (Tensor) first and then Variable, which raised an error. We fixed the bug in this PR by handling the case.

TODO: We just added testcase for the 1. shape comparison. Should add test case for 2. and 3. But since we are chasing 2.0RC, I will do it in the near future PR

* The leaf tensor concept is exposed and the gradient accumulation of leaf tensor

* The leaf tensor concept is exposed and the gradient accumulation of leaf tensor

* fix coverage

* fix api doc

* fix CI unittest

* fix CI unittest

* fix unitest

* empty tensor does’t need inner_var_

* fix some error message

* Add a class TensorInplaceVersion to count the inplace version and put it in framework::Tensor instead of Allocation or Variable.

* Add a new attribute `_inplace_version` for VarBase.

* Raise exception if an inplace operation can result in incorrect gradient computation.

* Add a new interface _bump_inplace_version() for VarBase to bump the version whenever the Tensor is modified through an inplace operation.

* For api assign, call _bump_inplace_version() when it's an inplace operation inn dynamic mode.

* Use original var_wrapper if the inplace_version is not changed.

* Replace SnapshotVarWrapperList with SnapshotVarWrapper to optimize performane.

* Changed a variable name error

* Add comments

* Move member functions of TranslatedLayer out of function

* edit code according to review

* Edit input argument of '_run_static_graph'

* reset due to Segmentation fault

* rename variables when stitching graph

* modify code according CI

* Add comments to '__i_m_p_l__'

* remove blanks befor 'Get...'

* edit code according to review

* Add a comment to '_execution_method_creator'

* Edit a comment to '_execution_method_creator'

[Dy2Stat] Fix bug: the return statement should be transformed to an equivalent Paddle/Python if statement, which depends on if conditions of the return stmt. (#29165)

* add reducer

* refine envent for memorycopy

* add concat&split for allreduce

* apply concat & split for fuse tensor

* fix nccl dep

* fix the untest, compile problem and ddp initialize problem

* fix untest for mac & add some comments & solve the repeated param in sublayers

* fix untest for windows & fix document

[Dynamic-to-Static] Support **kwargs as input of the function which is decorated by `jit.save.to_static` (#29098)

* split train_mode and has_grad

* fix format

* fix ci problems

* fix sample code

* Rename variables when use 'jit.load'

* Check whether the original graph contains the variable with the same name

* add comment

* rename output/input of op and edit unittest

* modify the code according to CI

* edit code according to CI

* edit code according to CI

* edit code according to CI

* edit code according to CI

* edit code according to CI

* edit code according to CI

* Support pop for dict in dy2stat

* Move convert_pop to convert_operators.py and polish convert_pop

* polish parallel api impl & doc details

* add unittest for coverage

* remove spawn test in py2.7

* add parallel api into white list

* fix some docs test=develop;test=document_fix

* add code example test=develop;test=document_fix

* fix code example test=develop;test=document_fix

* fix code example test=develop;test=document_fix

* fix code example test=develop;test=document_fix

Add support for using tuple as tensor.shape (For example: a, b, c, d = x.shape)