* refine launch and distributed repr string for print

* add auto parallel L1 implementation
test=develop

* fix fluid.embedding

* Add RNN related apis in paddl.nn
test=develop

* new rnn api, cell almost done

* add new progresses in rnn APIs for 2.0

* refine rnn APIs and docstrings.

* add unittets

* disable gpu tests when paddle is not compiled with cuda support

* remove unnecessary imports

* fix docstring

* add to no_sample wlist

* backport to python2 to avoid yield from

* add **kwargs, fix typos

* update docstrings for birnn

* rename argument for SimpleRNN and SimpleRNNCell, fix sample code

* add default value for initial_states in fluid.layers.birnn
Co-authored-by: Nguosheng <guosheng@baidu.com>

* add TensorDataset. test=develop

add collective op for cpu using gloo and paddle.distributed.* apis

* find timeout unittests

* setting timeout value

* fix some error

* fix some error

* fix some error

* fix no newline of end file error

* consider the combination of different strategies to work together

disable test_dist_fleet_ctr

* disable UT temp

* add save/load for parameter server

* add SyncBatchNorm,test=develop

set default python3,fix incompatible,cache dir for third party,unify error code,for windows (#26178)

* set default python3 for paddle windows,test=win

* set default python3,cache dir for third party,error code,test=win

* fix some incompatible

* fix some error

* set virtual environment,test=win

* move paddle.fleet to paddle.distributed.fleet

* add auto_cast, test=develop

* add loss scaler, test=develop

* add comments, test=develop

* refine code, test=develop

* refine code, test=develop

* do not set flags automatically, test=develop

* fix custom op bug, test=develop

* add more test, test=develop

* refine enable logic, test=develop

* enable amp test with GPU, test=develop

* add unittest

* add test for found_inf

* follow comments

* follow comments

* remove global variable, use singleton

* add some notes

* update comments

* update comments

* update comments

* add use_dynamic_loss_scaling argument

* refine found_inf

* refine found_inf

* add IterableDataset support in multiprocess DataLoader. test=develop

add lamb to fleet meta optimizer

* add the support for multi-node training

* test=develop,test=document_fix, remove the out args

* fleet_util move to paddle.fleet
Co-authored-by: NWuHaobo <wuhaobo1994@gmail.com>
Co-authored-by: Ntangwei12 <tangwei12@baidu.com>

Add dgc to fleet meta optimizer, rm dgc from optimizer all

* add fleetrun command for distributed running; test=develop

* split meta optimizer files
* add graph execution in execution, update two properties in DistributedStrategy, unit tests for these features

* add localsgd meta optimizer

* add pipeline optimizer

* fix test_pipeline, test=develop

* add gradient Merge optimizer to meta, test=develop

Integrated Trainer of Parameter Server (API add `fluid.contrib.layers.sparse_embedding` only) (#22957)

* Integrated Trainer of Parameter Server

* fix best_fit_allocator_test on windows, test=develop

* enable best_fit_allocator_test and test_math_op_patch_var_base, test=develop

just skip case of distribute on windows