* consider the combination of different strategies to work together

disable test_dist_fleet_ctr

* disable UT temp

* add save/load for parameter server

* add SyncBatchNorm,test=develop

set default python3,fix incompatible,cache dir for third party,unify error code,for windows (#26178)

* set default python3 for paddle windows,test=win

* set default python3,cache dir for third party,error code,test=win

* fix some incompatible

* fix some error

* set virtual environment,test=win

* move paddle.fleet to paddle.distributed.fleet

* add auto_cast, test=develop

* add loss scaler, test=develop

* add comments, test=develop

* refine code, test=develop

* refine code, test=develop

* do not set flags automatically, test=develop

* fix custom op bug, test=develop

* add more test, test=develop

* refine enable logic, test=develop

* enable amp test with GPU, test=develop

* add unittest

* add test for found_inf

* follow comments

* follow comments

* remove global variable, use singleton

* add some notes

* update comments

* update comments

* update comments

* add use_dynamic_loss_scaling argument

* refine found_inf

* refine found_inf

* add IterableDataset support in multiprocess DataLoader. test=develop

add lamb to fleet meta optimizer

* add the support for multi-node training

* test=develop,test=document_fix, remove the out args

* fleet_util move to paddle.fleet
Co-authored-by: NWuHaobo <wuhaobo1994@gmail.com>
Co-authored-by: Ntangwei12 <tangwei12@baidu.com>

Add dgc to fleet meta optimizer, rm dgc from optimizer all

* add fleetrun command for distributed running; test=develop

* split meta optimizer files
* add graph execution in execution, update two properties in DistributedStrategy, unit tests for these features

* add localsgd meta optimizer

* add pipeline optimizer

* fix test_pipeline, test=develop

* add gradient Merge optimizer to meta, test=develop

Integrated Trainer of Parameter Server (API add `fluid.contrib.layers.sparse_embedding` only) (#22957)

* Integrated Trainer of Parameter Server

* fix best_fit_allocator_test on windows, test=develop

* enable best_fit_allocator_test and test_math_op_patch_var_base, test=develop

just skip case of distribute on windows

refactor fleet api under paddle.fleet
update DistributedStrategy

* test_dist_fleet_ctr disable, test=develop

* modify pipeline optimizer, test=develop

test=develop

* add test_parallel_executor_profiler.py

test=develop

* set exec_strategy.num_threads=1

* skip check_profile_result

* add TODO for test_profiler

test=develop