* add save/load for parameter server

* add SyncBatchNorm,test=develop

set default python3,fix incompatible,cache dir for third party,unify error code,for windows (#26178)

* set default python3 for paddle windows,test=win

* set default python3,cache dir for third party,error code,test=win

* fix some incompatible

* fix some error

* set virtual environment,test=win

* move paddle.fleet to paddle.distributed.fleet

* add auto_cast, test=develop

* add loss scaler, test=develop

* add comments, test=develop

* refine code, test=develop

* refine code, test=develop

* do not set flags automatically, test=develop

* fix custom op bug, test=develop

* add more test, test=develop

* refine enable logic, test=develop

* enable amp test with GPU, test=develop

* add unittest

* add test for found_inf

* follow comments

* follow comments

* remove global variable, use singleton

* add some notes

* update comments

* update comments

* update comments

* add use_dynamic_loss_scaling argument

* refine found_inf

* refine found_inf

* add IterableDataset support in multiprocess DataLoader. test=develop

add lamb to fleet meta optimizer

* add the support for multi-node training

* test=develop,test=document_fix, remove the out args

* fleet_util move to paddle.fleet
Co-authored-by: NWuHaobo <wuhaobo1994@gmail.com>
Co-authored-by: Ntangwei12 <tangwei12@baidu.com>

Add dgc to fleet meta optimizer, rm dgc from optimizer all

* add fleetrun command for distributed running; test=develop

* split meta optimizer files
* add graph execution in execution, update two properties in DistributedStrategy, unit tests for these features

* add localsgd meta optimizer

* add pipeline optimizer

* fix test_pipeline, test=develop

* add gradient Merge optimizer to meta, test=develop

Integrated Trainer of Parameter Server (API add `fluid.contrib.layers.sparse_embedding` only) (#22957)

* Integrated Trainer of Parameter Server

* fix best_fit_allocator_test on windows, test=develop

* enable best_fit_allocator_test and test_math_op_patch_var_base, test=develop

just skip case of distribute on windows

refactor fleet api under paddle.fleet
update DistributedStrategy

* test_dist_fleet_ctr disable, test=develop

* modify pipeline optimizer, test=develop

test=develop

* add test_parallel_executor_profiler.py

test=develop

* set exec_strategy.num_threads=1

* skip check_profile_result

* add TODO for test_profiler

test=develop

* support selectedrows allreduce in multi-cards dygraph, test=develop

* remove useless import modules in unittests, test=develop

* add nccl cmake to get nccl version, test=develop

* add if-condition to compiled correctly, test=develop

* add detail version parseing for old nccl, test=develop

* polish camke details, test=develop

* fix remove test cmake error, test=develop

* fix cmake condition, test=develop

* change unittest camke list, test=develop

* fix unittest cmake rule, test=develop, test=framep0

* add crypto helper for paddle, test=develop

* cryptopp.cmake bug fixed, test=develop

* remove debug build type, test=develop

* fixed CMakeLists for new target, test=develop

* fix CI bug, test=develop

* add cmake option flag DWITH_CRYPTO, test=develop

* add crypto api for python, test=develop

* Revert "add crypto api for python, test=develop"

This reverts commit 3a1cfa9d.

* Revert "Add crypto api (#24694)"

This reverts commit 5a7a517c.

* Revert "Revert "Add crypto api (#24694)""

This reverts commit f952b19f.

* fixed cryptopp cmake building error, test=develop

* change WITH_CRYPTO building option to OFF, test=develop

* â€fixed cipher test failed, test=develop

* "add crypto api for python, test=develop"

This reverts commit 83fb55c0.

* travis CI bug fixed, test=develop

* fixed test in python3

* test=develop

* fixed unittest, test=develop

Enhance error message of checkpoint_notify_op, fake_init_op gen_nccl_id_op and listen_and_serv_op (#24554)

test=develop