* test=develop, save/load, shrink
Co-authored-by: NseiriosPlus <tangwei12@baidu.com>
Co-authored-by: N123malin <malin10@baidu.com>

* Fix the bug in fleet amp_init.

* Fix the amp_init unit test.

* add fleet amp.init()

* add unittest for fleet_amp_init

* Fix server.h include device_context (#30243)

* fix cmake
Co-authored-by: NseiriosPlus <tangwei12@baidu.com>

* 【Paddle.Fleet】Support local save sparse param (#30175)

* add save tensor support
Co-authored-by: NseiriosPlus <tangwei12@baidu.com>

* add sparse embedding & load vars for 2.0 & gloo bug fix (#30306)

* add sparse embedding & load vars for 2.0

Change-Id: I36b59ed5f015189dc9d9d2e34a9357722d369f1b

* fix hdfs gloo

Change-Id: Ia84d579053720ad804183e54c9a04b4f031c79c6

* fix gloo hdfs

Change-Id: I5ab982fd483cddc10adcdef0b8aa83aca976cb9e

* move loadvar/sparse embedding from incubute to static

Change-Id: I57081d3545ad2efab78c72420d2162c0eacaf3a0
Co-authored-by: Ntangwei12 <tangwei12@baidu.com>

* fix fleet for multi-stream

* fix memcpy for ncclid

* use sync to solve move operation

* fix the warning of reducer (#29323)

* fix warning of fleet (#29317)

* Fix doc of fleet api (#29282)

* Change the api of DataParallel and Fleet (#29224)

* fix fleet api doc

* add reducer

* refine envent for memorycopy

* add concat&split for allreduce

* apply concat & split for fuse tensor

* fix nccl dep

* fix the untest, compile problem and ddp initialize problem

* fix untest for mac & add some comments & solve the repeated param in sublayers

* fix untest for windows & fix document

fleet support paddle.optimzier

* bug fix

* fix fleet_base

* bug fix

* fix coverage

* add size method for large scale

* add large scale UT

* add ut for checkpoint

* raise error if use multi-cards in fleet non_distributed mode; test=develop

* fleet support non_distributed training in dygraph mode; test=develop

* add get final strategy for user to print final strategy

* test=develop, bug fix

* fix fleet util and gloo

* fix worker endpoints

* fix

* fix UT

* fix gloo

* fix gloo

* update gloo

* update gloo

* update gloo

* update gloo

* update gloo

* fix gloo wrapper for hdfs

* add file gloo and UT

* fix UT

* fix UT

* fix UT

* hide public method of RoleMaker

* fix UT

* GPU fleetrun support gloo

* parameterserver fleetrun support gloo

* add UT

* add UT

* fix UT

* fix get server endpoint

* fix get server endpoint

* fix UT

* hide public method of rolemaker

* hide public method of rolemaker

* hide public method of rolemaker

* Update test_fleet_rolemaker_new.py

* hide public method of rolemaker

* hide public method of rolemaker

* fix worker endpoints

* fix gloo wrapper for hdfs

* GPU fleetrun support gloo

* parameterserver fleetrun support gloo

* fix get server endpoint

* fix fleet dynamic-mode en-doc;test=develop

* parameter_server_optimizer support auto_strategy

* add auto parallel L1 implementation
test=develop

paddle.distributed.fleet supports dynamic graph execution.

* support running python train.py for fleet-task; test=develop

* add FleetAPI doc
Co-authored-by: Nliuyuhui <liuyuhui@baidu.com>

* Support Heter Parameter Server

* fix api document

* add cudnn related strategies to DistributedStrategy

* solve the initial configuration about fleet and rolemaker
Co-authored-by: NseiriosPlus <tangwei12@baidu.com>

* consider the combination of different strategies to work together

* add save/load for parameter server

* set default value to strategy in distributed_optimizer test=develop

* move paddle.fleet to paddle.distributed.fleet

* Directory migration, test=develop

* Change imperative from paddle init to paddle framework, test=develop

* Fixed jit bug, test=develop

* default static mode, test=develop

* fixed format and create parameter belongs to framework, test=develop

* Fixed import package, test=develop

* fix __init__ format, test=develop

* fixed alias problem

* fixed paddle.enable_imperative problems, test=develop

* Add unittest

* delete install_check comment

* Fixed unittest timeout

* fixed unittest error

* move Program default_xx_program to static package

* optimize unittest method

* fixed framework __init__ format

* fixed jit path

* delete alias

* move jit to paddle

* Fixed unittest format

* fixed paddle.default_main_program

* Fixed save load API in paddle __init__.py

* fixed ci paddle.imperative.to_variable

* add paddle.fleet.AsyncOptimizer
Co-authored-by: Ndongdaxiang <dongdaxiang@baidu.com>

* generate context during compile

* split meta optimizer files
* add graph execution in execution, update two properties in DistributedStrategy, unit tests for these features