* Sharding add hybrid-dp feature

* update sharding in distributed_strategy

* update sharding unitest

* revise code format for sharding

* Change the api of DataParallel and Fleet (#29224)

* add sharding

* refine fleetrun.ps_launch

* update fleet run for multi device support

* ps_graph support ps-gpu

* fix heter save

* add heter save unittest

* fix unittest & simple code

* update fleetrun

* fix fleetrun

* fix launch barrier

* fix role maker

* add paddlecloud rolemaker unittest

* rename heter_worker_device_guard

* fix test_dist_fleet_heter_ctr & peformance update

* add adaptivelsgd

* Todo fix the code to avoid the conflict.

* rm auto from localsgd

add lars to fleet meta optimizer

* add cudnn related strategies to DistributedStrategy

add lamb to fleet meta optimizer

* add paddle.fleet.AsyncOptimizer
Co-authored-by: Ndongdaxiang <dongdaxiang@baidu.com>

Add dgc to fleet meta optimizer, rm dgc from optimizer all

* split meta optimizer files
* add graph execution in execution, update two properties in DistributedStrategy, unit tests for these features

* refine strategy compiler and meta optimizers
make async as a_sync

* add more settings for distributed strategy
Basically, DistributedStrategy has several parts of configurations:
- BuildStrategy: the same as paddle.fluid.BuildStrategy, but the distributed arguments are moved out of BuildStrategy
- ExecutionStrategy: the same as paddle.fluid.ExecutionStrategy
- collective communication configs: nccl_comm_num, hierarchical allreduce and so on
- distributed algorithms: async_update(mainly used in PS), lars, lamb and so on

refactor fleet api under paddle.fleet
update DistributedStrategy

* add paddle.fleet.DistributedStrategy for 2.0