* add dygraph parallel run interface

* polish implement & unified env property name

* add print config arg

* refactor init_parallel_env function

* Compatible with multiprocessing and launch modes

* set default trainer start port

* support run in python 2

* polish python2 support code

* remove python2 support

* refine launch import

* polish dome design details

* refactor api implemention & path

* use new method _set_expected_place

* add spawn unittest framework & mnist test

* add more unittests & doc

* fix unittest failed

* polish english doc

* self review and polish details

* refactor code by reviewer's comments

* fix unittest failed

* fix parallel_env unittest

* fix several typos

* fix error introduced when fixing typos

* add unpublic note for start_processes

* polish details by xiaoguang's comment

* verify correctly when spawn nprocs=-1

* refactor spawn & init_parallel_env design

* polish doc details

* open spawn unittests

* try to fix doc compile error

* try to fix unknown doc format error

* add skip unittest when not gpu

* modified timeout value on windows and mac (#26690)

* add Local SGD algorithm referenced paper
test=develop

add collective op for cpu using gloo and paddle.distributed.* apis

* add cudnn related strategies to DistributedStrategy

* solve the initial configuration about fleet and rolemaker
Co-authored-by: NseiriosPlus <tangwei12@baidu.com>

* add documentation for DistributedStrategy

* consider the combination of different strategies to work together

* add save/load for parameter server

* add feature to fleet2.0 role_maker, distribute_strategy, test=develop

* add check approval
test=develop

* make fleet_localsgd_meta_optimizer work

* fix bug in localsgd meta optimizer

* set default value to strategy in distributed_optimizer test=develop

* add unittest for paddlerolemaker with gloo

* move paddle.fleet to paddle.distributed.fleet

test=develop

* fix the compatibility of PY2 and PY3 in paddle.distributed.launch
test=develop

* only pull log of local rank 0
test=develop

* log exception if UnicodeEncodeError occurs when pulling log in
paddle.distributed.launch
test=develop
Co-authored-by: NSunGaofeng <peakbee@gmail.com>

test=develop

replace join to terminate

* add DataLoader, Dataset, BatchSampler

* add launch_ps module so that we can launch a parameter server training job
1) a user can specify worker_num and server_num
2) parameter server can be killed after all workers exit
3) unit test is added
test=develop

distribute.launch use poll to query subprocess

distribute launch : add use_paddlecloud argument 

node_num is not needed for users, so remove them and fix the bugs about it!

refine launch_ps and role_maker

add parameter server launch so that a user can easily launch parameter server