* Fix the bug in fleet amp_init.

* Fix the amp_init unit test.

* add fleet amp.init()

* add unittest for fleet_amp_init

* update, test=develop

* add trainers for pserver

Change-Id: I1a75793ec81ce126d07f4c47cae09b95d530bbc8

* test=develop, fix fleet.metrics(mse, rmse, mae)

Co-authored-by: NseiriosPlus <tangwei12@baidu.com>

* test=develop, add distributed_infer

* rename sendrecv.proto to namespace paddle.distributed

* split ps with distributed

* add save tensor support
Co-authored-by: NseiriosPlus <tangwei12@baidu.com>

* add tensor table

* Simplify the options of spawn based on fleetrun

* polish details

* polish doc details

* set flags_selectedd_gpus for spawn

* add cond for unittest

* Delete test_no_single_process_using_multi_gpus_in_spawn.py

* Update spawn.py

* Update nccl_context.cc

* update, test=develop

* add distributed.split, test=develop

* update, test=develop

* oneps (3/4)
Co-authored-by: NMrChengmo <cmchengmo@163.com>
Co-authored-by: Nmalin10 <malin10@baidu.com>
Co-authored-by: Nchengmo <chengmo@baidu.com>

* fix fleet for multi-stream

* fix memcpy for ncclid

* use sync to solve move operation

* Sharding add hybrid-dp feature

* update sharding in distributed_strategy

* update sharding unitest

* revise code format for sharding

* add tensor_indices in AssignGroupBySize

* add rebuild group in reducer

* update, test=develop

* fix doc, test=document_fix

* fix fleet api doc

* test=develop, rm pathlib

* add reducer

* refine envent for memorycopy

* add concat&split for allreduce

* apply concat & split for fuse tensor

* fix nccl dep

* fix the untest, compile problem and ddp initialize problem

* fix untest for mac & add some comments & solve the repeated param in sublayers

* fix untest for windows & fix document