Co-authored-by: NWangXi <wangxi16@baidu.com>

Co-authored-by: NWangXi <wangxi16@baidu.com>

[cherry-pick][Hybrid Performance] Move the cast op of AMP which cast fp32 param to fp16 param to the optimizer (#34965) (#35296)
Co-authored-by: NWangXi <wangxi16@baidu.com>

* fix bug of py36 import

* add recompute for pp

* add recompute offload

* add recompute partition

* add auto_parallel apis

* add save/load for pipelineparallel

* add save/load

* fix bug of p2p for partial

* fix error

* add layer segment

* add segement for transformer

* add utest

* support 1f1b for pipeline

* add utest

* add send_partial/recv_partial

* support amp for pp

* fix logger

* add trainer desc config to distributed strategy

* code style modified

* fix force kill for elastic

* fix the docs bug in collective file

Fix the bug in paddle.distributed.split demo. The paddle.distributed.split op just can be used in static mode. (#34306)

* fix the bug in paddle.distributed.split demo

* add wait_server_ready

* fix remove bug

* fix the bug that happened in alltoall in dygraph mode

* cache core.ops, test=develop

* refine, test=develop

* add scatter-gather opt

* fix topo for pp

* rename function

* kill process by group

* opt dygraph python code, test=develop

* refine, test=develop