[cherry-pick][Hybrid Performance] Move the cast op of AMP which cast fp32 param to fp16 param to the optimizer (#34965) (#35296)
Co-authored-by: NWangXi <wangxi16@baidu.com>

* fix bug of py36 import

* add recompute for pp

* add recompute offload

* add recompute partition

* add save/load for pipelineparallel

* add save/load

* fix bug of p2p for partial

* fix error

* add layer segment

* add segement for transformer

* add utest

* support 1f1b for pipeline

* add utest

* add send_partial/recv_partial

* support amp for pp

* fix logger

* add trainer desc config to distributed strategy

* code style modified

* fix force kill for elastic

* cache core.ops, test=develop

* refine, test=develop

* add scatter-gather opt

* fix topo for pp

* rename function

* kill process by group

* pipeline adaptive for heterps;test=develop
* fix finalize hang;test=develop
* add is_compiled_with_heterps for dataset;test=develop
* fix hashtable core when pass ins_num=0;test=develop

* add amp support for pp

* add amp untest

* fix hdfs and fleet_util for supporting save/load infer;test=develop

* add random and prevent deadlock