[cherry-pick][Hybrid Performance] Move the cast op of AMP which cast fp32 param to fp16 param to the optimizer (#34965) (#35296)
Co-authored-by: NWangXi <wangxi16@baidu.com>

* cache core.ops, test=develop

* refine, test=develop

* add amp support for pp

* add amp untest

* dygraph sharding

* update unitest hybrid_parallel_communicate_group

* update, test=develop

* fix pipeline

* fix mp pp dp

* fix utest of hybrid parallel

* add utest for tuple

* fix precision of mp

* fix bug of seed

* fix dp

* print group

* add raw program, test=develop

avoid polluting logging's root logger

* fix amp bug

* fix name of wordsize

* add pipeline layer