* fix_sharding_group

* fix_sharding_group

* support param groups in grad_clip

* update

* modify for review

* update

* fix bugs

* modify code style

* fix bugs of _get_global_group

* merge latest develop branch

* fix bugs

* update

* fix bugs for unittest

* modify for less use of gpu mem

* fix bugs of using _reset_grad_inplace_version

* update

* update

* modify for CI-Coverage

* retrick all CIs

* Debug

* Fixed issue with reset_grad_inplace_version when used with clear_gradient & cross-batch accumulation

* Rearranged interfaces

* Fixed ci issues

* [HybridParallel]Support fp16 in dygraph hybrid parallel

* update

* update

* update for recompute

* add unittest of pp+fp16

* add unittest of recompute+fp16

* update

* modify ut

* add no_sync for parameters sync

* add pipeline for moe

* refine amp level

* fix typo

* update tracer._amp_level

* fix calling bug of HybridParallelClipGrad

* fix bugs of HybridParallelClipGrad

* add unittest of pp with HybridParallelClipGrad

* fix bugs in mp_layers.py

* update

* fix bugs in pp_layers.py

* update

* add pure fp16 major function in auto_cast & tracer

* support master weight in dygraph for pure fp16

* check mix dtype of fp16&fp32 for check_finite_and_unscale op

* change pure fp16 funtion name

* refine some bug in auto_cast

* refine auto_cast interface logic

* add param _casted_by_pure_fp16 for class Layer

* support state_dict hook for save model by user appointed dtype in pure_fp16_decorator

* refine pure_fp16_decorator as decorator

* add unittest

* add comment

* add comment

* support recompute

* add comment for auto_cast and decorator

* support to_static_state_dict for paddle.jit.save

* unlimite models num and optimizers num

* add lookup_table in black_list

* fix momentum and layer state_dict

* fix bug in layer state_dict

* fix bug in layer state_dict_helper

* refine unittest

* refine test_momentun_op

* refine interface and some code

* refine amp_decorator interface

* refine pure fp16 interface

* refine master weight interface

* support grad group

* fix single card condition

* sharding support dp

* sharding support mp

* sharding support pp

* add cache for send_recv

* add eval_batch for pipeline

* add eval batch for pipelineparallel

* add style code

* fix bug of py36 import

* add recompute for pp

* add recompute offload

* add recompute partition

* add save/load for pipelineparallel

* add save/load

* fix bug of p2p for partial

* fix error

* add layer segment

* add segement for transformer

* add utest

* support 1f1b for pipeline

* add utest

* add send_partial/recv_partial

* support amp for pp

* fix logger

* add scatter-gather opt

* fix topo for pp

* rename function