• H
    [cherry-pick]Support FP16 in HybridParallel and Fix bugs in HybridOptimizer (#36707) · 5b357e02
    Haohongxiang 提交于
    * fix bugs in HybridParallelClipGrad of hybrid_parallel_optimizer (#36237)
    
    * fix bugs in HybridParallelClipGrad of hybrid_parallel_optimizer
    
    * update
    
    * update
    
    * fix bugs in mp_layers、pp_layers and HybridParallelClipGrad (#36144)
    
    * fix calling bug of HybridParallelClipGrad
    
    * fix bugs of HybridParallelClipGrad
    
    * add unittest of pp with HybridParallelClipGrad
    
    * fix bugs in mp_layers.py
    
    * update
    
    * fix bugs in pp_layers.py
    
    * update
    
    * [HybridParallel]Rebuild code for pipeline (#36396)
    
    * add no_sync for parameters sync
    
    * add pipeline for moe
    
    * [HybridParallel]Support fp16 in dygraph hybrid parallel (#36420)
    
    * [HybridParallel]Support fp16 in dygraph hybrid parallel
    
    * update
    
    * update
    
    * update for recompute
    
    * add unittest of pp+fp16
    
    * add unittest of recompute+fp16
    
    * update
    
    * modify ut
    
    * modify ut of cond (#36475)
    
    * fix bugs of ClipGradByGlobalNorm in HybridParallel (#36555)
    
    * fix bugs of ClipGradByGlobalNorm
    
    * add unittests
    
    * add unittests
    
    * [HybridParallel]fix bug of check_inf in fleet_base.py (#36651)
    
    * fix bug of check_inf
    
    * fix allreduce
    
    * support ClipGradByGlobalNorm in sharding (#36012)
    
    * support ClipGradByGlobalNorm in sharding
    
    * support ClipGradByGlobalNorm in sharding
    
    * test=allcase
    
    * Update test_linalg_cond.py
    
    * Update hybrid_parallel_util.py
    
    * Update hybrid_parallel_util.py
    Co-authored-by: NShenLiang <1422485404@qq.com>
    Co-authored-by: Nzhaoyingli <86812880+zhaoyinglia@users.noreply.github.com>
    5b357e02
test_dygraph_recompute.py 8.8 KB