• Z
    Support pure fp16 training for ResNet50. (#4979) · ba1b19c0
    Zhen Wang 提交于
    * add pure fp16 training.
    
    * update the usage of cast_parameters_to_fp16.
    
    * use fp32 to initilize parameter values and then cast to fp16.
    
    * enable multi_precision when training with fp16.
    
    * add static loss scaling.
    
    * add the rescale_grad function in the pure fp16 training.
    
    * use cast to complete the pure fp16 training.
    
    * use sum loss for fp16 trainging and mean loss for fp32 training.
    
    * refine codes according reviewers' suggestions
    ba1b19c0
train.py 11.7 KB