-
由 Zhen Wang 提交于
* add pure fp16 training. * update the usage of cast_parameters_to_fp16. * use fp32 to initilize parameter values and then cast to fp16. * enable multi_precision when training with fp16. * add static loss scaling. * add the rescale_grad function in the pure fp16 training. * use cast to complete the pure fp16 training. * use sum loss for fp16 trainging and mean loss for fp32 training. * refine codes according reviewers' suggestions
ba1b19c0