-
由 Joe Mayer 提交于
* BF16 optimizer only with ZeRO stage 1. * Updating to grad accum of fp32 for BF16 ZeRO1 case. Co-authored-by: NOlatunji Ruwase <olruwase@microsoft.com>
8d87c89e
* BF16 optimizer only with ZeRO stage 1.
* Updating to grad accum of fp32 for BF16 ZeRO1 case.
Co-authored-by: NOlatunji Ruwase <olruwase@microsoft.com>