未验证 提交 195563a2 编写于 作者: Y YiSheng5 提交者: GitHub

fix a small type error on bf16+pp (#3441)

Co-authored-by: NOlatunji Ruwase <olruwase@microsoft.com>
Co-authored-by: NLogan Adams <114770087+loadams@users.noreply.github.com>
上级 f1fab902
......@@ -243,7 +243,7 @@ class PipelineEngine(DeepSpeedEngine):
self._force_grad_boundary = True
if self.pipeline_enable_backward_allreduce:
if self.bfloat16_enabled():
if self.zero_optimization_stage() < ZeroStageEnum().gradients:
if self.zero_optimization_stage() < ZeroStageEnum.gradients:
self._bf16_reduce_grads()
else:
raise NotImplementedError("PP+BF16 only work for ZeRO Stage 1")
......
Markdown is supported
0% .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑!
想要评论请 注册