未验证 提交 94afa5ab 编写于 作者: Y Yulong Ao 提交者: GitHub

[Auto Parallel] Add the micro-bathsize config (#52912)

上级 1765d5d1
......@@ -107,6 +107,11 @@ class Parallelizer:
)
# Do reshard process
time0 = time.time()
micro_bsz = (
1
if not self._strategy.pipeline.enable
else self._strategy.pipeline.micro_batch_size
)
set_grad_var_shape(dist_main_prog, self._dist_context)
resharder = Resharder(
dist_main_prog,
......@@ -114,6 +119,7 @@ class Parallelizer:
rank,
self._dist_context,
dist_params_grads,
micro_bsz,
)
resharder.reshard()
self._logger.debug(
......
Markdown is supported
0% .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑!
想要评论请 注册