未验证 提交 722f880e 编写于 作者: Z zhaoyingli 提交者: GitHub

make micro bsz configurable (#52447)

上级 964497b5
...@@ -155,6 +155,11 @@ class Parallelizer: ...@@ -155,6 +155,11 @@ class Parallelizer:
time.time() - time0, self._mode time.time() - time0, self._mode
) )
) )
micro_bsz = (
1
if not self._strategy.pipeline.enable
else self._strategy.pipeline.micro_batch_size
)
time0 = time.time() time0 = time.time()
resharder = Resharder( resharder = Resharder(
dist_main_prog, dist_main_prog,
...@@ -162,7 +167,7 @@ class Parallelizer: ...@@ -162,7 +167,7 @@ class Parallelizer:
rank, rank,
self._dist_context, self._dist_context,
[], [],
1, micro_bsz,
) )
resharder.reshard() resharder.reshard()
self._logger.debug( self._logger.debug(
......
Markdown is supported
0% .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑!
想要评论请 注册