提交 9baf7fc4 编写于 作者: P Peng Li

Fix data provider bug in srl demo

Once encoutering a single sample whose size is larger than batch size,
the provider will yield empty batch and terminate the current pass unexpectedly
if can_over_batch_size=False.
上级 b53bdcdc
...@@ -43,7 +43,7 @@ def get_batch_size(yeild_data): ...@@ -43,7 +43,7 @@ def get_batch_size(yeild_data):
init_hook=hook, init_hook=hook,
should_shuffle=True, should_shuffle=True,
calc_batch_size=get_batch_size, calc_batch_size=get_batch_size,
can_over_batch_size=False, can_over_batch_size=True,
cache=CacheType.CACHE_PASS_IN_MEM) cache=CacheType.CACHE_PASS_IN_MEM)
def process(settings, file_name): def process(settings, file_name):
with open(file_name, 'r') as fdata: with open(file_name, 'r') as fdata:
......
Markdown is supported
0% .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑!
想要评论请 注册