Skip to content

E
ERNIE

PaddlePaddle / ERNIE
大约 1 年 前同步成功

通知 109
Star 5997
Fork 1270
E
ERNIE
提交 fe4e183f 编写于 作者: Y Yibing Liu
浏览文件
操作

Fix data yield

上级 68e04f38
隐藏空白更改
内联 并排
Showing with 6 addition and 4 deletion
BERT/reader/cls.py
浏览文件 @ fe4e183f
...@@ -196,10 +196,11 @@ class DataProcessor(object): ...@@ -196,10 +196,11 @@ class DataProcessor(object):
return_num_token=False) return_num_token=False)
if len(all_dev_batches) < dev_count: if len(all_dev_batches) < dev_count:
all_dev_batches.append(batch_data) all_dev_batches.append(batch_data)
else:
if len(all_dev_batches) == dev_count:
for batch in all_dev_batches: for batch in all_dev_batches:
yield batch yield batch
all_dev_batches = [batch_data] all_dev_batches = []
return wrapper return wrapper
......
BERT/reader/squad.py
浏览文件 @ fe4e183f
...@@ -566,10 +566,11 @@ class DataProcessor(object): ...@@ -566,10 +566,11 @@ class DataProcessor(object):
return_num_token=False) return_num_token=False)
if len(all_dev_batches) < dev_count: if len(all_dev_batches) < dev_count:
all_dev_batches.append(batch_data) all_dev_batches.append(batch_data)
else:
if len(all_dev_batches) == dev_count:
for batch in all_dev_batches: for batch in all_dev_batches:
yield batch yield batch
all_dev_batches = [batch_data] all_dev_batches = []
return wrapper return wrapper
......
Markdown is supported
0% 或 .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑！
想要评论请 注册 或