未验证 提交 04689f52 编写于 作者: J Jiabin Yang 提交者: GitHub

Merge pull request #14090 from JiabinYang/fix_test_parallel_executor_transformer

[1.1] fix python encoding problem in wmt16 related test
...@@ -78,7 +78,7 @@ def __build_dict(tar_file, dict_size, save_path, lang): ...@@ -78,7 +78,7 @@ def __build_dict(tar_file, dict_size, save_path, lang):
six.iteritems(word_dict), key=lambda x: x[1], six.iteritems(word_dict), key=lambda x: x[1],
reverse=True)): reverse=True)):
if idx + 3 == dict_size: break if idx + 3 == dict_size: break
fout.write("%s\n" % (word[0])) fout.write("%s\n" % (cpt.to_bytes(word[0])))
def __load_dict(tar_file, dict_size, lang, reverse=False): def __load_dict(tar_file, dict_size, lang, reverse=False):
......
Markdown is supported
0% .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑!
想要评论请 注册