提交 db831ae7 编写于 作者: H huangyuxin

fix the text_featurize

上级 cb49aba6
...@@ -118,8 +118,10 @@ class TextFeaturizer(): ...@@ -118,8 +118,10 @@ class TextFeaturizer():
""" """
text = text.strip() text = text.strip()
if replace_space: if replace_space:
text = text.replace(" ", SPACE) text_list = [SPACE if item == " " else item for item in list(text)]
return list(text) else:
text_list = list(text)
return text_list
def char_detokenize(self, tokens): def char_detokenize(self, tokens):
"""Character detokenizer. """Character detokenizer.
......
Markdown is supported
0% .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑!
想要评论请 注册