过一个pass就停止不动了,内存占用着,但是cpu和gpu都为0了,就一直不动了
Created by: bjkite
经确认,是在reader的实现中使用了 paddle.reader.xmap_readers的原因,去掉不用就不会出现 代码如下:
def reader():
UNK = word_id_dict['<UNK>']
with open(file_name) as file:
count = 0
for line in file:
image_path,image_tag = line.strip().split("\t")
words = image_tag.decode('utf-8', 'ignore').strip().split()
if len(words) < min_sentence_length or len(
words) > max_sentence_length:
continue
ids = [word_id_dict.get(w, UNK) for w in words]
ids.append(word_id_dict['<EOS>'])
target = ids[:]
target.append(word_id_dict['<EOS>'])
count += 1
yield image_path, ids[:], target[:]
return paddle.reader.xmap_readers(image_mapper, reader,
1, buffered_size)