预测时内存一直增长,直到溢出
Created by: Black-cup
使用的模型是ernie,模型已经训练好了,按照text-classification的demo写了一个预测demo,通过这个函数得到cls_task,之后for循环cls_task.predict(data=data),循环预测每个句子,但是执行之后发现内存持续在增长,不预测就不会增长,请问这是哪里出了问题呢? def loading_neg(module): dataset_neg = ChnSentiCorp_neg() inputs_neg, outputs_neg, program_neg = module.context( trainable=True, max_seq_len=128) reader = hub.reader.ClassifyReader( dataset=dataset_neg, vocab_path=module.get_vocab_path(), max_seq_len=128, use_task_id=False)
pooled_output = outputs_neg["pooled_output"]
# Setup feed list for data feeder
# Must feed all the tensor of ERNIE's module need
feed_list = [
inputs_neg["input_ids"].name,
inputs_neg["position_ids"].name,
inputs_neg["segment_ids"].name,
inputs_neg["input_mask"].name,
]
neg_config = hub.RunConfig(
use_data_parallel=False,
use_pyreader=True,
use_cuda=False,
batch_size=1,
enable_memory_optim=False,
checkpoint_dir=checkpoint_dir_neg,
strategy=hub.finetune.strategy.DefaultFinetuneStrategy())
neg_cls_task = hub.TextClassifierTask(
data_reader=reader,
feature=pooled_output,
feed_list=feed_list,
num_classes=dataset_neg.num_labels,
config=neg_config)
return neg_cls_task