写模型到本地目录的时候崩溃
Created by: fty8788
第一轮训练完成之后,写入文件,崩溃,代码是:
with gzip.open("dssm_%s_pass_%05d.tar.gz" %
(model_save_name_prefix, event.pass_id), "w") as f:
parameters.to_tar(f)
错误日志是:
[INFO 2017-07-13 15:26:48,320 train.py:199] Pass 0, Batch 4000, Cost 0.530298, {'auc_evaluator_0': 0.7309514880180359, 'classification_error_evaluator': 0.2705000042915344}
(paddle_box) (paddle_box) tail log.bigram
num_passes=num_passes)
File "/home/yanchunwei/third_party/paddle_env/python27/lib/python2.7/site-packages/paddle/v2/trainer.py", line 175, in train
event_handler(v2_event.EndPass(pass_id, evaluator=pass_evaluator))
File "train.py", line 211, in _event_handler
parameters.to_tar(f)
File "/home/yanchunwei/third_party/paddle_env/python27/lib/python2.7/site-packages/paddle/v2/parameters.py", line 274, in to_tar
self.serialize(nm, buf)
File "/home/yanchunwei/third_party/paddle_env/python27/lib/python2.7/site-packages/paddle/v2/parameters.py", line 256, in serialize
f.write(param.tostring())
OverflowError: length too large
用的DSSM模型,字典维度200w