paddle升级到1.6.2后,对话自动评估模型训练报错,paddle1.5下正常训练
Created by: sucuicong
为使您的问题得到快速解决,在建立Issues前,请您先通过如下方式搜索是否有相似问题:【搜索issue关键字】【使用labels筛选】【官方文档】
如果您没有查询到相似问题,为快速解决您的提问,建立issue时请提供如下细节信息:
-
标题:简洁、精准概括您的问题,例如“Insufficient Memory xxx" ”
-
版本、环境信息: 1)PaddlePaddle版本:paddle1.6.2 2)CPU:预测若用CPU,请提供CPU型号,MKL/OpenBlas/MKLDNN/等数学库使用情况 3)GPU:v100 4)系统环境:请您描述系统类型、版本,例如Mac OS 10.14,Python版本
-
训练信息 1)单机/单卡 2)显存信息 3)Operator信息
-
复现信息:如为报错,请给出复现环境、复现步骤 AI studio公开项目https://aistudio.baidu.com/aistudio/projectdetail/122301,fork后执行即可
-
问题描述:请详细描述您的问题,同步贴出报错信息、日志、可复现的代码片段 paddle1.5下模型正常训练,paddle升级到1.6.2后出现如下报错 /opt/conda/envs/python27-paddle120-env/lib/python2.7/site-packages/paddle/fluid/executor.py:779: UserWarning: The following exception is not an EOF exception. "The following exception is not an EOF exception.") Traceback (most recent call last): File "main.py", line 498, in main() File "main.py", line 484, in main train(args) File "main.py", line 187, in train best_recall = save_exe(global_step, best_recall) File "main.py", line 153, in save_exe recall_dict = evaluate() File "main.py", line 144, in evaluate scores.extend(test_with_feed(batch)) File "main.py", line 129, in test_with_feed score = test_exe.run(feed=feed_dict, fetch_list=[logits.name]) File "/opt/conda/envs/python27-paddle120-env/lib/python2.7/site-packages/paddle/fluid/parallel_executor.py", line 311, in run return_numpy=return_numpy) File "/opt/conda/envs/python27-paddle120-env/lib/python2.7/site-packages/paddle/fluid/executor.py", line 780, in run six.reraise(*sys.exc_info()) File "/opt/conda/envs/python27-paddle120-env/lib/python2.7/site-packages/paddle/fluid/executor.py", line 775, in run use_program_cache=use_program_cache) File "/opt/conda/envs/python27-paddle120-env/lib/python2.7/site-packages/paddle/fluid/executor.py", line 824, in _run_impl program._compile(scope, self.place) File "/opt/conda/envs/python27-paddle120-env/lib/python2.7/site-packages/paddle/fluid/compiler.py", line 392, in _compile places=self._places) File "/opt/conda/envs/python27-paddle120-env/lib/python2.7/site-packages/paddle/fluid/compiler.py", line 286, in _compile_data_parallel "share_vars_from is not compiled and run, so there is no " ValueError: share_vars_from is not compiled and run, so there is no var to share.
Thank you for contributing to PaddlePaddle. Before submitting the issue, you could search issue in the github in case that there was a similar issue submitted or resolved before. If there is no solution,please make sure that this is a training issue including the following details: System information -PaddlePaddle version (eg.1.1)or CommitID -CPU: including CPUMKL/OpenBlas/MKLDNN version -GPU: including CUDA/CUDNN version -OS Platform (eg.Mac OS 10.14) -Other imformation: Distriuted training/informantion of operator/ Graphics card storage To Reproduce Steps to reproduce the behavior Describe your current behavior Code to reproduce the issue Other info / logs