elementwise_add:parallel_for failed: invalid configuration argument
Created by: xygaoyuan
请问下,这个错误一般是什么问题造成的? 目前是在我这边的预测程序里面出错(预测run的时候),并且和我的预测数据有关系,一个测试集共9000多个样本一起预测没问题,单独预测某一条就会报这个错误。
错误信息如下: W0401 22:25:03.523263 76374 operator.cc:181] elementwise_add raises an exception thrust::system::system_error, parallel_for failed: invalid configuration argument /home/users/gaoyuan09/jumbo.gpu/lib/python2.7/site-packages/paddle/fluid/executor.py:773: UserWarning: The following exception is not an EOF exception. "The following exception is not an EOF exception.") Traceback (most recent call last): File "ernie/run_sequence_labeling.py", line 489, in main(args) File "ernie/run_sequence_labeling.py", line 335, in main steps) File "ernie/run_sequence_labeling.py", line 407, in predict_wrapper res = predict(exe, test_prog, test_pyreader, graph_vars, dev_count=1) File "/home/users/gaoyuan09/bert/ERNIE-develop.multi-task.knowledge.add_subword_mean_pool.routine.add_multi_head_attention.cand_use_multi_head_attention.gated_knowledge_fusion.ontonotes.add_label_predict_by_mem.bert.multitask/ernie/finetune/sequence_label.py", line 696, in predict return_numpy=False) File "/home/users/gaoyuan09/jumbo.gpu/lib/python2.7/site-packages/paddle/fluid/executor.py", line 774, in run six.reraise(*sys.exc_info()) File "/home/users/gaoyuan09/jumbo.gpu/lib/python2.7/site-packages/paddle/fluid/executor.py", line 769, in run use_program_cache=use_program_cache) File "/home/users/gaoyuan09/jumbo.gpu/lib/python2.7/site-packages/paddle/fluid/executor.py", line 816, in _run_impl use_program_cache=use_program_cache) File "/home/users/gaoyuan09/jumbo.gpu/lib/python2.7/site-packages/paddle/fluid/executor.py", line 890, in run_program fetch_var_name) RuntimeError: parallel_for failed: invalid configuration argument F0401 22:25:04.606678 76374 device_context.cc:334] cudaGetLastError invalid configuration argument errno: 9