paddlev2 CPU集群训练,报错Cuda Error: CUDA driver version is insufficient for CUDA runtime version
Created by: shiyazhou121
在mip-paddle集群使用CPU训练,结果报错
++ /home/disk1/normandy/maybach/app-user-20181012100630-27056/workspace/python27-gcc482//bin/python train.py
I1012 10:13:03.775209 22097 Util.cpp:166] commandline: --ports_num_for_sparse=1 --use_gpu=0 --trainer_id=0 --pservers=10.73.51.35,10.73.51.20,10.73.43.47,10.73.105.44,10.73.36.34,10.73.105.15,10.75.67.25,10.73.36.46,10.73.105.51,10.73.36.54 --trainer_count=1 --num_gradient_servers=10 --ports_num=1 --port=8000
I1012 10:13:04.378612 22097 GradientMachine.cpp:94] Initing parameters..
I1012 10:13:09.229614 22097 GradientMachine.cpp:101] Init parameters done.
F1012 10:13:49.429853 22097 hl_cuda_device.cc:565] Check failed: cudaSuccess == cudaStat (0 vs. 35) Cuda Error: CUDA driver version is insufficient for CUDA runtime version
* Check failure stack trace: *
@ 0x7f9681798f5d google::LogMessage::Fail()
@ 0x7f968179ca0c google::LogMessage::SendToLog()
@ 0x7f9681798a83 google::LogMessage::Flush()
@ 0x7f968179df1e google::LogMessageFatal::~LogMessageFatal()
@ 0x7f9681753b23 hl_stream_synchronize()
@ 0x7f96813fb57e paddle::LstmLayer::forwardBatch()
@ 0x7f96813fccd2 paddle::LstmLayer::forward()
@ 0x7f9681499b1d paddle::NeuralNetwork::forward()
@ 0x7f968149a813 paddle::GradientMachine::forwardBackward()
@ 0x7f9681774d84 GradientMachine::forwardBackward()
@ 0x7f9681344f69 _wrap_GradientMachine_forwardBackward
@ 0x4b4cb9 PyEval_EvalFrameEx
@ 0x4b6b28 PyEval_EvalCodeEx
@ 0x4b5d10 PyEval_EvalFrameEx
@ 0x4b6b28 PyEval_EvalCodeEx
@ 0x4b5d10 PyEval_EvalFrameEx
@ 0x4b6b28 PyEval_EvalCodeEx
@ 0x4b5d10 PyEval_EvalFrameEx
@ 0x4b6b28 PyEval_EvalCodeEx
@ 0x4b5d10 PyEval_EvalFrameEx
@ 0x4b6b28 PyEval_EvalCodeEx
@ 0x4b6c52 PyEval_EvalCode
@ 0x4e1c7d PyRun_FileExFlags
@ 0x4e3501 PyRun_SimpleFileExFlags
@ 0x4159dd Py_Main
@ 0x7f96e0ff2bd5 __libc_start_main
@ 0x414b71 (unknown)
@ (nil) (unknown)
求帮忙看下:http://10.73.51.35:8900/fileview.html?path=/home/disk1/normandy/maybach/app-user-20181012100630-27056/