Error: /paddle/paddle/fluid/operators/elementwise/elementwise_op_function.cu.h:88 Assertion `y_[id] != 0` failed. InvalidArgumentError: Integer division by zero encountered in divide.Please check.
Created by: Termset
我改动过的IMPALA算法,运行出现问题。不知道怎么定位到这个问题。
[03-02 14:47:41 MainThread @machine_info.py:86] nvidia-smi -L found gpu count: 1
[03-02 14:47:41 MainThread @machine_info.py:86] nvidia-smi -L found gpu count: 1
[03-02 14:47:41 MainThread @train.py:154] Waiting for 1 remote actors to connect.
[03-02 14:47:41 MainThread @train.py:158] Remote actor count: 1
Error: /paddle/paddle/fluid/operators/elementwise/elementwise_op_function.cu.h:88 Assertion `y_[id] != 0` failed. InvalidArgumentError: Integer division by zero encountered in divide.Please check.
Error: /paddle/paddle/fluid/operators/elementwise/elementwise_op_function.cu.h:88 Assertion `y_[id] != 0` failed. InvalidArgumentError: Integer division by zero encountered in divide.Please check.
Error: /paddle/paddle/fluid/operators/elementwise/elementwise_op_function.cu.h:88 Assertion `y_[id] != 0` failed. InvalidArgumentError: Integer division by zero encountered in divide.Please check.
Error: /paddle/paddle/fluid/operators/elementwise/elementwise_op_function.cu.h:88 Assertion `y_[id] != 0` failed. InvalidArgumentError: Integer division by zero encountered in divide.Please check.
Error: /paddle/paddle/fluid/operators/elementwise/elementwise_op_function.cu.h:88 Assertion `y_[id] != 0` failed. InvalidArgumentError: Integer division by zero encountered in divide.Please check.
......
Error: /paddle/paddle/fluid/operators/elementwise/elementwise_op_function.cu.h:88 Assertion `y_[id] != 0` failed. InvalidArgumentError: Integer division by zero encountered in divide.Please check.
F0302 14:47:44.609232 23176 device_context.cc:328] cudaStreamSynchronize unspecified launch failure errno: 4
*** Check failure stack trace: ***
@ 0x7f1e600d138d google::LogMessage::Fail()
@ 0x7f1e600d4e3c google::LogMessage::SendToLog()
@ 0x7f1e600d0eb3 google::LogMessage::Flush()
@ 0x7f1e600d634e google::LogMessageFatal::~LogMessageFatal()
@ 0x7f1e629ee8f7 paddle::platform::CUDADeviceContext::Wait()
@ 0x7f1e601184fb paddle::framework::Executor::RunPreparedContext()
@ 0x7f1e60a5c312 paddle::operators::RecurrentOp::RunImpl()
@ 0x7f1e6295783c paddle::framework::OperatorBase::Run()
@ 0x7f1e601184c6 paddle::framework::Executor::RunPreparedContext()
@ 0x7f1e6011bd0f paddle::framework::Executor::Run()
@ 0x7f1e5ff5609d _ZZN8pybind1112cpp_function10initializeIZN6paddle6pybindL22pybind11_init_core_avxERNS_6moduleEEUlRNS2_9framework8ExecutorERKNS6_11ProgramDescEPNS6_5ScopeEibbRKSt6vectorISsSaISsEEE103_vIS8_SB_SD_ibbSI_EINS_4nameENS_9is_methodENS_7siblingEEEEvOT_PFT0_DpT1_EDpRKT2_ENUlRNS_6detail13function_callEE1_4_FUNES10_
@ 0x7f1e5ff9edd1 pybind11::cpp_function::dispatcher()
@ 0x7f1eb6d6a302 _PyCFunction_FastCallDict
@ 0x7f1eb6def95b call_function
@ 0x7f1eb6df2d40 _PyEval_EvalFrameDefault
@ 0x7f1eb6dee100 _PyEval_EvalCodeWithName
@ 0x7f1eb6defb2a call_function
@ 0x7f1eb6df32cc _PyEval_EvalFrameDefault
@ 0x7f1eb6dee100 _PyEval_EvalCodeWithName
@ 0x7f1eb6defb2a call_function
@ 0x7f1eb6df32cc _PyEval_EvalFrameDefault
@ 0x7f1eb6dee100 _PyEval_EvalCodeWithName
@ 0x7f1eb6defb2a call_function
@ 0x7f1eb6df32cc _PyEval_EvalFrameDefault
@ 0x7f1eb6ded514 _PyFunction_FastCall
@ 0x7f1eb6defc88 call_function
@ 0x7f1eb6df2d40 _PyEval_EvalFrameDefault
@ 0x7f1eb6ded514 _PyFunction_FastCall
@ 0x7f1eb6dee515 _PyFunction_FastCallDict
@ 0x7f1eb6d12ce6 _PyObject_FastCallDict
@ 0x7f1eb6d12f3c _PyObject_Call_Prepend
@ 0x7f1eb6d12fd6 PyObject_Call
已放弃