Created by: chenjiachuan

W0102 12:50:41.764370 28848 grpc_server.cc:416] completion queue:RequestSend recv no regular event context:RequestSend name:[], ep:[] Process using req_id:0 W0102 12:50:41.764364 28847 grpc_server.cc:416] completion queue:RequestSend recv no regular event context:RequestSend name:[], ep:[] Process using req_id:99 W0102 12:50:41.764354 28845 grpc_server.cc:416] completion queue:RequestSend recv no regular event context:RequestSend name:[], ep:[] Process using req_id:95 W0102 12:50:41.764375 28849 grpc_server.cc:416] completion queue:RequestSend recv no regular event context:RequestSend name:[], ep:[] Process using req_id:1 Traceback (most recent call last): File "train.py", line 747, in train(args) File "train.py", line 689, in train trainer_id) File "train.py", line 521, in train_loop feed=feed_dict_list) File "/home/slurm/job/tmp/job-72767/python/lib/python2.7/site-packages/paddle/fluid/parallel_executor.py", line 272, in run self.executor.run(fetch_list, fetch_var_name) paddle.fluid.core.EnforceNotMet: holder_ should not be null Tensor not initialized yet when Tensor::type() is called. at [/paddle/paddle/fluid/framework/tensor.h:143] PaddlePaddle Call Stacks: 0 0x7fa1eef15e86p paddle::platform::EnforceNotMet::EnforceNotMet(std::__exception_ptr::exception_ptr, char const*, int) + 486 1 0x7fa1eef18569p paddle::framework::Tensor::type() const + 137 2 0x7fa1f071a3b0p paddle::framework::GetDataTypeOfVar(paddle::framework::Variable const*) + 272 3 0x7fa1f0224839p paddle::operators::ElementwiseOp::GetExpectedKernelType(paddle::framework::ExecutionContext const&) const + 137 4 0x7fa1f071c9edp paddle::framework::OperatorWithKernel::RunImpl(paddle::framework::Scope const&, boost::variant const&) const + 237 5 0x7fa1f0718c2fp paddle::framework::OperatorBase::Run(paddle::framework::Scope const&, boost::variant const&) + 463 6 0x7fa1f06d9a37p 7 0x7fa1f06f63f0p 8 0x7fa1f06f5c55p paddle::framework::details::OpHandleBase::RunAndRecordEvent(std::function const&) + 805 9 0x7fa1f06d9530p paddle::framework::details::ComputationOpHandle::RunImpl() + 128 10 0x7fa1f06f6cf5p paddle::framework::details::OpHandleBase::Run(bool) + 117 11 0x7fa1f06badaap 12 0x7fa1eeff0233p std::_Function_handler (), std::__future_base::_Task_setter, std::__future_base::_Result_base::_Deleter>, void> >::_M_invoke(std::_Any_data const&) + 35 13 0x7fa1eefefa37p std::__future_base::_State_base::_M_do_set(std::function ()>&, bool&) + 39 14 0x7fa250b31973p pthread_once + 83 15 0x7fa1f06ba072p 16 0x7fa1eeff1d84p ThreadPool::ThreadPool(unsigned long)::{lambda()#1 (closed)}::operator()() const + 404 17 0x7fa23ec178a0p 18 0x7fa250b2c1c3p 19 0x7fa25015412dp clone + 109