Created by: xiegegege

在paddle1.5分支下，图像分类InceptionV4偶尔会报illegal memory access，截图如下： Pass 0, trainbatch 470, loss 6.65145, acc1 0.00391, acc5 0.01953, lr 0.10000, time 0.47 sec F0624 09:12:36.741020 50191 device_context.cc:333] cudaStreamSynchronize an illegal memory access was encountered errno:77

* Check failure stack trace: *

F0624 09:1F:36.74102662 50194 9:device_context.cc:127336cudaStreamSynchronize .741029egal memory access was encountered errno:7750180 all_reduce_op_handle.cc:73] cudaStreamSynchronize an illegal memory access was encountered

* Check failure stack trace: *

F0624 09:1F:36.74102662 50194 9:device_context.cc:127336cudaStreamSynchronize .741029egal memory access was encountered errno:7750180 all_reduce_op_handle.cc:73] cudaStreamSynchronize an illegal memory access was encountered

* Check failure stack trace: *

@ 0x7f7a737c777d google::LogMessage::Fail() @ 0x7f7a737c777d google::LogMessage::Fail() @ 0x7f7a737c777d google::LogMessage::Fail() @ 0x7f7a737cb22c google::LogMessage::SendToLog() @ 0x7f7a737cb22c google::LogMessage::SendToLog() @ 0x7f7a737cb22c google::LogMessage::SendToLog() @ 0x7f7a737c72a3 google::LogMessage::Flush() @ 0x7f7a737c72a3 google::LogMessage::Flush() @ 0x7f7a737c72a3 google::LogMessage::Flush() @ 0x7f7a737cc73e google::LogMessageFatal::~LogMessageFatal() @ 0x7f7a737cc73e google::LogMessageFatal::~LogMessageFatal() @ 0x7f7a737cc73e google::LogMessageFatal::~LogMessageFatal() @ 0x7f7a7544a2ed _ZNSt17_Function_handlerIFvvEZNK6paddle8platform17CUDADeviceContext4WaitEvEUlvE_E9_M_invokeERKSt9_Any_data