Created by: miraclebiu

为使您的问题得到快速解决，在建立Issues前，请您先通过如下方式搜索是否有相似问题:【搜索issue关键字】【使用labels筛选】【官方文档】

如果您没有查询到相似问题，为快速解决您的提问，建立issue时请提供如下细节信息：

• 标题：简洁、精准概括您的问题，例如“Insufficient Memory xxx" ”
• 版本、环境信息：    1）PaddlePaddle版本：1.5.0 post87    2）CPU：    3）GPU：k40    4）系统环境：centos
• 训练信息    1）单机，多卡    2）显存信息    3）Operator信息
• 复现信息：如为报错，请给出复现环境、复现步骤
• 问题描述：请详细描述您的问题，同步贴出报错信息、日志、可复现的代码片段 cudaStreamSynchronize an illegal memory access was encountered errno:77 *** Check failure stack trace: *** @ 0x7fd3393026ad google::LogMessage::Fail() @ 0x7fd33930615c google::LogMessage::SendToLog() @ 0x7fd3393021d3 google::LogMessage::Flush() @ 0x7fd33930766e google::LogMessageFatal::~LogMessageFatal() @ 0x7fd33b2e001d _ZNSt17_Function_handlerIFvvEZNK6paddle8platform17CUDADeviceContext4WaitEvEUlvE_E9_M_invokeERKSt9_Any_data @ 0x7fd33b2edaa4 paddle::platform::TemporaryAllocator::Release() @ 0x7fd33b2e2ff1 paddle::platform::CUDADeviceContext::Wait() @ 0x7fd33b239ad1 paddle::framework::TransDataDevice() @ 0x7fd33b238b6e paddle::framework::TransformData() @ 0x7fd33b22ffad paddle::framework::OperatorWithKernel::PrepareData() @ 0x7fd33b2310dd paddle::framework::OperatorWithKernel::RunImpl() @ 0x7fd33b231581 paddle::framework::OperatorWithKernel::RunImpl() @ 0x7fd33b22eb7c paddle::framework::OperatorBase::Run() @ 0x7fd33b02be6a paddle::framework::details::ComputationOpHandle::RunImpl() @ 0x7fd33b01e810 paddle::framework::details::OpHandleBase::Run() @ 0x7fd33afffb86 paddle::framework::details::FastThreadedSSAGraphExecutor::RunOpSync() @ 0x7fd33affe7ef paddle::framework::details::FastThreadedSSAGraphExecutor::RunOp() @ 0x7fd33affebaf _ZNSt17_Function_handlerIFvvESt17reference_wrapperISt12_Bind_simpleIFS1_ISt5_BindIFZN6paddle9framework7details28FastThreadedSSAGraphExecutor10RunOpAsyncEPSt13unordered_mapIPNS6_12OpHandleBaseESt6atomicIiESt4hashISA_ESt8equal_toISA_ESaISt4pairIKSA_SC_EEESA_RKSt10shared_ptrINS5_13BlockingQueueImEEEEUlvE_vEEEvEEEE9_M_invokeERKSt9_Any_data @ 0x7fd3393ef2b3 std::_Function_handler<>::_M_invoke() @ 0x7fd339285ef7 std::__future_base::_State_base::_M_do_set() @ 0x38c040cbe0 (unknown) @ 0x7fd33affa232 _ZNSt13__future_base11_Task_stateISt5_BindIFZN6paddle9framework7details28FastThreadedSSAGraphExecutor10RunOpAsyncEPSt13unordered_mapIPNS4_12OpHandleBaseESt6atomicIiESt4hashIS8_ESt8equal_toIS8_ESaISt4pairIKS8_SA_EEES8_RKSt10shared_ptrINS3_13BlockingQueueImEEEEUlvE_vEESaIiEFvvEE6_M_runEv @ 0x7fd339287474 _ZZN10ThreadPoolC1EmENKUlvE_clEv @ 0x7fd37483d678 execute_native_thread_routine_compat @ 0x38c0407df3 (unknown) @ 0x38bfcf62cd (unknown) @ (nil) (unknown) ^[[A^C [1]+ Aborted nohup python train_gpl.py > exp1.out