Created by: zhengzhe97

Paddle版本：1.7.1 训练环境：Aistudio 迭代几次后出现如下error

2020-03-31 09:23:15,855-INFO: iter: 0, lr: 0.000050, 'B': '3.100250', 'M': '5.603091', 'C': '28.284410', 'S': '58.862827', 'I': '1.536018', 'loss': '97.386597', time: 0.000, eta: 0:00:06

2020-03-31 09:24:31,154-INFO: iter: 20, lr: 0.000059, 'B': '3.221717', 'M': '5.426755', 'C': '17.999424', 'S': '2.390736', 'I': '1.326720', 'loss': '30.490068', time: 3.780, eta: 3 days, 22:28:02
W0331 09:24:36.084980   571 operator.cc:181] cast raises an exception thrust::system::system_error, parallel_for failed: invalid configuration argument
F0331 09:24:36.085136   571 exception_holder.h:37] std::exception caught, parallel_for failed: invalid configuration argument
*** Check failure stack trace: ***
W0331 09:24:36.087306   572 operator.cc:181] elementwise_add raises an exception thrust::system::system_error, parallel_for failed: invalid configuration argument
F0331 09:24:36.087427   572 exception_holder.h:37] std::exception caught, parallel_for failed: invalid configuration argument
*** Check failure stack trace: ***
    @     0x7fd497419fed  google::LogMessage::Fail()
    @     0x7fd497419fed  google::LogMessage::Fail()
    @     0x7fd49741da9c  google::LogMessage::SendToLog()
    @     0x7fd49741da9c  google::LogMessage::SendToLog()
    @     0x7fd497419b13  google::LogMessage::Flush()
    @     0x7fd497419b13  google::LogMessage::Flush()
    @     0x7fd49741efae  google::LogMessageFatal::~LogMessageFatal()
    @     0x7fd49741efae  google::LogMessageFatal::~LogMessageFatal()
    @     0x7fd4998e3748  paddle::framework::details::ExceptionHolder::Catch()
    @     0x7fd4998e3748  paddle::framework::details::ExceptionHolder::Catch()
    @     0x7fd49998eb6e  paddle::framework::details::FastThreadedSSAGraphExecutor::RunOpSync()
    @     0x7fd49998eb6e  paddle::framework::details::FastThreadedSSAGraphExecutor::RunOpSync()
    @     0x7fd49998d77f  paddle::framework::details::FastThreadedSSAGraphExecutor::RunOp()
    @     0x7fd49998d77f  paddle::framework::details::FastThreadedSSAGraphExecutor::RunOp()
    @     0x7fd49998da44  _ZNSt17_Function_handlerIFvvESt17reference_wrapperISt12_Bind_simpleIFS1_ISt5_BindIFZN6paddle9framework7details28FastThreadedSSAGraphExecutor10RunOpAsyncEPSt13unordered_mapIPNS6_12OpHandleBaseESt6atomicIiESt4hashISA_ESt8equal_toISA_ESaISt4pairIKSA_SC_EEESA_RKSt10shared_ptrINS5_13BlockingQueueImEEEEUlvE_vEEEvEEEE9_M_invokeERKSt9_Any_data
    @     0x7fd49998da44  _ZNSt17_Function_handlerIFvvESt17reference_wrapperISt12_Bind_simpleIFS1_ISt5_BindIFZN6paddle9framework7details28FastThreadedSSAGraphExecutor10RunOpAsyncEPSt13unordered_mapIPNS6_12OpHandleBaseESt6atomicIiESt4hashISA_ESt8equal_toISA_ESaISt4pairIKSA_SC_EEESA_RKSt10shared_ptrINS5_13BlockingQueueImEEEEUlvE_vEEEvEEEE9_M_invokeERKSt9_Any_data
    @     0x7fd497472d43  std::_Function_handler<>::_M_invoke()
    @     0x7fd497472d43  std::_Function_handler<>::_M_invoke()
    @     0x7fd497202537  std::__future_base::_State_base::_M_do_set()
    @     0x7fd497202537  std::__future_base::_State_base::_M_do_set()
    @     0x7fd4fd980a99  __pthread_once_slow
    @     0x7fd4fd980a99  __pthread_once_slow
    @     0x7fd499988f32  _ZNSt13__future_base11_Task_stateISt5_BindIFZN6paddle9framework7details28FastThreadedSSAGraphExecutor10RunOpAsyncEPSt13unordered_mapIPNS4_12OpHandleBaseESt6atomicIiESt4hashIS8_ESt8equal_toIS8_ESaISt4pairIKS8_SA_EEES8_RKSt10shared_ptrINS3_13BlockingQueueImEEEEUlvE_vEESaIiEFvvEE6_M_runEv
    @     0x7fd499988f32  _ZNSt13__future_base11_Task_stateISt5_BindIFZN6paddle9framework7details28FastThreadedSSAGraphExecutor10RunOpAsyncEPSt13unordered_mapIPNS4_12OpHandleBaseESt6atomicIiESt4hashIS8_ESt8equal_toIS8_ESaISt4pairIKS8_SA_EEES8_RKSt10shared_ptrINS3_13BlockingQueueImEEEEUlvE_vEESaIiEFvvEE6_M_runEv
    @     0x7fd497204764  _ZZN10ThreadPoolC1EmENKUlvE_clEv
    @     0x7fd497204764  _ZZN10ThreadPoolC1EmENKUlvE_clEv
    @     0x7fd4be3b0421  execute_native_thread_routine_compat
    @     0x7fd4be3b0421  execute_native_thread_routine_compat
    @     0x7fd4fd9796ba  start_thread
    @     0x7fd4fd9796ba  start_thread
    @     0x7fd4fd6af41d  clone
    @     0x7fd4fd6af41d  clone
    @              (nil)  (unknown)
    @              (nil)  (unknown)
Aborted (core dumped)