OCR End-to-end Training
Created by: Yipeng-Sun
- 版本、环境信息: 1)PaddlePaddle版本:Fluid 1.2 3)GPU:CUDA8.0 CUDNNv5 4)系统环境:Python2.7
- 训练信息 1)单机,单卡
- 复现信息:end-to-end training, 迭代若干次报错
- 问题描述: 1547704800 casted_label Tensor[cast_10.tmp_0] shape: [14,1,] dtype: l LoD: [[ 0,4,8,14, ]] data: 66,68,85,1,88,66,84,1,77,70, Traceback (most recent call last): File "./train_end2end.py", line 184, in main() File "./train_end2end.py", line 180, in main train(args) File "./train_end2end.py", line 150, in train recog_loss]) File "/home/users/sunyipeng/paddle-fluid/venv/lib/python2.7/site-packages/paddle/fluid/executor.py", line 470, in run self.executor.run(program.desc, scope, 0, True, True) paddle.fluid.core.EnforceNotMet: an illegal memory access was encountered at [/home/users/sunyipeng/paddle-fluid/Paddle-ocr-end2end/Paddle/paddle/fluid/platform/device_context.cc:230] PaddlePaddle Call Stacks: 0 0x7fee8803cb76p paddle::platform::EnforceNotMet::EnforceNotMet(std::__exception_ptr::exception_ptr, char const*, int) + 486 1 0x7fee89565532p paddle::platform::CUDADeviceContext::Wait() const + 258