执行脚本参数iteration 超过recordio文件提供的batch数,出core
Created by: ccmeteorljh
pr #11121 下单机8卡训练,启动脚本如下: python fluid_benchmark.py --model resnet --data_set flowers --use_reader_op --memory_optimize --iterations 20 --device GPU --gpus 8 --batch_size 64 --pass_num 1 --data_path recordio --no_test 报错如下: *** Aborted at 1528350059 (unix time) try "date -d @1528350059" if you are using GNU date *** PC: @ 0x0 (unknown) *** SIGSEGV (@0x18529) received by PID 21665 (TID 0x7f653f5fe700) from PID 99625; stack trace: *** @ 0x7f6604e98390 (unknown) @ 0x7f65abf19fdf std::_Sp_counted_base<>::_M_release() @ 0x7f65abfcd726 std::vector<>::operator=() @ 0x7f65ac026260 paddle::operators::reader::BlockingQueue<>::Receive() @ 0x7f65ac020f81 paddle::operators::reader::MultiFileReader::ReadNext() @ 0x7f65ac03c959 paddle::operators::reader::ThreadedReader::ReadNext() @ 0x7f65ac0301c4 paddle::operators::reader::BatchReader::ReadNext() @ 0x7f65ac037778 paddle::operators::reader::DoubleBufferReader::PrefetchThreadFunc() @ 0x7f65fd4e27e0 execute_native_thread_routine @ 0x7f6604e8e6ba start_thread @ 0x7f6604bc441d clone @ 0x0 (unknown) Segmentation fault 复现情况: 只要iterations 参数的值 > recordio 文件提供的能跑得最大batch数就会稳定复现,单机和多机都是如此