Created by: mc261670164

在自己的数据集上训练ppyolo时提前终止了，报错如下： W0907 11:18:28.866688 4224 init.cc:226] Warning: PaddlePaddle catches a failure signal, it may not work properly W0907 11:18:28.866739 4224 init.cc:228] You could check whether you killed PaddlePaddle thread/process accidentally or report the case to PaddlePaddle W0907 11:18:28.866745 4224 init.cc:231] The detail failure signal is:

W0907 11:18:28.866752 4224 init.cc:234] *** Aborted at 1599448708 (unix time) try "date -d @1599448708" if you are using GNU date *** W0907 11:18:28.869943 4224 init.cc:234] PC: @ 0x0 (unknown) W0907 11:18:28.872963 4224 init.cc:234] *** SIGSEGV (@0x7f7acc3c70bc) received by PID 4160 (TID 0x7f7d9a482700) from PID 18446744072841097404; stack trace: *** W0907 11:18:28.874167 4224 init.cc:234] @ 0x7f7f7e64b8a0 (unknown) W0907 11:18:28.881690 4224 init.cc:234] @ 0x7f7f6082a695 DmcnIm2colBilinear<>() W0907 11:18:28.884872 4224 init.cc:234] @ 0x7f7f6082d38b paddle::operators::ModulatedDeformableCol2imCoordCPUKernel<>() W0907 11:18:28.886778 4224 init.cc:234] @ 0x7f7f6082f873 paddle::operators::DeformableConvGradCPUKernel<>::Compute() W0907 11:18:28.889729 4224 init.cc:234] @ 0x7f7f60830453 ZNSt17_Function_handlerIFvRKN6paddle9framework16ExecutionContextEEZNKS1_24OpKernelRegistrarFunctorINS0_8platform8CPUPlaceELb0ELm0EJNS0_9operators27DeformableConvGradCPUKernelIfEENSA_IdEEEEclEPKcSF_iEUlS4_E_E9_M_invokeERKSt9_Any_dataS4 W0907 11:18:28.892365 4224 init.cc:234] @ 0x7f7f61a381b0 paddle::framework::OperatorWithKernel::RunImpl() W0907 11:18:28.896667 4224 init.cc:234] @ 0x7f7f61a38d71 paddle::framework::OperatorWithKernel::RunImpl() W0907 11:18:28.899250 4224 init.cc:234] @ 0x7f7f61a33ce1 paddle::framework::OperatorBase::Run() W0907 11:18:28.903023 4224 init.cc:234] @ 0x7f7f6176729b paddle::framework::details::ComputationOpHandle::RunImpl() W0907 11:18:28.907918 4224 init.cc:234] @ 0x7f7f6177fc5c paddle::framework::details::OpHandleBase::Run() W0907 11:18:28.910671 4224 init.cc:234] @ 0x7f7f6170df91 paddle::framework::details::FastThreadedSSAGraphExecutor::RunOpSync() W0907 11:18:28.913704 4224 init.cc:234] @ 0x7f7f6170ba8f paddle::framework::details::FastThreadedSSAGraphExecutor::RunOp() W0907 11:18:28.914446 4224 init.cc:234] @ 0x7f7f6170bd54 _ZNSt17_Function_handlerIFvvESt17reference_wrapperISt12_Bind_simpleIFS1_ISt5_BindIFZN6paddle9framework7details28FastThreadedSSAGraphExecutor10RunOpAsyncEPSt13unordered_mapIPNS6_12OpHandleBaseESt6atomicIiESt4hashISA_ESt8equal_toISA_ESaISt4pairIKSA_SC_EEESA_RKSt10shared_ptrINS5_13BlockingQueueImEEEEUlvE_vEEEvEEEE9_M_invokeERKSt9_Any_data W0907 11:18:28.918684 4224 init.cc:234] @ 0x7f7f5fbd3653 std::_Function_handler<>::_M_invoke() W0907 11:18:28.923187 4224 init.cc:234] @ 0x7f7f5f9da6a7 std::__future_base::_State_base::_M_do_set() W0907 11:18:28.923985 4224 init.cc:234] @ 0x7f7f7e648827 __pthread_once_slow W0907 11:18:28.924475 4224 init.cc:234] @ 0x7f7f61708432 _ZNSt13__future_base11_Task_stateISt5_BindIFZN6paddle9framework7details28FastThreadedSSAGraphExecutor10RunOpAsyncEPSt13unordered_mapIPNS4_12OpHandleBaseESt6atomicIiESt4hashIS8_ESt8equal_toIS8_ESaISt4pairIKS8_SA_EEES8_RKSt10shared_ptrINS3_13BlockingQueueImEEEEUlvE_vEESaIiEFvvEE6_M_runEv W0907 11:18:28.928139 4224 init.cc:234] @ 0x7f7f5f9dcb04 _ZZN10ThreadPoolC1EmENKUlvE_clEv W0907 11:18:28.929417 4224 init.cc:234] @ 0x7f7f6f9b7421 execute_native_thread_routine_compat W0907 11:18:28.930181 4224 init.cc:234] @ 0x7f7f7e6406db start_thread W0907 11:18:28.931035 4224 init.cc:234] @ 0x7f7f7e369a3f clone W0907 11:18:28.931797 4224 init.cc:234] @ 0x0 (unknown) Segmentation fault (core dumped)

配置文件如下： architecture: YOLOv3 use_gpu: false max_iters: 15000 #20000 log_smooth_window: 20 log_iter: 10 save_dir: save_models snapshot_iter: 200 metric: VOC pretrain_weights: https://paddle-imagenet-models-name.bj.bcebos.com/ResNet50_vd_ssld_pretrained.tar weights: output/ppyolo_cartoon/model_final num_classes: 1 use_fine_grained_loss: true use_ema: true ema_decay: 0.9998

YOLOv3: backbone: ResNet yolo_head: YOLOv3Head use_fine_grained_loss: true

ResNet: norm_type: sync_bn freeze_at: 0 freeze_norm: false norm_decay: 0. depth: 50 feature_maps: [3, 4, 5] variant: d dcn_v2_stages: [5]

YOLOv3Head: anchor_masks: [[6, 7, 8], [3, 4, 5], [0, 1, 2]] anchors: [[10, 13], [16, 30], [33, 23], [30, 61], [62, 45], [59, 119], [116, 90], [156, 198], [373, 326]] norm_decay: 0. coord_conv: true iou_aware: true iou_aware_factor: 0.4 scale_x_y: 1.05 spp: true yolo_loss: YOLOv3Loss nms: MatrixNMS drop_block: true

YOLOv3Loss: batch_size: 4 ignore_thresh: 0.7 scale_x_y: 1.05 label_smooth: false use_fine_grained_loss: true iou_loss: IouLoss iou_aware_loss: IouAwareLoss

IouLoss: loss_weight: 2.5 max_height: 608 max_width: 608

IouAwareLoss: loss_weight: 1.0 max_height: 608 max_width: 608

MatrixNMS: background_label: -1 keep_top_k: 100 normalized: false score_threshold: 0.01 post_threshold: 0.01

LearningRate: base_lr: 0.001 schedulers:

• !PiecewiseDecay gamma: 0.1 milestones:
  • 10000
  • 12000
• !LinearWarmup start_factor: 0. steps: 100

OptimizerBuilder: optimizer: momentum: 0.9 type: Momentum regularizer: factor: 0.0005 type: L2

TrainReader: inputs_def: fields: ['image', 'gt_bbox', 'gt_class', 'gt_score'] num_max_boxes: 50 dataset: !VOCDataSet anno_path: train.txt dataset_dir: dataset/cartoon_face use_default_label: false with_background: false sample_transforms: - !DecodeImage to_rgb: True with_mixup: True - !MixupImage alpha: 1.5 beta: 1.5 - !ColorDistort {} - !RandomExpand fill_value: [123.675, 116.28, 103.53] - !RandomCrop {} - !RandomFlipImage is_normalized: false - !NormalizeBox {} - !PadBox num_max_boxes: 50 - !BboxXYXY2XYWH {} batch_transforms:

• !RandomShape sizes: [320, 352, 384, 416, 448, 480, 512, 544, 576, 608] random_inter: True
• !NormalizeImage mean: [0.485, 0.456, 0.406] std: [0.229, 0.224, 0.225] is_scale: True is_channel_first: false
• !Permute to_bgr: false channel_first: True
• !Gt2YoloTarget anchor_masks: [[6, 7, 8], [3, 4, 5], [0, 1, 2]] anchors: [[10, 13], [16, 30], [33, 23], [30, 61], [62, 45], [59, 119], [116, 90], [156, 198], [373, 326]] downsample_ratios: [32, 16, 8] batch_size: 4 shuffle: true mixup_epoch: 2500 drop_last: true worker_num: 8 bufsize: 4 use_process: true

EvalReader: inputs_def: fields: ['image', 'im_size', 'im_id', 'gt_bbox', 'gt_class', 'is_difficult'] num_max_boxes: 50 dataset: !VOCDataSet anno_path: test.txt dataset_dir: dataset/cartoon_face use_default_label: false with_background: false sample_transforms: - !DecodeImage to_rgb: True - !ResizeImage target_size: 608 interp: 2 - !NormalizeImage mean: [0.485, 0.456, 0.406] std: [0.229, 0.224, 0.225] is_scale: True is_channel_first: false - !PadBox num_max_boxes: 50 - !Permute to_bgr: false channel_first: True batch_size: 4 drop_empty: false worker_num: 8 bufsize: 4

TestReader: inputs_def: image_shape: [3, 608, 608] fields: ['image', 'im_size', 'im_id'] dataset: !ImageFolder anno_path: dataset/cartoon_face/label_list.txt with_background: false sample_transforms: - !DecodeImage to_rgb: True - !ResizeImage target_size: 608 interp: 2 - !NormalizeImage mean: [0.485, 0.456, 0.406] std: [0.229, 0.224, 0.225] is_scale: True is_channel_first: false - !Permute to_bgr: false channel_first: True batch_size: 1

有没有大佬遇到过？