使用自己的数据集训练yolov3增强模型 最后loss不下降
Created by: llzhaoshuo
命令: python -u tools/train.py -c configs/dcn/yolov3_r50vd_dcn_db_iouaware_obj365_pretrained_coco.yml -o pretrain_weights=/home/ResNet50_vd_dcn_db_obj365_pretrained 硬件设备:GTX 1060(单GPU) 参数设置: max_iters: 168000 LearningRate: base_lr: 0.0004 schedulers:
- !PiecewiseDecay
gamma: 0.1
milestones:
- 120000
- 160000
- !LinearWarmup start_factor: 0. steps: 4000 部分输出日志如下: 2020-05-30 06:14:26,967-INFO: iter: 167520, lr: 0.000004, 'loss_xy': '2.246058', 'loss_wh': '0.131906', 'loss_obj': '0.012410', 'loss_cls': '0.001954', 'loss_iou': '0.682529', 'loss_iou_aware': '0.001468', 'loss': '3.076323', time: 0.186, eta: 0:01:29 2020-05-30 06:14:31,415-INFO: iter: 167540, lr: 0.000004, 'loss_xy': '3.273055', 'loss_wh': '0.209258', 'loss_obj': '0.115878', 'loss_cls': '0.006119', 'loss_iou': '1.545025', 'loss_iou_aware': '0.006425', 'loss': '5.155761', time: 0.164, eta: 0:01:15 2020-05-30 06:14:36,009-INFO: iter: 167560, lr: 0.000004, 'loss_xy': '2.693145', 'loss_wh': '0.172948', 'loss_obj': '0.075382', 'loss_cls': '0.001816', 'loss_iou': '0.839077', 'loss_iou_aware': '0.001352', 'loss': '3.783719', time: 0.290, eta: 0:02:07 2020-05-30 06:14:40,466-INFO: iter: 167580, lr: 0.000004, 'loss_xy': '3.943757', 'loss_wh': '0.276499', 'loss_obj': '0.394455', 'loss_cls': '0.004380', 'loss_iou': '1.617814', 'loss_iou_aware': '0.004830', 'loss': '6.241735', time: 0.200, eta: 0:01:24 2020-05-30 06:14:44,840-INFO: iter: 167600, lr: 0.000004, 'loss_xy': '4.160210', 'loss_wh': '0.181095', 'loss_obj': '0.227801', 'loss_cls': '0.003091', 'loss_iou': '1.076061', 'loss_iou_aware': '0.003005', 'loss': '5.651264', time: 0.289, eta: 0:01:55 2020-05-30 06:14:49,508-INFO: iter: 167620, lr: 0.000004, 'loss_xy': '2.431662', 'loss_wh': '0.069355', 'loss_obj': '0.013711', 'loss_cls': '0.003777', 'loss_iou': '0.558544', 'loss_iou_aware': '0.002806', 'loss': '3.079854', time: 0.186, eta: 0:01:10 2020-05-30 06:14:54,052-INFO: iter: 167640, lr: 0.000004, 'loss_xy': '1.926714', 'loss_wh': '0.067578', 'loss_obj': '0.010637', 'loss_cls': '0.003065', 'loss_iou': '0.368102', 'loss_iou_aware': '0.002851', 'loss': '2.378947', time: 0.289, eta: 0:01:44 2020-05-30 06:14:58,558-INFO: iter: 167660, lr: 0.000004, 'loss_xy': '3.000982', 'loss_wh': '0.038119', 'loss_obj': '0.025343', 'loss_cls': '0.003633', 'loss_iou': '0.545230', 'loss_iou_aware': '0.002434', 'loss': '3.615741', time: 0.245, eta: 0:01:23 2020-05-30 06:15:02,981-INFO: iter: 167680, lr: 0.000004, 'loss_xy': '4.154136', 'loss_wh': '0.132925', 'loss_obj': '0.069874', 'loss_cls': '0.003423', 'loss_iou': '0.988418', 'loss_iou_aware': '0.004481', 'loss': '5.353256', time: 0.165, eta: 0:00:52 2020-05-30 06:15:07,615-INFO: iter: 167700, lr: 0.000004, 'loss_xy': '3.450465', 'loss_wh': '0.105813', 'loss_obj': '0.004646', 'loss_cls': '0.002043', 'loss_iou': '0.758110', 'loss_iou_aware': '0.002518', 'loss': '4.323595', time: 0.289, eta: 0:01:26 2020-05-30 06:15:12,247-INFO: iter: 167720, lr: 0.000004, 'loss_xy': '5.094399', 'loss_wh': '0.155303', 'loss_obj': '0.229020', 'loss_cls': '0.003947', 'loss_iou': '1.167494', 'loss_iou_aware': '0.005018', 'loss': '6.655182', time: 0.271, eta: 0:01:15 2020-05-30 06:15:16,721-INFO: iter: 167740, lr: 0.000004, 'loss_xy': '5.470705', 'loss_wh': '0.162787', 'loss_obj': '0.050715', 'loss_cls': '0.006103', 'loss_iou': '1.153234', 'loss_iou_aware': '0.010553', 'loss': '6.854097', time: 0.164, eta: 0:00:42 2020-05-30 06:15:21,137-INFO: iter: 167760, lr: 0.000004, 'loss_xy': '2.837036', 'loss_wh': '0.119636', 'loss_obj': '0.048652', 'loss_cls': '0.002462', 'loss_iou': '0.745960', 'loss_iou_aware': '0.002393', 'loss': '3.756140', time: 0.269, eta: 0:01:04 2020-05-30 06:15:25,637-INFO: iter: 167780, lr: 0.000004, 'loss_xy': '2.362906', 'loss_wh': '0.087851', 'loss_obj': '4.997545', 'loss_cls': '0.003284', 'loss_iou': '0.630845', 'loss_iou_aware': '0.002806', 'loss': '8.085238', time: 0.270, eta: 0:00:59 2020-05-30 06:15:30,318-INFO: iter: 167800, lr: 0.000004, 'loss_xy': '3.572321', 'loss_wh': '0.144000', 'loss_obj': '0.010618', 'loss_cls': '0.003705', 'loss_iou': '0.856327', 'loss_iou_aware': '0.002462', 'loss': '4.589433', time: 0.245, eta: 0:00:48 2020-05-30 06:15:34,912-INFO: iter: 167820, lr: 0.000004, 'loss_xy': '5.696221', 'loss_wh': '0.243897', 'loss_obj': '0.027483', 'loss_cls': '0.007104', 'loss_iou': '1.703367', 'loss_iou_aware': '0.011312', 'loss': '7.689384', time: 0.223, eta: 0:00:40 2020-05-30 06:15:39,400-INFO: iter: 167840, lr: 0.000004, 'loss_xy': '4.652727', 'loss_wh': '0.129715', 'loss_obj': '0.012127', 'loss_cls': '0.002893', 'loss_iou': '0.823375', 'loss_iou_aware': '0.004305', 'loss': '5.625143', time: 0.184, eta: 0:00:29 2020-05-30 06:15:43,859-INFO: iter: 167860, lr: 0.000004, 'loss_xy': '3.307869', 'loss_wh': '0.052787', 'loss_obj': '0.011986', 'loss_cls': '0.003379', 'loss_iou': '0.523182', 'loss_iou_aware': '0.004549', 'loss': '3.903752', time: 0.244, eta: 0:00:34 2020-05-30 06:15:48,443-INFO: iter: 167880, lr: 0.000004, 'loss_xy': '6.311714', 'loss_wh': '0.322868', 'loss_obj': '0.050633', 'loss_cls': '0.006870', 'loss_iou': '1.937109', 'loss_iou_aware': '0.005934', 'loss': '8.635129', time: 0.227, eta: 0:00:27 2020-05-30 06:15:52,802-INFO: iter: 167900, lr: 0.000004, 'loss_xy': '2.030791', 'loss_wh': '0.128073', 'loss_obj': '0.006715', 'loss_cls': '0.001170', 'loss_iou': '0.816985', 'loss_iou_aware': '0.000785', 'loss': '2.984518', time: 0.185, eta: 0:00:18 2020-05-30 06:15:57,428-INFO: iter: 167920, lr: 0.000004, 'loss_xy': '3.877166', 'loss_wh': '0.103015', 'loss_obj': '0.011861', 'loss_cls': '0.003750', 'loss_iou': '0.692197', 'loss_iou_aware': '0.005119', 'loss': '4.693107', time: 0.287, eta: 0:00:22 2020-05-30 06:16:01,863-INFO: iter: 167940, lr: 0.000004, 'loss_xy': '1.875529', 'loss_wh': '0.045435', 'loss_obj': '0.672520', 'loss_cls': '0.002949', 'loss_iou': '0.407925', 'loss_iou_aware': '0.000878', 'loss': '3.005235', time: 0.184, eta: 0:00:11 2020-05-30 06:16:06,361-INFO: iter: 167960, lr: 0.000004, 'loss_xy': '1.779233', 'loss_wh': '0.058582', 'loss_obj': '0.237887', 'loss_cls': '0.004172', 'loss_iou': '0.831138', 'loss_iou_aware': '0.003687', 'loss': '2.914700', time: 0.186, eta: 0:00:07 2020-05-30 06:16:10,906-INFO: iter: 167980, lr: 0.000004, 'loss_xy': '3.984343', 'loss_wh': '0.268123', 'loss_obj': '1.528533', 'loss_cls': '0.003928', 'loss_iou': '2.151222', 'loss_iou_aware': '0.018473', 'loss': '7.954621', time: 0.183, eta: 0:00:03 有些参数到底应该怎么修改?有详细具体的说明文档吗?