Created by: kenyeung128

Hi I tried to train a model but seems the validation cost increase after each pass as below, is it normal?

[2018-03-21 00:00:10] ------- Time: 22227 sec, Pass: 0, ValidationCost: 18.345830903 [2018-03-21 06:10:07] ------- Time: 22185 sec, Pass: 1, ValidationCost: 18.3184596212 [2018-03-21 12:20:30] ------- Time: 22209 sec, Pass: 2, ValidationCost: 17.5856525281 [2018-03-21 18:27:57] ------- Time: 22034 sec, Pass: 3, ValidationCost: 18.2304718057 [2018-03-22 00:34:54] ------- Time: 22004 sec, Pass: 4, ValidationCost: 17.9108289895 [2018-03-22 06:42:42] ------- Time: 22056 sec, Pass: 5, ValidationCost: 18.7010796109 [2018-03-22 12:49:11] ------- Time: 21976 sec, Pass: 6, ValidationCost: 19.2663258581 [2018-03-22 18:57:12] ------- Time: 22070 sec, Pass: 7, ValidationCost: 19.5299177824 [2018-03-23 01:10:56] ------- Time: 22409 sec, Pass: 8, ValidationCost: 20.5845607126 [2018-03-23 07:20:32] ------- Time: 22163 sec, Pass: 9, ValidationCost: 20.757628163 [2018-03-23 13:28:11] ------- Time: 22045 sec, Pass: 10, ValidationCost: 20.8393640953 [2018-03-23 19:41:08] ------- Time: 22363 sec, Pass: 11, ValidationCost: 21.0724439657 [2018-03-24 01:53:30] ------- Time: 22327 sec, Pass: 12, ValidationCost: 21.9896265413 [2018-03-24 08:01:36] ------- Time: 22073 sec, Pass: 13, ValidationCost: 22.531096637 [2018-03-24 14:09:42] ------- Time: 22073 sec, Pass: 14, ValidationCost: 22.3551265387 [2018-03-24 20:21:39] ------- Time: 22303 sec, Pass: 15, ValidationCost: 23.8237918068 [2018-03-25 02:33:52] ------- Time: 22318 sec, Pass: 16, ValidationCost: 24.2891556082 [2018-03-25 08:41:56] ------- Time: 22072 sec, Pass: 17, ValidationCost: 24.6508512178 [2018-03-25 14:49:34] ------- Time: 22044 sec, Pass: 18, ValidationCost: 25.2140230672 [2018-03-25 21:01:05] ------- Time: 22276 sec, Pass: 19, ValidationCost: 23.3633104355 [2018-03-26 03:12:44] ------- Time: 22285 sec, Pass: 20, ValidationCost: 27.1979689747 [2018-03-26 09:23:12] ------- Time: 22214 sec, Pass: 21, ValidationCost: 23.4449897665 [2018-03-26 15:32:47] ------- Time: 22160 sec, Pass: 22, ValidationCost: 24.0375935756 [2018-03-26 21:42:57] ------- Time: 22197 sec, Pass: 23, ValidationCost: 24.7978525064 [2018-03-27 03:51:56] ------- Time: 22125 sec, Pass: 24, ValidationCost: 24.6670613937 [2018-03-27 10:01:58] ------- Time: 22187 sec, Pass: 25, ValidationCost: 25.072318309 [2018-03-27 16:17:09] ------- Time: 22496 sec, Pass: 26, ValidationCost: 26.3841101534 [2018-03-27 22:32:06] ------- Time: 22482 sec, Pass: 27, ValidationCost: 26.4974455705 [2018-03-28 04:45:17] ------- Time: 22377 sec, Pass: 28, ValidationCost: 25.0781797638