Created by: isofun

你好 文档 中给出的实验结果准确率在90%左右，我理解的是表格中的结果左边是没用预训练，右边用了预训练；但是我用目前的代码只能跑到70%左右，然后devloss就开始上涨了，默认参数基本上没有改动，只是把batch_size改成了32，希望能够帮忙看一下问题出在哪里了。 以下是部分日志，dev loss最低的时候大概为0.62： [dev evaluation] ave loss: 0.648370, ave acc: 0.710833, elapsed time: 1.372104 s step: 7860, ave loss: 0.330971, ave acc: 0.843750, speed: 3.124809 steps/s step: 7870, ave loss: 0.449688, ave acc: 0.687500, speed: 5.640100 steps/s step: 7880, ave loss: 0.437291, ave acc: 0.656250, speed: 5.932377 steps/s step: 7890, ave loss: 0.399466, ave acc: 0.875000, speed: 6.088658 steps/s step: 7900, ave loss: 0.367463, ave acc: 0.906250, speed: 6.215408 steps/s do evalatation [dev evaluation] ave loss: 0.647289, ave acc: 0.706667, elapsed time: 1.353274 s step: 7910, ave loss: 0.358904, ave acc: 0.812500, speed: 3.332746 steps/s step: 7920, ave loss: 0.398318, ave acc: 0.781250, speed: 6.083976 steps/s step: 7930, ave loss: 0.325438, ave acc: 0.906250, speed: 3.674678 steps/s step: 7940, ave loss: 0.315872, ave acc: 0.843750, speed: 5.414906 steps/s step: 7950, ave loss: 0.315895, ave acc: 0.875000, speed: 5.962798 steps/s do evalatation [dev evaluation] ave loss: 0.647060, ave acc: 0.707500, elapsed time: 1.633162 s step: 7960, ave loss: 0.524899, ave acc: 0.687500, speed: 2.950875 steps/s step: 7970, ave loss: 0.454106, ave acc: 0.812500, speed: 5.759536 steps/s step: 7980, ave loss: 0.245226, ave acc: 0.937500, speed: 6.064020 steps/s step: 7990, ave loss: 0.312347, ave acc: 0.843750, speed: 6.042671 steps/s step: 8000, ave loss: 0.327485, ave acc: 0.843750, speed: 4.937799 steps/s do evalatation [dev evaluation] ave loss: 0.649432, ave acc: 0.709167, elapsed time: 1.446141 s step: 8010, ave loss: 0.269960, ave acc: 0.906250, speed: 3.160801 steps/s step: 8020, ave loss: 0.310272, ave acc: 0.812500, speed: 6.020429 steps/s step: 8030, ave loss: 0.374136, ave acc: 0.750000, speed: 5.688697 steps/s step: 8040, ave loss: 0.432397, ave acc: 0.781250, speed: 6.041522 steps/s step: 8050, ave loss: 0.366272, ave acc: 0.843750, speed: 6.063380 steps/s do evalatation [dev evaluation] ave loss: 0.651746, ave acc: 0.710000, elapsed time: 1.460297 s step: 8060, ave loss: 0.431585, ave acc: 0.750000, speed: 3.163170 steps/s step: 8070, ave loss: 0.365779, ave acc: 0.750000, speed: 5.453197 steps/s step: 8080, ave loss: 0.406466, ave acc: 0.781250, speed: 5.655510 steps/s step: 8090, ave loss: 0.314121, ave acc: 0.875000, speed: 5.909949 steps/s step: 8100, ave loss: 0.357805, ave acc: 0.875000, speed: 6.067416 steps/s do evalatation [dev evaluation] ave loss: 0.651485, ave acc: 0.706667, elapsed time: 1.423898 s step: 8110, ave loss: 0.330592, ave acc: 0.843750, speed: 3.281906 steps/s step: 8120, ave loss: 0.406624, ave acc: 0.781250, speed: 4.848600 steps/s step: 8130, ave loss: 0.274350, ave acc: 0.875000, speed: 5.724158 steps/s