我训练一个汉语模型,将aishell的数据集降采样为8k的,结果不能收敛,最左还报错了。
Created by: songxia928
................................................................................................... Pass: 0, Batch: 100, TrainCost: 68.859889 ................................................................................................... Pass: 0, Batch: 200, TrainCost: 68.140386 ................................................................................................... Pass: 0, Batch: 300, TrainCost: 73.981499 ................................................................................................... Pass: 0, Batch: 400, TrainCost: 79.325998 ................................................................................................... Pass: 0, Batch: 500, TrainCost: 84.907326 ................................................................................................... Pass: 0, Batch: 600, TrainCost: 91.324884 ................................................................................................... Pass: 0, Batch: 700, TrainCost: 98.817860 ................................................................................................... Pass: 0, Batch: 800, TrainCost: 106.118329 ................................................................................................... Pass: 0, Batch: 900, TrainCost: 111.627918 ................................................................................................... Pass: 0, Batch: 1000, TrainCost: 85.846820 ........................................................................Traceback (most recent call last): File "train.py", line 129, in main() File "train.py", line 125, in main train() File "train.py", line 116, in train test_off=args.test_off) File "/data/chensong/code/20180522_deepspeech/DeepSpeech/model_utils/model.py", line 155, in train feeding=adapted_feeding_dict) File "/usr/local/lib/python2.7/dist-packages/paddle/v2/trainer.py", line 201, in train gm=self.gradient_machine)) File "/data/chensong/code/20180522_deepspeech/DeepSpeech/model_utils/model.py", line 140, in event_handler feeding=adapted_feeding_dict) File "/usr/local/lib/python2.7/dist-packages/paddle/v2/trainer.py", line 234, in test evaluator=evaluator, cost=total_cost / num_samples) ZeroDivisionError: float division by zero Failed in training!
原始的16k的数据可以正常训练,并收敛。但降采样为8k就不行了。还有自己准备的其他8k数据也不行。 求指教!!!