IOError: got end of file during message when training with CPU
Created by: witkeyshare
root@537527a1fed8:/DeepSpeech/examples/tiny# sh run_train.sh
----------- Configuration Arguments -----------
augment_conf_path: conf/augmentation.config
batch_size: 2
dev_manifest: data/tiny/manifest.tiny
init_model_path: None
is_local: 1
learning_rate: 1e-05
max_duration: 27.0
mean_std_path: data/tiny/mean_std.npz
min_duration: 0.0
num_conv_layers: 2
num_iter_print: 100
num_passes: 10
num_proc_data: 1
num_rnn_layers: 3
output_model_dir: ./checkpoints/tiny
rnn_layer_size: 2048
share_rnn_weights: 1
shuffle_method: batch_shuffle_clipped
specgram_type: linear
test_off: 0
train_manifest: data/tiny/manifest.tiny
trainer_count: 4
use_gpu: 0
use_gru: 0
use_sortagrad: 1
vocab_path: data/tiny/vocab.txt
------------------------------------------------
I1222 08:04:21.331934 17693 Util.cpp:166] commandline: --use_gpu=0 --rnn_use_batch=True --log_clipping=True --trainer_count=4
[INFO 2017-12-22 08:04:21,345 layers.py:2696] output for __conv_0__: c = 32, h = 81, w = 54, size = 139968
[INFO 2017-12-22 08:04:21,347 layers.py:3264] output for __batch_norm_0__: c = 32, h = 81, w = 54, size = 139968
[INFO 2017-12-22 08:04:21,347 layers.py:7436] output for __scale_sub_region_0__: c = 32, h = 81, w = 54, size = 139968
[INFO 2017-12-22 08:04:21,348 layers.py:2696] output for __conv_1__: c = 32, h = 41, w = 54, size = 70848
[INFO 2017-12-22 08:04:21,348 layers.py:3264] output for __batch_norm_1__: c = 32, h = 41, w = 54, size = 70848
[INFO 2017-12-22 08:04:21,349 layers.py:7436] output for __scale_sub_region_1__: c = 32, h = 41, w = 54, size = 70848
I1222 08:04:21.697677 17693 GradientMachine.cpp:94] Initing parameters..
I1222 08:04:23.947732 17693 GradientMachine.cpp:101] Init parameters done.
.Exception in thread Thread-1:
Traceback (most recent call last):
File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
self.run()
File "/usr/lib/python2.7/threading.py", line 754, in run
self.__target(*self.__args, **self.__kwargs)
File "/DeepSpeech/data_utils/utility.py", line 153, in flush_worker
sample = in_queue.get()
File "<string>", line 2, in get
File "/usr/lib/python2.7/multiprocessing/managers.py", line 759, in _callmethod
kind, result = conn.recv()
IOError: got end of file during message
need to set something ?