Skip to content

D
DeepSpeech

PaddlePaddle / DeepSpeech
大约 2 年 前同步成功

通知 210
Star 8425
Fork 1598
D
DeepSpeech
未验证 提交 4cab9f62 编写于 作者: H Hui Zhang 提交者： GitHub
浏览文件
操作

Merge pull request #1225 from Jackwaterveg/new_config 

[ASR][Config]refactor the train and test config
上级 03f8accd 455bf477
展开全部 隐藏空白更改
内联 并排
Showing with 3266 addition and 3792 deletion
examples/aishell/asr0/conf/deepspeech2.yaml
浏览文件 @ 4cab9f62
# https://yaml.org/type/float.html # https://yaml.org/type/float.html
data: ###########################################
train_manifest: data/manifest.train # Data #
dev_manifest: data/manifest.dev ###########################################
test_manifest: data/manifest.test train_manifest: data/manifest.train
min_input_len: 0.0 dev_manifest: data/manifest.dev
max_input_len: 27.0 # second test_manifest: data/manifest.test
min_output_len: 0.0 min_input_len: 0.0
max_output_len: .inf max_input_len: 27.0 # second
min_output_input_ratio: 0.00 min_output_len: 0.0
max_output_input_ratio: .inf max_output_len: .inf
min_output_input_ratio: 0.00
max_output_input_ratio: .inf
collator: ###########################################
batch_size: 64 # one gpu # Dataloader #
mean_std_filepath: data/mean_std.json ###########################################
unit_type: char batch_size: 64 # one gpu
vocab_filepath: data/lang_char/vocab.txt mean_std_filepath: data/mean_std.json
augmentation_config: conf/augmentation.json unit_type: char
random_seed: 0 vocab_filepath: data/lang_char/vocab.txt
spm_model_prefix: augmentation_config: conf/augmentation.json
spectrum_type: linear random_seed: 0
feat_dim: spm_model_prefix:
delta_delta: False spectrum_type: linear
stride_ms: 10.0 feat_dim:
window_ms: 20.0 delta_delta: False
n_fft: None stride_ms: 10.0
max_freq: None window_ms: 20.0
target_sample_rate: 16000 n_fft: None
use_dB_normalization: True max_freq: None
target_dB: -20 target_sample_rate: 16000
dither: 1.0 use_dB_normalization: True
keep_transcription_text: False target_dB: -20
sortagrad: True dither: 1.0
shuffle_method: batch_shuffle keep_transcription_text: False
num_workers: 2 sortagrad: True
shuffle_method: batch_shuffle
num_workers: 2
model: ############################################
num_conv_layers: 2 # Network Architecture #
num_rnn_layers: 3 ############################################
rnn_layer_size: 1024 num_conv_layers: 2
use_gru: True num_rnn_layers: 3
share_rnn_weights: False rnn_layer_size: 1024
blank_id: 0 use_gru: True
ctc_grad_norm_type: instance share_rnn_weights: False
blank_id: 0
ctc_grad_norm_type: instance
training: ###########################################
n_epoch: 80 # Training #
accum_grad: 1 ###########################################
lr: 2e-3 n_epoch: 80
lr_decay: 0.83 accum_grad: 1
weight_decay: 1e-06 lr: 2e-3
global_grad_clip: 3.0 lr_decay: 0.83
log_interval: 100 weight_decay: 1e-06
checkpoint: global_grad_clip: 3.0
kbest_n: 50 log_interval: 100
latest_n: 5 checkpoint:
kbest_n: 50
decoding: latest_n: 5
batch_size: 128
error_rate_type: cer
decoding_method: ctc_beam_search
lang_model_path: data/lm/zh_giga.no_cna_cmn.prune01244.klm
alpha: 1.9
beta: 5.0
beam_size: 300
cutoff_prob: 0.99
cutoff_top_n: 40
num_proc_bsearch: 10
examples/aishell/asr0/conf/deepspeech2_online.yaml
浏览文件 @ 4cab9f62
# https://yaml.org/type/float.html # https://yaml.org/type/float.html
data: ###########################################
train_manifest: data/manifest.train # Data #
dev_manifest: data/manifest.dev ###########################################
test_manifest: data/manifest.test train_manifest: data/manifest.train
min_input_len: 0.0 dev_manifest: data/manifest.dev
max_input_len: 27.0 # second test_manifest: data/manifest.test
min_output_len: 0.0 min_input_len: 0.0
max_output_len: .inf max_input_len: 27.0 # second
min_output_input_ratio: 0.00 min_output_len: 0.0
max_output_input_ratio: .inf max_output_len: .inf
min_output_input_ratio: 0.00
max_output_input_ratio: .inf
collator: ###########################################
batch_size: 64 # one gpu # Dataloader #
mean_std_filepath: data/mean_std.json ###########################################
unit_type: char batch_size: 64 # one gpu
vocab_filepath: data/lang_char/vocab.txt mean_std_filepath: data/mean_std.json
augmentation_config: conf/augmentation.json unit_type: char
random_seed: 0 vocab_filepath: data/lang_char/vocab.txt
spm_model_prefix: augmentation_config: conf/augmentation.json
spectrum_type: linear #linear, mfcc, fbank random_seed: 0
feat_dim: spm_model_prefix:
delta_delta: False spectrum_type: linear #linear, mfcc, fbank
stride_ms: 10.0 feat_dim:
window_ms: 20.0 delta_delta: False
n_fft: None stride_ms: 10.0
max_freq: None window_ms: 20.0
target_sample_rate: 16000 n_fft: None
use_dB_normalization: True max_freq: None
target_dB: -20 target_sample_rate: 16000
dither: 1.0 use_dB_normalization: True
keep_transcription_text: False target_dB: -20
sortagrad: True dither: 1.0
shuffle_method: batch_shuffle keep_transcription_text: False
num_workers: 0 sortagrad: True
shuffle_method: batch_shuffle
num_workers: 0
model: ############################################
num_conv_layers: 2 # Network Architecture #
num_rnn_layers: 5 ############################################
rnn_layer_size: 1024 num_conv_layers: 2
rnn_direction: forward # [forward, bidirect] num_rnn_layers: 5
num_fc_layers: 0 rnn_layer_size: 1024
fc_layers_size_list: -1, rnn_direction: forward # [forward, bidirect]
use_gru: False num_fc_layers: 0
blank_id: 0 fc_layers_size_list: -1,
use_gru: False
blank_id: 0
training: ###########################################
n_epoch: 65 # Training #
accum_grad: 1 ###########################################
lr: 5e-4 n_epoch: 65
lr_decay: 0.93 accum_grad: 1
weight_decay: 1e-06 lr: 5e-4
global_grad_clip: 3.0 lr_decay: 0.93
log_interval: 100 weight_decay: 1e-06
checkpoint: global_grad_clip: 3.0
kbest_n: 50 log_interval: 100
latest_n: 5 checkpoint:
kbest_n: 50
latest_n: 5
decoding:
batch_size: 32
error_rate_type: cer
decoding_method: ctc_beam_search
lang_model_path: data/lm/zh_giga.no_cna_cmn.prune01244.klm
alpha: 2.2 #1.9
beta: 4.3
beam_size: 300
cutoff_prob: 0.99
cutoff_top_n: 40
num_proc_bsearch: 10
examples/aishell/asr0/conf/tuning/chunk_decode.yaml 0 → 100644
浏览文件 @ 4cab9f62
chunk_batch_size: 32
error_rate_type: cer
decoding_method: ctc_beam_search
lang_model_path: data/lm/zh_giga.no_cna_cmn.prune01244.klm
alpha: 2.2 #1.9
beta: 4.3
beam_size: 300
cutoff_prob: 0.99
cutoff_top_n: 40
num_proc_bsearch: 10
examples/aishell/asr0/conf/tuning/decode.yaml 0 → 100644
浏览文件 @ 4cab9f62
decode_batch_size: 128
error_rate_type: cer
decoding_method: ctc_beam_search
lang_model_path: data/lm/zh_giga.no_cna_cmn.prune01244.klm
alpha: 1.9
beta: 5.0
beam_size: 300
cutoff_prob: 0.99
cutoff_top_n: 40
num_proc_bsearch: 10
examples/aishell/asr0/local/test.sh
浏览文件 @ 4cab9f62
#!/bin/bash #!/bin/bash
if [ $# != 3 ];then if [ $# != 4 ];then
echo "usage: ${0} config_path ckpt_path_prefix model_type" echo "usage: ${0} config_path decode_config_path ckpt_path_prefix model_type"
exit -1 exit -1
fi fi
...@@ -9,8 +9,9 @@ ngpu=$(echo $CUDA_VISIBLE_DEVICES | awk -F "," '{print NF}') ...@@ -9,8 +9,9 @@ ngpu=$(echo $CUDA_VISIBLE_DEVICES | awk -F "," '{print NF}')
echo "using $ngpu gpus..." echo "using $ngpu gpus..."
config_path=$1 config_path=$1
ckpt_prefix=$2 decode_config_path=$2
model_type=$3 ckpt_prefix=$3
model_type=$4
# download language model # download language model
bash local/download_lm_ch.sh bash local/download_lm_ch.sh
...@@ -21,6 +22,7 @@ fi ...@@ -21,6 +22,7 @@ fi
python3 -u ${BIN_DIR}/test.py \ python3 -u ${BIN_DIR}/test.py \
--ngpu ${ngpu} \ --ngpu ${ngpu} \
--config ${config_path} \ --config ${config_path} \
--decode_cfg ${decode_config_path} \
--result_file ${ckpt_prefix}.rsl \ --result_file ${ckpt_prefix}.rsl \
--checkpoint_path ${ckpt_prefix} \ --checkpoint_path ${ckpt_prefix} \
--model_type ${model_type} --model_type ${model_type}
......
examples/aishell/asr0/local/test_export.sh
浏览文件 @ 4cab9f62
#!/bin/bash #!/bin/bash
if [ $# != 3 ];then if [ $# != 4 ];then
echo "usage: ${0} config_path ckpt_path_prefix model_type" echo "usage: ${0} config_path decode_config_path ckpt_path_prefix model_type"
exit -1 exit -1
fi fi
...@@ -9,8 +9,9 @@ ngpu=$(echo $CUDA_VISIBLE_DEVICES | awk -F "," '{print NF}') ...@@ -9,8 +9,9 @@ ngpu=$(echo $CUDA_VISIBLE_DEVICES | awk -F "," '{print NF}')
echo "using $ngpu gpus..." echo "using $ngpu gpus..."
config_path=$1 config_path=$1
jit_model_export_path=$2 decode_config_path=$2
model_type=$3 jit_model_export_path=$3
model_type=$4
# download language model # download language model
bash local/download_lm_ch.sh > /dev/null 2>&1 bash local/download_lm_ch.sh > /dev/null 2>&1
...@@ -21,6 +22,7 @@ fi ...@@ -21,6 +22,7 @@ fi
python3 -u ${BIN_DIR}/test_export.py \ python3 -u ${BIN_DIR}/test_export.py \
--ngpu ${ngpu} \ --ngpu ${ngpu} \
--config ${config_path} \ --config ${config_path} \
--decode_cfg ${decode_config_path} \
--result_file ${jit_model_export_path}.rsl \ --result_file ${jit_model_export_path}.rsl \
--export_path ${jit_model_export_path} \ --export_path ${jit_model_export_path} \
--model_type ${model_type} --model_type ${model_type}
......
examples/aishell/asr0/local/test_wav.sh
浏览文件 @ 4cab9f62
#!/bin/bash #!/bin/bash
if [ $# != 4 ];then if [ $# != 5 ];then
echo "usage: ${0} config_path ckpt_path_prefix model_type audio_file" echo "usage: ${0} config_path decode_config_path ckpt_path_prefix model_type audio_file"
exit -1 exit -1
fi fi
...@@ -9,9 +9,10 @@ ngpu=$(echo $CUDA_VISIBLE_DEVICES | awk -F "," '{print NF}') ...@@ -9,9 +9,10 @@ ngpu=$(echo $CUDA_VISIBLE_DEVICES | awk -F "," '{print NF}')
echo "using $ngpu gpus..." echo "using $ngpu gpus..."
config_path=$1 config_path=$1
ckpt_prefix=$2 decode_config_path=$2
model_type=$3 ckpt_prefix=$3
audio_file=$4 model_type=$4
audio_file=$5
mkdir -p data mkdir -p data
wget -nc https://paddlespeech.bj.bcebos.com/datasets/single_wav/zh/demo_01_03.wav -P data/ wget -nc https://paddlespeech.bj.bcebos.com/datasets/single_wav/zh/demo_01_03.wav -P data/
...@@ -33,6 +34,7 @@ fi ...@@ -33,6 +34,7 @@ fi
python3 -u ${BIN_DIR}/test_wav.py \ python3 -u ${BIN_DIR}/test_wav.py \
--ngpu ${ngpu} \ --ngpu ${ngpu} \
--config ${config_path} \ --config ${config_path} \
--decode_cfg ${decode_config_path} \
--result_file ${ckpt_prefix}.rsl \ --result_file ${ckpt_prefix}.rsl \
--checkpoint_path ${ckpt_prefix} \ --checkpoint_path ${ckpt_prefix} \
--model_type ${model_type} \ --model_type ${model_type} \
......
examples/aishell/asr0/run.sh
浏览文件 @ 4cab9f62
...@@ -6,6 +6,7 @@ gpus=0,1,2,3 ...@@ -6,6 +6,7 @@ gpus=0,1,2,3
stage=0 stage=0
stop_stage=100 stop_stage=100
conf_path=conf/deepspeech2.yaml #conf/deepspeech2.yaml or conf/deepspeeech2_online.yaml conf_path=conf/deepspeech2.yaml #conf/deepspeech2.yaml or conf/deepspeeech2_online.yaml
decode_conf_path=conf/tuning/decode.yaml
avg_num=1 avg_num=1
model_type=offline # offline or online model_type=offline # offline or online
audio_file=data/demo_01_03.wav audio_file=data/demo_01_03.wav
...@@ -34,7 +35,7 @@ fi ...@@ -34,7 +35,7 @@ fi
if [ ${stage} -le 3 ] && [ ${stop_stage} -ge 3 ]; then if [ ${stage} -le 3 ] && [ ${stop_stage} -ge 3 ]; then
# test ckpt avg_n # test ckpt avg_n
CUDA_VISIBLE_DEVICES=0 ./local/test.sh ${conf_path} exp/${ckpt}/checkpoints/${avg_ckpt} ${model_type}|| exit -1 CUDA_VISIBLE_DEVICES=0 ./local/test.sh ${conf_path} ${decode_conf_path} exp/${ckpt}/checkpoints/${avg_ckpt} ${model_type}|| exit -1
fi fi
if [ ${stage} -le 4 ] && [ ${stop_stage} -ge 4 ]; then if [ ${stage} -le 4 ] && [ ${stop_stage} -ge 4 ]; then
...@@ -44,11 +45,11 @@ fi ...@@ -44,11 +45,11 @@ fi
if [ ${stage} -le 5 ] && [ ${stop_stage} -ge 5 ]; then if [ ${stage} -le 5 ] && [ ${stop_stage} -ge 5 ]; then
# test export ckpt avg_n # test export ckpt avg_n
CUDA_VISIBLE_DEVICES=0 ./local/test_export.sh ${conf_path} exp/${ckpt}/checkpoints/${avg_ckpt}.jit ${model_type}|| exit -1 CUDA_VISIBLE_DEVICES=0 ./local/test_export.sh ${conf_path} ${decode_conf_path} exp/${ckpt}/checkpoints/${avg_ckpt}.jit ${model_type}|| exit -1
fi fi
# Optionally, you can add LM and test it with runtime. # Optionally, you can add LM and test it with runtime.
if [ ${stage} -le 6 ] && [ ${stop_stage} -ge 6 ]; then if [ ${stage} -le 6 ] && [ ${stop_stage} -ge 6 ]; then
# test a single .wav file # test a single .wav file
CUDA_VISIBLE_DEVICES=0 ./local/test_wav.sh ${conf_path} exp/${ckpt}/checkpoints/${avg_ckpt} ${model_type} ${audio_file} || exit -1 CUDA_VISIBLE_DEVICES=0 ./local/test_wav.sh ${conf_path} ${decode_conf_path} exp/${ckpt}/checkpoints/${avg_ckpt} ${model_type} ${audio_file} || exit -1
fi fi
examples/aishell/asr1/conf/chunk_conformer.yaml
浏览文件 @ 4cab9f62
# network architecture ############################################
model: # Network Architecture #
cmvn_file: ############################################
cmvn_file_type: "json" cmvn_file:
# encoder related cmvn_file_type: "json"
encoder: conformer # encoder related
encoder_conf: encoder: conformer
output_size: 256 # dimension of attention encoder_conf:
attention_heads: 4 output_size: 256 # dimension of attention
linear_units: 2048 # the number of units of position-wise feed forward attention_heads: 4
num_blocks: 12 # the number of encoder blocks linear_units: 2048 # the number of units of position-wise feed forward
dropout_rate: 0.1 num_blocks: 12 # the number of encoder blocks
positional_dropout_rate: 0.1 dropout_rate: 0.1
attention_dropout_rate: 0.0 positional_dropout_rate: 0.1
input_layer: conv2d # encoder input type, you can chose conv2d, conv2d6 and conv2d8 attention_dropout_rate: 0.0
normalize_before: True input_layer: conv2d # encoder input type, you can chose conv2d, conv2d6 and conv2d8
cnn_module_kernel: 15 normalize_before: True
use_cnn_module: True cnn_module_kernel: 15
activation_type: 'swish' use_cnn_module: True
pos_enc_layer_type: 'rel_pos' activation_type: 'swish'
selfattention_layer_type: 'rel_selfattn' pos_enc_layer_type: 'rel_pos'
causal: true selfattention_layer_type: 'rel_selfattn'
use_dynamic_chunk: true causal: true
cnn_module_norm: 'layer_norm' # using nn.LayerNorm makes model converge faster use_dynamic_chunk: true
use_dynamic_left_chunk: false cnn_module_norm: 'layer_norm' # using nn.LayerNorm makes model converge faster
use_dynamic_left_chunk: false
# decoder related
decoder: transformer
decoder_conf:
attention_heads: 4
linear_units: 2048
num_blocks: 6
dropout_rate: 0.1
positional_dropout_rate: 0.1
self_attention_dropout_rate: 0.0
src_attention_dropout_rate: 0.0
# hybrid CTC/attention
model_conf:
ctc_weight: 0.3
lsm_weight: 0.1 # label smoothing option
length_normalized_loss: false
# decoder related ###########################################
decoder: transformer # Data #
decoder_conf: ###########################################
attention_heads: 4
linear_units: 2048
num_blocks: 6
dropout_rate: 0.1
positional_dropout_rate: 0.1
self_attention_dropout_rate: 0.0
src_attention_dropout_rate: 0.0
# hybrid CTC/attention train_manifest: data/manifest.train
model_conf: dev_manifest: data/manifest.dev
ctc_weight: 0.3 test_manifest: data/manifest.test
lsm_weight: 0.1 # label smoothing option
length_normalized_loss: false
data: ###########################################
train_manifest: data/manifest.train # Dataloader #
dev_manifest: data/manifest.dev ###########################################
test_manifest: data/manifest.test
vocab_filepath: data/lang_char/vocab.txt
spm_model_prefix: ''
unit_type: 'char'
preprocess_config: conf/preprocess.yaml
feat_dim: 80
stride_ms: 10.0
window_ms: 25.0
sortagrad: 0 # Feed samples from shortest to longest ; -1: enabled for all epochs, 0: disabled, other: enabled for 'other' epochs
batch_size: 64
maxlen_in: 512 # if input length > maxlen-in, batchsize is automatically reduced
maxlen_out: 150 # if output length > maxlen-out, batchsize is automatically reduced
minibatches: 0 # for debug
batch_count: auto
batch_bins: 0
batch_frames_in: 0
batch_frames_out: 0
batch_frames_inout: 0
num_workers: 0
subsampling_factor: 1
num_encs: 1
collator: ###########################################
vocab_filepath: data/lang_char/vocab.txt # Training #
unit_type: 'char' ###########################################
augmentation_config: conf/preprocess.yaml n_epoch: 240
feat_dim: 80 accum_grad: 2
stride_ms: 10.0 global_grad_clip: 5.0
window_ms: 25.0 optim: adam
sortagrad: 0 # Feed samples from shortest to longest ; -1: enabled for all epochs, 0: disabled, other: enabled for 'other' epochs optim_conf:
batch_size: 64 lr: 0.002
maxlen_in: 512 # if input length > maxlen-in, batchsize is automatically reduced weight_decay: 1.0e-6
maxlen_out: 150 # if output length > maxlen-out, batchsize is automatically reduced scheduler: warmuplr
minibatches: 0 # for debug scheduler_conf:
batch_count: auto warmup_steps: 25000
batch_bins: 0 lr_decay: 1.0
batch_frames_in: 0 log_interval: 100
batch_frames_out: 0 checkpoint:
batch_frames_inout: 0 kbest_n: 50
num_workers: 0 latest_n: 5
subsampling_factor: 1
num_encs: 1
training:
n_epoch: 240
accum_grad: 2
global_grad_clip: 5.0
optim: adam
optim_conf:
lr: 0.002
weight_decay: 1e-6
scheduler: warmuplr
scheduler_conf:
warmup_steps: 25000
lr_decay: 1.0
log_interval: 100
checkpoint:
kbest_n: 50
latest_n: 5
decoding:
beam_size: 10
batch_size: 128
error_rate_type: cer
decoding_method: attention # 'attention', 'ctc_greedy_search', 'ctc_prefix_beam_search', 'attention_rescoring'
ctc_weight: 0.5 # ctc weight for attention rescoring decode mode.
decoding_chunk_size: -1 # decoding chunk size. Defaults to -1.
# <0: for decoding, use full chunk.
# >0: for decoding, use fixed chunk size as set.
# 0: used for training, it's prohibited here.
num_decoding_left_chunks: -1 # number of left chunks for decoding. Defaults to -1.
simulate_streaming: False # simulate streaming inference. Defaults to False.
examples/aishell/asr1/conf/conformer.yaml
浏览文件 @ 4cab9f62
# network architecture ############################################
model: # Network Architecture #
cmvn_file: ############################################
cmvn_file_type: "json" cmvn_file:
# encoder related cmvn_file_type: "json"
encoder: conformer # encoder related
encoder_conf: encoder: conformer
output_size: 256 # dimension of attention encoder_conf:
attention_heads: 4 output_size: 256 # dimension of attention
linear_units: 2048 # the number of units of position-wise feed forward attention_heads: 4
num_blocks: 12 # the number of encoder blocks linear_units: 2048 # the number of units of position-wise feed forward
dropout_rate: 0.1 num_blocks: 12 # the number of encoder blocks
positional_dropout_rate: 0.1 dropout_rate: 0.1
attention_dropout_rate: 0.0 positional_dropout_rate: 0.1
input_layer: conv2d # encoder input type, you can chose conv2d, conv2d6 and conv2d8 attention_dropout_rate: 0.0
normalize_before: True input_layer: conv2d # encoder input type, you can chose conv2d, conv2d6 and conv2d8
cnn_module_kernel: 15 normalize_before: True
use_cnn_module: True cnn_module_kernel: 15
activation_type: 'swish' use_cnn_module: True
pos_enc_layer_type: 'rel_pos' activation_type: 'swish'
selfattention_layer_type: 'rel_selfattn' pos_enc_layer_type: 'rel_pos'
selfattention_layer_type: 'rel_selfattn'
# decoder related # decoder related
decoder: transformer decoder: transformer
decoder_conf: decoder_conf:
attention_heads: 4 attention_heads: 4
linear_units: 2048 linear_units: 2048
num_blocks: 6 num_blocks: 6
dropout_rate: 0.1 dropout_rate: 0.1
positional_dropout_rate: 0.1 positional_dropout_rate: 0.1
self_attention_dropout_rate: 0.0 self_attention_dropout_rate: 0.0
src_attention_dropout_rate: 0.0 src_attention_dropout_rate: 0.0
# hybrid CTC/attention # hybrid CTC/attention
model_conf: model_conf:
ctc_weight: 0.3 ctc_weight: 0.3
lsm_weight: 0.1 # label smoothing option lsm_weight: 0.1 # label smoothing option
length_normalized_loss: false length_normalized_loss: false
data: ###########################################
train_manifest: data/manifest.train # Data #
dev_manifest: data/manifest.dev ###########################################
test_manifest: data/manifest.test train_manifest: data/manifest.train
dev_manifest: data/manifest.dev
test_manifest: data/manifest.test
###########################################
# Dataloader #
###########################################
vocab_filepath: data/lang_char/vocab.txt
spm_model_prefix: ''
unit_type: 'char'
preprocess_config: conf/preprocess.yaml
feat_dim: 80
stride_ms: 10.0
window_ms: 25.0
sortagrad: 0 # Feed samples from shortest to longest ; -1: enabled for all epochs, 0: disabled, other: enabled for 'other' epochs
batch_size: 64
maxlen_in: 512 # if input length > maxlen-in, batchsize is automatically reduced
maxlen_out: 150 # if output length > maxlen-out, batchsize is automatically reduced
minibatches: 0 # for debug
batch_count: auto
batch_bins: 0
batch_frames_in: 0
batch_frames_out: 0
batch_frames_inout: 0
num_workers: 2
subsampling_factor: 1
num_encs: 1
collator: ###########################################
vocab_filepath: data/lang_char/vocab.txt # Training #
unit_type: 'char' ###########################################
augmentation_config: conf/preprocess.yaml n_epoch: 240
feat_dim: 80 accum_grad: 2
stride_ms: 10.0 global_grad_clip: 5.0
window_ms: 25.0 optim: adam
sortagrad: 0 # Feed samples from shortest to longest ; -1: enabled for all epochs, 0: disabled, other: enabled for 'other' epochs optim_conf:
batch_size: 64 lr: 0.002
maxlen_in: 512 # if input length > maxlen-in, batchsize is automatically reduced weight_decay: 1.0e-6
maxlen_out: 150 # if output length > maxlen-out, batchsize is automatically reduced scheduler: warmuplr
minibatches: 0 # for debug scheduler_conf:
batch_count: auto warmup_steps: 25000
batch_bins: 0 lr_decay: 1.0
batch_frames_in: 0 log_interval: 100
batch_frames_out: 0 checkpoint:
batch_frames_inout: 0 kbest_n: 50
num_workers: 8 latest_n: 5
subsampling_factor: 1
num_encs: 1
training:
n_epoch: 240
accum_grad: 2
global_grad_clip: 5.0
optim: adam
optim_conf:
lr: 0.002
weight_decay: 1e-6
scheduler: warmuplr
scheduler_conf:
warmup_steps: 25000
lr_decay: 1.0
log_interval: 100
checkpoint:
kbest_n: 50
latest_n: 5
decoding:
beam_size: 10
batch_size: 128
error_rate_type: cer
decoding_method: attention # 'attention', 'ctc_greedy_search', 'ctc_prefix_beam_search', 'attention_rescoring'
ctc_weight: 0.5 # ctc weight for attention rescoring decode mode.
decoding_chunk_size: -1 # decoding chunk size. Defaults to -1.
# <0: for decoding, use full chunk.
# >0: for decoding, use fixed chunk size as set.
# 0: used for training, it's prohibited here.
num_decoding_left_chunks: -1 # number of left chunks for decoding. Defaults to -1.
simulate_streaming: False # simulate streaming inference. Defaults to False.
examples/aishell/asr1/conf/transformer.yaml
浏览文件 @ 4cab9f62
# network architecture ############################################
model: # Network Architecture #
cmvn_file: ############################################
cmvn_file_type: "json" cmvn_file:
# encoder related cmvn_file_type: "json"
encoder: transformer # encoder related
encoder_conf: encoder: transformer
output_size: 256 # dimension of attention encoder_conf:
attention_heads: 4 output_size: 256 # dimension of attention
linear_units: 2048 # the number of units of position-wise feed forward attention_heads: 4
num_blocks: 12 # the number of encoder blocks linear_units: 2048 # the number of units of position-wise feed forward
dropout_rate: 0.1 num_blocks: 12 # the number of encoder blocks
positional_dropout_rate: 0.1 dropout_rate: 0.1
attention_dropout_rate: 0.0 positional_dropout_rate: 0.1
input_layer: conv2d # encoder input type, you can chose conv2d, conv2d6 and conv2d8 attention_dropout_rate: 0.0
normalize_before: true input_layer: conv2d # encoder input type, you can chose conv2d, conv2d6 and conv2d8
normalize_before: true
# decoder related # decoder related
decoder: transformer decoder: transformer
decoder_conf: decoder_conf:
attention_heads: 4 attention_heads: 4
linear_units: 2048 linear_units: 2048
num_blocks: 6 num_blocks: 6
dropout_rate: 0.1 dropout_rate: 0.1
positional_dropout_rate: 0.1 positional_dropout_rate: 0.1
self_attention_dropout_rate: 0.0 self_attention_dropout_rate: 0.0
src_attention_dropout_rate: 0.0 src_attention_dropout_rate: 0.0
# hybrid CTC/attention
model_conf:
ctc_weight: 0.3
lsm_weight: 0.1 # label smoothing option
length_normalized_loss: false
# hybrid CTC/attention
model_conf:
ctc_weight: 0.3
lsm_weight: 0.1 # label smoothing option
length_normalized_loss: false
###########################################
# Data #
###########################################
# https://yaml.org/type/float.html # https://yaml.org/type/float.html
data: train_manifest: data/manifest.train
train_manifest: data/manifest.train dev_manifest: data/manifest.dev
dev_manifest: data/manifest.dev test_manifest: data/manifest.test
test_manifest: data/manifest.test
collator:
unit_type: 'char'
vocab_filepath: data/lang_char/vocab.txt
feat_dim: 80
stride_ms: 10.0
window_ms: 25.0
sortagrad: 0 # Feed samples from shortest to longest ; -1: enabled for all epochs, 0: disabled, other: enabled for 'other' epochs
batch_size: 64
maxlen_in: 512 # if input length > maxlen-in, batchsize is automatically reduced
maxlen_out: 150 # if output length > maxlen-out, batchsize is automatically reduced
minibatches: 0 # for debug
batch_count: auto
batch_bins: 0
batch_frames_in: 0
batch_frames_out: 0
batch_frames_inout: 0
augmentation_config: conf/preprocess.yaml
num_workers: 0
subsampling_factor: 1
num_encs: 1
training:
n_epoch: 240
accum_grad: 2
global_grad_clip: 5.0
optim: adam
optim_conf:
lr: 0.002
weight_decay: 1e-6
scheduler: warmuplr
scheduler_conf:
warmup_steps: 25000
lr_decay: 1.0
log_interval: 100
checkpoint:
kbest_n: 50
latest_n: 5
###########################################
# Dataloader #
###########################################
unit_type: 'char'
vocab_filepath: data/lang_char/vocab.txt
spm_model_prefix: ''
feat_dim: 80
stride_ms: 10.0
window_ms: 25.0
sortagrad: 0 # Feed samples from shortest to longest ; -1: enabled for all epochs, 0: disabled, other: enabled for 'other' epochs
batch_size: 64
maxlen_in: 512 # if input length > maxlen-in, batchsize is automatically reduced
maxlen_out: 150 # if output length > maxlen-out, batchsize is automatically reduced
minibatches: 0 # for debug
batch_count: auto
batch_bins: 0
batch_frames_in: 0
batch_frames_out: 0
batch_frames_inout: 0
preprocess_config: conf/preprocess.yaml
num_workers: 0
subsampling_factor: 1
num_encs: 1
decoding: ###########################################
beam_size: 10 # Training #
batch_size: 128 ###########################################
error_rate_type: cer n_epoch: 240
decoding_method: attention # 'attention', 'ctc_greedy_search', 'ctc_prefix_beam_search', 'attention_rescoring' accum_grad: 2
ctc_weight: 0.5 # ctc weight for attention rescoring decode mode. global_grad_clip: 5.0
decoding_chunk_size: -1 # decoding chunk size. Defaults to -1. optim: adam
# <0: for decoding, use full chunk. optim_conf:
# >0: for decoding, use fixed chunk size as set. lr: 0.002
# 0: used for training, it's prohibited here. weight_decay: 1.0e-6
num_decoding_left_chunks: -1 # number of left chunks for decoding. Defaults to -1. scheduler: warmuplr
simulate_streaming: False # simulate streaming inference. Defaults to False. scheduler_conf:
\ No newline at end of file warmup_steps: 25000
lr_decay: 1.0
log_interval: 100
checkpoint:
kbest_n: 50
latest_n: 5
examples/aishell/asr1/conf/tuning/chunk_decode.yaml 0 → 100644
浏览文件 @ 4cab9f62
beam_size: 10
decode_batch_size: 128
error_rate_type: cer
decoding_method: attention # 'attention', 'ctc_greedy_search', 'ctc_prefix_beam_search', 'attention_rescoring'
ctc_weight: 0.5 # ctc weight for attention rescoring decode mode.
decoding_chunk_size: -1 # decoding chunk size. Defaults to -1.
# <0: for decoding, use full chunk.
# >0: for decoding, use fixed chunk size as set.
# 0: used for training, it's prohibited here.
num_decoding_left_chunks: -1 # number of left chunks for decoding. Defaults to -1.
simulate_streaming: False # simulate streaming inference. Defaults to False.
examples/aishell/asr1/conf/tuning/decode.yaml 0 → 100644
浏览文件 @ 4cab9f62
beam_size: 10
decode_batch_size: 128
error_rate_type: cer
decoding_method: attention # 'attention', 'ctc_greedy_search', 'ctc_prefix_beam_search', 'attention_rescoring'
ctc_weight: 0.5 # ctc weight for attention rescoring decode mode.
decoding_chunk_size: -1 # decoding chunk size. Defaults to -1.
# <0: for decoding, use full chunk.
# >0: for decoding, use fixed chunk size as set.
# 0: used for training, it's prohibited here.
num_decoding_left_chunks: -1 # number of left chunks for decoding. Defaults to -1.
simulate_streaming: False # simulate streaming inference. Defaults to False.
examples/aishell/asr1/local/align.sh
浏览文件 @ 4cab9f62
#!/bin/bash #!/bin/bash
if [ $# != 2 ];then if [ $# != 3 ];then
echo "usage: ${0} config_path ckpt_path_prefix" echo "usage: ${0} config_path decode_config_path ckpt_path_prefix"
exit -1 exit -1
fi fi
...@@ -9,7 +9,8 @@ ngpu=$(echo $CUDA_VISIBLE_DEVICES | awk -F "," '{print NF}') ...@@ -9,7 +9,8 @@ ngpu=$(echo $CUDA_VISIBLE_DEVICES | awk -F "," '{print NF}')
echo "using $ngpu gpus..." echo "using $ngpu gpus..."
config_path=$1 config_path=$1
ckpt_prefix=$2 decode_config_path=$2
ckpt_prefix=$3
batch_size=1 batch_size=1
output_dir=${ckpt_prefix} output_dir=${ckpt_prefix}
...@@ -20,9 +21,10 @@ mkdir -p ${output_dir} ...@@ -20,9 +21,10 @@ mkdir -p ${output_dir}
python3 -u ${BIN_DIR}/alignment.py \ python3 -u ${BIN_DIR}/alignment.py \
--ngpu ${ngpu} \ --ngpu ${ngpu} \
--config ${config_path} \ --config ${config_path} \
--decode_cfg ${decode_config_path} \
--result_file ${output_dir}/${type}.align \ --result_file ${output_dir}/${type}.align \
--checkpoint_path ${ckpt_prefix} \ --checkpoint_path ${ckpt_prefix} \
--opts decoding.batch_size ${batch_size} --opts decode.decode_batch_size ${batch_size}
if [ $? -ne 0 ]; then if [ $? -ne 0 ]; then
echo "Failed in ctc alignment!" echo "Failed in ctc alignment!"
......
examples/aishell/asr1/local/test.sh
浏览文件 @ 4cab9f62
#!/bin/bash #!/bin/bash
if [ $# != 2 ];then if [ $# != 3 ];then
echo "usage: ${0} config_path ckpt_path_prefix" echo "usage: ${0} config_path decode_config_path ckpt_path_prefix"
exit -1 exit -1
fi fi
...@@ -9,7 +9,8 @@ ngpu=$(echo $CUDA_VISIBLE_DEVICES | awk -F "," '{print NF}') ...@@ -9,7 +9,8 @@ ngpu=$(echo $CUDA_VISIBLE_DEVICES | awk -F "," '{print NF}')
echo "using $ngpu gpus..." echo "using $ngpu gpus..."
config_path=$1 config_path=$1
ckpt_prefix=$2 decode_config_path=$2
ckpt_prefix=$3
chunk_mode=false chunk_mode=false
if [[ ${config_path} =~ ^.*chunk_.*yaml$ ]];then if [[ ${config_path} =~ ^.*chunk_.*yaml$ ]];then
...@@ -36,10 +37,11 @@ for type in attention ctc_greedy_search; do ...@@ -36,10 +37,11 @@ for type in attention ctc_greedy_search; do
python3 -u ${BIN_DIR}/test.py \ python3 -u ${BIN_DIR}/test.py \
--ngpu ${ngpu} \ --ngpu ${ngpu} \
--config ${config_path} \ --config ${config_path} \
--decode_cfg ${decode_config_path} \
--result_file ${output_dir}/${type}.rsl \ --result_file ${output_dir}/${type}.rsl \
--checkpoint_path ${ckpt_prefix} \ --checkpoint_path ${ckpt_prefix} \
--opts decoding.decoding_method ${type} \ --opts decode.decoding_method ${type} \
--opts decoding.batch_size ${batch_size} --opts decode.decode_batch_size ${batch_size}
if [ $? -ne 0 ]; then if [ $? -ne 0 ]; then
echo "Failed in evaluation!" echo "Failed in evaluation!"
...@@ -55,10 +57,11 @@ for type in ctc_prefix_beam_search attention_rescoring; do ...@@ -55,10 +57,11 @@ for type in ctc_prefix_beam_search attention_rescoring; do
python3 -u ${BIN_DIR}/test.py \ python3 -u ${BIN_DIR}/test.py \
--ngpu ${ngpu} \ --ngpu ${ngpu} \
--config ${config_path} \ --config ${config_path} \
--decode_cfg ${decode_config_path} \
--result_file ${output_dir}/${type}.rsl \ --result_file ${output_dir}/${type}.rsl \
--checkpoint_path ${ckpt_prefix} \ --checkpoint_path ${ckpt_prefix} \
--opts decoding.decoding_method ${type} \ --opts decode.decoding_method ${type} \
--opts decoding.batch_size ${batch_size} --opts decode.decode_batch_size ${batch_size}
if [ $? -ne 0 ]; then if [ $? -ne 0 ]; then
echo "Failed in evaluation!" echo "Failed in evaluation!"
......
examples/aishell/asr1/local/test_wav.sh
浏览文件 @ 4cab9f62
#!/bin/bash #!/bin/bash
if [ $# != 3 ];then if [ $# != 4 ];then
echo "usage: ${0} config_path ckpt_path_prefix audio_file" echo "usage: ${0} config_path decode_config_path ckpt_path_prefix audio_file"
exit -1 exit -1
fi fi
...@@ -9,8 +9,9 @@ ngpu=$(echo $CUDA_VISIBLE_DEVICES | awk -F "," '{print NF}') ...@@ -9,8 +9,9 @@ ngpu=$(echo $CUDA_VISIBLE_DEVICES | awk -F "," '{print NF}')
echo "using $ngpu gpus..." echo "using $ngpu gpus..."
config_path=$1 config_path=$1
ckpt_prefix=$2 decode_config_path=$2
audio_file=$3 ckpt_prefix=$3
audio_file=$4
mkdir -p data mkdir -p data
wget -nc https://paddlespeech.bj.bcebos.com/datasets/single_wav/zh/demo_01_03.wav -P data/ wget -nc https://paddlespeech.bj.bcebos.com/datasets/single_wav/zh/demo_01_03.wav -P data/
...@@ -42,10 +43,11 @@ for type in attention_rescoring; do ...@@ -42,10 +43,11 @@ for type in attention_rescoring; do
python3 -u ${BIN_DIR}/test_wav.py \ python3 -u ${BIN_DIR}/test_wav.py \
--ngpu ${ngpu} \ --ngpu ${ngpu} \
--config ${config_path} \ --config ${config_path} \
--decode_cfg ${decode_config_path} \
--result_file ${output_dir}/${type}.rsl \ --result_file ${output_dir}/${type}.rsl \
--checkpoint_path ${ckpt_prefix} \ --checkpoint_path ${ckpt_prefix} \
--opts decoding.decoding_method ${type} \ --opts decode.decoding_method ${type} \
--opts decoding.batch_size ${batch_size} \ --opts decode.decode_batch_size ${batch_size} \
--audio_file ${audio_file} --audio_file ${audio_file}
if [ $? -ne 0 ]; then if [ $? -ne 0 ]; then
......
examples/aishell/asr1/run.sh
浏览文件 @ 4cab9f62
...@@ -6,6 +6,7 @@ gpus=0,1,2,3 ...@@ -6,6 +6,7 @@ gpus=0,1,2,3
stage=0 stage=0
stop_stage=50 stop_stage=50
conf_path=conf/conformer.yaml conf_path=conf/conformer.yaml
decode_conf_path=conf/tuning/decode.yaml
avg_num=20 avg_num=20
audio_file=data/demo_01_03.wav audio_file=data/demo_01_03.wav
...@@ -32,18 +33,18 @@ fi ...@@ -32,18 +33,18 @@ fi
if [ ${stage} -le 3 ] && [ ${stop_stage} -ge 3 ]; then if [ ${stage} -le 3 ] && [ ${stop_stage} -ge 3 ]; then
# test ckpt avg_n # test ckpt avg_n
CUDA_VISIBLE_DEVICES=0 ./local/test.sh ${conf_path} exp/${ckpt}/checkpoints/${avg_ckpt} || exit -1 CUDA_VISIBLE_DEVICES=0 ./local/test.sh ${conf_path} ${decode_conf_path} exp/${ckpt}/checkpoints/${avg_ckpt} || exit -1
fi fi
if [ ${stage} -le 4 ] && [ ${stop_stage} -ge 4 ]; then if [ ${stage} -le 4 ] && [ ${stop_stage} -ge 4 ]; then
# ctc alignment of test data # ctc alignment of test data
CUDA_VISIBLE_DEVICES=0 ./local/align.sh ${conf_path} exp/${ckpt}/checkpoints/${avg_ckpt} || exit -1 CUDA_VISIBLE_DEVICES=0 ./local/align.sh ${conf_path} ${decode_conf_path} exp/${ckpt}/checkpoints/${avg_ckpt} || exit -1
fi fi
# Optionally, you can add LM and test it with runtime. # Optionally, you can add LM and test it with runtime.
if [ ${stage} -le 5 ] && [ ${stop_stage} -ge 5 ]; then if [ ${stage} -le 5 ] && [ ${stop_stage} -ge 5 ]; then
# test a single .wav file # test a single .wav file
CUDA_VISIBLE_DEVICES=0 ./local/test_wav.sh ${conf_path} exp/${ckpt}/checkpoints/${avg_ckpt} ${audio_file} || exit -1 CUDA_VISIBLE_DEVICES=0 ./local/test_wav.sh ${conf_path} ${decode_conf_path} exp/${ckpt}/checkpoints/${avg_ckpt} ${audio_file} || exit -1
fi fi
# Not supported at now!!! # Not supported at now!!!
......
examples/callcenter/asr1/conf/chunk_conformer.yaml
浏览文件 @ 4cab9f62
# https://yaml.org/type/float.html # https://yaml.org/type/float.html
data: ###########################################
train_manifest: data/manifest.train # Data #
dev_manifest: data/manifest.dev ###########################################
test_manifest: data/manifest.test train_manifest: data/manifest.train
min_input_len: 0.5 dev_manifest: data/manifest.dev
max_input_len: 20.0 # second test_manifest: data/manifest.test
min_output_len: 0.0
max_output_len: 400.0
min_output_input_ratio: 0.05
max_output_input_ratio: 10.0
collator:
vocab_filepath: data/lang_char/vocab.txt
unit_type: 'char'
spm_model_prefix: ''
augmentation_config: conf/preprocess.yaml
batch_size: 32
raw_wav: True # use raw_wav or kaldi feature
spectrum_type: fbank #linear, mfcc, fbank
feat_dim: 80
delta_delta: False
dither: 1.0
target_sample_rate: 8000
max_freq: None
n_fft: None
stride_ms: 10.0
window_ms: 25.0
use_dB_normalization: True
target_dB: -20
random_seed: 0
keep_transcription_text: False
sortagrad: True
shuffle_method: batch_shuffle
num_workers: 2
# network architecture
model:
cmvn_file:
cmvn_file_type: "json"
# encoder related
encoder: conformer
encoder_conf:
output_size: 256 # dimension of attention
attention_heads: 4
linear_units: 2048 # the number of units of position-wise feed forward
num_blocks: 12 # the number of encoder blocks
dropout_rate: 0.1
positional_dropout_rate: 0.1
attention_dropout_rate: 0.0
input_layer: conv2d # encoder input type, you can chose conv2d, conv2d6 and conv2d8
normalize_before: True
use_cnn_module: True
cnn_module_kernel: 15
activation_type: 'swish'
pos_enc_layer_type: 'rel_pos'
selfattention_layer_type: 'rel_selfattn'
causal: true
use_dynamic_chunk: true
cnn_module_norm: 'layer_norm' # using nn.LayerNorm makes model converge faster
use_dynamic_left_chunk: false
# decoder related
decoder: transformer
decoder_conf:
attention_heads: 4
linear_units: 2048
num_blocks: 6
dropout_rate: 0.1
positional_dropout_rate: 0.1
self_attention_dropout_rate: 0.0
src_attention_dropout_rate: 0.0
# hybrid CTC/attention
model_conf:
ctc_weight: 0.3
lsm_weight: 0.1 # label smoothing option
length_normalized_loss: false
###########################################
# Dataloader #
###########################################
vocab_filepath: data/lang_char/vocab.txt
unit_type: 'char'
spm_model_prefix: ''
preprocess_config: conf/preprocess.yaml
batch_size: 32
raw_wav: True # use raw_wav or kaldi feature
spectrum_type: fbank #linear, mfcc, fbank
feat_dim: 80
delta_delta: False
dither: 1.0
target_sample_rate: 8000
max_freq: None
n_fft: None
stride_ms: 10.0
window_ms: 25.0
use_dB_normalization: True
target_dB: -20
random_seed: 0
keep_transcription_text: False
sortagrad: True
shuffle_method: batch_shuffle
num_workers: 2
training:
n_epoch: 240
accum_grad: 4
global_grad_clip: 5.0
optim: adam
optim_conf:
lr: 0.001
weight_decay: 1e-6
scheduler: warmuplr
scheduler_conf:
warmup_steps: 25000
lr_decay: 1.0
log_interval: 100
checkpoint:
kbest_n: 50
latest_n: 5
############################################
# Network Architecture #
############################################
cmvn_file:
cmvn_file_type: "json"
# encoder related
encoder: conformer
encoder_conf:
output_size: 256 # dimension of attention
attention_heads: 4
linear_units: 2048 # the number of units of position-wise feed forward
num_blocks: 12 # the number of encoder blocks
dropout_rate: 0.1
positional_dropout_rate: 0.1
attention_dropout_rate: 0.0
input_layer: conv2d # encoder input type, you can chose conv2d, conv2d6 and conv2d8
normalize_before: True
use_cnn_module: True
cnn_module_kernel: 15
activation_type: 'swish'
pos_enc_layer_type: 'rel_pos'
selfattention_layer_type: 'rel_selfattn'
causal: true
use_dynamic_chunk: true
cnn_module_norm: 'layer_norm' # using nn.LayerNorm makes model converge faster
use_dynamic_left_chunk: false
decoding: # decoder related
batch_size: 128 decoder: transformer
error_rate_type: cer decoder_conf:
decoding_method: attention # 'attention', 'ctc_greedy_search', 'ctc_prefix_beam_search', 'attention_rescoring' attention_heads: 4
lang_model_path: data/lm/common_crawl_00.prune01111.trie.klm linear_units: 2048
alpha: 2.5 num_blocks: 6
beta: 0.3 dropout_rate: 0.1
beam_size: 10 positional_dropout_rate: 0.1
cutoff_prob: 1.0 self_attention_dropout_rate: 0.0
cutoff_top_n: 0 src_attention_dropout_rate: 0.0
num_proc_bsearch: 8
ctc_weight: 0.5 # ctc weight for attention rescoring decode mode.
decoding_chunk_size: -1 # decoding chunk size. Defaults to -1.
# <0: for decoding, use full chunk.
# >0: for decoding, use fixed chunk size as set.
# 0: used for training, it's prohibited here.
num_decoding_left_chunks: -1 # number of left chunks for decoding. Defaults to -1.
simulate_streaming: true # simulate streaming inference. Defaults to False.
# hybrid CTC/attention
model_conf:
ctc_weight: 0.3
lsm_weight: 0.1 # label smoothing option
length_normalized_loss: false
###########################################
# Training #
###########################################
n_epoch: 240
accum_grad: 4
global_grad_clip: 5.0
optim: adam
optim_conf:
lr: 0.001
weight_decay: 1.0e-6
scheduler: warmuplr
scheduler_conf:
warmup_steps: 25000
lr_decay: 1.0
log_interval: 100
checkpoint:
kbest_n: 50
latest_n: 5
examples/callcenter/asr1/conf/conformer.yaml
浏览文件 @ 4cab9f62
# https://yaml.org/type/float.html # https://yaml.org/type/float.html
data: ###########################################
train_manifest: data/manifest.train # Data #
dev_manifest: data/manifest.dev ###########################################
test_manifest: data/manifest.test train_manifest: data/manifest.train
min_input_len: 0.5 dev_manifest: data/manifest.dev
max_input_len: 20.0 # second test_manifest: data/manifest.test
min_output_len: 0.0
max_output_len: 400.0
min_output_input_ratio: 0.0
max_output_input_ratio: .inf
collator: ###########################################
vocab_filepath: data/lang_char/vocab.txt # Dataloader #
unit_type: 'char' ###########################################
spm_model_prefix: '' vocab_filepath: data/lang_char/vocab.txt
augmentation_config: conf/preprocess.yaml unit_type: 'char'
batch_size: 32 spm_model_prefix: ''
raw_wav: True # use raw_wav or kaldi feature preprocess_config: conf/preprocess.yaml
spectrum_type: fbank #linear, mfcc, fbank feat_dim: 80
feat_dim: 80 stride_ms: 10.0
delta_delta: False window_ms: 25.0
dither: 1.0 sortagrad: 0 # Feed samples from shortest to longest ; -1: enabled for all epochs, 0: disabled, other: enabled for 'other' epochs
target_sample_rate: 8000 batch_size: 64
max_freq: None maxlen_in: 512 # if input length > maxlen-in, batchsize is automatically reduced
n_fft: None maxlen_out: 150 # if output length > maxlen-out, batchsize is automatically reduced
stride_ms: 10.0 minibatches: 0 # for debug
window_ms: 25.0 batch_count: auto
use_dB_normalization: True batch_bins: 0
target_dB: -20 batch_frames_in: 0
random_seed: 0 batch_frames_out: 0
keep_transcription_text: False batch_frames_inout: 0
sortagrad: True num_workers: 0
shuffle_method: batch_shuffle subsampling_factor: 1
num_workers: 2 num_encs: 1
# network architecture ############################################
model: # Network Architecture #
cmvn_file: ############################################
cmvn_file_type: "json" cmvn_file:
# encoder related cmvn_file_type: "json"
encoder: conformer # encoder related
encoder_conf: encoder: conformer
output_size: 256 # dimension of attention encoder_conf:
attention_heads: 4 output_size: 256 # dimension of attention
linear_units: 2048 # the number of units of position-wise feed forward attention_heads: 4
num_blocks: 12 # the number of encoder blocks linear_units: 2048 # the number of units of position-wise feed forward
dropout_rate: 0.1 num_blocks: 12 # the number of encoder blocks
positional_dropout_rate: 0.1 dropout_rate: 0.1
attention_dropout_rate: 0.0 positional_dropout_rate: 0.1
input_layer: conv2d # encoder input type, you can chose conv2d, conv2d6 and conv2d8 attention_dropout_rate: 0.0
normalize_before: True input_layer: conv2d # encoder input type, you can chose conv2d, conv2d6 and conv2d8
use_cnn_module: True normalize_before: True
cnn_module_kernel: 15 use_cnn_module: True
activation_type: 'swish' cnn_module_kernel: 15
pos_enc_layer_type: 'rel_pos' activation_type: 'swish'
selfattention_layer_type: 'rel_selfattn' pos_enc_layer_type: 'rel_pos'
selfattention_layer_type: 'rel_selfattn'
# decoder related # decoder related
decoder: transformer decoder: transformer
decoder_conf: decoder_conf:
attention_heads: 4 attention_heads: 4
linear_units: 2048 linear_units: 2048
num_blocks: 6 num_blocks: 6
dropout_rate: 0.1 dropout_rate: 0.1
positional_dropout_rate: 0.1 positional_dropout_rate: 0.1
self_attention_dropout_rate: 0.0 self_attention_dropout_rate: 0.0
src_attention_dropout_rate: 0.0 src_attention_dropout_rate: 0.0
# hybrid CTC/attention # hybrid CTC/attention
model_conf: model_conf:
ctc_weight: 0.3 ctc_weight: 0.3
lsm_weight: 0.1 # label smoothing option lsm_weight: 0.1 # label smoothing option
length_normalized_loss: false length_normalized_loss: false
training:
n_epoch: 100 # 50 will be lowest
accum_grad: 4
global_grad_clip: 5.0
optim: adam
optim_conf:
lr: 0.002
weight_decay: 1e-6
scheduler: warmuplr
scheduler_conf:
warmup_steps: 25000
lr_decay: 1.0
log_interval: 100
checkpoint:
kbest_n: 50
latest_n: 5
decoding:
batch_size: 128
error_rate_type: cer
decoding_method: attention # 'attention', 'ctc_greedy_search', 'ctc_prefix_beam_search', 'attention_rescoring'
lang_model_path: data/lm/common_crawl_00.prune01111.trie.klm
alpha: 2.5
beta: 0.3
beam_size: 10
cutoff_prob: 1.0
cutoff_top_n: 0
num_proc_bsearch: 8
ctc_weight: 0.5 # ctc weight for attention rescoring decode mode.
decoding_chunk_size: -1 # decoding chunk size. Defaults to -1.
# <0: for decoding, use full chunk.
# >0: for decoding, use fixed chunk size as set.
# 0: used for training, it's prohibited here.
num_decoding_left_chunks: -1 # number of left chunks for decoding. Defaults to -1.
simulate_streaming: False # simulate streaming inference. Defaults to False.
###########################################
# Training #
###########################################
n_epoch: 100 # 50 will be lowest
accum_grad: 4
global_grad_clip: 5.0
optim: adam
optim_conf:
lr: 0.002
weight_decay: 1.0e-6
scheduler: warmuplr
scheduler_conf:
warmup_steps: 25000
lr_decay: 1.0
log_interval: 100
checkpoint:
kbest_n: 50
latest_n: 5
examples/callcenter/asr1/conf/preprocess.yaml
浏览文件 @ 4cab9f62
process: process:
# extract kaldi fbank from PCM # extract kaldi fbank from PCM
- type: fbank_kaldi - type: fbank_kaldi
fs: 16000 fs: 8000
n_mels: 80 n_mels: 80
n_shift: 160 n_shift: 160
win_length: 400 win_length: 400
......
examples/callcenter/asr1/conf/tuning/chunk_decode.yaml 0 → 100644
浏览文件 @ 4cab9f62
decode_batch_size: 128
error_rate_type: cer
decoding_method: attention # 'attention', 'ctc_greedy_search', 'ctc_prefix_beam_search', 'attention_rescoring'
beam_size: 10
ctc_weight: 0.5 # ctc weight for attention rescoring decode mode.
decoding_chunk_size: -1 # decoding chunk size. Defaults to -1.
# <0: for decoding, use full chunk.
# >0: for decoding, use fixed chunk size as set.
# 0: used for training, it's prohibited here.
num_decoding_left_chunks: -1 # number of left chunks for decoding. Defaults to -1.
simulate_streaming: true # simulate streaming inference. Defaults to False.
\ No newline at end of file
examples/callcenter/asr1/conf/tuning/decode.yaml 0 → 100644
浏览文件 @ 4cab9f62
decode_batch_size: 128
error_rate_type: cer
decoding_method: attention # 'attention', 'ctc_greedy_search', 'ctc_prefix_beam_search', 'attention_rescoring'
beam_size: 10
ctc_weight: 0.5 # ctc weight for attention rescoring decode mode.
decoding_chunk_size: -1 # decoding chunk size. Defaults to -1.
# <0: for decoding, use full chunk.
# >0: for decoding, use fixed chunk size as set.
# 0: used for training, it's prohibited here.
num_decoding_left_chunks: -1 # number of left chunks for decoding. Defaults to -1.
simulate_streaming: False # simulate streaming inference. Defaults to False.
examples/callcenter/asr1/local/align.sh
浏览文件 @ 4cab9f62
#! /usr/bin/env bash #! /usr/bin/env bash
if [ $# != 2 ];then if [ $# != 3 ];then
echo "usage: ${0} config_path ckpt_path_prefix" echo "usage: ${0} config_path decode_config_path ckpt_path_prefix"
exit -1 exit -1
fi fi
...@@ -9,7 +9,8 @@ ngpu=$(echo $CUDA_VISIBLE_DEVICES | awk -F "," '{print NF}') ...@@ -9,7 +9,8 @@ ngpu=$(echo $CUDA_VISIBLE_DEVICES | awk -F "," '{print NF}')
echo "using $ngpu gpus..." echo "using $ngpu gpus..."
config_path=$1 config_path=$1
ckpt_prefix=$2 decode_config_path=$2
ckpt_prefix=$3
ckpt_name=$(basename ${ckpt_prefxi}) ckpt_name=$(basename ${ckpt_prefxi})
...@@ -25,9 +26,10 @@ mkdir -p ${output_dir} ...@@ -25,9 +26,10 @@ mkdir -p ${output_dir}
python3 -u ${BIN_DIR}/alignment.py \ python3 -u ${BIN_DIR}/alignment.py \
--ngpu ${ngpu} \ --ngpu ${ngpu} \
--config ${config_path} \ --config ${config_path} \
--decode_cfg ${decode_config_path} \
--result_file ${output_dir}/${type}.align \ --result_file ${output_dir}/${type}.align \
--checkpoint_path ${ckpt_prefix} \ --checkpoint_path ${ckpt_prefix} \
--opts decoding.batch_size ${batch_size} --opts decode.decode_batch_size ${batch_size}
if [ $? -ne 0 ]; then if [ $? -ne 0 ]; then
echo "Failed in ctc alignment!" echo "Failed in ctc alignment!"
......
examples/callcenter/asr1/local/test.sh
浏览文件 @ 4cab9f62
#! /usr/bin/env bash #! /usr/bin/env bash
if [ $# != 2 ];then if [ $# != 3 ];then
echo "usage: ${0} config_path ckpt_path_prefix" echo "usage: ${0} config_path decode_config_path ckpt_path_prefix"
exit -1 exit -1
fi fi
...@@ -9,7 +9,9 @@ ngpu=$(echo $CUDA_VISIBLE_DEVICES | awk -F "," '{print NF}') ...@@ -9,7 +9,9 @@ ngpu=$(echo $CUDA_VISIBLE_DEVICES | awk -F "," '{print NF}')
echo "using $ngpu gpus..." echo "using $ngpu gpus..."
config_path=$1 config_path=$1
ckpt_prefix=$2 decode_config_path=$2
ckpt_prefix=$3
ckpt_name=$(basename ${ckpt_prefxi}) ckpt_name=$(basename ${ckpt_prefxi})
...@@ -30,10 +32,11 @@ for type in attention ctc_greedy_search; do ...@@ -30,10 +32,11 @@ for type in attention ctc_greedy_search; do
python3 -u ${BIN_DIR}/test.py \ python3 -u ${BIN_DIR}/test.py \
--ngpu ${ngpu} \ --ngpu ${ngpu} \
--config ${config_path} \ --config ${config_path} \
--decode_cfg ${decode_config_path} \
--result_file ${output_dir}/${type}.rsl \ --result_file ${output_dir}/${type}.rsl \
--checkpoint_path ${ckpt_prefix} \ --checkpoint_path ${ckpt_prefix} \
--opts decoding.decoding_method ${type} \ --opts decode.decoding_method ${type} \
--opts decoding.batch_size ${batch_size} --opts decode.decode_batch_size ${batch_size}
if [ $? -ne 0 ]; then if [ $? -ne 0 ]; then
echo "Failed in evaluation!" echo "Failed in evaluation!"
...@@ -49,10 +52,11 @@ for type in ctc_prefix_beam_search attention_rescoring; do ...@@ -49,10 +52,11 @@ for type in ctc_prefix_beam_search attention_rescoring; do
python3 -u ${BIN_DIR}/test.py \ python3 -u ${BIN_DIR}/test.py \
--ngpu ${ngpu} \ --ngpu ${ngpu} \
--config ${config_path} \ --config ${config_path} \
--decode_cfg ${decode_config_path} \
--result_file ${output_dir}/${type}.rsl \ --result_file ${output_dir}/${type}.rsl \
--checkpoint_path ${ckpt_prefix} \ --checkpoint_path ${ckpt_prefix} \
--opts decoding.decoding_method ${type} \ --opts decode.decoding_method ${type} \
--opts decoding.batch_size ${batch_size} --opts decode.decode_batch_size ${batch_size}
if [ $? -ne 0 ]; then if [ $? -ne 0 ]; then
echo "Failed in evaluation!" echo "Failed in evaluation!"
......
examples/callcenter/asr1/run.sh
浏览文件 @ 4cab9f62
...@@ -4,8 +4,9 @@ source path.sh ...@@ -4,8 +4,9 @@ source path.sh
gpus=0,1,2,3 gpus=0,1,2,3
stage=0 stage=0
stop_stage=100 stop_stage=50
conf_path=conf/conformer.yaml conf_path=conf/conformer.yaml
decode_conf_path=conf/tuning/decode.yaml
avg_num=20 avg_num=20
source ${MAIN_ROOT}/utils/parse_options.sh || exit 1; source ${MAIN_ROOT}/utils/parse_options.sh || exit 1;
...@@ -31,15 +32,15 @@ fi ...@@ -31,15 +32,15 @@ fi
if [ ${stage} -le 3 ] && [ ${stop_stage} -ge 3 ]; then if [ ${stage} -le 3 ] && [ ${stop_stage} -ge 3 ]; then
# test ckpt avg_n # test ckpt avg_n
CUDA_VISIBLE_DEVICES=0 ./local/test.sh ${conf_path} exp/${ckpt}/checkpoints/${avg_ckpt} || exit -1 CUDA_VISIBLE_DEVICES=0 ./local/test.sh ${conf_path} ${decode_conf_path} exp/${ckpt}/checkpoints/${avg_ckpt} || exit -1
fi fi
if [ ${stage} -le 4 ] && [ ${stop_stage} -ge 4 ]; then if [ ${stage} -le 4 ] && [ ${stop_stage} -ge 4 ]; then
# ctc alignment of test data # ctc alignment of test data
CUDA_VISIBLE_DEVICES=0 ./local/align.sh ${conf_path} exp/${ckpt}/checkpoints/${avg_ckpt} || exit -1 CUDA_VISIBLE_DEVICES=0 ./local/align.sh ${conf_path} ${decode_conf_path} exp/${ckpt}/checkpoints/${avg_ckpt} || exit -1
fi fi
if [ ${stage} -le 5 ] && [ ${stop_stage} -ge 5 ]; then if [ ${stage} -le 51 ] && [ ${stop_stage} -ge 51 ]; then
# export ckpt avg_n # export ckpt avg_n
CUDA_VISIBLE_DEVICES= ./local/export.sh ${conf_path} exp/${ckpt}/checkpoints/${avg_ckpt} exp/${ckpt}/checkpoints/${avg_ckpt}.jit CUDA_VISIBLE_DEVICES= ./local/export.sh ${conf_path} exp/${ckpt}/checkpoints/${avg_ckpt} exp/${ckpt}/checkpoints/${avg_ckpt}.jit
fi fi
examples/librispeech/asr0/conf/deepspeech2.yaml
浏览文件 @ 4cab9f62
# https://yaml.org/type/float.html # https://yaml.org/type/float.html
data: ###########################################
train_manifest: data/manifest.train # Data #
dev_manifest: data/manifest.dev-clean ###########################################
test_manifest: data/manifest.test-clean train_manifest: data/manifest.train
min_input_len: 0.0 dev_manifest: data/manifest.dev-clean
max_input_len: 30.0 # second test_manifest: data/manifest.test-clean
min_output_len: 0.0 min_input_len: 0.0
max_output_len: .inf max_input_len: 30.0 # second
min_output_input_ratio: 0.00 min_output_len: 0.0
max_output_input_ratio: .inf max_output_len: .inf
min_output_input_ratio: 0.00
max_output_input_ratio: .inf
collator: ###########################################
batch_size: 20 # Dataloader #
mean_std_filepath: data/mean_std.json ###########################################
unit_type: char batch_size: 20
vocab_filepath: data/lang_char/vocab.txt mean_std_filepath: data/mean_std.json
augmentation_config: conf/augmentation.json unit_type: char
random_seed: 0 vocab_filepath: data/lang_char/vocab.txt
spm_model_prefix: augmentation_config: conf/augmentation.json
spectrum_type: linear random_seed: 0
target_sample_rate: 16000 spm_model_prefix:
max_freq: None spectrum_type: linear
n_fft: None feat_dim:
stride_ms: 10.0 target_sample_rate: 16000
window_ms: 20.0 max_freq: None
delta_delta: False n_fft: None
dither: 1.0 stride_ms: 10.0
use_dB_normalization: True window_ms: 20.0
target_dB: -20 delta_delta: False
random_seed: 0 dither: 1.0
keep_transcription_text: False use_dB_normalization: True
sortagrad: True target_dB: -20
shuffle_method: batch_shuffle random_seed: 0
num_workers: 2 keep_transcription_text: False
sortagrad: True
shuffle_method: batch_shuffle
num_workers: 2
model: ############################################
num_conv_layers: 2 # Network Architecture #
num_rnn_layers: 3 ############################################
rnn_layer_size: 2048 num_conv_layers: 2
use_gru: False num_rnn_layers: 3
share_rnn_weights: True rnn_layer_size: 2048
blank_id: 0 use_gru: False
share_rnn_weights: True
blank_id: 0
training: ###########################################
n_epoch: 50 # Training #
accum_grad: 1 ###########################################
lr: 1e-3 n_epoch: 50
lr_decay: 0.83 accum_grad: 1
weight_decay: 1e-06 lr: 1e-3
global_grad_clip: 5.0 lr_decay: 0.83
log_interval: 100 weight_decay: 1e-06
checkpoint: global_grad_clip: 5.0
kbest_n: 50 log_interval: 100
latest_n: 5 checkpoint:
kbest_n: 50
decoding: latest_n: 5
batch_size: 128
error_rate_type: wer
decoding_method: ctc_beam_search
lang_model_path: data/lm/common_crawl_00.prune01111.trie.klm
alpha: 1.9
beta: 0.3
beam_size: 500
cutoff_prob: 1.0
cutoff_top_n: 40
num_proc_bsearch: 8
examples/librispeech/asr0/conf/deepspeech2_online.yaml
浏览文件 @ 4cab9f62
# https://yaml.org/type/float.html # https://yaml.org/type/float.html
data: ###########################################
train_manifest: data/manifest.train # Data #
dev_manifest: data/manifest.dev-clean ###########################################
test_manifest: data/manifest.test-clean train_manifest: data/manifest.train
min_input_len: 0.0 dev_manifest: data/manifest.dev-clean
max_input_len: 30.0 # second test_manifest: data/manifest.test-clean
min_output_len: 0.0 min_input_len: 0.0
max_output_len: .inf max_input_len: 30.0 # second
min_output_input_ratio: 0.00 min_output_len: 0.0
max_output_input_ratio: .inf max_output_len: .inf
min_output_input_ratio: 0.00
max_output_input_ratio: .inf
collator: ###########################################
batch_size: 15 # Dataloader #
mean_std_filepath: data/mean_std.json ###########################################
unit_type: char batch_size: 15
vocab_filepath: data/lang_char/vocab.txt mean_std_filepath: data/mean_std.json
augmentation_config: conf/augmentation.json unit_type: char
random_seed: 0 vocab_filepath: data/lang_char/vocab.txt
spm_model_prefix: augmentation_config: conf/augmentation.json
spectrum_type: linear random_seed: 0
target_sample_rate: 16000 spm_model_prefix:
max_freq: None spectrum_type: linear
n_fft: None feat_dim:
stride_ms: 10.0 target_sample_rate: 16000
window_ms: 20.0 max_freq: None
delta_delta: False n_fft: None
dither: 1.0 stride_ms: 10.0
use_dB_normalization: True window_ms: 20.0
target_dB: -20 delta_delta: False
random_seed: 0 dither: 1.0
keep_transcription_text: False use_dB_normalization: True
sortagrad: True target_dB: -20
shuffle_method: batch_shuffle random_seed: 0
num_workers: 0 keep_transcription_text: False
sortagrad: True
shuffle_method: batch_shuffle
num_workers: 0
model: ############################################
num_conv_layers: 2 # Network Architecture #
num_rnn_layers: 3 ############################################
rnn_layer_size: 2048 num_conv_layers: 2
rnn_direction: forward num_rnn_layers: 3
num_fc_layers: 2 rnn_layer_size: 2048
fc_layers_size_list: 512, 256 rnn_direction: forward
use_gru: False num_fc_layers: 2
blank_id: 0 fc_layers_size_list: 512, 256
use_gru: False
blank_id: 0
training: ###########################################
n_epoch: 50 # Training #
accum_grad: 4 ###########################################
lr: 1e-3 n_epoch: 50
lr_decay: 0.83 accum_grad: 4
weight_decay: 1e-06 lr: 1e-3
global_grad_clip: 5.0 lr_decay: 0.83
log_interval: 100 weight_decay: 1e-06
checkpoint: global_grad_clip: 5.0
kbest_n: 50 log_interval: 100
latest_n: 5 checkpoint:
kbest_n: 50
decoding: latest_n: 5
batch_size: 128
error_rate_type: wer
decoding_method: ctc_beam_search
lang_model_path: data/lm/common_crawl_00.prune01111.trie.klm
alpha: 1.9
beta: 0.3
beam_size: 500
cutoff_prob: 1.0
cutoff_top_n: 40
num_proc_bsearch: 8
examples/librispeech/asr0/conf/tuning/chunk_decode.yaml 0 → 100644
浏览文件 @ 4cab9f62
decode_batch_size: 128
error_rate_type: wer
decoding_method: ctc_beam_search
lang_model_path: data/lm/common_crawl_00.prune01111.trie.klm
alpha: 1.9
beta: 0.3
beam_size: 500
cutoff_prob: 1.0
cutoff_top_n: 40
num_proc_bsearch: 8
\ No newline at end of file
examples/librispeech/asr0/conf/tuning/decode.yaml 0 → 100644
浏览文件 @ 4cab9f62
decode_batch_size: 128
error_rate_type: wer
decoding_method: ctc_beam_search
lang_model_path: data/lm/common_crawl_00.prune01111.trie.klm
alpha: 1.9
beta: 0.3
beam_size: 500
cutoff_prob: 1.0
cutoff_top_n: 40
num_proc_bsearch: 8
\ No newline at end of file
examples/librispeech/asr0/local/test.sh
浏览文件 @ 4cab9f62
#!/bin/bash #!/bin/bash
if [ $# != 3 ];then if [ $# != 4 ];then
echo "usage: ${0} config_path ckpt_path_prefix model_type" echo "usage: ${0} config_path decode_config_path ckpt_path_prefix model_type"
exit -1 exit -1
fi fi
...@@ -9,8 +9,9 @@ ngpu=$(echo $CUDA_VISIBLE_DEVICES | awk -F "," '{print NF}') ...@@ -9,8 +9,9 @@ ngpu=$(echo $CUDA_VISIBLE_DEVICES | awk -F "," '{print NF}')
echo "using $ngpu gpus..." echo "using $ngpu gpus..."
config_path=$1 config_path=$1
ckpt_prefix=$2 decode_config_path=$2
model_type=$3 ckpt_prefix=$3
model_type=$4
# download language model # download language model
bash local/download_lm_en.sh bash local/download_lm_en.sh
...@@ -21,6 +22,7 @@ fi ...@@ -21,6 +22,7 @@ fi
python3 -u ${BIN_DIR}/test.py \ python3 -u ${BIN_DIR}/test.py \
--ngpu ${ngpu} \ --ngpu ${ngpu} \
--config ${config_path} \ --config ${config_path} \
--decode_cfg ${decode_config_path} \
--result_file ${ckpt_prefix}.rsl \ --result_file ${ckpt_prefix}.rsl \
--checkpoint_path ${ckpt_prefix} \ --checkpoint_path ${ckpt_prefix} \
--model_type ${model_type} --model_type ${model_type}
......
examples/librispeech/asr0/local/test_wav.sh
浏览文件 @ 4cab9f62
#!/bin/bash #!/bin/bash
if [ $# != 4 ];then if [ $# != 5 ];then
echo "usage: ${0} config_path ckpt_path_prefix model_type audio_file" echo "usage: ${0} config_path decode_config_path ckpt_path_prefix model_type audio_file"
exit -1 exit -1
fi fi
...@@ -9,9 +9,10 @@ ngpu=$(echo $CUDA_VISIBLE_DEVICES | awk -F "," '{print NF}') ...@@ -9,9 +9,10 @@ ngpu=$(echo $CUDA_VISIBLE_DEVICES | awk -F "," '{print NF}')
echo "using $ngpu gpus..." echo "using $ngpu gpus..."
config_path=$1 config_path=$1
ckpt_prefix=$2 decode_config_path=$2
model_type=$3 ckpt_prefix=$3
audio_file=$4 model_type=$4
audio_file=$5
mkdir -p data mkdir -p data
wget -nc https://paddlespeech.bj.bcebos.com/datasets/single_wav/en/demo_002_en.wav -P data/ wget -nc https://paddlespeech.bj.bcebos.com/datasets/single_wav/en/demo_002_en.wav -P data/
...@@ -33,6 +34,7 @@ fi ...@@ -33,6 +34,7 @@ fi
python3 -u ${BIN_DIR}/test_wav.py \ python3 -u ${BIN_DIR}/test_wav.py \
--ngpu ${ngpu} \ --ngpu ${ngpu} \
--config ${config_path} \ --config ${config_path} \
--decode_cfg ${decode_config_path} \
--result_file ${ckpt_prefix}.rsl \ --result_file ${ckpt_prefix}.rsl \
--checkpoint_path ${ckpt_prefix} \ --checkpoint_path ${ckpt_prefix} \
--model_type ${model_type} \ --model_type ${model_type} \
......
examples/librispeech/asr0/run.sh
浏览文件 @ 4cab9f62
...@@ -6,6 +6,7 @@ gpus=0,1,2,3,4,5,6,7 ...@@ -6,6 +6,7 @@ gpus=0,1,2,3,4,5,6,7
stage=0 stage=0
stop_stage=100 stop_stage=100
conf_path=conf/deepspeech2.yaml conf_path=conf/deepspeech2.yaml
decode_conf_path=conf/tuning/decode.yaml
avg_num=30 avg_num=30
model_type=offline model_type=offline
audio_file=data/demo_002_en.wav audio_file=data/demo_002_en.wav
...@@ -33,7 +34,7 @@ fi ...@@ -33,7 +34,7 @@ fi
if [ ${stage} -le 3 ] && [ ${stop_stage} -ge 3 ]; then if [ ${stage} -le 3 ] && [ ${stop_stage} -ge 3 ]; then
# test ckpt avg_n # test ckpt avg_n
CUDA_VISIBLE_DEVICES=0 ./local/test.sh ${conf_path} exp/${ckpt}/checkpoints/${avg_ckpt} ${model_type} || exit -1 CUDA_VISIBLE_DEVICES=0 ./local/test.sh ${conf_path} ${decode_conf_path} exp/${ckpt}/checkpoints/${avg_ckpt} ${model_type} || exit -1
fi fi
if [ ${stage} -le 4 ] && [ ${stop_stage} -ge 4 ]; then if [ ${stage} -le 4 ] && [ ${stop_stage} -ge 4 ]; then
...@@ -43,5 +44,5 @@ fi ...@@ -43,5 +44,5 @@ fi
if [ ${stage} -le 5 ] && [ ${stop_stage} -ge 5 ]; then if [ ${stage} -le 5 ] && [ ${stop_stage} -ge 5 ]; then
# test a single .wav file # test a single .wav file
CUDA_VISIBLE_DEVICES=0 ./local/test_wav.sh ${conf_path} exp/${ckpt}/checkpoints/${avg_ckpt} ${model_type} ${audio_file} || exit -1 CUDA_VISIBLE_DEVICES=0 ./local/test_wav.sh ${conf_path} ${decode_conf_path} exp/${ckpt}/checkpoints/${avg_ckpt} ${model_type} ${audio_file} || exit -1
fi fi
examples/librispeech/asr1/conf/chunk_conformer.yaml
浏览文件 @ 4cab9f62
# network architecture ############################################
model: # Network Architecture #
cmvn_file: ############################################
cmvn_file_type: "json" cmvn_file:
# encoder related cmvn_file_type: "json"
encoder: conformer # encoder related
encoder_conf: encoder: conformer
output_size: 256 # dimension of attention encoder_conf:
attention_heads: 4 output_size: 256 # dimension of attention
linear_units: 2048 # the number of units of position-wise feed forward attention_heads: 4
num_blocks: 12 # the number of encoder blocks linear_units: 2048 # the number of units of position-wise feed forward
dropout_rate: 0.1 num_blocks: 12 # the number of encoder blocks
positional_dropout_rate: 0.1 dropout_rate: 0.1
attention_dropout_rate: 0.0 positional_dropout_rate: 0.1
input_layer: conv2d # encoder input type, you can chose conv2d, conv2d6 and conv2d8 attention_dropout_rate: 0.0
normalize_before: True input_layer: conv2d # encoder input type, you can chose conv2d, conv2d6 and conv2d8
use_cnn_module: True normalize_before: True
cnn_module_kernel: 15 use_cnn_module: True
activation_type: 'swish' cnn_module_kernel: 15
pos_enc_layer_type: 'rel_pos' activation_type: 'swish'
selfattention_layer_type: 'rel_selfattn' pos_enc_layer_type: 'rel_pos'
causal: True selfattention_layer_type: 'rel_selfattn'
use_dynamic_chunk: true causal: True
cnn_module_norm: 'layer_norm' # using nn.LayerNorm makes model converge faster use_dynamic_chunk: true
use_dynamic_left_chunk: false cnn_module_norm: 'layer_norm' # using nn.LayerNorm makes model converge faster
use_dynamic_left_chunk: false
# decoder related # decoder related
decoder: transformer decoder: transformer
decoder_conf: decoder_conf:
attention_heads: 4 attention_heads: 4
linear_units: 2048 linear_units: 2048
num_blocks: 6 num_blocks: 6
dropout_rate: 0.1 dropout_rate: 0.1
positional_dropout_rate: 0.1 positional_dropout_rate: 0.1
self_attention_dropout_rate: 0.0 self_attention_dropout_rate: 0.0
src_attention_dropout_rate: 0.0 src_attention_dropout_rate: 0.0
# hybrid CTC/attention # hybrid CTC/attention
model_conf: model_conf:
ctc_weight: 0.3 ctc_weight: 0.3
lsm_weight: 0.1 # label smoothing option lsm_weight: 0.1 # label smoothing option
length_normalized_loss: false length_normalized_loss: false
data: ###########################################
train_manifest: data/manifest.train # Data #
dev_manifest: data/manifest.dev ###########################################
test_manifest: data/manifest.test train_manifest: data/manifest.train
dev_manifest: data/manifest.dev
test_manifest: data/manifest.test
collator: ###########################################
vocab_filepath: data/lang_char/vocab.txt # Dataloader #
unit_type: 'spm' ###########################################
spm_model_prefix: 'data/lang_char/bpe_unigram_5000' vocab_filepath: data/lang_char/vocab.txt
mean_std_filepath: "" unit_type: 'spm'
augmentation_config: conf/preprocess.yaml spm_model_prefix: 'data/lang_char/bpe_unigram_5000'
feat_dim: 80 mean_std_filepath: ""
stride_ms: 10.0 preprocess_config: conf/preprocess.yaml
window_ms: 25.0 feat_dim: 80
sortagrad: 0 # Feed samples from shortest to longest ; -1: enabled for all epochs, 0: disabled, other: enabled for 'other' epochs stride_ms: 10.0
batch_size: 16 window_ms: 25.0
maxlen_in: 512 # if input length > maxlen-in, batchsize is automatically reduced sortagrad: 0 # Feed samples from shortest to longest ; -1: enabled for all epochs, 0: disabled, other: enabled for 'other' epochs
maxlen_out: 150 # if output length > maxlen-out, batchsize is automatically reduced batch_size: 16
minibatches: 0 # for debug maxlen_in: 512 # if input length > maxlen-in, batchsize is automatically reduced
batch_count: auto maxlen_out: 150 # if output length > maxlen-out, batchsize is automatically reduced
batch_bins: 0 minibatches: 0 # for debug
batch_frames_in: 0 batch_count: auto
batch_frames_out: 0 batch_bins: 0
batch_frames_inout: 0 batch_frames_in: 0
augmentation_config: conf/preprocess.yaml batch_frames_out: 0
num_workers: 0 batch_frames_inout: 0
subsampling_factor: 1 num_workers: 0
num_encs: 1 subsampling_factor: 1
num_encs: 1
###########################################
# Training #
###########################################
n_epoch: 120
accum_grad: 8
global_grad_clip: 5.0
optim: adam
optim_conf:
lr: 0.001
weight_decay: 1.0e-06
scheduler: warmuplr
scheduler_conf:
warmup_steps: 25000
lr_decay: 1.0
log_interval: 100
checkpoint:
kbest_n: 50
latest_n: 5
training:
n_epoch: 120
accum_grad: 8
global_grad_clip: 5.0
optim: adam
optim_conf:
lr: 0.001
weight_decay: 1e-06
scheduler: warmuplr
scheduler_conf:
warmup_steps: 25000
log_interval: 100
checkpoint:
kbest_n: 50
latest_n: 5
decoding:
batch_size: 128
error_rate_type: wer
decoding_method: attention # 'attention', 'ctc_greedy_search', 'ctc_prefix_beam_search', 'attention_rescoring'
beam_size: 10
ctc_weight: 0.5 # ctc weight for attention rescoring decode mode.
decoding_chunk_size: -1 # decoding chunk size. Defaults to -1.
# <0: for decoding, use full chunk.
# >0: for decoding, use fixed chunk size as set.
# 0: used for training, it's prohibited here.
num_decoding_left_chunks: -1 # number of left chunks for decoding. Defaults to -1.
simulate_streaming: true # simulate streaming inference. Defaults to False.
examples/librispeech/asr1/conf/chunk_transformer.yaml
浏览文件 @ 4cab9f62
# network architecture ############################################
model: # Network Architecture #
cmvn_file: ############################################
cmvn_file_type: "json" cmvn_file:
# encoder related cmvn_file_type: "json"
encoder: transformer # encoder related
encoder_conf: encoder: transformer
output_size: 256 # dimension of attention encoder_conf:
attention_heads: 4 output_size: 256 # dimension of attention
linear_units: 2048 # the number of units of position-wise feed forward attention_heads: 4
num_blocks: 12 # the number of encoder blocks linear_units: 2048 # the number of units of position-wise feed forward
dropout_rate: 0.1 num_blocks: 12 # the number of encoder blocks
positional_dropout_rate: 0.1 dropout_rate: 0.1
attention_dropout_rate: 0.0 positional_dropout_rate: 0.1
input_layer: conv2d # encoder input type, you can chose conv2d, conv2d6 and conv2d8 attention_dropout_rate: 0.0
normalize_before: true input_layer: conv2d # encoder input type, you can chose conv2d, conv2d6 and conv2d8
use_dynamic_chunk: true normalize_before: true
use_dynamic_left_chunk: false use_dynamic_chunk: true
use_dynamic_left_chunk: false
# decoder related # decoder related
decoder: transformer decoder: transformer
decoder_conf: decoder_conf:
attention_heads: 4 attention_heads: 4
linear_units: 2048 linear_units: 2048
num_blocks: 6 num_blocks: 6
dropout_rate: 0.1 dropout_rate: 0.1
positional_dropout_rate: 0.1 positional_dropout_rate: 0.1
self_attention_dropout_rate: 0.0 self_attention_dropout_rate: 0.0
src_attention_dropout_rate: 0.0 src_attention_dropout_rate: 0.0
# hybrid CTC/attention # hybrid CTC/attention
model_conf: model_conf:
ctc_weight: 0.3 ctc_weight: 0.3
lsm_weight: 0.1 # label smoothing option lsm_weight: 0.1 # label smoothing option
length_normalized_loss: false length_normalized_loss: false
data: ###########################################
train_manifest: data/manifest.train # Data #
dev_manifest: data/manifest.dev ###########################################
test_manifest: data/manifest.test train_manifest: data/manifest.train
dev_manifest: data/manifest.dev
test_manifest: data/manifest.test
collator: ###########################################
vocab_filepath: data/lang_char/vocab.txt # Dataloader #
unit_type: 'spm' ###########################################
spm_model_prefix: 'data/lang_char/bpe_unigram_5000' vocab_filepath: data/lang_char/vocab.txt
mean_std_filepath: "" unit_type: 'spm'
augmentation_config: conf/preprocess.yaml spm_model_prefix: 'data/lang_char/bpe_unigram_5000'
feat_dim: 80 mean_std_filepath: ""
stride_ms: 10.0 preprocess_config: conf/preprocess.yaml
window_ms: 25.0 feat_dim: 80
sortagrad: 0 # Feed samples from shortest to longest ; -1: enabled for all epochs, 0: disabled, other: enabled for 'other' epochs stride_ms: 10.0
batch_size: 64 window_ms: 25.0
maxlen_in: 512 # if input length > maxlen-in, batchsize is automatically reduced sortagrad: 0 # Feed samples from shortest to longest ; -1: enabled for all epochs, 0: disabled, other: enabled for 'other' epochs
maxlen_out: 150 # if output length > maxlen-out, batchsize is automatically reduced batch_size: 64
minibatches: 0 # for debug maxlen_in: 512 # if input length > maxlen-in, batchsize is automatically reduced
batch_count: auto maxlen_out: 150 # if output length > maxlen-out, batchsize is automatically reduced
batch_bins: 0 minibatches: 0 # for debug
batch_frames_in: 0 batch_count: auto
batch_frames_out: 0 batch_bins: 0
batch_frames_inout: 0 batch_frames_in: 0
augmentation_config: conf/preprocess.yaml batch_frames_out: 0
num_workers: 0 batch_frames_inout: 0
subsampling_factor: 1 num_workers: 0
num_encs: 1 subsampling_factor: 1
num_encs: 1
training: ###########################################
n_epoch: 120 # Training #
accum_grad: 1 ###########################################
global_grad_clip: 5.0 n_epoch: 120
optim: adam accum_grad: 1
optim_conf: global_grad_clip: 5.0
lr: 0.001 optim: adam
weight_decay: 1e-06 optim_conf:
scheduler: warmuplr lr: 0.001
scheduler_conf: weight_decay: 1.0e-06
warmup_steps: 25000 scheduler: warmuplr
lr_decay: 1.0 scheduler_conf:
log_interval: 100 warmup_steps: 25000
checkpoint: lr_decay: 1.0
kbest_n: 50 log_interval: 100
latest_n: 5 checkpoint:
kbest_n: 50
latest_n: 5
decoding: \ No newline at end of file
batch_size: 64
error_rate_type: wer
decoding_method: attention # 'attention', 'ctc_greedy_search', 'ctc_prefix_beam_search', 'attention_rescoring'
lang_model_path: data/lm/common_crawl_00.prune01111.trie.klm
alpha: 2.5
beta: 0.3
beam_size: 10
cutoff_prob: 1.0
cutoff_top_n: 0
num_proc_bsearch: 8
ctc_weight: 0.5 # ctc weight for attention rescoring decode mode.
decoding_chunk_size: -1 # decoding chunk size. Defaults to -1.
# <0: for decoding, use full chunk.
# >0: for decoding, use fixed chunk size as set.
# 0: used for training, it's prohibited here.
num_decoding_left_chunks: -1 # number of left chunks for decoding. Defaults to -1.
simulate_streaming: true # simulate streaming inference. Defaults to False.
\ No newline at end of file
examples/librispeech/asr1/conf/conformer.yaml
浏览文件 @ 4cab9f62
# network architecture ############################################
model: # Network Architecture #
cmvn_file: ############################################
cmvn_file_type: "json" cmvn_file:
# encoder related cmvn_file_type: "json"
encoder: conformer # encoder related
encoder_conf: encoder: conformer
output_size: 256 # dimension of attention encoder_conf:
attention_heads: 4 output_size: 256 # dimension of attention
linear_units: 2048 # the number of units of position-wise feed forward attention_heads: 4
num_blocks: 12 # the number of encoder blocks linear_units: 2048 # the number of units of position-wise feed forward
dropout_rate: 0.1 num_blocks: 12 # the number of encoder blocks
positional_dropout_rate: 0.1 dropout_rate: 0.1
attention_dropout_rate: 0.0 positional_dropout_rate: 0.1
input_layer: conv2d # encoder input type, you can chose conv2d, conv2d6 and conv2d8 attention_dropout_rate: 0.0
normalize_before: True input_layer: conv2d # encoder input type, you can chose conv2d, conv2d6 and conv2d8
use_cnn_module: True normalize_before: True
cnn_module_kernel: 15 use_cnn_module: True
activation_type: 'swish' cnn_module_kernel: 15
pos_enc_layer_type: 'rel_pos' activation_type: 'swish'
selfattention_layer_type: 'rel_selfattn' pos_enc_layer_type: 'rel_pos'
selfattention_layer_type: 'rel_selfattn'
# decoder related # decoder related
decoder: transformer decoder: transformer
decoder_conf: decoder_conf:
attention_heads: 4 attention_heads: 4
linear_units: 2048 linear_units: 2048
num_blocks: 6 num_blocks: 6
dropout_rate: 0.1 dropout_rate: 0.1
positional_dropout_rate: 0.1 positional_dropout_rate: 0.1
self_attention_dropout_rate: 0.0 self_attention_dropout_rate: 0.0
src_attention_dropout_rate: 0.0 src_attention_dropout_rate: 0.0
# hybrid CTC/attention # hybrid CTC/attention
model_conf: model_conf:
ctc_weight: 0.3 ctc_weight: 0.3
ctc_grad_norm_type: null ctc_grad_norm_type: null
lsm_weight: 0.1 # label smoothing option lsm_weight: 0.1 # label smoothing option
length_normalized_loss: false length_normalized_loss: false
data: ###########################################
train_manifest: data/manifest.train # Data #
dev_manifest: data/manifest.dev ###########################################
test_manifest: data/manifest.test-clean train_manifest: data/manifest.train
dev_manifest: data/manifest.dev
test_manifest: data/manifest.test-clean
collator: ###########################################
vocab_filepath: data/lang_char/vocab.txt # Dataloader #
unit_type: 'spm' ###########################################
spm_model_prefix: 'data/lang_char/bpe_unigram_5000' vocab_filepath: data/lang_char/vocab.txt
mean_std_filepath: "" unit_type: 'spm'
augmentation_config: conf/preprocess.yaml spm_model_prefix: 'data/lang_char/bpe_unigram_5000'
feat_dim: 80 mean_std_filepath: ""
stride_ms: 10.0 preprocess_config: conf/preprocess.yaml
window_ms: 25.0 feat_dim: 80
sortagrad: 0 # Feed samples from shortest to longest ; -1: enabled for all epochs, 0: disabled, other: enabled for 'other' epochs stride_ms: 10.0
batch_size: 16 window_ms: 25.0
maxlen_in: 512 # if input length > maxlen-in, batchsize is automatically reduced sortagrad: 0 # Feed samples from shortest to longest ; -1: enabled for all epochs, 0: disabled, other: enabled for 'other' epochs
maxlen_out: 150 # if output length > maxlen-out, batchsize is automatically reduced batch_size: 16
minibatches: 0 # for debug maxlen_in: 512 # if input length > maxlen-in, batchsize is automatically reduced
batch_count: auto maxlen_out: 150 # if output length > maxlen-out, batchsize is automatically reduced
batch_bins: 0 minibatches: 0 # for debug
batch_frames_in: 0 batch_count: auto
batch_frames_out: 0 batch_bins: 0
batch_frames_inout: 0 batch_frames_in: 0
augmentation_config: conf/preprocess.yaml batch_frames_out: 0
num_workers: 0 batch_frames_inout: 0
subsampling_factor: 1 num_workers: 0
num_encs: 1 subsampling_factor: 1
num_encs: 1
training: ###########################################
n_epoch: 70 # Training #
accum_grad: 8 ###########################################
global_grad_clip: 3.0 n_epoch: 70
optim: adam accum_grad: 8
optim_conf: global_grad_clip: 3.0
lr: 0.004 optim: adam
weight_decay: 1e-06 optim_conf:
scheduler: warmuplr lr: 0.004
scheduler_conf: weight_decay: 1.0e-06
warmup_steps: 25000 scheduler: warmuplr
lr_decay: 1.0 scheduler_conf:
log_interval: 100 warmup_steps: 25000
checkpoint: lr_decay: 1.0
kbest_n: 50 log_interval: 100
latest_n: 5 checkpoint:
kbest_n: 50
latest_n: 5
decoding:
batch_size: 64
error_rate_type: wer
decoding_method: attention # 'attention', 'ctc_greedy_search', 'ctc_prefix_beam_search', 'attention_rescoring'
beam_size: 10
ctc_weight: 0.5 # ctc weight for attention rescoring decode mode.
decoding_chunk_size: -1 # decoding chunk size. Defaults to -1.
# <0: for decoding, use full chunk.
# >0: for decoding, use fixed chunk size as set.
# 0: used for training, it's prohibited here.
num_decoding_left_chunks: -1 # number of left chunks for decoding. Defaults to -1.
simulate_streaming: False # simulate streaming inference. Defaults to False.
examples/librispeech/asr1/conf/transformer.yaml
浏览文件 @ 4cab9f62
# network architecture ############################################
model: # Network Architecture #
cmvn_file: ############################################
cmvn_file_type: "json" cmvn_file:
# encoder related cmvn_file_type: "json"
encoder: transformer # encoder related
encoder_conf: encoder: transformer
output_size: 256 # dimension of attention encoder_conf:
attention_heads: 4 output_size: 256 # dimension of attention
linear_units: 2048 # the number of units of position-wise feed forward attention_heads: 4
num_blocks: 12 # the number of encoder blocks linear_units: 2048 # the number of units of position-wise feed forward
dropout_rate: 0.1 num_blocks: 12 # the number of encoder blocks
positional_dropout_rate: 0.1 dropout_rate: 0.1
attention_dropout_rate: 0.0 positional_dropout_rate: 0.1
input_layer: conv2d # encoder input type, you can chose conv2d, conv2d6 and conv2d8 attention_dropout_rate: 0.0
normalize_before: true input_layer: conv2d # encoder input type, you can chose conv2d, conv2d6 and conv2d8
normalize_before: true
# decoder related # decoder related
decoder: transformer decoder: transformer
decoder_conf: decoder_conf:
attention_heads: 4 attention_heads: 4
linear_units: 2048 linear_units: 2048
num_blocks: 6 num_blocks: 6
dropout_rate: 0.1 dropout_rate: 0.1
positional_dropout_rate: 0.1 positional_dropout_rate: 0.1
self_attention_dropout_rate: 0.0 self_attention_dropout_rate: 0.0
src_attention_dropout_rate: 0.0 src_attention_dropout_rate: 0.0
# hybrid CTC/attention # hybrid CTC/attention
model_conf: model_conf:
ctc_weight: 0.3 ctc_weight: 0.3
lsm_weight: 0.1 # label smoothing option lsm_weight: 0.1 # label smoothing option
length_normalized_loss: false length_normalized_loss: false
# https://yaml.org/type/float.html # https://yaml.org/type/float.html
data: ###########################################
train_manifest: data/manifest.train # Data #
dev_manifest: data/manifest.dev ###########################################
test_manifest: data/manifest.test-clean train_manifest: data/manifest.train
min_input_len: 0.5 # second dev_manifest: data/manifest.dev
max_input_len: 30.0 # second test_manifest: data/manifest.test-clean
min_output_len: 0.0 # tokens
max_output_len: 400.0 # tokens
min_output_input_ratio: 0.05
max_output_input_ratio: 100.0
collator: ###########################################
vocab_filepath: data/lang_char/vocab.txt # Dataloader #
unit_type: 'spm' ###########################################
spm_model_prefix: 'data/lang_char/bpe_unigram_5000' vocab_filepath: data/lang_char/vocab.txt
mean_std_filepath: "" unit_type: 'spm'
augmentation_config: conf/preprocess.yaml spm_model_prefix: 'data/lang_char/bpe_unigram_5000'
feat_dim: 80 mean_std_filepath: ""
stride_ms: 10.0 preprocess_config: conf/preprocess.yaml
window_ms: 25.0 feat_dim: 80
sortagrad: 0 # Feed samples from shortest to longest ; -1: enabled for all epochs, 0: disabled, other: enabled for 'other' epochs stride_ms: 10.0
batch_size: 32 window_ms: 25.0
maxlen_in: 512 # if input length > maxlen-in, batchsize is automatically reduced sortagrad: 0 # Feed samples from shortest to longest ; -1: enabled for all epochs, 0: disabled, other: enabled for 'other' epochs
maxlen_out: 150 # if output length > maxlen-out, batchsize is automatically reduced batch_size: 32
minibatches: 0 # for debug maxlen_in: 512 # if input length > maxlen-in, batchsize is automatically reduced
batch_count: auto maxlen_out: 150 # if output length > maxlen-out, batchsize is automatically reduced
batch_bins: 0 minibatches: 0 # for debug
batch_frames_in: 0 batch_count: auto
batch_frames_out: 0 batch_bins: 0
batch_frames_inout: 0 batch_frames_in: 0
augmentation_config: conf/preprocess.yaml batch_frames_out: 0
num_workers: 0 batch_frames_inout: 0
subsampling_factor: 1 num_workers: 0
num_encs: 1 subsampling_factor: 1
num_encs: 1
training:
n_epoch: 120
accum_grad: 4
global_grad_clip: 5.0
optim: adam
optim_conf:
lr: 0.004
weight_decay: 1e-06
scheduler: warmuplr
scheduler_conf:
warmup_steps: 25000
lr_decay: 1.0
log_interval: 100
checkpoint:
kbest_n: 50
latest_n: 5
decoding:
batch_size: 64
error_rate_type: wer
decoding_method: attention # 'attention', 'ctc_greedy_search', 'ctc_prefix_beam_search', 'attention_rescoring'
lang_model_path: data/lm/common_crawl_00.prune01111.trie.klm
alpha: 2.5
beta: 0.3
beam_size: 10
cutoff_prob: 1.0
cutoff_top_n: 0
num_proc_bsearch: 8
ctc_weight: 0.5 # ctc weight for attention rescoring decode mode.
decoding_chunk_size: -1 # decoding chunk size. Defaults to -1.
# <0: for decoding, use full chunk.
# >0: for decoding, use fixed chunk size as set.
# 0: used for training, it's prohibited here.
num_decoding_left_chunks: -1 # number of left chunks for decoding. Defaults to -1.
simulate_streaming: False # simulate streaming inference. Defaults to False.
###########################################
# Training #
###########################################
n_epoch: 120
accum_grad: 4
global_grad_clip: 5.0
optim: adam
optim_conf:
lr: 0.004
weight_decay: 1.0e-06
scheduler: warmuplr
scheduler_conf:
warmup_steps: 25000
lr_decay: 1.0
log_interval: 100
checkpoint:
kbest_n: 50
latest_n: 5
examples/librispeech/asr1/conf/tuning/chunk_decode.yaml 0 → 100644
浏览文件 @ 4cab9f62
decode_batch_size: 128
error_rate_type: wer
decoding_method: attention # 'attention', 'ctc_greedy_search', 'ctc_prefix_beam_search', 'attention_rescoring'
beam_size: 10
ctc_weight: 0.5 # ctc weight for attention rescoring decode mode.
decoding_chunk_size: -1 # decoding chunk size. Defaults to -1.
# <0: for decoding, use full chunk.
# >0: for decoding, use fixed chunk size as set.
# 0: used for training, it's prohibited here.
num_decoding_left_chunks: -1 # number of left chunks for decoding. Defaults to -1.
simulate_streaming: true # simulate streaming inference. Defaults to False.
examples/librispeech/asr1/conf/tuning/decode.yaml 0 → 100644
浏览文件 @ 4cab9f62
decode_batch_size: 64
error_rate_type: wer
decoding_method: attention # 'attention', 'ctc_greedy_search', 'ctc_prefix_beam_search', 'attention_rescoring'
beam_size: 10
ctc_weight: 0.5 # ctc weight for attention rescoring decode mode.
decoding_chunk_size: -1 # decoding chunk size. Defaults to -1.
# <0: for decoding, use full chunk.
# >0: for decoding, use fixed chunk size as set.
# 0: used for training, it's prohibited here.
num_decoding_left_chunks: -1 # number of left chunks for decoding. Defaults to -1.
simulate_streaming: False # simulate streaming inference. Defaults to False.
examples/librispeech/asr1/local/align.sh
浏览文件 @ 4cab9f62
#!/bin/bash #!/bin/bash
if [ $# != 2 ];then if [ $# != 3 ];then
echo "usage: ${0} config_path ckpt_path_prefix" echo "usage: ${0} config_path decode_config_path ckpt_path_prefix"
exit -1 exit -1
fi fi
...@@ -9,7 +9,8 @@ ngpu=$(echo $CUDA_VISIBLE_DEVICES | awk -F "," '{print NF}') ...@@ -9,7 +9,8 @@ ngpu=$(echo $CUDA_VISIBLE_DEVICES | awk -F "," '{print NF}')
echo "using $ngpu gpus..." echo "using $ngpu gpus..."
config_path=$1 config_path=$1
ckpt_prefix=$2 decode_config_path=$2
ckpt_prefix=$3
batch_size=1 batch_size=1
output_dir=${ckpt_prefix} output_dir=${ckpt_prefix}
...@@ -20,9 +21,10 @@ mkdir -p ${output_dir} ...@@ -20,9 +21,10 @@ mkdir -p ${output_dir}
python3 -u ${BIN_DIR}/alignment.py \ python3 -u ${BIN_DIR}/alignment.py \
--ngpu ${ngpu} \ --ngpu ${ngpu} \
--config ${config_path} \ --config ${config_path} \
--decode_cfg ${decode_config_path} \
--result_file ${output_dir}/${type}.align \ --result_file ${output_dir}/${type}.align \
--checkpoint_path ${ckpt_prefix} \ --checkpoint_path ${ckpt_prefix} \
--opts decoding.batch_size ${batch_size} --opts decode.decode_batch_size ${batch_size}
if [ $? -ne 0 ]; then if [ $? -ne 0 ]; then
echo "Failed in ctc alignment!" echo "Failed in ctc alignment!"
......
examples/librispeech/asr1/local/test.sh
浏览文件 @ 4cab9f62
...@@ -15,8 +15,8 @@ recog_set="test-clean" ...@@ -15,8 +15,8 @@ recog_set="test-clean"
stage=0 stage=0
stop_stage=100 stop_stage=100
if [ $# != 2 ];then if [ $# != 3 ];then
echo "usage: ${0} config_path ckpt_path_prefix" echo "usage: ${0} config_path decode_config_path ckpt_path_prefix"
exit -1 exit -1
fi fi
...@@ -24,7 +24,8 @@ ngpu=$(echo $CUDA_VISIBLE_DEVICES | awk -F "," '{print NF}') ...@@ -24,7 +24,8 @@ ngpu=$(echo $CUDA_VISIBLE_DEVICES | awk -F "," '{print NF}')
echo "using $ngpu gpus..." echo "using $ngpu gpus..."
config_path=$1 config_path=$1
ckpt_prefix=$2 decode_config_path=$2
ckpt_prefix=$3
chunk_mode=false chunk_mode=false
if [[ ${config_path} =~ ^.*chunk_.*yaml$ ]];then if [[ ${config_path} =~ ^.*chunk_.*yaml$ ]];then
...@@ -52,10 +53,11 @@ if [ ${stage} -le 0 ] && [ ${stop_stage} -ge 0 ]; then ...@@ -52,10 +53,11 @@ if [ ${stage} -le 0 ] && [ ${stop_stage} -ge 0 ]; then
python3 -u ${BIN_DIR}/test.py \ python3 -u ${BIN_DIR}/test.py \
--ngpu ${ngpu} \ --ngpu ${ngpu} \
--config ${config_path} \ --config ${config_path} \
--decode_cfg ${decode_config_path} \
--result_file ${ckpt_prefix}.${type}.rsl \ --result_file ${ckpt_prefix}.${type}.rsl \
--checkpoint_path ${ckpt_prefix} \ --checkpoint_path ${ckpt_prefix} \
--opts decoding.decoding_method ${type} \ --opts decode.decoding_method ${type} \
--opts decoding.batch_size ${batch_size} --opts decode.decode_batch_size ${batch_size}
if [ $? -ne 0 ]; then if [ $? -ne 0 ]; then
echo "Failed in evaluation!" echo "Failed in evaluation!"
...@@ -76,10 +78,11 @@ for type in ctc_greedy_search; do ...@@ -76,10 +78,11 @@ for type in ctc_greedy_search; do
python3 -u ${BIN_DIR}/test.py \ python3 -u ${BIN_DIR}/test.py \
--ngpu ${ngpu} \ --ngpu ${ngpu} \
--config ${config_path} \ --config ${config_path} \
--decode_cfg ${decode_config_path} \
--result_file ${ckpt_prefix}.${type}.rsl \ --result_file ${ckpt_prefix}.${type}.rsl \
--checkpoint_path ${ckpt_prefix} \ --checkpoint_path ${ckpt_prefix} \
--opts decoding.decoding_method ${type} \ --opts decode.decoding_method ${type} \
--opts decoding.batch_size ${batch_size} --opts decode.decode_batch_size ${batch_size}
if [ $? -ne 0 ]; then if [ $? -ne 0 ]; then
echo "Failed in evaluation!" echo "Failed in evaluation!"
...@@ -96,10 +99,11 @@ for type in ctc_prefix_beam_search attention_rescoring; do ...@@ -96,10 +99,11 @@ for type in ctc_prefix_beam_search attention_rescoring; do
python3 -u ${BIN_DIR}/test.py \ python3 -u ${BIN_DIR}/test.py \
--ngpu ${ngpu} \ --ngpu ${ngpu} \
--config ${config_path} \ --config ${config_path} \
--decode_cfg ${decode_config_path} \
--result_file ${ckpt_prefix}.${type}.rsl \ --result_file ${ckpt_prefix}.${type}.rsl \
--checkpoint_path ${ckpt_prefix} \ --checkpoint_path ${ckpt_prefix} \
--opts decoding.decoding_method ${type} \ --opts decode.decoding_method ${type} \
--opts decoding.batch_size ${batch_size} --opts decode.decode_batch_size ${batch_size}
if [ $? -ne 0 ]; then if [ $? -ne 0 ]; then
echo "Failed in evaluation!" echo "Failed in evaluation!"
......
examples/librispeech/asr1/local/test_wav.sh
浏览文件 @ 4cab9f62
#!/bin/bash #!/bin/bash
if [ $# != 3 ];then if [ $# != 4 ];then
echo "usage: ${0} config_path ckpt_path_prefix audio_file" echo "usage: ${0} config_path decode_config_path ckpt_path_prefix audio_file"
exit -1 exit -1
fi fi
...@@ -9,8 +9,9 @@ ngpu=$(echo $CUDA_VISIBLE_DEVICES | awk -F "," '{print NF}') ...@@ -9,8 +9,9 @@ ngpu=$(echo $CUDA_VISIBLE_DEVICES | awk -F "," '{print NF}')
echo "using $ngpu gpus..." echo "using $ngpu gpus..."
config_path=$1 config_path=$1
ckpt_prefix=$2 decode_config_path=$2
audio_file=$3 ckpt_prefix=$3
audio_file=$4
mkdir -p data mkdir -p data
wget -nc https://paddlespeech.bj.bcebos.com/datasets/single_wav/en/demo_002_en.wav -P data/ wget -nc https://paddlespeech.bj.bcebos.com/datasets/single_wav/en/demo_002_en.wav -P data/
...@@ -49,10 +50,11 @@ for type in attention_rescoring; do ...@@ -49,10 +50,11 @@ for type in attention_rescoring; do
python3 -u ${BIN_DIR}/test_wav.py \ python3 -u ${BIN_DIR}/test_wav.py \
--ngpu ${ngpu} \ --ngpu ${ngpu} \
--config ${config_path} \ --config ${config_path} \
--decode_cfg ${decode_config_path} \
--result_file ${output_dir}/${type}.rsl \ --result_file ${output_dir}/${type}.rsl \
--checkpoint_path ${ckpt_prefix} \ --checkpoint_path ${ckpt_prefix} \
--opts decoding.decoding_method ${type} \ --opts decode.decoding_method ${type} \
--opts decoding.batch_size ${batch_size} \ --opts decode.decode_batch_size ${batch_size} \
--audio_file ${audio_file} --audio_file ${audio_file}
#score_sclite.sh --bpe ${nbpe} --bpemodel ${bpemodel}.model --wer true ${expdir}/${decode_dir} ${dict} #score_sclite.sh --bpe ${nbpe} --bpemodel ${bpemodel}.model --wer true ${expdir}/${decode_dir} ${dict}
......
examples/librispeech/asr1/run.sh
浏览文件 @ 4cab9f62
...@@ -8,6 +8,7 @@ gpus=0,1,2,3 ...@@ -8,6 +8,7 @@ gpus=0,1,2,3
stage=0 stage=0
stop_stage=50 stop_stage=50
conf_path=conf/transformer.yaml conf_path=conf/transformer.yaml
decode_conf_path=conf/tuning/decode.yaml
avg_num=30 avg_num=30
audio_file=data/demo_002_en.wav audio_file=data/demo_002_en.wav
...@@ -34,17 +35,17 @@ fi ...@@ -34,17 +35,17 @@ fi
if [ ${stage} -le 3 ] && [ ${stop_stage} -ge 3 ]; then if [ ${stage} -le 3 ] && [ ${stop_stage} -ge 3 ]; then
# test ckpt avg_n # test ckpt avg_n
CUDA_VISIBLE_DEVICES=0 ./local/test.sh ${conf_path} exp/${ckpt}/checkpoints/${avg_ckpt} || exit -1 CUDA_VISIBLE_DEVICES=0 ./local/test.sh ${conf_path} ${decode_conf_path} exp/${ckpt}/checkpoints/${avg_ckpt} || exit -1
fi fi
if [ ${stage} -le 4 ] && [ ${stop_stage} -ge 4 ]; then if [ ${stage} -le 4 ] && [ ${stop_stage} -ge 4 ]; then
# ctc alignment of test data # ctc alignment of test data
CUDA_VISIBLE_DEVICES=0 ./local/align.sh ${conf_path} exp/${ckpt}/checkpoints/${avg_ckpt} || exit -1 CUDA_VISIBLE_DEVICES=0 ./local/align.sh ${conf_path} ${decode_conf_path} exp/${ckpt}/checkpoints/${avg_ckpt} || exit -1
fi fi
if [ ${stage} -le 5 ] && [ ${stop_stage} -ge 5 ]; then if [ ${stage} -le 5 ] && [ ${stop_stage} -ge 5 ]; then
# test a single .wav file # test a single .wav file
CUDA_VISIBLE_DEVICES=0 ./local/test_wav.sh ${conf_path} exp/${ckpt}/checkpoints/${avg_ckpt} ${audio_file} || exit -1 CUDA_VISIBLE_DEVICES=0 ./local/test_wav.sh ${conf_path} ${decode_conf_path} exp/${ckpt}/checkpoints/${avg_ckpt} ${audio_file} || exit -1
fi fi
if [ ${stage} -le 51 ] && [ ${stop_stage} -ge 51 ]; then if [ ${stage} -le 51 ] && [ ${stop_stage} -ge 51 ]; then
......
examples/librispeech/asr2/conf/decode/decode_base.yaml 0 → 100644
浏览文件 @ 4cab9f62
decode_batch_size: 1
error_rate_type: wer
decoding_method: attention # 'attention', 'ctc_greedy_search', 'ctc_prefix_beam_search', 'attention_rescoring'
beam_size: 10
ctc_weight: 0.5 # ctc weight for attention rescoring decode mode.
decoding_chunk_size: -1 # decoding chunk size. Defaults to -1.
# <0: for decoding, use full chunk.
# >0: for decoding, use fixed chunk size as set.
# 0: used for training, it's prohibited here.
num_decoding_left_chunks: -1 # number of left chunks for decoding. Defaults to -1.
simulate_streaming: False # simulate streaming inference. Defaults to False.
examples/librispeech/asr2/conf/transformer.yaml
浏览文件 @ 4cab9f62
# https://yaml.org/type/float.html # https://yaml.org/type/float.html
# network architecture ############################################
model: # Network Architecture #
cmvn_file: ############################################
cmvn_file_type: "json" cmvn_file:
# encoder related cmvn_file_type: "json"
encoder: transformer # encoder related
encoder_conf: encoder: transformer
output_size: 256 # dimension of attention encoder_conf:
attention_heads: 4 output_size: 256 # dimension of attention
linear_units: 2048 # the number of units of position-wise feed forward attention_heads: 4
num_blocks: 12 # the number of encoder blocks linear_units: 2048 # the number of units of position-wise feed forward
dropout_rate: 0.1 num_blocks: 12 # the number of encoder blocks
positional_dropout_rate: 0.1 dropout_rate: 0.1
attention_dropout_rate: 0.0 positional_dropout_rate: 0.1
input_layer: conv2d # encoder input type, you can chose conv2d, conv2d6 and conv2d8 attention_dropout_rate: 0.0
normalize_before: true input_layer: conv2d # encoder input type, you can chose conv2d, conv2d6 and conv2d8
normalize_before: true
# decoder related # decoder related
decoder: transformer decoder: transformer
decoder_conf: decoder_conf:
attention_heads: 4 attention_heads: 4
linear_units: 2048 linear_units: 2048
num_blocks: 6 num_blocks: 6
dropout_rate: 0.1 dropout_rate: 0.1
positional_dropout_rate: 0.1 positional_dropout_rate: 0.1
self_attention_dropout_rate: 0.0 self_attention_dropout_rate: 0.0
src_attention_dropout_rate: 0.0 src_attention_dropout_rate: 0.0
# hybrid CTC/attention # hybrid CTC/attention
model_conf: model_conf:
ctc_weight: 0.3 ctc_weight: 0.3
lsm_weight: 0.1 # label smoothing option lsm_weight: 0.1 # label smoothing option
length_normalized_loss: false length_normalized_loss: false
data: ###########################################
train_manifest: data/manifest.train # Data #
dev_manifest: data/manifest.dev ###########################################
test_manifest: data/manifest.test-clean train_manifest: data/manifest.train
dev_manifest: data/manifest.dev
test_manifest: data/manifest.test-clean
collator: ###########################################
vocab_filepath: data/lang_char/train_960_unigram5000_units.txt # Dataloader #
unit_type: spm ###########################################
spm_model_prefix: data/lang_char/train_960_unigram5000 vocab_filepath: data/lang_char/train_960_unigram5000_units.txt
feat_dim: 83 unit_type: spm
stride_ms: 10.0 spm_model_prefix: data/lang_char/train_960_unigram5000
window_ms: 25.0 feat_dim: 83
sortagrad: 0 # Feed samples from shortest to longest ; -1: enabled for all epochs, 0: disabled, other: enabled for 'other' epochs stride_ms: 10.0
batch_size: 30 window_ms: 25.0
maxlen_in: 512 # if input length > maxlen-in, batchsize is automatically reduced sortagrad: 0 # Feed samples from shortest to longest ; -1: enabled for all epochs, 0: disabled, other: enabled for 'other' epochs
maxlen_out: 150 # if output length > maxlen-out, batchsize is automatically reduced batch_size: 30
minibatches: 0 # for debug maxlen_in: 512 # if input length > maxlen-in, batchsize is automatically reduced
batch_count: auto maxlen_out: 150 # if output length > maxlen-out, batchsize is automatically reduced
batch_bins: 0 minibatches: 0 # for debug
batch_frames_in: 0 batch_count: auto
batch_frames_out: 0 batch_bins: 0
batch_frames_inout: 0 batch_frames_in: 0
augmentation_config: conf/preprocess.yaml batch_frames_out: 0
num_workers: 0 batch_frames_inout: 0
subsampling_factor: 1 preprocess_config: conf/preprocess.yaml
num_encs: 1 num_workers: 0
subsampling_factor: 1
num_encs: 1
training: ###########################################
n_epoch: 120 # Training #
accum_grad: 2 ###########################################
log_interval: 100 n_epoch: 120
checkpoint: accum_grad: 2
kbest_n: 50 log_interval: 1
latest_n: 5 checkpoint:
kbest_n: 50
latest_n: 5
optim: adam optim: adam
optim_conf: optim_conf:
...@@ -79,23 +86,5 @@ scheduler_conf: ...@@ -79,23 +86,5 @@ scheduler_conf:
warmup_steps: 25000 warmup_steps: 25000
lr_decay: 1.0 lr_decay: 1.0
decoding:
batch_size: 1
error_rate_type: wer
decoding_method: attention # 'attention', 'ctc_greedy_search', 'ctc_prefix_beam_search', 'attention_rescoring'
lang_model_path: data/lm/common_crawl_00.prune01111.trie.klm
alpha: 2.5
beta: 0.3
beam_size: 10
cutoff_prob: 1.0
cutoff_top_n: 0
num_proc_bsearch: 8
ctc_weight: 0.5 # ctc weight for attention rescoring decode mode.
decoding_chunk_size: -1 # decoding chunk size. Defaults to -1.
# <0: for decoding, use full chunk.
# >0: for decoding, use fixed chunk size as set.
# 0: used for training, it's prohibited here.
num_decoding_left_chunks: -1 # number of left chunks for decoding. Defaults to -1.
simulate_streaming: False # simulate streaming inference. Defaults to False.
examples/librispeech/asr2/local/align.sh
浏览文件 @ 4cab9f62
#!/bin/bash #!/bin/bash
if [ $# != 3 ];then if [ $# != 4 ];then
echo "usage: ${0} config_path dict_path ckpt_path_prefix" echo "usage: ${0} config_path decode_config_path dict_path ckpt_path_prefix"
exit -1 exit -1
fi fi
...@@ -9,8 +9,9 @@ ngpu=$(echo $CUDA_VISIBLE_DEVICES | awk -F "," '{print NF}') ...@@ -9,8 +9,9 @@ ngpu=$(echo $CUDA_VISIBLE_DEVICES | awk -F "," '{print NF}')
echo "using $ngpu gpus..." echo "using $ngpu gpus..."
config_path=$1 config_path=$1
dict_path=$2 decode_config_path=$2
ckpt_prefix=$3 dict_path=$3
ckpt_prefix=$4
batch_size=1 batch_size=1
output_dir=${ckpt_prefix} output_dir=${ckpt_prefix}
...@@ -24,9 +25,10 @@ python3 -u ${BIN_DIR}/test.py \ ...@@ -24,9 +25,10 @@ python3 -u ${BIN_DIR}/test.py \
--dict-path ${dict_path} \ --dict-path ${dict_path} \
--ngpu ${ngpu} \ --ngpu ${ngpu} \
--config ${config_path} \ --config ${config_path} \
--decode_cfg ${decode_config_path} \
--result-file ${output_dir}/${type}.align \ --result-file ${output_dir}/${type}.align \
--checkpoint_path ${ckpt_prefix} \ --checkpoint_path ${ckpt_prefix} \
--opts decoding.batch_size ${batch_size} --opts decode.decode_batch_size ${batch_size}
if [ $? -ne 0 ]; then if [ $? -ne 0 ]; then
echo "Failed in ctc alignment!" echo "Failed in ctc alignment!"
......
examples/librispeech/asr2/local/test.sh
浏览文件 @ 4cab9f62
...@@ -19,6 +19,7 @@ bpeprefix=data/lang_char/${train_set}_${bpemode}${nbpe} ...@@ -19,6 +19,7 @@ bpeprefix=data/lang_char/${train_set}_${bpemode}${nbpe}
bpemodel=${bpeprefix}.model bpemodel=${bpeprefix}.model
config_path=conf/transformer.yaml config_path=conf/transformer.yaml
decode_config_path=conf/decode/decode_base.yaml
dict=data/lang_char/${train_set}_${bpemode}${nbpe}_units.txt dict=data/lang_char/${train_set}_${bpemode}${nbpe}_units.txt
ckpt_prefix= ckpt_prefix=
...@@ -79,11 +80,12 @@ for dmethd in attention ctc_greedy_search ctc_prefix_beam_search attention_resco ...@@ -79,11 +80,12 @@ for dmethd in attention ctc_greedy_search ctc_prefix_beam_search attention_resco
--ngpu ${ngpu} \ --ngpu ${ngpu} \
--dict-path ${dict} \ --dict-path ${dict} \
--config ${config_path} \ --config ${config_path} \
--decode_cfg ${decode_config_path} \
--checkpoint_path ${ckpt_prefix} \ --checkpoint_path ${ckpt_prefix} \
--result-file ${decode_dir}/data.JOB.json \ --result-file ${decode_dir}/data.JOB.json \
--opts decoding.decoding_method ${dmethd} \ --opts decode.decoding_method ${dmethd} \
--opts decoding.batch_size ${batch_size} \ --opts decode.decode_batch_size ${batch_size} \
--opts data.test_manifest ${feat_recog_dir}/split${nj}/JOB/manifest.${rtask} --opts test_manifest ${feat_recog_dir}/split${nj}/JOB/manifest.${rtask}
score_sclite.sh --bpe ${nbpe} --bpemodel ${bpemodel} --wer false ${decode_dir} ${dict} score_sclite.sh --bpe ${nbpe} --bpemodel ${bpemodel} --wer false ${decode_dir} ${dict}
......
examples/librispeech/asr2/run.sh
浏览文件 @ 4cab9f62
...@@ -9,7 +9,8 @@ gpus=0,1,2,3,4,5,6,7 ...@@ -9,7 +9,8 @@ gpus=0,1,2,3,4,5,6,7
stage=0 stage=0
stop_stage=50 stop_stage=50
conf_path=conf/transformer.yaml conf_path=conf/transformer.yaml
dict_path=lang_char/train_960_unigram5000_units.txt decode_conf_path=conf/decode/decode_base.yaml
dict_path=data/lang_char/train_960_unigram5000_units.txt
avg_num=10 avg_num=10
source ${MAIN_ROOT}/utils/parse_options.sh || exit 1; source ${MAIN_ROOT}/utils/parse_options.sh || exit 1;
...@@ -35,7 +36,7 @@ fi ...@@ -35,7 +36,7 @@ fi
if [ ${stage} -le 3 ] && [ ${stop_stage} -ge 3 ]; then if [ ${stage} -le 3 ] && [ ${stop_stage} -ge 3 ]; then
# attetion resocre decoder # attetion resocre decoder
./local/test.sh ${conf_path} ${dict_path} exp/${ckpt}/checkpoints/${avg_ckpt} || exit -1 ./local/test.sh ${conf_path} ${decode_conf_path} ${dict_path} exp/${ckpt}/checkpoints/${avg_ckpt} || exit -1
fi fi
if [ ${stage} -le 4 ] && [ ${stop_stage} -ge 4 ]; then if [ ${stage} -le 4 ] && [ ${stop_stage} -ge 4 ]; then
...@@ -45,7 +46,7 @@ fi ...@@ -45,7 +46,7 @@ fi
if [ ${stage} -le 5 ] && [ ${stop_stage} -ge 5 ]; then if [ ${stage} -le 5 ] && [ ${stop_stage} -ge 5 ]; then
# ctc alignment of test data # ctc alignment of test data
CUDA_VISIBLE_DEVICES=0 ./local/align.sh ${conf_path} ${dict_path} exp/${ckpt}/checkpoints/${avg_ckpt} || exit -1 CUDA_VISIBLE_DEVICES=0 ./local/align.sh ${conf_path} ${decode_conf_path} ${dict_path} exp/${ckpt}/checkpoints/${avg_ckpt} || exit -1
fi fi
if [ ${stage} -le 6 ] && [ ${stop_stage} -ge 6 ]; then if [ ${stage} -le 6 ] && [ ${stop_stage} -ge 6 ]; then
......
examples/other/1xt2x/aishell/conf/deepspeech2.yaml
浏览文件 @ 4cab9f62
# https://yaml.org/type/float.html # https://yaml.org/type/float.html
data: ###########################################
train_manifest: data/manifest.train # Data #
dev_manifest: data/manifest.dev ###########################################
test_manifest: data/manifest.test train_manifest: data/manifest.train
min_input_len: 0.0 dev_manifest: data/manifest.dev
max_input_len: 27.0 # second test_manifest: data/manifest.test
min_output_len: 0.0 min_input_len: 0.0
max_output_len: .inf max_input_len: 27.0 # second
min_output_input_ratio: 0.00 min_output_len: 0.0
max_output_input_ratio: .inf max_output_len: .inf
min_output_input_ratio: 0.00
max_output_input_ratio: .inf
collator: ###########################################
batch_size: 64 # one gpu # Dataloader #
mean_std_filepath: data/mean_std.npz ###########################################
unit_type: char batch_size: 64 # one gpu
vocab_filepath: data/vocab.txt mean_std_filepath: data/mean_std.npz
augmentation_config: conf/augmentation.json unit_type: char
random_seed: 0 vocab_filepath: data/vocab.txt
spm_model_prefix: augmentation_config: conf/augmentation.json
spectrum_type: linear random_seed: 0
feat_dim: spm_model_prefix:
delta_delta: False spectrum_type: linear
stride_ms: 10.0 feat_dim:
window_ms: 20.0 delta_delta: False
n_fft: None stride_ms: 10.0
max_freq: None window_ms: 20.0
target_sample_rate: 16000 n_fft: None
use_dB_normalization: True max_freq: None
target_dB: -20 target_sample_rate: 16000
dither: 1.0 use_dB_normalization: True
keep_transcription_text: False target_dB: -20
sortagrad: True dither: 1.0
shuffle_method: batch_shuffle keep_transcription_text: False
num_workers: 2 sortagrad: True
shuffle_method: batch_shuffle
num_workers: 2
model: ############################################
num_conv_layers: 2 # Network Architecture #
num_rnn_layers: 3 ############################################
rnn_layer_size: 1024 num_conv_layers: 2
use_gru: True num_rnn_layers: 3
share_rnn_weights: False rnn_layer_size: 1024
blank_id: 4333 use_gru: True
share_rnn_weights: False
blank_id: 4333
training: ###########################################
n_epoch: 80 # Training #
accum_grad: 1 ###########################################
lr: 2e-3 n_epoch: 80
lr_decay: 0.83 accum_grad: 1
weight_decay: 1e-06 lr: 2e-3
global_grad_clip: 3.0 lr_decay: 0.83
log_interval: 100 weight_decay: 1e-06
checkpoint: global_grad_clip: 3.0
kbest_n: 50 log_interval: 100
latest_n: 5 checkpoint:
kbest_n: 50
latest_n: 5
decoding:
batch_size: 32
error_rate_type: cer
decoding_method: ctc_beam_search
lang_model_path: data/lm/zh_giga.no_cna_cmn.prune01244.klm
alpha: 2.6
beta: 5.0
beam_size: 300
cutoff_prob: 0.99
cutoff_top_n: 40
num_proc_bsearch: 8
examples/other/1xt2x/aishell/conf/tuning/decode.yaml 0 → 100644
浏览文件 @ 4cab9f62
decode_batch_size: 32
error_rate_type: cer
decoding_method: ctc_beam_search
lang_model_path: data/lm/zh_giga.no_cna_cmn.prune01244.klm
alpha: 2.6
beta: 5.0
beam_size: 300
cutoff_prob: 0.99
cutoff_top_n: 40
num_proc_bsearch: 8
\ No newline at end of file
examples/other/1xt2x/aishell/local/test.sh
浏览文件 @ 4cab9f62
#!/bin/bash #!/bin/bash
if [ $# != 3 ];then if [ $# != 4 ];then
echo "usage: ${0} config_path ckpt_path_prefix model_type" echo "usage: ${0} config_path decode_config_path ckpt_path_prefix model_type"
exit -1 exit -1
fi fi
...@@ -9,8 +9,9 @@ ngpu=$(echo $CUDA_VISIBLE_DEVICES | awk -F "," '{print NF}') ...@@ -9,8 +9,9 @@ ngpu=$(echo $CUDA_VISIBLE_DEVICES | awk -F "," '{print NF}')
echo "using $ngpu gpus..." echo "using $ngpu gpus..."
config_path=$1 config_path=$1
ckpt_prefix=$2 decode_config_path=$2
model_type=$3 ckpt_prefix=$3
model_type=$4
# download language model # download language model
bash local/download_lm_ch.sh bash local/download_lm_ch.sh
...@@ -21,6 +22,7 @@ fi ...@@ -21,6 +22,7 @@ fi
python3 -u ${BIN_DIR}/test.py \ python3 -u ${BIN_DIR}/test.py \
--ngpu ${ngpu} \ --ngpu ${ngpu} \
--config ${config_path} \ --config ${config_path} \
--decode_cfg ${decode_config_path} \
--result_file ${ckpt_prefix}.rsl \ --result_file ${ckpt_prefix}.rsl \
--checkpoint_path ${ckpt_prefix} \ --checkpoint_path ${ckpt_prefix} \
--model_type ${model_type} --model_type ${model_type}
......
examples/other/1xt2x/aishell/run.sh
浏览文件 @ 4cab9f62
...@@ -5,6 +5,7 @@ source path.sh ...@@ -5,6 +5,7 @@ source path.sh
stage=0 stage=0
stop_stage=100 stop_stage=100
conf_path=conf/deepspeech2.yaml conf_path=conf/deepspeech2.yaml
decode_conf_path=conf/tuning/decode.yaml
avg_num=1 avg_num=1
model_type=offline model_type=offline
gpus=2 gpus=2
...@@ -23,6 +24,6 @@ fi ...@@ -23,6 +24,6 @@ fi
if [ ${stage} -le 1 ] && [ ${stop_stage} -ge 1 ]; then if [ ${stage} -le 1 ] && [ ${stop_stage} -ge 1 ]; then
# test ckpt avg_n # test ckpt avg_n
CUDA_VISIBLE_DEVICES=${gpus} ./local/test.sh ${conf_path} exp/${ckpt}/checkpoints/${v18_ckpt} ${model_type}|| exit -1 CUDA_VISIBLE_DEVICES=${gpus} ./local/test.sh ${conf_path} ${decode_conf_path} exp/${ckpt}/checkpoints/${v18_ckpt} ${model_type}|| exit -1
fi fi
examples/other/1xt2x/baidu_en8k/conf/deepspeech2.yaml
浏览文件 @ 4cab9f62
# https://yaml.org/type/float.html # https://yaml.org/type/float.html
data: ###########################################
train_manifest: data/manifest.train # Data #
dev_manifest: data/manifest.dev ###########################################
test_manifest: data/manifest.test-clean train_manifest: data/manifest.train
min_input_len: 0.0 dev_manifest: data/manifest.dev
max_input_len: .inf # second test_manifest: data/manifest.test-clean
min_output_len: 0.0 min_input_len: 0.0
max_output_len: .inf max_input_len: .inf # second
min_output_input_ratio: 0.00 min_output_len: 0.0
max_output_input_ratio: .inf max_output_len: .inf
min_output_input_ratio: 0.00
max_output_input_ratio: .inf
collator: ###########################################
batch_size: 64 # one gpu # Dataloader #
mean_std_filepath: data/mean_std.npz ###########################################
unit_type: char batch_size: 64 # one gpu
vocab_filepath: data/vocab.txt mean_std_filepath: data/mean_std.npz
augmentation_config: conf/augmentation.json unit_type: char
random_seed: 0 vocab_filepath: data/vocab.txt
spm_model_prefix: augmentation_config: conf/augmentation.json
spectrum_type: linear random_seed: 0
feat_dim: spm_model_prefix:
delta_delta: False spectrum_type: linear
stride_ms: 10.0 feat_dim:
window_ms: 20.0 delta_delta: False
n_fft: None stride_ms: 10.0
max_freq: None window_ms: 20.0
target_sample_rate: 16000 n_fft: None
use_dB_normalization: True max_freq: None
target_dB: -20 target_sample_rate: 16000
dither: 1.0 use_dB_normalization: True
keep_transcription_text: False target_dB: -20
sortagrad: True dither: 1.0
shuffle_method: batch_shuffle keep_transcription_text: False
num_workers: 2 sortagrad: True
shuffle_method: batch_shuffle
num_workers: 2
model: ############################################
num_conv_layers: 2 # Network Architecture #
num_rnn_layers: 3 ############################################
rnn_layer_size: 1024 num_conv_layers: 2
use_gru: True num_rnn_layers: 3
share_rnn_weights: False rnn_layer_size: 1024
blank_id: 28 use_gru: True
share_rnn_weights: False
blank_id: 28
###########################################
# Training #
###########################################
n_epoch: 80
accum_grad: 1
lr: 2e-3
lr_decay: 0.83
weight_decay: 1e-06
global_grad_clip: 3.0
log_interval: 100
checkpoint:
kbest_n: 50
latest_n: 5
training:
n_epoch: 80
accum_grad: 1
lr: 2e-3
lr_decay: 0.83
weight_decay: 1e-06
global_grad_clip: 3.0
log_interval: 100
checkpoint:
kbest_n: 50
latest_n: 5
decoding:
batch_size: 32
error_rate_type: wer
decoding_method: ctc_beam_search
lang_model_path: data/lm/common_crawl_00.prune01111.trie.klm
alpha: 1.4
beta: 0.35
beam_size: 500
cutoff_prob: 1.0
cutoff_top_n: 40
num_proc_bsearch: 8
examples/other/1xt2x/baidu_en8k/conf/tuning/decode.yaml 0 → 100644
浏览文件 @ 4cab9f62
decode_batch_size: 32
error_rate_type: wer
decoding_method: ctc_beam_search
lang_model_path: data/lm/common_crawl_00.prune01111.trie.klm
alpha: 1.4
beta: 0.35
beam_size: 500
cutoff_prob: 1.0
cutoff_top_n: 40
num_proc_bsearch: 8
\ No newline at end of file
examples/other/1xt2x/baidu_en8k/local/test.sh
浏览文件 @ 4cab9f62
#!/bin/bash #!/bin/bash
if [ $# != 3 ];then if [ $# != 4 ];then
echo "usage: ${0} config_path ckpt_path_prefix model_type" echo "usage: ${0} config_path decode_config_path ckpt_path_prefix model_type"
exit -1 exit -1
fi fi
...@@ -9,8 +9,9 @@ ngpu=$(echo $CUDA_VISIBLE_DEVICES | awk -F "," '{print NF}') ...@@ -9,8 +9,9 @@ ngpu=$(echo $CUDA_VISIBLE_DEVICES | awk -F "," '{print NF}')
echo "using $ngpu gpus..." echo "using $ngpu gpus..."
config_path=$1 config_path=$1
ckpt_prefix=$2 decode_config_path=$2
model_type=$3 ckpt_prefix=$3
model_type=$4
# download language model # download language model
bash local/download_lm_en.sh bash local/download_lm_en.sh
...@@ -21,6 +22,7 @@ fi ...@@ -21,6 +22,7 @@ fi
python3 -u ${BIN_DIR}/test.py \ python3 -u ${BIN_DIR}/test.py \
--ngpu ${ngpu} \ --ngpu ${ngpu} \
--config ${config_path} \ --config ${config_path} \
--decode_cfg ${decode_config_path} \
--result_file ${ckpt_prefix}.rsl \ --result_file ${ckpt_prefix}.rsl \
--checkpoint_path ${ckpt_prefix} \ --checkpoint_path ${ckpt_prefix} \
--model_type ${model_type} --model_type ${model_type}
......
examples/other/1xt2x/baidu_en8k/run.sh
浏览文件 @ 4cab9f62
...@@ -5,6 +5,7 @@ source path.sh ...@@ -5,6 +5,7 @@ source path.sh
stage=0 stage=0
stop_stage=100 stop_stage=100
conf_path=conf/deepspeech2.yaml conf_path=conf/deepspeech2.yaml
decode_conf_path=conf/tuning/decode.yaml
avg_num=1 avg_num=1
model_type=offline model_type=offline
gpus=0 gpus=0
...@@ -23,6 +24,6 @@ fi ...@@ -23,6 +24,6 @@ fi
if [ ${stage} -le 1 ] && [ ${stop_stage} -ge 1 ]; then if [ ${stage} -le 1 ] && [ ${stop_stage} -ge 1 ]; then
# test ckpt avg_n # test ckpt avg_n
CUDA_VISIBLE_DEVICES=${gpus} ./local/test.sh ${conf_path} exp/${ckpt}/checkpoints/${v18_ckpt} ${model_type}|| exit -1 CUDA_VISIBLE_DEVICES=${gpus} ./local/test.sh ${conf_path} ${decode_conf_path} exp/${ckpt}/checkpoints/${v18_ckpt} ${model_type}|| exit -1
fi fi
examples/other/1xt2x/librispeech/conf/deepspeech2.yaml
浏览文件 @ 4cab9f62
# https://yaml.org/type/float.html # https://yaml.org/type/float.html
data: ###########################################
train_manifest: data/manifest.train # Data #
dev_manifest: data/manifest.dev ###########################################
test_manifest: data/manifest.test-clean train_manifest: data/manifest.train
min_input_len: 0.0 dev_manifest: data/manifest.dev
max_input_len: 1000.0 # second test_manifest: data/manifest.test-clean
min_output_len: 0.0 min_input_len: 0.0
max_output_len: .inf max_input_len: 1000.0 # second
min_output_input_ratio: 0.00 min_output_len: 0.0
max_output_input_ratio: .inf max_output_len: .inf
min_output_input_ratio: 0.00
max_output_input_ratio: .inf
collator: ###########################################
batch_size: 64 # one gpu # Dataloader #
mean_std_filepath: data/mean_std.npz ###########################################
unit_type: char batch_size: 64 # one gpu
vocab_filepath: data/vocab.txt mean_std_filepath: data/mean_std.npz
augmentation_config: conf/augmentation.json unit_type: char
random_seed: 0 vocab_filepath: data/vocab.txt
spm_model_prefix: augmentation_config: conf/augmentation.json
spectrum_type: linear random_seed: 0
feat_dim: spm_model_prefix:
delta_delta: False spectrum_type: linear
stride_ms: 10.0 feat_dim:
window_ms: 20.0 delta_delta: False
n_fft: None stride_ms: 10.0
max_freq: None window_ms: 20.0
target_sample_rate: 16000 n_fft: None
use_dB_normalization: True max_freq: None
target_dB: -20 target_sample_rate: 16000
dither: 1.0 use_dB_normalization: True
keep_transcription_text: False target_dB: -20
sortagrad: True dither: 1.0
shuffle_method: batch_shuffle keep_transcription_text: False
num_workers: 2 sortagrad: True
shuffle_method: batch_shuffle
num_workers: 2
model: ############################################
num_conv_layers: 2 # Network Architecture #
num_rnn_layers: 3 ############################################
rnn_layer_size: 2048 num_conv_layers: 2
use_gru: False num_rnn_layers: 3
share_rnn_weights: True rnn_layer_size: 2048
blank_id: 28 use_gru: False
share_rnn_weights: True
blank_id: 28
###########################################
# Training #
###########################################
n_epoch: 80
accum_grad: 1
lr: 2e-3
lr_decay: 0.83
weight_decay: 1e-06
global_grad_clip: 3.0
log_interval: 100
checkpoint:
kbest_n: 50
latest_n: 5
training:
n_epoch: 80
accum_grad: 1
lr: 2e-3
lr_decay: 0.83
weight_decay: 1e-06
global_grad_clip: 3.0
log_interval: 100
checkpoint:
kbest_n: 50
latest_n: 5
decoding:
batch_size: 32
error_rate_type: wer
decoding_method: ctc_beam_search
lang_model_path: data/lm/common_crawl_00.prune01111.trie.klm
alpha: 2.5
beta: 0.3
beam_size: 500
cutoff_prob: 1.0
cutoff_top_n: 40
num_proc_bsearch: 8
examples/other/1xt2x/librispeech/conf/tuning/decode.yaml 0 → 100644
浏览文件 @ 4cab9f62
decode_batch_size: 32
error_rate_type: wer
decoding_method: ctc_beam_search
lang_model_path: data/lm/common_crawl_00.prune01111.trie.klm
alpha: 2.5
beta: 0.3
beam_size: 500
cutoff_prob: 1.0
cutoff_top_n: 40
num_proc_bsearch: 8
\ No newline at end of file
examples/other/1xt2x/librispeech/local/test.sh
浏览文件 @ 4cab9f62
#!/bin/bash #!/bin/bash
if [ $# != 3 ];then if [ $# != 4 ];then
echo "usage: ${0} config_path ckpt_path_prefix model_type" echo "usage: ${0} config_path decode_config_path ckpt_path_prefix model_type"
exit -1 exit -1
fi fi
...@@ -9,8 +9,9 @@ ngpu=$(echo $CUDA_VISIBLE_DEVICES | awk -F "," '{print NF}') ...@@ -9,8 +9,9 @@ ngpu=$(echo $CUDA_VISIBLE_DEVICES | awk -F "," '{print NF}')
echo "using $ngpu gpus..." echo "using $ngpu gpus..."
config_path=$1 config_path=$1
ckpt_prefix=$2 decode_config_path=$2
model_type=$3 ckpt_prefix=$3
model_type=$4
# download language model # download language model
bash local/download_lm_en.sh bash local/download_lm_en.sh
...@@ -21,6 +22,7 @@ fi ...@@ -21,6 +22,7 @@ fi
python3 -u ${BIN_DIR}/test.py \ python3 -u ${BIN_DIR}/test.py \
--ngpu ${ngpu} \ --ngpu ${ngpu} \
--config ${config_path} \ --config ${config_path} \
--decode_cfg ${decode_config_path} \
--result_file ${ckpt_prefix}.rsl \ --result_file ${ckpt_prefix}.rsl \
--checkpoint_path ${ckpt_prefix} \ --checkpoint_path ${ckpt_prefix} \
--model_type ${model_type} --model_type ${model_type}
......
examples/other/1xt2x/librispeech/run.sh
浏览文件 @ 4cab9f62
...@@ -5,6 +5,7 @@ source path.sh ...@@ -5,6 +5,7 @@ source path.sh
stage=0 stage=0
stop_stage=100 stop_stage=100
conf_path=conf/deepspeech2.yaml conf_path=conf/deepspeech2.yaml
decode_conf_path=conf/tuning/decode.yaml
avg_num=1 avg_num=1
model_type=offline model_type=offline
gpus=1 gpus=1
...@@ -23,5 +24,5 @@ fi ...@@ -23,5 +24,5 @@ fi
if [ ${stage} -le 1 ] && [ ${stop_stage} -ge 1 ]; then if [ ${stage} -le 1 ] && [ ${stop_stage} -ge 1 ]; then
# test ckpt avg_n # test ckpt avg_n
CUDA_VISIBLE_DEVICES=${gpus} ./local/test.sh ${conf_path} exp/${ckpt}/checkpoints/${v18_ckpt} ${model_type}|| exit -1 CUDA_VISIBLE_DEVICES=${gpus} ./local/test.sh ${conf_path} ${decode_conf_path} exp/${ckpt}/checkpoints/${v18_ckpt} ${model_type}|| exit -1
fi fi
examples/other/1xt2x/src_deepspeech2x/bin/test.py
浏览文件 @ 4cab9f62
...@@ -13,8 +13,8 @@ ...@@ -13,8 +13,8 @@
# limitations under the License. # limitations under the License.
"""Evaluation for DeepSpeech2 model.""" """Evaluation for DeepSpeech2 model."""
from src_deepspeech2x.test_model import DeepSpeech2Tester as Tester from src_deepspeech2x.test_model import DeepSpeech2Tester as Tester
from yacs.config import CfgNode
from paddlespeech.s2t.exps.deepspeech2.config import get_cfg_defaults
from paddlespeech.s2t.training.cli import default_argument_parser from paddlespeech.s2t.training.cli import default_argument_parser
from paddlespeech.s2t.utils.utility import print_arguments from paddlespeech.s2t.utils.utility import print_arguments
...@@ -41,9 +41,13 @@ if __name__ == "__main__": ...@@ -41,9 +41,13 @@ if __name__ == "__main__":
print("model_type:{}".format(args.model_type)) print("model_type:{}".format(args.model_type))
# https://yaml.org/type/float.html # https://yaml.org/type/float.html
config = get_cfg_defaults(args.model_type) config = CfgNode(new_allowed=True)
if args.config: if args.config:
config.merge_from_file(args.config) config.merge_from_file(args.config)
if args.decode_cfg:
decode_confs = CfgNode(new_allowed=True)
decode_confs.merge_from_file(args.decode_cfg)
config.decode = decode_confs
if args.opts: if args.opts:
config.merge_from_list(args.opts) config.merge_from_list(args.opts)
config.freeze() config.freeze()
......
examples/other/1xt2x/src_deepspeech2x/models/ds2/deepspeech2.py
浏览文件 @ 4cab9f62
...@@ -120,20 +120,6 @@ class DeepSpeech2Model(nn.Layer): ...@@ -120,20 +120,6 @@ class DeepSpeech2Model(nn.Layer):
:rtype: tuple of LayerOutput :rtype: tuple of LayerOutput
""" """
@classmethod
def params(cls, config: Optional[CfgNode]=None) -> CfgNode:
default = CfgNode(
dict(
num_conv_layers=2, #Number of stacking convolution layers.
num_rnn_layers=3, #Number of stacking RNN layers.
rnn_layer_size=1024, #RNN layer size (number of RNN cells).
use_gru=True, #Use gru if set True. Use simple rnn if set False.
share_rnn_weights=True #Whether to share input-hidden weights between forward and backward directional RNNs.Notice that for GRU, weight sharing is not supported.
))
if config is not None:
config.merge_from_other_cfg(default)
return default
def __init__(self, def __init__(self,
feat_size, feat_size,
dict_size, dict_size,
...@@ -233,11 +219,11 @@ class DeepSpeech2Model(nn.Layer): ...@@ -233,11 +219,11 @@ class DeepSpeech2Model(nn.Layer):
""" """
model = cls(feat_size=dataloader.collate_fn.feature_size, model = cls(feat_size=dataloader.collate_fn.feature_size,
dict_size=len(dataloader.collate_fn.vocab_list), dict_size=len(dataloader.collate_fn.vocab_list),
num_conv_layers=config.model.num_conv_layers, num_conv_layers=config.num_conv_layers,
num_rnn_layers=config.model.num_rnn_layers, num_rnn_layers=config.num_rnn_layers,
rnn_size=config.model.rnn_layer_size, rnn_size=config.rnn_layer_size,
use_gru=config.model.use_gru, use_gru=config.use_gru,
share_rnn_weights=config.model.share_rnn_weights) share_rnn_weights=config.share_rnn_weights)
infos = Checkpoint().load_parameters( infos = Checkpoint().load_parameters(
model, checkpoint_path=checkpoint_path) model, checkpoint_path=checkpoint_path)
logger.info(f"checkpoint info: {infos}") logger.info(f"checkpoint info: {infos}")
...@@ -250,7 +236,7 @@ class DeepSpeech2Model(nn.Layer): ...@@ -250,7 +236,7 @@ class DeepSpeech2Model(nn.Layer):
Parameters Parameters
config: yacs.config.CfgNode config: yacs.config.CfgNode
config.model config
Returns Returns
------- -------
DeepSpeech2Model DeepSpeech2Model
......
examples/other/1xt2x/src_deepspeech2x/test_model.py
浏览文件 @ 4cab9f62
...@@ -44,27 +44,11 @@ logger = Log(__name__).getlog() ...@@ -44,27 +44,11 @@ logger = Log(__name__).getlog()
class DeepSpeech2Trainer(Trainer): class DeepSpeech2Trainer(Trainer):
@classmethod
def params(cls, config: Optional[CfgNode]=None) -> CfgNode:
# training config
default = CfgNode(
dict(
lr=5e-4, # learning rate
lr_decay=1.0, # learning rate decay
weight_decay=1e-6, # the coeff of weight decay
global_grad_clip=5.0, # the global norm clip
n_epoch=50, # train epochs
))
if config is not None:
config.merge_from_other_cfg(default)
return default
def __init__(self, config, args): def __init__(self, config, args):
super().__init__(config, args) super().__init__(config, args)
def train_batch(self, batch_index, batch_data, msg): def train_batch(self, batch_index, batch_data, msg):
train_conf = self.config.training train_conf = self.config
start = time.time() start = time.time()
# forward # forward
...@@ -98,7 +82,7 @@ class DeepSpeech2Trainer(Trainer): ...@@ -98,7 +82,7 @@ class DeepSpeech2Trainer(Trainer):
iteration_time = time.time() - start iteration_time = time.time() - start
msg += "train time: {:>.3f}s, ".format(iteration_time) msg += "train time: {:>.3f}s, ".format(iteration_time)
msg += "batch size: {}, ".format(self.config.collator.batch_size) msg += "batch size: {}, ".format(self.config.batch_size)
msg += "accum: {}, ".format(train_conf.accum_grad) msg += "accum: {}, ".format(train_conf.accum_grad)
msg += ', '.join('{}: {:>.6f}'.format(k, v) msg += ', '.join('{}: {:>.6f}'.format(k, v)
for k, v in losses_np.items()) for k, v in losses_np.items())
...@@ -126,7 +110,7 @@ class DeepSpeech2Trainer(Trainer): ...@@ -126,7 +110,7 @@ class DeepSpeech2Trainer(Trainer):
total_loss += float(loss) * num_utts total_loss += float(loss) * num_utts
valid_losses['val_loss'].append(float(loss)) valid_losses['val_loss'].append(float(loss))
if (i + 1) % self.config.training.log_interval == 0: if (i + 1) % self.config.log_interval == 0:
valid_dump = {k: np.mean(v) for k, v in valid_losses.items()} valid_dump = {k: np.mean(v) for k, v in valid_losses.items()}
valid_dump['val_history_loss'] = total_loss / num_seen_utts valid_dump['val_history_loss'] = total_loss / num_seen_utts
...@@ -146,15 +130,15 @@ class DeepSpeech2Trainer(Trainer): ...@@ -146,15 +130,15 @@ class DeepSpeech2Trainer(Trainer):
def setup_model(self): def setup_model(self):
config = self.config.clone() config = self.config.clone()
config.defrost() config.defrost()
config.model.feat_size = self.train_loader.collate_fn.feature_size config.feat_size = self.train_loader.collate_fn.feature_size
#config.model.dict_size = self.train_loader.collate_fn.vocab_size #config.dict_size = self.train_loader.collate_fn.vocab_size
config.model.dict_size = len(self.train_loader.collate_fn.vocab_list) config.dict_size = len(self.train_loader.collate_fn.vocab_list)
config.freeze() config.freeze()
if self.args.model_type == 'offline': if self.args.model_type == 'offline':
model = DeepSpeech2Model.from_config(config.model) model = DeepSpeech2Model.from_config(config)
elif self.args.model_type == 'online': elif self.args.model_type == 'online':
model = DeepSpeech2ModelOnline.from_config(config.model) model = DeepSpeech2ModelOnline.from_config(config)
else: else:
raise Exception("wrong model type") raise Exception("wrong model type")
if self.parallel: if self.parallel:
...@@ -163,17 +147,13 @@ class DeepSpeech2Trainer(Trainer): ...@@ -163,17 +147,13 @@ class DeepSpeech2Trainer(Trainer):
logger.info(f"{model}") logger.info(f"{model}")
layer_tools.print_params(model, logger.info) layer_tools.print_params(model, logger.info)
grad_clip = ClipGradByGlobalNormWithLog( grad_clip = ClipGradByGlobalNormWithLog(config.global_grad_clip)
config.training.global_grad_clip)
lr_scheduler = paddle.optimizer.lr.ExponentialDecay( lr_scheduler = paddle.optimizer.lr.ExponentialDecay(
learning_rate=config.training.lr, learning_rate=config.lr, gamma=config.lr_decay, verbose=True)
gamma=config.training.lr_decay,
verbose=True)
optimizer = paddle.optimizer.Adam( optimizer = paddle.optimizer.Adam(
learning_rate=lr_scheduler, learning_rate=lr_scheduler,
parameters=model.parameters(), parameters=model.parameters(),
weight_decay=paddle.regularizer.L2Decay( weight_decay=paddle.regularizer.L2Decay(config.weight_decay),
config.training.weight_decay),
grad_clip=grad_clip) grad_clip=grad_clip)
self.model = model self.model = model
...@@ -184,59 +164,59 @@ class DeepSpeech2Trainer(Trainer): ...@@ -184,59 +164,59 @@ class DeepSpeech2Trainer(Trainer):
def setup_dataloader(self): def setup_dataloader(self):
config = self.config.clone() config = self.config.clone()
config.defrost() config.defrost()
config.collator.keep_transcription_text = False config.keep_transcription_text = False
config.data.manifest = config.data.train_manifest config.manifest = config.train_manifest
train_dataset = ManifestDataset.from_config(config) train_dataset = ManifestDataset.from_config(config)
config.data.manifest = config.data.dev_manifest config.manifest = config.dev_manifest
dev_dataset = ManifestDataset.from_config(config) dev_dataset = ManifestDataset.from_config(config)
config.data.manifest = config.data.test_manifest config.manifest = config.test_manifest
test_dataset = ManifestDataset.from_config(config) test_dataset = ManifestDataset.from_config(config)
if self.parallel: if self.parallel:
batch_sampler = SortagradDistributedBatchSampler( batch_sampler = SortagradDistributedBatchSampler(
train_dataset, train_dataset,
batch_size=config.collator.batch_size, batch_size=config.batch_size,
num_replicas=None, num_replicas=None,
rank=None, rank=None,
shuffle=True, shuffle=True,
drop_last=True, drop_last=True,
sortagrad=config.collator.sortagrad, sortagrad=config.sortagrad,
shuffle_method=config.collator.shuffle_method) shuffle_method=config.shuffle_method)
else: else:
batch_sampler = SortagradBatchSampler( batch_sampler = SortagradBatchSampler(
train_dataset, train_dataset,
shuffle=True, shuffle=True,
batch_size=config.collator.batch_size, batch_size=config.batch_size,
drop_last=True, drop_last=True,
sortagrad=config.collator.sortagrad, sortagrad=config.sortagrad,
shuffle_method=config.collator.shuffle_method) shuffle_method=config.shuffle_method)
collate_fn_train = SpeechCollator.from_config(config) collate_fn_train = SpeechCollator.from_config(config)
config.collator.augmentation_config = "" config.augmentation_config = ""
collate_fn_dev = SpeechCollator.from_config(config) collate_fn_dev = SpeechCollator.from_config(config)
config.collator.keep_transcription_text = True config.keep_transcription_text = True
config.collator.augmentation_config = "" config.augmentation_config = ""
collate_fn_test = SpeechCollator.from_config(config) collate_fn_test = SpeechCollator.from_config(config)
self.train_loader = DataLoader( self.train_loader = DataLoader(
train_dataset, train_dataset,
batch_sampler=batch_sampler, batch_sampler=batch_sampler,
collate_fn=collate_fn_train, collate_fn=collate_fn_train,
num_workers=config.collator.num_workers) num_workers=config.num_workers)
self.valid_loader = DataLoader( self.valid_loader = DataLoader(
dev_dataset, dev_dataset,
batch_size=config.collator.batch_size, batch_size=config.batch_size,
shuffle=False, shuffle=False,
drop_last=False, drop_last=False,
collate_fn=collate_fn_dev) collate_fn=collate_fn_dev)
self.test_loader = DataLoader( self.test_loader = DataLoader(
test_dataset, test_dataset,
batch_size=config.decoding.batch_size, batch_size=config.decode.decode_batch_size,
shuffle=False, shuffle=False,
drop_last=False, drop_last=False,
collate_fn=collate_fn_test) collate_fn=collate_fn_test)
...@@ -250,31 +230,10 @@ class DeepSpeech2Trainer(Trainer): ...@@ -250,31 +230,10 @@ class DeepSpeech2Trainer(Trainer):
class DeepSpeech2Tester(DeepSpeech2Trainer): class DeepSpeech2Tester(DeepSpeech2Trainer):
@classmethod
def params(cls, config: Optional[CfgNode]=None) -> CfgNode:
# testing config
default = CfgNode(
dict(
alpha=2.5, # Coef of LM for beam search.
beta=0.3, # Coef of WC for beam search.
cutoff_prob=1.0, # Cutoff probability for pruning.
cutoff_top_n=40, # Cutoff number for pruning.
lang_model_path='models/lm/common_crawl_00.prune01111.trie.klm', # Filepath for language model.
decoding_method='ctc_beam_search', # Decoding method. Options: ctc_beam_search, ctc_greedy
error_rate_type='wer', # Error rate type for evaluation. Options `wer`, 'cer'
num_proc_bsearch=8, # # of CPUs for beam search.
beam_size=500, # Beam search width.
batch_size=128, # decoding batch size
))
if config is not None:
config.merge_from_other_cfg(default)
return default
def __init__(self, config, args): def __init__(self, config, args):
self._text_featurizer = TextFeaturizer( self._text_featurizer = TextFeaturizer(
unit_type=config.collator.unit_type, vocab_filepath=None) unit_type=config.unit_type, vocab=None)
super().__init__(config, args) super().__init__(config, args)
def ordid2token(self, texts, texts_len): def ordid2token(self, texts, texts_len):
...@@ -293,7 +252,7 @@ class DeepSpeech2Tester(DeepSpeech2Trainer): ...@@ -293,7 +252,7 @@ class DeepSpeech2Tester(DeepSpeech2Trainer):
texts, texts,
texts_len, texts_len,
fout=None): fout=None):
cfg = self.config.decoding cfg = self.config.decode
errors_sum, len_refs, num_ins = 0.0, 0, 0 errors_sum, len_refs, num_ins = 0.0, 0, 0
errors_func = error_rate.char_errors if cfg.error_rate_type == 'cer' else error_rate.word_errors errors_func = error_rate.char_errors if cfg.error_rate_type == 'cer' else error_rate.word_errors
error_rate_func = error_rate.cer if cfg.error_rate_type == 'cer' else error_rate.wer error_rate_func = error_rate.cer if cfg.error_rate_type == 'cer' else error_rate.wer
...@@ -399,31 +358,3 @@ class DeepSpeech2Tester(DeepSpeech2Trainer): ...@@ -399,31 +358,3 @@ class DeepSpeech2Tester(DeepSpeech2Trainer):
self.export() self.export()
except KeyboardInterrupt: except KeyboardInterrupt:
exit(-1) exit(-1)
def setup(self):
"""Setup the experiment.
"""
paddle.set_device('gpu' if self.args.ngpu > 0 else 'cpu')
self.setup_output_dir()
self.setup_checkpointer()
self.setup_dataloader()
self.setup_model()
self.iteration = 0
self.epoch = 0
def setup_output_dir(self):
"""Create a directory used for output.
"""
# output dir
if self.args.output:
output_dir = Path(self.args.output).expanduser()
output_dir.mkdir(parents=True, exist_ok=True)
else:
output_dir = Path(
self.args.checkpoint_path).expanduser().parent.parent
output_dir.mkdir(parents=True, exist_ok=True)
self.output_dir = output_dir
examples/ted_en_zh/st0/conf/preprocess.yaml 0 → 100644
浏览文件 @ 4cab9f62
process:
# extract kaldi fbank from PCM
- type: fbank_kaldi
fs: 16000
n_mels: 80
n_shift: 160
win_length: 400
dither: 0.1
- type: cmvn_json
cmvn_path: data/mean_std.json
# these three processes are a.k.a. SpecAugument
- type: time_warp
max_time_warp: 5
inplace: true
mode: PIL
- type: freq_mask
F: 30
n_mask: 2
inplace: true
replace_with_zero: false
- type: time_mask
T: 40
n_mask: 2
inplace: true
replace_with_zero: false
examples/ted_en_zh/st0/conf/transformer.yaml
浏览文件 @ 4cab9f62
# https://yaml.org/type/float.html # https://yaml.org/type/float.html
data: ###########################################
train_manifest: data/manifest.train # Data #
dev_manifest: data/manifest.dev ###########################################
test_manifest: data/manifest.test train_manifest: data/manifest.train
min_input_len: 0.05 # second dev_manifest: data/manifest.dev
max_input_len: 30.0 # second test_manifest: data/manifest.test
min_output_len: 0.0 # tokens min_input_len: 0.05 # second
max_output_len: 400.0 # tokens max_input_len: 30.0 # second
min_output_input_ratio: 0.01 min_output_len: 0.0 # tokens
max_output_input_ratio: 20.0 max_output_len: 400.0 # tokens
min_output_input_ratio: 0.01
max_output_input_ratio: 20.0
collator: ###########################################
vocab_filepath: data/lang_char/vocab.txt # Dataloader #
unit_type: 'spm' ###########################################
spm_model_prefix: data/lang_char/bpe_unigram_8000 vocab_filepath: data/lang_char/vocab.txt
mean_std_filepath: "" unit_type: 'spm'
augmentation_config: conf/preprocess.yaml spm_model_prefix: data/lang_char/bpe_unigram_8000
batch_size: 16 mean_std_filepath: ""
maxlen_in: 5 # if input length > maxlen-in, batchsize is automatically reduced preprocess_config: conf/preprocess.yaml
maxlen_out: 150 # if output length > maxlen-out, batchsize is automatically reduced batch_size: 16
raw_wav: True # use raw_wav or kaldi feature maxlen_in: 5 # if input length > maxlen-in, batchsize is automatically reduced
spectrum_type: fbank #linear, mfcc, fbank maxlen_out: 150 # if output length > maxlen-out, batchsize is automatically reduced
feat_dim: 80 raw_wav: True # use raw_wav or kaldi feature
delta_delta: False spectrum_type: fbank #linear, mfcc, fbank
dither: 1.0 feat_dim: 80
target_sample_rate: 16000 delta_delta: False
max_freq: None dither: 1.0
n_fft: None target_sample_rate: 16000
stride_ms: 10.0 max_freq: None
window_ms: 25.0 n_fft: None
use_dB_normalization: True stride_ms: 10.0
target_dB: -20 window_ms: 25.0
random_seed: 0 use_dB_normalization: True
keep_transcription_text: False target_dB: -20
sortagrad: True random_seed: 0
shuffle_method: batch_shuffle keep_transcription_text: False
num_workers: 2 sortagrad: True
shuffle_method: batch_shuffle
num_workers: 2
# network architecture ############################################
model: # Network Architecture #
cmvn_file: "data/mean_std.json" ############################################
cmvn_file_type: "json" cmvn_file: "data/mean_std.json"
# encoder related cmvn_file_type: "json"
encoder: transformer # encoder related
encoder_conf: encoder: transformer
output_size: 256 # dimension of attention encoder_conf:
attention_heads: 4 output_size: 256 # dimension of attention
linear_units: 2048 # the number of units of position-wise feed forward attention_heads: 4
num_blocks: 12 # the number of encoder blocks linear_units: 2048 # the number of units of position-wise feed forward
dropout_rate: 0.1 num_blocks: 12 # the number of encoder blocks
positional_dropout_rate: 0.1 dropout_rate: 0.1
attention_dropout_rate: 0.0 positional_dropout_rate: 0.1
input_layer: conv2d # encoder input type, you can chose conv2d, conv2d6 and conv2d8 attention_dropout_rate: 0.0
normalize_before: true input_layer: conv2d # encoder input type, you can chose conv2d, conv2d6 and conv2d8
normalize_before: true
# decoder related # decoder related
decoder: transformer decoder: transformer
decoder_conf: decoder_conf:
attention_heads: 4 attention_heads: 4
linear_units: 2048 linear_units: 2048
num_blocks: 6 num_blocks: 6
dropout_rate: 0.1 dropout_rate: 0.1
positional_dropout_rate: 0.1 positional_dropout_rate: 0.1
self_attention_dropout_rate: 0.0 self_attention_dropout_rate: 0.0
src_attention_dropout_rate: 0.0 src_attention_dropout_rate: 0.0
# hybrid CTC/attention
model_conf:
asr_weight: 0.0
ctc_weight: 0.0
lsm_weight: 0.1 # label smoothing option
length_normalized_loss: false
training:
n_epoch: 120
accum_grad: 2
global_grad_clip: 5.0
optim: adam
optim_conf:
lr: 2.5
weight_decay: 1e-06
scheduler: noam
scheduler_conf:
warmup_steps: 25000
lr_decay: 1.0
log_interval: 50
checkpoint:
kbest_n: 50
latest_n: 5
decoding:
batch_size: 5
error_rate_type: char-bleu
decoding_method: fullsentence # 'fullsentence', 'simultaneous'
alpha: 2.5
beta: 0.3
beam_size: 10
word_reward: 0.7
cutoff_prob: 1.0
cutoff_top_n: 0
num_proc_bsearch: 8
ctc_weight: 0.5 # ctc weight for attention rescoring decode mode.
decoding_chunk_size: -1 # decoding chunk size. Defaults to -1.
# <0: for decoding, use full chunk.
# >0: for decoding, use fixed chunk size as set.
# 0: used for training, it's prohibited here.
num_decoding_left_chunks: -1 # number of left chunks for decoding. Defaults to -1.
simulate_streaming: False # simulate streaming inference. Defaults to False.
# hybrid CTC/attention
model_conf:
asr_weight: 0.0
ctc_weight: 0.0
lsm_weight: 0.1 # label smoothing option
length_normalized_loss: false
###########################################
# Training #
###########################################
n_epoch: 120
accum_grad: 2
global_grad_clip: 5.0
optim: adam
optim_conf:
lr: 2.5
weight_decay: 1.0e-06
scheduler: noam
scheduler_conf:
warmup_steps: 25000
lr_decay: 1.0
log_interval: 50
checkpoint:
kbest_n: 50
latest_n: 5
examples/ted_en_zh/st0/conf/transformer_mtl_noam.yaml
浏览文件 @ 4cab9f62
# https://yaml.org/type/float.html # https://yaml.org/type/float.html
data: ###########################################
train_manifest: data/manifest.train # Data #
dev_manifest: data/manifest.dev ###########################################
test_manifest: data/manifest.test train_manifest: data/manifest.train
min_input_len: 0.05 # second dev_manifest: data/manifest.dev
max_input_len: 30.0 # second test_manifest: data/manifest.test
min_output_len: 0.0 # tokens min_input_len: 0.05 # second
max_output_len: 400.0 # tokens max_input_len: 30.0 # second
min_output_input_ratio: 0.01 min_output_len: 0.0 # tokens
max_output_input_ratio: 20.0 max_output_len: 400.0 # tokens
min_output_input_ratio: 0.01
max_output_input_ratio: 20.0
collator: ###########################################
vocab_filepath: data/lang_char/vocab.txt # Dataloader #
unit_type: 'spm' ###########################################
spm_model_prefix: data/lang_char/bpe_unigram_8000 vocab_filepath: data/lang_char/vocab.txt
mean_std_filepath: "" unit_type: 'spm'
augmentation_config: conf/preprocess.yaml spm_model_prefix: data/lang_char/bpe_unigram_8000
batch_size: 16 mean_std_filepath: ""
maxlen_in: 5 # if input length > maxlen-in, batchsize is automatically reduced preprocess_config: conf/preprocess.yaml
maxlen_out: 150 # if output length > maxlen-out, batchsize is automatically reduced batch_size: 16
raw_wav: True # use raw_wav or kaldi feature maxlen_in: 5 # if input length > maxlen-in, batchsize is automatically reduced
spectrum_type: fbank #linear, mfcc, fbank maxlen_out: 150 # if output length > maxlen-out, batchsize is automatically reduced
feat_dim: 80 raw_wav: True # use raw_wav or kaldi feature
delta_delta: False spectrum_type: fbank #linear, mfcc, fbank
dither: 1.0 feat_dim: 80
target_sample_rate: 16000 delta_delta: False
max_freq: None dither: 1.0
n_fft: None target_sample_rate: 16000
stride_ms: 10.0 max_freq: None
window_ms: 25.0 n_fft: None
use_dB_normalization: True stride_ms: 10.0
target_dB: -20 window_ms: 25.0
random_seed: 0 use_dB_normalization: True
keep_transcription_text: False target_dB: -20
sortagrad: True random_seed: 0
shuffle_method: batch_shuffle keep_transcription_text: False
num_workers: 2 sortagrad: True
shuffle_method: batch_shuffle
num_workers: 2
# network architecture ############################################
model: # Network Architecture #
cmvn_file: "data/mean_std.json" ############################################
cmvn_file_type: "json" cmvn_file: "data/mean_std.json"
# encoder related cmvn_file_type: "json"
encoder: transformer # encoder related
encoder_conf: encoder: transformer
output_size: 256 # dimension of attention encoder_conf:
attention_heads: 4 output_size: 256 # dimension of attention
linear_units: 2048 # the number of units of position-wise feed forward attention_heads: 4
num_blocks: 12 # the number of encoder blocks linear_units: 2048 # the number of units of position-wise feed forward
dropout_rate: 0.1 num_blocks: 12 # the number of encoder blocks
positional_dropout_rate: 0.1 dropout_rate: 0.1
attention_dropout_rate: 0.0 positional_dropout_rate: 0.1
input_layer: conv2d # encoder input type, you can chose conv2d, conv2d6 and conv2d8 attention_dropout_rate: 0.0
normalize_before: true input_layer: conv2d # encoder input type, you can chose conv2d, conv2d6 and conv2d8
normalize_before: true
# decoder related # decoder related
decoder: transformer decoder: transformer
decoder_conf: decoder_conf:
attention_heads: 4 attention_heads: 4
linear_units: 2048 linear_units: 2048
num_blocks: 6 num_blocks: 6
dropout_rate: 0.1 dropout_rate: 0.1
positional_dropout_rate: 0.1 positional_dropout_rate: 0.1
self_attention_dropout_rate: 0.0 self_attention_dropout_rate: 0.0
src_attention_dropout_rate: 0.0 src_attention_dropout_rate: 0.0
# hybrid CTC/attention # hybrid CTC/attention
model_conf: model_conf:
asr_weight: 0.5 asr_weight: 0.5
ctc_weight: 0.3 ctc_weight: 0.3
lsm_weight: 0.1 # label smoothing option lsm_weight: 0.1 # label smoothing option
length_normalized_loss: false length_normalized_loss: false
training: ###########################################
n_epoch: 120 # Training #
accum_grad: 2 ###########################################
global_grad_clip: 5.0 n_epoch: 120
optim: adam accum_grad: 2
optim_conf: global_grad_clip: 5.0
lr: 2.5 optim: adam
weight_decay: 1e-06 optim_conf:
scheduler: noam lr: 2.5
scheduler_conf: weight_decay: 1.0e-06
warmup_steps: 25000 scheduler: noam
lr_decay: 1.0 scheduler_conf:
log_interval: 50 warmup_steps: 25000
checkpoint: lr_decay: 1.0
kbest_n: 50 log_interval: 50
latest_n: 5 checkpoint:
kbest_n: 50
latest_n: 5
decoding:
batch_size: 5
error_rate_type: char-bleu
decoding_method: fullsentence # 'fullsentence', 'simultaneous'
alpha: 2.5
beta: 0.3
beam_size: 10
word_reward: 0.7
cutoff_prob: 1.0
cutoff_top_n: 0
num_proc_bsearch: 8
ctc_weight: 0.5 # ctc weight for attention rescoring decode mode.
decoding_chunk_size: -1 # decoding chunk size. Defaults to -1.
# <0: for decoding, use full chunk.
# >0: for decoding, use fixed chunk size as set.
# 0: used for training, it's prohibited here.
num_decoding_left_chunks: -1 # number of left chunks for decoding. Defaults to -1.
simulate_streaming: False # simulate streaming inference. Defaults to False.
examples/ted_en_zh/st0/conf/tuning/decode.yaml 0 → 100644
浏览文件 @ 4cab9f62
batch_size: 5
error_rate_type: char-bleu
decoding_method: fullsentence # 'fullsentence', 'simultaneous'
beam_size: 10
word_reward: 0.7
decoding_chunk_size: -1 # decoding chunk size. Defaults to -1.
# <0: for decoding, use full chunk.
# >0: for decoding, use fixed chunk size as set.
# 0: used for training, it's prohibited here.
num_decoding_left_chunks: -1 # number of left chunks for decoding. Defaults to -1.
simulate_streaming: False # simulate streaming inference. Defaults to False.
\ No newline at end of file
examples/ted_en_zh/st0/local/test.sh
浏览文件 @ 4cab9f62
#! /usr/bin/env bash #! /usr/bin/env bash
if [ $# != 2 ];then if [ $# != 3 ];then
echo "usage: ${0} config_path ckpt_path_prefix" echo "usage: ${0} config_path decode_config_path ckpt_path_prefix"
exit -1 exit -1
fi fi
...@@ -9,16 +9,18 @@ ngpu=$(echo $CUDA_VISIBLE_DEVICES | awk -F "," '{print NF}') ...@@ -9,16 +9,18 @@ ngpu=$(echo $CUDA_VISIBLE_DEVICES | awk -F "," '{print NF}')
echo "using $ngpu gpus..." echo "using $ngpu gpus..."
config_path=$1 config_path=$1
ckpt_prefix=$2 decode_config_path=$2
ckpt_prefix=$3
for type in fullsentence; do for type in fullsentence; do
echo "decoding ${type}" echo "decoding ${type}"
python3 -u ${BIN_DIR}/test.py \ python3 -u ${BIN_DIR}/test.py \
--ngpu ${ngpu} \ --ngpu ${ngpu} \
--config ${config_path} \ --config ${config_path} \
--decode_cfg ${decode_config_path} \
--result_file ${ckpt_prefix}.${type}.rsl \ --result_file ${ckpt_prefix}.${type}.rsl \
--checkpoint_path ${ckpt_prefix} \ --checkpoint_path ${ckpt_prefix} \
--opts decoding.decoding_method ${type} \ --opts decode.decoding_method ${type} \
if [ $? -ne 0 ]; then if [ $? -ne 0 ]; then
echo "Failed in evaluation!" echo "Failed in evaluation!"
......
examples/ted_en_zh/st0/run.sh
浏览文件 @ 4cab9f62
...@@ -6,6 +6,7 @@ gpus=0,1,2,3 ...@@ -6,6 +6,7 @@ gpus=0,1,2,3
stage=0 stage=0
stop_stage=50 stop_stage=50
conf_path=conf/transformer_mtl_noam.yaml conf_path=conf/transformer_mtl_noam.yaml
decode_conf_path=conf/tuning/decode.yaml
avg_num=5 avg_num=5
data_path=./TED_EnZh # path to unzipped data data_path=./TED_EnZh # path to unzipped data
source ${MAIN_ROOT}/utils/parse_options.sh || exit 1; source ${MAIN_ROOT}/utils/parse_options.sh || exit 1;
...@@ -32,7 +33,7 @@ fi ...@@ -32,7 +33,7 @@ fi
if [ ${stage} -le 3 ] && [ ${stop_stage} -ge 3 ]; then if [ ${stage} -le 3 ] && [ ${stop_stage} -ge 3 ]; then
# test ckpt avg_n # test ckpt avg_n
CUDA_VISIBLE_DEVICES=0 ./local/test.sh ${conf_path} exp/${ckpt}/checkpoints/${avg_ckpt} || exit -1 CUDA_VISIBLE_DEVICES=0 ./local/test.sh ${conf_path} ${decode_conf_path} exp/${ckpt}/checkpoints/${avg_ckpt} || exit -1
fi fi
if [ ${stage} -le 51 ] && [ ${stop_stage} -ge 51 ]; then if [ ${stage} -le 51 ] && [ ${stop_stage} -ge 51 ]; then
......
examples/ted_en_zh/st1/conf/preprocess.yaml 0 → 100644
浏览文件 @ 4cab9f62
process:
# these three processes are a.k.a. SpecAugument
- type: time_warp
max_time_warp: 5
inplace: true
mode: PIL
- type: freq_mask
F: 30
n_mask: 2
inplace: true
replace_with_zero: false
- type: time_mask
T: 40
n_mask: 2
inplace: true
replace_with_zero: false
examples/ted_en_zh/st1/conf/transformer.yaml
浏览文件 @ 4cab9f62
# https://yaml.org/type/float.html # https://yaml.org/type/float.html
data: ###########################################
train_manifest: data/manifest.train # Data #
dev_manifest: data/manifest.dev ###########################################
test_manifest: data/manifest.test train_manifest: data/manifest.train
dev_manifest: data/manifest.dev
test_manifest: data/manifest.test
collator: ###########################################
vocab_filepath: data/lang_char/ted_en_zh_bpe8000.txt # Dataloader #
unit_type: 'spm' ###########################################
spm_model_prefix: data/lang_char/ted_en_zh_bpe8000 vocab_filepath: data/lang_char/ted_en_zh_bpe8000.txt
mean_std_filepath: "" unit_type: 'spm'
# augmentation_config: conf/augmentation.json spm_model_prefix: data/lang_char/ted_en_zh_bpe8000
batch_size: 20 mean_std_filepath: ""
feat_dim: 83 # preprocess_config: conf/augmentation.json
stride_ms: 10.0 batch_size: 20
window_ms: 25.0 feat_dim: 83
sortagrad: 0 # Feed samples from shortest to longest ; -1: enabled for all epochs, 0: disabled, other: enabled for 'other' epochs stride_ms: 10.0
maxlen_in: 512 # if input length > maxlen-in, batchsize is automatically reduced window_ms: 25.0
maxlen_out: 150 # if output length > maxlen-out, batchsize is automatically reduced sortagrad: 0 # Feed samples from shortest to longest ; -1: enabled for all epochs, 0: disabled, other: enabled for 'other' epochs
minibatches: 0 # for debug maxlen_in: 512 # if input length > maxlen-in, batchsize is automatically reduced
batch_count: auto maxlen_out: 150 # if output length > maxlen-out, batchsize is automatically reduced
batch_bins: 0 minibatches: 0 # for debug
batch_frames_in: 0 batch_count: auto
batch_frames_out: 0 batch_bins: 0
batch_frames_inout: 0 batch_frames_in: 0
augmentation_config: batch_frames_out: 0
num_workers: 0 batch_frames_inout: 0
subsampling_factor: 1 preprocess_config:
num_encs: 1 num_workers: 0
subsampling_factor: 1
num_encs: 1
############################################
# Network Architecture #
############################################
cmvn_file: None
cmvn_file_type: "json"
# encoder related
encoder: transformer
encoder_conf:
output_size: 256 # dimension of attention
attention_heads: 4
linear_units: 2048 # the number of units of position-wise feed forward
num_blocks: 12 # the number of encoder blocks
dropout_rate: 0.1
positional_dropout_rate: 0.1
attention_dropout_rate: 0.0
input_layer: conv2d # encoder input type, you can chose conv2d, conv2d6 and conv2d8
normalize_before: true
# network architecture # decoder related
model: decoder: transformer
cmvn_file: None decoder_conf:
cmvn_file_type: "json" attention_heads: 4
# encoder related linear_units: 2048
encoder: transformer num_blocks: 6
encoder_conf: dropout_rate: 0.1
output_size: 256 # dimension of attention positional_dropout_rate: 0.1
attention_heads: 4 self_attention_dropout_rate: 0.0
linear_units: 2048 # the number of units of position-wise feed forward src_attention_dropout_rate: 0.0
num_blocks: 12 # the number of encoder blocks
dropout_rate: 0.1
positional_dropout_rate: 0.1
attention_dropout_rate: 0.0
input_layer: conv2d # encoder input type, you can chose conv2d, conv2d6 and conv2d8
normalize_before: true
# decoder related # hybrid CTC/attention
decoder: transformer model_conf:
decoder_conf: asr_weight: 0.0
attention_heads: 4 ctc_weight: 0.0
linear_units: 2048 lsm_weight: 0.1 # label smoothing option
num_blocks: 6 length_normalized_loss: false
dropout_rate: 0.1
positional_dropout_rate: 0.1
self_attention_dropout_rate: 0.0
src_attention_dropout_rate: 0.0
# hybrid CTC/attention
model_conf:
asr_weight: 0.0
ctc_weight: 0.0
lsm_weight: 0.1 # label smoothing option
length_normalized_loss: false
###########################################
training: # Training #
n_epoch: 40 ###########################################
accum_grad: 2 n_epoch: 40
global_grad_clip: 5.0 accum_grad: 2
optim: adam global_grad_clip: 5.0
optim_conf: optim: adam
lr: 2.5 optim_conf:
weight_decay: 0. lr: 2.5
scheduler: noam weight_decay: 0.
scheduler_conf: scheduler: noam
warmup_steps: 25000 scheduler_conf:
lr_decay: 1.0 warmup_steps: 25000
log_interval: 50 lr_decay: 1.0
checkpoint: log_interval: 50
kbest_n: 50 checkpoint:
latest_n: 5 kbest_n: 50
latest_n: 5
decoding:
batch_size: 5
error_rate_type: char-bleu
decoding_method: fullsentence # 'fullsentence', 'simultaneous'
alpha: 2.5
beta: 0.3
beam_size: 10
word_reward: 0.7
cutoff_prob: 1.0
cutoff_top_n: 0
num_proc_bsearch: 8
ctc_weight: 0.5 # ctc weight for attention rescoring decode mode.
decoding_chunk_size: -1 # decoding chunk size. Defaults to -1.
# <0: for decoding, use full chunk.
# >0: for decoding, use fixed chunk size as set.
# 0: used for training, it's prohibited here.
num_decoding_left_chunks: -1 # number of left chunks for decoding. Defaults to -1.
simulate_streaming: False # simulate streaming inference. Defaults to False.
\ No newline at end of file
examples/ted_en_zh/st1/conf/transformer_mtl_noam.yaml
浏览文件 @ 4cab9f62
# https://yaml.org/type/float.html # https://yaml.org/type/float.html
data: ###########################################
train_manifest: data/manifest.train # Data #
dev_manifest: data/manifest.dev ###########################################
test_manifest: data/manifest.test train_manifest: data/manifest.train
dev_manifest: data/manifest.dev
test_manifest: data/manifest.test
collator: ###########################################
vocab_filepath: data/lang_char/ted_en_zh_bpe8000.txt # Dataloader #
unit_type: 'spm' ###########################################
spm_model_prefix: data/lang_char/ted_en_zh_bpe8000 vocab_filepath: data/lang_char/ted_en_zh_bpe8000.txt
mean_std_filepath: "" unit_type: 'spm'
# augmentation_config: conf/augmentation.json spm_model_prefix: data/lang_char/ted_en_zh_bpe8000
batch_size: 20 mean_std_filepath: ""
feat_dim: 83 # preprocess_config: conf/augmentation.json
stride_ms: 10.0 batch_size: 20
window_ms: 25.0 feat_dim: 83
sortagrad: 0 # Feed samples from shortest to longest ; -1: enabled for all epochs, 0: disabled, other: enabled for 'other' epochs stride_ms: 10.0
maxlen_in: 512 # if input length > maxlen-in, batchsize is automatically reduced window_ms: 25.0
maxlen_out: 150 # if output length > maxlen-out, batchsize is automatically reduced sortagrad: 0 # Feed samples from shortest to longest ; -1: enabled for all epochs, 0: disabled, other: enabled for 'other' epochs
minibatches: 0 # for debug maxlen_in: 512 # if input length > maxlen-in, batchsize is automatically reduced
batch_count: auto maxlen_out: 150 # if output length > maxlen-out, batchsize is automatically reduced
batch_bins: 0 minibatches: 0 # for debug
batch_frames_in: 0 batch_count: auto
batch_frames_out: 0 batch_bins: 0
batch_frames_inout: 0 batch_frames_in: 0
augmentation_config: batch_frames_out: 0
num_workers: 0 batch_frames_inout: 0
subsampling_factor: 1 preprocess_config:
num_encs: 1 num_workers: 0
subsampling_factor: 1
num_encs: 1
############################################
# Network Architecture #
############################################
cmvn_file: None
cmvn_file_type: "json"
# encoder related
encoder: transformer
encoder_conf:
output_size: 256 # dimension of attention
attention_heads: 4
linear_units: 2048 # the number of units of position-wise feed forward
num_blocks: 12 # the number of encoder blocks
dropout_rate: 0.1
positional_dropout_rate: 0.1
attention_dropout_rate: 0.0
input_layer: conv2d # encoder input type, you can chose conv2d, conv2d6 and conv2d8
normalize_before: true
# network architecture # decoder related
model: decoder: transformer
cmvn_file: None decoder_conf:
cmvn_file_type: "json" attention_heads: 4
# encoder related linear_units: 2048
encoder: transformer num_blocks: 6
encoder_conf: dropout_rate: 0.1
output_size: 256 # dimension of attention positional_dropout_rate: 0.1
attention_heads: 4 self_attention_dropout_rate: 0.0
linear_units: 2048 # the number of units of position-wise feed forward src_attention_dropout_rate: 0.0
num_blocks: 12 # the number of encoder blocks
dropout_rate: 0.1
positional_dropout_rate: 0.1
attention_dropout_rate: 0.0
input_layer: conv2d # encoder input type, you can chose conv2d, conv2d6 and conv2d8
normalize_before: true
# decoder related # hybrid CTC/attention
decoder: transformer model_conf:
decoder_conf: asr_weight: 0.5
attention_heads: 4 ctc_weight: 0.3
linear_units: 2048 lsm_weight: 0.1 # label smoothing option
num_blocks: 6 length_normalized_loss: false
dropout_rate: 0.1
positional_dropout_rate: 0.1
self_attention_dropout_rate: 0.0
src_attention_dropout_rate: 0.0
# hybrid CTC/attention
model_conf:
asr_weight: 0.5
ctc_weight: 0.3
lsm_weight: 0.1 # label smoothing option
length_normalized_loss: false
###########################################
training: # Training #
n_epoch: 40 ###########################################
accum_grad: 2 n_epoch: 40
global_grad_clip: 5.0 accum_grad: 2
optim: adam global_grad_clip: 5.0
optim_conf: optim: adam
lr: 2.5 optim_conf:
weight_decay: 0. lr: 2.5
scheduler: noam weight_decay: 0.
scheduler_conf: scheduler: noam
warmup_steps: 25000 scheduler_conf:
lr_decay: 1.0 warmup_steps: 25000
log_interval: 50 lr_decay: 1.0
checkpoint: log_interval: 50
kbest_n: 50 checkpoint:
latest_n: 5 kbest_n: 50
latest_n: 5
decoding:
batch_size: 5
error_rate_type: char-bleu
decoding_method: fullsentence # 'fullsentence', 'simultaneous'
alpha: 2.5
beta: 0.3
beam_size: 10
word_reward: 0.7
cutoff_prob: 1.0
cutoff_top_n: 0
num_proc_bsearch: 8
ctc_weight: 0.5 # ctc weight for attention rescoring decode mode.
decoding_chunk_size: -1 # decoding chunk size. Defaults to -1.
# <0: for decoding, use full chunk.
# >0: for decoding, use fixed chunk size as set.
# 0: used for training, it's prohibited here.
num_decoding_left_chunks: -1 # number of left chunks for decoding. Defaults to -1.
simulate_streaming: False # simulate streaming inference. Defaults to False.
\ No newline at end of file
examples/ted_en_zh/st1/conf/tuning/decode.yaml 0 → 100644
浏览文件 @ 4cab9f62
batch_size: 5
error_rate_type: char-bleu
decoding_method: fullsentence # 'fullsentence', 'simultaneous'
beam_size: 10
word_reward: 0.7
decoding_chunk_size: -1 # decoding chunk size. Defaults to -1.
# <0: for decoding, use full chunk.
# >0: for decoding, use fixed chunk size as set.
# 0: used for training, it's prohibited here.
num_decoding_left_chunks: -1 # number of left chunks for decoding. Defaults to -1.
simulate_streaming: False # simulate streaming inference. Defaults to False.
\ No newline at end of file
examples/ted_en_zh/st1/local/test.sh
浏览文件 @ 4cab9f62
#! /usr/bin/env bash #! /usr/bin/env bash
if [ $# != 2 ];then if [ $# != 3 ];then
echo "usage: ${0} config_path ckpt_path_prefix" echo "usage: ${0} config_path decode_config_path ckpt_path_prefix"
exit -1 exit -1
fi fi
...@@ -9,16 +9,18 @@ ngpu=$(echo $CUDA_VISIBLE_DEVICES | awk -F "," '{print NF}') ...@@ -9,16 +9,18 @@ ngpu=$(echo $CUDA_VISIBLE_DEVICES | awk -F "," '{print NF}')
echo "using $ngpu gpus..." echo "using $ngpu gpus..."
config_path=$1 config_path=$1
ckpt_prefix=$2 decode_config_path=$2
ckpt_prefix=$3
for type in fullsentence; do for type in fullsentence; do
echo "decoding ${type}" echo "decoding ${type}"
python3 -u ${BIN_DIR}/test.py \ python3 -u ${BIN_DIR}/test.py \
--ngpu ${ngpu} \ --ngpu ${ngpu} \
--config ${config_path} \ --config ${config_path} \
--decode_cfg ${decode_config_path} \
--result_file ${ckpt_prefix}.${type}.rsl \ --result_file ${ckpt_prefix}.${type}.rsl \
--checkpoint_path ${ckpt_prefix} \ --checkpoint_path ${ckpt_prefix} \
--opts decoding.decoding_method ${type} \ --opts decode.decoding_method ${type} \
if [ $? -ne 0 ]; then if [ $? -ne 0 ]; then
echo "Failed in evaluation!" echo "Failed in evaluation!"
......
examples/ted_en_zh/st1/run.sh
浏览文件 @ 4cab9f62
...@@ -7,6 +7,7 @@ gpus=0,1,2,3 ...@@ -7,6 +7,7 @@ gpus=0,1,2,3
stage=1 stage=1
stop_stage=4 stop_stage=4
conf_path=conf/transformer_mtl_noam.yaml conf_path=conf/transformer_mtl_noam.yaml
decode_conf_path=conf/tuning/decode.yaml
ckpt_path= # paddle.98 # (finetune from FAT-ST pretrained model) ckpt_path= # paddle.98 # (finetune from FAT-ST pretrained model)
avg_num=5 avg_num=5
data_path=./TED_EnZh # path to unzipped data data_path=./TED_EnZh # path to unzipped data
...@@ -27,7 +28,7 @@ if [ ${stage} -le 1 ] && [ ${stop_stage} -ge 1 ]; then ...@@ -27,7 +28,7 @@ if [ ${stage} -le 1 ] && [ ${stop_stage} -ge 1 ]; then
if [ -n "${ckpt_path}" ]; then if [ -n "${ckpt_path}" ]; then
echo "Finetune from Pretrained Model" ${ckpt_path} echo "Finetune from Pretrained Model" ${ckpt_path}
./local/download_pretrain.sh || exit -1 ./local/download_pretrain.sh || exit -1
fi fi
CUDA_VISIBLE_DEVICES=${gpus} ./local/train.sh ${conf_path} ${ckpt} "${ckpt_path}" CUDA_VISIBLE_DEVICES=${gpus} ./local/train.sh ${conf_path} ${ckpt} "${ckpt_path}"
fi fi
...@@ -38,5 +39,5 @@ fi ...@@ -38,5 +39,5 @@ fi
if [ ${stage} -le 3 ] && [ ${stop_stage} -ge 3 ]; then if [ ${stage} -le 3 ] && [ ${stop_stage} -ge 3 ]; then
# test ckpt avg_n # test ckpt avg_n
CUDA_VISIBLE_DEVICES=0 ./local/test.sh ${conf_path} exp/${ckpt}/checkpoints/${avg_ckpt} || exit -1 CUDA_VISIBLE_DEVICES=0 ./local/test.sh ${conf_path} ${decode_conf_path} exp/${ckpt}/checkpoints/${avg_ckpt} || exit -1
fi fi
\ No newline at end of file
examples/timit/asr1/conf/tuning/decode.yaml 0 → 100644
浏览文件 @ 4cab9f62
decode_batch_size: 64
error_rate_type: wer
decoding_method: attention # 'attention', 'ctc_greedy_search', 'ctc_prefix_beam_search', 'attention_rescoring'
beam_size: 10
ctc_weight: 0.5 # ctc weight for attention rescoring decode mode.
decoding_chunk_size: -1 # decoding chunk size. Defaults to -1.
# <0: for decoding, use full chunk.
# >0: for decoding, use fixed chunk size as set.
# 0: used for training, it's prohibited here.
num_decoding_left_chunks: -1 # number of left chunks for decoding. Defaults to -1.
simulate_streaming: False # simulate streaming inference. Defaults to False.
examples/timit/asr1/local/align.sh
浏览文件 @ 4cab9f62
此差异已折叠。
examples/timit/asr1/local/test.sh
浏览文件 @ 4cab9f62
此差异已折叠。
examples/timit/asr1/run.sh
浏览文件 @ 4cab9f62
此差异已折叠。
examples/tiny/asr0/conf/tuning/chunk_decode.yaml 0 → 100644
浏览文件 @ 4cab9f62
decode_batch_size: 128
error_rate_type: wer
decoding_method: ctc_beam_search
lang_model_path: data/lm/common_crawl_00.prune01111.trie.klm
alpha: 2.5
beta: 0.3
beam_size: 500
cutoff_prob: 1.0
cutoff_top_n: 40
num_proc_bsearch: 8
examples/tiny/asr0/conf/tuning/decode.yaml 0 → 100644
浏览文件 @ 4cab9f62
decode_batch_size: 128
error_rate_type: wer
decoding_method: ctc_beam_search
lang_model_path: data/lm/common_crawl_00.prune01111.trie.klm
alpha: 2.5
beta: 0.3
beam_size: 500
cutoff_prob: 1.0
cutoff_top_n: 40
num_proc_bsearch: 8
examples/tiny/asr0/local/test.sh
浏览文件 @ 4cab9f62
此差异已折叠。
examples/tiny/asr0/run.sh
浏览文件 @ 4cab9f62
此差异已折叠。
examples/tiny/asr1/conf/tuning/chunk_decode.yaml 0 → 100644
浏览文件 @ 4cab9f62
此差异已折叠。
examples/tiny/asr1/conf/tuning/decode.yaml 0 → 100644
浏览文件 @ 4cab9f62
此差异已折叠。
examples/tiny/asr1/local/align.sh
浏览文件 @ 4cab9f62
此差异已折叠。
examples/tiny/asr1/local/test.sh
浏览文件 @ 4cab9f62
此差异已折叠。
examples/tiny/asr1/run.sh
浏览文件 @ 4cab9f62
此差异已折叠。
examples/wenetspeech/asr1/conf/tuning/decode.yaml 0 → 100644
浏览文件 @ 4cab9f62
此差异已折叠。
examples/wenetspeech/asr1/run.sh
浏览文件 @ 4cab9f62
此差异已折叠。
paddlespeech/s2t/decoders/recog.py
浏览文件 @ 4cab9f62
此差异已折叠。
paddlespeech/s2t/exps/deepspeech2/config.py 已删除 100644 → 0
浏览文件 @ 03f8accd
此差异已折叠。
paddlespeech/s2t/exps/u2/config.py 已删除 100644 → 0
浏览文件 @ 03f8accd
此差异已折叠。
paddlespeech/s2t/exps/u2/model.py
浏览文件 @ 4cab9f62
此差异已折叠。
paddlespeech/s2t/exps/u2/trainer.py
浏览文件 @ 4cab9f62
此差异已折叠。
paddlespeech/s2t/exps/u2_st/config.py 已删除 100644 → 0
浏览文件 @ 03f8accd
此差异已折叠。
paddlespeech/s2t/io/collator.py
浏览文件 @ 4cab9f62
此差异已折叠。
paddlespeech/s2t/io/dataset.py
浏览文件 @ 4cab9f62
此差异已折叠。
paddlespeech/s2t/models/u2/u2.py
浏览文件 @ 4cab9f62
此差异已折叠。
paddlespeech/s2t/training/cli.py
浏览文件 @ 4cab9f62
此差异已折叠。
paddlespeech/s2t/utils/utility.py
浏览文件 @ 4cab9f62
此差异已折叠。
tests/benchmark/conformer/run.sh
浏览文件 @ 4cab9f62
此差异已折叠。
tests/chains/ds2/prepare.sh
浏览文件 @ 4cab9f62
此差异已折叠。
tests/chains/ds2/test.sh
浏览文件 @ 4cab9f62
此差异已折叠。
Markdown is supported
0% 或 .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑！
想要评论请 注册 或