Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
PaddlePaddle
DeepSpeech
提交
e79e00a6
D
DeepSpeech
项目概览
PaddlePaddle
/
DeepSpeech
大约 1 年 前同步成功
通知
206
Star
8425
Fork
1598
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
245
列表
看板
标记
里程碑
合并请求
3
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
D
DeepSpeech
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
245
Issue
245
列表
看板
标记
里程碑
合并请求
3
合并请求
3
Pages
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
提交
Issue看板
体验新版 GitCode,发现更多精彩内容 >>
提交
e79e00a6
编写于
11月 22, 2021
作者:
H
Hui Zhang
浏览文件
操作
浏览文件
下载
电子邮件补丁
差异文件
pack model
上级
0defc658
变更
5
隐藏空白更改
内联
并排
Showing
5 changed file
with
278 addition
and
20 deletion
+278
-20
examples/wenetspeech/asr1/README.md
examples/wenetspeech/asr1/README.md
+10
-20
examples/wenetspeech/asr1/RESULTS.md
examples/wenetspeech/asr1/RESULTS.md
+24
-0
examples/wenetspeech/asr1/utils
examples/wenetspeech/asr1/utils
+1
-0
utils/pack_model.sh
utils/pack_model.sh
+169
-0
utils/show_results.sh
utils/show_results.sh
+74
-0
未找到文件。
examples/wenetspeech/asr1/README.md
浏览文件 @
e79e00a6
#
WenetSpeech
#
# Pack Model
pack model to tar.gz, e.g.
## Conformer
```
bash
./utils/pack_model.sh
--preprocess_conf
conf/preprocess.yaml
--dict
data/vocab.txt conf/conformer.yaml
''
data/mean_std.json exp/conformer/checkpoints/wenetspeec
h.pdparams
| Model | Params | Config | Augmentation| Test set | Decode method | Loss | WER |
| --- | --- | --- | --- | --- | --- | --- | --- |
| conformer | 32.52 M | conf/conformer.yaml | spec_aug | dev | attention | | |
| conformer | 32.52 M | conf/conformer.yaml | spec_aug | test net | ctc_greedy_search | | |
| conformer | 32.52 M | conf/conformer.yaml | spec_aug | test meeting | ctc_prefix_beam_search | | |
| conformer | 32.52 M | conf/conformer.yaml | spec_aug | test net | attention_rescoring | | |
```
## Conformer Pretrain Model
Pretrain model from http://mobvoi-speech-public.ufile.ucloud.cn/public/wenet/wenetspeech/20211025_conformer_exp.tar.gz
| Model | Params | Config | Augmentation| Test set | Decode method | Loss | WER |
| --- | --- | --- | --- | --- | --- | --- | --- |
| conformer | 32.52 M | conf/conformer.yaml | spec_aug | aishell1 | attention | - | 0.048456 |
| conformer | 32.52 M | conf/conformer.yaml | spec_aug | aishell1 | ctc_greedy_search | - | 0.052534 |
| conformer | 32.52 M | conf/conformer.yaml | spec_aug | aishell1 | ctc_prefix_beam_search | - | 0.052915 |
| conformer | 32.52 M | conf/conformer.yaml | spec_aug | aishell1 | attention_rescoring | - | 0.047904 |
\ No newline at end of file
show model.tar.gz
```
tar tf model.tar.gz
```
examples/wenetspeech/asr1/RESULTS.md
0 → 100644
浏览文件 @
e79e00a6
# WenetSpeech
## Conformer
| Model | Params | Config | Augmentation| Test set | Decode method | Loss | WER |
| --- | --- | --- | --- | --- | --- | --- | --- |
| conformer | 32.52 M | conf/conformer.yaml | spec_aug | dev | attention | | |
| conformer | 32.52 M | conf/conformer.yaml | spec_aug | test net | ctc_greedy_search | | |
| conformer | 32.52 M | conf/conformer.yaml | spec_aug | test meeting | ctc_prefix_beam_search | | |
| conformer | 32.52 M | conf/conformer.yaml | spec_aug | test net | attention_rescoring | | |
## Conformer Pretrain Model
Pretrain model from http://mobvoi-speech-public.ufile.ucloud.cn/public/wenet/wenetspeech/20211025_conformer_exp.tar.gz
| Model | Params | Config | Augmentation| Test set | Decode method | Loss | WER |
| --- | --- | --- | --- | --- | --- | --- | --- |
| conformer | 32.52 M | conf/conformer.yaml | spec_aug | aishell1 | attention | - | 0.048456 |
| conformer | 32.52 M | conf/conformer.yaml | spec_aug | aishell1 | ctc_greedy_search | - | 0.052534 |
| conformer | 32.52 M | conf/conformer.yaml | spec_aug | aishell1 | ctc_prefix_beam_search | - | 0.052915 |
| conformer | 32.52 M | conf/conformer.yaml | spec_aug | aishell1 | attention_rescoring | - | 0.047904 |
\ No newline at end of file
examples/wenetspeech/asr1/utils
0 → 120000
浏览文件 @
e79e00a6
../../../utils
\ No newline at end of file
utils/pack_model.sh
0 → 100755
浏览文件 @
e79e00a6
#!/usr/bin/env bash
# Copyright 2019 Johns Hopkins University (Shinji Watanabe)
# Apache 2.0 (http://www.apache.org/licenses/LICENSE-2.0)
[
-f
./path.sh
]
&&
.
./path.sh
results
=
""
# e.g., "exp/tr_it_pytorch_train/decode_dt_it_decode/result.wrd.txt
# exp/tr_it_pytorch_train/decode_et_it_decode/result.wrd.txt"'
lm
=
""
dict
=
""
etc
=
""
outfile
=
"model"
preprocess_conf
=
""
help_message
=
$(
cat
<<
EOF
Usage:
$0
--lm <lm> --dict <dict> <train_conf> <dec_conf> <cmvn> <e2e>, for example:
<lm>: exp/train_rnnlm/rnnlm.model.best
<dict>: data/lang_char
<train_conf>: conf/train.yaml
<dec_conf>: conf/decode.yaml
<cmvn>: data/tr_it/cmvn.ark
<e2e>: exp/tr_it_pytorch_train/results/model.last10.avg.best
EOF
)
.
utils/parse_options.sh
echo
$PWD
echo
$dict
if
[
$#
!=
4
]
;
then
echo
"
${
help_message
}
"
exit
1
fi
tr_conf
=
$1
dec_conf
=
$2
cmvn
=
$3
e2e
=
$4
echo
" - Model files (archived to
${
outfile
}
.tar.gz by
\`\$
pack_model.sh
\`
)"
echo
" - model link: (put the model link manually.)"
# configs
if
[
-e
${
tr_conf
}
]
;
then
tar
cfh
${
outfile
}
.tar
${
tr_conf
}
echo
-n
" - training config file:
\`
"
echo
${
tr_conf
}
|
sed
-e
"s/
$/
\`
/"
else
echo
"missing
${
tr_conf
}
"
exit
1
fi
if
[
-e
${
dec_conf
}
]
;
then
tar
rfh
${
outfile
}
.tar
${
dec_conf
}
echo
-n
" - decoding config file:
\`
"
echo
${
dec_conf
}
|
sed
-e
"s/
$/
\`
/"
else
echo
"missing
${
dec_conf
}
"
exit
1
fi
# NOTE(kan-bayashi): preprocess conf is optional
if
[
-n
"
${
preprocess_conf
}
"
]
;
then
tar
rfh
${
outfile
}
.tar
${
preprocess_conf
}
echo
-n
" - preprocess config file:
\`
"
echo
${
preprocess_conf
}
|
sed
-e
"s/
$/
\`
/"
fi
# cmvn
if
[
-e
${
cmvn
}
]
;
then
tar
rfh
${
outfile
}
.tar
${
cmvn
}
echo
-n
" - cmvn file:
\`
"
echo
${
cmvn
}
|
sed
-e
"s/
$/
\`
/"
else
echo
"missing
${
cmvn
}
"
exit
1
fi
# e2e
if
[
-e
${
e2e
}
]
;
then
tar
rfh
${
outfile
}
.tar
${
e2e
}
echo
-n
" - e2e file:
\`
"
echo
${
e2e
}
|
sed
-e
"s/
$/
\`
/"
e2e_conf
=
$(
dirname
${
e2e
}
)
/model.json
if
[
!
-e
${
e2e_conf
}
]
;
then
echo
missing
${
e2e_conf
}
#exit 1
else
echo
-n
" - e2e JSON file:
\`
"
echo
${
e2e_conf
}
|
sed
-e
"s/
$/
\`
/"
tar
rfh
${
outfile
}
.tar
${
e2e_conf
}
fi
else
echo
"missing
${
e2e
}
"
exit
1
fi
# lm
if
[
-n
"
${
lm
}
"
]
;
then
if
[
-e
${
lm
}
]
;
then
tar
rfh
${
outfile
}
.tar
${
lm
}
echo
-n
" - lm file:
\`
"
echo
${
lm
}
|
sed
-e
"s/
$/
\`
/"
lm_conf
=
$(
dirname
${
lm
}
)
/model.json
if
[
!
-e
${
lm_conf
}
]
;
then
echo
missing
${
lm_conf
}
exit
1
else
echo
-n
" - lm JSON file:
\`
"
echo
${
lm_conf
}
|
sed
-e
"s/
$/
\`
/"
tar
rfh
${
outfile
}
.tar
${
lm_conf
}
fi
else
echo
"missing
${
lm
}
"
exit
1
fi
fi
echo
${
dict
}
echo test
-e
${
dict
}
# dict
if
[
-n
"
${
dict
}
"
]
;
then
if
[
-e
${
dict
}
]
;
then
tar
rfh
${
outfile
}
.tar
${
dict
}
echo
-n
" - dict file:
\`
"
echo
${
dict
}
|
sed
-e
"s/
$/
\`
/"
else
echo
"missing
${
dict
}
"
exit
1
fi
fi
# etc
for
x
in
${
etc
}
;
do
if
[
-e
${
x
}
]
;
then
tar
rfh
${
outfile
}
.tar
${
x
}
echo
-n
" - etc file:
\`
"
echo
${
x
}
|
sed
-e
"s/
$/
\`
/"
else
echo
"missing
${
x
}
"
exit
1
fi
done
# finally compress the tar file
gzip
-f
${
outfile
}
.tar
# results
if
[
-n
"
${
results
}
"
]
;
then
echo
" - Results (paste them by yourself or obtained by
\`\$
pack_model.sh --results <results>
\`
)"
echo
"
\`\`\`
"
fi
for
x
in
${
results
}
;
do
if
[
-e
${
x
}
]
;
then
echo
"
${
x
}
"
grep
-e
Avg
-e
SPKR
-m
2
${
x
}
else
echo
"missing
${
x
}
"
exit
1
fi
done
if
[
-n
"
${
results
}
"
]
;
then
echo
"
\`\`\`
"
fi
exit
0
utils/show_results.sh
0 → 100755
浏览文件 @
e79e00a6
#!/usr/bin/env bash
mindepth
=
0
maxdepth
=
1
.
utils/parse_options.sh
if
[
$#
-gt
1
]
;
then
echo
"Usage:
$0
--mindepth 0 --maxdepth 1 [exp]"
1>&2
echo
""
echo
"Show the system environments and the evaluation results in Markdown format."
echo
'The default of <exp> is "exp/".'
exit
1
fi
[
-f
./path.sh
]
&&
.
./path.sh
set
-euo
pipefail
if
[
$#
-eq
1
]
;
then
exp
=
$1
else
exp
=
exp
fi
cat
<<
EOF
<!-- Generated by
$0
-->
# RESULTS
## Environments
- date:
\`
$(
LC_ALL
=
C
date
)
\`
EOF
python3
<<
EOF
import sys, paddle
pyversion = sys.version.replace('
\n
', ' ')
print(f"""- python version:
\`
{pyversion}
\`
- paddle version:
\`
paddle {paddle.__version__}
\`
""")
EOF
cat
<<
EOF
- Git hash:
\`
$(
git rev-parse HEAD
)
\`
- Commit date:
\`
$(
git log
-1
--format
=
'%cd'
)
\`
EOF
while
IFS
=
read
-r
expdir
;
do
if
ls
${
expdir
}
/decode_
*
/result.txt &> /dev/null
;
then
# 1. Show the result table
cat
<<
EOF
##
$(
basename
${
expdir
}
)
### CER
|dataset|Snt|Wrd|Corr|Sub|Del|Ins|Err|S.Err|
|---|---|---|---|---|---|---|---|---|
EOF
grep
-e
Avg
${
expdir
}
/decode_
*
/result.txt
\
|
sed
-e
"s#
${
expdir
}
/
\(
[^/]*
\)
/result.txt:#|
\1
#g"
\
|
sed
-e
's#Sum/Avg##g'
|
tr
'|'
' '
|
tr
-s
' '
'|'
echo
# 2. Show the result table for WER
if
ls
${
expdir
}
/decode_
*
/result.wrd.txt &> /dev/null
;
then
cat
<<
EOF
### WER
|dataset|Snt|Wrd|Corr|Sub|Del|Ins|Err|S.Err|
|---|---|---|---|---|---|---|---|---|
EOF
grep
-e
Avg
${
expdir
}
/decode_
*
/result.wrd.txt
\
|
sed
-e
"s#
${
expdir
}
/
\(
[^/]*
\)
/result.wrd.txt:#|
\1
#g"
\
|
sed
-e
's#Sum/Avg##g'
|
tr
'|'
' '
|
tr
-s
' '
'|'
echo
fi
fi
done
< <
(
find
${
exp
}
-mindepth
${
mindepth
}
-maxdepth
${
maxdepth
}
-type
d
)
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录