提交 fe83adfb 编写于 作者: H Hui Zhang

nproc to ngpu

上级 4d46cc93
...@@ -290,7 +290,7 @@ The following is the basic `ArgumentParser`: ...@@ -290,7 +290,7 @@ The following is the basic `ArgumentParser`:
1. `--config` is used to support configuration file parsing, and the configuration file itself handles the unique options of each experiment. 1. `--config` is used to support configuration file parsing, and the configuration file itself handles the unique options of each experiment.
2. `--train-metadata` is the path to the training data. 2. `--train-metadata` is the path to the training data.
3. `--output-dir` is the dir to save the training results.(if there are checkpoints in `checkpoints/` of `--output-dir` , it's defalut to reload the newest checkpoint to train) 3. `--output-dir` is the dir to save the training results.(if there are checkpoints in `checkpoints/` of `--output-dir` , it's defalut to reload the newest checkpoint to train)
4. `--device` and `--nprocs` determine operation modes,`--device` specifies the type of running device, whether to run on `cpu` or `gpu`. `--nprocs` refers to the number of training processes. If `nprocs` > 1, it means that multi process parallel training is used. (Note: currently only GPU multi card multi process training is supported.) 4. `--ngpu` determine operation modes,`--ngpu` refers to the number of training processes. If `ngpu` > 0, it means using GPU, else CPU is used.
Developers can refer to the examples in `examples` to write the default configuration file when adding new experiments. Developers can refer to the examples in `examples` to write the default configuration file when adding new experiments.
......
...@@ -14,7 +14,7 @@ jit_model_export_path=$3 ...@@ -14,7 +14,7 @@ jit_model_export_path=$3
model_type=$4 model_type=$4
python3 -u ${BIN_DIR}/export.py \ python3 -u ${BIN_DIR}/export.py \
--nproc ${ngpu} \ --ngpu ${ngpu} \
--config ${config_path} \ --config ${config_path} \
--checkpoint_path ${ckpt_path_prefix} \ --checkpoint_path ${ckpt_path_prefix} \
--export_path ${jit_model_export_path} \ --export_path ${jit_model_export_path} \
......
...@@ -19,7 +19,7 @@ if [ $? -ne 0 ]; then ...@@ -19,7 +19,7 @@ if [ $? -ne 0 ]; then
fi fi
python3 -u ${BIN_DIR}/test.py \ python3 -u ${BIN_DIR}/test.py \
--nproc ${ngpu} \ --ngpu ${ngpu} \
--config ${config_path} \ --config ${config_path} \
--result_file ${ckpt_prefix}.rsl \ --result_file ${ckpt_prefix}.rsl \
--checkpoint_path ${ckpt_prefix} \ --checkpoint_path ${ckpt_prefix} \
......
...@@ -19,7 +19,7 @@ if [ $? -ne 0 ]; then ...@@ -19,7 +19,7 @@ if [ $? -ne 0 ]; then
fi fi
python3 -u ${BIN_DIR}/test_export.py \ python3 -u ${BIN_DIR}/test_export.py \
--nproc ${ngpu} \ --ngpu ${ngpu} \
--config ${config_path} \ --config ${config_path} \
--result_file ${jit_model_export_path}.rsl \ --result_file ${jit_model_export_path}.rsl \
--export_path ${jit_model_export_path} \ --export_path ${jit_model_export_path} \
......
...@@ -20,7 +20,7 @@ if [ $? -ne 0 ]; then ...@@ -20,7 +20,7 @@ if [ $? -ne 0 ]; then
fi fi
python3 -u ${BIN_DIR}/test_hub.py \ python3 -u ${BIN_DIR}/test_hub.py \
--nproc ${ngpu} \ --ngpu ${ngpu} \
--config ${config_path} \ --config ${config_path} \
--result_file ${ckpt_prefix}.rsl \ --result_file ${ckpt_prefix}.rsl \
--checkpoint_path ${ckpt_prefix} \ --checkpoint_path ${ckpt_prefix} \
......
...@@ -21,7 +21,7 @@ if [ ${seed} != 0 ]; then ...@@ -21,7 +21,7 @@ if [ ${seed} != 0 ]; then
fi fi
python3 -u ${BIN_DIR}/train.py \ python3 -u ${BIN_DIR}/train.py \
--nproc ${ngpu} \ --ngpu ${ngpu} \
--config ${config_path} \ --config ${config_path} \
--output exp/${ckpt_name} \ --output exp/${ckpt_name} \
--model_type ${model_type} \ --model_type ${model_type} \
......
...@@ -18,7 +18,7 @@ mkdir -p ${output_dir} ...@@ -18,7 +18,7 @@ mkdir -p ${output_dir}
# align dump in `result_file` # align dump in `result_file`
# .tier, .TextGrid dump in `dir of result_file` # .tier, .TextGrid dump in `dir of result_file`
python3 -u ${BIN_DIR}/alignment.py \ python3 -u ${BIN_DIR}/alignment.py \
--nproc ${ngpu} \ --ngpu ${ngpu} \
--config ${config_path} \ --config ${config_path} \
--result_file ${output_dir}/${type}.align \ --result_file ${output_dir}/${type}.align \
--checkpoint_path ${ckpt_prefix} \ --checkpoint_path ${ckpt_prefix} \
......
...@@ -13,7 +13,7 @@ ckpt_path_prefix=$2 ...@@ -13,7 +13,7 @@ ckpt_path_prefix=$2
jit_model_export_path=$3 jit_model_export_path=$3
python3 -u ${BIN_DIR}/export.py \ python3 -u ${BIN_DIR}/export.py \
--nproc ${ngpu} \ --ngpu ${ngpu} \
--config ${config_path} \ --config ${config_path} \
--checkpoint_path ${ckpt_path_prefix} \ --checkpoint_path ${ckpt_path_prefix} \
--export_path ${jit_model_export_path} --export_path ${jit_model_export_path}
......
...@@ -34,7 +34,7 @@ for type in attention ctc_greedy_search; do ...@@ -34,7 +34,7 @@ for type in attention ctc_greedy_search; do
output_dir=${ckpt_prefix} output_dir=${ckpt_prefix}
mkdir -p ${output_dir} mkdir -p ${output_dir}
python3 -u ${BIN_DIR}/test.py \ python3 -u ${BIN_DIR}/test.py \
--nproc ${ngpu} \ --ngpu ${ngpu} \
--config ${config_path} \ --config ${config_path} \
--result_file ${output_dir}/${type}.rsl \ --result_file ${output_dir}/${type}.rsl \
--checkpoint_path ${ckpt_prefix} \ --checkpoint_path ${ckpt_prefix} \
...@@ -53,7 +53,7 @@ for type in ctc_prefix_beam_search attention_rescoring; do ...@@ -53,7 +53,7 @@ for type in ctc_prefix_beam_search attention_rescoring; do
output_dir=${ckpt_prefix} output_dir=${ckpt_prefix}
mkdir -p ${output_dir} mkdir -p ${output_dir}
python3 -u ${BIN_DIR}/test.py \ python3 -u ${BIN_DIR}/test.py \
--nproc ${ngpu} \ --ngpu ${ngpu} \
--config ${config_path} \ --config ${config_path} \
--result_file ${output_dir}/${type}.rsl \ --result_file ${output_dir}/${type}.rsl \
--checkpoint_path ${ckpt_prefix} \ --checkpoint_path ${ckpt_prefix} \
......
...@@ -29,7 +29,7 @@ for type in attention_rescoring; do ...@@ -29,7 +29,7 @@ for type in attention_rescoring; do
output_dir=${ckpt_prefix} output_dir=${ckpt_prefix}
mkdir -p ${output_dir} mkdir -p ${output_dir}
python3 -u ${BIN_DIR}/test_hub.py \ python3 -u ${BIN_DIR}/test_hub.py \
--nproc ${ngpu} \ --ngpu ${ngpu} \
--config ${config_path} \ --config ${config_path} \
--result_file ${output_dir}/${type}.rsl \ --result_file ${output_dir}/${type}.rsl \
--checkpoint_path ${ckpt_prefix} \ --checkpoint_path ${ckpt_prefix} \
......
...@@ -29,7 +29,7 @@ mkdir -p exp ...@@ -29,7 +29,7 @@ mkdir -p exp
python3 -u ${BIN_DIR}/train.py \ python3 -u ${BIN_DIR}/train.py \
--seed ${seed} \ --seed ${seed} \
--nproc ${ngpu} \ --ngpu ${ngpu} \
--config ${config_path} \ --config ${config_path} \
--output exp/${ckpt_name} \ --output exp/${ckpt_name} \
--profiler-options "${profiler_options}" \ --profiler-options "${profiler_options}" \
......
...@@ -23,7 +23,7 @@ mkdir -p ${output_dir} ...@@ -23,7 +23,7 @@ mkdir -p ${output_dir}
# align dump in `result_file` # align dump in `result_file`
# .tier, .TextGrid dump in `dir of result_file` # .tier, .TextGrid dump in `dir of result_file`
python3 -u ${BIN_DIR}/alignment.py \ python3 -u ${BIN_DIR}/alignment.py \
--nproc ${ngpu} \ --ngpu ${ngpu} \
--config ${config_path} \ --config ${config_path} \
--result_file ${output_dir}/${type}.align \ --result_file ${output_dir}/${type}.align \
--checkpoint_path ${ckpt_prefix} \ --checkpoint_path ${ckpt_prefix} \
......
...@@ -13,7 +13,7 @@ ckpt_path_prefix=$2 ...@@ -13,7 +13,7 @@ ckpt_path_prefix=$2
jit_model_export_path=$3 jit_model_export_path=$3
python3 -u ${BIN_DIR}/export.py \ python3 -u ${BIN_DIR}/export.py \
--nproc ${ngpu} \ --ngpu ${ngpu} \
--config ${config_path} \ --config ${config_path} \
--checkpoint_path ${ckpt_path_prefix} \ --checkpoint_path ${ckpt_path_prefix} \
--export_path ${jit_model_export_path} --export_path ${jit_model_export_path}
......
...@@ -28,7 +28,7 @@ for type in attention ctc_greedy_search; do ...@@ -28,7 +28,7 @@ for type in attention ctc_greedy_search; do
output_dir=${ckpt_prefix} output_dir=${ckpt_prefix}
mkdir -p ${output_dir} mkdir -p ${output_dir}
python3 -u ${BIN_DIR}/test.py \ python3 -u ${BIN_DIR}/test.py \
--nproc ${ngpu} \ --ngpu ${ngpu} \
--config ${config_path} \ --config ${config_path} \
--result_file ${output_dir}/${type}.rsl \ --result_file ${output_dir}/${type}.rsl \
--checkpoint_path ${ckpt_prefix} \ --checkpoint_path ${ckpt_prefix} \
...@@ -47,7 +47,7 @@ for type in ctc_prefix_beam_search attention_rescoring; do ...@@ -47,7 +47,7 @@ for type in ctc_prefix_beam_search attention_rescoring; do
output_dir=${ckpt_prefix} output_dir=${ckpt_prefix}
mkdir -p ${output_dir} mkdir -p ${output_dir}
python3 -u ${BIN_DIR}/test.py \ python3 -u ${BIN_DIR}/test.py \
--nproc ${ngpu} \ --ngpu ${ngpu} \
--config ${config_path} \ --config ${config_path} \
--result_file ${output_dir}/${type}.rsl \ --result_file ${output_dir}/${type}.rsl \
--checkpoint_path ${ckpt_prefix} \ --checkpoint_path ${ckpt_prefix} \
......
...@@ -22,7 +22,7 @@ if [ ${seed} != 0 ]; then ...@@ -22,7 +22,7 @@ if [ ${seed} != 0 ]; then
fi fi
python3 -u ${BIN_DIR}/train.py \ python3 -u ${BIN_DIR}/train.py \
--nproc ${ngpu} \ --ngpu ${ngpu} \
--config ${config_path} \ --config ${config_path} \
--output exp/${ckpt_name} \ --output exp/${ckpt_name} \
--seed ${seed} --seed ${seed}
......
...@@ -14,7 +14,7 @@ jit_model_export_path=$3 ...@@ -14,7 +14,7 @@ jit_model_export_path=$3
model_type=$4 model_type=$4
python3 -u ${BIN_DIR}/export.py \ python3 -u ${BIN_DIR}/export.py \
--nproc ${ngpu} \ --ngpu ${ngpu} \
--config ${config_path} \ --config ${config_path} \
--checkpoint_path ${ckpt_path_prefix} \ --checkpoint_path ${ckpt_path_prefix} \
--export_path ${jit_model_export_path} \ --export_path ${jit_model_export_path} \
......
...@@ -19,7 +19,7 @@ if [ $? -ne 0 ]; then ...@@ -19,7 +19,7 @@ if [ $? -ne 0 ]; then
fi fi
python3 -u ${BIN_DIR}/test.py \ python3 -u ${BIN_DIR}/test.py \
--nproc ${ngpu} \ --ngpu ${ngpu} \
--config ${config_path} \ --config ${config_path} \
--result_file ${ckpt_prefix}.rsl \ --result_file ${ckpt_prefix}.rsl \
--checkpoint_path ${ckpt_prefix} \ --checkpoint_path ${ckpt_prefix} \
......
...@@ -20,7 +20,7 @@ if [ $? -ne 0 ]; then ...@@ -20,7 +20,7 @@ if [ $? -ne 0 ]; then
fi fi
python3 -u ${BIN_DIR}/test_hub.py \ python3 -u ${BIN_DIR}/test_hub.py \
--nproc ${ngpu} \ --ngpu ${ngpu} \
--config ${config_path} \ --config ${config_path} \
--result_file ${ckpt_prefix}.rsl \ --result_file ${ckpt_prefix}.rsl \
--checkpoint_path ${ckpt_prefix} \ --checkpoint_path ${ckpt_prefix} \
......
...@@ -21,7 +21,7 @@ if [ ${seed} != 0 ]; then ...@@ -21,7 +21,7 @@ if [ ${seed} != 0 ]; then
fi fi
python3 -u ${BIN_DIR}/train.py \ python3 -u ${BIN_DIR}/train.py \
--nproc ${ngpu} \ --ngpu ${ngpu} \
--config ${config_path} \ --config ${config_path} \
--output exp/${ckpt_name} \ --output exp/${ckpt_name} \
--model_type ${model_type} \ --model_type ${model_type} \
......
...@@ -18,7 +18,7 @@ mkdir -p ${output_dir} ...@@ -18,7 +18,7 @@ mkdir -p ${output_dir}
# align dump in `result_file` # align dump in `result_file`
# .tier, .TextGrid dump in `dir of result_file` # .tier, .TextGrid dump in `dir of result_file`
python3 -u ${BIN_DIR}/alignment.py \ python3 -u ${BIN_DIR}/alignment.py \
--nproc ${ngpu} \ --ngpu ${ngpu} \
--config ${config_path} \ --config ${config_path} \
--result_file ${output_dir}/${type}.align \ --result_file ${output_dir}/${type}.align \
--checkpoint_path ${ckpt_prefix} \ --checkpoint_path ${ckpt_prefix} \
......
...@@ -13,7 +13,7 @@ ckpt_path_prefix=$2 ...@@ -13,7 +13,7 @@ ckpt_path_prefix=$2
jit_model_export_path=$3 jit_model_export_path=$3
python3 -u ${BIN_DIR}/export.py \ python3 -u ${BIN_DIR}/export.py \
--nproc ${ngpu} \ --ngpu ${ngpu} \
--config ${config_path} \ --config ${config_path} \
--checkpoint_path ${ckpt_path_prefix} \ --checkpoint_path ${ckpt_path_prefix} \
--export_path ${jit_model_export_path} --export_path ${jit_model_export_path}
......
...@@ -50,7 +50,7 @@ if [ ${stage} -le 0 ] && [ ${stop_stage} -ge 0 ]; then ...@@ -50,7 +50,7 @@ if [ ${stage} -le 0 ] && [ ${stop_stage} -ge 0 ]; then
batch_size=64 batch_size=64
fi fi
python3 -u ${BIN_DIR}/test.py \ python3 -u ${BIN_DIR}/test.py \
--nproc ${ngpu} \ --ngpu ${ngpu} \
--config ${config_path} \ --config ${config_path} \
--result_file ${ckpt_prefix}.${type}.rsl \ --result_file ${ckpt_prefix}.${type}.rsl \
--checkpoint_path ${ckpt_prefix} \ --checkpoint_path ${ckpt_prefix} \
...@@ -74,7 +74,7 @@ for type in ctc_greedy_search; do ...@@ -74,7 +74,7 @@ for type in ctc_greedy_search; do
batch_size=64 batch_size=64
fi fi
python3 -u ${BIN_DIR}/test.py \ python3 -u ${BIN_DIR}/test.py \
--nproc ${ngpu} \ --ngpu ${ngpu} \
--config ${config_path} \ --config ${config_path} \
--result_file ${ckpt_prefix}.${type}.rsl \ --result_file ${ckpt_prefix}.${type}.rsl \
--checkpoint_path ${ckpt_prefix} \ --checkpoint_path ${ckpt_prefix} \
...@@ -94,7 +94,7 @@ for type in ctc_prefix_beam_search attention_rescoring; do ...@@ -94,7 +94,7 @@ for type in ctc_prefix_beam_search attention_rescoring; do
echo "decoding ${type}" echo "decoding ${type}"
batch_size=1 batch_size=1
python3 -u ${BIN_DIR}/test.py \ python3 -u ${BIN_DIR}/test.py \
--nproc ${ngpu} \ --ngpu ${ngpu} \
--config ${config_path} \ --config ${config_path} \
--result_file ${ckpt_prefix}.${type}.rsl \ --result_file ${ckpt_prefix}.${type}.rsl \
--checkpoint_path ${ckpt_prefix} \ --checkpoint_path ${ckpt_prefix} \
......
...@@ -36,7 +36,7 @@ for type in attention_rescoring; do ...@@ -36,7 +36,7 @@ for type in attention_rescoring; do
output_dir=${ckpt_prefix} output_dir=${ckpt_prefix}
mkdir -p ${output_dir} mkdir -p ${output_dir}
python3 -u ${BIN_DIR}/test_hub.py \ python3 -u ${BIN_DIR}/test_hub.py \
--nproc ${ngpu} \ --ngpu ${ngpu} \
--config ${config_path} \ --config ${config_path} \
--result_file ${output_dir}/${type}.rsl \ --result_file ${output_dir}/${type}.rsl \
--checkpoint_path ${ckpt_prefix} \ --checkpoint_path ${ckpt_prefix} \
......
...@@ -23,7 +23,7 @@ fi ...@@ -23,7 +23,7 @@ fi
# export FLAGS_conv_workspace_size_limit=4000 # export FLAGS_conv_workspace_size_limit=4000
python3 -u ${BIN_DIR}/train.py \ python3 -u ${BIN_DIR}/train.py \
--nproc ${ngpu} \ --ngpu ${ngpu} \
--config ${config_path} \ --config ${config_path} \
--output exp/${ckpt_name} \ --output exp/${ckpt_name} \
--seed ${seed} --seed ${seed}
......
...@@ -22,7 +22,7 @@ python3 -u ${BIN_DIR}/test.py \ ...@@ -22,7 +22,7 @@ python3 -u ${BIN_DIR}/test.py \
--model-name 'u2_kaldi' \ --model-name 'u2_kaldi' \
--run-mode 'align' \ --run-mode 'align' \
--dict-path ${dict_path} \ --dict-path ${dict_path} \
--nproc ${ngpu} \ --ngpu ${ngpu} \
--config ${config_path} \ --config ${config_path} \
--result-file ${output_dir}/${type}.align \ --result-file ${output_dir}/${type}.align \
--checkpoint_path ${ckpt_prefix} \ --checkpoint_path ${ckpt_prefix} \
......
...@@ -15,7 +15,7 @@ jit_model_export_path=$3 ...@@ -15,7 +15,7 @@ jit_model_export_path=$3
python3 -u ${BIN_DIR}/test.py \ python3 -u ${BIN_DIR}/test.py \
--model-name 'u2_kaldi' \ --model-name 'u2_kaldi' \
--run-mode 'export' \ --run-mode 'export' \
--nproc ${ngpu} \ --ngpu ${ngpu} \
--config ${config_path} \ --config ${config_path} \
--checkpoint_path ${ckpt_path_prefix} \ --checkpoint_path ${ckpt_path_prefix} \
--export_path ${jit_model_export_path} --export_path ${jit_model_export_path}
......
...@@ -76,7 +76,7 @@ for dmethd in attention ctc_greedy_search ctc_prefix_beam_search attention_resco ...@@ -76,7 +76,7 @@ for dmethd in attention ctc_greedy_search ctc_prefix_beam_search attention_resco
python3 -u ${BIN_DIR}/test.py \ python3 -u ${BIN_DIR}/test.py \
--model-name u2_kaldi \ --model-name u2_kaldi \
--run-mode test \ --run-mode test \
--nproc ${ngpu} \ --ngpu ${ngpu} \
--dict-path ${dict} \ --dict-path ${dict} \
--config ${config_path} \ --config ${config_path} \
--checkpoint_path ${ckpt_prefix} \ --checkpoint_path ${ckpt_prefix} \
......
...@@ -21,7 +21,7 @@ fi ...@@ -21,7 +21,7 @@ fi
python3 -u ${BIN_DIR}/train.py \ python3 -u ${BIN_DIR}/train.py \
--model-name u2_kaldi \ --model-name u2_kaldi \
--nproc ${ngpu} \ --ngpu ${ngpu} \
--config ${config_path} \ --config ${config_path} \
--output exp/${ckpt_name} \ --output exp/${ckpt_name} \
--seed ${seed} --seed ${seed}
......
...@@ -19,7 +19,7 @@ if [ $? -ne 0 ]; then ...@@ -19,7 +19,7 @@ if [ $? -ne 0 ]; then
fi fi
python3 -u ${BIN_DIR}/test.py \ python3 -u ${BIN_DIR}/test.py \
--nproc ${ngpu} \ --ngpu ${ngpu} \
--config ${config_path} \ --config ${config_path} \
--result_file ${ckpt_prefix}.rsl \ --result_file ${ckpt_prefix}.rsl \
--checkpoint_path ${ckpt_prefix} \ --checkpoint_path ${ckpt_prefix} \
......
...@@ -19,7 +19,7 @@ if [ $? -ne 0 ]; then ...@@ -19,7 +19,7 @@ if [ $? -ne 0 ]; then
fi fi
python3 -u ${BIN_DIR}/test.py \ python3 -u ${BIN_DIR}/test.py \
--nproc ${ngpu} \ --ngpu ${ngpu} \
--config ${config_path} \ --config ${config_path} \
--result_file ${ckpt_prefix}.rsl \ --result_file ${ckpt_prefix}.rsl \
--checkpoint_path ${ckpt_prefix} \ --checkpoint_path ${ckpt_prefix} \
......
...@@ -19,7 +19,7 @@ if [ $? -ne 0 ]; then ...@@ -19,7 +19,7 @@ if [ $? -ne 0 ]; then
fi fi
python3 -u ${BIN_DIR}/test.py \ python3 -u ${BIN_DIR}/test.py \
--nproc ${ngpu} \ --ngpu ${ngpu} \
--config ${config_path} \ --config ${config_path} \
--result_file ${ckpt_prefix}.rsl \ --result_file ${ckpt_prefix}.rsl \
--checkpoint_path ${ckpt_prefix} \ --checkpoint_path ${ckpt_prefix} \
......
...@@ -403,7 +403,7 @@ class DeepSpeech2Tester(DeepSpeech2Trainer): ...@@ -403,7 +403,7 @@ class DeepSpeech2Tester(DeepSpeech2Trainer):
def setup(self): def setup(self):
"""Setup the experiment. """Setup the experiment.
""" """
paddle.set_device('gpu' if self.args.nprocs > 0 else 'cpu') paddle.set_device('gpu' if self.args.ngpu > 0 else 'cpu')
self.setup_output_dir() self.setup_output_dir()
self.setup_checkpointer() self.setup_checkpointer()
......
...@@ -15,7 +15,7 @@ for type in fullsentence; do ...@@ -15,7 +15,7 @@ for type in fullsentence; do
echo "decoding ${type}" echo "decoding ${type}"
batch_size=32 batch_size=32
python3 -u ${BIN_DIR}/test.py \ python3 -u ${BIN_DIR}/test.py \
--nproc ${ngpu} \ --ngpu ${ngpu} \
--config ${config_path} \ --config ${config_path} \
--result_file ${ckpt_prefix}.${type}.rsl \ --result_file ${ckpt_prefix}.${type}.rsl \
--checkpoint_path ${ckpt_prefix} \ --checkpoint_path ${ckpt_prefix} \
......
...@@ -20,7 +20,7 @@ if [ ${seed} != 0 ]; then ...@@ -20,7 +20,7 @@ if [ ${seed} != 0 ]; then
fi fi
python3 -u ${BIN_DIR}/train.py \ python3 -u ${BIN_DIR}/train.py \
--nproc ${ngpu} \ --ngpu ${ngpu} \
--config ${config_path} \ --config ${config_path} \
--output exp/${ckpt_name} \ --output exp/${ckpt_name} \
--seed ${seed} --seed ${seed}
......
...@@ -15,7 +15,7 @@ for type in fullsentence; do ...@@ -15,7 +15,7 @@ for type in fullsentence; do
echo "decoding ${type}" echo "decoding ${type}"
batch_size=32 batch_size=32
python3 -u ${BIN_DIR}/test.py \ python3 -u ${BIN_DIR}/test.py \
--nproc ${ngpu} \ --ngpu ${ngpu} \
--config ${config_path} \ --config ${config_path} \
--result_file ${ckpt_prefix}.${type}.rsl \ --result_file ${ckpt_prefix}.${type}.rsl \
--checkpoint_path ${ckpt_prefix} \ --checkpoint_path ${ckpt_prefix} \
......
...@@ -21,7 +21,7 @@ if [ ${seed} != 0 ]; then ...@@ -21,7 +21,7 @@ if [ ${seed} != 0 ]; then
fi fi
python3 -u ${BIN_DIR}/train.py \ python3 -u ${BIN_DIR}/train.py \
--nproc ${ngpu} \ --ngpu ${ngpu} \
--config ${config_path} \ --config ${config_path} \
--output exp/${ckpt_name} \ --output exp/${ckpt_name} \
--checkpoint_path ${ckpt_path} \ --checkpoint_path ${ckpt_path} \
......
...@@ -18,7 +18,7 @@ mkdir -p ${output_dir} ...@@ -18,7 +18,7 @@ mkdir -p ${output_dir}
# align dump in `result_file` # align dump in `result_file`
# .tier, .TextGrid dump in `dir of result_file` # .tier, .TextGrid dump in `dir of result_file`
python3 -u ${BIN_DIR}/alignment.py \ python3 -u ${BIN_DIR}/alignment.py \
--nproc ${ngpu} \ --ngpu ${ngpu} \
--config ${config_path} \ --config ${config_path} \
--result_file ${output_dir}/${type}.align \ --result_file ${output_dir}/${type}.align \
--checkpoint_path ${ckpt_prefix} \ --checkpoint_path ${ckpt_prefix} \
......
...@@ -13,7 +13,7 @@ ckpt_path_prefix=$2 ...@@ -13,7 +13,7 @@ ckpt_path_prefix=$2
jit_model_export_path=$3 jit_model_export_path=$3
python3 -u ${BIN_DIR}/export.py \ python3 -u ${BIN_DIR}/export.py \
--nproc ${ngpu} \ --ngpu ${ngpu} \
--config ${config_path} \ --config ${config_path} \
--checkpoint_path ${ckpt_path_prefix} \ --checkpoint_path ${ckpt_path_prefix} \
--export_path ${jit_model_export_path} --export_path ${jit_model_export_path}
......
...@@ -41,7 +41,7 @@ if [ ${stage} -le 0 ] && [ ${stop_stage} -ge 0 ]; then ...@@ -41,7 +41,7 @@ if [ ${stage} -le 0 ] && [ ${stop_stage} -ge 0 ]; then
batch_size=64 batch_size=64
fi fi
python3 -u ${BIN_DIR}/test.py \ python3 -u ${BIN_DIR}/test.py \
--nproc ${ngpu} \ --ngpu ${ngpu} \
--config ${config_path} \ --config ${config_path} \
--result_file ${ckpt_prefix}.${type}.rsl \ --result_file ${ckpt_prefix}.${type}.rsl \
--checkpoint_path ${ckpt_prefix} \ --checkpoint_path ${ckpt_prefix} \
...@@ -61,7 +61,7 @@ if [ ${stage} -le 1 ] && [ ${stop_stage} -ge 1 ]; then ...@@ -61,7 +61,7 @@ if [ ${stage} -le 1 ] && [ ${stop_stage} -ge 1 ]; then
echo "decoding ${type}" echo "decoding ${type}"
batch_size=1 batch_size=1
python3 -u ${BIN_DIR}/test.py \ python3 -u ${BIN_DIR}/test.py \
--nproc ${ngpu} \ --ngpu ${ngpu} \
--config ${config_path} \ --config ${config_path} \
--result_file ${ckpt_prefix}.${type}.rsl \ --result_file ${ckpt_prefix}.${type}.rsl \
--checkpoint_path ${ckpt_prefix} \ --checkpoint_path ${ckpt_prefix} \
...@@ -80,7 +80,7 @@ if [ ${stage} -le 2 ] && [ ${stop_stage} -ge 2 ]; then ...@@ -80,7 +80,7 @@ if [ ${stage} -le 2 ] && [ ${stop_stage} -ge 2 ]; then
echo "decoding ${type}" echo "decoding ${type}"
batch_size=1 batch_size=1
python3 -u ${BIN_DIR}/test.py \ python3 -u ${BIN_DIR}/test.py \
--nproc ${ngpu} \ --ngpu ${ngpu} \
--config ${config_path} \ --config ${config_path} \
--result_file ${ckpt_prefix}.${type}.rsl \ --result_file ${ckpt_prefix}.${type}.rsl \
--checkpoint_path ${ckpt_prefix} \ --checkpoint_path ${ckpt_prefix} \
......
...@@ -20,7 +20,7 @@ if [ ${seed} != 0 ]; then ...@@ -20,7 +20,7 @@ if [ ${seed} != 0 ]; then
fi fi
python3 -u ${BIN_DIR}/train.py \ python3 -u ${BIN_DIR}/train.py \
--nproc ${ngpu} \ --ngpu ${ngpu} \
--config ${config_path} \ --config ${config_path} \
--output exp/${ckpt_name} \ --output exp/${ckpt_name} \
--seed ${seed} --seed ${seed}
......
...@@ -14,7 +14,7 @@ jit_model_export_path=$3 ...@@ -14,7 +14,7 @@ jit_model_export_path=$3
model_type=$4 model_type=$4
python3 -u ${BIN_DIR}/export.py \ python3 -u ${BIN_DIR}/export.py \
--nproc ${ngpu} \ --ngpu ${ngpu} \
--config ${config_path} \ --config ${config_path} \
--checkpoint_path ${ckpt_path_prefix} \ --checkpoint_path ${ckpt_path_prefix} \
--export_path ${jit_model_export_path} \ --export_path ${jit_model_export_path} \
......
...@@ -19,7 +19,7 @@ if [ $? -ne 0 ]; then ...@@ -19,7 +19,7 @@ if [ $? -ne 0 ]; then
fi fi
python3 -u ${BIN_DIR}/test.py \ python3 -u ${BIN_DIR}/test.py \
--nproc ${ngpu} \ --ngpu ${ngpu} \
--config ${config_path} \ --config ${config_path} \
--result_file ${ckpt_prefix}.rsl \ --result_file ${ckpt_prefix}.rsl \
--checkpoint_path ${ckpt_prefix} \ --checkpoint_path ${ckpt_prefix} \
......
...@@ -27,7 +27,7 @@ model_type=$3 ...@@ -27,7 +27,7 @@ model_type=$3
mkdir -p exp mkdir -p exp
python3 -u ${BIN_DIR}/train.py \ python3 -u ${BIN_DIR}/train.py \
--nproc ${ngpu} \ --ngpu ${ngpu} \
--config ${config_path} \ --config ${config_path} \
--output exp/${ckpt_name} \ --output exp/${ckpt_name} \
--model_type ${model_type} \ --model_type ${model_type} \
......
...@@ -18,7 +18,7 @@ mkdir -p ${output_dir} ...@@ -18,7 +18,7 @@ mkdir -p ${output_dir}
# align dump in `result_file` # align dump in `result_file`
# .tier, .TextGrid dump in `dir of result_file` # .tier, .TextGrid dump in `dir of result_file`
python3 -u ${BIN_DIR}/alignment.py \ python3 -u ${BIN_DIR}/alignment.py \
--nproc ${ngpu} \ --ngpu ${ngpu} \
--config ${config_path} \ --config ${config_path} \
--result_file ${output_dir}/${type}.align \ --result_file ${output_dir}/${type}.align \
--checkpoint_path ${ckpt_prefix} \ --checkpoint_path ${ckpt_prefix} \
......
...@@ -13,7 +13,7 @@ ckpt_path_prefix=$2 ...@@ -13,7 +13,7 @@ ckpt_path_prefix=$2
jit_model_export_path=$3 jit_model_export_path=$3
python3 -u ${BIN_DIR}/export.py \ python3 -u ${BIN_DIR}/export.py \
--nproc ${ngpu} \ --ngpu ${ngpu} \
--config ${config_path} \ --config ${config_path} \
--checkpoint_path ${ckpt_path_prefix} \ --checkpoint_path ${ckpt_path_prefix} \
--export_path ${jit_model_export_path} --export_path ${jit_model_export_path}
......
...@@ -31,7 +31,7 @@ for type in attention ctc_greedy_search; do ...@@ -31,7 +31,7 @@ for type in attention ctc_greedy_search; do
batch_size=64 batch_size=64
fi fi
python3 -u ${BIN_DIR}/test.py \ python3 -u ${BIN_DIR}/test.py \
--nproc ${ngpu} \ --ngpu ${ngpu} \
--config ${config_path} \ --config ${config_path} \
--result_file ${ckpt_prefix}.${type}.rsl \ --result_file ${ckpt_prefix}.${type}.rsl \
--checkpoint_path ${ckpt_prefix} \ --checkpoint_path ${ckpt_prefix} \
...@@ -48,7 +48,7 @@ for type in ctc_prefix_beam_search attention_rescoring; do ...@@ -48,7 +48,7 @@ for type in ctc_prefix_beam_search attention_rescoring; do
echo "decoding ${type}" echo "decoding ${type}"
batch_size=1 batch_size=1
python3 -u ${BIN_DIR}/test.py \ python3 -u ${BIN_DIR}/test.py \
--nproc ${ngpu} \ --ngpu ${ngpu} \
--config ${config_path} \ --config ${config_path} \
--result_file ${ckpt_prefix}.${type}.rsl \ --result_file ${ckpt_prefix}.${type}.rsl \
--checkpoint_path ${ckpt_prefix} \ --checkpoint_path ${ckpt_prefix} \
......
...@@ -29,7 +29,7 @@ mkdir -p exp ...@@ -29,7 +29,7 @@ mkdir -p exp
python3 -u ${BIN_DIR}/train.py \ python3 -u ${BIN_DIR}/train.py \
--seed ${seed} \ --seed ${seed} \
--nproc ${ngpu} \ --ngpu ${ngpu} \
--config ${config_path} \ --config ${config_path} \
--output exp/${ckpt_name} \ --output exp/${ckpt_name} \
--profiler-options "${profiler_options}" \ --profiler-options "${profiler_options}" \
......
...@@ -34,7 +34,7 @@ for type in attention ctc_greedy_search; do ...@@ -34,7 +34,7 @@ for type in attention ctc_greedy_search; do
output_dir=${ckpt_prefix} output_dir=${ckpt_prefix}
mkdir -p ${output_dir} mkdir -p ${output_dir}
python3 -u ${BIN_DIR}/test.py \ python3 -u ${BIN_DIR}/test.py \
--nproc ${ngpu} \ --ngpu ${ngpu} \
--config ${config_path} \ --config ${config_path} \
--result_file ${output_dir}/${type}.rsl \ --result_file ${output_dir}/${type}.rsl \
--checkpoint_path ${ckpt_prefix} \ --checkpoint_path ${ckpt_prefix} \
...@@ -53,7 +53,7 @@ for type in ctc_prefix_beam_search attention_rescoring; do ...@@ -53,7 +53,7 @@ for type in ctc_prefix_beam_search attention_rescoring; do
output_dir=${ckpt_prefix} output_dir=${ckpt_prefix}
mkdir -p ${output_dir} mkdir -p ${output_dir}
python3 -u ${BIN_DIR}/test.py \ python3 -u ${BIN_DIR}/test.py \
--nproc ${ngpu} \ --ngpu ${ngpu} \
--config ${config_path} \ --config ${config_path} \
--result_file ${output_dir}/${type}.rsl \ --result_file ${output_dir}/${type}.rsl \
--checkpoint_path ${ckpt_prefix} \ --checkpoint_path ${ckpt_prefix} \
......
...@@ -40,7 +40,6 @@ def get_config(config_path): ...@@ -40,7 +40,6 @@ def get_config(config_path):
def load_trained_model(args): def load_trained_model(args):
args.nprocs = args.ngpu
confs = get_config(args.model_conf) confs = get_config(args.model_conf)
class_obj = dynamic_import_tester(args.model_name) class_obj = dynamic_import_tester(args.model_name)
exp = class_obj(confs, args) exp = class_obj(confs, args)
......
...@@ -87,7 +87,7 @@ class DeepSpeech2Tester_hub(): ...@@ -87,7 +87,7 @@ class DeepSpeech2Tester_hub():
def setup(self): def setup(self):
"""Setup the experiment. """Setup the experiment.
""" """
paddle.set_device('gpu' if self.args.nprocs > 0 else 'cpu') paddle.set_device('gpu' if self.args.ngpu > 0 else 'cpu')
self.setup_output_dir() self.setup_output_dir()
self.setup_checkpointer() self.setup_checkpointer()
......
...@@ -27,8 +27,8 @@ def main_sp(config, args): ...@@ -27,8 +27,8 @@ def main_sp(config, args):
def main(config, args): def main(config, args):
if args.nprocs > 0: if args.ngpu > 1:
dist.spawn(main_sp, args=(config, args), nprocs=args.nprocs) dist.spawn(main_sp, args=(config, args), nprocs=args.ngpu)
else: else:
main_sp(config, args) main_sp(config, args)
......
...@@ -47,7 +47,7 @@ class U2Infer(): ...@@ -47,7 +47,7 @@ class U2Infer():
vocab_filepath=config.collator.vocab_filepath, vocab_filepath=config.collator.vocab_filepath,
spm_model_prefix=config.collator.spm_model_prefix) spm_model_prefix=config.collator.spm_model_prefix)
paddle.set_device('gpu' if self.args.nprocs > 0 else 'cpu') paddle.set_device('gpu' if self.args.ngpu > 0 else 'cpu')
# model # model
model_conf = config.model model_conf = config.model
......
...@@ -32,8 +32,8 @@ def main_sp(config, args): ...@@ -32,8 +32,8 @@ def main_sp(config, args):
def main(config, args): def main(config, args):
if args.nprocs > 0: if args.ngpu > 1:
dist.spawn(main_sp, args=(config, args), nprocs=args.nprocs) dist.spawn(main_sp, args=(config, args), nprocs=args.ngpu)
else: else:
main_sp(config, args) main_sp(config, args)
......
...@@ -257,7 +257,7 @@ class U2Trainer(Trainer): ...@@ -257,7 +257,7 @@ class U2Trainer(Trainer):
maxlen_in=float('inf'), maxlen_in=float('inf'),
maxlen_out=float('inf'), maxlen_out=float('inf'),
minibatches=0, minibatches=0,
mini_batch_size=self.args.nprocs, mini_batch_size=self.args.ngpu,
batch_count='auto', batch_count='auto',
batch_bins=0, batch_bins=0,
batch_frames_in=0, batch_frames_in=0,
...@@ -277,7 +277,7 @@ class U2Trainer(Trainer): ...@@ -277,7 +277,7 @@ class U2Trainer(Trainer):
maxlen_in=float('inf'), maxlen_in=float('inf'),
maxlen_out=float('inf'), maxlen_out=float('inf'),
minibatches=0, minibatches=0,
mini_batch_size=self.args.nprocs, mini_batch_size=self.args.ngpu,
batch_count='auto', batch_count='auto',
batch_bins=0, batch_bins=0,
batch_frames_in=0, batch_frames_in=0,
......
...@@ -36,8 +36,8 @@ def main_sp(config, args): ...@@ -36,8 +36,8 @@ def main_sp(config, args):
def main(config, args): def main(config, args):
if args.nprocs > 0: if args.ngpu > 1:
dist.spawn(main_sp, args=(config, args), nprocs=args.nprocs) dist.spawn(main_sp, args=(config, args), nprocs=args.ngpu)
else: else:
main_sp(config, args) main_sp(config, args)
......
...@@ -239,7 +239,7 @@ class U2Trainer(Trainer): ...@@ -239,7 +239,7 @@ class U2Trainer(Trainer):
maxlen_in=float('inf'), maxlen_in=float('inf'),
maxlen_out=float('inf'), maxlen_out=float('inf'),
minibatches=0, minibatches=0,
mini_batch_size=self.args.nprocs, mini_batch_size=self.args.ngpu,
batch_count='auto', batch_count='auto',
batch_bins=0, batch_bins=0,
batch_frames_in=0, batch_frames_in=0,
...@@ -258,7 +258,7 @@ class U2Trainer(Trainer): ...@@ -258,7 +258,7 @@ class U2Trainer(Trainer):
maxlen_in=float('inf'), maxlen_in=float('inf'),
maxlen_out=float('inf'), maxlen_out=float('inf'),
minibatches=0, minibatches=0,
mini_batch_size=self.args.nprocs, mini_batch_size=self.args.ngpu,
batch_count='auto', batch_count='auto',
batch_bins=0, batch_bins=0,
batch_frames_in=0, batch_frames_in=0,
......
...@@ -30,8 +30,8 @@ def main_sp(config, args): ...@@ -30,8 +30,8 @@ def main_sp(config, args):
def main(config, args): def main(config, args):
if args.nprocs > 0: if args.ngpu > 1:
dist.spawn(main_sp, args=(config, args), nprocs=args.nprocs) dist.spawn(main_sp, args=(config, args), nprocs=args.ngpu)
else: else:
main_sp(config, args) main_sp(config, args)
......
...@@ -51,7 +51,7 @@ def default_argument_parser(parser=None): ...@@ -51,7 +51,7 @@ def default_argument_parser(parser=None):
The ``--checkpoint_path`` specifies the checkpoint to load from. The ``--checkpoint_path`` specifies the checkpoint to load from.
The ``--nprocs`` specifies how to run the training. The ``--ngpu`` specifies how to run the training.
See Also See Also
...@@ -78,7 +78,7 @@ def default_argument_parser(parser=None): ...@@ -78,7 +78,7 @@ def default_argument_parser(parser=None):
help="seed to use for paddle, np and random. None or 0 for random, else set seed." help="seed to use for paddle, np and random. None or 0 for random, else set seed."
) )
train_group.add_argument( train_group.add_argument(
"--nprocs", "--ngpu",
type=int, type=int,
default=1, default=1,
help="number of parallel processes. 0 for cpu.") help="number of parallel processes. 0 for cpu.")
......
...@@ -88,8 +88,8 @@ class Trainer(): ...@@ -88,8 +88,8 @@ class Trainer():
>>> config.merge_from_list(args.opts) >>> config.merge_from_list(args.opts)
>>> config.freeze() >>> config.freeze()
>>> >>>
>>> if args.nprocs > 0: >>> if args.ngpu > 1:
>>> dist.spawn(main_sp, args=(config, args), nprocs=args.nprocs) >>> dist.spawn(main_sp, args=(config, args), nprocs=args.ngpu)
>>> else: >>> else:
>>> main_sp(config, args) >>> main_sp(config, args)
""" """
...@@ -112,7 +112,7 @@ class Trainer(): ...@@ -112,7 +112,7 @@ class Trainer():
logger.info(f"Rank: {self.rank}/{self.world_size}") logger.info(f"Rank: {self.rank}/{self.world_size}")
# set device # set device
paddle.set_device('gpu' if self.args.nprocs > 0 else 'cpu') paddle.set_device('gpu' if self.args.ngpu > 0 else 'cpu')
if self.parallel: if self.parallel:
self.init_parallel() self.init_parallel()
...@@ -162,7 +162,7 @@ class Trainer(): ...@@ -162,7 +162,7 @@ class Trainer():
"""A flag indicating whether the experiment should run with """A flag indicating whether the experiment should run with
multiprocessing. multiprocessing.
""" """
return self.args.nprocs > 1 return self.args.ngpu > 1
def init_parallel(self): def init_parallel(self):
"""Init environment for multiprocess training. """Init environment for multiprocess training.
......
...@@ -241,7 +241,7 @@ def main_sp(config, args): ...@@ -241,7 +241,7 @@ def main_sp(config, args):
def main(config, args): def main(config, args):
if args.ngpu: if args.ngpu > 1:
dist.spawn(main_sp, args=(config, args), nprocs=args.ngpu) dist.spawn(main_sp, args=(config, args), nprocs=args.ngpu)
else: else:
main_sp(config, args) main_sp(config, args)
......
...@@ -30,7 +30,7 @@ def default_argument_parser(): ...@@ -30,7 +30,7 @@ def default_argument_parser():
The ``--checkpoint_path`` specifies the checkpoint to load from. The ``--checkpoint_path`` specifies the checkpoint to load from.
The ``--device`` and ``--nprocs`` specifies how to run the training. The ``--ngpu`` specifies how to run the training.
See Also See Also
-------- --------
......
...@@ -82,8 +82,8 @@ class ExperimentBase(object): ...@@ -82,8 +82,8 @@ class ExperimentBase(object):
>>> config.merge_from_list(args.opts) >>> config.merge_from_list(args.opts)
>>> config.freeze() >>> config.freeze()
>>> >>>
>>> if args.nprocs > 1 and args.device == "gpu": >>> if args.ngpu > 1:
>>> dist.spawn(main_sp, args=(config, args), nprocs=args.nprocs) >>> dist.spawn(main_sp, args=(config, args), nprocs=args.ngpu)
>>> else: >>> else:
>>> main_sp(config, args) >>> main_sp(config, args)
""" """
......
...@@ -9,17 +9,11 @@ fi ...@@ -9,17 +9,11 @@ fi
ngpu=$(echo $CUDA_VISIBLE_DEVICES | awk -F "," '{print NF}') ngpu=$(echo $CUDA_VISIBLE_DEVICES | awk -F "," '{print NF}')
echo "using $ngpu gpus..." echo "using $ngpu gpus..."
device=gpu
if [ ${ngpu} == 0 ];then
device=cpu
fi
config_path=$1 config_path=$1
ckpt_prefix=$2 ckpt_prefix=$2
python3 -u ${BIN_DIR}/test.py \ python3 -u ${BIN_DIR}/test.py \
--device ${device} \ --ngpu 1 \
--nproc 1 \
--config ${config_path} \ --config ${config_path} \
--result_file ${ckpt_prefix}.rsl \ --result_file ${ckpt_prefix}.rsl \
--checkpoint_path ${ckpt_prefix} --checkpoint_path ${ckpt_prefix}
......
...@@ -11,16 +11,10 @@ echo "using $ngpu gpus..." ...@@ -11,16 +11,10 @@ echo "using $ngpu gpus..."
config_path=$1 config_path=$1
ckpt_name=$2 ckpt_name=$2
device=gpu
if [ ${ngpu} == 0 ];then
device=cpu
fi
mkdir -p exp mkdir -p exp
python3 -u ${BIN_DIR}/train.py \ python3 -u ${BIN_DIR}/train.py \
--device ${device} \ --ngpu ${ngpu} \
--nproc ${ngpu} \
--config ${config_path} \ --config ${config_path} \
--output exp/${ckpt_name} --output exp/${ckpt_name}
......
...@@ -9,17 +9,12 @@ fi ...@@ -9,17 +9,12 @@ fi
ngpu=$(echo $CUDA_VISIBLE_DEVICES | awk -F "," '{print NF}') ngpu=$(echo $CUDA_VISIBLE_DEVICES | awk -F "," '{print NF}')
echo "using $ngpu gpus..." echo "using $ngpu gpus..."
device=gpu
if [ ${ngpu} == 0 ];then
device=cpu
fi
config_path=$1 config_path=$1
ckpt_prefix=$2 ckpt_prefix=$2
python3 -u ${BIN_DIR}/test.py \ python3 -u ${BIN_DIR}/test.py \
--device ${device} \ --ngpu 1 \
--nproc 1 \
--config ${config_path} \ --config ${config_path} \
--result_file ${ckpt_prefix}.rsl \ --result_file ${ckpt_prefix}.rsl \
--checkpoint_path ${ckpt_prefix} --checkpoint_path ${ckpt_prefix}
......
...@@ -11,16 +11,10 @@ echo "using $ngpu gpus..." ...@@ -11,16 +11,10 @@ echo "using $ngpu gpus..."
config_path=$1 config_path=$1
ckpt_name=$2 ckpt_name=$2
device=gpu
if [ ${ngpu} == 0 ];then
device=cpu
fi
mkdir -p exp mkdir -p exp
python3 -u ${BIN_DIR}/train.py \ python3 -u ${BIN_DIR}/train.py \
--device ${device} \ --ngpu ${ngpu} \
--nproc ${ngpu} \
--config ${config_path} \ --config ${config_path} \
--output exp/${ckpt_name} --output exp/${ckpt_name}
......
...@@ -26,8 +26,8 @@ def main_sp(config, args): ...@@ -26,8 +26,8 @@ def main_sp(config, args):
def main(config, args): def main(config, args):
if args.device == "gpu" and args.nprocs > 1: if args.ngpu > 1:
dist.spawn(main_sp, args=(config, args), nprocs=args.nprocs) dist.spawn(main_sp, args=(config, args), nprocs=args.ngpu)
else: else:
main_sp(config, args) main_sp(config, args)
......
...@@ -106,8 +106,8 @@ class Trainer(): ...@@ -106,8 +106,8 @@ class Trainer():
>>> config.merge_from_list(args.opts) >>> config.merge_from_list(args.opts)
>>> config.freeze() >>> config.freeze()
>>> >>>
>>> if args.nprocs > 1 and args.device == "gpu": >>> if args.ngpu > 1:
>>> dist.spawn(main_sp, args=(config, args), nprocs=args.nprocs) >>> dist.spawn(main_sp, args=(config, args), nprocs=args.ngpu)
>>> else: >>> else:
>>> main_sp(config, args) >>> main_sp(config, args)
""" """
...@@ -147,7 +147,7 @@ class Trainer(): ...@@ -147,7 +147,7 @@ class Trainer():
"""A flag indicating whether the experiment should run with """A flag indicating whether the experiment should run with
multiprocessing. multiprocessing.
""" """
return self.args.device == "gpu" and self.args.nprocs > 1 return self.args.ngpu > 1
def init_parallel(self): def init_parallel(self):
"""Init environment for multiprocess training. """Init environment for multiprocess training.
......
...@@ -30,7 +30,7 @@ def default_argument_parser(): ...@@ -30,7 +30,7 @@ def default_argument_parser():
The ``--checkpoint_path`` specifies the checkpoint to load from. The ``--checkpoint_path`` specifies the checkpoint to load from.
The ``--device`` and ``--nprocs`` specifies how to run the training. The ``--ngpu`` specifies how to run the training.
See Also See Also
...@@ -60,9 +60,7 @@ def default_argument_parser(): ...@@ -60,9 +60,7 @@ def default_argument_parser():
parser.add_argument("--result_file", type=str, help="path of save the asr result") parser.add_argument("--result_file", type=str, help="path of save the asr result")
# running # running
parser.add_argument("--device", type=str, default='gpu', choices=["cpu", "gpu"], parser.add_argument("--ngpu", type=int, default=1, help="number of parallel processes to use. if ngpu=0, using cpu.")
help="device type to use, cpu and gpu are supported.")
parser.add_argument("--nprocs", type=int, default=1, help="number of parallel processes to use.")
# overwrite extra config and default config # overwrite extra config and default config
# parser.add_argument("--opts", nargs=argparse.REMAINDER, # parser.add_argument("--opts", nargs=argparse.REMAINDER,
......
...@@ -38,7 +38,7 @@ function _train(){ ...@@ -38,7 +38,7 @@ function _train(){
train_cmd="--config=${config_path} train_cmd="--config=${config_path}
--output=${output} --output=${output}
--seed=${seed} --seed=${seed}
--nproc=${ngpu} --ngpu=${ngpu}
--profiler-options "${profiler_options}" --profiler-options "${profiler_options}"
--benchmark-batch-size ${batch_size} --benchmark-batch-size ${batch_size}
--benchmark-max-step ${benchmark_max_step} " --benchmark-max-step ${benchmark_max_step} "
......
...@@ -21,13 +21,13 @@ null:null ...@@ -21,13 +21,13 @@ null:null
null:null null:null
## ##
===========================eval_params=========================== ===========================eval_params===========================
eval: ../../../paddlespeech/s2t/exps/deepspeech2/bin/test.py --nproc 1 --config conf/deepspeech2.yaml --checkpoint_path exp/deepspeech_tiny/checkpoints/9 --result_file tests/9.rsl --model_type offline eval: ../../../paddlespeech/s2t/exps/deepspeech2/bin/test.py --ngpu 1 --config conf/deepspeech2.yaml --checkpoint_path exp/deepspeech_tiny/checkpoints/9 --result_file tests/9.rsl --model_type offline
null:null null:null
## ##
===========================infer_params=========================== ===========================infer_params===========================
null:null null:null
null:null null:null
norm_export: ../../../paddlespeech/s2t/exps/deepspeech2/bin/export.py --nproc 1 --config conf/deepspeech2.yaml --model_type offline --checkpoint_path exp/deepspeech_tiny/checkpoints/9 --export_path exp/deepspeech_tiny/checkpoints/9.jit norm_export: ../../../paddlespeech/s2t/exps/deepspeech2/bin/export.py --ngpu 1 --config conf/deepspeech2.yaml --model_type offline --checkpoint_path exp/deepspeech_tiny/checkpoints/9 --export_path exp/deepspeech_tiny/checkpoints/9.jit
quant_export:null quant_export:null
fpgm_export:null fpgm_export:null
distill_export:null distill_export:null
......
...@@ -21,13 +21,13 @@ null:null ...@@ -21,13 +21,13 @@ null:null
null:null null:null
## ##
===========================eval_params=========================== ===========================eval_params===========================
eval: ../../../paddlespeech/s2t/exps/deepspeech2/bin/test.py --nproc 1 --config conf/deepspeech2.yaml --result_file tests/49.rsl --checkpoint_path exp/deepspeech_whole/checkpoints/49 --model_type offline eval: ../../../paddlespeech/s2t/exps/deepspeech2/bin/test.py --ngpu 1 --config conf/deepspeech2.yaml --result_file tests/49.rsl --checkpoint_path exp/deepspeech_whole/checkpoints/49 --model_type offline
null:null null:null
## ##
===========================infer_params=========================== ===========================infer_params===========================
null:null null:null
null:null null:null
norm_export: ../../../paddlespeech/s2t/exps/deepspeech2/bin/export.py --nproc 1 --config conf/deepspeech2.yaml --model_type offline --checkpoint_path exp/deepspeech_whole/checkpoints/49 --export_path exp/deepspeech_whole/checkpoints/49.jit norm_export: ../../../paddlespeech/s2t/exps/deepspeech2/bin/export.py --ngpu 1 --config conf/deepspeech2.yaml --model_type offline --checkpoint_path exp/deepspeech_whole/checkpoints/49 --export_path exp/deepspeech_whole/checkpoints/49.jit
quant_export:null quant_export:null
fpgm_export:null fpgm_export:null
distill_export:null distill_export:null
......
...@@ -21,7 +21,7 @@ null:null ...@@ -21,7 +21,7 @@ null:null
null:null null:null
## ##
===========================eval_params=========================== ===========================eval_params===========================
eval:../examples/speedyspeech/baker/synthesize_e2e.py --speedyspeech-config=../examples/speedyspeech/baker/conf/default.yaml --speedyspeech-checkpoint=exp/default/checkpoints/snapshot_iter_90.pdz --speedyspeech-stat=pretrain_models/speedyspeech_baker_ckpt_0.4/speedy_speech_stats.npy --pwg-config=../examples/parallelwave_gan/baker/conf/default.yaml --pwg-checkpoint=pretrain_models/pwg_baker_ckpt_0.4/pwg_snapshot_iter_400000.pdz --pwg-stat=pretrain_models/pwg_baker_ckpt_0.4/pwg_stats.npy --text=../examples/speedyspeech/baker/sentences.txt --output-dir=e2e --inference-dir=inference --device="gpu" --phones-dict=../examples/speedyspeech/baker/phones.txt --tones-dict=../examples/speedyspeech/baker/tones.txt eval:../examples/speedyspeech/baker/synthesize_e2e.py --speedyspeech-config=../examples/speedyspeech/baker/conf/default.yaml --speedyspeech-checkpoint=exp/default/checkpoints/snapshot_iter_90.pdz --speedyspeech-stat=pretrain_models/speedyspeech_baker_ckpt_0.4/speedy_speech_stats.npy --pwg-config=../examples/parallelwave_gan/baker/conf/default.yaml --pwg-checkpoint=pretrain_models/pwg_baker_ckpt_0.4/pwg_snapshot_iter_400000.pdz --pwg-stat=pretrain_models/pwg_baker_ckpt_0.4/pwg_stats.npy --text=../examples/speedyspeech/baker/sentences.txt --output-dir=e2e --inference-dir=inference --ngpu=1 --phones-dict=../examples/speedyspeech/baker/phones.txt --tones-dict=../examples/speedyspeech/baker/tones.txt
null:null null:null
## ##
===========================infer_params=========================== ===========================infer_params===========================
......
...@@ -323,7 +323,7 @@ else ...@@ -323,7 +323,7 @@ else
elif [ ${#gpu} -le 15 ];then # train with multi-gpu elif [ ${#gpu} -le 15 ];then # train with multi-gpu
gsu=${gpu//,/ } gsu=${gpu//,/ }
nump=`echo $gsu | wc -w` nump=`echo $gsu | wc -w`
cmd="${python} ${run_train} --nproc=$nump" cmd="${python} ${run_train} --ngpu=$nump"
else # train with multi-machine else # train with multi-machine
cmd="${python} -m paddle.distributed.launch --ips=${ips} --gpus=${gpu} ${run_train} ${set_save_model} ${set_pretrain} ${set_epoch} ${set_autocast} ${set_batchsize} ${set_train_params1}" cmd="${python} -m paddle.distributed.launch --ips=${ips} --gpus=${gpu} ${run_train} ${set_save_model} ${set_pretrain} ${set_epoch} ${set_autocast} ${set_batchsize} ${set_train_params1}"
fi fi
......
...@@ -21,7 +21,7 @@ null:null ...@@ -21,7 +21,7 @@ null:null
null:null null:null
## ##
===========================eval_params=========================== ===========================eval_params===========================
eval:../../../paddlespeech/t2s/exps/speedyspeech/synthesize_e2e.py --speedyspeech-config=../../../examples/csmsc/tts2/conf/default.yaml --speedyspeech-checkpoint=exp/default/checkpoints/snapshot_iter_20.pdz --speedyspeech-stat=train_data/mini_BZNSYP/train/feats_stats.npy --pwg-config=pretrain_models/pwg_baker_ckpt_0.4/pwg_default.yaml --pwg-checkpoint=pretrain_models/pwg_baker_ckpt_0.4/pwg_snapshot_iter_400000.pdz --pwg-stat=pretrain_models/pwg_baker_ckpt_0.4/pwg_stats.npy --text=../../../paddlespeech/t2s/exps/sentences.txt --output-dir=e2e --inference-dir=inference --device="gpu" --phones-dict=train_data/mini_BZNSYP/phone_id_map.txt --tones-dict=train_data/mini_BZNSYP/tone_id_map.txt eval:../../../paddlespeech/t2s/exps/speedyspeech/synthesize_e2e.py --speedyspeech-config=../../../examples/csmsc/tts2/conf/default.yaml --speedyspeech-checkpoint=exp/default/checkpoints/snapshot_iter_20.pdz --speedyspeech-stat=train_data/mini_BZNSYP/train/feats_stats.npy --pwg-config=pretrain_models/pwg_baker_ckpt_0.4/pwg_default.yaml --pwg-checkpoint=pretrain_models/pwg_baker_ckpt_0.4/pwg_snapshot_iter_400000.pdz --pwg-stat=pretrain_models/pwg_baker_ckpt_0.4/pwg_stats.npy --text=../../../paddlespeech/t2s/exps/sentences.txt --output-dir=e2e --inference-dir=inference --ngpu=1 --phones-dict=train_data/mini_BZNSYP/phone_id_map.txt --tones-dict=train_data/mini_BZNSYP/tone_id_map.txt
null:null null:null
## ##
===========================infer_params=========================== ===========================infer_params===========================
......
...@@ -21,7 +21,7 @@ null:null ...@@ -21,7 +21,7 @@ null:null
null:null null:null
## ##
===========================eval_params=========================== ===========================eval_params===========================
eval:../../../paddlespeech/t2s/exps/speedyspeech/synthesize_e2e.py --speedyspeech-config=../../../examples/csmsc/tts2/conf/default.yaml --speedyspeech-checkpoint=exp/default/checkpoints/snapshot_iter_30.pdz --speedyspeech-stat=train_data/mini_BZNSYP/train/feats_stats.npy --pwg-config=pretrain_models/pwg_baker_ckpt_0.4/pwg_default.yaml --pwg-checkpoint=pretrain_models/pwg_baker_ckpt_0.4/pwg_snapshot_iter_400000.pdz --pwg-stat=pretrain_models/pwg_baker_ckpt_0.4/pwg_stats.npy --text=../../../paddlespeech/t2s/exps/sentences.txt --output-dir=e2e --inference-dir=inference --device="gpu" --phones-dict=train_data/mini_BZNSYP/phone_id_map.txt --tones-dict=train_data/mini_BZNSYP/tone_id_map.txt eval:../../../paddlespeech/t2s/exps/speedyspeech/synthesize_e2e.py --speedyspeech-config=../../../examples/csmsc/tts2/conf/default.yaml --speedyspeech-checkpoint=exp/default/checkpoints/snapshot_iter_30.pdz --speedyspeech-stat=train_data/mini_BZNSYP/train/feats_stats.npy --pwg-config=pretrain_models/pwg_baker_ckpt_0.4/pwg_default.yaml --pwg-checkpoint=pretrain_models/pwg_baker_ckpt_0.4/pwg_snapshot_iter_400000.pdz --pwg-stat=pretrain_models/pwg_baker_ckpt_0.4/pwg_stats.npy --text=../../../paddlespeech/t2s/exps/sentences.txt --output-dir=e2e --inference-dir=inference --ngpu=1 --phones-dict=train_data/mini_BZNSYP/phone_id_map.txt --tones-dict=train_data/mini_BZNSYP/tone_id_map.txt
null:null null:null
## ##
===========================infer_params=========================== ===========================infer_params===========================
......
...@@ -21,7 +21,7 @@ null:null ...@@ -21,7 +21,7 @@ null:null
null:null null:null
## ##
===========================eval_params=========================== ===========================eval_params===========================
eval:../../../paddlespeech/t2s/exps/speedyspeech/synthesize_e2e.py --speedyspeech-config=pretrain_models/speedyspeech_nosil_baker_ckpt_0.5/default.yaml --speedyspeech-checkpoint=pretrain_models/speedyspeech_nosil_baker_ckpt_0.5/snapshot_iter_11400.pdz --speedyspeech-stat=pretrain_models/speedyspeech_nosil_baker_ckpt_0.5/feats_stats.npy --pwg-config=pretrain_models/pwg_baker_ckpt_0.4/pwg_default.yaml --pwg-checkpoint=pretrain_models/pwg_baker_ckpt_0.4/pwg_snapshot_iter_400000.pdz --pwg-stat=pretrain_models/pwg_baker_ckpt_0.4/pwg_stats.npy --text=../../../paddlespeech/t2s/exps/sentences.txt --output-dir=e2e --inference-dir=inference --device="gpu" --phones-dict=pretrain_models/speedyspeech_nosil_baker_ckpt_0.5/phone_id_map.txt --tones-dict=pretrain_models/speedyspeech_nosil_baker_ckpt_0.5/tone_id_map.txt eval:../../../paddlespeech/t2s/exps/speedyspeech/synthesize_e2e.py --speedyspeech-config=pretrain_models/speedyspeech_nosil_baker_ckpt_0.5/default.yaml --speedyspeech-checkpoint=pretrain_models/speedyspeech_nosil_baker_ckpt_0.5/snapshot_iter_11400.pdz --speedyspeech-stat=pretrain_models/speedyspeech_nosil_baker_ckpt_0.5/feats_stats.npy --pwg-config=pretrain_models/pwg_baker_ckpt_0.4/pwg_default.yaml --pwg-checkpoint=pretrain_models/pwg_baker_ckpt_0.4/pwg_snapshot_iter_400000.pdz --pwg-stat=pretrain_models/pwg_baker_ckpt_0.4/pwg_stats.npy --text=../../../paddlespeech/t2s/exps/sentences.txt --output-dir=e2e --inference-dir=inference --ngpu=1 --phones-dict=pretrain_models/speedyspeech_nosil_baker_ckpt_0.5/phone_id_map.txt --tones-dict=pretrain_models/speedyspeech_nosil_baker_ckpt_0.5/tone_id_map.txt
null:null null:null
## ##
===========================infer_params=========================== ===========================infer_params===========================
......
...@@ -21,7 +21,7 @@ null:null ...@@ -21,7 +21,7 @@ null:null
null:null null:null
## ##
===========================eval_params=========================== ===========================eval_params===========================
eval:../../../paddlespeech/t2s/exps/speedyspeech/synthesize_e2e.py --speedyspeech-config=pretrain_models/speedyspeech_nosil_baker_ckpt_0.5/default.yaml --speedyspeech-checkpoint=pretrain_models/speedyspeech_nosil_baker_ckpt_0.5/snapshot_iter_11400.pdz --speedyspeech-stat=pretrain_models/speedyspeech_nosil_baker_ckpt_0.5/feats_stats.npy --pwg-config=pretrain_models/pwg_baker_ckpt_0.4/pwg_default.yaml --pwg-checkpoint=pretrain_models/pwg_baker_ckpt_0.4/pwg_snapshot_iter_400000.pdz --pwg-stat=pretrain_models/pwg_baker_ckpt_0.4/pwg_stats.npy --text=../../../paddlespeech/t2s/exps/sentences.txt --output-dir=e2e --inference-dir=inference --device="gpu" --phones-dict=pretrain_models/speedyspeech_nosil_baker_ckpt_0.5/phone_id_map.txt --tones-dict=pretrain_models/speedyspeech_nosil_baker_ckpt_0.5/tone_id_map.txt eval:../../../paddlespeech/t2s/exps/speedyspeech/synthesize_e2e.py --speedyspeech-config=pretrain_models/speedyspeech_nosil_baker_ckpt_0.5/default.yaml --speedyspeech-checkpoint=pretrain_models/speedyspeech_nosil_baker_ckpt_0.5/snapshot_iter_11400.pdz --speedyspeech-stat=pretrain_models/speedyspeech_nosil_baker_ckpt_0.5/feats_stats.npy --pwg-config=pretrain_models/pwg_baker_ckpt_0.4/pwg_default.yaml --pwg-checkpoint=pretrain_models/pwg_baker_ckpt_0.4/pwg_snapshot_iter_400000.pdz --pwg-stat=pretrain_models/pwg_baker_ckpt_0.4/pwg_stats.npy --text=../../../paddlespeech/t2s/exps/sentences.txt --output-dir=e2e --inference-dir=inference --ngpu=1 --phones-dict=pretrain_models/speedyspeech_nosil_baker_ckpt_0.5/phone_id_map.txt --tones-dict=pretrain_models/speedyspeech_nosil_baker_ckpt_0.5/tone_id_map.txt
null:null null:null
## ##
===========================infer_params=========================== ===========================infer_params===========================
......
Markdown is supported
0% .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑!
想要评论请 注册