Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
正统之独孤求败
mindspore
提交
8aae0a18
M
mindspore
项目概览
正统之独孤求败
/
mindspore
与 Fork 源项目一致
Fork自
MindSpore / mindspore
通知
1
Star
0
Fork
0
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
0
列表
看板
标记
里程碑
合并请求
0
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
M
mindspore
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
0
Issue
0
列表
看板
标记
里程碑
合并请求
0
合并请求
0
Pages
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
提交
Issue看板
体验新版 GitCode,发现更多精彩内容 >>
提交
8aae0a18
编写于
5月 29, 2020
作者:
U
unknown
浏览文件
操作
浏览文件
下载
电子邮件补丁
差异文件
add scripts
上级
a728b328
变更
3
显示空白变更内容
内联
并排
Showing
3 changed file
with
134 addition
and
0 deletion
+134
-0
model_zoo/deeplabv3/scripts/run_distribute_pretrain.sh
model_zoo/deeplabv3/scripts/run_distribute_pretrain.sh
+66
-0
model_zoo/deeplabv3/scripts/run_eval.sh
model_zoo/deeplabv3/scripts/run_eval.sh
+31
-0
model_zoo/deeplabv3/scripts/run_standalone_train.sh
model_zoo/deeplabv3/scripts/run_standalone_train.sh
+37
-0
未找到文件。
model_zoo/deeplabv3/scripts/run_distribute_pretrain.sh
0 → 100644
浏览文件 @
8aae0a18
#!/bin/bash
# Copyright 2020 Huawei Technologies Co., Ltd
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ============================================================================
echo
"=============================================================================================================="
echo
"Please run the scipt as: "
echo
"bash run_distribute_pretrain.sh DEVICE_NUM EPOCH_SIZE DATA_DIR MINDSPORE_HCCL_CONFIG_PATH"
echo
"for example: bash run_distribute_train.sh 8 40 /path/zh-wiki/ /path/hccl.json"
echo
"It is better to use absolute path."
echo
"=============================================================================================================="
EPOCH_SIZE
=
$2
DATA_DIR
=
$3
export
MINDSPORE_HCCL_CONFIG_PATH
=
$4
export
RANK_TABLE_FILE
=
$4
export
RANK_SIZE
=
$1
cores
=
`
cat
/proc/cpuinfo|grep
"processor"
|wc
-l
`
echo
"the number of logical core"
$cores
avg_core_per_rank
=
`
expr
$cores
\/
$RANK_SIZE
`
core_gap
=
`
expr
$avg_core_per_rank
\-
1
`
echo
"avg_core_per_rank"
$avg_core_per_rank
echo
"core_gap"
$core_gap
for
((
i
=
0
;
i<RANK_SIZE
;
i++
))
do
start
=
`
expr
$i
\*
$avg_core_per_rank
`
export
DEVICE_ID
=
$i
export
RANK_ID
=
$i
export
DEPLOY_MODE
=
0
export
GE_USE_STATIC_MEMORY
=
1
end
=
`
expr
$start
\+
$core_gap
`
cmdopt
=
$start
"-"
$end
rm
-rf
LOG
$i
mkdir
./LOG
$i
cp
*
.py ./LOG
$i
cd
./LOG
$i
||
exit
echo
"start training for rank
$i
, device
$DEVICE_ID
"
mkdir
-p
ms_log
CUR_DIR
=
`
pwd
`
export
GLOG_log_dir
=
${
CUR_DIR
}
/ms_log
export
GLOG_logtostderr
=
0
env
>
env.log
taskset
-c
$cmdopt
python ../train.py
\
--distribute
=
"true"
\
--epoch_size
=
$EPOCH_SIZE
\
--device_id
=
$DEVICE_ID
\
--enable_save_ckpt
=
"true"
\
--checkpoint_url
=
"/store1/deeplabv3/deeplabv3_split_url/train/checkpoint/CKP-12_732.ckpt"
\
--save_checkpoint_steps
=
10000
\
--save_checkpoint_num
=
1
\
--data_url
=
$DATA_DIR
>
log.txt 2>&1 &
cd
../
done
\ No newline at end of file
model_zoo/deeplabv3/scripts/run_eval.sh
0 → 100644
浏览文件 @
8aae0a18
# Copyright 2020 Huawei Technologies Co., Ltd
#
# Licensed under the Apache License, Version 2.0 (the License);
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# httpwww.apache.orglicensesLICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an AS IS BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ============================================================================
echo
"=============================================================================================================="
echo
"Please run the scipt as: "
echo
"bash run_eval.sh DEVICE_ID EPOCH_SIZE DATA_DIR"
echo
"for example: bash run_eval.sh 0 /path/zh-wiki/ "
echo
"=============================================================================================================="
DEVICE_ID
=
$1
DATA_DIR
=
$2
mkdir
-p
ms_log
CUR_DIR
=
`
pwd
`
export
GLOG_log_dir
=
${
CUR_DIR
}
/ms_log
export
GLOG_logtostderr
=
0
python evaluation.py
\
--device_id
=
$DEVICE_ID
\
--checkpoint_url
=
"/store1/deeplabv3/deeplabv3_split_url/train/checkpoint/CKP-12_732.ckpt"
\
--data_url
=
$DATA_DIR
>
log.txt 2>&1 &
\ No newline at end of file
model_zoo/deeplabv3/scripts/run_standalone_train.sh
0 → 100644
浏览文件 @
8aae0a18
# Copyright 2020 Huawei Technologies Co., Ltd
#
# Licensed under the Apache License, Version 2.0 (the License);
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# httpwww.apache.orglicensesLICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an AS IS BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ============================================================================
echo
"=============================================================================================================="
echo
"Please run the scipt as: "
echo
"bash run_standalone_pretrain.sh DEVICE_ID EPOCH_SIZE DATA_DIR"
echo
"for example: bash run_standalone_train.sh 0 40 /path/zh-wiki/ "
echo
"=============================================================================================================="
DEVICE_ID
=
$1
EPOCH_SIZE
=
$2
DATA_DIR
=
$3
mkdir
-p
ms_log
CUR_DIR
=
`
pwd
`
export
GLOG_log_dir
=
${
CUR_DIR
}
/ms_log
export
GLOG_logtostderr
=
0
python train.py
\
--distribute
=
"false"
\
--epoch_size
=
$EPOCH_SIZE
\
--device_id
=
$DEVICE_ID
\
--enable_save_ckpt
=
"true"
\
--checkpoint_url
=
"/store1/deeplabv3/deeplabv3_split_url/train/checkpoint/CKP-12_732.ckpt"
\
--save_checkpoint_steps
=
10000
\
--save_checkpoint_num
=
1
\
--data_url
=
$DATA_DIR
>
log.txt 2>&1 &
\ No newline at end of file
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录