Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
magicwindyyd
mindspore
提交
1afb8749
M
mindspore
项目概览
magicwindyyd
/
mindspore
与 Fork 源项目一致
Fork自
MindSpore / mindspore
通知
1
Star
1
Fork
0
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
0
列表
看板
标记
里程碑
合并请求
0
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
M
mindspore
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
0
Issue
0
列表
看板
标记
里程碑
合并请求
0
合并请求
0
Pages
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
提交
Issue看板
提交
1afb8749
编写于
4年前
作者:
M
mindspore-ci-bot
提交者:
Gitee
4年前
浏览文件
操作
浏览文件
下载
差异文件
!4933 Improve the Outputs Quality and Change Saving Frequency
Merge pull request !4933 from huangxinjing/wide-deep-checkpoint
上级
caccffed
4c1ae4de
master
无相关合并请求
变更
5
隐藏空白更改
内联
并排
Showing
5 changed file
with
12 addition
and
9 deletion
+12
-9
model_zoo/official/recommend/wide_and_deep_multitable/src/callbacks.py
...icial/recommend/wide_and_deep_multitable/src/callbacks.py
+2
-2
model_zoo/official/recommend/wide_and_deep_multitable/src/config.py
...official/recommend/wide_and_deep_multitable/src/config.py
+1
-1
model_zoo/official/recommend/wide_and_deep_multitable/src/metrics.py
...fficial/recommend/wide_and_deep_multitable/src/metrics.py
+1
-2
model_zoo/official/recommend/wide_and_deep_multitable/train_and_eval.py
...cial/recommend/wide_and_deep_multitable/train_and_eval.py
+4
-2
model_zoo/official/recommend/wide_and_deep_multitable/train_and_eval_distribute.py
...end/wide_and_deep_multitable/train_and_eval_distribute.py
+4
-2
未找到文件。
model_zoo/official/recommend/wide_and_deep_multitable/src/callbacks.py
浏览文件 @
1afb8749
...
...
@@ -49,7 +49,7 @@ class LossCallBack(Callback):
cb_params
.
net_outputs
[
1
].
asnumpy
()
cur_step_in_epoch
=
(
cb_params
.
cur_step_num
-
1
)
%
cb_params
.
batch_num
+
1
cur_num
=
cb_params
.
cur_step_num
print
(
"
===loss===
"
,
cb_params
.
cur_epoch_num
,
cur_step_in_epoch
,
wide_loss
,
deep_loss
,
flush
=
True
)
print
(
"
Status:
"
,
cb_params
.
cur_epoch_num
,
cur_step_in_epoch
,
wide_loss
,
deep_loss
,
flush
=
True
)
if
self
.
_per_print_times
!=
0
and
cur_num
%
self
.
_per_print_times
==
0
:
loss_file
=
open
(
self
.
config
.
loss_file_name
,
"a+"
)
loss_file
.
write
(
...
...
@@ -91,6 +91,6 @@ class EvalCallBack(Callback):
end_time
=
time
.
time
()
eval_time
=
int
(
end_time
-
start_time
)
time_str
=
time
.
strftime
(
"%Y-%m-%d %H:%M:%S"
,
time
.
localtime
())
out_str
=
"{}
=====
EvalCallBack model.eval(): {} ; eval_time:{}s"
.
format
(
time_str
,
out
.
values
(),
eval_time
)
out_str
=
"{}
:
EvalCallBack model.eval(): {} ; eval_time:{}s"
.
format
(
time_str
,
out
.
values
(),
eval_time
)
print
(
out_str
)
add_write
(
self
.
eval_file_name
,
out_str
)
This diff is collapsed.
Click to expand it.
model_zoo/official/recommend/wide_and_deep_multitable/src/config.py
浏览文件 @
1afb8749
...
...
@@ -22,7 +22,7 @@ def argparse_init():
parser
=
argparse
.
ArgumentParser
(
description
=
'WideDeep'
)
parser
.
add_argument
(
"--data_path"
,
type
=
str
,
default
=
"./test_raw_data/"
)
# The location of the input data.
parser
.
add_argument
(
"--epochs"
,
type
=
int
,
default
=
200
)
# The number of epochs used to train.
parser
.
add_argument
(
"--epochs"
,
type
=
int
,
default
=
8
)
# The number of epochs used to train.
parser
.
add_argument
(
"--batch_size"
,
type
=
int
,
default
=
131072
)
# Batch size for training and evaluation
parser
.
add_argument
(
"--eval_batch_size"
,
type
=
int
,
default
=
131072
)
# The batch size used for evaluation.
parser
.
add_argument
(
"--deep_layers_dim"
,
type
=
int
,
nargs
=
'+'
,
default
=
[
1024
,
512
,
256
,
128
])
# The sizes of hidden layers for MLP
...
...
This diff is collapsed.
Click to expand it.
model_zoo/official/recommend/wide_and_deep_multitable/src/metrics.py
浏览文件 @
1afb8749
...
...
@@ -148,6 +148,5 @@ class AUCMetric(Metric):
auc
=
roc_auc_score
(
self
.
true_labels
,
self
.
pred_probs
)
MAP
=
new_compute_mAP
(
result_df
,
gb_key
=
"display_ids"
,
top_k
=
12
)
print
(
"====="
*
20
+
" auc_metric end "
)
print
(
"====="
*
20
+
" auc: {}, map: {}"
.
format
(
auc
,
MAP
))
print
(
"Eval result:"
+
" auc: {}, map: {}"
.
format
(
auc
,
MAP
))
return
auc
This diff is collapsed.
Click to expand it.
model_zoo/official/recommend/wide_and_deep_multitable/train_and_eval.py
浏览文件 @
1afb8749
...
...
@@ -89,13 +89,15 @@ def train_and_eval(config):
eval_callback
=
EvalCallBack
(
model
,
ds_eval
,
auc_metric
,
config
)
callback
=
LossCallBack
(
config
)
ckptconfig
=
CheckpointConfig
(
save_checkpoint_steps
=
ds_train
.
get_dataset_size
(),
# Only save the last checkpoint at the last epoch. For saving epochs at each epoch, please
# set save_checkpoint_steps=ds_train.get_dataset_size()
ckptconfig
=
CheckpointConfig
(
save_checkpoint_steps
=
ds_train
.
get_dataset_size
()
*
config
.
epochs
,
keep_checkpoint_max
=
10
)
ckpoint_cb
=
ModelCheckpoint
(
prefix
=
'widedeep_train'
,
directory
=
config
.
ckpt_path
,
config
=
ckptconfig
)
model
.
train
(
epochs
,
ds_train
,
callbacks
=
[
TimeMonitor
(
ds_train
.
get_dataset_size
()),
eval_callback
,
callback
,
ckpoint_cb
])
callback
,
ckpoint_cb
]
,
sink_size
=
ds_train
.
get_dataset_size
()
)
if
__name__
==
"__main__"
:
...
...
This diff is collapsed.
Click to expand it.
model_zoo/official/recommend/wide_and_deep_multitable/train_and_eval_distribute.py
浏览文件 @
1afb8749
...
...
@@ -94,14 +94,16 @@ def train_and_eval(config):
eval_callback
=
EvalCallBack
(
model
,
ds_eval
,
auc_metric
,
config
)
callback
=
LossCallBack
(
config
)
ckptconfig
=
CheckpointConfig
(
save_checkpoint_steps
=
ds_train
.
get_dataset_size
(),
# Only save the last checkpoint at the last epoch. For saving epochs at each epoch, please
# set save_checkpoint_steps=ds_train.get_dataset_size()
ckptconfig
=
CheckpointConfig
(
save_checkpoint_steps
=
ds_train
.
get_dataset_size
()
*
config
.
epochs
,
keep_checkpoint_max
=
10
)
ckpoint_cb
=
ModelCheckpoint
(
prefix
=
'widedeep_train'
,
directory
=
config
.
ckpt_path
,
config
=
ckptconfig
)
callback_list
=
[
TimeMonitor
(
ds_train
.
get_dataset_size
()),
eval_callback
,
callback
]
if
int
(
get_rank
())
==
0
:
callback_list
.
append
(
ckpoint_cb
)
model
.
train
(
epochs
,
ds_train
,
callbacks
=
callback_list
)
model
.
train
(
epochs
,
ds_train
,
callbacks
=
callback_list
,
sink_size
=
ds_train
.
get_dataset_size
()
)
if
__name__
==
"__main__"
:
...
...
This diff is collapsed.
Click to expand it.
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录
反馈
建议
客服
返回
顶部