Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
Oneflow-Inc
OneFlow-Benchmark
提交
6015332e
O
OneFlow-Benchmark
项目概览
Oneflow-Inc
/
OneFlow-Benchmark
上一次同步 2 年多
通知
1
Star
92
Fork
0
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
0
列表
看板
标记
里程碑
合并请求
0
DevOps
流水线
流水线任务
计划
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
O
OneFlow-Benchmark
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
0
Issue
0
列表
看板
标记
里程碑
合并请求
0
合并请求
0
Pages
DevOps
DevOps
流水线
流水线任务
计划
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
流水线任务
提交
Issue看板
前往新版Gitcode,体验更适合开发者的 AI 搜索 >>
提交
6015332e
编写于
11月 23, 2020
作者:
O
ouyangyu
浏览文件
操作
浏览文件
下载
电子邮件补丁
差异文件
reduce mean
上级
3583a5f4
变更
4
隐藏空白更改
内联
并排
Showing
4 changed file
with
11 addition
and
8 deletion
+11
-8
Classification/cnns/of_cnn_train_val.py
Classification/cnns/of_cnn_train_val.py
+1
-2
Classification/cnns/optimizer_util.py
Classification/cnns/optimizer_util.py
+5
-2
LanguageModeling/BERT/pretrain.py
LanguageModeling/BERT/pretrain.py
+2
-3
LanguageModeling/BERT/util.py
LanguageModeling/BERT/util.py
+3
-1
未找到文件。
Classification/cnns/of_cnn_train_val.py
浏览文件 @
6015332e
...
...
@@ -86,8 +86,7 @@ def TrainNet():
else
:
loss
=
flow
.
nn
.
sparse_softmax_cross_entropy_with_logits
(
labels
,
logits
,
name
=
"softmax_loss"
)
if
not
args
.
use_fp16
:
loss
=
flow
.
math
.
reduce_mean
(
loss
)
loss
=
flow
.
math
.
reduce_mean
(
loss
)
predictions
=
flow
.
nn
.
softmax
(
logits
)
outputs
=
{
"loss"
:
loss
,
"predictions"
:
predictions
,
"labels"
:
labels
}
...
...
Classification/cnns/optimizer_util.py
浏览文件 @
6015332e
...
...
@@ -101,9 +101,11 @@ def set_up_optimizer(loss, args):
grad_clipping
=
grad_clipping
).
minimize
(
loss
)
elif
args
.
optimizer
==
'adam'
:
loss_scale_policy
=
None
if
args
.
use_fp16
:
loss_scale_policy
=
flow
.
optimizer
.
loss_scale
.
dynamic_loss_scale
(
increment_period
=
2000
);
if
args
.
wd
>
0
and
args
.
wd
<
1.0
:
print
(
"Optimizer: AdamW"
)
loss_scale_policy
=
flow
.
optimizer
.
loss_scale
.
dynamic_loss_scale
(
increment_period
=
20
);
flow
.
optimizer
.
AdamW
(
lr_scheduler
=
lr_scheduler
,
weight_decay
=
args
.
wd
,
...
...
@@ -116,7 +118,8 @@ def set_up_optimizer(loss, args):
print
(
"Optimizer: Adam"
)
flow
.
optimizer
.
Adam
(
lr_scheduler
=
lr_scheduler
,
grad_clipping
=
grad_clipping
,
epsilon
=
args
.
epsilon
epsilon
=
args
.
epsilon
,
loss_scale_policy
=
loss_scale_policy
).
minimize
(
loss
)
elif
args
.
optimizer
==
'rmsprop'
:
print
(
"Optimizer: RMSProp"
)
...
...
LanguageModeling/BERT/pretrain.py
浏览文件 @
6015332e
...
...
@@ -82,9 +82,8 @@ def PreTrain(
initializer_range
=
initializer_range
,
)
with
flow
.
scope
.
namespace
(
"cls-loss"
):
if
not
use_fp16
:
lm_loss
=
flow
.
math
.
reduce_mean
(
lm_loss
)
ns_loss
=
flow
.
math
.
reduce_mean
(
ns_loss
)
lm_loss
=
flow
.
math
.
reduce_mean
(
lm_loss
)
ns_loss
=
flow
.
math
.
reduce_mean
(
ns_loss
)
total_loss
=
lm_loss
+
ns_loss
return
total_loss
,
lm_loss
,
ns_loss
...
...
LanguageModeling/BERT/util.py
浏览文件 @
6015332e
...
...
@@ -166,7 +166,9 @@ def CreateOptimizer(args):
lr_warmup
=
flow
.
optimizer
.
warmup
.
linear
(
warmup_batches
,
0
)
lr_scheduler
=
flow
.
optimizer
.
PolynomialSchduler
(
args
.
learning_rate
,
args
.
iter_num
,
0.0
,
warmup
=
lr_warmup
)
loss_scale_policy
=
flow
.
optimizer
.
loss_scale
.
dynamic_loss_scale
(
increment_period
=
20
);
loss_scale_policy
=
None
if
args
.
use_fp16
:
loss_scale_policy
=
flow
.
optimizer
.
loss_scale
.
dynamic_loss_scale
(
increment_period
=
2000
);
return
flow
.
optimizer
.
AdamW
(
lr_scheduler
,
epsilon
=
1e-6
,
weight_decay
=
args
.
weight_decay_rate
,
weight_decay_excludes
=
[
"bias"
,
"LayerNorm"
,
"layer_norm"
],
grad_clipping
=
flow
.
optimizer
.
grad_clipping
.
by_global_norm
(
1.0
),
...
...
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录