Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
magicwindyyd
mindspore
提交
352c6faf
M
mindspore
项目概览
magicwindyyd
/
mindspore
与 Fork 源项目一致
Fork自
MindSpore / mindspore
通知
1
Star
1
Fork
0
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
0
列表
看板
标记
里程碑
合并请求
0
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
M
mindspore
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
0
Issue
0
列表
看板
标记
里程碑
合并请求
0
合并请求
0
Pages
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
提交
Issue看板
提交
352c6faf
编写于
3月 31, 2020
作者:
M
mindspore-ci-bot
提交者:
Gitee
3月 31, 2020
浏览文件
操作
浏览文件
下载
差异文件
!18 enable use float type learning rate in lars optimizer
Merge pull request !18 from gziyan/master
上级
da447b8d
4cbcd8e9
变更
2
隐藏空白更改
内联
并排
Showing
2 changed file
with
24 addition
and
4 deletion
+24
-4
mindspore/nn/optim/lars.py
mindspore/nn/optim/lars.py
+6
-3
tests/ut/python/nn/optim/test_lars.py
tests/ut/python/nn/optim/test_lars.py
+18
-1
未找到文件。
mindspore/nn/optim/lars.py
浏览文件 @
352c6faf
...
...
@@ -13,12 +13,14 @@
# limitations under the License.
# ============================================================================
"""lars optimizer"""
from
typing
import
Iterable
from
mindspore.common
import
dtype
as
mstype
from
mindspore.common
import
Tensor
from
mindspore.common.initializer
import
initializer
from
mindspore.common.parameter
import
Parameter
from
mindspore.ops
import
operations
as
P
from
mindspore.ops
import
composite
as
C
from
mindspore.ops
import
functional
as
F
from
mindspore.common.parameter
import
Parameter
from
mindspore.nn.cell
import
Cell
from
.optimizer
import
grad_scale
...
...
@@ -111,7 +113,8 @@ class LARS(Cell):
self
.
gather
=
None
self
.
global_step
=
None
self
.
axis
=
None
if
not
isinstance
(
self
.
learning_rate
,
float
):
if
isinstance
(
self
.
learning_rate
.
default_input
,
Iterable
)
or
\
(
isinstance
(
self
.
learning_rate
.
default_input
,
Tensor
)
and
self
.
learning_rate
.
default_input
.
dim
()
==
1
):
self
.
dynamic_lr
=
True
self
.
assignadd
=
P
.
AssignAdd
()
self
.
gather
=
P
.
GatherV2
()
...
...
@@ -124,7 +127,7 @@ class LARS(Cell):
lr
=
self
.
gather
(
self
.
learning_rate
,
self
.
global_step
,
self
.
axis
)
F
.
control_depend
(
lr
,
self
.
assignadd
(
self
.
global_step
,
1
))
else
:
lr
=
F
.
scalar_to_array
(
self
.
learning_rate
)
lr
=
self
.
learning_rate
if
self
.
reciprocal_scale
!=
1.0
:
gradients
=
self
.
hyper_map
(
F
.
partial
(
grad_scale
,
self
.
reciprocal_scale
),
gradients
)
...
...
tests/ut/python/nn/optim/test_lars.py
浏览文件 @
352c6faf
...
...
@@ -46,7 +46,7 @@ class Net(nn.Cell):
return
x
def
test_lars
():
def
test_lars
_multi_step_lr
():
inputs
=
Tensor
(
np
.
ones
([
1
,
64
]).
astype
(
np
.
float32
))
label
=
Tensor
(
np
.
zeros
([
1
,
10
]).
astype
(
np
.
float32
))
net
=
Net
()
...
...
@@ -61,3 +61,20 @@ def test_lars():
net_with_loss
=
WithLossCell
(
net
,
loss
)
train_network
=
TrainOneStepCell
(
net_with_loss
,
optimizer
)
_executor
.
compile
(
train_network
,
inputs
,
label
)
def
test_lars_float_lr
():
inputs
=
Tensor
(
np
.
ones
([
1
,
64
]).
astype
(
np
.
float32
))
label
=
Tensor
(
np
.
zeros
([
1
,
10
]).
astype
(
np
.
float32
))
net
=
Net
()
net
.
set_train
()
loss
=
nn
.
SoftmaxCrossEntropyWithLogits
()
lr
=
0.1
SGD
=
Momentum
(
net
.
trainable_params
(),
lr
,
0.9
)
optimizer
=
LARS
(
SGD
,
epsilon
=
1e-08
,
hyperpara
=
0.02
,
decay_filter
=
lambda
x
:
'bn'
not
in
x
.
name
,
lars_filter
=
lambda
x
:
'bn'
not
in
x
.
name
)
net_with_loss
=
WithLossCell
(
net
,
loss
)
train_network
=
TrainOneStepCell
(
net_with_loss
,
optimizer
)
_executor
.
compile
(
train_network
,
inputs
,
label
)
\ No newline at end of file
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录