Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
兽拳
Paddle
提交
ed1a0536
P
Paddle
项目概览
兽拳
/
Paddle
与 Fork 源项目一致
Fork自
PaddlePaddle / Paddle
通知
1
Star
1
Fork
0
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
1
列表
看板
标记
里程碑
合并请求
0
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
P
Paddle
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
1
Issue
1
列表
看板
标记
里程碑
合并请求
0
合并请求
0
Pages
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
提交
Issue看板
体验新版 GitCode,发现更多精彩内容 >>
未验证
提交
ed1a0536
编写于
2月 26, 2018
作者:
Q
Qiao Longfei
提交者:
GitHub
2月 26, 2018
浏览文件
操作
浏览文件
下载
差异文件
Merge pull request #8545 from jacquesqiao/fix-optimize-multi-program
create learning rate for multi program
上级
2a3b9ff1
ea9e62b8
变更
1
隐藏空白更改
内联
并排
Showing
1 changed file
with
34 addition
and
19 deletion
+34
-19
python/paddle/fluid/optimizer.py
python/paddle/fluid/optimizer.py
+34
-19
未找到文件。
python/paddle/fluid/optimizer.py
浏览文件 @
ed1a0536
...
...
@@ -36,10 +36,18 @@ class Optimizer(object):
"""
def
__init__
(
self
,
learning_rate
,
global_step
=
None
,
regularization
=
None
):
assert
learning_rate
is
not
None
if
not
isinstance
(
learning_rate
,
float
)
and
\
not
isinstance
(
learning_rate
,
framework
.
Variable
):
raise
TypeError
(
"learning rate should be float or Variable"
)
self
.
_global_step
=
global_step
self
.
regularization
=
regularization
self
.
_global_learning_rate
=
learning_rate
self
.
_learning_rate
=
learning_rate
# each program should have a independent learning rate
# program -> Variable(learning_rate)
self
.
_learning_rate_map
=
dict
()
if
isinstance
(
self
.
_learning_rate
,
framework
.
Variable
):
self
.
_learning_rate_map
[
framework
.
default_main_program
(
)]
=
self
.
_learning_rate
# Dictionary of accumulators. Some optimizer subclasses need to
# allocate and manage extra variables associated with the parameters
# to train. These variables are called accumulators.
...
...
@@ -48,26 +56,33 @@ class Optimizer(object):
self
.
helper
=
None
def
_create_global_learning_rate
(
self
):
if
isinstance
(
self
.
_global_learning_rate
,
float
):
self
.
_global_learning_rate
=
layers
.
create_global_var
(
name
=
unique_name
.
generate
(
"learning_rate"
),
shape
=
[
1
],
value
=
float
(
self
.
_global_learning_rate
),
dtype
=
'float32'
,
persistable
=
True
)
if
not
isinstance
(
self
.
_global_learning_rate
,
framework
.
Variable
):
raise
ValueError
(
"learning rate should be a Variable, "
"actual type is %s"
,
type
(
self
.
_global_learning_rate
))
@
property
def
global_learning_rate
(
self
):
lr
=
self
.
global_learning_rate
()
if
isinstance
(
lr
,
framework
.
Variable
):
return
else
:
if
not
isinstance
(
self
.
_learning_rate
,
float
):
raise
TypeError
(
"learning rate variable is create outside optimizer,"
"can not create new learning rate variable for new program"
)
# create learning rate in the current main program
self
.
_learning_rate_map
[
framework
.
default_main_program
(
)]
=
layers
.
create_global_var
(
name
=
unique_name
.
generate
(
"learning_rate"
),
shape
=
[
1
],
value
=
float
(
self
.
_learning_rate
),
dtype
=
'float32'
,
persistable
=
True
)
def
global_learning_rate
(
self
,
program
=
None
):
"""
get global decayed learning rate
:return:
"""
return
self
.
_global_learning_rate
if
program
is
None
:
program
=
framework
.
default_main_program
()
return
self
.
_learning_rate_map
.
get
(
program
,
None
)
def
_append_optimize_op
(
self
,
block
,
param_and_grad
):
""" append optimize operator to block and return all the added optimize_op
...
...
@@ -78,7 +93,7 @@ class Optimizer(object):
# create learning rate variable for every parameter
param
=
param_and_grad
[
0
]
param_lr
=
param
.
optimize_attr
[
'learning_rate'
]
return
self
.
_global_learning_rate
*
param_lr
return
self
.
global_learning_rate
()
*
param_lr
def
_create_accumulators
(
self
,
block
,
parameters
):
"""Create all accumulators needed by the parameters
...
...
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录