提交 810ece8f 编写于 作者: Y Yang Zhang

Fix learning rate scaling bug

this bug is quite peculiar and hard to track down, when learning rate for a
parameter is scaled via param_attr and learning rate schedulers are used,
`append_optimizer_op` will error out complaining `LearningRate` input is null

turns out learning rate scaling is done in `_create_param_lr`, which basically
add a scale op, the problem is: it is appended to `orig_prog` (since
`global_learning_rate()` variable is in it), therefore the resulting scaled
learning rate variable can not be found in `train_prog`.

the reason it works previously w/o lr scaling is this:
`clone()` will create a variable with the same name as the
`global_learning_rate()` variable, which will be used in `append_optimizer_op`
上级 d8d5176d
......@@ -410,7 +410,8 @@ class StaticGraphAdapter(object):
and self.model._optimizer._learning_rate_map:
# HACK workaround learning rate map issue
lr_var = self.model._optimizer._learning_rate_map[self._orig_prog]
self.model._optimizer._learning_rate_map[prog] = lr_var
new_lr_var = prog.global_block().vars[lr_var.name]
self.model._optimizer._learning_rate_map[prog] = new_lr_var
losses = []
metrics = []
......
Markdown is supported
0% .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑!
想要评论请 注册