如何在训练过程中使得参数通过计算进行自适应更新?
Created by: linshufei
版本、环境信息: 1)PaddlePaddle版本:1.5 2)CPU:CPU 3)GPU:无 4)系统环境:Linux python2.7 复现信息:如为报错,请给出复现环境、复现步骤 复现代码如下:
python实现:
threshold = 0.3
lmbda = 1.0
batches = [np.array([x], np.float32) for x in range(1, 6)]
for x in batches:
cond = 1 if threshold <= x else 0
lmbda = lmbda / (1 + (1 - 2 * cond) * 0.1)
print lmbda
python输出
1.11111111111
1.23456790123
1.37174211248
1.52415790276
1.69350878084
paddle实现:
import numpy as np
import paddle.fluid as fluid
import paddle.fluid.layers as layers
threshold = fluid.layers.fill_constant(shape=[1], value=0.3, dtype='float32')
lmbda = fluid.layers.fill_constant(shape=[1], value=1.0, dtype='float32')
x = fluid.data(name='x', shape=[1], dtype='float32')
cond = fluid.layers.cast(x=(threshold <= x), dtype='float32')
lmbda = lmbda / (1 + (1 - 2 * cond) * 0.1)
place = fluid.CPUPlace()
exe = fluid.Executor(place)
batches = [np.array([x], np.float32) for x in range(1, 6)]
for x in batches:
print(exe.run(fluid.default_main_program(), fetch_list=[lmbda, cond], feed={'x': x}))
paddle输出:
[array([1.1111112], dtype=float32), array([1.], dtype=float32)]
[array([1.1111112], dtype=float32), array([1.], dtype=float32)]
[array([1.1111112], dtype=float32), array([1.], dtype=float32)]
[array([1.1111112], dtype=float32), array([1.], dtype=float32)]
[array([1.1111112], dtype=float32), array([1.], dtype=float32)]
问题描述
用python代码实现的lmbda的输出会根据以上一次计算的结果进行更新,而paddle实现时每一次lmbda都为1.0,再做一次运算lmbda = lmbda / (1 + (1 - 2 * cond) * 0.1)变成1.1111112。 需求:
需求: paddle有没有python接口可以使得lmdba也能根据上一次计算的结果进行更新,就像是模型参数会依据上一次forward和backward的结果进行更新?