未验证 提交 168bbd6d 编写于 作者: J juncaipeng 提交者: GitHub

Update L1Decay and L2Decay (#1367)

* modify L1DecayRegularizer, L1Decay, L2DecayRegularizer and L2Decay, test=develop
上级 256c00be
......@@ -5,33 +5,7 @@ L1DecayRegularizer
.. py:class:: paddle.fluid.regularizer.L1DecayRegularizer(regularization_coeff=0.0)
L1DecayRegularizer实现L1权重衰减正则化,用于模型训练,使得权重矩阵稀疏。
**注意:paddle.fluid.regularizer.L1DecayRegularizer是paddle.fluid.regularizer.L1Decay的别名,推荐使用paddle.fluid.regularizer.L1Decay。**
具体实现中,L1权重衰减正则化的计算公式如下:
详见 :ref:`cn_api_fluid_regularizer_L1Decay` 接口的使用文档。
.. math::
\\L1WeightDecay=reg\_coeff∗sign(parameter)\\
参数:
- **regularization_coeff** (float) – L1正则化系数,默认值为0.0。
**代码示例**
.. code-block:: python
import paddle.fluid as fluid
main_prog = fluid.Program()
startup_prog = fluid.Program()
with fluid.program_guard(main_prog, startup_prog):
data = fluid.layers.data(name='image', shape=[3, 28, 28], dtype='float32')
label = fluid.layers.data(name='label', shape=[1], dtype='int64')
hidden = fluid.layers.fc(input=data, size=128, act='relu')
prediction = fluid.layers.fc(input=hidden, size=10, act='softmax')
loss = fluid.layers.cross_entropy(input=prediction, label=label)
avg_loss = fluid.layers.mean(loss)
optimizer = fluid.optimizer.Adagrad(
learning_rate=1e-4,
regularization=fluid.regularizer.L1DecayRegularizer(
regularization_coeff=0.1))
optimizer.minimize(avg_loss)
......@@ -6,8 +6,6 @@ L1Decay
.. py:attribute:: paddle.fluid.regularizer.L1Decay(regularization_coeff=0.0)
``L1Decay`` 是 ``L1DecayRegularizer`` 的别名。
L1Decay实现L1权重衰减正则化,用于模型训练,使得权重矩阵稀疏。
具体实现中,L1权重衰减正则化的计算公式如下:
......
......@@ -5,33 +5,8 @@ L2DecayRegularizer
.. py:class:: paddle.fluid.regularizer.L2DecayRegularizer(regularization_coeff=0.0)
L2DecayRegularizer实现L2权重衰减正则化,用于模型训练,有助于防止模型对训练数据过拟合。
**注意:paddle.fluid.regularizer.L2DecayRegularizer是paddle.fluid.regularizer.L2Decay的别名,推荐使用paddle.fluid.regularizer.L2Decay。**
具体实现中,L2权重衰减正则化的计算公式如下:
详见 :ref:`cn_api_fluid_regularizer_L2Decay` 接口的使用文档。
.. math::
\\L2WeightDecay=reg\_coeff*parameter\\
参数:
- **regularization_coeff** (float) – 正则化系数,默认值为0.0。
**代码示例**
.. code-block:: python
import paddle.fluid as fluid
main_prog = fluid.Program()
startup_prog = fluid.Program()
with fluid.program_guard(main_prog, startup_prog):
data = fluid.layers.data(name='image', shape=[3, 28, 28], dtype='float32')
label = fluid.layers.data(name='label', shape=[1], dtype='int64')
hidden = fluid.layers.fc(input=data, size=128, act='relu')
prediction = fluid.layers.fc(input=hidden, size=10, act='softmax')
loss = fluid.layers.cross_entropy(input=prediction, label=label)
avg_loss = fluid.layers.mean(loss)
optimizer = fluid.optimizer.Adagrad(
learning_rate=1e-4,
regularization=fluid.regularizer.L2DecayRegularizer(
regularization_coeff=0.1))
optimizer.minimize(avg_loss)
......@@ -5,8 +5,6 @@ L2Decay
.. py:attribute:: paddle.fluid.regularizer.L2Decay
``L2Decay`` 是 ``L2DecayRegularizer`` 的别名。
L2Decay实现L2权重衰减正则化,用于模型训练,有助于防止模型对训练数据过拟合。
具体实现中,L2权重衰减正则化的计算公式如下:
......
Markdown is supported
0% .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑!
想要评论请 注册