Parameter Attributes

class paddle.trainer_config_helpers.attrs.ParameterAttribute(name=None, is_static=False, initial_std=None, initial_mean=None, initial_max=None, initial_min=None, l1_rate=None, l2_rate=None, learning_rate=None, momentum=None, gradient_clipping_threshold=None, sparse_update=False)

Parameter Attributes object. To fine-tuning network training process, user can set attribute to control training details, such as l1,l2 rate / learning rate / how to init param.

NOTE: IT IS A HIGH LEVEL USER INTERFACE.

Parameters:
  • is_static (bool) – True if this parameter will be fixed while training.
  • initial_std (float or None) – Gauss Random initialization standard deviation. None if not using Gauss Random initialize parameter.
  • initial_mean (float or None) – Gauss Random initialization mean. None if not using Gauss Random initialize parameter.
  • initial_max (float or None) – Uniform initialization max value.
  • initial_min (float or None) – Uniform initialization min value.
  • l1_rate (float or None) – the l1 regularization factor
  • l2_rate (float or None) – the l2 regularization factor
  • learning_rate (float or None) – The parameter learning rate. None means 1. The learning rate when optimize is LEARNING_RATE = GLOBAL_LEARNING_RATE * PARAMETER_LEARNING_RATE * SCHEDULER_FACTOR.
  • momentum (float or None) – The parameter momentum. None means use global value.
  • gradient_clipping_threshold (float) – gradient clipping threshold. If gradient value larger than some value, will be clipped.
  • sparse_update (bool) – Enable sparse update for this parameter. It will enable both local and remote sparse update.
set_default_parameter_name(name)

Set default parameter name. If parameter not set, then will use default parameter name.

Parameters:name (basestring) – default parameter name.
class paddle.trainer_config_helpers.attrs.ExtraLayerAttribute(error_clipping_threshold=None, drop_rate=None, device=None)

Some high level layer attributes config. You can set all attributes here, but some layer doesn’t support all attributes. If you set an attribute to a layer that not support this attribute, paddle will print an error and core.

Parameters:
  • error_clipping_threshold (float) – Error clipping threshold.
  • drop_rate (float) – Dropout rate. Dropout will create a mask on layer output. The dropout rate is the zero rate of this mask. The details of what dropout is please refer to here.
  • device (int) –

    device ID of layer. device=-1, use CPU. device>=0, use GPU. The details allocation in parallel_nn please refer to here.
paddle.trainer_config_helpers.attrs.ParamAttr

alias of ParameterAttribute

paddle.trainer_config_helpers.attrs.ExtraAttr

alias of ExtraLayerAttribute