Optimizer¶

class paddle.v2.fluid.optimizer.Optimizer(learning_rate, global_step=None, regularization=None)

Optimizer Base class.

Define the common interface of an optimizer. User should not use this class directly, but need to use one of it’s implementation.

create_optimization_pass(parameters_and_grads, loss, startup_program=None)

Add optimization operators to update gradients to variables.

Parameters:	loss – the target that this optimization is for. parameters_and_grads – a list of (variable, gradient) pair to update.
Returns:	a list of operators that will complete one step of optimization. This will include parameter update ops, global step update ops and any other custom ops required by subclasses to manage their internal state. :param startup_program:
Return type:	return_op_list

minimize(loss, startup_program=None, parameter_list=None, no_grad_set=None)

Add operations to minimize loss by updating parameter_list.

This method combines interface append_backward() and create_optimization_pass() into one.

SGDOptimizer¶

class paddle.v2.fluid.optimizer.SGDOptimizer(learning_rate, **kwargs): Simple SGD optimizer without any state.

class paddle.v2.fluid.optimizer.MomentumOptimizer(learning_rate, momentum, use_nesterov=False, **kwargs): Simple Momentum optimizer with velocity state

class paddle.v2.fluid.optimizer.AdagradOptimizer(learning_rate, epsilon=1e-06, **kwargs): Simple Adagrad optimizer with moment state

class paddle.v2.fluid.optimizer.AdamOptimizer(learning_rate=0.001, beta1=0.9, beta2=0.999, epsilon=1e-08, **kwargs): Implements the Adam Optimizer

class paddle.v2.fluid.optimizer.AdamaxOptimizer(learning_rate=0.001, beta1=0.9, beta2=0.999, epsilon=1e-08, **kwargs): Implements the Adamax Optimizer

class paddle.v2.fluid.optimizer.DecayedAdagradOptimizer(learning_rate, decay=0.95, epsilon=1e-06, **kwargs): Simple Decayed Adagrad optimizer with moment state