Activations

class paddle::ActivationFunction

Activation function is a function that transforms a set of input signals into an output signals. The purpose of the activation function is to introduce non-liearilty into the network.

Note

Common activation function are provieded, including linear, sigmoid, softmax, sequence_max, relu, brelu, tanh, stanh, softrelu, abs, square, exponential.

Subclassed by paddle::IdentityActivation

Public Functions

ActivationFunction()

virtual ~ActivationFunction()

virtual void forward(Argument &act) = 0

Foward propagation.

act.value <- f(act.value), where f is the activation function. Suppose that before calling forward(), act.value is x and after forward() is called, act.value is y, then y = f(x).

Usually, act is Layer::output_

virtual void backward(Argument &act) = 0

Backward propagaion.

x and y are defined in the above comment for forward().

• Before calling backward(), act.grad = dE / dy, where E is the error/cost
• After backward() returns, act.grad = dE / dx = (dE/dy) * (dy/dx)

virtual const std::string &getName() const = 0

Public Static Functions

ActivationFunction *create(const std::string &type)

std::vector<std::string> getAllRegisteredTypes()