# Activations

## BaseActivation

class `paddle.trainer_config_helpers.activations.``BaseActivation`(name, support_hppl)

  A mark for activation class. Each activation inherit BaseActivation, which has two parameters.

  Parameters:

  - name (basestring) – activation name in paddle config.
  - support_hppl (bool) – True if supported by hppl. HPPL is a library used by paddle internally. Currently, lstm layer can only use activations supported by hppl.

## AbsActivation

class `paddle.trainer_config_helpers.activations.``AbsActivation`

  Abs Activation.

  Forward: \(f(z) = abs(z)\)

  Derivative:

  \[\begin{split}1 &\quad if \quad z > 0 \\ -1 &\quad if \quad z < 0 \\ 0 &\quad if \quad z = 0\end{split}\]

## ExpActivation

class `paddle.trainer_config_helpers.activations.``ExpActivation`

  Exponential Activation.

  \[f(z) = e^z.\]

## IdentityActivation

class `paddle.trainer_config_helpers.activations.``IdentityActivation`

  Identity Activation.

  Just do nothing for output both forward/backward.

## LinearActivation

`paddle.trainer_config_helpers.activations.``LinearActivation`

  alias of `IdentityActivation`

## LogActivation

class `paddle.trainer_config_helpers.activations.``LogActivation`

  Logarithm Activation.

  \[f(z) = log(z)\]

## SquareActivation

class `paddle.trainer_config_helpers.activations.``SquareActivation`

  Square Activation.

  \[f(z) = z^2.\]

## SigmoidActivation

class `paddle.trainer_config_helpers.activations.``SigmoidActivation`

  Sigmoid activation.

  \[f(z) = \frac{1}{1+exp(-z)}\]

## SoftmaxActivation

class `paddle.trainer_config_helpers.activations.``SoftmaxActivation`

  Softmax activation for simple input

  \[P(y=j|x) = \frac{e^{x_j}} {\sum^K_{k=1} e^{x_j} }\]

## SequenceSoftmaxActivation

class `paddle.trainer_config_helpers.activations.``SequenceSoftmaxActivation`

  Softmax activation for one sequence. The dimension of input feature must be 1 and a sequence.

      result = softmax(for each_feature_vector[0] in input_feature)
      for i, each_time_step_output in enumerate(output):
          each_time_step_output = result[i]

## ReluActivation

class `paddle.trainer_config_helpers.activations.``ReluActivation`

  Relu activation.

  forward. \(y = max(0, z)\)

  derivative:

  \[\begin{split}1 &\quad if z > 0 \\ 0 &\quad\mathrm{otherwize}\end{split}\]

## BReluActivation

class `paddle.trainer_config_helpers.activations.``BReluActivation`

  BRelu Activation.

  forward. \(y = min(24, max(0, z))\)

  derivative:

  \[\begin{split}1 &\quad if 0 < z < 24 \\ 0 &\quad \mathrm{otherwise}\end{split}\]

## SoftReluActivation

class `paddle.trainer_config_helpers.activations.``SoftReluActivation`

  SoftRelu Activation.

## TanhActivation

class `paddle.trainer_config_helpers.activations.``TanhActivation`

  Tanh activation.

  \[f(z)=tanh(z)=\frac{e^z-e^{-z}}{e^z+e^{-z}}\]

## STanhActivation

class `paddle.trainer_config_helpers.activations.``STanhActivation`

  Scaled Tanh Activation.

  \[f(z) = 1.7159 * tanh(2/3*z)\]