BaseActivation¶
-
class
paddle.trainer_config_helpers.activations.BaseActivation(name, support_hppl) A mark for activation class. Each activation inherit BaseActivation, which has two parameters.
Parameters: - name (basestring) – activation name in paddle config.
- support_hppl (bool) – True if supported by hppl. HPPL is a library used by paddle internally. Currently, lstm layer can only use activations supported by hppl.
AbsActivation¶
-
class
paddle.trainer_config_helpers.activations.AbsActivation Abs Activation.
Forward: \(f(z) = abs(z)\)
Derivative:
\[\begin{split}1 &\quad if \quad z > 0 \\ -1 &\quad if \quad z < 0 \\ 0 &\quad if \quad z = 0\end{split}\]
ExpActivation¶
-
class
paddle.trainer_config_helpers.activations.ExpActivation Exponential Activation.
\[f(z) = e^z.\]
IdentityActivation¶
-
class
paddle.trainer_config_helpers.activations.IdentityActivation Identity Activation.
Just do nothing for output both forward/backward.
LinearActivation¶
-
paddle.trainer_config_helpers.activations.LinearActivation alias of
IdentityActivation
SquareActivation¶
-
class
paddle.trainer_config_helpers.activations.SquareActivation Square Activation.
\[f(z) = z^2.\]
SigmoidActivation¶
-
class
paddle.trainer_config_helpers.activations.SigmoidActivation Sigmoid activation.
\[f(z) = \frac{1}{1+exp(-z)}\]
SoftmaxActivation¶
-
class
paddle.trainer_config_helpers.activations.SoftmaxActivation Softmax activation for simple input
\[P(y=j|x) = \frac{e^{x_j}} {\sum^K_{k=1} e^{x_j} }\]
SequenceSoftmaxActivation¶
-
class
paddle.trainer_config_helpers.activations.SequenceSoftmaxActivation Softmax activation for one sequence. The dimension of input feature must be 1 and a sequence.
result = softmax(for each_feature_vector[0] in input_feature) for i, each_time_step_output in enumerate(output): each_time_step_output = result[i]
ReluActivation¶
-
class
paddle.trainer_config_helpers.activations.ReluActivation Relu activation.
forward. \(y = max(0, z)\)
derivative:
\[\begin{split}1 &\quad if z > 0 \\ 0 &\quad\mathrm{otherwize}\end{split}\]
BReluActivation¶
-
class
paddle.trainer_config_helpers.activations.BReluActivation BRelu Activation.
forward. \(y = min(24, max(0, z))\)
derivative:
\[\begin{split}1 &\quad if 0 < z < 24 \\ 0 &\quad \mathrm{otherwise}\end{split}\]
SoftReluActivation¶
-
class
paddle.trainer_config_helpers.activations.SoftReluActivation SoftRelu Activation.
TanhActivation¶
-
class
paddle.trainer_config_helpers.activations.TanhActivation Tanh activation.
\[f(z)=tanh(z)=\frac{e^z-e^{-z}}{e^z+e^{-z}}\]
STanhActivation¶
-
class
paddle.trainer_config_helpers.activations.STanhActivation Scaled Tanh Activation.
\[f(z) = 1.7159 * tanh(2/3*z)\]