Created by: qili93

PR types

Bug fixes

PR changes

OPs

Describe

Fix the grad output of ELUGradFunctor when alpha is less then ZERO.

out=max(0,x)+min(0,α∗(e^x−1))

if alpha > 0 and x > 0, then out = x, then dx = dout if alpha > 0 and x <=0, then out = α∗(e^x−1), then dx = dout ∗ α ∗ e^x if alpha <= 0 and x > 0, then out = x + α∗(e^x−1), then dx = dout ∗ (1 + α ∗ e^x) if alpha <= 0 and x <= 0, then out = 0, then dx = 0