Add double grad op for sigmoid activation, test=develop (#32971)
Sigmoid: Out = Sigmoid(X) SigmoidGrad: DX = DOut*(1-Out)*Out [This Patch] Out DOut -> SigmoidGradGrad -> DOutNew DDX DDOut DDOut = (1-Out)*Out*DDX DOutNew = (1-2*Out)*DOut*DDX
Showing
想要评论请 注册 或 登录