diff --git a/doc/fluid/api_cn/layers_cn/batch_norm_cn.rst b/doc/fluid/api_cn/layers_cn/batch_norm_cn.rst index 0d9ab01c4130f5313c38aa345fe70639527323bd..0ddd6f8a027f57157320d4038e9dee226a1300e4 100644 --- a/doc/fluid/api_cn/layers_cn/batch_norm_cn.rst +++ b/doc/fluid/api_cn/layers_cn/batch_norm_cn.rst @@ -17,13 +17,15 @@ batch_norm ``input`` 是mini-batch的输入。 .. math:: - \mu_{\beta} &\gets \frac{1}{m} \sum_{i=1}^{m} x_i \quad &// mini-batch-mean \\ - \sigma_{\beta}^{2} &\gets \frac{1}{m} \sum_{i=1}^{m}(x_i - \mu_{\beta})^2 \quad &// mini-batch-variance \\ - \hat{x_i} &\gets \frac{x_i - \mu_\beta} {\sqrt{\sigma_{\beta}^{2} + \epsilon}} \quad &// normalize \\ - y_i &\gets \gamma \hat{x_i} + \beta \quad &// scale-and-shift - - moving\_mean = moving\_mean * momentum + mini\_batch\_mean * (1. - momentum) \global mean - moving\_variance = moving\_variance * momentum + mini\_batch\_var * (1. - momentum) \global variance + \mu_{\beta} &\gets \frac{1}{m} \sum_{i=1}^{m} x_i \qquad &//\ + \ mini-batch\ mean \\ + \sigma_{\beta}^{2} &\gets \frac{1}{m} \sum_{i=1}^{m}(x_i - \mu_{\beta})^2 \qquad &//\ + \ mini-batch\ variance \\ + \hat{x_i} &\gets \frac{x_i - \mu_\beta} {\sqrt{\sigma_{\beta}^{2} + \epsilon}} \qquad &//\ normalize \\ + y_i &\gets \gamma \hat{x_i} + \beta \qquad &//\ scale\ and\ shift + + moving\_mean = moving\_mean * momentum + mini\_batch\_mean * (1. - momentum) \\ + moving\_variance = moving\_variance * momentum + mini\_batch\_var * (1. - momentum) moving_mean和moving_var是训练过程中统计得到的全局均值和方差,在预测或者评估中使用。