From 164b642bcc02955aaa8a02f8f0cdd8480cf3d038 Mon Sep 17 00:00:00 2001 From: Lv Mengsi Date: Wed, 11 Dec 2019 16:15:18 +0800 Subject: [PATCH] update bn (#1653) * update bn * update --- doc/fluid/api_cn/layers_cn/batch_norm_cn.rst | 16 +++++++++------- 1 file changed, 9 insertions(+), 7 deletions(-) diff --git a/doc/fluid/api_cn/layers_cn/batch_norm_cn.rst b/doc/fluid/api_cn/layers_cn/batch_norm_cn.rst index 0d9ab01c4..0ddd6f8a0 100644 --- a/doc/fluid/api_cn/layers_cn/batch_norm_cn.rst +++ b/doc/fluid/api_cn/layers_cn/batch_norm_cn.rst @@ -17,13 +17,15 @@ batch_norm ``input`` 是mini-batch的输入。 .. math:: - \mu_{\beta} &\gets \frac{1}{m} \sum_{i=1}^{m} x_i \quad &// mini-batch-mean \\ - \sigma_{\beta}^{2} &\gets \frac{1}{m} \sum_{i=1}^{m}(x_i - \mu_{\beta})^2 \quad &// mini-batch-variance \\ - \hat{x_i} &\gets \frac{x_i - \mu_\beta} {\sqrt{\sigma_{\beta}^{2} + \epsilon}} \quad &// normalize \\ - y_i &\gets \gamma \hat{x_i} + \beta \quad &// scale-and-shift - - moving\_mean = moving\_mean * momentum + mini\_batch\_mean * (1. - momentum) \global mean - moving\_variance = moving\_variance * momentum + mini\_batch\_var * (1. - momentum) \global variance + \mu_{\beta} &\gets \frac{1}{m} \sum_{i=1}^{m} x_i \qquad &//\ + \ mini-batch\ mean \\ + \sigma_{\beta}^{2} &\gets \frac{1}{m} \sum_{i=1}^{m}(x_i - \mu_{\beta})^2 \qquad &//\ + \ mini-batch\ variance \\ + \hat{x_i} &\gets \frac{x_i - \mu_\beta} {\sqrt{\sigma_{\beta}^{2} + \epsilon}} \qquad &//\ normalize \\ + y_i &\gets \gamma \hat{x_i} + \beta \qquad &//\ scale\ and\ shift + + moving\_mean = moving\_mean * momentum + mini\_batch\_mean * (1. - momentum) \\ + moving\_variance = moving\_variance * momentum + mini\_batch\_var * (1. - momentum) moving_mean和moving_var是训练过程中统计得到的全局均值和方差，在预测或者评估中使用。 -- GitLab