Skip to content

P
Paddle

PaddlePaddle / Paddle
大约 1 年 前同步成功

通知 2298
Star 20931
Fork 5422
P
Paddle

Enhance batch_norm_op: support 2d and 5d data and unbiased variance estimation !5845

!5845 开放中 由 saxon_zh@saxon_zh 创建4 of 6 tasks completed4/6 tasks

Created by: zhouxiao-coder

Solve #5811 (closed)

As part of the job of another PR #5686 , I separate the code that related to batch_norm_op enhancement as a standalone PR for better code review.

This PR does several modifications to the original batch_norm_op implementation.

  • Extract normalization logic to a common header file, check issue #5685 (closed) for disscussion.

  • Fix assertion logic. Some outputs should appear only in training mode.

  • Support unbiased variance estimation, i.e. the bessel's correction.

  • Support 2D tensor (tensor_format="NC") and 5D tensor (tensor_format="NCDHW" or "NDHWC") in CPU

  • Support 2D tensor (tensor_format="NC") and 5D tensor (tensor_format="NCDHW" or "NDHWC") in GPU.

  • Operator unit test. See issue #5842 (closed) .

2D tensor is not supported by cudnn batch norm implementation, so changes need to be made to the GPU code.