Created by: ceci3

PR types

Others

PR changes

APIs

Describe

Add SyncBatchNorm API for dygraph

1. because moving mean and moving variance computed all the time in the batch norm C++ kernel , so 'track_running_stats' is not kick in the API now. need to change batch norm kernel.
2. in the train mode, use mini-batch stats, in the eval mode, use global stats, not use use_global_stas and trainable_stats to jugde whether to use global or mini-batch stats