Created by: wangxicoding
-
DGC doesn't support fuse for now, add PADDLE_ENFORCE_NE in fuse_all_reduce_op_pass.cc to judge if use DGC and fuse same time。see https://github.com/PaddlePaddle/Paddle/pull/20935
-
Fix DGC gather buffer illegal problem by alloc buffer in dgc_op, avoid multi-stream bug in memory::Alloc. Reuse velocity in dgc_op & dgc_momentum_op, save memory & reserve global momentum information. see https://github.com/PaddlePaddle/Paddle/pull/21012