Created by: mapingshuo

PR types

New features

PR changes

Ops

Describe

Sometimes, we need to do allreduce_sum for multiple variables at the same time, with only one sync_comm op after these allreduce_sum ops. These vars should be the input of the sync_comm op. This PR adds the AsDuplicable function after the AddInput and AddOutput calls, thus enable multiple inputs and outputs.

如果有俩reduce var: x, y, 如果不在sync的输入里面加入这俩var，gc就会在reduce完成前就释放x和y，导致nan错误