Created by: sandyhouse

use allreduce operation to sync trainers for some tasks such as downloading.

Todo: maybe run a global startup once to avoid generating a new ring at each startup program running.