Created by: zuowang

When training on multi nodes, trainers on all nodes train a pass together. But trainers on each node get DataBatch from DataProvider independently, how do they know which part of the whole data should it fetch from the source?