Created by: sandyhouse

PR types

New features

PR changes

APIs

Describe

Initialize gloo by default for distributed training. Initialize gloo for low level api. Now, we only support to initialize gloo with http server as it is much more common than hdfs store and local files.

Usage:

1. Use with spawn

import paddle
def train():
    paddle.distributed.init_parallel_env()
    ...

if __name__ == "__main__":
    paddle.distributed.spawn(train)

2. Use with fleetrun

import paddle
def train():
    paddle.distributed.init_parallel_env()
    ...

if __name__ == "__main__":
    train()

Start a train with the following command:

fleetrun train.py