Created by: seiriosPlus

By using MPI API, we enables PaddlePaddle to take advantage of high performance low latency networks such as Infiniband. There are two benefits:

1. Enable RDMA with PaddlePaddle, which bring high performance low latency networks.
2. Enable GPUDriect with PaddlePaddle, which bring highest throughput and lowest latency GPU read and write.