Created by: reyoung

Fluid supports multi-GPUs and cluster, and high usability

Deadline:

KPI:

1. Make fluid supports all models in PaddlePaddle/Book, PaddlePaddle/models. Complete the inference framework of Fluid on linux and mobile.
   • Make Baidu teams (Speech, NLP, Image, Abacus) use fluid to train and inference models.
2. The speed of training models in PaddlePaddle/book is not slower than TF in MultiGPUs and cluster.
3. The memory consumption of training models in PaddlePaddle/book is not larger than TF in MultiGPUs and a cluster.

Fluid distributed computing

Deadline:

KPI:

1. Make Fluid support EDL (Elastic Deep Learning). Make cluster training of Fluid can adapt to workload changes by provisioning and de-provisioning resources in an autonomic manner.
2. Support model parallelism as well as data parallelism.
3. Make Fluid support OpenMPI APIs to do distributed all-reduce.
4. Make Fluid support GPU direct when possible.

Compatible with ONNX

Deadline:

KPI:

1. Make ProgramDesc can be converted to ONNX model files.
2. Make ONNX can be converted into ProgramDesc, and make Fluid can train ONNX model.

Support CSP program model and imperative programming

Deadline:

KPI:

1. Users only use Python as a compiler frontend and produce the ProgramDesc, and an interpreter will execute the ProgramDesc.
2. The ProgramDesc includes IfElse operator and While, and supports auto diff.
3. Saving and loading model, printing metrics are all configured in ProgramDesc. Deeply integrate with VisualDL to give a GUI.
4. Support to configure CSP(coroutines, channel, select) in ProgramDesc. Use CSP to implement multi GPUs and cluster training.