Created by: QiJune

Our neural network topology will be serialized to ProgramDesc. And we hope that this ProgramDesc can be transpose to different hardware environments, such as single CPU/GPU, multi-GPUs, or multi-nodes.

However, we should also expose device configuration to users. Following is an example:

with paddle.device("CPU"):
   a = layers.data()
   b = layers.data()
   c = layers.sum(a, b)

with paddle.device("GPU"):
   d = layer.tanh(c)

with paddle.device("FPGA"):
   e = layers.softmax(d)

I think that this feature is important, we should provides such interface for users to set device for specific layers.

Then, there comes a question, where to store these device information. At now, there is no device field in framework.proto.

In tensorflow, there is a device field in node_def.proto. In Mxnet, there is a device attribute of graph to store users's device configuration.

If we want to support such feature(user can set device for specific operator), we have to handle these device configuration. Maybe we should add a device field in framework.proto, or we create another struct to store the device configuration.