How can we load/produce data into paddle core in our new design
Created by: QiJune
There are mainly two kinds of data Paddle core have to handle, training data and model parameters. And There are commonly two ways to put them into Paddle core. The first one is to load them from a data source(Python Numpy object will be a good choice). And the second one is to produce them in Paddle, which users just set a few attributes to give a data distribution, such as zero/one/constant/rand.
In GPU training process, we have to make copy in loading ways(load to CPU first, and then copy to GPU). But, we want to produce specific data distribution in GPU directly(It's more efficient and copy is no need).
So, an efficient and easy-to-use solution have to be put forward.