Design document of refactoring Paddle Paddle implementation.
Created by: reyoung
There are several docs about refactoring Paddle Paddle implementation.
-
readme.md Summarize overview and the user interfaces in C++. How to configure a neural network and train a neural network in C++.
-
memory.md Type of Variable memory. Maintains all variables' memory. How to implement a faster algorithm to allocate/reallocate memory.
-
variable_and_op.md What is metadata of an op? What is an op? How to register an op?
-
kernels.md How to implement each op's computation kernel? Should we have a single tensor library or we could let each op's implementation use a different library? What is the suggestion tensor library that we need to be used? Is there some common method should be shared by each kernel implementation?
-
graph.md Given a list of op, i.e., a graph. How to convert a forward Graph to a backward Graph? How to convert a single device graph to multiple devices? How to optimize a graph, make computing in place? How to plan to recompute a tensor to save memory?
-
engine.md Given a graph, which devices of variable could be different, how to run on each device? How to synchronize each device? How to get each variable's result immediately?
Please [Thunbs up] the comments you want to investigate or discuss.