diff --git a/doc/design/executor.md b/doc/design/executor.md new file mode 100644 index 0000000000000000000000000000000000000000..bf7f05552c98e7b5ecf2861300e9c82b63df63a4 --- /dev/null +++ b/doc/design/executor.md @@ -0,0 +1,95 @@ +# Executor Desgin Doc + +## Overview + +`Executor` evaluates a `ProgramDesc`. Essentially, it instantializes Variables and Operators, then run all the operators + +```c++ +void Executor::Run(const ProgramDesc& pdesc, Scope* scope) { + auto& block = pdesc.blocks(0); + auto& device = device_contexts_[0]; + + // Instantiate all the vars in the global scope + for (auto& var : block.vars()) { + scope->NewVar(var.name()); + } + + // Decide which operator should be run + std::vector should_run = Preprocess(pdesc); + + // Run the block + Scope& local_scope = scope->NewScope(); + for (size_t i = 0; i < should_run.size(); ++i) { + if (should_run[i]) { + for (auto var : block.ops(i).outputs()) { + for (auto argu : var.arguments()) { + // Create variable in the local_scope + if (local_scope.FindVar(argu) == nullptr) { + local_scope.NewVar(argu); + } + } + } + auto op = paddle::framework::OpRegistry::CreateOp(block.ops(i)); + op->Run(local_scope, *device); + } + } +} +``` + +## Tasks + +As shown above, it is not hard to simply evaluate the graph. The real problem +is how do we actually construct the `ProgramDesc`. There are several different +situations that we need to consider. + +### 1. Init @tony @qijun + +##### Problem: + +Not sure which block to put init ops. Same concerns applys to `Load Model`. + +##### Solution: In seperate Blocks + +All `initop` and `parameter` goes to `block[0]`. Actual run starts from `block[1]`. + +When user writes `a = Parameter(Variable, init)`, a init op is inserted into +`block[0]`, and a `NOP` is inserted into `block[1]` to substitute init op. + +- Pro: + - Init Op can be run multiple times. + - Compatiable with current `Executor::Preprocessing` + - Still only one `ProgramDesc` +- Con: + - Let others know! + +### 2. IO + +#### 2.1 FeedOp and FetchOp + +Design Doc: https://github.com/PaddlePaddle/Paddle/pull/4599 + +FeedOp and FetchOp in distributed environment: +https://github.com/PaddlePaddle/Paddle/issues/4613 + +#### 2.2 ReaderOp and WriterOp + +### 3. Backward @jiayi + +Executor test case is a good place to test `backward` module, even though executor +is not necessarily depends on `backward`. Currently exposed issue: + +- Fill One: https://github.com/PaddlePaddle/Paddle/issues/4627 +- Attribute map: https://github.com/PaddlePaddle/Paddle/issues/4642 + +### 4. Optimizer @longfei + +Executor test case is a good place to test `optimizer `module, even though executor +is not necessarily depends on `optimizer `. + +### 5. RNN @chunwei + +To be discussed. + +- How to deal with multiple blocks +- How to deal with LoDTensor +