diff --git a/doc/design/executor.md b/doc/design/executor.md index bf7f05552c98e7b5ecf2861300e9c82b63df63a4..7376ecaef0444a57adddf68c2a1323de30865d9a 100644 --- a/doc/design/executor.md +++ b/doc/design/executor.md @@ -15,7 +15,7 @@ void Executor::Run(const ProgramDesc& pdesc, Scope* scope) { } // Decide which operator should be run - std::vector should_run = Preprocess(pdesc); + std::vector should_run = Prune(pdesc); // Run the block Scope& local_scope = scope->NewScope(); @@ -23,7 +23,7 @@ void Executor::Run(const ProgramDesc& pdesc, Scope* scope) { if (should_run[i]) { for (auto var : block.ops(i).outputs()) { for (auto argu : var.arguments()) { - // Create variable in the local_scope + // Create temp variable in the local_scope if (local_scope.FindVar(argu) == nullptr) { local_scope.NewVar(argu); } @@ -36,60 +36,33 @@ void Executor::Run(const ProgramDesc& pdesc, Scope* scope) { } ``` -## Tasks +## Challenge -As shown above, it is not hard to simply evaluate the graph. The real problem -is how do we actually construct the `ProgramDesc`. There are several different -situations that we need to consider. +It is not hard to simply evaluate a graph. However, it is hard to determine which op should be run. Consider the following different situations. -### 1. Init @tony @qijun - -##### Problem: - -Not sure which block to put init ops. Same concerns applys to `Load Model`. - -##### Solution: In seperate Blocks - -All `initop` and `parameter` goes to `block[0]`. Actual run starts from `block[1]`. - -When user writes `a = Parameter(Variable, init)`, a init op is inserted into -`block[0]`, and a `NOP` is inserted into `block[1]` to substitute init op. - -- Pro: - - Init Op can be run multiple times. - - Compatiable with current `Executor::Preprocessing` - - Still only one `ProgramDesc` -- Con: - - Let others know! - -### 2. IO - -#### 2.1 FeedOp and FetchOp - -Design Doc: https://github.com/PaddlePaddle/Paddle/pull/4599 - -FeedOp and FetchOp in distributed environment: -https://github.com/PaddlePaddle/Paddle/issues/4613 - -#### 2.2 ReaderOp and WriterOp - -### 3. Backward @jiayi - -Executor test case is a good place to test `backward` module, even though executor -is not necessarily depends on `backward`. Currently exposed issue: - -- Fill One: https://github.com/PaddlePaddle/Paddle/issues/4627 -- Attribute map: https://github.com/PaddlePaddle/Paddle/issues/4642 - -### 4. Optimizer @longfei +```python +# Case 1: run foward pass. +cost_np = executor.run(target=cost) +# Case 2: run backward passing. +opts_np, _ = executor.run(target=[cost, opt]) +# Case 3: run checkpointing +_ = executor.run(target=checkpoint) +``` -Executor test case is a good place to test `optimizer `module, even though executor -is not necessarily depends on `optimizer `. +We want to support the evaluation of both variables and operators. -### 5. RNN @chunwei +## Solution -To be discussed. +To support evaluation of operators, we add `is_target` field in the `OpDesc`. -- How to deal with multiple blocks -- How to deal with LoDTensor +```c++ +message OpDesc { + required string type = 3; + repeated Var inputs = 1; + repeated Var outputs = 2; + repeated Attr attrs = 4; + required bool is_target = 5 [ default = false ]; // true if the op is target +}; +``` +To support evaluation of variables, we add [fetch_op](https://github.com/PaddlePaddle/Paddle/pull/4599). For each variable in the `target`, we insert a `fetch_op` into the `ProgramDesc`. (Also, a user may want to overwrite a variable, so we also added [feed_op](https://github.com/PaddlePaddle/Paddle/pull/4599). )