From 7837e10d6fdcda6d18ad37a6dc0478c845f1d71d Mon Sep 17 00:00:00 2001 From: guru4elephant <35550832+guru4elephant@users.noreply.github.com> Date: Mon, 12 Nov 2018 10:28:25 +0800 Subject: [PATCH] Update async_executor.md --- .../design/async_executor/async_executor.md | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-) diff --git a/doc/fluid/design/async_executor/async_executor.md b/doc/fluid/design/async_executor/async_executor.md index b19581213..6c5b78332 100644 --- a/doc/fluid/design/async_executor/async_executor.md +++ b/doc/fluid/design/async_executor/async_executor.md @@ -42,21 +42,27 @@ for i, (slots, label) in enumerate(train_loader): ``` +## Difference between async_executor and other executors +async_executor is mainly designed for cpu training scenarios where data throughputs are high and the computation part of training is not intensive compared with GPU trained models such as resnet-50. Since data throughputs ability is very important in async_executor, we have to design very fast data IO modules to handle very large scale data reading. Another different key aspect is that memory is not a problem in cpu training scenarios given 128G or 256G RAW in modern server. + +executor and parallel_executor are designed for geneneral training cases in particular for gpu training. Executor is a single thread implementation for model training and it is mostly used for startup_program running currently. Another application scenario of executor is reinforcement learning where input data and main_program may change through training. Parallel_executor is mainly designed for synchronous training on high performance devices such as gpu. Operators are executed concurrently following topological orders on different graphs and model parameter gradients are synchrounized iteratively. ## Data Feeding Approach -TBA +to be discussed. ## Inside Structure of Async Executor -TBA +will be added. ## How to print variable information during execution -TBA +Inside async_executor, no information is printed. Variable can be fetched through an execution of async_executor. The fetched variables can be printed through python. ## How to save models -TBA +Models can be saved between execution of async_executor through io.save method. + + ## references 1. [Sentiment Analysis](https://arxiv.org/pdf/1801.07883.pdf) 2. [Word2Vec](https://arxiv.org/abs/1301.3781) 3. [Click Through Rate Estimation](https://static.googleusercontent.com/media/research.google.com/zh-CN//pubs/archive/45530.pdf) -4. [Hogwild](https://people.eecs.berkeley.edu/~brecht/papers/hogwildTR.pdf) \ No newline at end of file +4. [Hogwild](https://people.eecs.berkeley.edu/~brecht/papers/hogwildTR.pdf) -- GitLab