FluidDoc
+ +[English](./README.md) | 简体中文 + +# 介绍 + +FluidDoc包含了所有PaddlePaddle相关的文档，它通过CI系统为PaddlePaddle.org提供文档内容 + +# 架构 + +FluidDoc将Paddle, Book, Models, Mobile and Anakin作为子模块，并放置在 `external` 目录下。按照标准做法，所有的子模块应当置于`external` 目录下 + +FluidDoc通过引用这些子模块来加载这些Repo中的文档。FluidDoc在 `FluidDoc/doc/fluid` 目录下构建了文档的整体树形结构。可以分别在 `FluidDoc/doc/fluid/index_cn.rst` 和 `FluidDoc/doc/fluid/index_en.rst` 查看。 + +当一个新发布的分支被push到了Github上，Travis-CI 将会自动启动编译文档并把文档部署到服务器 + +## 注意： +FluidDoc 需要Paddle Repo的python模块去编译生成API文档。但由于Paddle的python模块过于庞大，超过了Travis CI允许的最大时长，通常Travis CI将会因为超时问题失败。这是Travis上有三项作业的原因，其中两项用于构建库。当Travis缓存了这些库以后，下一次的构建将会变得非常的快。 + +## 通过PaddlePaddle.org预览文档 + +为了预览FluidDoc的文档，请按照[常规预览步骤](https://github.com/PaddlePaddle/PaddlePaddle.org/blob/develop/README.md)，但请在这一步将 paddle 的路径替换为 Fluid 的路径 +`./runserver --paddle| Argument | -Optional Values | -Default | -
|---|---|---|
| ANDROID_ABI | -armeabi-v7a, arm64-v8a | -armeabi-v7a | -
| ANDROID_API | ->= 16 | -21 | -
| Argument | -Optional Values | -Default | -
|---|---|---|
| ANDROID_ABI | -armeabi-v7a, arm64-v8a | -armeabi-v7a | -
| ANDROID_API | ->= 16 | -21 | -
| IOS_PLATFORM | -IOS_ARCH | -
|---|---|
| OS | -armv7, armv7s, arm64 | -
| SIMULATOR | -i386, x86_64 | -
| IOS_PLATFORM | -IOS_ARCH | -
|---|---|
| OS | -armv7, armv7s, arm64 | -
| SIMULATOR | -i386, x86_64 | -
-
-By coordinating these processes, PaddlePaddle supports use both Synchronize Stochastic Gradient Descent (sync SGD) and Asynchronous Stochastic Gradient Descent (async SGD) to train user-defined neural network topologies.
-
-When training with sync SGD, parameter servers wait for all trainers to finish gradients update and then send the updated parameters to trainers, training can not proceed until the trainer received the updated parameters. This creates a synchronization point between trainers. When training with async SGD, each trainer upload gradient and download new parameters individually, without the synchronization with other trainers. Using asyc SGD will be faster in terms of time per pass, but have more noise in gradient since trainers are likely to have a stale model.
-
-### Master Server Process
-
-The master server process will:
-
-- Partition a dataset into [tasks](#task) and dispatch tasks to trainers.
-- Keep track of training progress on the dataset with [task queue](#task-queue). A training job will iterate on the dataset for a full pass until it goes into next pass.
-
-
-#### Task
-
-A task is a data shard to be trained. The total number of tasks will be much bigger than the total number of trainers. The number of data instances inside a task will be much bigger than the mini-batch size.
-
-#### Task Queue
-
-The master server has three task queues to track training progress. As illustrated in the graph below, Job A and Job B both have one master server. Each master server process has three task queues.
-
-
-
-- The todo queue holds tasks to be dispatched. When a job starts, the master server fills in the todo queue with all tasks.
-- The pending queue holds tasks that are currently training by trainers.
-- the done queue holds tasks that are already trained.
-
-The life cycle of a single task is illustrated below:
-
-
-
-1. When a new pass of training starts, all tasks will be placed in the todo queue.
-1. Upon trainer requests for new task, the master server will dispatch a task from todo queue to it, put the task in the pending queue and wait for completion.
-1. The trainer will work on its task and tell the master server once the task is completed and ask for new task. The master server will dispatch a new task to that trainer.
-1. If a task fails for any reason in trainer, or takes longer than a specific period of time, the master server will move the task back to the todo queue. The timeout count for that task will increase by one. If the timeout count is above a threshold, the task is likely to cause a trainer to crash, then it will be discarded.
-1. The master server will move completed task to the done queue. When the todo queue is empty, the master server will start a new pass by moving all tasks in the done queue to todo queue and reset the timeout counter of all tasks to zero.
-
-### Trainer Process
-
-The trainer process will:
-
-- Request tasks from the master.
-- Work on the tasks
-- Upload gradient to parameter servers, and update local model by downloading new parameters from parameter servers.
-
-### Parameter Server Process
-
-Parameter server processes hold the parameters collaboratively. The parameters are partitioned on different parameter servers.
-
-The parameter server will:
-
-- Receive gradient from the trainers, update its parameters, and give the trainers the latest parameters.
-- Periodically save its parameters to distributed file system by overriding the previous save.
-
-### Optimization Algorithms
-
-The communication pattern between the trainers and the parameter servers depends on the category of optimization algorithm:
-
-- Synchronous Stochastic Gradient Descent (sync-SGD)
-
- Parameter server will wait for all trainer finish n-th mini-batch calculation and send their gradients before broadcasting new parameters to every trainer. Every trainer will wait for the new parameters before starting n+1-th mini-batch.
-
-- Asynchronous Stochastic Gradient Descent (async-SGD)
-
- There will no synchronization between different trainers, and parameter server updates its parameter as soon as it receives new gradient:
-
- - Each trainer uploads its accumulated gradient every n mini-batches.
- - Every m mini-batches, the trainer downloads new parameters from parameter server.
- - n and m do not have to be equal.
-
-## Fault Tolerant
-
-The training job will pause if the master server processes is dead, or any of the parameter server process is dead. They will be started by [Kubernetes](https://kubernetes.io/) and recover in few minutes. Please refer to [fault recovery](#fault-recovery).
-
-The training job will continue to make progress if there is at least one training process running. The strategy depends on the type of optimization algorithm:
-
-- sync-SGD
-
- TODO
-
-- async-SGD
-
- Since async-SGD does not require synchronization between mini-batches, the system will by definition make process if at least one trainer is running.
-
-## Fault Recovery
-
-PaddlePaddle uses [etcd](https://github.com/coreos/etcd) to keep track of the states of processes. Because etcd is a distributed reliable key-value store, the restarted process can recover its states from etcd. The model parameters are periodically saved into distributed file system, so a restarted parameter server can recover its parameters from the saved file.
-
-Now we will introduce how each process recovers from a failure, the graph below shows how etcd is used:
-
-
-
-### Master Server Process
-
-When the master is started by the Kubernetes, it executes the following steps at startup:
-
-1. Grabs a unique *master* lock in etcd, which prevents concurrent master instantiations.
-1. Recovers the task queues from etcd if they already exist, otherwise, the master will create them.
-1. Write its ip address to */master/addr* so that trainers can discover it.
-1. Listens to trainers' request of task, dispatch one upon request, and updates task queue using an etcd transaction to ensure lock is held during the update.
-
-When the master server process is dead for any reason, Kubernetes will restart it. It will be online again with all states recovered from etcd in few minutes.
-
-### Trainer Process
-
-When the trainer is started by the Kubernetes, it executes the following steps at startup:
-
-1. Watches the available parameter server prefix keys `/ps/` on etcd and waits until the count of parameter servers reaches the desired count */ps_desired*.
-1. Finds and watches */master/addr* to get master's address.
-1. Requests for tasks from the master to start training.
-
-When a trainer fails, Kuberentes would try to restart it. The recovered trainer would fetch tasks from master and go on training.
-
-### Parameter Server Process
-
-When the parameter server is started by Kubernetes, it executes the following steps at startup:
-
-1. Read desired total number of parameter servers from etcd `/ps_desired`
-1. Search through etcd keys `/ps/
-
- The third parameter server joined:
-
- 
-
-1. The parameter server can load parameters if there are already saved parameters in the save path (inferred from its index).
-1. Now the parameter server is ready for the trainers' requests.
-
-If the parameter server's etcd lease expires, the parameter server will kill itself.
-
-
-## Parameter Server Checkpointing
-See [here](./checkpointing.md)
-
-## Store and dispatching trainning data
-See [here](./data_dispatch.md)
-
-
-## Dynamic Scaling
-
-### Trainer Scaling
-
-TODO
-
-### Parameter Server Scaling
-
-Not planned for v1.
-
-## Training Dataset Format
-
-TODO
-
-## User Interface
-
-TODO
diff --git a/doc/v2/design/cluster_train/checkpointing.md b/doc/v2/design/cluster_train/checkpointing.md
deleted file mode 100644
index c87ef2c7d2636208866d05456d5d44316d0bb200..0000000000000000000000000000000000000000
--- a/doc/v2/design/cluster_train/checkpointing.md
+++ /dev/null
@@ -1,44 +0,0 @@
-## 模型参数检查点（Checkpointing）
-模型数据检查点的实现，可以有效的避免parameter server的单点或多点同时故障。模型参数检查点通过定期向磁盘上保存一份存储在parameter server内存中的模型数据的完整镜像，来保证训练过程可以从中间状态重新启动。在一个不可中断并缺少备份的训练任务中，可以通过阶段性的保存每个parameter server的数据快照（snapshot）到 ***分布式存储服务*** 达到容灾的目的，比如每隔10分钟最新的快照，并删除更早的快照。在出现单点故障时，只需要恢复这台节点，或者将这台节点迁移到另一个节点并启动即可恢复训练任务。
-
-
-
-### 快照保存的设计如下：
-
-说明：
-
-* parameter server在集群中启动后，自动挂载分布式存储目录，并把快照保存到这个目录下。
-* ***注：每个parameter server的检查点各自独立保存，暂时不考虑多个parameter server同步的保存一个特定时间点的全局检查点，因为这样做也没法保证消除随机性。***
-
-检查点保存程序流程：
-
-1. 如果满足条件"每隔10分钟"时，parameter server会获取parameters内存的`read_lock`，启动一个新的线程开始保存检查点。如果已经正在执行保存检查点的线程，则忽略。由于对parameters的更新需要获取parameters内存的`write_lock`，所以在写入快照的过程中，parameter server会暂停参数更新并等待。
-2. parameter server生成一个UUID，向指定的目录中一个新的文件（文件名为此UUID）写入快照数据。在快照写入完成后，计算这个文件的MD5 sum。然后在etcd的`/checkpoints/[pserver_id]`中写入json内容：`{"uuid": [UUID], "md5", "MD5 sum", "timestamp": xxxx}`。
-3. 删除磁盘目录中不是当前uuid的快照文件。
-4. 释放对paramters内存的锁定，停止保存检查点的线程。
-
-这里需要用户额外注意，在您的实际环境中，训练任务的运行可能会占满trainer和parameter server之间的网络带宽，如果parameter server此时还需要通过网络访问分布式存储以保存快照，可能会造成网络拥塞，而出现阶段性的运行停滞。
-
-### 从快照恢复
-
-在parameter server第一次启动或任意时间parameter server故障后被Kubernetes重新启动，则需要回滚到上一个检查点：
-
- 1. 从etcd中读取节点：`/checkpoints/[pserver_id]`获取最新的检查点的文件uuid
- 1. 从磁盘文件中加载uuid文件名的检查点快照文件，并加载其中的参数
- 1. 如果上面两步出现错误，则使用启动参数定义的初始化方法初始化参数
- 1. 开始提供服务
-
-## TODO List
-### 推测执行/加速执行（TODO）
-在异构集群中，如果存在某些trainer执行速度过慢会影响整体集群的速度（如图中Trainer 1），此时master将负责启动一个新的Trainer（Accelerate Trainer 2），使用同样的训练数据block。哪个trainer先完成block的训练，则把另一个慢速的kill掉。
-
-### 动态扩容/缩容
-目前只考虑动态扩容trainer数量，可以减小系统复杂性。
-
-## 术语
-* model: 指深度学习训练之后得到的所有参数，使用这个神经网络可以完成对新数据的预测
-* parameters: 神经网络中的参数，包括权重w和偏置b。一个神经网络的模型由大量的参数组成
-* shard: 分片，通常指将一个整体拆分成多份的其中的一份。
-* model shard: 将一个神经网络参数拆分成多份，每个shard分别存储在其中一台parameter server之上
-* parameter block: 多个parameter block构成一个model shard
-* 单点故障: 任意时刻只可能同时有一台服务器故障。由于集群中同时存在两台机器故障的概率极低（（平均故障率*平均故障修复时间）^2）只对特殊在线系统考虑两台以上同时故障的容灾。
diff --git a/doc/v2/design/cluster_train/data_dispatch.md b/doc/v2/design/cluster_train/data_dispatch.md
deleted file mode 100644
index 1f5d22ff5e6abcb576d16cbe7391da1967a1ab8e..0000000000000000000000000000000000000000
--- a/doc/v2/design/cluster_train/data_dispatch.md
+++ /dev/null
@@ -1,160 +0,0 @@
-## 训练数据的存储和分发
-
-### 概念解释
-
-### 流程介绍
-生产环境中的训练数据集通常体积很大，并被存储在诸如Hadoop HDFS，Ceph，AWS S3之类的分布式存储之上。这些分布式存储服务通常会把数据切割成多个分片分布式的存储在多个节点之上。这样就可以在云端执行多种数据类计算任务，包括：
-
-* 数据预处理任务
-* Paddle训练任务
-* 在线模型预测服务
-
-
-
-
-在上图中显示了在一个实际生产环境中的应用（人脸识别）的数据流图。生产环境的日志数据会通过实时流的方式（Kafka）和离线数据的方式（HDFS）存储，并在集群中运行多个分布式数据处理任务，比如流式数据处理（online data process），离线批处理（offline data process）完成数据的预处理，提供给paddle作为训练数据。用户也可以上传labeled data到分布式存储补充训练数据。在paddle之上运行的深度学习训练输出的模型会提供给在线人脸识别的应用使用。
-
-### 训练数据存储
-我们选择[CephFS](http://docs.ceph.com/docs/master/cephfs/)作为存储系统。
-
-- 无论是从[PFSClient](../file_manager/README.md)的角度，还是从[Pod](https://kubernetes.io/docs/concepts/workloads/pods/pod/)中运行任务的角度，统一用`/pfs/$DATACENTER/home/$USER`来访问用户自己的数据。
-- `/pfs/$DATACENTER/common`下存放公共数据集合
- - 做只读挂载
-
-
-
-
-
-
-### 文件预处理
-
-
-在开始训练之前, 数据集需要预先被转换成PaddlePaddle分布式训练使用的存储格[RecordIO](https://github.com/PaddlePaddle/Paddle/issues/1947)。我们提供两个转换方式：
-
-1. 用户在本地转换好再上传
-1. 用户上传数据后，在机群上运行转换程序
-
-转换生成的文件名会是以下格式：
-
-```text
-name_prefix-aaaaa-of-bbbbb
-```
-
-"aaaaa"和"bbbbb"都是五位的数字，每一个文件是数据集的一个shard，"aaaaa"代表shard的index，"bbbbb"代表这个shard的最大index。
-
-比如ImageNet这个数据集可能被分成1000个shard，它们的文件名是：
-```text
-imagenet-00000-of-00999
-imagenet-00001-of-00999
-...
-imagenet-00999-of-00999
-```
-
-#### 转换库
-
-无论是在本地或是云端转换，我们都提供Python的转换库，接口是：
-```python
-def convert(output_path, reader, num_shards, name_prefix)
-```
-
-- `output_path`: directory in which output files will be saved.
-- `reader`: a [data reader](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/reader/README.md#data-reader-interface), from which the convert program will read data instances.
-- `num_shards`: the number of shards that the dataset will be partitioned into.
-- `name_prefix`: the name prefix of generated files.
-
-`reader`每次输出一个data instance，这个instance可以是单个值，或者用tuple表示的多个值：
-
-```python
-yield 1 # 单个值
-yield numpy.random.uniform(-1, 1, size=28*28) # 单个值
-yield numpy.random.uniform(-1, 1, size=28*28), 0 # 多个值
-```
-
-每个值的类型可以是整形、浮点型数据、字符串，或者由它们组成的list，以及numpy.ndarray。如果是其它类型，会被Pickle序列化成字符串。
-
-### 示例程序
-
-#### 使用转换库
-
-以下`reader_creator`生成的`reader`每次输出一个data instance，每个data instance包涵两个值：numpy.ndarray类型的值和整型的值：
-```python
-def reader_creator():
- def reader():
- for i in range(1000):
- yield numpy.random.uniform(-1, 1, size=28*28), 0 # 多个值
- return reader
-```
-
-把`reader_creator`生成的`reader`传入`convert`函数即可完成转换：
-```python
-convert("./", reader_creator(), 100, random_images)
-```
-
-以上命令会在当前目录下生成100个文件：
-```text
-random_images-00000-of-00099
-random_images-00001-of-00099
-...
-random_images-00099-of-00099
-```
-
-#### 进行训练
-
-
-PaddlePaddle提供专用的[data reader creator](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/reader/README.md#python-data-reader-design-doc)，生成给定`RecordIO`文件对应的data reader。**无论在本地还是在云端，reader的使用方式都是一致的**：
-
-```python
-# ...
-reader = paddle.reader.creator.RecordIO("/pfs/datacenter_name/home/user_name/random_images-*-of-*")
-batch_reader = paddle.batch(paddle.dataset.mnist.train(), 128)
-trainer.train(batch_reader, ...)
-```
-
-以上代码的reader输出的data instance与生成数据集时，reader输出的data instance是一模一样的。
-
-### 上传训练文件
-
-使用下面命令，可以把本地的数据上传到存储集群中。
-
-```bash
-paddle pfs cp filename /pfs/$DATACENTER/home/$USER/folder/
-```
-
-比如，把之前示例中转换完毕的random_images数据集上传到云端的`/home/`可以用以下指令：
-
-```bash
-paddle pfs cp random_images-*-of-* /pfs/$DATACENTER/home/$USER/folder/
-```
-
-需要`$DATACENTER`的配置写到配置文件中，例如
-
-```
-# config file
-[datacenter_1]
-username=user
-usercert=user.pem
-userkey=user-key.pem
-endpoint=datacenter1.paddlepaddle.org
-
-[datacenter_2]
-username=user
-usercert=user.pem
-userkey=user-key.pem
-endpoint=datacenter2.paddlepaddle.org
-```
-## TODO
-### 文件访问的权限
-控制用户权限
-
-- 用户可以把自己的数据分享给别人
-
-### 文件访问方式
-不用mount的方式来访问数据，而是直接用API的接口远程访问
-
-例如：
-
-```
-f = open('/pfs/datacenter_name/home/user_name/test1.dat')
-```
-
-
-### 支持用户自定义的数据预处理job
diff --git a/doc/v2/design/cluster_train/large_model_dist_train.md b/doc/v2/design/cluster_train/large_model_dist_train.md
deleted file mode 100644
index edb0245ea083e791b7f32ac57a330698299fceda..0000000000000000000000000000000000000000
--- a/doc/v2/design/cluster_train/large_model_dist_train.md
+++ /dev/null
@@ -1,101 +0,0 @@
-# Alalysis of large model distributed training in Paddle
-
-***NOTE: This is only some note for how we implemeted this scheme in V1, not a new design.***
-
-## What is it
-
-We often encounter cases that the embedding layer parameters(sparse) are so large that we can not store it in the trainer's memory when training. So we need to put them to several servers, and fetch them row by row instead of fetch all of the parameters.
-
-## How to use
-
-Specify command-line argument like `--loadsave_parameters_in_pserver=true --ports_num_for_sparse=1 --use_old_updater=1` when starting the paddle trainer. And also add something like `--ports_num_for_sparse=1 --pserver_num_threads=5` when starting pserver processes.
-
-Accrodingly, configure your embedding layers like:
-
-```python
-SPARSE_REMOTE=True
-
-w1 = data_layer(name="w1", size=dict_size)
-emb1 = embedding_layer(input=w1, size=32, param_attr=ParameterAttribute(sparse_update=SPARSE_REMOTE))
-w2 = data_layer(name="w2", size=dict_size)
-emb2 = embedding_layer(input=w2, size=32, param_attr=ParameterAttribute(sparse_update=SPARSE_REMOTE))
-...
-```
-
-## Implementation details
-
-```c++
-enum MatType {
- MAT_NORMAL,
- MAT_NORMAL_SHARED,
- MAT_VALUE_SHARED,
- MAT_SPARSE_ROW_IDS,
- MAT_SPARSE_ROW_AUTO_GROW,
- MAT_CACHE_ROW,
- MAT_SPARSE_ROW,
- MAT_SPARSE_ROW_PREFETCH,
- MAT_SPARSE_ROW_PREFETCH_FULL_SIZE,
-};
-```
-
-`MAT_SPARSE_ROW_PREFETCH` is what we use when configured to fetch only row of matrix when training.
-
-In `trainer_internal.cpp:L93 trainOneBatch`:
-
-```c++
- if (config_->getOptConfig().use_sparse_remote_updater()) {
- REGISTER_TIMER("prefetch");
- gradientMachine_->prefetch(inArgs);
- parameterUpdater_->getParametersRemote();
- }
-```
-
-When doing actual network forward and backward, at the beginning of each batch, the trainer will try to download one row of data from pserver.
-
-In `legacy/trainer/RemoteParameterUpdater.cpp`: `parameterUpdater_->getParametersRemote();`:
-
-```c++
-if (fullSize) {
- ...
-} else {
-getParams = [&] {
- parameterClient_->getParameterSparse(
- /* recvParameterType= */ PARAMETER_VALUE, sendBackParameterType);
-};
-applyL1 = [](Parameter& para, real decayRate) {
- para.getMat(PARAMETER_VALUE)->applyL1(/*lr=*/1.0f, decayRate);
-};
-}
-```
-
-Calling `parameterClient_->getParameterSparse` will do remote call to pserver's `getParameterSparse`:
-
-```c++
-void ParameterServer2::getParameterSparse(const SendParameterRequest& request,
- std::vector
-
-
-A dataset is a list of files in *RecordIO* format. A RecordIO file consists of chunks, whereas each chunk consists some records.
-
-## Task Queue
-
-As mentioned in [distributed training design doc](./README.md), a *task* is a data shard that the master server assigns to the trainer process to train on. A task consists of one or multiple *chunks* from one or multiple files. The master server maintains *task queues* to track the training progress.
-
-### Task Queue Creation
-
-1. Each trainer will make an RPC call (using Go's [rpc](https://golang.org/pkg/net/rpc/) package) to the master server, telling it the RecordIO files representing the dataset specified by the user. Since every trainer will tell the master server the same dataset, only the first RPC call will be honored.
-
- The RPC interface is:
- ```go
- func (m *RPCServer) ReportDataset(Paths []string, dummy *int) error {
- }
- ```
-1. The master server will scan through each RecordIO file to generate the *chunk index* and know how many chunks does each file have. A chunk can be referenced by the file path and the index of the chunk within the file. The chunk index is in memory data structure that enables fast access to each chunk, and the index of the chunk with the file is an integer start from 0, representing the n-th chunk within the file.
-
- The definition of the chunk is:
- ```go
- type Chunk struct {
- Idx int // index of the chunk within the file
- Path string
- Index recordio.Index // chunk index
- }
- ```
-1. Chunks are grouped into tasks, and tasks are filled into the todo queue. The pending queue and the done queue are initialized with no element.
-
- The definition of the task is:
- ```go
- type Task struct {
- Index int
- Chunks []Chunk
- }
- ```
-
- The elements in the tasks queues is of type `TaskEntry`, containing a timeout counter (described in [task retry logic](#task-retry-logic)), and a task:
- ```go
- type TaskEntry struct {
- NumTimeout int
- Task Task
- }
- ```
-
- The definition of task queues is:
- ```go
- type TaskQueues struct {
- Todo []TaskEntry
- Pending map[int]TaskEntry // map from task index to task entry
- Done []TaskEntry
- }
- ```
-
-### Task Queue Persistence
-
-The task queues need to be persisted on [etcd](https://github.com/coreos/etcd) for fault recovery. Since the task queues only change once a task is completed or timed out, which is not very frequent, we can afford to synchronize with etcd every time the task queues change.
-
-We will serialize the task queues data structure with [gob encoding](https://golang.org/pkg/encoding/gob/), compress with gzip, and save into etcd synchronously under key `/task_queues`.
-
-### Task Dispatch
-
-The trainer will make an RPC call to master to get a new task when:
-
-- the trainer first started, or
-- the trainer finishes a task.
-
-The RPC interface is:
-```go
-func (m *RPCServer) GetTask(finished *Task, result *Task) error {
-}
-```
-Argument `finished` will be `nil` when the trainer is just started.
-
-During the RPC call the master will do the following:
-
-- Make a copy of the task queues, and update the copy reflecting the finished tasks and the new pending tasks.
-- Synchronize the copy of task queues with etcd using a transaction conditioned on holding the master lock.
-- Replace the task queues with the copy and report to the trainer with the new tasks if succeeded, or discard the copy and report the error to the trainer if failed.
-
-### Task Retry Logic
-
-When a task is dispatched to the trainer, the master will schedule a function for execution after the timeout duration (based on the moving average of task completion time). If the task entry in still in the pending queue, its timeout counter will increase by one, and the task will be moved to todo queue. If the timeout counter is above the threshold, the master will log the error and discard the task.
-
-Please note that since a timed out task could be completed after it has been dispatched for retry, so it is possible for a task to be processed multiple times. We do not try to prevent it from happening since it's fine to train on the same task multiple times due to the stochastic nature of the stochastic gradient decent algorithm.
diff --git a/doc/v2/design/cluster_train/pserver_client.md b/doc/v2/design/cluster_train/pserver_client.md
deleted file mode 100644
index 474b8c572cd92fc87e9f7f3f2b19d12cccd158de..0000000000000000000000000000000000000000
--- a/doc/v2/design/cluster_train/pserver_client.md
+++ /dev/null
@@ -1,171 +0,0 @@
-# Design Doc: The Client Library of Parameter Server
-
-For an overview of trainer's role, please refer to [distributed training design doc](README.md). In this design doc, we will discuss the parameter server's client library, which will manage communication with parameter servers. The library will be implemented in [Go](https://golang.org/) and made available as a static or dynamic library with a C header file.
-
-## Parameter Partition
-
-Each parameter will be partitioned into parameter blocks to make the parameters evenly distributed on parameter servers. The partition is done automatically by the client library. The *sparse parameter* require a little different treatment:
-
-### Sparse Parameter
-
-The sparse parameter is a parameter that is updated sparsely. The name is somewhat misleading, it does not have a sparse representation, it has the same representation as a dense vector.
-
-Because a sparse parameter is updated sparsely, the trainer will have to partition the sparse parameter. Because the parameter server will merge all sparse parameter shard into the same file when saving the parameter. It needs special naming convention:
-
-If a sparse parameter is partitioned into n shards, they should be named as:
-
-```text
-name:sparse-0
-name:sparse-1
-...
-name:sparse-n-1
-```
-
-The library is unaware of the partition, and treat each parameter independently. Only when saving parameters, the parameter servers will merge the sparse parameters according to the naming convention.
-
-## Model Optimization Using Gradients
-
-There are two ways to perform model optimization using gradients:
-
-- On Client
-
- The client does multiple steps of forward and backward update. In each step, the gradients are calculated and a new model is generated. After some steps, the client will calculate the difference between the newest model and the old model at step 0. The difference will be updated to parameter servers. Parameter servers will just update parameters using the difference without any optimization using gradients (such as Adam and L1 regularization).
-
-- On Parameter Server
-
- The client will send accumulated gradients to parameter servers, the parameter server will do the optimization using gradients.
-
-## L1 and L2 Regularization
-
-PaddlePaddle allows L1 or L2 regularizations to be specified per parameter, so when the trainer initializes the parameter it needs include a parameter configuration when L1 or L2 regularization is necessary.
-
-## Parameter Initialization
-
-The parameters on parameter servers need to be initialized. To provide maximum flexibility, the trainer will initialize the parameters. Only one trainer will do the initialization, the other trainers will wait for the completion of initialization and get the parameters from the parameter servers.
-
-### Trainer Selection
-
-To select the trainer for initialization, every trainer will try to get a distributed lock, whoever owns the lock will do the initialization. As illustrated below:
-
-
-
-### Trainer Selection Process
-
-The trainer select process is encapsulated in the C API function:
-```c
-int paddle_begin_init_params(paddle_pserver_client* client, const char* config_proto);
-```
-The selected trainer's call to `paddle_begin_init_params` will return with 1, and the other trainers' call to `paddle_begin_init_params` will return 0. `paddle_get_params` will be blocked until initialization is completed. As illustrated below:
-
-
-
-## C Interface
-
-```c
-typedef enum {
- PADDLE_ELEMENT_TYPE_INT32 = 0,
- PADDLE_ELEMENT_TYPE_UINT32 = 1,
- PADDLE_ELEMENT_TYPE_INT64 = 2,
- PADDLE_ELEMENT_TYPE_UINT64 = 3,
- PADDLE_ELEMENT_TYPE_FLOAT32 = 4,
- PADDLE_ELEMENT_TYPE_FLOAT64 = 5,
-} paddle_element_type;
-
-typedef struct {
- char* name;
- paddle_element_type element_type;
- unsigned char* content;
- int content_len;
-} paddle_parameter, paddle_gradient;
-
-typedef int paddle_pserver_client;
-
-/**
- * @brief creates a pserver client that talks to etcd for coordination.
- */
-paddle_pserver_client paddle_new_etcd_pserver_client(char* etcd_addr);
-
-/**
- * @brief creates a pserver client given pserver addresses.
- *
- * @param pserver_addrs comma-separated pserver addresses.
- * @param selected if current pserver client is selected to initialize all parameter servers.
- */
-paddle_pserver_client paddle_new_pserver_client(char* pserver_addrs, int selected);
-void paddle_pserver_client_release(paddle_pserver_client c);
-
-/**
- * @brief paddle_begin_init_params begins to initialize parameters on
- * parameter servers.
- *
- * paddle_begin_init_params will be called from multiple trainers,
- * only one trainer will be selected to initialize the parameters on
- * parameter servers. Other trainers need to get the initialized
- * parameters from parameter servers using @paddle_get_params.
- *
- * @return 1 if the trainer is selected to initialize parameter
- * servers, otherwise 0.
- */
-int paddle_begin_init_params(paddle_pserver_client client);
-
-/**
- * @brief paddle_init_param initializes the parameter on parameter
- * servers.
- *
- * @param param the parameter to initialize.
- * @param param_config_proto the configuration for the parameter.
- * @param config_len the length of param_config_proto
- * @return 0 if successful, otherwise -1. On failure, the trainer
- * needs to restart the entire initialization process (starting from
- * @paddle_begin_init_param). Or simply exit the program and wait for
- * the cluster management system to restart the trainer.
- */
-int paddle_init_param(paddle_pserver_client client, paddle_parameter param, const unsigned char* param_config_proto, int config_len);
-
-/**
- * @brief paddle_finish_init_params tells parameter servers client has
- * sent all parameters to parameter servers as initialization.
- *
- * @return 0 if successful, otherwise -1. On failure, the trainer
- * needs to restart the entire initialization process (starting from
- * @paddle_begin_init_param). Or simply exit the program and wait for
- * the cluster management system to restart the trainer.
- */
-int paddle_finish_init_params(paddle_pserver_client client);
-
-/**
- * @brief paddle_send_grads sends gradients to parameter servers for
- * updating parameters.
- *
- * @param grads the array of gradients to send.
- * @param len the length of the gradient array.
- * @param learning_rate the learning rate for the gradients.
- * @return 0 if successful, otherwise -1.
- */
-int paddle_send_grads(paddle_pserver_client client, const paddle_gradient* grads, int len);
-
-/**
- * @brief paddle_get_params gets parameters from parameter servers.
- *
- * paddle_get_params will block until parameters are initialized on
- * the parameter servers.
- *
- * @param dst the destination array of parameter pointers to save to.
- * The parameter pointer must be pre-popullated with required parameter name,
- * and the content of parameter must be pre-allocated of the size of required
- * parameter on pserver.
- * @param len the length of the names array and the paddle_parameter
- * array.
- * @return 0 if successful, otherwise -1.
- */
-int paddle_get_params(paddle_pserver_client client, paddle_parameter** dst, int len);
-
-/**
- * @brief paddle_save_model indicates parameters to save the parameter
- * to the given path
- *
- * @param path the path to save parameters.
- * @return 0 if successful, otherwise -1.
- */
-int paddle_save_model(paddle_pserver_client client, const char* path);
-```
diff --git a/doc/v2/design/cluster_train/remote_parameter_updater.md b/doc/v2/design/cluster_train/remote_parameter_updater.md
deleted file mode 100644
index 6e8e5938455b869e0f3367794c41250340b37f77..0000000000000000000000000000000000000000
--- a/doc/v2/design/cluster_train/remote_parameter_updater.md
+++ /dev/null
@@ -1,21 +0,0 @@
-# Design Doc: Remote Parameter Updater for Cluster Train
-
-For an overview of distribute training, please refer to [distributed training design doc](README.md). In this design doc, we will discuss the parameter updater that will use parameter server cclient [The Client Library of Parameter Server Design Doc](pserver_client.md) to manage and update parameters.
-
-## Parameter Updater
-
-Parameter Updater is used by trainer to manage and update parameter, there are mainly two kind of parameter updater: local and remote, since this design is for cluster train, we will only discuss remote parameter updater here.
-
-### Remote Parameter Updater
-
-Remote Parameter Updater manage parameters through remote parameter server with the client that communicate with pserver([The Client Library of Parameter Server Design Doc](pserver_client.md))
-
-In PaddlePaddle Python V2 API, trainer is implemented in python, and the trainer will hold a instance of parameter updater and call it's functions directly. In this design, we will also expose the api of RemoteParameterUpdater to python with swig.
-
-#### Sparse Remote Parameter Updater
-
-Since we will only implement dense parameter management new, the mechanism for sparse parameter will be discussed in next stage.
-
-### Interface Design
-
-TBD
diff --git a/doc/v2/design/cluster_train/save_model.md b/doc/v2/design/cluster_train/save_model.md
deleted file mode 100644
index b755185c81ad617b9c85c47de0f5f65d2201c658..0000000000000000000000000000000000000000
--- a/doc/v2/design/cluster_train/save_model.md
+++ /dev/null
@@ -1,111 +0,0 @@
-# Design Doc: Save Model
-
-## Overview
-
-The model is the output of the training process. There are two
-ways from which user can obtain a model:
-
-- Save model triggered by user code: user code asks PaddlePaddle to
- save a model.
-- Convert model from the checkpoint: model being converted from
- pservers' periodic checkpoint. In this way, the user can cancel a
- job at any time, and still have a relatively fresh model (we
- checkpoint around every 5 minutes).
-
-### Trainer Saving Model vs. Pservers Saving Model
-
-Both trainers and pservers have access to the model. So the model can
-be saved from a trainer or pservers. We need to decide where the model
-is saved from.
-
-#### Dense Update vs. Sparse Update
-
-There are two types of model update methods: dense update and sparse
-update (when the model parameter is configured to be sparse).
-
-- Dense update
-
- Every trainer has it's own full copy of the model. Every model
- update will update the entire model.
-
-- Sparse update
-
- The training input is sparse, and the trainer does not have the
- entire model. It will only download the sub-model necessary related
- to the input. When updating the model, only the sub-model related to
- the training input is updated.
-
-
-#### Pservers Saving Model
-
-The benefit of letting pservers save model is they have the entire
-model all the time. However, since pservers are on different nodes, it
-requires a merging process to merge model shards into the same
-model. Thus requires the pservers to write models to a distributed
-filesystem, making the checkpoint shards visible to the merge program.
-
-#### Trainer Saving Model
-
-The benefit of letting one trainer to save the model is it does not
-require a distributed filesystem. And it's reusing the same save model
-logic when training locally - except when doing sparse update, the
-trainer needs to download the entire model during the saving process.
-
-#### Conclusion
-
-Given trainer saving model does not require a distributed filesystem,
-and is an intuitive extension to trainer saving model when training
-locally, we decide to let the trainer save the model when doing
-distributed training.
-
-
-### Convert Model from Checkpoint
-
-TODO
-
-
-## Timeline
-
-We first implement trainer save the model. Converting the latest
-snapshot to a model will be a TODO for future.
-
-
-## Trainer Save Model
-
-### Trainer Election
-
-One trainer will be elected as the one to save the model. When using
-etcd, trainer ID is a randomly generated UUID, the trainer will
-contact the master server requesting to save the model, and find out
-if itself is elected. When the master server is not used, unique
-trainer IDs will be given by the administrator, the trainer whose ID
-is "0" is elected to save the model.
-
-### Model Save Path
-
-Each trainer will be given the directory to save the model. The
-elected trainer will save the model to
-`given-directory/trainerID`. Since the trainer ID is unique, this
-would prevent concurrent save to the same file when multiple trainers
-are elected to save the model when split-brain problem happens.
-
-### What Happens When Model Is Saving
-
-It takes some time to save model, we need to define what will happen
-when save model is taking place.
-
-When doing dense update, the trainer uses the local model. Pservers
-does not need to pause model update.
-
-When doing sparse update. The trainer needs to download the entire
-model while saving. To get the most accurate model, the model update
-needs to be paused before the download starts and resumed after the
-download finishes. Otherwise, the trainer gets a model that is
-"polluted": some part of the model is old, some part of the model is
-new.
-
-It's unclear that the "polluted" model will be inferior due to the
-stochastic nature of deep learning, and pausing the model update will
-add more complexity to the system. Since supporting sparse update is a
-TODO item. We defer the evaluation of pause the model update or not
-during saving model to the future.
diff --git a/doc/v2/design/cluster_train/src/checkpointing.png b/doc/v2/design/cluster_train/src/checkpointing.png
deleted file mode 100644
index c221e8474f90f37e31416cbb19c9452207a0d14c..0000000000000000000000000000000000000000
Binary files a/doc/v2/design/cluster_train/src/checkpointing.png and /dev/null differ
diff --git a/doc/v2/design/cluster_train/src/data_dispatch.png b/doc/v2/design/cluster_train/src/data_dispatch.png
deleted file mode 100644
index 5bdcc24d6a6d193cb014f8c38b362451fded5e54..0000000000000000000000000000000000000000
Binary files a/doc/v2/design/cluster_train/src/data_dispatch.png and /dev/null differ
diff --git a/doc/v2/design/cluster_train/src/dataset.graffle b/doc/v2/design/cluster_train/src/dataset.graffle
deleted file mode 100644
index c10a423ed16a23229a9ee33d11bfc82bb59646c8..0000000000000000000000000000000000000000
Binary files a/doc/v2/design/cluster_train/src/dataset.graffle and /dev/null differ
diff --git a/doc/v2/design/cluster_train/src/dataset.png b/doc/v2/design/cluster_train/src/dataset.png
deleted file mode 100644
index 2fb7f1cce3b6dd21489392557826e95a9f207c34..0000000000000000000000000000000000000000
Binary files a/doc/v2/design/cluster_train/src/dataset.png and /dev/null differ
diff --git a/doc/v2/design/cluster_train/src/file_storage.graffle b/doc/v2/design/cluster_train/src/file_storage.graffle
deleted file mode 100644
index 50a17e70fa255495337c529a3bf12a5c0024a5be..0000000000000000000000000000000000000000
Binary files a/doc/v2/design/cluster_train/src/file_storage.graffle and /dev/null differ
diff --git a/doc/v2/design/cluster_train/src/file_storage.png b/doc/v2/design/cluster_train/src/file_storage.png
deleted file mode 100644
index fccb4e3e7e738224c7f1584326bd5f351ce799aa..0000000000000000000000000000000000000000
Binary files a/doc/v2/design/cluster_train/src/file_storage.png and /dev/null differ
diff --git a/doc/v2/design/cluster_train/src/init_lock.graffle b/doc/v2/design/cluster_train/src/init_lock.graffle
deleted file mode 100644
index fa9149f21b1311eed48ef72ec55e556559d0fc94..0000000000000000000000000000000000000000
Binary files a/doc/v2/design/cluster_train/src/init_lock.graffle and /dev/null differ
diff --git a/doc/v2/design/cluster_train/src/init_lock.png b/doc/v2/design/cluster_train/src/init_lock.png
deleted file mode 100644
index 92404ee6d6c0f9a7727952bae3c869ba338ecd7f..0000000000000000000000000000000000000000
Binary files a/doc/v2/design/cluster_train/src/init_lock.png and /dev/null differ
diff --git a/doc/v2/design/cluster_train/src/paddle-cloud-in-data-center.png b/doc/v2/design/cluster_train/src/paddle-cloud-in-data-center.png
deleted file mode 100644
index da5d1a77562480ad1d886f5f21dbd84001d3d508..0000000000000000000000000000000000000000
Binary files a/doc/v2/design/cluster_train/src/paddle-cloud-in-data-center.png and /dev/null differ
diff --git a/doc/v2/design/cluster_train/src/paddle-etcd.graffle b/doc/v2/design/cluster_train/src/paddle-etcd.graffle
deleted file mode 100644
index f973dc9b9dbf72e9bc31e2d32822916cd281f8d9..0000000000000000000000000000000000000000
Binary files a/doc/v2/design/cluster_train/src/paddle-etcd.graffle and /dev/null differ
diff --git a/doc/v2/design/cluster_train/src/paddle-etcd.png b/doc/v2/design/cluster_train/src/paddle-etcd.png
deleted file mode 100644
index 57981ceb4b94f0f7d6dfa63f3d28c0402bf9cc31..0000000000000000000000000000000000000000
Binary files a/doc/v2/design/cluster_train/src/paddle-etcd.png and /dev/null differ
diff --git a/doc/v2/design/cluster_train/src/paddle-model-sharding.graffle b/doc/v2/design/cluster_train/src/paddle-model-sharding.graffle
deleted file mode 100644
index fba30f0ca2b47f0d202a432821d95e55aac37ec8..0000000000000000000000000000000000000000
Binary files a/doc/v2/design/cluster_train/src/paddle-model-sharding.graffle and /dev/null differ
diff --git a/doc/v2/design/cluster_train/src/paddle-model-sharding.png b/doc/v2/design/cluster_train/src/paddle-model-sharding.png
deleted file mode 100644
index 8c3f6724ef46c6527e63a4cd8cb0b50fe0167124..0000000000000000000000000000000000000000
Binary files a/doc/v2/design/cluster_train/src/paddle-model-sharding.png and /dev/null differ
diff --git a/doc/v2/design/cluster_train/src/paddle-ps-0.png b/doc/v2/design/cluster_train/src/paddle-ps-0.png
deleted file mode 100644
index 47ef32806f182cab003da77f1556823b3f6d1721..0000000000000000000000000000000000000000
Binary files a/doc/v2/design/cluster_train/src/paddle-ps-0.png and /dev/null differ
diff --git a/doc/v2/design/cluster_train/src/paddle-ps-1.png b/doc/v2/design/cluster_train/src/paddle-ps-1.png
deleted file mode 100644
index f3125db73096c52bac6e7c60e1675552857c0774..0000000000000000000000000000000000000000
Binary files a/doc/v2/design/cluster_train/src/paddle-ps-1.png and /dev/null differ
diff --git a/doc/v2/design/cluster_train/src/paddle-ps.graffle b/doc/v2/design/cluster_train/src/paddle-ps.graffle
deleted file mode 100644
index 0e536ffdd91cd696008b4c01bad3cb53edebdc16..0000000000000000000000000000000000000000
Binary files a/doc/v2/design/cluster_train/src/paddle-ps.graffle and /dev/null differ
diff --git a/doc/v2/design/cluster_train/src/paddle-task-queues.graffle b/doc/v2/design/cluster_train/src/paddle-task-queues.graffle
deleted file mode 100644
index 4263ed8bfd2ef0e55058828bf23f2fac3595e5fd..0000000000000000000000000000000000000000
Binary files a/doc/v2/design/cluster_train/src/paddle-task-queues.graffle and /dev/null differ
diff --git a/doc/v2/design/cluster_train/src/paddle-task-queues.png b/doc/v2/design/cluster_train/src/paddle-task-queues.png
deleted file mode 100644
index 5f980266795776752cebd0c346b85c4a75a47780..0000000000000000000000000000000000000000
Binary files a/doc/v2/design/cluster_train/src/paddle-task-queues.png and /dev/null differ
diff --git a/doc/v2/design/cluster_train/src/paddle-task-states.graffle b/doc/v2/design/cluster_train/src/paddle-task-states.graffle
deleted file mode 100644
index cf1a0b9246d9386a949d2dbb8c32fe84f72eea83..0000000000000000000000000000000000000000
Binary files a/doc/v2/design/cluster_train/src/paddle-task-states.graffle and /dev/null differ
diff --git a/doc/v2/design/cluster_train/src/paddle-task-states.png b/doc/v2/design/cluster_train/src/paddle-task-states.png
deleted file mode 100644
index 4ae43cb66c071aee9eb90d875e2373b29af9c3e0..0000000000000000000000000000000000000000
Binary files a/doc/v2/design/cluster_train/src/paddle-task-states.png and /dev/null differ
diff --git a/doc/v2/design/cluster_train/src/pserver_init.graffle b/doc/v2/design/cluster_train/src/pserver_init.graffle
deleted file mode 100644
index 5f3f1f52be8aa7f9049a8fcd6b7c93c8560c1676..0000000000000000000000000000000000000000
Binary files a/doc/v2/design/cluster_train/src/pserver_init.graffle and /dev/null differ
diff --git a/doc/v2/design/cluster_train/src/pserver_init.png b/doc/v2/design/cluster_train/src/pserver_init.png
deleted file mode 100644
index dfe491ff98dd7db1c336093c80964a260df2cd90..0000000000000000000000000000000000000000
Binary files a/doc/v2/design/cluster_train/src/pserver_init.png and /dev/null differ
diff --git a/doc/v2/design/cluster_train/src/submit-job.graffle b/doc/v2/design/cluster_train/src/submit-job.graffle
deleted file mode 100644
index 677cdfb6d9a32168bf71729eb841fa1ca0dd31d6..0000000000000000000000000000000000000000
Binary files a/doc/v2/design/cluster_train/src/submit-job.graffle and /dev/null differ
diff --git a/doc/v2/design/cluster_train/src/submit-job.png b/doc/v2/design/cluster_train/src/submit-job.png
deleted file mode 100644
index 3046a460a7ba708079e88a560debaa215a694680..0000000000000000000000000000000000000000
Binary files a/doc/v2/design/cluster_train/src/submit-job.png and /dev/null differ
diff --git a/doc/v2/design/cluster_train/src/trainer.graffle b/doc/v2/design/cluster_train/src/trainer.graffle
deleted file mode 100644
index 43415ed8cf61a5acfa34f8e56b9577f338dbf254..0000000000000000000000000000000000000000
Binary files a/doc/v2/design/cluster_train/src/trainer.graffle and /dev/null differ
diff --git a/doc/v2/design/cluster_train/src/trainer.png b/doc/v2/design/cluster_train/src/trainer.png
deleted file mode 100644
index 6537d3d56589ca9f19a77a50a970e4b5275e6ce0..0000000000000000000000000000000000000000
Binary files a/doc/v2/design/cluster_train/src/trainer.png and /dev/null differ
diff --git a/doc/v2/design/cluster_train/submit-job.md b/doc/v2/design/cluster_train/submit-job.md
deleted file mode 100644
index 8377d5489dc64bd2fdc5bb4f7bc737e7b489000d..0000000000000000000000000000000000000000
--- a/doc/v2/design/cluster_train/submit-job.md
+++ /dev/null
@@ -1,127 +0,0 @@
-# Submit a Distributed Training Job
-
-The user can submit a distributed training job with Python code, rather than with a command-line interface.
-
-## Runtime Environment On Kubernetes
-
-For a distributed training job, there is two Docker image called *runtime Docker image* and *base Docker image*. The runtime Docker image is the Docker image that gets scheduled by Kubernetes to run during training. The base Docker image is for building the runtime Docker image.
-
-### Base Docker Image
-
-Usually, the base Docker image is PaddlePaddle product Docker image including paddle binary files and python package. And of course, users can specify any image name hosted on any docker registry which users have the access right.
-
-### Runtime Docker Image
-
-The trainer package which user upload and some Python dependencies are packaged into a runtime Docker image based on base Docker image.
-
-- Handle Python Dependencies
-
- You need to provide requirements.txt file in your `trainer-package` folder. Example:
-
- ```txt
- pillow
- protobuf==3.1.0
- ```
- More [details](https://pip.readthedocs.io/en/1.1/requirements.html) about requirements, an example project looks like:
- ```bash
- paddle_example
- |-quick_start
- |-trainer.py
- |-dataset.py
- |-requirements.txt
- ```
-
-## Submit Distributed Training Job With Python Code
-
-
-- `paddle.job.dist_train()` will call the Job Server API `/v1/packages` to upload the trainer package and save them on CephFS, and then call `/v1/trainer/job` to submit the PaddlePaddle distributed job.
-- `/v1/trainer/job` will start a building job for preparing the runtime Docker image. When the building job is finished, Job Server will submit the PaddlePaddle distributed job to Kubernetes.
-- *NOTE*: For the first version, we will not prepare the runtime Docker image, instead, the package is uploaded to Paddle Cloud, and Paddle Cloud will mount the package in a temporary folder into the base Docker image. We will not support custom Python dependencies in the first version as well.
-
-You can call `paddle.job.dist_train` and provide distributed training configuration as the parameters:
-```python
-paddle.job.dist_train(
- trainer=dist_trainer(),
- paddle_job=PaddleJob(
- job_name = "paddle-cloud",
- entry_point = "python %s"%__file__,
- trainer_package = "/example/word2vec",
- image = "yancey1989/paddle-job",
- trainers = 10,
- pservers = 3,
- trainer_cpu = 1,
- trainer_gpu = 1,
- trainer_mem = "10G",
- pserver_cpu = 1,
- pserver_mem = "2G"
- ))
-```
-
-The parameter `trainer` of `paddle.job.dist_train` is a function and you can implement it as follows:
-```python
-def dist_trainer():
- def trainer_creator():
- trainer = paddle.v2.trainer.SGD(...)
- trainer.train(...)
- return trainer_creator
-```
-
-The pseudo code of `paddle.job.dist_train` is as follows:
-```python
-def dist_train(trainer, paddle_job):
- # if the code is running on cloud, set PADDLE_ON_CLOUD=YES
- if os.getenv("RUNNING_ON_CLOUD", "NO") == "NO":
- #submit the paddle job
- paddle_job.submit()
- else:
- #start the training
- trainer()
-```
-### PaddleJob Parameters
-parameter | type | explanation
- --- | --- | ---
-job_name | str | the unique name for the training job
-entry_point | str | entry point for startup trainer process
-trainer_package | str | trainer package file path which user have the access right
-image|str|the [base image](#base-docker-image) for building the [runtime image](#runtime-docker-image)
-pservers|int| Parameter Server process count
-trainers|int| Trainer process count
-pserver_cpu|int| CPU count for each Parameter Server process
-pserver_mem|str| memory allocated for each Parameter Server process, a plain integer using one of these suffixes: E, P, T, G, M, K
-trainer_cpu|int| CPU count for each Trainer process
-trainer_mem|str| memory allocated for each Trainer process, a plain integer using one of these suffixes: E, P, T, G, M, K
-trainer_gpu|int| GPU count for each Trainer process, if you only want CPU, do not set this parameter
-
-### Deploy Parameter Server, Trainer and Master Process
- - Deploy PaddlePaddle Parameter Server processes, it's a Kubernetes ReplicaSet.
- - Deploy PaddlePaddle Trainer processes, it's a Kubernetes Job.
- - Deploy PaddlePaddle Master processes, it's a Kubernetes ReplicaSet.
-
-## Job Server
-
-- RESTful API
-
- Job server provides RESTful HTTP API for receiving the trainer package and displaying
- PaddlePaddle job related informations.
- - `POST /v1/package` receive the trainer package and save them on CephFS
- - `POST /v1/trainer/job` submit a trainer job
- - `GET /v1/jobs/` list all jobs
- - `GET /v1/jobs/
-
-Figure 1. PaddlePaddle on IA -
-
-近期目标
-
-- 完成常用Layer的MKL-DNN实现。
-- 完成常见深度神经网络VGG，GoogLeNet 和 ResNet的MKL-DNN实现。
-
-目前的优化，主要针对PaddlePaddle在重构之前的代码框架以及V1的API。
-具体的完成状态可以参见[这里](https://github.com/PaddlePaddle/Paddle/projects/21)。
-
-## Contents
-
-- [Overview](#overview)
-- [Actions](#actions)
- - [CMake](#cmake)
- - [Matrix](#matrix)
- - [Layers](#layers)
- - [Activations](#activations)
- - [Parameters](#parameters)
- - [Gradients](#gradients)
- - [Unit Tests](#unit-tests)
- - [Python API](#python-api)
- - [Benchmarking](#benchmarking)
- - [Others](#others)
-- [Design Concerns](#design-concerns)
-
-## Overview
-
-我们会把MKL-DNN会作为第三方库集成进PaddlePaddle，与其他第三方库一样，会在编译PaddlePaddle的时候下载并编译MKL-DNN。
-
-同时，为了进一步提升PaddlePaddle在基本数学运算的计算速度，我们也将MKLML即(MKL small library\[[1](#references)\])
-作为另一个第三方库集成进PaddlePaddle，它只会包括生成好的动态库和头文件。
-
-MKL，MKLML以及MKL-DNN三者关系如下表：
-
--Figure 1. PaddlePaddle on IA -
| Name | -Open Source | -License | -Descriptions | -
|---|---|---|---|
| MKL | -No | -Proprietary | -Accelerate math processing routines | -
| MKLML | -No | -Proprietary | -Small package of MKL, especially for Machine Learning | -
| MKL-DNN | -Yes | -Apache 2.0 | -Accelerate primitives processing routines especially for Deep Neural Networks | -
-
-Figure 2. PaddlePaddle with MKL Engines -
-
-## Actions
-
-添加的相关文件和目录结构如下：
-
-```txt
-PaddlePaddle/Paddle
-├── ...
-├── cmake/
-│ ├── external/
-│ │ ├── ...
-│ │ ├── mkldnn.cmake
-│ │ └── mklml.cmake
-└── paddle/
- ├── ...
- ├── math/
- │ ├── ...
- │ └── MKLDNNMatrix.*
- └── gserver/
- ├── ...
- ├── layers/
- │ ├── ...
- │ └── MKLDNN*Layer.*
- ├── activations/
- │ ├── ...
- │ └── MKLDNNActivations.*
- └── tests/
- ├── ...
- ├── MKLDNNTester.*
- └── test_MKLDNN.cpp
-```
-
-### CMake
-在`CMakeLists.txt`中提供一个与MKL有关的总开关：`WITH_MKL`，它负责决定编译时是否使用MKLML和MKL-DNN
-
-- `WITH_MKLML` 控制是否使用MKLML库。
-当打开`WITH_MKL`时，会自动使用MKLML库作为PaddlePaddle的CBLAS和LAPACK库，同时会开启Intel OpenMP用于提高MKLML的性能。
-编译时会把对应的头文件和库放在`build/third_party/install/mklml/*`目录下对应的地方。
-MKLML的库目前都是动态库，主要包括`libiomp5.so`和`libmklml_intel.so`。
-- `WITH_MKLDNN` 控制是否使用MKL-DNN。
-当开启`WITH_MKL`时，会自动根据硬件配置[[2](#references)]选择是否编译MKL-DNN。
-编译时会把对应的头文件和库放在`build/third_party/install/mkldnn/*`目录下对应的地方。
-MKL-DNN的库目前只有动态库`libmkldnn.so`。
-
-### Matrix
-目前在PaddlePaddle中数据都是以`NCHW`的格式存储，但是在MKL-DNN中的排列方式不止这一种。
-所以我们定义了一个`MKLDNNMatrix`用于管理MKL-DNN数据的不同格式以及相互之间的转换。
-
--Figure 2. PaddlePaddle with MKL Engines -
-
-Figure 3. MKLDNNMatrix -
-
-### Layers
-所有MKL-DNN的Layers都会继承于`MKLDNNLayer`，该类继承于PaddlePaddle的基类`Layer`。
-在`MKLDNNLayer`中会提供一些必要的接口和函数，并且会写好`forward`和`backward`的基本逻辑，
-子类只需要使用定义好的接口，实现具体的函数功能即可。
-
--Figure 3. MKLDNNMatrix -
-
-Figure 4. MKLDNNLayer -
-
-每个MKLDNNLayer都包含用于内部存储和外部存储的一系列MKLDNNMatrix：
-
-- 内部存储（internel memory）：`inVal_`,`inGrad_`,`outVal_`和`outGrad_`，分别代表输入数据，输入梯度，输出数据和输出梯度。
-- 外部存储（external memory）：都是以ext开头，比如`extInVal_`和`extInGrad_`，它们主要是用于，
-当数据格式与PaddlePaddle默认的`NCHW`格式不匹配时，转换内存的工作。
-需要注意的是，PaddlePaddle的activation会直接使用`output_.value`和`output_.grad`，
-所以`extOutVal_`和`extOutGrad_`必须分别与`output_.value`和`output_.grad`共享内存，
-如果不需要外部存储用于转换，那么对应的内部存储也会与它们共享内存。
-- 转换函数（resetXXX）： 包括`resetInValue`，`resetInGrad`，`resetOutValue`和`resetOutGrad`，
-表示对输入数据，输入梯度，输出数据和输出梯度的转换。
-这些函数会根据输入参数重新设置内部和外部存储，当然这两者也可以相等，即表示不需要转换。
-
-注意：每个`MKLDNNlayer`的子类只需要使用内部存储就可以了，所有外部的转换工作都会在reset系列函数中都准备好。
-
-### Activations
-在重构前的PaddlePaddle中，激活函数是独立于`Layer`的概念，并且输入输出都是共用一块内存，
-所以添加了对应的`MKLDNNActivation`来实现，方式类似于`MKLDNNLayer`。
-
-### Parameters
-对于有参数的层，我们会保证`MKLDNNLayer`使用的参数与PaddlePaddle申请的buffer共用一块内存。
-如果存在数据排列格式不一样的情况时，我们会在网络训练之前把格式转换为MKL-DNN希望的格式，
-在训练结束的时候再保存为PaddlePaddle的格式，但是整个训练过程中不需要任何转换。
-这样既使得最终保存的参数格式与PaddlePaddle一致，又可以避免不必要的转换。
-
-### Gradients
-由于MKL-DNN的操作都是直接覆盖的形式，也就是说输出的结果不会在原来的数据上累加，
-这样带来的好处就是不需要一直清空memory，节省了不必要的操作。
-但是注意的是，当网络出现分支且在`backward`的时候，需要累加不同Layer传过来的梯度。
-所以在`MKLDNNlayer`中实现了一个merge的方法，此时每个小分支的`Input Gradient`
-会先临时保存在`MKLDNNMatrix`中，由分支处的Layer负责求和，并把结果放到当前层的`output_.grad`中。
-所以整体上，在实现每个子类的时候就不需要关心分支的事情了。
-
--Figure 4. MKLDNNLayer -
-
-Figure 5. Merge Gradients -
-
-### Unit Tests
-我们会添加`test_MKLDNN.cpp`和`MKLDNNTester.*`用于MKL-DNN的测试。
-测试分为每个Layer（或Activation）的单元测试和简单网络的整体测试。
-每个测试会对比PaddlePaddle中CPU算出的结果与MKL-DNN的结果，小于某个比较小的阈值认为通过。
-
-### Python API
-目前只考虑**v1 API**。
-
-计划在`python/paddle/trainer/config_parser.py`里面添加`use_mkldnn`这个选择，方便用户选择使用MKL-DNN的layers。
-
-具体实现方式比如：
-
-```python
-use_mkldnn = bool(int(g_command_config_args.get("use_mkldnn", 0)))
-if use_mkldnn
- self.layer_type = mkldnn_*
-```
-
-所有MKL-DNN的`layer_type`会以*mkldnn_*开头，这些会在`MKLDNN*Layer`注册layer的时候保证，以示区分。
-
-同时,会在`paddle/utils.Flags`中添加一个`use_mkldnn`的flag，用于选择是否使用MKL-DNN的相关功能。
-
-### Benchmarking
-会添加相应的脚本在[这里](https://github.com/PaddlePaddle/Paddle/tree/develop/benchmark/paddle/image)，用于测试和对比在使用MKL-DNN前后的CNN网络性能。
-测试的性能对比结果会在[IntelOptimizedPaddle.md](https://github.com/PaddlePaddle/Paddle/blob/develop/benchmark/IntelOptimizedPaddle.md)
-
-### Others
-1. 如果在使用MKL-DNN的情况下，会把CPU的Buffer对齐为4096，具体可以参考MKL-DNN中的[memory](https://github.com/01org/mkl-dnn/blob/master/include/mkldnn.hpp#L673)。
-2. 深入PaddlePaddle，寻找有没有其他可以优化的可能，进一步优化。比如可能会用OpenMP改进SGD的更新性能。
-
-## Design Concerns
-
-为了更好的符合PaddlePaddle的代码风格\[[3](#references)\]，同时又尽可能少的牺牲MKL-DNN的性能\[[4](#references)\]。
-
-我们总结出一些特别需要注意的点：
-
-1. 使用**deviceId_**。为了尽可能少的在父类Layer中添加变量或者函数，
-我们决定使用已有的`deviceId_`变量来区分layer的属性，定义`-2`为`MKLDNNLayer`特有的设备ID。
-2. 重写父类Layer的**init**函数，修改`deviceId_`为`-2`，代表这个layer是用于跑在MKL-DNN的环境下。
-3. 创建`MKLDNNBase`，定义一些除了layer和memory相关的类和函数。
-包括MKL-DNN会用到`MKLDNNStream`和`CPUEngine`，和未来可能还会用到`FPGAEngine`等。
-4. 如果MKL-DNN layer的后面接有cpu device，那么就会使`output_.value`与`extOutVal_`共享内存，
-同时数据格式就是`NCHW`，这样下一个cpu device就能拿到正确的数据。
-在有普通的CPU layer时， `extOutVal_`和`extOutGrad_`的格式始终是`NCHW`或者`NC`。
-
-## References
-1. [MKL small library](https://github.com/01org/mkl-dnn#linking-your-application)是[Intel MKL](https://software.intel.com/en-us/mkl)的一个子集。
-主要包括了深度学习相关的数学原语与操作，一般由MKL-DNN在发布[新版本](https://github.com/01org/mkl-dnn/releases)时一起更新。
-2. [MKL-DNN System Requirements](https://github.com/01org/mkl-dnn#system-requirements)。
-目前在PaddlePaddle中，仅会在支持AVX2指令集及以上的机器才使用MKL-DNN。
-3. [原来的方案](https://github.com/PaddlePaddle/Paddle/pull/3096)会引入**nextLayer**的信息。
-但是在PaddlePaddle中，无论是重构前的layer还是重构后的op，都不会想要知道next layer/op的信息。
-4. MKL-DNN的高性能格式与PaddlePaddle原有的`NCHW`不同(PaddlePaddle中的cuDNN部分使用的也是`NCHW`，所以不存在这个问题)。
-所以需要引入一个转换方法，并且只需要在必要的时候转换这种格式，才能更好的发挥MKL-DNN的性能。
diff --git a/doc/v2/dev/contribute_to_paddle_cn.md b/doc/v2/dev/contribute_to_paddle_cn.md
deleted file mode 100644
index 3244eedf918b93f9351258f1218dfb2d507c1a9c..0000000000000000000000000000000000000000
--- a/doc/v2/dev/contribute_to_paddle_cn.md
+++ /dev/null
@@ -1,243 +0,0 @@
-# 如何贡献代码
-
-我们真诚地感谢您的贡献，欢迎通过 GitHub 的 fork 和 pull request 流程来提交代码。
-
-## 代码要求
-- 代码注释请遵守 [Doxygen](http://www.stack.nl/~dimitri/doxygen/) 的样式。
-- 确保编译器选项 `WITH_STYLE_CHECK` 已打开，并且编译能通过代码样式检查。
-- 所有代码必须具有单元测试。
-- 通过所有单元测试。
-- 请遵守[提交代码的一些约定](#提交代码的一些约定)。
-
-以下教程将指导您提交代码。
-## [Fork](https://help.github.com/articles/fork-a-repo/)
-
-跳转到[PaddlePaddle](https://github.com/PaddlePaddle/Paddle) GitHub首页，然后单击 `Fork` 按钮，生成自己目录下的仓库，比如 -Figure 5. Merge Gradients -
-
-选择目标分支：
-
-
-
-在 PR 的描述说明中，填写 `resolve #Issue编号` 可以在这个 PR 被 merge 后，自动关闭对应的 Issue，具体请见 
-
-也可以使用 `git push origin :分支名` 删除远程分支，如：
-
-```bash
-➜ git push origin :my-cool-stuff
-```
-
-## 删除本地分支
-
-最后，删除本地分支。
-
-```bash
-# 切换到 develop 分支
-➜ git checkout develop
-
-# 删除 my-cool-stuff 分支
-➜ git branch -D my-cool-stuff
-```
-
-至此，我们就完成了一次代码贡献的过程。
-
-## 提交代码的一些约定
-
-为了使评审人在评审代码时更好地专注于代码本身，请您每次提交代码时，遵守以下约定：
-
-1. 请保证Travis-CI 中单元测试能顺利通过。如果没过，说明提交的代码存在问题，评审人一般不做评审。
-2. 提交PUll Request前：
- - 请注意commit的数量：
- - 原因：如果仅仅修改一个文件但提交了十几个commit，每个commit只做了少量的修改，这会给评审人带来很大困扰。评审人需要逐一查看每个commit才能知道做了哪些修改，且不排除commit之间的修改存在相互覆盖的情况。
- - 建议：每次提交时，保持尽量少的commit，可以通过`git commit --amend`补充上次的commit。对已经Push到远程仓库的多个commit，可以参考[squash commits after push](http://stackoverflow.com/questions/5667884/how-to-squash-commits-in-git-after-they-have-been-pushed)。
- - 请注意每个commit的名称：应能反映当前commit的内容，不能太随意。
-3. 如果解决了某个Issue的问题，请在该PUll Request的**第一个**评论框中加上：`fix #issue_number`，这样当该PUll Request被合并后，会自动关闭对应的Issue。关键词包括：close, closes, closed, fix, fixes, fixed, resolve, resolves, resolved，请选择合适的词汇。详细可参考[Closing issues via commit messages](https://help.github.com/articles/closing-issues-via-commit-messages)。
-
-此外，在回复评审人意见时，请您遵守以下约定：
-
-1. 评审人的每个意见都必须回复（这是开源社区的基本礼貌，别人帮了忙，应该说谢谢）：
- - 对评审意见同意且按其修改完的，给个简单的`Done`即可；
- - 对评审意见不同意的，请给出您自己的反驳理由。
-2. 如果评审意见比较多：
- - 请给出总体的修改情况。
- - 请采用[start a review](https://help.github.com/articles/reviewing-proposed-changes-in-a-pull-request/)进行回复，而非直接回复的方式。原因是每个回复都会发送一封邮件，会造成邮件灾难。
diff --git a/doc/v2/dev/contribute_to_paddle_en.md b/doc/v2/dev/contribute_to_paddle_en.md
deleted file mode 100644
index b878f37a5b8e807e5aa346e0074a741f2f8b6cc5..0000000000000000000000000000000000000000
--- a/doc/v2/dev/contribute_to_paddle_en.md
+++ /dev/null
@@ -1,162 +0,0 @@
-# Contribute Code
-
-You are welcome to contribute to project PaddlePaddle. To contribute to PaddlePaddle, you have to agree with the
-[PaddlePaddle Contributor License Agreement](https://gist.github.com/wangkuiyi/0c22c7b1bd3bb7eb27d76f85c3a3e329).
-
-We sincerely appreciate your contribution. This document explains our workflow and work style.
-
-## Workflow
-
-PaddlePaddle uses this [Git branching model](http://nvie.com/posts/a-successful-git-branching-model/). The following steps guide usual contributions.
-
-1. Fork
-
- Our development community has been growing fastly; it doesn't make sense for everyone to write into the official repo. So, please file Pull Requests from your fork. To make a fork, just head over to the GitHub page and click the ["Fork" button](https://help.github.com/articles/fork-a-repo/).
-
-1. Clone
-
- To make a copy of your fork to your local computers, please run
-
- ```bash
- git clone https://github.com/your-github-account/paddle
- cd paddle
- ```
-
-1. Create the local feature branch
-
- For daily works like adding a new feature or fixing a bug, please open your feature branch before coding:
-
- ```bash
- git checkout -b my-cool-stuff
- ```
-
-1. Commit
-
- Before issuing your first `git commit` command, please install [`pre-commit`](http://pre-commit.com/) by running the following commands:
-
- ```bash
- pip install pre-commit
- pre-commit install
- ```
-
- Our pre-commit configuration requires clang-format 3.8 for auto-formating C/C++ code and yapf for Python.
-
- Once installed, `pre-commit` checks the style of code and documentation in every commit. We will see something like the following when you run `git commit`:
-
- ```
- ➜ git commit
- CRLF end-lines remover...............................(no files to check)Skipped
- yapf.................................................(no files to check)Skipped
- Check for added large files..............................................Passed
- Check for merge conflicts................................................Passed
- Check for broken symlinks................................................Passed
- Detect Private Key...................................(no files to check)Skipped
- Fix End of Files.....................................(no files to check)Skipped
- clang-formater.......................................(no files to check)Skipped
- [my-cool-stuff c703c041] add test file
- 1 file changed, 0 insertions(+), 0 deletions(-)
- create mode 100644 233
- ```
-
- NOTE: The `yapf` installed by `pip install pre-commit` and `conda install -c conda-forge pre-commit` is slightly different. Paddle developers use `pip install pre-commit`.
-
-1. Build and test
-
- Users can build PaddlePaddle natively on Linux and Mac OS X. But to unify the building environment and to make it easy for debugging, the recommended way is [using Docker](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/howto/dev/build_en.md).
-
-1. Keep pulling
-
- An experienced Git user pulls from the official repo often -- daily or even hourly, so they notice conflicts with others work early, and it's easier to resolve smaller conflicts.
-
- ```bash
- git remote add upstream https://github.com/PaddlePaddle/Paddle
- git pull upstream develop
- ```
-
-1. Push and file a pull request
-
- You can "push" your local work into your forked repo:
-
- ```bash
- git push origin my-cool-stuff
- ```
-
- The push allows you to create a pull request, requesting owners of this [official repo](https://github.com/PaddlePaddle/Paddle) to pull your change into the official one.
-
- To create a pull request, please follow [these steps](https://help.github.com/articles/creating-a-pull-request/).
-
- If your change is for fixing an issue, please write ["Fixes | 版本说明 | -C-API | -
|---|---|
| cpu_avx_mkl | -paddle.tgz | -
| cpu_avx_openblas | -paddle.tgz | -
| cpu_noavx_openblas | -paddle.tgz | -
| cuda7.5_cudnn5_avx_mkl | -paddle.tgz | -
| cuda8.0_cudnn5_avx_mkl | -paddle.tgz | -
| cuda8.0_cudnn7_avx_mkl | -paddle.tgz | -
| cuda9.0_cudnn7_avx_mkl | -paddle.tgz | -
| 选项 | -值 | -
|---|---|
| WITH_C_API | -ON | -
| WITH_PYTHON | -OFF（推荐） | -
| WITH_SWIG_PY | -OFF（推荐） | -
| WITH_GOLANG | -OFF（推荐） | -
| WITH_GPU | -ON/OFF | -
| WITH_MKL | -ON/OFF | -
| Version Tips | -C-API | -
|---|---|
| cpu_avx_mkl | -paddle.tgz | -
| cpu_avx_openblas | -paddle.tgz | -
| cpu_noavx_openblas | -paddle.tgz | -
| cuda7.5_cudnn5_avx_mkl | -paddle.tgz | -
| cuda8.0_cudnn5_avx_mkl | -paddle.tgz | -
| cuda8.0_cudnn7_avx_mkl | -paddle.tgz | -
| cuda9.0_cudnn7_avx_mkl | -paddle.tgz | -
| Options | -Value | -
|---|---|
| WITH_C_API | -ON | -
| WITH_PYTHON | -OFF（recommended） | -
| WITH_SWIG_PY | -OFF（recommended） | -
| WITH_GOLANG | -OFF（recommended） | -
| WITH_GPU | -ON/OFF | -
| WITH_MKL | -ON/OFF | -
- 
图1. 稀疏矩阵存储示意图
-
-
图2. 序列输入示意图
-
| Python 端数据类型 | -C-API 输入数据类型 | -
|---|---|
| paddle.data_type.integer_value | -整型数组，无需附加序列信息 | -
| paddle.data_type.dense_vector | -浮点型稠密矩阵，无需附加序列信息 | -
| paddle.data_type.sparse_binary_vector | -浮点型稀疏矩阵，无需提供非零元的值，默认为1，无需附加序列信息 | -
| paddle.data_type.sparse_vector | -浮点型稀疏矩阵，需提供非零元的值，无需附加序列信息 | -
| paddle.data_type.integer_value_sequence | -整型数组，需附加序列信息 | -
| paddle.data_type.dense_vector_sequence | -浮点型稠密矩阵，需附加序列信息 | -
| paddle.data_type.sparse_binary_vector_sequence | -浮点型稀疏矩阵，无需提供非零元的值，默认为1，需附加序列信息 | -
| paddle.data_type.sparse_vector_sequence | -浮点型稀疏矩阵，需提供非零元的值，需附加序列信息 | -
| paddle.data_type.integer_value_sub_sequence | -整型数组，需附加双层序列信息 | -
| paddle.data_type.dense_vector_sub_sequence | -浮点型稠密矩阵，需附加双层序列信息 | -
| paddle.data_type.sparse_binary_vector_sub_sequence | -浮点型稀疏矩阵，无需提供非零元的值，默认为1，需附加双层序列信息 | -
| paddle.data_type.sparse_vector_sub_sequence | -浮点型稀疏矩阵，需提供非零元的值，需附加双层序列信息 | -
- - -### 输出数据 - -PaddlePaddle中一个计算层的输出数据组织方式和输入数据组织方式完全相同。一个输出数据同样被组织为一个`argument`，`argument`通过`paddle_matrix`或`paddle_ivector`存数数据，如果输出是一个序列，那么会携带有`sequence_start_positions`信息。调用C-API相关接口，读取需要的结果即可。 - -### 总结 - -- 在PaddlePaddle内部，神经网络中一个计算层的输入/输出被组织为`argument`。 -- `argument`并不真正“存储”数据，而是将输入/输出信息有机地组织在一起。 -- 在`argument`内部由`paddle_ivector`（一维整型数组）和`paddle_matrix`（二维浮点型矩阵）来实际存储数据。 -如果是一个序列输入/输出由 `sequence start positions` 来记录输入/输出的序列信息。 - -于是，在组织神经网络输入时，需要思考完成以下工作： - -1. 为每一个输入/输出创建`argument`。 - - C-API 中操作`argument`的接口请查看[argument.h](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/capi/arguments.h)。 -1. 为每一个`argument`创建`paddle_matrix`或者`paddle_ivector`来存储数据。 - - C-API 中操作`paddle_ivector`的接口请查看 [vector.h](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/capi/vector.h)。 - - C-API 中操作`paddle_matrix`的接口请查看[matrix.h](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/capi/matrix.h)。 -1. 如果输入是序列数据，需要创建并填写`sequence_start_positions`信息。 - - 通过调用 [`paddle_arguments_set_sequence_start_pos`](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/capi/arguments.h#L137) 来为一个`argument`添加序列信息。 - - 通过调用 [`paddle_arguments_get_sequence_start_pos`](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/capi/arguments.h#L150) 来读取一个`argument`添加序列信息。 - - 接口说明请查看 [argument.h](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/capi/arguments.h) 文件。 diff --git a/doc/v2/howto/capi/organization_of_the_inputs_en.md b/doc/v2/howto/capi/organization_of_the_inputs_en.md deleted file mode 100644 index 250d3b2f749aed018e63527e817899c843dff996..0000000000000000000000000000000000000000 --- a/doc/v2/howto/capi/organization_of_the_inputs_en.md +++ /dev/null @@ -1,3 +0,0 @@ -## Input/Output Data Organization - -TBD diff --git a/doc/v2/howto/capi/workflow_of_capi_cn.md b/doc/v2/howto/capi/workflow_of_capi_cn.md deleted file mode 100644 index db1568a2afbea3cca0d4e1fe053ba9536a60ab3d..0000000000000000000000000000000000000000 --- a/doc/v2/howto/capi/workflow_of_capi_cn.md +++ /dev/null @@ -1,124 +0,0 @@ -## C-API使用流程 - -这篇文档介绍 PaddlePaddle C-API 整体使用流程。 - -### 使用流程 - -使用 C-API 的工作流程如图1所示，分为（1）准备预测模型和（2）预测程序开发两大部分。 - -
-
图1. C-API使用流程示意图
-
| - | 参数 | -本地训练 | -集群训练 | -本地测试 | -集群测试 | -
|---|---|---|---|---|---|
| 通用 | -job | -√ | √ | √ | √ | -
| use_gpu | -√ | √ | √ | √ | -|
| local | -√ | √ | √ | √ | -|
| config | -√ | √ | √ | √ | -|
| config_args | -√ | √ | √ | √ | -|
| num_passes | -√ | √ | √ | √ | -|
| trainer_count | -√ | √ | √ | √ | -|
| version | -√ | √ | √ | √ | -|
| show_layer_stat | -√ | √ | √ | √ | -|
| 训练 | dot_period | -√ | √ | - | |
| test_period | -√ | √ | - | ||
| saving_period | -√ | √ | - | ||
| show_parameter_stats_period | -√ | √ | - | ||
| init_model_path | -√ | √ | √ | - | |
| load_missing_parameter_strategy | -√ | √ | - | ||
| saving_period_by_batches | -√ | √ | - | ||
| use_old_updater | -√ | √ | - | ||
| enable_grad_share | -√ | √ | - | ||
| grad_share_block_num | -√ | √ | - | ||
| log_error_clipping | -√ | √ | - | ||
| log_clipping | -√ | √ | - | ||
| save_only_one | -√ | √ | - | ||
| start_pass | -√ | √ | - | ||
| 训练/测试 | save_dir | -√ | √ | √ | √ | -
| 训练过程中测试 | test_period | -√ | √ | - | |
| average_test_period | -√ | √ | - | ||
| 测试 | model_list | -√ | √ | -||
| test_wait | -√ | √ | -|||
| test_pass | -√ | √ | -|||
| predict_output_dir | -√ | √ | -|||
| distribute_test | -√ | √ | -|||
| Auc/正负对验证(PnpairValidation) | predict_file | -√ | √ | -||
| GPU | gpu_id | -√ | √ | √ | √ | -
| parallel_nn | -√ | √ | √ | √ | -|
| allow_only_one_model_on_one_gpu | -√ | √ | √ | √ | -|
| cudnn_dir | -√ | √ | √ | √ | -|
| cuda_dir | -√ | √ | √ | √ | -|
| cudnn_conv_workspace_limit_in_mb | -√ | √ | √ | √ | -|
| 递归神经网络(RNN) | -beam_size | -√ | √ | -||
| rnn_use_batch | -√ | √ | √ | √ | -|
| prev_batch_state | -√ | √ | - | ||
| diy_beam_search_prob_so | -√ | √ | -|||
| 参数服务器(PServer) | start_pserver | -√ | √ | -||
| pservers | -√ | √ | -|||
| port | -√ | √ | -|||
| port_num | -√ | √ | -|||
| ports_num_for_sparse | -√ | √ | -|||
| nics | -√ | √ | -|||
| rdma_tcp | -√ | √ | -|||
| small_messages | -√ | - | |||
| loadsave_parameters_in_pserver | -√ | √ | -|||
| log_period_server | -√ | - | |||
| pserver_num_threads | -√ | - | |||
| sock_send_buf_size | -√ | - | |||
| sock_recv_buf_size | -√ | - | |||
| num_gradient_servers | -√ | - | |||
| parameter_block_size | -√ | - | |||
| parameter_block_size_for_sparse | -√ | - | |||
| 异步随机梯度下降(Async SGD) | async_count | -√ | - | ||
| async_lagged_ratio_min | -√ | - | |||
| async_lagged_ratio_default | -√ | - | |||
| 性能调优(Performance Tuning) | log_barrier_abstract | -√ | - | ||
| log_barrier_lowest_nodes | -√ | - | |||
| log_barrier_show_log | -√ | - | |||
| check_sparse_distribution_batches | -√ | - | |||
| check_sparse_distribution_ratio | -√ | - | |||
| check_sparse_distribution_unbalance_degree | -√ | - | |||
| check_sparse_distribution_in_pserver | -√ | - | |||
| show_check_sparse_distribution_log | -√ | - | |||
| 数据提供器(Data Provider) | memory_threshold_on_load_data | -√ | √ | - | |
| 随机数 | seed | -√ | √ | - | |
| thread_local_rand_use_global_seed | -√ | √ | - | ||
| 单元测试 | checkgrad_eps | -- | |||
| 矩阵/向量 | enable_parallel_vector | -√ | √ | √ | √ | -
| - | args | -local train | -cluster train | -local test | -cluster test | -
|---|---|---|---|---|---|
| common | -job | -√ | √ | √ | √ | -
| use_gpu | -√ | √ | √ | √ | -|
| local | -√ | √ | √ | √ | -|
| config | -√ | √ | √ | √ | -|
| config_args | -√ | √ | √ | √ | -|
| num_passes | -√ | √ | √ | √ | -|
| trainer_count | -√ | √ | √ | √ | -|
| version | -√ | √ | √ | √ | -|
| show_layer_stat | -√ | √ | √ | √ | -|
| train | dot_period | -√ | √ | - | |
| test_period | -√ | √ | - | ||
| saving_period | -√ | √ | - | ||
| show_parameter_stats_period | -√ | √ | - | ||
| init_model_path | -√ | √ | √ | - | |
| load_missing_parameter_strategy | -√ | √ | - | ||
| saving_period_by_batches | -√ | √ | - | ||
| use_old_updater | -√ | √ | - | ||
| enable_grad_share | -√ | √ | - | ||
| grad_share_block_num | -√ | √ | - | ||
| log_error_clipping | -√ | √ | - | ||
| log_clipping | -√ | √ | - | ||
| save_only_one | -√ | √ | - | ||
| start_pass | -√ | √ | - | ||
| train/test | save_dir | -√ | √ | √ | √ | -
| testing during training | test_period | -√ | √ | - | |
| average_test_period | -√ | √ | - | ||
| test | model_list | -√ | √ | -||
| test_wait | -√ | √ | -|||
| test_pass | -√ | √ | -|||
| predict_output_dir | -√ | √ | -|||
| distribute_test | -√ | √ | -|||
| Auc/PnpairValidation | predict_file | -√ | √ | -||
| GPU | gpu_id | -√ | √ | √ | √ | -
| parallel_nn | -√ | √ | √ | √ | -|
| allow_only_one_model_on_one_gpu | -√ | √ | √ | √ | -|
| cudnn_dir | -√ | √ | √ | √ | -|
| cuda_dir | -√ | √ | √ | √ | -|
| cudnn_conv_workspace_limit_in_mb | -√ | √ | √ | √ | -|
| RNN | -beam_size | -√ | √ | -||
| rnn_use_batch | -√ | √ | √ | √ | -|
| prev_batch_state | -√ | √ | - | ||
| diy_beam_search_prob_so | -√ | √ | -|||
| PServer | start_pserver | -√ | √ | -||
| pservers | -√ | √ | -|||
| port | -√ | √ | -|||
| port_num | -√ | √ | -|||
| ports_num_for_sparse | -√ | √ | -|||
| nics | -√ | √ | -|||
| rdma_tcp | -√ | √ | -|||
| small_messages | -√ | - | |||
| loadsave_parameters_in_pserver | -√ | √ | -|||
| log_period_server | -√ | - | |||
| pserver_num_threads | -√ | - | |||
| sock_send_buf_size | -√ | - | |||
| sock_recv_buf_size | -√ | - | |||
| num_gradient_servers | -√ | - | |||
| parameter_block_size | -√ | - | |||
| parameter_block_size_for_sparse | -√ | - | |||
| Async SGD | async_count | -√ | - | ||
| async_lagged_ratio_min | -√ | - | |||
| async_lagged_ratio_default | -√ | - | |||
| Performance Tuning | log_barrier_abstract | -√ | - | ||
| log_barrier_lowest_nodes | -√ | - | |||
| log_barrier_show_log | -√ | - | |||
| check_sparse_distribution_batches | -√ | - | |||
| check_sparse_distribution_ratio | -√ | - | |||
| check_sparse_distribution_unbalance_degree | -√ | - | |||
| check_sparse_distribution_in_pserver | -√ | - | |||
| show_check_sparse_distribution_log | -√ | - | |||
| Data Provider | memory_threshold_on_load_data | -√ | √ | - | |
| RandomNumber | seed | -√ | √ | - | |
| thread_local_rand_use_global_seed | -√ | √ | - | ||
| UnitTest | checkgrad_eps | -- | |||
| Matrix/Vector | enable_parallel_vector | -√ | √ | √ | √ | -