update the content of basic_concept and prepare_data (#1133)

Unify the guide content structure.

doc/fluid/advanced_usage/index_en.rst

@@ -6,7 +6,7 @@ Advanced User Guides
 
 So far you have already been familiar with Fluid. And the next expectation should be building a more efficient model or inventing your original Operator. If so, read more on:
 
-    - `Fluid Design Principles <../advanced_usage/design_idea/fluid_design_idea_en.html>`_ : Design principles underlying Fluid to help you understand how the framework runs.
+    - `Design Principles of Fluid <../advanced_usage/design_idea/fluid_design_idea_en.html>`_ : Design principles underlying Fluid to help you understand how the framework runs.
 
 	- `Deploy Inference Model <../advanced_usage/deploy/index_en.html>`_ ：How to deploy the trained network to perform practical inference
 

doc/fluid/user_guides/howto/basic_concept/index_cn.rst

-############
-基本概念
-############
+.. _cn_user_guide_lod_tensor:
 
-本文介绍Fluid版本基本使用概念：
+##################
+LoD-Tensor使用说明
+##################
 
-- `LoD-Tensor使用说明 <lod_tensor.html>`_ : LoD-Tensor是Fluid中特有的概念，它在Tensor基础上附加了序列信息，支持处理变长数据。
+LoD(Level-of-Detail) Tensor是Fluid中特有的概念，它在Tensor基础上附加了序列信息。Fluid中可传输的数据包括：输入、输出、网络中的可学习参数，全部统一使用LoD-Tensor表示。
 
-..  toctree::
-    :hidden:
+阅读本文档将帮助您了解 Fluid 中的 LoD-Tensor 设计思想，以便您更灵活的使用这一数据类型。
+
+变长序列的挑战
+================
+
+大多数的深度学习框架使用Tensor表示一个mini-batch。
+
+例如一个mini-batch中有10张图片，每幅图片大小为32x32，则这个mini-batch是一个10x32x32的 Tensor。
+
+或者在处理NLP任务中，一个mini-batch包含N个句子，每个字都用一个D维的one-hot向量表示，假设所有句子都用相同的长度L，那这个mini-batch可以被表示为NxLxD的Tensor。
+
+上述两个例子中序列元素都具有相同大小，但是在许多情况下，训练数据是变长序列。基于这一场景，大部分框架采取的方法是确定一个固定长度，对小于这一长度的序列数据以0填充。
+
+在Fluid中，由于LoD-Tensor的存在，我们不要求每个mini-batch中的序列数据必须保持长度一致，因此您不需要执行填充操作，也可以满足处理NLP等具有序列要求的任务需求。
+
+Fluid引入了一个索引数据结构（LoD）来将张量分割成序列。
+
+
+LoD 索引
+===========
+
+为了更好的理解LoD的概念，本节提供了几个例子供您参考：
+
+**句子组成的 mini-batch**
+
+假设一个mini-batch中有3个句子，每个句子中分别包含3个、1个和2个单词。我们可以用(3+1+2)xD维Tensor 加上一些索引信息来表示这个mini-batch:
+
+.. code-block :: text
+
+  3       1   2
+  | | |   |   | |
+
+上述表示中，每一个 :code:`|` 代表一个D维的词向量，数字3，1，2构成了 1-level LoD。
+
+**递归序列**
+
+让我们来看另一个2-level LoD-Tensor的例子：假设存在一个mini-batch中包含3个句子、1个句子和2个句子的文章，每个句子都由不同数量的单词组成，则这个mini-batch的样式可以看作：
+
+.. code-block:: text
+
+
+  3            1 2
+  3   2  4     1 2  3
+  ||| || ||||  | || |||
+
+
+表示的LoD信息为：
+
+.. code-block:: text
+
+  [[3，1，2]/*level=0*/，[3，2，4，1，2，3]/*level=1*/]
+
+
+**视频的mini-batch**
+
+在视觉任务中，时常需要处理视频和图像这些元素是高维的对象，假设现存的一个mini-batch包含3个视频，分别有3个，1个和2个帧，每个帧都具有相同大小：640x480，则这个mini-batch可以被表示为：
+
+.. code-block:: text
+
+  3     1  2
+  口口口 口 口口
+
+
+最底层tensor大小为（3+1+2）x640x480，每一个 :code:`口` 表示一个640x480的图像
+
+**图像的mini-batch**
+
+在传统的情况下，比如有N个固定大小的图像的mini-batch，LoD-Tensor表示为:
+
+.. code-block:: text
+
+  1 1 1 1     1
+  口口口口 ... 口
+
+在这种情况下，我们不会因为索引值都为1而忽略信息，仅仅把LoD-Tensor看作是一个普通的张量:
+
+.. code-block:: text
+
+  口口口口 ... 口
+
+**模型参数**
+
+模型参数只是一个普通的张量，在Fluid中它们被表示为一个0-level LoD-Tensor。
+
+LoDTensor的偏移表示
+=====================
+
+为了快速访问基本序列，Fluid提供了一种偏移表示的方法——保存序列的开始和结束元素，而不是保存长度。
+
+在上述例子中，您可以计算基本元素的长度：
+
+.. code-block:: text
+
+  3 2 4 1 2 3
+
+将其转换为偏移表示：
+
+.. code-block:: text
+
+  0  3  5   9   10  12   15
+     =  =   =   =   =    =
+     3  2+3 4+5 1+9 2+10 3+12
+
+所以我们知道第一个句子是从单词0到单词3，第二个句子是从单词3到单词5。
+
+类似的，LoD的顶层长度
+
+.. code-block:: text
+
+  3 1 2
+
+可以被转化成偏移形式：
+
+.. code-block:: text
+
+  0 3 4   6
+    = =   =
+    3 3+1 4+2
+
+因此该LoD-Tensor的偏移表示为：
+
+.. code-block:: text
+
+  0       3    4      6
+    3 5 9   10   12 15
+
+
+LoD-Tensor
+=============
+一个LoD-Tensor可以被看作是一个树的结构，树叶是基本的序列元素，树枝作为基本元素的标识。
+
+在 Fluid 中 LoD-Tensor 的序列信息有两种表述形式：原始长度和偏移量。在 Paddle 内部采用偏移量的形式表述 LoD-Tensor，以获得更快的序列访问速度；在 python API中采用原始长度的形式表述 LoD-Tensor 方便用户理解和计算，并将原始长度称为： :code:`recursive_sequence_lengths` 。
+
+以上文提到的一个2-level LoD-Tensor为例：
+
+.. code-block:: text
+
+  3           1  2
+  3   2  4    1  2  3
+  ||| || |||| |  || |||
+
+- 以偏移量表示此 LoD-Tensor:[ [0,3,4,6] , [0,3,5,9,10,12,15] ]，
+- 以原始长度表达此 Lod-Tensor：recursive_sequence_lengths=[ [3-0 , 4-3 , 6-4] , [3-0 , 5-3 , 9-5 , 10-9 , 12-10 , 15-12] ]。
+
+
+以文字序列为例： [3,1,2] 可以表示这个mini-batch中有3篇文章，每篇文章分别有3、1、2个句子，[3,2,4,1,2,3] 表示每个句子中分别含有3、2、4、1、2、3个字。
+
+recursive_seq_lens 是一个双层嵌套列表，也就是列表的列表，最外层列表的size表示嵌套的层数，也就是lod-level的大小；内部的每个列表，对应表示每个lod-level下，每个元素的大小。
+
+下面三段代码分别介绍如何创建一个LoD-Tensor，如何将LoD-Tensor转换成Tensor，如何将Tensor转换成LoD-Tensor：
+
+* 创建 LoD-Tensor
+
+.. code-block:: python
+
+  #创建lod-tensor
+  import paddle.fluid as fluid
+  import numpy as np
+  
+  a = fluid.create_lod_tensor(np.array([[1],[1],[1],
+                                    [1],[1],
+                                    [1],[1],[1],[1],
+                                    [1],
+                                    [1],[1],
+                                    [1],[1],[1]]).astype('int64') ,
+                            [[3,1,2] , [3,2,4,1,2,3]],
+                            fluid.CPUPlace())
+  
+  #查看lod-tensor嵌套层数
+  print (len(a.recursive_sequence_lengths()))
+  # output：2
+
+  #查看最基础元素个数
+  print (sum(a.recursive_sequence_lengths()[-1]))
+  # output:15 (3+2+4+1+2+3=15)
+
+* LoD-Tensor 转 Tensor
+
+.. code-block:: python
+
+  import paddle.fluid as fluid
+  import numpy as np
+
+  # 创建一个 LoD-Tensor
+  a = fluid.create_lod_tensor(np.array([[1.1], [2.2],[3.3],[4.4]]).astype('float32'), [[1,3]], fluid.CPUPlace())
+
+  def LodTensor_to_Tensor(lod_tensor):
+    # 获取 LoD-Tensor 的 lod 信息
+    lod = lod_tensor.lod()
+    # 转换成 array
+    array = np.array(lod_tensor)
+    new_array = []
+    # 依照原LoD-Tensor的层级信息，转换成Tensor
+    for i in range(len(lod[0]) - 1):
+        new_array.append(array[lod[0][i]:lod[0][i + 1]])
+    return new_array
+
+  new_array = LodTensor_to_Tensor(a)
+
+  # 输出结果
+  print(new_array)
+
+* Tensor 转 LoD-Tensor
+
+.. code-block:: python
+
+  import paddle.fluid as fluid
+  import numpy as np
+
+  def to_lodtensor(data, place):
+    # 存储Tensor的长度作为LoD信息
+    seq_lens = [len(seq) for seq in data]
+    cur_len = 0
+    lod = [cur_len]
+    for l in seq_lens:
+        cur_len += l
+        lod.append(cur_len)
+    # 对待转换的 Tensor 降维
+    flattened_data = np.concatenate(data, axis=0).astype("int64")
+    flattened_data = flattened_data.reshape([len(flattened_data), 1])
+    # 为 Tensor 数据添加lod信息
+    res = fluid.LoDTensor()
+    res.set(flattened_data, place)
+    res.set_lod([lod])
+    return res
+
+  # new_array 为上段代码中转换的Tensor
+  lod_tensor = to_lodtensor(new_array,fluid.CPUPlace())
+
+  # 输出 LoD 信息
+  print("The LoD of the result: {}.".format(lod_tensor.lod()))
+
+  # 检验与原Tensor数据是否一致
+  print("The array : {}.".format(np.array(lod_tensor)))
+
+
+
+
+代码示例
+===========
+
+本节代码将根据指定的级别y-lod，扩充输入变量x。本例综合了LoD-Tensor的多个重要概念，跟随代码实现，您将：
+
+-  直观理解Fluid中 :code:`fluid.layers.sequence_expand` 的实现过程
+-  掌握如何在Fluid中创建LoD-Tensor
+-  学习如何打印LoDTensor内容
+
+
+  
+**定义计算过程**
+
+layers.sequence_expand通过获取 y 的 lod 值对 x 的数据进行扩充，关于 :code:`fluid.layers.sequence_expand` 的功能说明，请先阅读 :ref:`cn_api_fluid_layers_sequence_expand` 。
+
+序列扩充代码实现：
+
+.. code-block:: python
+
+  x = fluid.layers.data(name='x', shape=[1], dtype='float32', lod_level=1)
+  y = fluid.layers.data(name='y', shape=[1], dtype='float32', lod_level=2)
+  out = fluid.layers.sequence_expand(x=x, y=y, ref_level=0)
+
+*说明*：输出LoD-Tensor的维度仅与传入的真实数据维度有关，在定义网络结构阶段为x、y设置的shape值，仅作为占位，并不影响结果。
+
+**创建Executor**
+
+.. code-block:: python
+
+  place = fluid.CPUPlace()
+  exe = fluid.Executor(place)
+  exe.run(fluid.default_startup_program())
+
+**准备数据**
+
+这里我们调用 :code:`fluid.create_lod_tensor` 创建 :code:`sequence_expand` 的输入数据，通过定义 y_d 的 LoD 值，对 x_d 进行扩充。其中，输出值只与 y_d 的 LoD 值有关，y_d 的 data 值在这里并不参与计算，维度上与LoD[-1]一致即可。
+
+:code:`fluid.create_lod_tensor()` 的使用说明请参考 :ref:`cn_api_fluid_create_lod_tensor` 。
+
+实现代码如下：
+
+.. code-block:: python
+
+  x_d = fluid.create_lod_tensor(np.array([[1.1],[2.2],[3.3],[4.4]]).astype('float32'), [[1,3]], place)
+  y_d = fluid.create_lod_tensor(np.array([[1.1],[1.1],[1.1],[1.1],[1.1],[1.1]]).astype('float32'), [[1,3], [2,1,2,1]],place)
+
+
+**执行运算**
+
+在Fluid中，LoD>1的Tensor与其他类型的数据一样，使用 :code:`feed` 定义数据传入顺序。此外，由于输出results是带有LoD信息的Tensor，需在exe.run( )中添加 :code:`return_numpy=False` 参数，获得LoD-Tensor的输出结果。
+
+.. code-block:: python
+
+  results = exe.run(fluid.default_main_program(),
+                    feed={'x':x_d, 'y': y_d },
+                    fetch_list=[out],return_numpy=False)
+
+**查看LodTensor结果**
+
+由于LoDTensor的特殊属性，无法直接print查看内容，常用操作时将LoD-Tensor作为网络的输出fetch出来，然后执行 numpy.array(lod_tensor), 就能转成numpy array：
+
+.. code-block:: python
+
+  np.array(results[0])
+
+输出结果为：
+
+.. code-block:: text
+
+  array([[1.1],[2.2],[3.3],[4.4],[2.2],[3.3],[4.4],[2.2],[3.3],[4.4]])
+
+**查看序列长度**
+
+可以通过查看序列长度得到 LoDTensor 的递归序列长度：
+
+.. code-block:: python
+
+    results[0].recursive_sequence_lengths()
+    
+输出结果为：
+
+.. code-block:: text
+    
+    [[1L, 3L, 3L, 3L]]
+
+**完整代码**
+
+您可以运行下列完整代码，观察输出结果：
+
+.. code-block:: python
+    
+    #加载库
+    import paddle
+    import paddle.fluid as fluid
+    import numpy as np
+    #定义前向计算
+    x = fluid.layers.data(name='x', shape=[1], dtype='float32', lod_level=1)
+    y = fluid.layers.data(name='y', shape=[1], dtype='float32', lod_level=2)
+    out = fluid.layers.sequence_expand(x=x, y=y, ref_level=0)
+    #定义运算场所
+    place = fluid.CPUPlace()
+    #创建执行器
+    exe = fluid.Executor(place)
+    exe.run(fluid.default_startup_program())
+    #创建LoDTensor
+    x_d = fluid.create_lod_tensor(np.array([[1.1], [2.2],[3.3],[4.4]]).astype('float32'), [[1,3]], place)
+    y_d = fluid.create_lod_tensor(np.array([[1.1],[1.1],[1.1],[1.1],[1.1],[1.1]]).astype('float32'), [[1,3], [1,2,1,2]], place)
+    #开始计算
+    results = exe.run(fluid.default_main_program(),
+                      feed={'x':x_d, 'y': y_d },
+                      fetch_list=[out],return_numpy=False)
+    #输出执行结果
+    print("The data of the result: {}.".format(np.array(results[0])))
+    #输出 result 的序列长度
+    print("The recursive sequence lengths of the result: {}.".format(results[0].recursive_sequence_lengths()))
+    #输出 result 的 LoD
+    print("The LoD of the result: {}.".format(results[0].lod()))
+
+
+总结
+========
+
+至此，相信您已经基本掌握了LoD-Tensor的概念，尝试修改上述代码中的 x_d 与 y_d，观察输出结果，有助于您更好的理解这一灵活的结构。
+
+更多LoDTensor的模型应用，可以参考新手入门中的 `词向量 <../../../beginners_guide/basics/word2vec/index.html>`_ 、`个性化推荐 <../../../beginners_guide/basics/recommender_system/index.html>`_、`情感分析 <../../../beginners_guide/basics/understand_sentiment/index.html>`_ 等指导教程。
+
+更高阶的应用案例，请参考 `模型库 <../../../user_guides/models/index_cn.html>`_ 中的相关内容。
 
-    lod_tensor.rst

doc/fluid/user_guides/howto/basic_concept/index_en.rst

-###############
-Basic Concepts
-###############
+#####################
+LoD-Tensor User Guide
+#####################
+
+LoD(Level-of-Detail) Tensor is a unique term in Fluid, which can be constructed by appending sequence information to Tensor. Data transferred in Fluid contain input, output and learnable parameters of the network, all of which are represented by LoD-Tensor.
+
+With the help of this user guide, you will learn the design idea of LoD-Tensor in Fluid so that you can use such a data type more flexibly.
+
+Challenge of variable-length sequences
+======================================
+
+In most deep learning frameworks, a mini-batch is represented by Tensor.
+
+For example, if there are 10 pictures in a mini-batch and the size of each picture is 32*32, the mini-batch will be a 10*32*32 Tensor.
+
+Or in the NLP task, there are N sentences in a mini-batch and the length of each sentence is L. Every word is represented by a one-hot vector with D dimensions. Then the mini-batch can be represented by an N*L*D Tensor.
+
+In the two examples above, the size of each sequence element remains the same. However, the data to be trained are variable-length sequences in many cases. For this scenario, method to be taken in most frameworks is to set a fixed length and sequence data shorter than the fixed length will be padded with 0 to reach the fixed length.
+
+Owing to the LoD-Tensor in Fluid, it is not necessary to keep the lengths of sequence data in every mini-batch constant.Therefore tasks sensitive to sequence formats like NLP can also be finished without padding.
+
+Index Data Structure (LoD) is introduced to Fluid to split Tensor into sequences.
+
+Index Structure - LoD 
+======================
+
+To have a better understanding of the concept of LoD, you can refer to the examples in this section.
+
+**mini-batch consisting of sentences**
+
+Suppose a mini-batch contains three sentences, and each contains 3, 1, 2 words respectively. Then the mini-batch can be represented by a (3+1+2)*D Tensor with some index information appended:
+
+.. code-block :: text
+
+  3       1   2
+  | | |   |   | |
+
+In the text above, each :code:`|` represents a word vector with D dimension and a 1-level LoD is made up of digits 3,1,2 .
+
+**recursive sequence**
+
+Take a 2-level LoD-Tensor for example, a mini-batch contains articles of 3 sentences, 1 sentence and 2 sentences. The number of words in every sentence is different. Then the mini-batch is formed as follows:
+
+.. code-block:: text
+
+
+  3            1 2
+  3   2  4     1 2  3
+  ||| || ||||  | || |||
+
+
+the LoD to express the format:
+
+.. code-block:: text
+
+  [[3，1，2]/*level=0*/，[3，2，4，1，2，3]/*level=1*/]
+
+
+**mini-batch consisting of video data**
+
+In the task of computer vision, it usually needs to deal objects with high dimension like videos and pictures. Suppose a mini-batch contains 3 videos, which is composed of 3 frames, 1 frames, 2 frames respectively. The size of each frame is 640*480. Then the mini-batch can be described as:
+
+.. code-block:: text
+
+  3     1  2
+  口口口 口 口口
+
+
+The size of the tensor at the bottom is (3+1+2)*640*480. Every :code:`口` represents a 640*480 picture.
+
+**mini-batch consisting of pictures**
+
+Traditionally, for a mini-batch of N pictures with fixed size, LoD-Tensor is described as:
+
+.. code-block:: text
+
+  1 1 1 1     1
+  口口口口 ... 口
+
+Under such circumstance, we will consider LoD-Tensor as a common tensor instead of ignoring information because of the indices of all elements are 1.
+
+.. code-block:: text
+
+  口口口口 ... 口
+
+**model parameter**
+
+model parameter is a common tensor which is described as a 0-level LoD-Tensor in Fluid.
+
+LoDTensor expressed by offset
+=============================
+
+To have a quick access to the original sequence, you can take the offset expression method——store the first and last element of a sequence instead of its length.
+
+In the example above, you can compute the length of fundamental elements:
+
+.. code-block:: text
+
+  3 2 4 1 2 3
+
+It is expressed by offset as follows:
+
+.. code-block:: text
+
+  0  3  5   9   10  12   15
+     =  =   =   =   =    =
+     3  2+3 4+5 1+9 2+10 3+12
+
+Therefore we infer that the first sentence starts from word 0 to word 3 and the second sentence starts from word 3 to word 5.
+
+Similarly, for the length of the top layer of LoD
+
+.. code-block:: text
+
+  3 1 2
+
+It can be expressed by offset:
+
+.. code-block:: text
+
+  0 3 4   6
+    = =   =
+    3 3+1 4+2
+
+Therefore the LoD-Tensor is expressed by offset:
+
+.. code-block:: text
+
+  0       3    4      6
+    3 5 9   10   12 15
+
+
+LoD-Tensor
+=============
+A LoD-Tensor can be regarded as a tree of which the leaf is an original sequence element and branch is the flag of fundamental element.
+
+There are two ways to express sequence information of LoD-Tensor in Fluid: primitive length and offset. LoD-Tensor is expressed by offset in Paddle to offer a quicker access to sequence;LoD-Tensor is expressed by primitive length in python API to make user understand and compute more easily. The primary length is named as  :code:`recursive_sequence_lengths` .
+
+Take a 2-level LoD-Tensor mentioned above as an example:
+
+.. code-block:: text
+
+  3           1  2
+  3   2  4    1  2  3
+  ||| || |||| |  || |||
+
+- LoD-Tensor expressed by offset: [ [0,3,4,6] , [0,3,5,9,10,12,15] ]
+- LoD-Tensor expressed by primitive length: recursive_sequence_lengths=[ [3-0 , 4-3 , 6-4] , [3-0 , 5-3 , 9-5 , 10-9 , 12-10 , 15-12] ]
+
+
+Take text sequence as an example,[3,1,2] indicates there are 3 articles in the mini-batch,which contains 3,1,2 sentences respectively.[3,2,4,1,2,3] indicates there are 3,2,4,1,2,3 words in sentences respectively.
+
+recursive_seq_lens is a double Layer nested list, and in other words, the element of the list is list. The size of the outermost list represents the nested layers, namely the size of lod-level; Each inner list represents the size of each element in each lod-level. 
+
+The following three pieces of codes introduce how to create LoD-Tensor, how to transform LoD-Tensor to Tensor and how to transform Tensor to LoD-Tensor respectively:
+
+  * Create LoD-Tensor
+
+.. code-block:: python
+
+  #Create lod-tensor
+  import paddle.fluid as fluid
+  import numpy as np
+  
+  a = fluid.create_lod_tensor(np.array([[1],[1],[1],
+                                    [1],[1],
+                                    [1],[1],[1],[1],
+                                    [1],
+                                    [1],[1],
+                                    [1],[1],[1]]).astype('int64') ,
+                            [[3,1,2] , [3,2,4,1,2,3]],
+                            fluid.CPUPlace())
+  
+  #Check lod-tensor nested layers
+  print (len(a.recursive_sequence_lengths()))
+  # output：2
+
+  #Check the number of the most fundamental elements
+  print (sum(a.recursive_sequence_lengths()[-1]))
+  # output:15 (3+2+4+1+2+3=15)
+
+* Transform LoD-Tensor to Tensor
+
+  .. code-block:: python
+
+   import paddle.fluid as fluid
+   import numpy as np
+
+   # create LoD-Tensor
+   a = fluid.create_lod_tensor(np.array([[1.1], [2.2],[3.3],[4.4]]).astype('float32'), [[1,3]], fluid.CPUPlace())
+
+   def LodTensor_to_Tensor(lod_tensor):
+     # get lod information of LoD-Tensor
+     lod = lod_tensor.lod()
+     # transform into array
+     array = np.array(lod_tensor)
+     new_array = []
+     # transform to Tensor according to the layer information of the original LoD-Tensor
+     for i in range(len(lod[0]) - 1):
+         new_array.append(array[lod[0][i]:lod[0][i + 1]])
+     return new_array
+
+   new_array = LodTensor_to_Tensor(a)
+
+   # output the result
+   print(new_array)
+
+ * Transform Tensor to LoD-Tensor
+
+  .. code-block:: python
+
+   import paddle.fluid as fluid
+   import numpy as np
+
+   def to_lodtensor(data, place):
+     # save the length of Tensor as LoD information
+     seq_lens = [len(seq) for seq in data]
+     cur_len = 0
+     lod = [cur_len]
+     for l in seq_lens:
+         cur_len += l
+         lod.append(cur_len)
+     # decrease the dimention of transformed Tensor
+     flattened_data = np.concatenate(data, axis=0).astype("int64")
+     flattened_data = flattened_data.reshape([len(flattened_data), 1])
+     # add lod information to Tensor data
+     res = fluid.LoDTensor()
+     res.set(flattened_data, place)
+     res.set_lod([lod])
+     return res
+
+   # new_array is the transformed Tensor above
+   lod_tensor = to_lodtensor(new_array,fluid.CPUPlace())
+
+   # output LoD information
+   print("The LoD of the result: {}.".format(lod_tensor.lod()))
+
+   # examine the consistency with Tensor data
+   print("The array : {}.".format(np.array(lod_tensor)))
+
+
+
+
+
+Code examples
+==============
+
+Input variable x is expanded according to specified layer level y-lod in the code example in this section. The example below contains some fundamental conception of LoD-Tensor. By following the code, you will
+
+-  Have a direct understanding of the implementation of :code:`fluid.layers.sequence_expand` in Fluid
+-  Know how to create LoD-Tensor in Fluid
+-  Learn how to print the content of LoDTensor
+
+
+  
+**Define the Process of Computing**
+
+layers.sequence_expand expands x by obtaining the lod value of y. About more explanation of :code:`fluid.layers.sequence_expand` , please read :ref:`api_fluid_layers_sequence_expand` first. 
+
+Code of sequence expanding:
+
+.. code-block:: python
+
+  x = fluid.layers.data(name='x', shape=[1], dtype='float32', lod_level=1)
+  y = fluid.layers.data(name='y', shape=[1], dtype='float32', lod_level=2)
+  out = fluid.layers.sequence_expand(x=x, y=y, ref_level=0)
+
+*Note*：The dimension of input LoD-Tensor is only associated with the dimension of real data transferred in. The shape value set for x and y in the definition of network structure is just a placeholder with little influence on the result.  
+
+**Create Executor**
+
+.. code-block:: python
+
+  place = fluid.CPUPlace()
+  exe = fluid.Executor(place)
+  exe.run(fluid.default_startup_program())
+
+**Prepare Data**
+
+Here we use :code:`fluid.create_lod_tensor` to create the input data of :code:`sequence_expand` and expand x_d by defining LoD of y_d. The output value is only associated with LoD of y_d. And the data of y_d is not invovled in the process of computation. The dimension of y_d must keep consistent with as its LoD[-1] .
+
+About the user guide of :code:`fluid.create_lod_tensor()` , please refer to :ref:`api_fluid_create_lod_tensor` .
+
+Code：
+
+.. code-block:: python
+
+  x_d = fluid.create_lod_tensor(np.array([[1.1],[2.2],[3.3],[4.4]]).astype('float32'), [[1,3]], place)
+  y_d = fluid.create_lod_tensor(np.array([[1.1],[1.1],[1.1],[1.1],[1.1],[1.1]]).astype('float32'), [[1,3], [2,1,2,1]],place)
+
+
+**Execute Computing**
+
+For tensor whose LoD > 1 in Fluid, like data of other types, the order of transfering data is defined by :code:`feed` . In addition, parameter :code:`return_numpy=False` needs to be added to exe.run() to get the output of LoD-Tensor because results are Tensors with LoD information.
+
+.. code-block:: python
+
+  results = exe.run(fluid.default_main_program(),
+                    feed={'x':x_d, 'y': y_d },
+                    fetch_list=[out],return_numpy=False)
+
+**Check the result of LodTensor**
+
+Because of the special attributes of LoDTensor, you could not print to check the content. The usual solution to the problem is to fetch the LoDTensor as the output of network and then execute  numpy.array(lod_tensor) to transfer LoDTensor into numpy array: 
+
+.. code-block:: python
+
+  np.array(results[0])
+
+Output:
+
+.. code-block:: text
+
+  array([[1.1],[2.2],[3.3],[4.4],[2.2],[3.3],[4.4],[2.2],[3.3],[4.4]])
+
+**Check the length of sequence**
+
+You can get the recursive sequence length of LoDTensor by checking the sequence length:
+
+.. code-block:: python
+
+    results[0].recursive_sequence_lengths()
+    
+Output
+
+.. code-block:: text
+    
+    [[1L, 3L, 3L, 3L]]
+
+**Complete Code**
+
+You can check the output by executing the following complete code:
+
+.. code-block:: python
+    
+    #Load 
+    import paddle
+    import paddle.fluid as fluid
+    import numpy as np
+    #Define forward computation
+    x = fluid.layers.data(name='x', shape=[1], dtype='float32', lod_level=1)
+    y = fluid.layers.data(name='y', shape=[1], dtype='float32', lod_level=2)
+    out = fluid.layers.sequence_expand(x=x, y=y, ref_level=0)
+    #Define place for computation
+    place = fluid.CPUPlace()
+    #Create executer
+    exe = fluid.Executor(place)
+    exe.run(fluid.default_startup_program())
+    #Create LoDTensor
+    x_d = fluid.create_lod_tensor(np.array([[1.1], [2.2],[3.3],[4.4]]).astype('float32'), [[1,3]], place)
+    y_d = fluid.create_lod_tensor(np.array([[1.1],[1.1],[1.1],[1.1],[1.1],[1.1]]).astype('float32'), [[1,3], [1,2,1,2]], place)
+    #Start computing
+    results = exe.run(fluid.default_main_program(),
+                      feed={'x':x_d, 'y': y_d },
+                      fetch_list=[out],return_numpy=False)
+    #Output result
+    print("The data of the result: {}.".format(np.array(results[0])))
+    #print the length of sequence of result
+    print("The recursive sequence lengths of the result: {}.".format(results[0].recursive_sequence_lengths()))
+    #print the LoD of result
+    print("The LoD of the result: {}.".format(results[0].lod()))
+
+
+Summary
+========
+
+Then, we believe that you have known about the concept LoD-Tensor. And an attempt to change x_d and y_d in code above and then to check the output may help you get a better understanding of this flexible structure.
+
+About more model applications of LoDTensor, you can refer to `Word2vec <../../../beginners_guide/basics/word2vec/index_en.html>`_ , `Individual Recommendation <../../../beginners_guide/basics/recommender_system/index_en.html>`_ , `Sentiment Analysis <../../../beginners_guide/basics/understand_sentiment/index_en.html>`_ in the beginner's guide. 
+
+About more difffiult and complex examples of application, please refer to associated information about `models <../../../user_guides/models/index_en.html>`_ .
 
-This section will introduce basic concepts in Fluid：
-
-- `LoD-Tensor User Guide <lod_tensor_en.html>`_ : LoD-Tensor is a unique term of Fluid. It appends sequence information to Tensor，and supports data of variable lengths.
-
-..  toctree::
-    :hidden:
-
-    lod_tensor_en.rst

doc/fluid/user_guides/howto/prepare_data/index_cn.rst

@@ -4,127 +4,12 @@
 准备数据
 ########
 
-使用PaddlePaddle Fluid准备数据分为三个步骤：
-
-Step1: 自定义Reader生成训练/预测数据
-###################################
-
-生成的数据类型可以为Numpy Array或LoDTensor。根据Reader返回的数据形式的不同，可分为Batch级的Reader和Sample（样本）级的Reader。
-
-Batch级的Reader每次返回一个Batch的数据，Sample级的Reader每次返回单个样本的数据
-
-如果您的数据是Sample级的数据，我们提供了一个可以数据预处理和组建batch的工具：:code:`Python Reader` 。
-
-
-Step2: 在网络配置中定义数据层变量
-###################################
-用户需使用 :code:`fluid.layers.data` 在网络中定义数据层变量。定义数据层变量时需指明数据层的名称name、数据类型dtype和维度shape。例如：
-
-.. code-block:: python
-
-    import paddle.fluid as fluid
-
-    image = fluid.layers.data(name='image', dtype='float32', shape=[28, 28])
-    label = fluid.layers.data(name='label', dtype='int64', shape=[1])
-
-
-需要注意的是，此处的shape是单个样本的维度，PaddlePaddle Fluid会在shape第0维位置添加-1，表示batch_size的维度，即此例中image.shape为[-1, 28, 28]，
-label.shape为[-1, 1]。
-
-若用户不希望框架在第0维位置添加-1，则可通过append_batch_size=False参数控制，即：
-
-.. code-block:: python
-
-   import paddle.fluid as fluid
-
-   image = fluid.layers.data(name='image', dtype='float32', shape=[28, 28], append_batch_size=False)
-   label = fluid.layers.data(name='label', dtype='int64', shape=[1], append_batch_size=False)
-
-此时，image.shape为[28, 28]，label.shape为[1]。
-
-Step3: 将数据送入网络进行训练/预测
-###################################
-
-Fluid提供两种方式，分别是异步PyReader接口方式或同步Feed方式，具体介绍如下：
-
-- 异步PyReader接口方式
-
-用户需要先使用 :code:`fluid.io.PyReader` 定义PyReader对象，然后通过PyReader对象的decorate方法设置数据源。
-使用PyReader接口时，数据传入与模型训练/预测过程是异步进行的，效率较高，推荐使用。
-
-- 同步Feed方式
-
-用户自行构造输入数据，并在 :code:`fluid.Executor` 或 :code:`fluid.ParallelExecutor`
-中使用 :code:`executor.run(feed=...)` 传入训练数据。数据准备和模型训练/预测的过程是同步进行的，
-效率较低。
-
-
-这两种准备数据方法的比较如下:
-
-========  =================================   =====================================
-对比项            同步Feed方式                          异步PyReader接口方式
-========  =================================   =====================================
-API接口     :code:`executor.run(feed=...)`          :code:`fluid.io.PyReader`
-数据格式         Numpy Array或LoDTensor               Numpy Array或LoDTensor
-数据增强          Python端使用其他库完成                  Python端使用其他库完成
-速度                     慢                                   快
-推荐用途                调试模型                              工业训练
-========  =================================   =====================================
-
-Reader数据类型对使用方式的影响
-###############################
-
-根据Reader数据类型的不同，上述步骤的具体操作将有所不同，具体介绍如下:
-
-读取Sample级Reader数据
-+++++++++++++++++++++
-
-若自定义的Reader每次返回单个样本的数据，用户需通过以下步骤完成数据送入：
-
-Step1. 组建数据
-=============================
-
-调用Fluid提供的Reader相关接口完成组batch和部分的数据预处理功能，具体请参见：
+本章详细介绍了如何为神经网络提供数据，包括数据的前期处理与后期的同步、异步读取。
 
 .. toctree::
    :maxdepth: 1
 
+   prepare_steps.rst
    reader_cn.md
-
-Step2. 送入数据
-=================================
-
-若使用异步PyReader接口方式送入数据，请调用 :code:`decorate_sample_generator` 或 :code:`decorate_sample_list_generator` 接口完成，具体请参见：
-
-- :ref:`user_guides_use_py_reader`
-
-若使用同步Feed方式送入数据，请使用DataFeeder接口将Reader数据转换为LoDTensor格式后送入网络，具体请参见 :ref:`cn_api_fluid_DataFeeder`
-
-读取Batch级Reader数据
-+++++++++++++++++++++++
-
-Step1. 组建数据
-=================
-
-由于Batch已经组好，已经满足了Step1的条件，可以直接进行Step2
-
-Step2. 送入数据
-=================================
-
-若使用异步PyReader接口方式送入数据，请调用PyReader的 :code:`decorate_batch_generator` 接口完成，具体方式请参见:
-
-.. toctree::
-   :maxdepth: 1
-
    use_py_reader.rst
-
-若使用同步Feed方式送入数据，具体请参见:
-
-.. toctree::
-   :maxdepth: 1
-
-   feeding_data.rst
-
-
-
-
+   feeding_data.rst
\ No newline at end of file

doc/fluid/user_guides/howto/prepare_data/index_en.rst

@@ -4,52 +4,12 @@
 Prepare Data
 #############
 
-PaddlePaddle Fluid supports two methods to feed data into networks:
-
-1. Synchronous method - Python Reader：Firstly, use :code:`fluid.layers.data` to set up data input layer. Then, feed in the training data through :code:`executor.run(feed=...)` in :code:`fluid.Executor` or :code:`fluid.ParallelExecutor` .
-
-2. Asynchronous method - py_reader：Firstly, use :code:`fluid.layers.py_reader` to set up data input layer. Then configure the data source with functions :code:`decorate_paddle_reader` or :code:`decorate_tensor_provider` of :code:`py_reader` . After that, call :code:`fluid.layers.read_file` to read data.
-
-
-
-Comparisons of the two methods:
-
-=========================  ====================================================   ===============================================
-Aspects                                   Synchronous Python Reader                       Asynchronous py_reader
-=========================  ====================================================   ===============================================
-API interface                          :code:`executor.run(feed=...)`                 :code:`fluid.layers.py_reader`
-data type                                   Numpy Array                                Numpy Array or LoDTensor
-data augmentation          carried out by other libraries on Python end            carried out by other libraries on Python end 
-velocity                                        slow                                            rapid
-recommended applications                model debugging                                      industrial training
-=========================  ====================================================   ===============================================
-
-Synchronous Python Reader
-##########################
-
-Fluid provides Python Reader to feed in data.
-
-Python Reader is a pure Python-side interface, and data feeding is synchronized with the model training/prediction process. Users can pass in data through Numpy Array. For specific operations, please refer to:
-
-
-.. toctree::
-   :maxdepth: 1
-
-   feeding_data_en.rst
-
-Python Reader supports advanced functions like group batch, shuffle. For specific operations, please refer to：
+This document mainly introduces how to provide data for the network, including Synchronous-method and Asynchronous-method.
 
 .. toctree::
    :maxdepth: 1
 
+   prepare_steps_en.rst
    reader.md
-
-Asynchronous py_reader
-########################
-
-Fluid provides asynchronous data feeding method PyReader. It is more efficient as data feeding is not synchronized with the model training/prediction process. For specific operations, please refer to：
-
-.. toctree::
-   :maxdepth: 1
-
    use_py_reader_en.rst
+   feeding_data_en.rst
\ No newline at end of file

doc/fluid/user_guides/howto/prepare_data/prepare_steps.rst

+..  _user_guide_prepare_steps:
+
+########
+准备步骤
+########
+
+使用PaddlePaddle Fluid准备数据分为三个步骤：
+
+Step1: 自定义Reader生成训练/预测数据
+###################################
+
+生成的数据类型可以为Numpy Array或LoDTensor。根据Reader返回的数据形式的不同，可分为Batch级的Reader和Sample（样本）级的Reader。
+
+Batch级的Reader每次返回一个Batch的数据，Sample级的Reader每次返回单个样本的数据
+
+如果您的数据是Sample级的数据，我们提供了一个可以数据预处理和组建batch的工具：:code:`Python Reader` 。
+
+
+Step2: 在网络配置中定义数据层变量
+###################################
+用户需使用 :code:`fluid.layers.data` 在网络中定义数据层变量。定义数据层变量时需指明数据层的名称name、数据类型dtype和维度shape。例如：
+
+.. code-block:: python
+
+    import paddle.fluid as fluid
+
+    image = fluid.layers.data(name='image', dtype='float32', shape=[28, 28])
+    label = fluid.layers.data(name='label', dtype='int64', shape=[1])
+
+
+需要注意的是，此处的shape是单个样本的维度，PaddlePaddle Fluid会在shape第0维位置添加-1，表示batch_size的维度，即此例中image.shape为[-1, 28, 28]，
+label.shape为[-1, 1]。
+
+若用户不希望框架在第0维位置添加-1，则可通过append_batch_size=False参数控制，即：
+
+.. code-block:: python
+
+   import paddle.fluid as fluid
+
+   image = fluid.layers.data(name='image', dtype='float32', shape=[28, 28], append_batch_size=False)
+   label = fluid.layers.data(name='label', dtype='int64', shape=[1], append_batch_size=False)
+
+此时，image.shape为[28, 28]，label.shape为[1]。
+
+Step3: 将数据送入网络进行训练/预测
+###################################
+
+Fluid提供两种方式，分别是异步PyReader接口方式或同步Feed方式，具体介绍如下：
+
+- 异步PyReader接口方式
+
+用户需要先使用 :code:`fluid.io.PyReader` 定义PyReader对象，然后通过PyReader对象的decorate方法设置数据源。
+使用PyReader接口时，数据传入与模型训练/预测过程是异步进行的，效率较高，推荐使用。
+
+- 同步Feed方式
+
+用户自行构造输入数据，并在 :code:`fluid.Executor` 或 :code:`fluid.ParallelExecutor`
+中使用 :code:`executor.run(feed=...)` 传入训练数据。数据准备和模型训练/预测的过程是同步进行的，
+效率较低。
+
+
+这两种准备数据方法的比较如下:
+
+========  =================================   =====================================
+对比项            同步Feed方式                          异步PyReader接口方式
+========  =================================   =====================================
+API接口     :code:`executor.run(feed=...)`          :code:`fluid.io.PyReader`
+数据格式         Numpy Array或LoDTensor               Numpy Array或LoDTensor
+数据增强          Python端使用其他库完成                  Python端使用其他库完成
+速度                     慢                                   快
+推荐用途                调试模型                              工业训练
+========  =================================   =====================================
+
+Reader数据类型对使用方式的影响
+###############################
+
+根据Reader数据类型的不同，上述步骤的具体操作将有所不同，具体介绍如下:
+
+读取Sample级Reader数据
++++++++++++++++++++++
+
+若自定义的Reader每次返回单个样本的数据，用户需通过以下步骤完成数据送入：
+
+Step1. 组建数据
+=============================
+
+调用Fluid提供的Reader相关接口完成组batch和部分的数据预处理功能，具体请参见：
+
+.. toctree::
+   :maxdepth: 1
+
+   reader_cn.md
+
+Step2. 送入数据
+=================================
+
+若使用异步PyReader接口方式送入数据，请调用 :code:`decorate_sample_generator` 或 :code:`decorate_sample_list_generator` 接口完成，具体请参见：
+
+- :ref:`user_guides_use_py_reader`
+
+若使用同步Feed方式送入数据，请使用DataFeeder接口将Reader数据转换为LoDTensor格式后送入网络，具体请参见 :ref:`cn_api_fluid_DataFeeder`
+
+读取Batch级Reader数据
++++++++++++++++++++++++
+
+Step1. 组建数据
+=================
+
+由于Batch已经组好，已经满足了Step1的条件，可以直接进行Step2
+
+Step2. 送入数据
+=================================
+
+若使用异步PyReader接口方式送入数据，请调用PyReader的 :code:`decorate_batch_generator` 接口完成，具体方式请参见:
+
+.. toctree::
+   :maxdepth: 1
+
+   use_py_reader.rst
+
+若使用同步Feed方式送入数据，具体请参见:
+
+.. toctree::
+   :maxdepth: 1
+
+   feeding_data.rst
+
+
+
+

doc/fluid/user_guides/howto/prepare_data/prepare_steps_en.rst

+..  _user_guide_prepare_steps_en:
+
+#############
+Prepare Steps
+#############
+
+PaddlePaddle Fluid supports two methods to feed data into networks:
+
+1. Synchronous method - Python Reader：Firstly, use :code:`fluid.layers.data` to set up data input layer. Then, feed in the training data through :code:`executor.run(feed=...)` in :code:`fluid.Executor` or :code:`fluid.ParallelExecutor` .
+
+2. Asynchronous method - py_reader：Firstly, use :code:`fluid.layers.py_reader` to set up data input layer. Then configure the data source with functions :code:`decorate_paddle_reader` or :code:`decorate_tensor_provider` of :code:`py_reader` . After that, call :code:`fluid.layers.read_file` to read data.
+
+
+
+Comparisons of the two methods:
+
+=========================  ====================================================   ===============================================
+Aspects                                   Synchronous Python Reader                       Asynchronous py_reader
+=========================  ====================================================   ===============================================
+API interface                          :code:`executor.run(feed=...)`                 :code:`fluid.layers.py_reader`
+data type                                   Numpy Array                                Numpy Array or LoDTensor
+data augmentation          carried out by other libraries on Python end            carried out by other libraries on Python end 
+velocity                                        slow                                            rapid
+recommended applications                model debugging                                      industrial training
+=========================  ====================================================   ===============================================
+
+Synchronous Python Reader
+##########################
+
+Fluid provides Python Reader to feed in data.
+
+Python Reader is a pure Python-side interface, and data feeding is synchronized with the model training/prediction process. Users can pass in data through Numpy Array. For specific operations, please refer to:
+
+
+.. toctree::
+   :maxdepth: 1
+
+   feeding_data_en.rst
+
+Python Reader supports advanced functions like group batch, shuffle. For specific operations, please refer to：
+
+.. toctree::
+   :maxdepth: 1
+
+   reader.md
+
+Asynchronous py_reader
+########################
+
+Fluid provides asynchronous data feeding method PyReader. It is more efficient as data feeding is not synchronized with the model training/prediction process. For specific operations, please refer to：
+
+.. toctree::
+   :maxdepth: 1
+
+   use_py_reader_en.rst

doc/fluid/user_guides/howto/prepare_data/use_py_reader_en.rst

@@ -4,7 +4,7 @@
 Use PyReader to read training and test data
 ############################################
 
-Besides Python Reader, we provide PyReader. The performance of PyReader is better than :ref:`user_guide_use_numpy_array_as_train_data` , because the process of loading data is asynchronous with the process of training model when PyReader is in use. And PyReader can coordinate with :code:`double_buffer_reader` to improve the performance of reading data. What's more, :code:`double_buffer_reader` can achieve the transformation from CPU Tensor to GPU Tensor, which improve the efficiency of reading data to some extent.
+Besides Python Reader, we provide PyReader. The performance of PyReader is better than :ref:`user_guide_use_numpy_array_as_train_data_en` , because the process of loading data is asynchronous with the process of training model when PyReader is in use. And PyReader can coordinate with :code:`double_buffer_reader` to improve the performance of reading data. What's more, :code:`double_buffer_reader` can achieve the transformation from CPU Tensor to GPU Tensor, which improve the efficiency of reading data to some extent.
 
 Create PyReader Object
 ################################

doc/fluid/user_guides/index_cn.rst

@@ -6,7 +6,7 @@
 
 如果您已经掌握了新手入门阶段的内容，期望可以针对实际问题建模、搭建自己网络，本模块提供了一些 Fluid 的使用细节供您参考：
 
-    - `基本概念 <../user_guides/howto/basic_concept/index_cn.html>`_ ：介绍了Fluid的基本使用概念
+    - `LoD-Tensor概念 <../user_guides/howto/basic_concept/index_cn.html>`_ ：介绍了Fluid LoD-Tensor的基本概念
 
     - `准备数据 <../user_guides/howto/prepare_data/index_cn.html>`_ ：介绍使用 Fluid 训练网络时，数据的支持类型及传输方法
 

doc/fluid/user_guides/index_en.rst

@@ -8,7 +8,7 @@ If you have got the hang of Beginner's Guide, and wish to model practical proble
 you with some detailed operations:
 
 
-    - `Basic Concepts <../user_guides/howto/basic_concept/index_en.html>`_ ：It explains basic concepts of Fluid.
+    - `LoD-Tensor Concepts <../user_guides/howto/basic_concept/index_en.html>`_ ：It explains basic concepts of Fluid LoD-Tensor.
 
     - `Prepare Data <../user_guides/howto/prepare_data/index_en.html>`_ ：This section introduces data types supported and data transmission methods when you are training your networks with Fluid.