Skip to content

F
FluidDoc

PaddlePaddle / FluidDoc

通知 7
Star 2
Fork 0
F
FluidDoc
未验证 提交 46946b6d 编写于 作者: D Double_V 提交者： GitHub
浏览文件
操作

Revert "add hapi readme and delete unused file in beginners_guide (#2132)" (#2135) 

This reverts commit 685da70d.
上级 685da70d
展开全部 隐藏空白更改
内联 并排
Showing with 5818 addition and 198 deletion
doc/fluid/beginners_guide/basic_concept/executor.rst 0 → 100644
浏览文件 @ 46946b6d
.. _cn_user_guide_Executor:
=======
Executor
=======
飞桨（PaddlePaddle，以下简称Paddle）的设计思想类似于高级编程语言C++和JAVA等。程序的执行过程被分为编译和执行两个阶段。
用户完成对 Program 的定义后，Executor 接受这段 Program 并转化为C++后端真正可执行的 FluidProgram，这一自动完成的过程叫做编译。
编译过后需要 Executor 来执行这段编译好的 FluidProgram。
例如上文实现的加法运算，当构建好 Program 后，需要创建 Executor，执行startup Program 和训练 Program：
.. code-block:: python
import paddle.fluid as fluid
import numpy
a = fluid.data(name="a",shape=[1],dtype='float32')
b = fluid.data(name="b",shape=[1],dtype='float32')
result = fluid.layers.elementwise_add(a,b)
# 定义执行器，并且制定执行的设备为CPU
cpu = fluid.core.CPUPlace()
exe = fluid.Executor(cpu)
exe.run(fluid.default_startup_program())
x = numpy.array([5]).astype("float32")
y = numpy.array([7]).astype("float32")
outs = exe.run(
feed={'a':x,'b':y},
fetch_list=[result])
# 打印输出结果，[array([12.], dtype=float32)]
print( outs )
doc/fluid/beginners_guide/basic_concept/index_cn.rst 0 → 100644
浏览文件 @ 46946b6d
############
基本概念
############
本文介绍飞桨核心框架中的基本概念：
- `编程指南 <./programming_guide/programming_guide.html>`_ : 介绍飞桨的基本概念和使用方法。
- `Variable <variable.html>`_ : Variable表示变量，在飞桨中可以包含任何类型的值，在大多数情况下是一个Lod-Tensor。
- `Tensor <tensor.html>`_ : Tensor表示数据。
- `LoD-Tensor <lod_tensor.html>`_ : LoD-Tensor是飞桨的高级特性，它在Tensor基础上附加了序列信息，支持处理变长数据。
- `Operator <operator.html>`_ : Operator表示对数据的操作。
- `Program <program.html>`_ : Program表示对计算过程的描述。
- `Executor <executor.html>`_ : Executor表示执行引擎。
- `命令式编程模式(动态图)机制-DyGraph <./dygraph/DyGraph.html>`_ : 介绍飞桨命令式编程模式执行机制。
.. toctree::
:hidden:
programming_guide/programming_guide.md
variable.rst
tensor.rst
lod_tensor.rst
operator.rst
program.rst
executor.rst
dygraph/DyGraph.md
doc/fluid/beginners_guide/basic_concept/index_en.rst 0 → 100644
浏览文件 @ 46946b6d
############
Basic Concept
############
This paper introduces the basic concepts of Paddle:
- `Guide to Fluid Programming <./programming_guide/programming_guide_en.html>`_ :introduces the basic concept and usage of Paddle.
- `LoD-Tensor User Guide <lod_tensor_en.html>`_ : LoD-Tensor is a high-level feature of Paddle. It adds sequence information on the basis of tensor and supports processing variable length data.
.. toctree::
:hidden:
programming_guide/programming_guide_en.md
lod_tensor_en.rst
doc/fluid/beginners_guide/basic_concept/lod_tensor.rst 0 → 100644
浏览文件 @ 46946b6d
.. _cn_user_guide_lod_tensor:
=========
LoDTensor
=========
LoD(Level-of-Detail) Tensor是Paddle的高级特性，是对Tensor的一种扩充。LoDTensor通过牺牲灵活性来提升训练的效率。
**注：对于大部分用户来说，无需关注LoDTensor的用法。**
变长序列的解决方案
================
现在主流的训练框架都采用batch的训练方式，即一个batch中包含多个样本。在nlp的任务中，一个batch中包含N个句子，句子的长度可能会不一致，为了解决这种长度不一致问题，Paddle提供了两种解决方案：1）padding，即在句子的结尾（或开头）添加padding id（建议的方式）；2）LoDTensor，tensor中同时保存序列的长度信息。
对于padding的方式，会增加框架的计算量，但是对于大部分nlp任务，可以通过分桶、排序等机制，使得一个batch内的句子长度尽可能接近、能够降低padding id的比例，padding对于训练的计算量影响可以忽略。而且可以通过引入mask（记录哪些位置是padding id）信息，来移除padding id对于训练效果的影响。
但是对于一部分nlp任务来说，一个batch内的句子长度无法做到接近，比如聊天任务，需要计算query和多个答案之间的相似度，答案必须在一个batch中，这些答案的长度差异可能会非常大，最长的几百个token，最短的10几个token，如果采用padding的方式，计算量会增加几十倍，这种场景非常适合LoDTensor。LoDTensor存储了样本的长度信息，不需要增加padding的词，能给大幅减少计算量，从而提高训练的速度。
LoDTensor将长度不一致的维度拼接为一个大的维度，并引入了一个索引数据结构（LoD）来将张量分割成序列。LoDTensor进行了维度拼接之后，rank大小和之前padding的方式不一致，在一些运算（如dot attention）逻辑比padding方式要复杂。
**注：如果训练样本无法通过排序、分桶等手段，使得一个batch内的样本的长度非常接近，推荐用户使用LoDTensor；其他情况下，建议用户使用padding的组网方式。**
LoD 索引
===========
为了更好的理解LoD的概念，本节提供了几个例子供您参考：
**句子组成的 mini-batch**
假设一个mini-batch中有3个句子，每个句子中分别包含3个、1个和2个单词。我们可以用(3+1+2)xD维Tensor 加上一些索引信息来表示这个mini-batch:
.. code-block :: text
3 1 2
| | | | | |
上述表示中，每一个 :code:`|` 代表一个D维的词向量，数字3，1，2构成了 1-level LoD。
**递归序列**
让我们来看另一个2-level LoD-Tensor的例子：假设存在一个mini-batch中包含3个句子、1个句子和2个句子的文章，每个句子都由不同数量的单词组成，则这个mini-batch的样式可以看作：
.. code-block:: text
3 1 2
3 2 4 1 2 3
||| || |||| | || |||
表示的LoD信息为：
.. code-block:: text
[[3，1，2]/*level=0*/，[3，2，4，1，2，3]/*level=1*/]
**视频的mini-batch**
在视觉任务中，时常需要处理视频和图像这些元素是高维的对象，假设现存的一个mini-batch包含3个视频，分别有3个，1个和2个帧，每个帧都具有相同大小：640x480，则这个mini-batch可以被表示为：
.. code-block:: text
3 1 2
口口口 口 口口
最底层tensor大小为（3+1+2）x640x480，每一个 :code:`口` 表示一个640x480的图像
**图像的mini-batch**
在传统的情况下，比如有N个固定大小的图像的mini-batch，LoD-Tensor表示为:
.. code-block:: text
1 1 1 1 1
口口口口 ... 口
在这种情况下，我们不会因为索引值都为1而忽略信息，仅仅把LoD-Tensor看作是一个普通的张量:
.. code-block:: text
口口口口 ... 口
**模型参数**
模型参数只是一个普通的张量，在Fluid中它们被表示为一个0-level LoD-Tensor。
LoDTensor的偏移表示
=====================
为了快速访问基本序列，Fluid提供了一种偏移表示的方法——保存序列的开始和结束元素，而不是保存长度。
在上述例子中，您可以计算基本元素的长度：
.. code-block:: text
3 2 4 1 2 3
将其转换为偏移表示：
.. code-block:: text
0 3 5 9 10 12 15
= = = = = =
3 2+3 4+5 1+9 2+10 3+12
所以我们知道第一个句子是从单词0到单词3，第二个句子是从单词3到单词5。
类似的，LoD的顶层长度
.. code-block:: text
3 1 2
可以被转化成偏移形式：
.. code-block:: text
0 3 4 6
= = =
3 3+1 4+2
因此该LoD-Tensor的偏移表示为：
.. code-block:: text
0 3 4 6
3 5 9 10 12 15
LoD-Tensor
=============
一个LoD-Tensor可以被看作是一个树的结构，树叶是基本的序列元素，树枝作为基本元素的标识。
在 Fluid 中 LoD-Tensor 的序列信息有两种表述形式：原始长度和偏移量。在 Paddle 内部采用偏移量的形式表述 LoD-Tensor，以获得更快的序列访问速度；在 python API中采用原始长度的形式表述 LoD-Tensor 方便用户理解和计算，并将原始长度称为： :code:`recursive_sequence_lengths` 。
以上文提到的一个2-level LoD-Tensor为例：
.. code-block:: text
3 1 2
3 2 4 1 2 3
||| || |||| | || |||
- 以偏移量表示此 LoD-Tensor:[ [0,3,4,6] , [0,3,5,9,10,12,15] ]，
- 以原始长度表达此 Lod-Tensor：recursive_sequence_lengths=[ [3-0 , 4-3 , 6-4] , [3-0 , 5-3 , 9-5 , 10-9 , 12-10 , 15-12] ]。
以文字序列为例： [3,1,2] 可以表示这个mini-batch中有3篇文章，每篇文章分别有3、1、2个句子，[3,2,4,1,2,3] 表示每个句子中分别含有3、2、4、1、2、3个字。
recursive_seq_lens 是一个双层嵌套列表，也就是列表的列表，最外层列表的size表示嵌套的层数，也就是lod-level的大小；内部的每个列表，对应表示每个lod-level下，每个元素的大小。
下面三段代码分别介绍如何创建一个LoD-Tensor，如何将LoD-Tensor转换成Tensor，如何将Tensor转换成LoD-Tensor：
* 创建 LoD-Tensor
.. code-block:: python
#创建lod-tensor
import paddle.fluid as fluid
import numpy as np
a = fluid.create_lod_tensor(np.array([[1],[1],[1],
[1],[1],
[1],[1],[1],[1],
[1],
[1],[1],
[1],[1],[1]]).astype('int64') ,
[[3,1,2] , [3,2,4,1,2,3]],
fluid.CPUPlace())
#查看lod-tensor嵌套层数
print (len(a.recursive_sequence_lengths()))
# output：2
#查看最基础元素个数
print (sum(a.recursive_sequence_lengths()[-1]))
# output:15 (3+2+4+1+2+3=15)
* LoD-Tensor 转 Tensor
.. code-block:: python
import paddle.fluid as fluid
import numpy as np
# 创建一个 LoD-Tensor
a = fluid.create_lod_tensor(np.array([[1.1], [2.2],[3.3],[4.4]]).astype('float32'), [[1,3]], fluid.CPUPlace())
def LodTensor_to_Tensor(lod_tensor):
# 获取 LoD-Tensor 的 lod 信息
lod = lod_tensor.lod()
# 转换成 array
array = np.array(lod_tensor)
new_array = []
# 依照原LoD-Tensor的层级信息，转换成Tensor
for i in range(len(lod[0]) - 1):
new_array.append(array[lod[0][i]:lod[0][i + 1]])
return new_array
new_array = LodTensor_to_Tensor(a)
# 输出结果
print(new_array)
* Tensor 转 LoD-Tensor
.. code-block:: python
import paddle.fluid as fluid
import numpy as np
def to_lodtensor(data, place):
# 存储Tensor的长度作为LoD信息
seq_lens = [len(seq) for seq in data]
cur_len = 0
lod = [cur_len]
for l in seq_lens:
cur_len += l
lod.append(cur_len)
# 对待转换的 Tensor 降维
flattened_data = np.concatenate(data, axis=0).astype("float32")
flattened_data = flattened_data.reshape([len(flattened_data), 1])
# 为 Tensor 数据添加lod信息
res = fluid.LoDTensor()
res.set(flattened_data, place)
res.set_lod([lod])
return res
# new_array 为上段代码中转换的Tensor
lod_tensor = to_lodtensor(new_array,fluid.CPUPlace())
# 输出 LoD 信息
print("The LoD of the result: {}.".format(lod_tensor.lod()))
# 检验与原Tensor数据是否一致
print("The array : {}.".format(np.array(lod_tensor)))
代码示例
===========
本节代码将根据指定的级别y-lod，扩充输入变量x。本例综合了LoD-Tensor的多个重要概念，跟随代码实现，您将：
- 直观理解Fluid中 :code:`fluid.layers.sequence_expand` 的实现过程
- 掌握如何在Fluid中创建LoD-Tensor
- 学习如何打印LoDTensor内容
**定义计算过程**
layers.sequence_expand通过获取 y 的 lod 值对 x 的数据进行扩充，关于 :code:`fluid.layers.sequence_expand` 的功能说明，请先阅读 :ref:`cn_api_fluid_layers_sequence_expand` 。
序列扩充代码实现：
.. code-block:: python
x = fluid.layers.data(name='x', shape=[1], dtype='float32', lod_level=1)
y = fluid.layers.data(name='y', shape=[1], dtype='float32', lod_level=2)
out = fluid.layers.sequence_expand(x=x, y=y, ref_level=0)
*说明*：输出LoD-Tensor的维度仅与传入的真实数据维度有关，在定义网络结构阶段为x、y设置的shape值，仅作为占位，并不影响结果。
**创建Executor**
.. code-block:: python
place = fluid.CPUPlace()
exe = fluid.Executor(place)
exe.run(fluid.default_startup_program())
**准备数据**
这里我们调用 :code:`fluid.create_lod_tensor` 创建 :code:`sequence_expand` 的输入数据，通过定义 y_d 的 LoD 值，对 x_d 进行扩充。其中，输出值只与 y_d 的 LoD 值有关，y_d 的 data 值在这里并不参与计算，维度上与LoD[-1]一致即可。
:code:`fluid.create_lod_tensor()` 的使用说明请参考 :ref:`cn_api_fluid_create_lod_tensor` 。
实现代码如下：
.. code-block:: python
x_d = fluid.create_lod_tensor(np.array([[1.1],[2.2],[3.3],[4.4]]).astype('float32'), [[1,3]], place)
y_d = fluid.create_lod_tensor(np.array([[1.1],[1.1],[1.1],[1.1],[1.1],[1.1]]).astype('float32'), [[1,3], [2,1,2,1]],place)
**执行运算**
在Fluid中，LoD>1的Tensor与其他类型的数据一样，使用 :code:`feed` 定义数据传入顺序。此外，由于输出results是带有LoD信息的Tensor，需在exe.run( )中添加 :code:`return_numpy=False` 参数，获得LoD-Tensor的输出结果。
.. code-block:: python
results = exe.run(fluid.default_main_program(),
feed={'x':x_d, 'y': y_d },
fetch_list=[out],return_numpy=False)
**查看LoDTensor结果**
由于LoDTensor的特殊属性，无法直接print查看内容，常用操作时将LoD-Tensor作为网络的输出fetch出来，然后执行 numpy.array(lod_tensor), 就能转成numpy array：
.. code-block:: python
np.array(results[0])
输出结果为：
.. code-block:: text
array([[1.1],[2.2],[3.3],[4.4],[2.2],[3.3],[4.4],[2.2],[3.3],[4.4]])
**查看序列长度**
可以通过查看序列长度得到 LoDTensor 的递归序列长度：
.. code-block:: python
results[0].recursive_sequence_lengths()
输出结果为：
.. code-block:: text
[[1L, 3L, 3L, 3L]]
**完整代码**
您可以运行下列完整代码，观察输出结果：
.. code-block:: python
#加载库
import paddle
import paddle.fluid as fluid
import numpy as np
#定义前向计算
x = fluid.layers.data(name='x', shape=[1], dtype='float32', lod_level=1)
y = fluid.layers.data(name='y', shape=[1], dtype='float32', lod_level=2)
out = fluid.layers.sequence_expand(x=x, y=y, ref_level=0)
#定义运算场所
place = fluid.CPUPlace()
#创建执行器
exe = fluid.Executor(place)
exe.run(fluid.default_startup_program())
#创建LoDTensor
x_d = fluid.create_lod_tensor(np.array([[1.1], [2.2],[3.3],[4.4]]).astype('float32'), [[1,3]], place)
y_d = fluid.create_lod_tensor(np.array([[1.1],[1.1],[1.1],[1.1],[1.1],[1.1]]).astype('float32'), [[1,3], [1,2,1,2]], place)
#开始计算
results = exe.run(fluid.default_main_program(),
feed={'x':x_d, 'y': y_d },
fetch_list=[out],return_numpy=False)
#输出执行结果
print("The data of the result: {}.".format(np.array(results[0])))
#输出 result 的序列长度
print("The recursive sequence lengths of the result: {}.".format(results[0].recursive_sequence_lengths()))
#输出 result 的 LoD
print("The LoD of the result: {}.".format(results[0].lod()))
FAQ：
=======
问：如何打印variable的lod 信息
答：
1. 可以使用 `executor.run` 将你需要查看的 `variable` fetch 出来，然后打印其 lod 信息，注意运行时设置 `executor.run` 方法的 `return_numpy` 参数为 `False`。
.. code-block:: python
results = exe.run(fluid.default_main_program(),
feed={'x':x_d, 'y': y_d },
fetch_list=[out],return_numpy=False)
lod_tensor = results[0]
print (lod_tensor.lod())
2. 可以使用fluid.layers.Print()
.. code-block:: python
y = fluid.layers.data(name='y', shape=[1], dtype='float32', lod_level=2)
fluid.layers.Print(y)
总结
========
至此，相信您已经基本掌握了LoD-Tensor的概念，尝试修改上述代码中的 x_d 与 y_d，观察输出结果，有助于您更好的理解这一灵活的结构。
更多LoDTensor的模型应用，可以参考新手入门中的 `词向量 <../../../beginners_guide/basics/word2vec/index.html>`_ 、`个性化推荐 <../../../beginners_guide/basics/recommender_system/index.html>`_、`情感分析 <../../../beginners_guide/basics/understand_sentiment/index.html>`_ 等指导教程。
更高阶的应用案例，请参考 `模型库 <../../../user_guides/models/index_cn.html>`_ 中的相关内容。
doc/fluid/beginners_guide/basic_concept/lod_tensor_en.rst 0 → 100644
浏览文件 @ 46946b6d
.. _user_guide_lod_tensor:
#####################
LoD-Tensor User Guide
#####################
LoD(Level-of-Detail) Tensor is a unique term in Fluid, which can be constructed by appending sequence information to Tensor. Data transferred in Fluid contain input, output and learnable parameters of the network, all of which are represented by LoD-Tensor.
With the help of this user guide, you will learn the design idea of LoD-Tensor in Fluid so that you can use such a data type more flexibly.
Challenge of variable-length sequences
======================================
In most deep learning frameworks, a mini-batch is represented by Tensor.
For example, if there are 10 pictures in a mini-batch and the size of each picture is 32*32, the mini-batch will be a 10*32*32 Tensor.
Or in the NLP task, there are N sentences in a mini-batch and the length of each sentence is L. Every word is represented by a one-hot vector with D dimensions. Then the mini-batch can be represented by an N*L*D Tensor.
In the two examples above, the size of each sequence element remains the same. However, the data to be trained are variable-length sequences in many cases. For this scenario, method to be taken in most frameworks is to set a fixed length and sequence data shorter than the fixed length will be padded with 0 to reach the fixed length.
Owing to the LoD-Tensor in Fluid, it is not necessary to keep the lengths of sequence data in every mini-batch constant.Therefore tasks sensitive to sequence formats like NLP can also be finished without padding.
Index Data Structure (LoD) is introduced to Fluid to split Tensor into sequences.
Index Structure - LoD
======================
To have a better understanding of the concept of LoD, you can refer to the examples in this section.
**mini-batch consisting of sentences**
Suppose a mini-batch contains three sentences, and each contains 3, 1, 2 words respectively. Then the mini-batch can be represented by a (3+1+2)*D Tensor with some index information appended:
.. code-block :: text
3 1 2
| | | | | |
In the text above, each :code:`|` represents a word vector with D dimension and a 1-level LoD is made up of digits 3,1,2 .
**recursive sequence**
Take a 2-level LoD-Tensor for example, a mini-batch contains articles of 3 sentences, 1 sentence and 2 sentences. The number of words in every sentence is different. Then the mini-batch is formed as follows:
.. code-block:: text
3 1 2
3 2 4 1 2 3
||| || |||| | || |||
the LoD to express the format:
.. code-block:: text
[[3，1，2]/*level=0*/，[3，2，4，1，2，3]/*level=1*/]
**mini-batch consisting of video data**
In the task of computer vision, it usually needs to deal objects with high dimension like videos and pictures. Suppose a mini-batch contains 3 videos, which is composed of 3 frames, 1 frames, 2 frames respectively. The size of each frame is 640*480. Then the mini-batch can be described as:
.. code-block:: text
3 1 2
口口口 口 口口
The size of the tensor at the bottom is (3+1+2)*640*480. Every :code:`口` represents a 640*480 picture.
**mini-batch consisting of pictures**
Traditionally, for a mini-batch of N pictures with fixed size, LoD-Tensor is described as:
.. code-block:: text
1 1 1 1 1
口口口口 ... 口
Under such circumstance, we will consider LoD-Tensor as a common tensor instead of ignoring information because of the indices of all elements are 1.
.. code-block:: text
口口口口 ... 口
**model parameter**
model parameter is a common tensor which is described as a 0-level LoD-Tensor in Fluid.
LoDTensor expressed by offset
=============================
To have a quick access to the original sequence, you can take the offset expression method——store the first and last element of a sequence instead of its length.
In the example above, you can compute the length of fundamental elements:
.. code-block:: text
3 2 4 1 2 3
It is expressed by offset as follows:
.. code-block:: text
0 3 5 9 10 12 15
= = = = = =
3 2+3 4+5 1+9 2+10 3+12
Therefore we infer that the first sentence starts from word 0 to word 3 and the second sentence starts from word 3 to word 5.
Similarly, for the length of the top layer of LoD
.. code-block:: text
3 1 2
It can be expressed by offset:
.. code-block:: text
0 3 4 6
= = =
3 3+1 4+2
Therefore the LoD-Tensor is expressed by offset:
.. code-block:: text
0 3 4 6
3 5 9 10 12 15
LoD-Tensor
=============
A LoD-Tensor can be regarded as a tree of which the leaf is an original sequence element and branch is the flag of fundamental element.
There are two ways to express sequence information of LoD-Tensor in Fluid: primitive length and offset. LoD-Tensor is expressed by offset in Paddle to offer a quicker access to sequence;LoD-Tensor is expressed by primitive length in python API to make user understand and compute more easily. The primary length is named as :code:`recursive_sequence_lengths` .
Take a 2-level LoD-Tensor mentioned above as an example:
.. code-block:: text
3 1 2
3 2 4 1 2 3
||| || |||| | || |||
- LoD-Tensor expressed by offset: [ [0,3,4,6] , [0,3,5,9,10,12,15] ]
- LoD-Tensor expressed by primitive length: recursive_sequence_lengths=[ [3-0 , 4-3 , 6-4] , [3-0 , 5-3 , 9-5 , 10-9 , 12-10 , 15-12] ]
Take text sequence as an example,[3,1,2] indicates there are 3 articles in the mini-batch,which contains 3,1,2 sentences respectively.[3,2,4,1,2,3] indicates there are 3,2,4,1,2,3 words in sentences respectively.
recursive_seq_lens is a double Layer nested list, and in other words, the element of the list is list. The size of the outermost list represents the nested layers, namely the size of lod-level; Each inner list represents the size of each element in each lod-level.
The following three pieces of codes introduce how to create LoD-Tensor, how to transform LoD-Tensor to Tensor and how to transform Tensor to LoD-Tensor respectively:
* Create LoD-Tensor
.. code-block:: python
#Create lod-tensor
import paddle.fluid as fluid
import numpy as np
a = fluid.create_lod_tensor(np.array([[1],[1],[1],
[1],[1],
[1],[1],[1],[1],
[1],
[1],[1],
[1],[1],[1]]).astype('int64') ,
[[3,1,2] , [3,2,4,1,2,3]],
fluid.CPUPlace())
#Check lod-tensor nested layers
print (len(a.recursive_sequence_lengths()))
# output：2
#Check the number of the most fundamental elements
print (sum(a.recursive_sequence_lengths()[-1]))
# output:15 (3+2+4+1+2+3=15)
* Transform LoD-Tensor to Tensor
.. code-block:: python
import paddle.fluid as fluid
import numpy as np
# create LoD-Tensor
a = fluid.create_lod_tensor(np.array([[1.1], [2.2],[3.3],[4.4]]).astype('float32'), [[1,3]], fluid.CPUPlace())
def LodTensor_to_Tensor(lod_tensor):
# get lod information of LoD-Tensor
lod = lod_tensor.lod()
# transform into array
array = np.array(lod_tensor)
new_array = []
# transform to Tensor according to the layer information of the original LoD-Tensor
for i in range(len(lod[0]) - 1):
new_array.append(array[lod[0][i]:lod[0][i + 1]])
return new_array
new_array = LodTensor_to_Tensor(a)
# output the result
print(new_array)
* Transform Tensor to LoD-Tensor
.. code-block:: python
import paddle.fluid as fluid
import numpy as np
def to_lodtensor(data, place):
# save the length of Tensor as LoD information
seq_lens = [len(seq) for seq in data]
cur_len = 0
lod = [cur_len]
for l in seq_lens:
cur_len += l
lod.append(cur_len)
# decrease the dimention of transformed Tensor
flattened_data = np.concatenate(data, axis=0).astype("float32")
flattened_data = flattened_data.reshape([len(flattened_data), 1])
# add lod information to Tensor data
res = fluid.LoDTensor()
res.set(flattened_data, place)
res.set_lod([lod])
return res
# new_array is the transformed Tensor above
lod_tensor = to_lodtensor(new_array,fluid.CPUPlace())
# output LoD information
print("The LoD of the result: {}.".format(lod_tensor.lod()))
# examine the consistency with Tensor data
print("The array : {}.".format(np.array(lod_tensor)))
Code examples
==============
Input variable x is expanded according to specified layer level y-lod in the code example in this section. The example below contains some fundamental conception of LoD-Tensor. By following the code, you will
- Have a direct understanding of the implementation of :code:`fluid.layers.sequence_expand` in Fluid
- Know how to create LoD-Tensor in Fluid
- Learn how to print the content of LoDTensor
**Define the Process of Computing**
layers.sequence_expand expands x by obtaining the lod value of y. About more explanation of :code:`fluid.layers.sequence_expand` , please read :ref:`api_fluid_layers_sequence_expand` first.
Code of sequence expanding:
.. code-block:: python
x = fluid.layers.data(name='x', shape=[1], dtype='float32', lod_level=1)
y = fluid.layers.data(name='y', shape=[1], dtype='float32', lod_level=2)
out = fluid.layers.sequence_expand(x=x, y=y, ref_level=0)
*Note*：The dimension of input LoD-Tensor is only associated with the dimension of real data transferred in. The shape value set for x and y in the definition of network structure is just a placeholder with little influence on the result.
**Create Executor**
.. code-block:: python
place = fluid.CPUPlace()
exe = fluid.Executor(place)
exe.run(fluid.default_startup_program())
**Prepare Data**
Here we use :code:`fluid.create_lod_tensor` to create the input data of :code:`sequence_expand` and expand x_d by defining LoD of y_d. The output value is only associated with LoD of y_d. And the data of y_d is not invovled in the process of computation. The dimension of y_d must keep consistent with as its LoD[-1] .
About the user guide of :code:`fluid.create_lod_tensor()` , please refer to :ref:`api_fluid_create_lod_tensor` .
Code：
.. code-block:: python
x_d = fluid.create_lod_tensor(np.array([[1.1],[2.2],[3.3],[4.4]]).astype('float32'), [[1,3]], place)
y_d = fluid.create_lod_tensor(np.array([[1.1],[1.1],[1.1],[1.1],[1.1],[1.1]]).astype('float32'), [[1,3], [2,1,2,1]],place)
**Execute Computing**
For tensor whose LoD > 1 in Fluid, like data of other types, the order of transfering data is defined by :code:`feed` . In addition, parameter :code:`return_numpy=False` needs to be added to exe.run() to get the output of LoD-Tensor because results are Tensors with LoD information.
.. code-block:: python
results = exe.run(fluid.default_main_program(),
feed={'x':x_d, 'y': y_d },
fetch_list=[out],return_numpy=False)
**Check the result of LodTensor**
Because of the special attributes of LoDTensor, you could not print to check the content. The usual solution to the problem is to fetch the LoDTensor as the output of network and then execute numpy.array(lod_tensor) to transfer LoDTensor into numpy array:
.. code-block:: python
np.array(results[0])
Output:
.. code-block:: text
array([[1.1],[2.2],[3.3],[4.4],[2.2],[3.3],[4.4],[2.2],[3.3],[4.4]])
**Check the length of sequence**
You can get the recursive sequence length of LoDTensor by checking the sequence length:
.. code-block:: python
results[0].recursive_sequence_lengths()
Output
.. code-block:: text
[[1L, 3L, 3L, 3L]]
**Complete Code**
You can check the output by executing the following complete code:
.. code-block:: python
#Load
import paddle
import paddle.fluid as fluid
import numpy as np
#Define forward computation
x = fluid.layers.data(name='x', shape=[1], dtype='float32', lod_level=1)
y = fluid.layers.data(name='y', shape=[1], dtype='float32', lod_level=2)
out = fluid.layers.sequence_expand(x=x, y=y, ref_level=0)
#Define place for computation
place = fluid.CPUPlace()
#Create executer
exe = fluid.Executor(place)
exe.run(fluid.default_startup_program())
#Create LoDTensor
x_d = fluid.create_lod_tensor(np.array([[1.1], [2.2],[3.3],[4.4]]).astype('float32'), [[1,3]], place)
y_d = fluid.create_lod_tensor(np.array([[1.1],[1.1],[1.1],[1.1],[1.1],[1.1]]).astype('float32'), [[1,3], [1,2,1,2]], place)
#Start computing
results = exe.run(fluid.default_main_program(),
feed={'x':x_d, 'y': y_d },
fetch_list=[out],return_numpy=False)
#Output result
print("The data of the result: {}.".format(np.array(results[0])))
#print the length of sequence of result
print("The recursive sequence lengths of the result: {}.".format(results[0].recursive_sequence_lengths()))
#print the LoD of result
print("The LoD of the result: {}.".format(results[0].lod()))
Summary
========
Then, we believe that you have known about the concept LoD-Tensor. And an attempt to change x_d and y_d in code above and then to check the output may help you get a better understanding of this flexible structure.
About more model applications of LoDTensor, you can refer to `Word2vec <../../../beginners_guide/basics/word2vec/index_en.html>`_ , `Individual Recommendation <../../../beginners_guide/basics/recommender_system/index_en.html>`_ , `Sentiment Analysis <../../../beginners_guide/basics/understand_sentiment/index_en.html>`_ in the beginner's guide.
About more difffiult and complex examples of application, please refer to associated information about `models <../../../user_guides/models/index_en.html>`_ .
doc/fluid/beginners_guide/basic_concept/operator.rst 0 → 100644
浏览文件 @ 46946b6d
.. _cn_user_guide_Operator:
=======
Operator
=======
在飞桨（PaddlePaddle，以下简称Paddle）中，所有对数据的操作都由Operator表示
为了便于用户使用，在Python端，Paddle中的Operator被封装入 :code:`paddle.fluid.layers` ， :code:`paddle.fluid.nets` 等模块。
因为一些常见的对Tensor的操作可能是由更多基础操作构成，为了提高使用的便利性，框架内部对基础 Operator 进行了一些封装，包括创建 Operator 依赖可学习参数，可学习参数的初始化细节等，减少用户重复开发的成本。
例如用户可以利用 :code:`paddle.fluid.layers.elementwise_add()` 实现两个输入Tensor的加法运算：
.. code-block:: python
import paddle.fluid as fluid
import numpy
a = fluid.data(name="a",shape=[1],dtype='float32')
b = fluid.data(name="b",shape=[1],dtype='float32')
result = fluid.layers.elementwise_add(a,b)
# 定义执行器，并且制定执行的设备为CPU
cpu = fluid.core.CPUPlace()
exe = fluid.Executor(cpu)
exe.run(fluid.default_startup_program())
x = numpy.array([5]).astype("float32")
y = numpy.array([7]).astype("float32")
outs = exe.run(
feed={'a':x,'b':y},
fetch_list=[result])
# 打印输出结果，[array([12.], dtype=float32)]
print( outs )
如果想获取网络执行过程中的a，b的具体值，可以将希望查看的变量添加在fetch_list中。
.. code-block:: python
#执行计算
outs = exe.run(
feed={'a':x,'b':y},
fetch_list=[a,b,result])
#查看输出结果
print( outs )
输出结果：
.. code-block:: python
[array([5.], dtype=float32), array([7.], dtype=float32), array([12.], dtype=float32)]
doc/fluid/beginners_guide/basic_concept/program.rst 0 → 100644
浏览文件 @ 46946b6d
.. _cn_user_guide_Program:
=======
Program
=======
飞桨（PaddlePaddle，以下简称Paddle）用Program的形式动态描述整个计算过程。这种描述方式，兼具网络结构修改的灵活性和模型搭建的便捷性，在保证性能的同时极大地提高了框架对模型的表达能力。
用户定义Operator会被顺序的放入Program中，在网络搭建过程中，由于不能使用python 的控制流，Paddle通过同时提供分支和循环两类控制流op结构的支持，让用户可以通过组合描述任意复杂的模型。
**顺序执行：**
用户可以使用顺序执行的方式搭建网络：
.. code-block:: python
x = fluid.data(name='x',shape=[None, 13], dtype='float32')
y_predict = fluid.layers.fc(input=x, size=1, act=None)
y = fluid.layers.data(name='y', shape=[1], dtype='float32')
cost = fluid.layers.square_error_cost(input=y_predict, label=y)
**条件分支——switch、if else：**
Fluid 中有 switch 和 if-else 类来实现条件选择，用户可以使用这一执行结构在学习率调节器中调整学习率或其他希望的操作：
.. code-block:: python
lr = fluid.layers.tensor.create_global_var(
shape=[1],
value=0.0,
dtype='float32',
persistable=True,
name="learning_rate")
one_var = fluid.layers.fill_constant(
shape=[1], dtype='float32', value=1.0)
two_var = fluid.layers.fill_constant(
shape=[1], dtype='float32', value=2.0)
with fluid.layers.control_flow.Switch() as switch:
with switch.case(global_step == zero_var):
fluid.layers.tensor.assign(input=one_var, output=lr)
with switch.default():
fluid.layers.tensor.assign(input=two_var, output=lr)
关于 Padldle 中 Program 的详细设计思想，可以参考阅读 `Fluid设计思想 <../../advanced_guide/addon_development/design_idea/fluid_design_idea.html>`_ 。
更多 Paddle 中的控制流，可以参考阅读 `API文档 <../../../api_guides/low_level/layers/control_flow.html>`_ 。
doc/fluid/beginners_guide/basic_concept/programming_guide/programming_guide.md 0 → 100644
浏览文件 @ 46946b6d
# 编程指南
目前飞桨（PaddlePaddle，以下简称Paddle）已经同时支持命令式编程模式(动态图)和声明式编程模式(静态图)两种编程方式，
本文主要侧重于介绍声明式编程模式的编程方法，关于命令式编程模式编程方法，请参考[命令式编程模式机制-DyGraph](../dygraph/DyGraph.html)。
阅读完本文档，您将了解在Paddle声明式编程模式编程方式中，如何表示和定义数据变量，以及如何完整的组建一个深度学习网络并进行训练。
## 数据的表示和定义
Paddle和其他主流框架一样，使用Tensor数据结构来承载数据，包括模型中的可学习参数（如网络权重、偏置等），
网络中每一层的输入输出数据，常量数据等。
Tensor可以简单理解成一个多维数组，一般而言可以有任意多的维度。
不同的Tensor可以具有自己的数据类型和形状，同一Tensor中每个元素的数据类型是一样的，
Tensor的形状就是Tensor的维度。关于Tensor的详细介绍请参阅：[Tensor](../tensor.html) 。
在Paddle中我们使用 `fluid.data` 来创建数据变量， `fluid.data` 需要指定Tensor的形状信息和数据类型，
当遇到无法确定的维度时，可以将相应维度指定为None，如下面的代码片段所示：
```python
import paddle.fluid as fluid
# 定义一个数据类型为int64的二维数据变量x，x第一维的维度为3，第二个维度未知，要在程序执行过程中才能确定，因此x的形状可以指定为[3, None]
x = fluid.data(name="x", shape=[3, None], dtype="int64")
# 大多数网络都会采用batch方式进行数据组织，batch大小在定义时不确定，因此batch所在维度（通常是第一维）可以指定为None
batched_x = fluid.data(name="batched_x", shape=[None, 3, None], dtype='int64')
```
除 `fluid.data` 之外，我们还可以使用 `fluid.layers.fill_constant` 来创建常量，
如下代码将创建一个维度为[3, 4], 数据类型为int64的Tensor，其中所有元素均为16（value参数所指定的值）。
```python
import paddle.fluid as fluid
data = fluid.layers.fill_constant(shape=[3, 4], value=16, dtype='int64')
```
以上例子中，我们只使用了一种数据类型"int64"，即有符号64位整数数据类型，更多Paddle目前支持的数据类型请查看：[支持的数据类型](../../../advanced_guide/data_preparing/feeding_data.html#fluid)。
需要注意的是，在声明式编程模式编程方式中，上述定义的Tensor并不具有值（即使创建常量的时候指定了value），
它们仅表示将要执行的操作，在网络执行时（训练或者预测）才会进行真正的赋值操作，
如您直接打印上例代码中的data将会得对其信息的描述：
```python
print data
```
输出结果:
```
name: "fill_constant_0.tmp_0"
type {
type: LOD_TENSOR
lod_tensor {
tensor {
data_type: INT64
dims: 3
dims: 4
}
}
}
persistable: false
```
在网络执行过程中，获取Tensor数值有两种方式：方式一是利用 `paddle.fluid.layers.Print` 创建一个打印操作，
打印正在访问的Tensor。方式二是将Variable添加在fetch_list中。
方式一的代码实现如下所示：
```python
import paddle.fluid as fluid
data = fluid.layers.fill_constant(shape=[3, 4], value=16, dtype='int64')
data = fluid.layers.Print(data, message="Print data:")
place = fluid.CPUPlace()
exe = fluid.Executor(place)
exe.run(fluid.default_startup_program())
ret = exe.run()
```
运行时的输出结果：
```
1571742368 Print data: The place is:CPUPlace
Tensor[fill_constant_0.tmp_0]
shape: [3,4,]
dtype: x
data: 16,16,16,16,16,16,16,16,16,16,16,16,
```
方式二Fetch_list的详细过程会在后文展开描述。
## 数据读取
使用 `fluid.data` 创建数据变量之后，我们需要把网络执行所需要的数据读取到对应变量中，
具体的数据准备过程，请阅读[准备数据](../../../advanced_guide/data_preparing/index_cn.html)。
## 组建网络
在Paddle中，数据计算类API统一称为Operator（算子），简称OP，大多数OP在 `paddle.fluid.layers` 模块中提供。
例如用户可以利用 `paddle.fluid.layers.elementwise_add()` 实现两个输入Tensor的加法运算：
```python
# 定义变量
import paddle.fluid as fluid
a = fluid.data(name="a", shape=[None, 1], dtype='int64')
b = fluid.data(name="b", shape=[None, 1], dtype='int64')
# 组建网络（此处网络仅由一个操作构成，即elementwise_add）
result = fluid.layers.elementwise_add(a,b)
# 准备运行网络
cpu = fluid.CPUPlace() # 定义运算设备，这里选择在CPU下训练
exe = fluid.Executor(cpu) # 创建执行器
exe.run(fluid.default_startup_program()) # 网络参数初始化
# 读取输入数据
import numpy
data_1 = int(input("Please enter an integer: a="))
data_2 = int(input("Please enter an integer: b="))
x = numpy.array([[data_1]])
y = numpy.array([[data_2]])
# 运行网络
outs = exe.run(
feed={'a':x, 'b':y}, # 将输入数据x, y分别赋值给变量a，b
fetch_list=[result] # 通过fetch_list参数指定需要获取的变量结果
)
# 输出计算结果
print "%d+%d=%d" % (data_1,data_2,outs[0][0])
```
输出结果：
```
Please enter an integer: a=7
Please enter an integer: b=3
7+3=10
```
本次运行时，输入a=7，b=3，得到outs=10。
您可以复制这段代码在本地执行，根据指示输入其他数值观察计算结果。
如果想获取网络执行过程中的a，b的具体值，可以将希望查看的变量添加在fetch_list中。
```python
...
# 运行网络
outs = exe.run(
feed={'a':x, 'b':y}, # 将输入数据x, y分别赋值给变量a，b
fetch_list=[a, b, result] # 通过fetch_list参数指定需要获取的变量结果
)
# 输出计算结果
print outs
```
输出结果：
```
[array([[7]]), array([[3]]), array([[10]])]
```
## 组建更加复杂的网络
某些场景下，用户需要根据当前网络中的某些状态，来具体决定后续使用哪一种操作，或者重复执行某些操作。在命令式编程模式中，可以方便的使用Python的控制流语句（如for，if-else等）来进行条件判断，但是在声明式编程模式中，由于组网阶段并没有实际执行操作，也没有产生中间计算结果，因此无法使用Python的控制流语句来进行条件判断，为此声明式编程模式提供了多个控制流API来实现条件判断。这里以[fluid.layers.while_loop](https://www.paddlepaddle.org.cn/documentation/docs/zh/develop/api_cn/layers_cn/while_loop_cn.html)为例来说明如何在声明式编程模式中实现条件循环的操作。
while_loop API用于实现类似while/for的循环控制功能，使用一个callable的方法cond作为参数来表示循环的条件，只要cond的返回值为True，while_loop就会循环执行循环体body（也是一个callable的方法），直到 cond 的返回值为False。对于while_loop API的详细定义和具体说明请参考文档[fluid.layers.while_loop](https://www.paddlepaddle.org.cn/documentation/docs/zh/develop/api_cn/layers_cn/while_loop_cn.html)。
下面的例子中，使用while_loop API进行条件循环操作，其实现的功能相当于在python中实现如下代码：
```python
i = 0
ten = 10
while i < ten:
i = i + 1
print('i =', i)
```
在声明式编程模式中使用while_loop API实现以上代码的逻辑：
```python
# 该代码要求安装飞桨1.7+版本
# 该示例代码展示整数循环+1，循环10次，输出计数结果
import paddle.fluid as fluid
import paddle.fluid.layers as layers
# 定义cond方法，作为while_loop的判断条件
def cond(i, ten):
return i < ten
# 定义body方法，作为while_loop的执行体，只要cond返回值为True，while_loop就会一直调用该方法进行计算
# 由于在使用while_loop OP时，cond和body的参数都是由while_loop的loop_vars参数指定的，所以cond和body必须有相同数量的参数列表，因此body中虽然只需要i这个参数，但是仍然要保持参数列表个数为2，此处添加了一个dummy参数来进行"占位"
def body(i, dummy):
# 计算过程是对输入参数i进行自增操作，即 i = i + 1
i = i + 1
return i, dummy
i = layers.fill_constant(shape=[1], dtype='int64', value=0) # 循环计数器
ten = layers.fill_constant(shape=[1], dtype='int64', value=10) # 循环次数
out, ten = layers.while_loop(cond=cond, body=body, loop_vars=[i, ten]) # while_loop的返回值是一个tensor列表，其长度，结构，类型与loop_vars相同
exe = fluid.Executor(fluid.CPUPlace())
res = exe.run(fluid.default_main_program(), feed={}, fetch_list=out)
print(res) #[array([10])]
```
限于篇幅，上面仅仅用一个最简单的例子来说明如何在声明式编程模式中实现循环操作。循环操作在很多应用中都有着重要作用，比如NLP中常用的Transformer模型，在解码（生成）阶段的Beam Search算法中，需要使用循环操作来进行候选的选取与生成，可以参考[Transformer](https://github.com/PaddlePaddle/models/tree/develop/PaddleNLP/PaddleMT/transformer)模型的实现来进一步学习while_loop在复杂场景下的用法。
除while_loop之外，飞桨还提供fluid.layers.cond API来实现条件分支的操作，以及fluid.layers.switch_case和fluid.layers.case API来实现分支控制功能，具体用法请参考文档：[cond](https://www.paddlepaddle.org.cn/documentation/docs/zh/develop/api_cn/layers_cn/cond_cn.html)，[switch_case](https://www.paddlepaddle.org.cn/documentation/docs/zh/develop/api_cn/layers_cn/switch_case_cn.html)和[case](https://www.paddlepaddle.org.cn/documentation/docs/zh/develop/api_cn/layers_cn/case_cn.html#case)
## 一个完整的网络示例
一个典型的模型通常包含4个部分，分别是：输入数据定义，搭建网络（模型前向计算逻辑），定义损失函数，以及选择优化算法。
下面我们通过一个非常简单的数据预测网络（线性回归），来完整的展示如何使用Paddle声明式编程模式方式完成一个深度学习模型的组建和训练。
问题描述：给定一组数据 $<X,Y>$，求解出函数 $f$，使得 $y=f(x)$，其中$X$,$Y$均为一维张量。最终网络可以依据输入$x$，准确预测出$y_{\_predict}$。
1. 定义数据
假设输入数据X=[1 2 3 4]，Y=[2 4 6 8]，在网络中定义：
```python
# 定义X数值
train_data=numpy.array([[1.0], [2.0], [3.0], [4.0]]).astype('float32')
# 定义期望预测的真实值y_true
y_true = numpy.array([[2.0], [4.0], [6.0], [8.0]]).astype('float32')
```
2. 搭建网络（定义前向计算逻辑）
接下来需要定义预测值与输入的关系，本次使用一个简单的线性回归函数进行预测：
```python
# 定义输入数据类型
x = fluid.data(name="x", shape=[None, 1], dtype='float32')
y = fluid.data(name="y", shape=[None, 1], dtype='float32')
# 搭建全连接网络
y_predict = fluid.layers.fc(input=x, size=1, act=None)
```
3. 添加损失函数
完成模型搭建后，如何评估预测结果的好坏呢？我们通常在设计的网络中添加损失函数，以计算真实值与预测值的差。
在本例中，损失函数采用[均方差函数](https://en.wikipedia.org/wiki/Mean_squared_error)：
```python
cost = fluid.layers.square_error_cost(input=y_predict, label=y)
avg_cost = fluid.layers.mean(cost)
```
4. 网络优化
确定损失函数后，可以通过前向计算得到损失值，并根据损失值对网络参数进行更新，最简单的算法是随机梯度下降法：w=w−η⋅g，由 `fluid.optimizer.SGD` 实现：
```python
sgd_optimizer = fluid.optimizer.SGD(learning_rate=0.01)
sgd_optimizer.minimize(avg_cost)
```
让我们的网络训练100次，查看结果：
```python
# 加载库
import paddle.fluid as fluid
import numpy
# 定义输入数据
train_data=numpy.array([[1.0],[2.0],[3.0],[4.0]]).astype('float32')
y_true = numpy.array([[2.0],[4.0],[6.0],[8.0]]).astype('float32')
# 组建网络
x = fluid.data(name="x",shape=[None, 1],dtype='float32')
y = fluid.data(name="y",shape=[None, 1],dtype='float32')
y_predict = fluid.layers.fc(input=x,size=1,act=None)
# 定义损失函数
cost = fluid.layers.square_error_cost(input=y_predict,label=y)
avg_cost = fluid.layers.mean(cost)
# 选择优化方法
sgd_optimizer = fluid.optimizer.SGD(learning_rate=0.01)
sgd_optimizer.minimize(avg_cost)
# 网络参数初始化
cpu = fluid.CPUPlace()
exe = fluid.Executor(cpu)
exe.run(fluid.default_startup_program())
# 开始训练，迭代100次
for i in range(100):
outs = exe.run(
feed={'x':train_data, 'y':y_true},
fetch_list=[y_predict, avg_cost])
# 输出训练结果
print outs
```
输出结果:
```
[array([[2.2075021],
[4.1005487],
[5.9935956],
[7.8866425]], dtype=float32), array([0.01651453], dtype=float32)]
```
可以看到100次迭代后，预测值已经非常接近真实值了，损失值也下降到了0.0165。
恭喜您！已经成功完成了第一个简单网络的搭建，想尝试线性回归的进阶版——房价预测模型，请阅读：[线性回归](../../../user_guides/simple_case/fit_a_line/README.cn.html)。更多丰富的模型实例可以在[典型案例](../../../user_guides/index_cn.html)中找到。
<a name="what_next"></a>
## 进一步学习
如果您已经掌握了基本操作，可以进行下一阶段的学习了：
跟随这一教程将学习到如何对实际问题建模并使用Paddle构建模型：[配置简单的网络](../../coding_practice/configure_simple_model/index_cn.html)。
完成网络搭建后，可以开始在单机上训练您的网络了，详细步骤请参考[单机训练](../../coding_practice/single_node.html)。
除此之外，使用文档模块根据开发者的不同背景划分了三个学习阶段：[快速上手](../../index_cn.html)、[典型案例](../../../user_guides/index_cn.html)和[进阶指南](../../../advanced_guide/index_cn.html)。
如果您希望阅读更多场景下的应用案例，可以参考[典型案例](../../../user_guides/index_cn.html)。已经具备深度学习基础知识的用户，也可以从[进阶指南](../../../advanced_guide/index_cn.html)开始阅读。
doc/fluid/beginners_guide/basic_concept/programming_guide/programming_guide_en.md 0 → 100644
浏览文件 @ 46946b6d
# Guide to Fluid Programming
This document will instruct you to program and create a simple nueral network with Fluid API. From this guide, you will get the hang of:
- Core concepts of Fluid
- How to define computing process in Fluid
- How to run fluid operators with executor
- How to model practical problems logically
- How to call API（layers, datasets, loss functions, optimization methods and so on)
Before building model, you need to figure out several core concepts of Fluid at first:
## Express data with Tensor
Like other mainstream frameworks, Fluid uses Tensor to hold data.
All data transferred in neural network are Tensor which can simply be regarded as a multi-dimensional array. In general, the number of dimensions can be any. Tensor features its own data type and shape. Data type of each element in single Tensor is the same. And **the shape of Tensor** refers to the dimensions of Tensor.
Picture below visually shows Tensor with dimension from one to six:
<p align="center">
<img src="https://raw.githubusercontent.com/PaddlePaddle/FluidDoc/develop/doc/fluid/beginners_guide/image/tensor.jpg" width="400">
</p>
There are three special kinds of Tensor in Fluid:
**1. Learnable parameters of models**
The lifetime of learnable parameters (such as network weight, bias and so on) of model is equal to the time of training task. The parameters will be updated by optimization algorithms. We use Parameter, the derived class of Variable, to express parameters.
We can create learnable parameters with `fluid.layers.create_parameter` in Fluid:
```python
w = fluid.layers.create_parameter(name="w",shape=[1],dtype='float32')
```
In general, you don't need to explicitly create learnable parameters of network. Fluid encapsulates most fundamental computing modules in common networks. Take the fully connected model as a simplest example, The codes below create connection weight(W) and bias(bias) for fully connected layer with no need to explicitly call associated APIs of Parameter.
```python
import paddle.fluid as fluid
y = fluid.layers.fc(input=x, size=128, bias_attr=True)
```
**2. Input and Output Tensor**
The input data of the whole neural network is also a special Tensor in which the sizes of some dimensions can not be decided at the definition time of models. Such dimensions usually includes batch size, or width and height of image when such data formats in a mini-batch are not constant. Placeholders for these uncertain dimension are necessary at the definition phase of model.
`fluid.layers.data` is used to receive input data in Fluid, and it needs to be provided with the shape of input Tensor. When the shape is not certain, the correspondent dimension is defined as None.
The code below exemplifies the usage of `fluid.layers.data` :
```python
import paddle.fluid as fluid
#Define the dimension of x : [3,None]. What we could make sure is that the first dimension of x is 3.
#The second dimension is unknown and can only be known at runtime.
x = fluid.layers.data(name="x", shape=[3,None], dtype="int64")
#batch size doesn't have to be defined explicitly.
#Fluid will automatically assign zeroth dimension as batch size dimension and fill right number at runtime.
a = fluid.layers.data(name="a",shape=[3,4],dtype='int64')
#If the width and height of image are variable, we can define the width and height as None.
#The meaning of three dimensions of shape is channel, width of image, height of image respectively.
b = fluid.layers.data(name="image",shape=[3,None,None],dtype="float32")
```
dtype=“int64” indicates signed int 64 bits data. For more data types supported by Fluid, please refer to [Data types currently supported by Fluid](../../user_guides/howto/prepare_data/feeding_data_en.html#fluid).
**3. Constant Tensor**
`fluid.layers.fill_constant` is used to define constant Tensor in Fluid. You can define the shape, data type and value of Constant Tensor. Code is as follows:
```python
import paddle.fluid as fluid
data = fluid.layers.fill_constant(shape=[1], value=0, dtype='int64')
```
Notice that the tensor defined above is not assigned with values. It merely represents the operation to perform. If you print data directly, you will get information about the description of this data:
```python
print data
```
Output:
```
name: "fill_constant_0.tmp_0"
type {
type: LOD_TENSOR
lod_tensor {
tensor {
data_type: INT64
dims: 1
}
}
}
persistable: false
```
Specific output value will be shown at the runtime of Executor. There are two ways to get runtime Variable value. The first way is to use `paddle.fluid.layers.Print` to create a print op that will print the tensor being accessed. The second way is to add Variable to Fetch_list.
Code of the first way is as follows:
```python
import paddle.fluid as fluid
data = fluid.layers.fill_constant(shape=[1], value=0, dtype='int64')
data = fluid.layers.Print(data, message="Print data: ")
```
Output at the runtime of Executor:
```
1563874307 Print data: The place is:CPUPlace
Tensor[fill_constant_0.tmp_0]
shape: [1,]
dtype: x
data: 0,
```
For more information on how to use the Print API, please refer to [Print operator](https://www.paddlepaddle.org.cn/documentation/docs/en/1.5/api/layers/control_flow.html#print).
Detailed process of the second way Fetch_list will be explained later.
## Feed data
The method to feed data in Fluid:
You need to use `fluid.layers.data` to configure data input layer and use ``executor.run(feed=...)`` to feed training data into `fluid.Executor` or `fluid.ParallelExecutor` .
For specific preparation for data, please refer to [Preparation for data](../../../advanced_guide/data_preparing/index_en.html).
## Operators -- operations on data
All operations on data are achieved by Operators in Fluid.
To facilitate development, on Python end, Operators in Fluid are further encapsulated into `paddle.fluid.layers` , `paddle.fluid.nets` and other modules.
It is because some common operations for Tensor may be composed of many fundamental operations. To make it more convenient, fundamental Operators are encapsulated in Fluid to reduce repeated coding, including the creation of learnable parameters which Operator relies on, details about initialization of learnable parameters and so on.
For example, you can use `paddle.fluid.layers.elementwise_add()` to add up two input Tensor:
```python
#Define network
import paddle.fluid as fluid
a = fluid.layers.data(name="a",shape=[1],dtype='float32')
b = fluid.layers.data(name="b",shape=[1],dtype='float32')
result = fluid.layers.elementwise_add(a,b)
#Define Exector
cpu = fluid.core.CPUPlace() #define computing place. Here we choose to train on CPU
exe = fluid.Executor(cpu) #create executor
exe.run(fluid.default_startup_program()) #initialize network parameters
#Prepare data
import numpy
data_1 = int(input("Please enter an integer: a="))
data_2 = int(input("Please enter an integer: b="))
x = numpy.array([[data_1]])
y = numpy.array([[data_2]])
#Run computing
outs = exe.run(
feed={'a':x,'b':y},
fetch_list=[result.name])
#Verify result
print "%d+%d=%d" % (data_1,data_2,outs[0][0])
```
Output:
```
a=7
b=3
7+3=10
```
At runtime, input a=7,b=3, and you will get output=10.
You can copy the code, run it locally, input different numbers following the prompt instructions and check the computed result.
If you want to get the specific value of a,b at the runtime of network, you can add variables you want to check into ``fetch_list`` .
```python
...
#Run computing
outs = exe.run(
feed={'a':x,'b':y},
fetch_list=[a,b,result.name]
#Check output
print outs
```
Output:
```
[array([[7]]), array([[3]]), array([[10]])]
```
## Use Program to describe neural network model
Fluid is different from most other deep learning frameworks. In Fluid, static computing map is replaced by Program to dynamically describe the network. This dynamic method delivers both flexible modifications to network structure and convenience to build model. Moreover, the capability of expressing a model is enhanced significantly while the performance is guaranteed.
All Operators will be written into Program, which will be automatically transformed into a descriptive language named ProgramDesc in Fluid. It's like to write a general program to define Program. If you are an experienced developer, you can naturally apply the knowledge you have acquired on Fluid programming.
You can describe any complex model by combining sequential processes, branches and loops supported by Fluid.
**Sequential Process**
You can use sequential structure to build network:
```python
x = fluid.layers.data(name='x',shape=[13], dtype='float32')
y_predict = fluid.layers.fc(input=x, size=1, act=None)
y = fluid.layers.data(name='y', shape=[1], dtype='float32')
cost = fluid.layers.square_error_cost(input=y_predict, label=y)
```
**Conditional branch——switch,if else：**
Switch and if-else class are used to implement conditional branch in Fluid. You can use the structure to adjust learning rate in learning rate adapter or perform other operations :
```python
lr = fluid.layers.tensor.create_global_var(
shape=[1],
value=0.0,
dtype='float32',
persistable=True,
name="learning_rate")
one_var = fluid.layers.fill_constant(
shape=[1], dtype='float32', value=1.0)
two_var = fluid.layers.fill_constant(
shape=[1], dtype='float32', value=2.0)
with fluid.layers.control_flow.Switch() as switch:
with switch.case(global_step == zero_var):
fluid.layers.tensor.assign(input=one_var, output=lr)
with switch.default():
fluid.layers.tensor.assign(input=two_var, output=lr)
```
For detailed design principles of Program, please refer to [Design principle of Fluid](../../../advanced_guide/addon_development/design_idea/fluid_design_idea_en.html).
For more about control flow in Fluid, please refer to [Control Flow](../../api/layers.html#control-flow).
## Use Executor to run Program
The design principle of Fluid is similar to C++, JAVA and other advanced programming language. The execution of program is divided into two steps: compile and run.
Executor accepts the defined Program and transforms it to a real executable Fluid Program at the back-end of C++. This process performed automatically is the compilation.
After compilation, it needs Executor to run the compiled Fluid Program.
Take add operator above as an example, you need to create an Executor to initialize and train Program after the construction of Program:
```python
#define Executor
cpu = fluid.core.CPUPlace() #define computing place. Here we choose training on CPU
exe = fluid.Executor(cpu) #create executor
exe.run(fluid.default_startup_program()) #initialize Program
#train Program and start computing
#feed defines the order of data transferred to network in the form of dict
#fetch_list defines the output of network
outs = exe.run(
feed={'a':x,'b':y},
fetch_list=[result.name])
```
## Code example
So far, you have got a primary knowledge of core concepts in Fluid. Why not try to configure a simple network ? You can finish a very simple data prediction under the guide of the part if you are interested. If you have learned this part, you can skip this section and read [What's next](#what_next).
Firstly, define input data format, model structure,loss function and optimized algorithm logically. Then you need to use PaddlePaddle APIs and operators to implement the logic of model. A typical model mainly contains four parts. They are: definition of input data format; forward computing logic; loss function; optimization algorithm.
1. Problem
Given a pair of data $<X,Y>$，construct a function $f$ so that $y=f(x)$ . $X$ , $Y$ are both one dimensional Tensor. Network finally can predict $y_{\_predict}$ accurately according to input $x$.
2. Define data
Supposing input data X=[1 2 3 4]，Y=[2,4,6,8], make a definition in network:
```python
#define X
train_data=numpy.array([[1.0],[2.0],[3.0],[4.0]]).astype('float32')
#define ground-truth y_true expected to get from the model prediction
y_true = numpy.array([[2.0],[4.0],[6.0],[8.0]]).astype('float32')
```
3. Create network (define forward computing logic)
Next you need to define the relationship between the predicted value and the input. Take a simple linear regression function for example:
```python
#define input data type
x = fluid.layers.data(name="x",shape=[1],dtype='float32')
#create fully connected network
y_predict = fluid.layers.fc(input=x,size=1,act=None)
```
Now the network can predict output. Although the output is just a group of random numbers, which is far from expected results:
```python
#load library
import paddle.fluid as fluid
import numpy
#define data
train_data=numpy.array([[1.0],[2.0],[3.0],[4.0]]).astype('float32')
y_true = numpy.array([[2.0],[4.0],[6.0],[8.0]]).astype('float32')
#define predict function
x = fluid.layers.data(name="x",shape=[1],dtype='float32')
y_predict = fluid.layers.fc(input=x,size=1,act=None)
#initialize parameters
cpu = fluid.core.CPUPlace()
exe = fluid.Executor(cpu)
exe.run(fluid.default_startup_program())
#start training
outs = exe.run(
feed={'x':train_data},
fetch_list=[y_predict.name])
#observe result
print outs
```
Output:
```
[array([[0.74079144],
[1.4815829 ],
[2.2223744 ],
[2.9631658 ]], dtype=float32)]
```
4. Add loss function
After the construction of model, we need to evaluate the output result in order to make accurate predictions. How do we evaluate the result of prediction? We usually add loss function to network to compute the *distance* between ground-truth value and predict value.
In this example, we adopt [mean-square function](https://en.wikipedia.org/wiki/Mean_squared_error) as our loss function ：
```python
cost = fluid.layers.square_error_cost(input=y_predict, label=y)
avg_cost = fluid.layers.mean(cost)
```
Output predicted value and loss function after a process of computing:
```python
#load library
import paddle.fluid as fluid
import numpy
#define data
train_data=numpy.array([[1.0],[2.0],[3.0],[4.0]]).astype('float32')
y_true = numpy.array([[2.0],[4.0],[6.0],[8.0]]).astype('float32')
#define network
x = fluid.layers.data(name="x",shape=[1],dtype='float32')
y = fluid.layers.data(name="y",shape=[1],dtype='float32')
y_predict = fluid.layers.fc(input=x,size=1,act=None)
#define loss function
cost = fluid.layers.square_error_cost(input=y_predict,label=y)
avg_cost = fluid.layers.mean(cost)
#initialize parameters
cpu = fluid.core.CPUPlace()
exe = fluid.Executor(cpu)
exe.run(fluid.default_startup_program())
#start training
outs = exe.run(
feed={'x':train_data,'y':y_true},
fetch_list=[y_predict.name,avg_cost.name])
#observe output
print outs
```
Output:
```
[array([[0.9010564],
[1.8021128],
[2.7031693],
[3.6042256]], dtype=float32), array([9.057577], dtype=float32)]
```
We discover that the loss function after the first iteration of computing is 9.0, which shows there is a great improve space.
5. Optimization of network
After the definition of loss function,you can get loss value by forward computing and then get gradients of parameters with chain derivative method.
Parameters should be updated after you have obtained gradients. The simplest algorithm is random gradient algorithm: w=w−η⋅g,which is implemented by `fluid.optimizer.SGD`:
```python
sgd_optimizer = fluid.optimizer.SGD(learning_rate=0.01)
```
Let's train the network for 100 times and check the results:
```python
#load library
import paddle.fluid as fluid
import numpy
#define data
train_data=numpy.array([[1.0],[2.0],[3.0],[4.0]]).astype('float32')
y_true = numpy.array([[2.0],[4.0],[6.0],[8.0]]).astype('float32')
#define network
x = fluid.layers.data(name="x",shape=[1],dtype='float32')
y = fluid.layers.data(name="y",shape=[1],dtype='float32')
y_predict = fluid.layers.fc(input=x,size=1,act=None)
#define loss function
cost = fluid.layers.square_error_cost(input=y_predict,label=y)
avg_cost = fluid.layers.mean(cost)
#define optimization algorithm
sgd_optimizer = fluid.optimizer.SGD(learning_rate=0.01)
sgd_optimizer.minimize(avg_cost)
#initialize parameters
cpu = fluid.core.CPUPlace()
exe = fluid.Executor(cpu)
exe.run(fluid.default_startup_program())
##start training and iterate for 100 times
for i in range(100):
outs = exe.run(
feed={'x':train_data,'y':y_true},
fetch_list=[y_predict.name,avg_cost.name])
#observe result
print outs
```
Output:
```
[array([[2.2075021],
[4.1005487],
[5.9935956],
[7.8866425]], dtype=float32), array([0.01651453], dtype=float32)]
```
Now we discover that predicted value is nearly close to real value and the loss value descends from original value 9.05 to 0.01 after iteration for 100 times.
Congratulations! You have succeed to create a simple network. If you want to try advanced linear regression —— predict model of housing price, please read [linear regression](../../../user_guides/simple_case/fit_a_line/README.html). More examples of model can be found in [User Guides](../../../user_guides/index_en.html).
<a name="what_next"></a>
## What's next
If you have been familiar with fundamental operations, you can start your next journey to learn fluid:
You will learn how to build model for practical problem with fluid: [The configuration of simple network](../../coding_practice/configure_simple_model/index_en.html).
After the construction of network, you can start training your network in single node. For detailed procedures, please refer to [Single-node training](../../coding_practice/single_node_en.html).
In addition, there are three learning levels in documentation according to developer's background and experience: [Beginner's Guide](../../index_en.html) , [User Guides](../../../user_guides/index_en.html) and [Advanced Guide](../../../advanced_guide/index_en.html).
If you want to read examples in more application scenarios, you can go to [User Guides](../../../user_guides/index_en.html) .If you have learned basic knowledge of deep learning, you can read from [Advanced Guide](../../../advanced_guide/index_en.html).
doc/fluid/beginners_guide/basic_concept/tensor.rst 0 → 100644
浏览文件 @ 46946b6d
.. _cn_user_guide_tensor:
=========
Tensor
=========
飞桨（PaddlePaddle，以下简称Paddle）和其他框架一样，使用Tensor来表示数据。
在神经网络中传递的数据都是Tensor。Tensor可以简单理解成一个多维数组，一般而言可以有任意多的维度。
不同的Tensor可以具有自己的数据类型和形状，同一Tensor中每个元素的数据类型是一样的，Tensor的形状就是Tensor的维度。
下图直观地表示1～6维的Tensor：
.. image:: ../image/tensor.jpg
**Paddle 高级特性**
:ref:`Lod-Tensor <cn_user_guide_lod_tensor>`
对于一些任务中batch内样本大小不一致的问题，Paddle提供了两种解决方案：
1. padding， 将大小不一致的样本padding到同样的大小，这是一种常用且推荐的使用方式；
2. :ref:`Lod-Tensor <cn_user_guide_lod_tensor>` ，记录每一个样本的大小，减少无用的计算量，LoD 牺牲灵活性来提升性能。
如果一个batch内的样本无法通过分桶、排序等方式使得大小接近， 建议使用 :ref:`Lod-Tensor <cn_user_guide_lod_tensor>` 。
doc/fluid/beginners_guide/basic_concept/variable.rst 0 → 100644
浏览文件 @ 46946b6d
.. _cn_user_guide_Variable:
=========
Variable
=========
飞桨（PaddlePaddle，以下简称Paddle）中的 :code:`Variable` 可以包含任何类型的值变量，提供的API中用到的类型是 :ref:`Tensor <cn_user_guide_tensor>` 。
后续的文档介绍中提到的 :code:`Variable` 基本等价于 :ref:`Tensor <cn_user_guide_tensor>` （特殊的地方会标注说明）。
在 Paddle 中存在三种 :code:`Variable`：
**1. 模型中的可学习参数**
模型中的可学习参数（包括网络权重、偏置等）生存期和整个训练任务一样长，会接受优化算法的更新，在 Paddle中以 Variable 的子类 Parameter 表示。
在Paddle中可以通过 :code:`fluid.layers.create_parameter` 来创建可学习参数：
.. code-block:: python
w = fluid.layers.create_parameter(name="w",shape=[1],dtype='float32')
Paddle 为大部分常见的神经网络基本计算模块都提供了封装。以最简单的全连接模型为例，下面的代码片段会直接为全连接层创建连接权值（W）和偏置（ bias ）两个可学习参数，无需显式地调用 Parameter 相关接口来创建。
.. code-block:: python
import paddle.fluid as fluid
y = fluid.layers.fc(input=x, size=128, bias_attr=True)
**2. 占位 Variable**
在声明式编程模式(静态图)模式下，组网的时候通常不知道实际输入的信息，此刻需要一个占位的 :code:`Variable`，表示一个待提供输入的 :code:`Variable`
Paddle 中使用 :code:`fluid.data` 来接收输入数据， :code:`fluid.data` 需要提供输入 Tensor 的形状信息，当遇到无法确定的维度时，相应维度指定为 None ，如下面的代码片段所示：
.. code-block:: python
import paddle.fluid as fluid
#定义x的维度为[3,None]，其中我们只能确定x的第一的维度为3，第二个维度未知，要在程序执行过程中才能确定
x = fluid.data(name="x", shape=[3,None], dtype="int64")
#若图片的宽度和高度在运行时可变，将宽度和高度定义为None。
#shape的三个维度含义分别是：batch_size, channel、图片的宽度、图片的高度
b = fluid.data(name="image",shape=[None, 3,None,None],dtype="float32")
其中，dtype=“int64”表示有符号64位整数数据类型，更多Fluid目前支持的数据类型请查看： :ref:`Paddle目前支持的数据类型 <user_guide_use_numpy_array_as_train_data>` 。
**3. 常量 Variable**
Fluid 通过 :code:`fluid.layers.fill_constant` 来实现常量Variable，用户可以指定内部包含Tensor的形状，数据类型和常量值。代码实现如下所示：
.. code-block:: python
import paddle.fluid as fluid
data = fluid.layers.fill_constant(shape=[1], value=0, dtype='int64')
doc/fluid/beginners_guide/book/fit_a_line/README.md 0 → 100644
浏览文件 @ 46946b6d
../../../../external/book/01.fit_a_line/README.md
doc/fluid/beginners_guide/book/fit_a_line/image 0 → 100644
浏览文件 @ 46946b6d
../../../../external/book/01.fit_a_line/image/
\ No newline at end of file
doc/fluid/beginners_guide/book/fit_a_line/index.html 0 → 100644
浏览文件 @ 46946b6d
<html>
<head>
<script type="text/x-mathjax-config">
MathJax.Hub.Config({
extensions: ["tex2jax.js", "TeX/AMSsymbols.js", "TeX/AMSmath.js"],
jax: ["input/TeX", "output/HTML-CSS"],
tex2jax: {
inlineMath: [ ['$','$'] ],
displayMath: [ ['$$','$$'] ],
processEscapes: true
},
"HTML-CSS": { availableFonts: ["TeX"] }
});
</script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.0/MathJax.js" async></script>
<script type="text/javascript" src="../.tools/theme/marked.js">
</script>
<link href="http://cdn.bootcss.com/highlight.js/9.9.0/styles/darcula.min.css" rel="stylesheet">
<script src="http://cdn.bootcss.com/highlight.js/9.9.0/highlight.min.js"></script>
<link href="http://cdn.bootcss.com/bootstrap/4.0.0-alpha.6/css/bootstrap.min.css" rel="stylesheet">
<link href="https://cdn.jsdelivr.net/perfect-scrollbar/0.6.14/css/perfect-scrollbar.min.css" rel="stylesheet">
<link href="../.tools/theme/github-markdown.css" rel='stylesheet'>
</head>
<style type="text/css" >
.markdown-body {
box-sizing: border-box;
min-width: 200px;
max-width: 980px;
margin: 0 auto;
padding: 45px;
}
</style>
<body>
<div id="context" class="container-fluid markdown-body">
</div>
<!-- This block will be replaced by each markdown file content. Please do not change lines below.-->
<div id="markdown" style='display:none'>
../../../../external/book/01.fit_a_line/README.md
</div>
<!-- You can change the lines below now. -->
<script type="text/javascript">
marked.setOptions({
renderer: new marked.Renderer(),
gfm: true,
breaks: false,
smartypants: true,
highlight: function(code, lang) {
code = code.replace(/&amp;/g, "&")
code = code.replace(/&gt;/g, ">")
code = code.replace(/&lt;/g, "<")
code = code.replace(/&nbsp;/g, " ")
return hljs.highlightAuto(code, [lang]).value;
}
});
document.getElementById("context").innerHTML = marked(
document.getElementById("markdown").innerHTML)
</script>
</body>
doc/fluid/beginners_guide/book/image_classification/README.md 0 → 100644
浏览文件 @ 46946b6d
../../../../external/book/03.image_classification/README.md
doc/fluid/beginners_guide/book/image_classification/image 0 → 100644
浏览文件 @ 46946b6d
../../../../external/book/03.image_classification/image/
\ No newline at end of file
doc/fluid/beginners_guide/book/image_classification/index.html 0 → 100644
浏览文件 @ 46946b6d
<html>
<head>
<script type="text/x-mathjax-config">
MathJax.Hub.Config({
extensions: ["tex2jax.js", "TeX/AMSsymbols.js", "TeX/AMSmath.js"],
jax: ["input/TeX", "output/HTML-CSS"],
tex2jax: {
inlineMath: [ ['$','$'] ],
displayMath: [ ['$$','$$'] ],
processEscapes: true
},
"HTML-CSS": { availableFonts: ["TeX"] }
});
</script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.0/MathJax.js" async></script>
<script type="text/javascript" src="../.tools/theme/marked.js">
</script>
<link href="http://cdn.bootcss.com/highlight.js/9.9.0/styles/darcula.min.css" rel="stylesheet">
<script src="http://cdn.bootcss.com/highlight.js/9.9.0/highlight.min.js"></script>
<link href="http://cdn.bootcss.com/bootstrap/4.0.0-alpha.6/css/bootstrap.min.css" rel="stylesheet">
<link href="https://cdn.jsdelivr.net/perfect-scrollbar/0.6.14/css/perfect-scrollbar.min.css" rel="stylesheet">
<link href="../.tools/theme/github-markdown.css" rel='stylesheet'>
</head>
<style type="text/css" >
.markdown-body {
box-sizing: border-box;
min-width: 200px;
max-width: 980px;
margin: 0 auto;
padding: 45px;
}
</style>
<body>
<div id="context" class="container-fluid markdown-body">
</div>
<!-- This block will be replaced by each markdown file content. Please do not change lines below.-->
<div id="markdown" style='display:none'>
../../../../external/book/03.image_classification/README.md
</div>
<!-- You can change the lines below now. -->
<script type="text/javascript">
marked.setOptions({
renderer: new marked.Renderer(),
gfm: true,
breaks: false,
smartypants: true,
highlight: function(code, lang) {
code = code.replace(/&amp;/g, "&")
code = code.replace(/&gt;/g, ">")
code = code.replace(/&lt;/g, "<")
code = code.replace(/&nbsp;/g, " ")
return hljs.highlightAuto(code, [lang]).value;
}
});
document.getElementById("context").innerHTML = marked(
document.getElementById("markdown").innerHTML)
</script>
</body>
doc/fluid/beginners_guide/book/index_en.rst 0 → 100644
浏览文件 @ 46946b6d
Book
======
.. toctree::
:maxdepth: 1
fit_a_line/README.md
recognize_digits/README.md
image_classification/README.md
word2vec/README.md
recommender_system/README.md
understand_sentiment/README.md
label_semantic_roles/README.md
machine_translation/README.md
doc/fluid/beginners_guide/book/label_semantic_roles/README.md 0 → 100644
浏览文件 @ 46946b6d
../../../../external/book/07.label_semantic_roles/README.md
doc/fluid/beginners_guide/book/label_semantic_roles/image 0 → 100644
浏览文件 @ 46946b6d
../../../../external/book/07.label_semantic_roles/image/
\ No newline at end of file
doc/fluid/beginners_guide/book/label_semantic_roles/index.html 0 → 100644
浏览文件 @ 46946b6d
<html>
<head>
<script type="text/x-mathjax-config">
MathJax.Hub.Config({
extensions: ["tex2jax.js", "TeX/AMSsymbols.js", "TeX/AMSmath.js"],
jax: ["input/TeX", "output/HTML-CSS"],
tex2jax: {
inlineMath: [ ['$','$'] ],
displayMath: [ ['$$','$$'] ],
processEscapes: true
},
"HTML-CSS": { availableFonts: ["TeX"] }
});
</script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.0/MathJax.js" async></script>
<script type="text/javascript" src="../.tools/theme/marked.js">
</script>
<link href="http://cdn.bootcss.com/highlight.js/9.9.0/styles/darcula.min.css" rel="stylesheet">
<script src="http://cdn.bootcss.com/highlight.js/9.9.0/highlight.min.js"></script>
<link href="http://cdn.bootcss.com/bootstrap/4.0.0-alpha.6/css/bootstrap.min.css" rel="stylesheet">
<link href="https://cdn.jsdelivr.net/perfect-scrollbar/0.6.14/css/perfect-scrollbar.min.css" rel="stylesheet">
<link href="../.tools/theme/github-markdown.css" rel='stylesheet'>
</head>
<style type="text/css" >
.markdown-body {
box-sizing: border-box;
min-width: 200px;
max-width: 980px;
margin: 0 auto;
padding: 45px;
}
</style>
<body>
<div id="context" class="container-fluid markdown-body">
</div>
<!-- This block will be replaced by each markdown file content. Please do not change lines below.-->
<div id="markdown" style='display:none'>
../../../../external/book/07.label_semantic_roles/README.md
</div>
<!-- You can change the lines below now. -->
<script type="text/javascript">
marked.setOptions({
renderer: new marked.Renderer(),
gfm: true,
breaks: false,
smartypants: true,
highlight: function(code, lang) {
code = code.replace(/&amp;/g, "&")
code = code.replace(/&gt;/g, ">")
code = code.replace(/&lt;/g, "<")
code = code.replace(/&nbsp;/g, " ")
return hljs.highlightAuto(code, [lang]).value;
}
});
document.getElementById("context").innerHTML = marked(
document.getElementById("markdown").innerHTML)
</script>
</body>
doc/fluid/beginners_guide/book/machine_translation/README.md 0 → 100644
浏览文件 @ 46946b6d
../../../../external/book/08.machine_translation/README.md
doc/fluid/beginners_guide/book/machine_translation/image 0 → 100644
浏览文件 @ 46946b6d
../../../../external/book/08.machine_translation/image/
\ No newline at end of file
doc/fluid/beginners_guide/book/machine_translation/index.html 0 → 100644
浏览文件 @ 46946b6d
<html>
<head>
<script type="text/x-mathjax-config">
MathJax.Hub.Config({
extensions: ["tex2jax.js", "TeX/AMSsymbols.js", "TeX/AMSmath.js"],
jax: ["input/TeX", "output/HTML-CSS"],
tex2jax: {
inlineMath: [ ['$','$'] ],
displayMath: [ ['$$','$$'] ],
processEscapes: true
},
"HTML-CSS": { availableFonts: ["TeX"] }
});
</script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.0/MathJax.js" async></script>
<script type="text/javascript" src="../.tools/theme/marked.js">
</script>
<link href="http://cdn.bootcss.com/highlight.js/9.9.0/styles/darcula.min.css" rel="stylesheet">
<script src="http://cdn.bootcss.com/highlight.js/9.9.0/highlight.min.js"></script>
<link href="http://cdn.bootcss.com/bootstrap/4.0.0-alpha.6/css/bootstrap.min.css" rel="stylesheet">
<link href="https://cdn.jsdelivr.net/perfect-scrollbar/0.6.14/css/perfect-scrollbar.min.css" rel="stylesheet">
<link href="../.tools/theme/github-markdown.css" rel='stylesheet'>
</head>
<style type="text/css" >
.markdown-body {
box-sizing: border-box;
min-width: 200px;
max-width: 980px;
margin: 0 auto;
padding: 45px;
}
</style>
<body>
<div id="context" class="container-fluid markdown-body">
</div>
<!-- This block will be replaced by each markdown file content. Please do not change lines below.-->
<div id="markdown" style='display:none'>
../../../../external/book/08.machine_translation/README.md
</div>
<!-- You can change the lines below now. -->
<script type="text/javascript">
marked.setOptions({
renderer: new marked.Renderer(),
gfm: true,
breaks: false,
smartypants: true,
highlight: function(code, lang) {
code = code.replace(/&amp;/g, "&")
code = code.replace(/&gt;/g, ">")
code = code.replace(/&lt;/g, "<")
code = code.replace(/&nbsp;/g, " ")
return hljs.highlightAuto(code, [lang]).value;
}
});
document.getElementById("context").innerHTML = marked(
document.getElementById("markdown").innerHTML)
</script>
</body>
doc/fluid/beginners_guide/book/recognize_digits/README.md 0 → 100644
浏览文件 @ 46946b6d
../../../../external/book/02.recognize_digits/README.md
doc/fluid/beginners_guide/book/recognize_digits/image 0 → 100644
浏览文件 @ 46946b6d
../../../../external/book/02.recognize_digits/image/
\ No newline at end of file
doc/fluid/beginners_guide/book/recognize_digits/index.html 0 → 100644
浏览文件 @ 46946b6d
<html>
<head>
<script type="text/x-mathjax-config">
MathJax.Hub.Config({
extensions: ["tex2jax.js", "TeX/AMSsymbols.js", "TeX/AMSmath.js"],
jax: ["input/TeX", "output/HTML-CSS"],
tex2jax: {
inlineMath: [ ['$','$'] ],
displayMath: [ ['$$','$$'] ],
processEscapes: true
},
"HTML-CSS": { availableFonts: ["TeX"] }
});
</script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.0/MathJax.js" async></script>
<script type="text/javascript" src="../.tools/theme/marked.js">
</script>
<link href="http://cdn.bootcss.com/highlight.js/9.9.0/styles/darcula.min.css" rel="stylesheet">
<script src="http://cdn.bootcss.com/highlight.js/9.9.0/highlight.min.js"></script>
<link href="http://cdn.bootcss.com/bootstrap/4.0.0-alpha.6/css/bootstrap.min.css" rel="stylesheet">
<link href="https://cdn.jsdelivr.net/perfect-scrollbar/0.6.14/css/perfect-scrollbar.min.css" rel="stylesheet">
<link href="../.tools/theme/github-markdown.css" rel='stylesheet'>
</head>
<style type="text/css" >
.markdown-body {
box-sizing: border-box;
min-width: 200px;
max-width: 980px;
margin: 0 auto;
padding: 45px;
}
</style>
<body>
<div id="context" class="container-fluid markdown-body">
</div>
<!-- This block will be replaced by each markdown file content. Please do not change lines below.-->
<div id="markdown" style='display:none'>
../../../../external/book/02.recognize_digits/README.md
</div>
<!-- You can change the lines below now. -->
<script type="text/javascript">
marked.setOptions({
renderer: new marked.Renderer(),
gfm: true,
breaks: false,
smartypants: true,
highlight: function(code, lang) {
code = code.replace(/&amp;/g, "&")
code = code.replace(/&gt;/g, ">")
code = code.replace(/&lt;/g, "<")
code = code.replace(/&nbsp;/g, " ")
return hljs.highlightAuto(code, [lang]).value;
}
});
document.getElementById("context").innerHTML = marked(
document.getElementById("markdown").innerHTML)
</script>
</body>
doc/fluid/beginners_guide/book/recommender_system/README.md 0 → 100644
浏览文件 @ 46946b6d
../../../../external/book/05.recommender_system/README.md
doc/fluid/beginners_guide/book/recommender_system/image 0 → 100644
浏览文件 @ 46946b6d
../../../../external/book/05.recommender_system/image/
\ No newline at end of file
doc/fluid/beginners_guide/book/recommender_system/index.html 0 → 100644
浏览文件 @ 46946b6d
<html>
<head>
<script type="text/x-mathjax-config">
MathJax.Hub.Config({
extensions: ["tex2jax.js", "TeX/AMSsymbols.js", "TeX/AMSmath.js"],
jax: ["input/TeX", "output/HTML-CSS"],
tex2jax: {
inlineMath: [ ['$','$'] ],
displayMath: [ ['$$','$$'] ],
processEscapes: true
},
"HTML-CSS": { availableFonts: ["TeX"] }
});
</script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.0/MathJax.js" async></script>
<script type="text/javascript" src="../.tools/theme/marked.js">
</script>
<link href="http://cdn.bootcss.com/highlight.js/9.9.0/styles/darcula.min.css" rel="stylesheet">
<script src="http://cdn.bootcss.com/highlight.js/9.9.0/highlight.min.js"></script>
<link href="http://cdn.bootcss.com/bootstrap/4.0.0-alpha.6/css/bootstrap.min.css" rel="stylesheet">
<link href="https://cdn.jsdelivr.net/perfect-scrollbar/0.6.14/css/perfect-scrollbar.min.css" rel="stylesheet">
<link href="../.tools/theme/github-markdown.css" rel='stylesheet'>
</head>
<style type="text/css" >
.markdown-body {
box-sizing: border-box;
min-width: 200px;
max-width: 980px;
margin: 0 auto;
padding: 45px;
}
</style>
<body>
<div id="context" class="container-fluid markdown-body">
</div>
<!-- This block will be replaced by each markdown file content. Please do not change lines below.-->
<div id="markdown" style='display:none'>
../../../../external/book/05.recommender_system/README.md
</div>
<!-- You can change the lines below now. -->
<script type="text/javascript">
marked.setOptions({
renderer: new marked.Renderer(),
gfm: true,
breaks: false,
smartypants: true,
highlight: function(code, lang) {
code = code.replace(/&amp;/g, "&")
code = code.replace(/&gt;/g, ">")
code = code.replace(/&lt;/g, "<")
code = code.replace(/&nbsp;/g, " ")
return hljs.highlightAuto(code, [lang]).value;
}
});
document.getElementById("context").innerHTML = marked(
document.getElementById("markdown").innerHTML)
</script>
</body>
doc/fluid/beginners_guide/book/understand_sentiment/README.md 0 → 100644
浏览文件 @ 46946b6d
../../../../external/book/06.understand_sentiment/README.md
doc/fluid/beginners_guide/book/understand_sentiment/image 0 → 100644
浏览文件 @ 46946b6d
../../../../external/book/06.understand_sentiment/image/
\ No newline at end of file
doc/fluid/beginners_guide/book/understand_sentiment/index.html 0 → 100644
浏览文件 @ 46946b6d
<html>
<head>
<script type="text/x-mathjax-config">
MathJax.Hub.Config({
extensions: ["tex2jax.js", "TeX/AMSsymbols.js", "TeX/AMSmath.js"],
jax: ["input/TeX", "output/HTML-CSS"],
tex2jax: {
inlineMath: [ ['$','$'] ],
displayMath: [ ['$$','$$'] ],
processEscapes: true
},
"HTML-CSS": { availableFonts: ["TeX"] }
});
</script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.0/MathJax.js" async></script>
<script type="text/javascript" src="../.tools/theme/marked.js">
</script>
<link href="http://cdn.bootcss.com/highlight.js/9.9.0/styles/darcula.min.css" rel="stylesheet">
<script src="http://cdn.bootcss.com/highlight.js/9.9.0/highlight.min.js"></script>
<link href="http://cdn.bootcss.com/bootstrap/4.0.0-alpha.6/css/bootstrap.min.css" rel="stylesheet">
<link href="https://cdn.jsdelivr.net/perfect-scrollbar/0.6.14/css/perfect-scrollbar.min.css" rel="stylesheet">
<link href="../.tools/theme/github-markdown.css" rel='stylesheet'>
</head>
<style type="text/css" >
.markdown-body {
box-sizing: border-box;
min-width: 200px;
max-width: 980px;
margin: 0 auto;
padding: 45px;
}
</style>
<body>
<div id="context" class="container-fluid markdown-body">
</div>
<!-- This block will be replaced by each markdown file content. Please do not change lines below.-->
<div id="markdown" style='display:none'>
../../../../external/book/06.understand_sentiment/README.md
</div>
<!-- You can change the lines below now. -->
<script type="text/javascript">
marked.setOptions({
renderer: new marked.Renderer(),
gfm: true,
breaks: false,
smartypants: true,
highlight: function(code, lang) {
code = code.replace(/&amp;/g, "&")
code = code.replace(/&gt;/g, ">")
code = code.replace(/&lt;/g, "<")
code = code.replace(/&nbsp;/g, " ")
return hljs.highlightAuto(code, [lang]).value;
}
});
document.getElementById("context").innerHTML = marked(
document.getElementById("markdown").innerHTML)
</script>
</body>
doc/fluid/beginners_guide/book/word2vec/README.md 0 → 100644
浏览文件 @ 46946b6d
../../../../external/book/04.word2vec/README.md
doc/fluid/beginners_guide/book/word2vec/images 0 → 100644
浏览文件 @ 46946b6d
../../../../external/book/04.word2vec/image/
\ No newline at end of file
doc/fluid/beginners_guide/book/word2vec/index.html 0 → 100644
浏览文件 @ 46946b6d
<html>
<head>
<script type="text/x-mathjax-config">
MathJax.Hub.Config({
extensions: ["tex2jax.js", "TeX/AMSsymbols.js", "TeX/AMSmath.js"],
jax: ["input/TeX", "output/HTML-CSS"],
tex2jax: {
inlineMath: [ ['$','$'] ],
displayMath: [ ['$$','$$'] ],
processEscapes: true
},
"HTML-CSS": { availableFonts: ["TeX"] }
});
</script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.0/MathJax.js" async></script>
<script type="text/javascript" src="../.tools/theme/marked.js">
</script>
<link href="http://cdn.bootcss.com/highlight.js/9.9.0/styles/darcula.min.css" rel="stylesheet">
<script src="http://cdn.bootcss.com/highlight.js/9.9.0/highlight.min.js"></script>
<link href="http://cdn.bootcss.com/bootstrap/4.0.0-alpha.6/css/bootstrap.min.css" rel="stylesheet">
<link href="https://cdn.jsdelivr.net/perfect-scrollbar/0.6.14/css/perfect-scrollbar.min.css" rel="stylesheet">
<link href="../.tools/theme/github-markdown.css" rel='stylesheet'>
</head>
<style type="text/css" >
.markdown-body {
box-sizing: border-box;
min-width: 200px;
max-width: 980px;
margin: 0 auto;
padding: 45px;
}
</style>
<body>
<div id="context" class="container-fluid markdown-body">
</div>
<!-- This block will be replaced by each markdown file content. Please do not change lines below.-->
<div id="markdown" style='display:none'>
../../../../external/book/04.word2vec/README.md
</div>
<!-- You can change the lines below now. -->
<script type="text/javascript">
marked.setOptions({
renderer: new marked.Renderer(),
gfm: true,
breaks: false,
smartypants: true,
highlight: function(code, lang) {
code = code.replace(/&amp;/g, "&")
code = code.replace(/&gt;/g, ">")
code = code.replace(/&lt;/g, "<")
code = code.replace(/&nbsp;/g, " ")
return hljs.highlightAuto(code, [lang]).value;
}
});
document.getElementById("context").innerHTML = marked(
document.getElementById("markdown").innerHTML)
</script>
</body>
doc/fluid/beginners_guide/coding_practice/configure_simple_model/index_cn.rst 0 → 100644
浏览文件 @ 46946b6d
.. _user_guide_configure_simple_model:
##############
配置简单的网络
##############
在解决实际问题时，可以先从逻辑层面对问题进行建模，明确模型所需要的 **输入数据类型**、**计算逻辑**、**求解目标** 以及 **优化算法**。PaddlePaddle提供了丰富的算子来实现模型逻辑。下面以一个简单回归任务举例说明如何使用PaddlePaddle构建模型。该例子完整代码参见 `fit_a_line <https://github.com/PaddlePaddle/Paddle/blob/develop/python/paddle/fluid/tests/book/test_fit_a_line.py>`_。
问题描述及定义
##############
问题描述: 给定一组数据 :math:`<X, Y>`，求解出函数 :math:`f`，使得 :math:`y=f(x)`，其中 :math:`x\subset X` 表示一条样本的特征，为 :math:`13` 维的实数向量；:math:`y \subset Y` 为一实数表示该样本对应的值。
我们可以尝试用回归模型来对问题建模，回归问题的损失函数有很多，这里选择常用的均方误差。为简化问题，这里假定 :math:`f` 为简单的线性变换函数，同时选用随机梯度下降算法来求解模型。
+----------------+----------------------------------------------+
| 输入数据类型 | 样本特征: 13 维 实数 |
+ +----------------------------------------------+
| | 样本标签: 1 维 实数 |
+----------------+----------------------------------------------+
| 计算逻辑 | 使用线性模型，产生 1维实数作为模型的预测输出 |
+----------------+----------------------------------------------+
| 求解目标 | 最小化模型预测输出与样本标签间的均方误差 |
+----------------+----------------------------------------------+
| 优化算法 | 随机梯度下降 |
+----------------+----------------------------------------------+
使用PaddlePaddle建模
###################
从逻辑层面明确了输入数据格式、模型结构、损失函数以及优化算法后，需要使用PaddlePaddle提供的API及算子来实现模型逻辑。一个典型的模型主要包含4个部分，分别是：输入数据格式定义，模型前向计算逻辑，损失函数以及优化算法。
数据层
------
PaddlePaddle提供了 :code:`fluid.data()` 算子来描述输入数据的格式。
:code:`fluid.data()` 算子的输出是一个Variable。这个Variable的实际类型是Tensor。Tensor具有强大的表征能力，可以表示多维数据。为了精确描述数据结构，通常需要指定数据shape以及数值类型type。其中shape为一个整数向量，type可以是一个字符串类型。目前支持的数据类型参考 :ref:`user_guide_paddle_support_data_types` 。 模型训练一般会使用batch的方式读取数据，而batch的size在训练过程中可能不固定。data算子会依据实际数据来推断batch size，所以这里提供shape时不用关心batch size，只需关心一条样本的shape即可，更高级用法请参考 :ref:`user_guide_customize_batch_size_rank`。从上知，:math:`x` 为 :math:`13` 维的实数向量，:math:`y` 为实数，可使用下面代码定义数据层：
.. code-block:: python
x = fluid.data(name='x', shape=[13], dtype='float32')
y = fluid.data(name='y', shape=[1], dtype='float32')
该模型使用的数据比较简单，事实上data算子还可以描述变长的、嵌套的序列数据。更详细的文档可参照 :ref:`user_guide_prepare_data`。
前向计算逻辑
------------
实现一个模型最重要的部分是实现计算逻辑，PaddlePaddle提供了丰富的算子。这些算子的封装粒度不同，通常对应一种或一组变换逻辑。算子输出即为对输入数据执行变换后的结果。用户可以灵活使用算子来完成复杂的模型逻辑。比如图像相关任务中会使用较多的卷积算子、序列任务中会使用LSTM/GRU等算子。复杂模型通常会组合多种算子，以完成复杂的变换。PaddlePaddle提供了非常自然的方式来组合算子，一般地可以使用下面的方式：
.. code-block:: python
op_1_out = fluid.layers.op_1(input=op_1_in, ...)
op_2_out = fluid.layers.op_2(input=op_1_out, ...)
...
其中op_1和op_2表示算子类型，可以是fc来执行线性变换(全连接)，也可以是conv来执行卷积变换等。通过算子的输入输出的连接来定义算子的计算顺序以及数据流方向。上面的例子中，op_1的输出是op_2的输入，那么在执行计算时，会先计算op_1，然后计算op_2。更复杂的模型可能需要使用控制流算子，依据输入数据来动态执行，针对这种情况，PaddlePaddle提供了IfElseOp和WhileOp等。算子的文档可参考 :code:`fluid.layers`。具体到这个任务, 我们使用一个fc算子：
.. code-block:: python
y_predict = fluid.layers.fc(input=x, size=1, act=None)
损失函数
--------
损失函数对应求解目标，我们可以通过最小化损失来求解模型。大多数模型使用的损失函数，输出是一个实数值。但是PaddlePaddle提供的损失算子一般是针对一条样本计算。当输入一个batch的数据时，损失算子的输出有多个值，每个值对应一条样本的损失，所以通常会在损失算子后面使用mean等算子，来对损失做归约。模型在一次前向迭代后会得到一个损失值，PaddlePaddle会自动执行链式求导法则计算模型里面每个参数和变量对应的梯度值。这里使用均方误差损失：
.. code-block:: python
cost = fluid.layers.square_error_cost(input=y_predict, label=y)
avg_cost = fluid.layers.mean(cost)
优化方法
--------
确定损失函数后，可以通过前向计算得到损失值，然后通过链式求导法则得到参数的梯度值。获取梯度值后需要更新参数，最简单的算法是随机梯度下降法：:math:`w=w - \eta \cdot g`。但是普通的随机梯度下降算法存在一些问题: 比如收敛不稳定等。为了改善模型的训练速度以及效果，学术界先后提出了很多优化算法，包括： :code:`Momentum`、:code:`RMSProp`、:code:`Adam` 等。这些优化算法采用不同的策略来更新模型参数，一般可以针对具体任务和具体模型来选择优化算法。不管使用何种优化算法，学习率一般是一个需要指定的比较重要的超参数，需要通过实验仔细调整。这里采用随机梯度下降算法：
.. code-block:: python
sgd_optimizer = fluid.optimizer.SGD(learning_rate=0.001)
更多优化算子可以参考 :code:`fluid.optimizer()` 。
下一步做什么？
##############
使用PaddlePaddle实现模型时需要关注 **数据层**、**前向计算逻辑**、**损失函数** 和 **优化方法**。不同的任务需要的数据格式不同，涉及的计算逻辑不同，损失函数不同，优化方法也不同。PaddlePaddle提供了丰富的模型示例，可以以这些示例为参考来构建自己的模型结构。用户可以访问 `模型库 <https://github.com/PaddlePaddle/models/tree/develop/fluid>`_ 查看官方提供的示例。
doc/fluid/beginners_guide/coding_practice/configure_simple_model/index_en.rst 0 → 100644
浏览文件 @ 46946b6d
.. _user_guide_configure_simple_model_en:
#######################
Set up Simple Model
#######################
When solving practical problems, in the beginning you can model the problem logically, and get a clear picture of **input data type** , **computing logic** , **target solution** and **optimization algorithm** of model.
PaddlePaddle provides abundant operators to implement logics of a model. In this article, we take a simple regression task as an example to clarify how to build model with PaddlePaddle.
About complete code of the example,please refer to `fit_a_line <https://github.com/PaddlePaddle/Paddle/blob/develop/python/paddle/fluid/tests/book/test_fit_a_line.py>`_ 。
Description and Definition of Problem
######################################
Description : Given a pair of data :math:`<X, Y>`, figure out a function :math:`f` to make :math:`y=f(x)` . :math:`x\subset X` represents the feature of a sample, which is a real number vector with 13 dimensions; :math:`y \subset Y` is a real number representing corresponding value of the given sample.
We can try to model the problem with a regression model. Though lots of loss functions are available for regression problem, here we choose commonly used mean-square error. To simplify the problem, assuming :math:`f` is a simple linear transformation funtion, we choose random gradient descent algorithm to solve problem.
+--------------------------+-------------------------------------------------------------------------------------+
| input data type | sample feature: 13-dimension real number |
+ +-------------------------------------------------------------------------------------+
| | sample label: 1-dimension real number |
+--------------------------+-------------------------------------------------------------------------------------+
| computing logic | use linear model to generate 1-dimensional real number as predicted output of model |
+--------------------------+-------------------------------------------------------------------------------------+
| target solution | minimize mean-squre error between predicted output of model and sample label |
+--------------------------+-------------------------------------------------------------------------------------+
| optimization algorithm | random gradient descent |
+--------------------------+-------------------------------------------------------------------------------------+
Model with PaddlePaddle
#######################
After getting clear of the of input data format, model structure, loss function and optimization algorithm in terms of logic, you need to use PaddlePaddle API and operators to implement logic of model. A typical model includes four parts: format of input data, forward computing logic, loss function and optimization algorithm.
Data Layer
-----------
PaddlePaddle provides :code:`fluid.data()` to describe format of input data.
The output of :code:`fluid.data()` is a Variable which is in fact a Tensor. Tensor can represent multi-demensional data with its great expressive feature.In order to accurately describe data structure, it is usually necessary to indicate the shape and type of data. The shape is int vector and type can be a string. About current supported data type, please refer to :ref:`user_guide_paddle_support_data_types_en` . Data is often read in form of batch to train model. Since batch size may vary and data operator infers batch size according to actual data, here the batch size is ignored when shape is provided. It's enough to care for the shape of single sample. For more advanced usage, please refer to :ref:`user_guide_customize_batch_size_rank_en` . :math:`x` is real number vector of :math:`13` dimenstions while :math:`y` is a real number. Data layer can be defined as follows:
.. code-block:: python
x = fluid.data(name='x', shape=[13], dtype='float32')
y = fluid.data(name='y', shape=[1], dtype='float32')
Data in this example model are relatively simple. In fact, data operator can describe variable-length and nested sequence data. For more detailed documentation, please refer to :ref:`user_guide_prepare_data_en` .
Logic of Forward Computing
---------------------------
The most important part of a model is to implement logic of computing. PaddlePaddle provides lots of operators encapsulated in different granularity. These operators usually are correspondent to a kind or a group of transformation logic. The output of operator is the result of transfomation for input data. User can flexiblely use operators to implement models with complex logics. For example, many convolutional operators will be used in tasks associated with image tasks and LSTM/GRU operators will be used in sequence tasks. Various operators are usually combined in complex models to implement complex transformation. PaddlePaddle provides natural methods to combine operators. The following example displays the typical combination method:
.. code-block:: python
op_1_out = fluid.layers.op_1(input=op_1_in, ...)
op_2_out = fluid.layers.op_2(input=op_1_out, ...)
...
In the example above, op_1 and op_2 represent types of operators,such as fc performing linear transformation(full connection) or conv performing convolutional transformation. The computing order of operators and direction of data stream are defined by the connection of input and output of operators. In the example above, the output of op_1 is the input of op_2. It will firstly compute op_1 and then op_2 in the process of computing. For more complex models, we may need to use control flow operators to make it perform dynamically according to the input data. In this situation, IfElseOp, WhileOp and other operators are provided in PaddlePaddle. About documentation of these operators, please refer to :code:`fluid.layers` . As for this task, we use a fc operator:
.. code-block:: python
y_predict = fluid.layers.fc(input=x, size=1, act=None)
Loss Function
--------------
Loss function is correspondent with the target solution. We can resolve the model by minimizing the loss value. The outputs of loss functions of most models are real numbers. But the loss operator in PaddlePaddle is only aimed at a single sample. When a batch is feeded, there will be many outputs from the loss operator, each of which is correspondent with the loss of a single sample. Therefore we usually append operators like ``mean`` after loss function to conduct reduction of losses. After each forward iteration, a loss value will be returned. After that, Chain derivation theorem will be performed automatically in PaddlePaddle to compute gradient value of every parameter and variable in computing model. Here we use mean square error cost:
.. code-block:: python
cost = fluid.layers.square_error_cost(input=y_predict, label=y)
avg_cost = fluid.layers.mean(cost)
Optimization Method
---------------------
After the definition of loss function, we can get loss value by forward computing and then get gradient value of parameters with chain deravation theorem. Having obtained the gradients, parameters have to be updated and the simplest algorithm is the random gradient descent algorithm: :math:`w=w - \eta \cdot g` .But common random gradient descent algorithms have some disadvantages, such as unstable convergency. To improve the training speed and effect of model, academic scholars have come up with many optimized algorithm, including :code:`Momentum` , :code:`RMSProp` , :code:`Adam` . Strategies vary from optimization algorithm to another to update parameters of model. Usually we can choose appropriate algorthm according to specific tasks and models. No matter what optimization algorithm we adopt, learning rate is usually an important super parameter to be specified and carefully adjusted by trials. Take random gradient descent algorithm as an example here:
.. code-block:: python
sgd_optimizer = fluid.optimizer.SGD(learning_rate=0.001)
For more optimization operators,please refer to :code:`fluid.optimizer()` .
What to do next?
#################
Attention needs to be paid for **Data Layer**, **Forward Computing Logic**, **Loss function** and **Optimization Function** while you use PaddlePaddle to implement models.
The data format, computing logic, loss function and optimization function are all different in different tasks. A rich number of examples of model are provided in PaddlePaddle. You can build your own model structure by referring to these examples. You can visit `Model Repository <https://github.com/PaddlePaddle/models/tree/develop/fluid>`_ to refer to examples in official documentation.
doc/fluid/beginners_guide/coding_practice/index_cn.rst 0 → 100644
浏览文件 @ 46946b6d
############
编程实践
############
如果您已经掌握了基本概念中的内容，期望可以针对实际问题建模、搭建自己网络，本模块提供了一些 Paddle 的使用细节供您参考：
.. toctree::
:maxdepth: 1
configure_simple_model/index_cn.rst
single_node.rst
save_load_variables.rst
doc/fluid/beginners_guide/coding_practice/index_en.rst 0 → 100644
浏览文件 @ 46946b6d
############
Coding Practice
############
If you have mastered the basic concepts and you expect to model and build your own network according to the actual problems, this module provides you with some details about the use of paddle for your reference:
.. toctree::
:maxdepth: 1
configure_simple_model/index_en.rst
single_node_en.rst
test_while_training_en.rst
save_load_variables_en.rst
doc/fluid/beginners_guide/coding_practice/save_load_variables.rst 0 → 100644
浏览文件 @ 46946b6d
.. _user_guide_save_load_vars:
################################
模型/变量的保存、载入与增量训练
################################
模型变量分类
############
在PaddlePaddle Fluid中，所有的模型变量都用 :code:`fluid.framework.Variable()` 作为基类。
在该基类之下，模型变量主要可以分为以下几种类别：
1. 模型参数
模型参数是深度学习模型中被训练和学习的变量，在训练过程中，训练框架根据反向传播(backpropagation)算法计算出每一个模型参数当前的梯度，
并用优化器(optimizer)根据梯度对参数进行更新。模型的训练过程本质上可以看做是模型参数不断迭代更新的过程。
在PaddlePaddle Fluid中，模型参数用 :code:`fluid.framework.Parameter` 来表示，
这是一个 :code:`fluid.framework.Variable()` 的派生类，除了具有 :code:`fluid.framework.Variable()` 的各项性质以外，
:code:`fluid.framework.Parameter` 还可以配置自身的初始化方法、更新率等属性。
2. 长期变量
长期变量指的是在整个训练过程中持续存在、不会因为一个迭代的结束而被销毁的变量，例如动态调节的全局学习率等。
在PaddlePaddle Fluid中，长期变量通过将 :code:`fluid.framework.Variable()` 的 :code:`persistable`
属性设置为 :code:`True` 来表示。所有的模型参数都是长期变量，但并非所有的长期变量都是模型参数。
3. 临时变量
不属于上面两个类别的所有模型变量都是临时变量，这种类型的变量只在一个训练迭代中存在，在每一个迭代结束后，
所有的临时变量都会被销毁，然后在下一个迭代开始之前，又会先构造出新的临时变量供本轮迭代使用。
一般情况下模型中的大部分变量都属于这一类别，例如输入的训练数据、一个普通的layer的输出等等。
如何保存模型变量
################
根据用途的不同，我们需要保存的模型变量也是不同的。例如，如果我们只是想保存模型用来进行以后的预测，
那么只保存模型参数就够用了。但如果我们需要保存一个checkpoint（检查点，类似于存档，存有复现目前模型的必要信息）以备将来恢复训练，
那么我们应该将各种长期变量都保存下来，甚至还需要记录一下当前的epoch和step的id。
因为一些模型变量虽然不是参数，但对于模型的训练依然必不可少。
save_vars、save_params、save_persistables 以及 save_inference_model的区别
##########################################################################
1. :code:`save_inference_model` 会根据用户配置的 :code:`feeded_var_names` 和 :code:`target_vars` 进行网络裁剪，保存下裁剪后的网络结构的 ``__model__`` 以及裁剪后网络中的长期变量
2. :code:`save_persistables` 不会保存网络结构，会保存网络中的全部长期变量到指定位置。
3. :code:`save_params` 不会保存网络结构，会保存网络中的全部模型参数到指定位置。
4. :code:`save_vars` 不会保存网络结构，会根据用户指定的 :code:`fluid.framework.Parameter` 列表进行保存。
:code:`save_persistables` 保存的网络参数是最全面的，如果是增量训练或者恢复训练， 请选择 :code:`save_persistables` 进行变量保存。
:code:`save_inference_model` 会保存网络参数及裁剪后的模型，如果后续要做预测相关的工作， 请选择 :code:`save_inference_model` 进行变量和网络的保存。
:code:`save_vars 和 save_params` 仅在用户了解清楚用途及特殊目的情况下使用， 一般不建议使用。
保存模型用于对新样本的预测
==========================
如果我们保存模型的目的是用于对新样本的预测，那么只保存模型参数就足够了。我们可以使用
:code:`fluid.io.save_params()` 接口来进行模型参数的保存。
例如：
.. code-block:: python
import paddle.fluid as fluid
exe = fluid.Executor(fluid.CPUPlace())
param_path = "./my_paddle_model"
prog = fluid.default_main_program()
fluid.io.save_params(executor=exe, dirname=param_path, main_program=None)
上面的例子中，通过调用 :code:`fluid.io.save_params` 函数，PaddlePaddle Fluid会对默认
:code:`fluid.Program` 也就是 :code:`prog` 中的所有模型变量进行扫描，
筛选出其中所有的模型参数，并将这些模型参数保存到指定的 :code:`param_path` 之中。
如何载入模型变量
################
与模型变量的保存相对应，我们提供了两套API来分别载入模型的参数和载入模型的长期变量，分别为保存、加载模型参数的 ``save_params()`` 、 ``load_params()`` 和
保存、加载长期变量的 ``save_persistables`` 、 ``load_persistables`` 。
载入模型用于对新样本的预测
==========================
对于通过 :code:`fluid.io.save_params` 保存的模型，可以使用 :code:`fluid.io.load_params`
来进行载入。
例如：
.. code-block:: python
import paddle.fluid as fluid
exe = fluid.Executor(fluid.CPUPlace())
param_path = "./my_paddle_model"
prog = fluid.default_main_program()
fluid.io.load_params(executor=exe, dirname=param_path,
main_program=prog)
上面的例子中，通过调用 :code:`fluid.io.load_params` 函数，PaddlePaddle Fluid会对
:code:`prog` 中的所有模型变量进行扫描，筛选出其中所有的模型参数，
并尝试从 :code:`param_path` 之中读取加载它们。
需要格外注意的是，这里的 :code:`prog` 必须和调用 :code:`fluid.io.save_params`
时所用的 :code:`prog` 中的前向部分完全一致，且不能包含任何参数更新的操作。如果两者存在不一致，
那么可能会导致一些变量未被正确加载；如果错误地包含了参数更新操作，那可能会导致正常预测过程中参数被更改。
这两个 :code:`fluid.Program` 之间的关系类似于训练 :code:`fluid.Program`
和测试 :code:`fluid.Program` 之间的关系，详见： :ref:`user_guide_test_while_training`。
另外，需特别注意运行 :code:`fluid.default_startup_program()` 必须在调用 :code:`fluid.io.load_params`
之前。如果在之后运行，可能会覆盖已加载的模型参数导致错误。
通过numpy数组设置模型参数值
===========================
用户可以灵活地使用numpy数组设置模型参数的值，具体示例如下：
.. code-block:: python
import paddle.fluid as fluid
import numpy as np
main_prog = fluid.Program()
startup_prog = fluid.Program()
with fluid.program_guard(main_prog, startup_prog):
data = fluid.layers.data(name="img", shape=[64, 784], append_batch_size=False)
w = fluid.layers.create_parameter(shape=[784, 200], dtype='float32', name='fc_w')
b = fluid.layers.create_parameter(shape=[200], dtype='float32', name='fc_b')
hidden_w = fluid.layers.matmul(x=data, y=w)
hidden_b = fluid.layers.elementwise_add(hidden_w, b)
place = fluid.CPUPlace()
exe = fluid.Executor(place)
exe.run(startup_prog)
for block in main_prog.blocks:
for param in block.all_parameters():
pd_var = fluid.global_scope().find_var(param.name)
pd_param = pd_var.get_tensor()
print("load: {}, shape: {}".format(param.name, param.shape))
print("Before setting the numpy array value: {}".format(np.array(pd_param).ravel()[:5]))
pd_param.set(np.ones(param.shape), place)
print("After setting the numpy array value: {}".format(np.array(pd_param).ravel()[:5]))
# 输出结果：
# load: fc_w, shape: (784, 200)
# Before setting the numpy array value: [ 0.00121664 0.00700346 -0.05220041 -0.05879825 0.05155897]
# After setting the numpy array value: [1. 1. 1. 1. 1.]
# load: fc_b, shape: (200,)
# Before setting the numpy array value: [-0.098886 -0.00530401 -0.05821943 -0.01038218 0.00760134]
# After setting the numpy array value: [1. 1. 1. 1. 1.]
预测模型的保存和加载
##############################
预测引擎提供了存储预测模型 :code:`fluid.io.save_inference_model` 和加载预测模型 :code:`fluid.io.load_inference_model` 两个接口。
- :code:`fluid.io.save_inference_model`：请参考 :ref:`api_guide_inference`。
- :code:`fluid.io.load_inference_model`：请参考 :ref:`api_guide_inference`。
增量训练
############
增量训练指一个学习系统能不断地从新样本中学习新的知识，并能保存大部分以前已经学习到的知识。因此增量学习涉及到两点：在上一次训练结束的时候保存需要的长期变量， 在下一次训练开始的时候加载上一次保存的这些长期变量。 因此增量训练涉及到如下几个API:
:code:`fluid.io.save_persistables`、:code:`fluid.io.load_persistables` 。
单机增量训练
==========================
单机的增量训练的一般步骤如下：
1. 在训练的最后调用 :code:`fluid.io.save_persistables` 保存持久性参数到指定的位置。
2. 在训练的startup_program通过执行器 :code:`Executor` 执行成功之后调用 :code:`fluid.io.load_persistables` 加载之前保存的持久性参数。
3. 通过执行器 :code:`Executor` 或者 :code:`ParallelExecutor` 继续训练。
例如：
.. code-block:: python
import paddle.fluid as fluid
exe = fluid.Executor(fluid.CPUPlace())
path = "./models"
prog = fluid.default_main_program()
fluid.io.save_persistables(exe, path, prog)
上面的例子中，通过调用 :code:`fluid.io.save_persistables` 函数，PaddlePaddle Fluid会从默认 :code:`fluid.Program` 也就是 :code:`prog` 的所有模型变量中找出长期变量，并将他们保存到指定的 :code:`path` 目录下。
.. code-block:: python
import paddle.fluid as fluid
exe = fluid.Executor(fluid.CPUPlace())
path = "./models"
startup_prog = fluid.default_startup_program()
exe.run(startup_prog)
main_prog = fluid.default_main_program()
fluid.io.load_persistables(exe, path, main_prog)
exe.run(main_prog)
上面的例子中，通过调用 :code:`fluid.io.load_persistables` 函数，PaddlePaddle Fluid会从默认
:code:`fluid.Program` 也就是 :code:`prog` 的所有模型变量中找出长期变量，从指定的 :code:`path` 目录中将它们一一加载， 然后再继续进行训练。
多机增量（不带分布式大规模稀疏矩阵）训练的一般步骤为
==========================
多机增量训练和单机增量训练有若干不同点：
1. 在训练的最后调用 :code:`fluid.io.save_persistables` 保存长期变量时，不必要所有的trainer都调用这个方法来保存，一般0号trainer来保存即可。
2. 多机增量训练的参数加载在PServer端，trainer端不用加载参数。在PServer全部启动后，trainer会从PServer端同步参数。
3. 在确认需要使用增量的情况下， 多机在调用 :code:`fluid.DistributeTranspiler.transpile` 时需要指定 ``current_endpoint`` 参数。
多机增量（不带分布式大规模稀疏矩阵）训练的一般步骤为：
1. 0号trainer在训练的最后调用 :code:`fluid.io.save_persistables` 保存持久性参数到指定的 :code:`path` 下。
2. 通过HDFS等方式将0号trainer保存下来的所有的参数共享给所有的PServer(每个PServer都需要有完整的参数)。
3. PServer在训练的startup_program通过执行器（:code:`Executor`）执行成功之后调用 :code:`fluid.io.load_persistables` 加载0号trainer保存的持久性参数。
4. PServer通过执行器 :code:`Executor` 继续启动PServer_program.
5. 所有的训练节点trainer通过执行器 :code:`Executor` 或者 :code:`ParallelExecutor` 正常训练。
对于训练过程中待保存参数的trainer， 例如：
.. code-block:: python
import paddle.fluid as fluid
exe = fluid.Executor(fluid.CPUPlace())
path = "./models"
trainer_id = 0
if trainer_id == 0:
prog = fluid.default_main_program()
fluid.io.save_persistables(exe, path, prog)
.. code-block:: bash
hadoop fs -mkdir /remote/$path
hadoop fs -put $path /remote/$path
上面的例子中，0号trainer通过调用 :code:`fluid.io.save_persistables` 函数，PaddlePaddle Fluid会从默认
:code:`fluid.Program` 也就是 :code:`prog` 的所有模型变量中找出长期变量，并将他们保存到指定的 :code:`path` 目录下。然后通过调用第三方的文件系统（如HDFS）将存储的模型进行上传到所有PServer都可访问的位置。
对于训练过程中待载入参数的PServer， 例如：
.. code-block:: bash
hadoop fs -get /remote/$path $path
.. code-block:: python
import paddle.fluid as fluid
exe = fluid.Executor(fluid.CPUPlace())
path = "./models"
pserver_endpoints = "127.0.0.1:1001,127.0.0.1:1002"
trainers = 4
training_role == "PSERVER"
config = fluid.DistributeTranspilerConfig()
t = fluid.DistributeTranspiler(config=config)
t.transpile(trainer_id, pservers=pserver_endpoints, trainers=trainers, sync_mode=True, current_endpoint=current_endpoint)
if training_role == "PSERVER":
current_endpoint = "127.0.0.1:1001"
pserver_prog = t.get_pserver_program(current_endpoint)
pserver_startup = t.get_startup_program(current_endpoint, pserver_prog)
exe.run(pserver_startup)
fluid.io.load_persistables(exe, path, pserver_prog)
exe.run(pserver_prog)
if training_role == "TRAINER":
main_program = t.get_trainer_program()
exe.run(main_program)
上面的例子中，每个PServer通过调用HDFS的命令获取到0号trainer保存的参数，通过配置获取到PServer的 :code:`fluid.Program` ，PaddlePaddle Fluid会从此
:code:`fluid.Program` 也就是 :code:`pserver_startup` 的所有模型变量中找出长期变量，并通过指定的 :code:`path` 目录下一一加载。
doc/fluid/beginners_guide/coding_practice/save_load_variables_en.rst 0 → 100644
浏览文件 @ 46946b6d
.. _user_guide_save_load_vars_en:
######################################################
Save, Load Models or Variables & Incremental Learning
######################################################
Model variable classification
##############################
In PaddlePaddle Fluid, all model variables are represented by :code:`fluid.Variable()` as the base class. Under this base class, model variables can be divided into the following categories:
1. Model parameter
The model parameters are the variables trained and learned in the deep learning model. During the training process, the training framework calculates the current gradient of each model parameter according to the back propagation algorithm, and updates the parameters according to their gradients by the optimizer. The essence of the training process of a model can be seen as the process of continuously iterative updating of model parameters. In PaddlePaddle Fluid, the model parameters are represented by :code:`fluid.framework.Parameter` , which is a derived class of :code:`fluid.Variable()` . Besides various properties of :code:`fluid.Variable()` , :code:`fluid.framework.Parameter` can also be configured with its own initialization methods, update rate and other properties.
2. Persistable variable
Persistable variables refer to variables that persist throughout the training process and are not destroyed by the end of an iteration, such as the global learning rate which is dynamically adjusted. In PaddlePaddle Fluid, persistable variables are represented by setting the :code:`persistable` property of :code:`fluid.Variable()` to :code:`True`. All model parameters are persistable variables, but not all persistable variables are model parameters.
3. Temporary variables
All model variables that do not belong to the above two categories are temporary variables. This type of variable exists only in one training iteration. After each iteration, all temporary variables will be destroyed, and before the next iteration, A new set of temporary variables will be constructed first for this iteration. In general, most of the variables in the model belong to this category, such as the input training data, the output of a normal layer, and so on.
How to save model variables
############################
The model variables we need to save are different depending on the application. For example, if we just want to save the model for future predictions, just saving the model parameters will be enough. But if we need to save a checkpoint for future recovery of current training, then we should save all the persistable variables, and even record the current epoch and step id. It is because even though some model variables are not parameters, they are still essential for model training.
Difference between save_vars、save_params、save_persistables and save_inference_model
##########################################################################
1. :code:`save_inference_model` will prune the inference model based on :code:`feeded_var_names` and :code:`target_vars` , this method will save the ``__model__`` file of the pruned program and the persistable variables in the program.
2. :code:`save_persistables` this method will not save model, it will save all the persistable variables in the program.
3. :code:`save_params` this method will not save model, it will save all the parameters in the program.
4. :code:`save_vars` this method will not save model, it will save the given parameter by user.
:code:`save_persistables` this method is useful for increment training or checkpoint training, it can save persistable variables in program comprehensively, such as parameter variables, optimizer variables, if you need increment training or checkpoint training, please choose this one.
:code:`save_inference_model` this method is useful for inference, it will save persistable variables and pruned program, if you need program and variables for follow-up high performance inference, please choose this one.
:code:`save_vars 和 save_params` there methods are only needed in particular cases, we suppose you already know the purpose of there APIs, there are not recommended for use normally.
Save the model to make prediction for new samples
===================================================
If we save the model to make prediction for new samples, just saving the model parameters will be sufficient. We can use the :code:`fluid.io.save_params()` interface to save model parameters.
For example:
.. code-block:: python
import paddle.fluid as fluid
exe = fluid.Executor(fluid.CPUPlace())
param_path = "./my_paddle_model"
prog = fluid.default_main_program()
fluid.io.save_params(executor=exe, dirname=param_path, main_program=None)
In the example above, by calling the :code:`fluid.io.save_params` function, PaddlePaddle Fluid scans all model variables in the default :code:`fluid.Program` , i.e. :code:`prog` and picks out all model parameters. All these model parameters are saved to the specified :code:`param_path` .
How to load model variables
#############################
Corresponding to saving of model variables, we provide two sets of APIs to load the model parameters and the persistable variables of model.
Load model to make predictions for new samples
================================================
For models saved with :code:`fluid.io.save_params` , you can load them with :code:`fluid.io.load_params`.
For example:
.. code-block:: python
import paddle.fluid as fluid
exe = fluid.Executor(fluid.CPUPlace())
param_path = "./my_paddle_model"
prog = fluid.default_main_program()
fluid.io.load_params(executor=exe, dirname=param_path,
main_program=prog)
In the above example, by calling the :code:`fluid.io.load_params` function, PaddlePaddle Fluid will scan all the model variables in :code:`prog`, filter out all the model parameters, and try to load them from :code:`param_path` .
It is important to note that the :code:`prog` here must be exactly the same as the forward part of the :code:`prog` used when calling :code:`fluid.io.save_params` and cannot contain any operations of parameter updates. If there is an inconsistency between the two, it may cause some variables not to be loaded correctly; if the parameter update operation is incorrectly included, it may cause the parameters to be changed during normal prediction. The relationship between these two :code:`fluid.Program` is similar to the relationship between training :code:`fluid.Program` and test :code:`fluid.Program`, see: :ref:`user_guide_test_while_training_en` .
In addition, special care must be taken that :code:`fluid.default_startup_program()` **must** be run before calling :code:`fluid.io.load_params` . If you run it later, it may overwrite the loaded model parameters and cause an error.
Prediction of the used models and parameters saving
#######################################################
The inference engine provides two interfaces : prediction model saving :code:`fluid.io.save_inference_model` and the prediction model loading :code:`fluid.io.load_inference_model`.
- :code:`fluid.io.save_inference_model`: Please refer to :ref:`api_guide_inference` .
- :code:`fluid.io.load_inference_model`: Please refer to :ref:`api_guide_inference` .
Incremental training
#####################
Incremental training means that a learning system can continuously learn new knowledge from new samples and preserve most of the knowledge that has been learned before. Therefore, incremental learning involves two points: saving the parameters that need to be persisted at the end of the last training, and loading the last saved persistent parameters at the beginning of the next training. Therefore incremental training involves the following APIs:
:code:`fluid.io.save_persistables`, :code:`fluid.io.load_persistables` .
Single-node incremental training
=================================
The general steps of incremental training on a single unit are as follows:
1. At the end of the training, call :code:`fluid.io.save_persistables` to save the persistable parameter to the specified location.
2. After the training startup_program is executed successfully by the executor :code:`Executor`, call :code:`fluid.io.load_persistables` to load the previously saved persistable parameters.
3. Continue training with the executor :code:`Executor` or :code:`ParallelExecutor`.
Example:
.. code-block:: python
import paddle.fluid as fluid
exe = fluid.Executor(fluid.CPUPlace())
path = "./models"
prog = fluid.default_main_program()
fluid.io.save_persistables(exe, path, prog)
In the above example, by calling the :code:`fluid.io.save_persistables` function, PaddlePaddle Fluid will find all persistable variables from all model variables in the default :code:`fluid.Program`, e.t. :code:`prog` , and save them to the specified :code:`path` directory.
.. code-block:: python
import paddle.fluid as fluid
exe = fluid.Executor(fluid.CPUPlace())
path = "./models"
startup_prog = fluid.default_startup_program()
exe.run(startup_prog)
main_prog = fluid.default_main_program()
fluid.io.load_persistables(exe, path, main_prog)
exe.run(main_prog)
In the above example, by calling the :code:`fluid.io.load_persistables` function, PaddlePaddle Fluid will find persistable variables from all model variables in the default :code:`fluid.Program` , e.t. :code:`prog` . and load them one by one from the specified :code:`path` directory to continue training.
The general steps for multi-node incremental training (without distributed large-scale sparse matrices)
=========================================================================================================
There are several differences between multi-node incremental training and single-node incremental training:
1. At the end of the training, when :code:`fluid.io.save_persistables` is called to save the persistence parameters, it is not necessary for all trainers to call this method, usually it is called on the 0th trainer.
2. The parameters of multi-node incremental training are loaded on the PServer side, and the trainer side does not need to load parameters. After the PServers are fully started, the trainer will synchronize the parameters from the PServer.
The general steps for multi-node incremental training (do not enable distributed large-scale sparse matrices) are:
1. At the end of the training, Trainer 0 will call :code:`fluid.io.save_persistables` to save the persistable parameters to the specified :code:`path`.
2. Share all the parameters saved by trainer 0 to all PServers through HDFS or other methods. (each PServer needs to have complete parameters).
3. After the training startup_program is successfully executed by the executor ( :code:`Executor` ), the PServer calls :code:`fluid.io.load_persistables` to load the persistable parameters saved by the 0th trainer.
4. The PServer continues to start PServer_program via the executor :code:`Executor`.
5. All training node trainers conduct training process normally through the executor :code:`Executor` or :code:`ParallelExecutor` .
For trainers whose parameters are to be saved during training, for example:
.. code-block:: python
import paddle.fluid as fluid
exe = fluid.Executor(fluid.CPUPlace())
path = "./models"
trainer_id = 0
if trainer_id == 0:
prog = fluid.default_main_program()
fluid.io.save_persistables(exe, path, prog)
.. code-block:: bash
hadoop fs -mkdir /remote/$path
hadoop fs -put $path /remote/$path
In the above example, the 0 trainer calls the :code:`fluid.io.save_persistables` function. By calling this function, PaddlePaddle Fluid will find all persistable variables in all model variables from default :code:`fluid.Program` , e.t. :code:`prog` , and save them to the specified :code:`path` directory. The stored model is then uploaded to a location accessible for all PServers by invoking a third-party file system (such as HDFS).
For the PServer to be loaded with parameters during training, for example:
.. code-block:: python
import paddle.fluid as fluid
exe = fluid.Executor(fluid.CPUPlace())
path = "./models"
pserver_endpoints = "127.0.0.1:1001,127.0.0.1:1002"
trainers = 4
Training_role == "PSERVER"
config = fluid.DistributeTranspilerConfig()
t = fluid.DistributeTranspiler(config=config)
t.transpile(trainer_id, pservers=pserver_endpoints, trainers=trainers, sync_mode=True)
if training_role == "PSERVER":
current_endpoint = "127.0.0.1:1001"
pserver_prog = t.get_pserver_program(current_endpoint)
pserver_startup = t.get_startup_program(current_endpoint, pserver_prog)
exe.run(pserver_startup)
fluid.io.load_persistables(exe, path, pserver_startup)
exe.run(pserver_prog)
if training_role == "TRAINER":
main_program = t.get_trainer_program()
exe.run(main_program)
In the above example, each PServer obtains the parameters saved by trainer 0 by calling the HDFS command, and obtains the PServer's :code:`fluid.Program` by configuration. PaddlePaddle Fluid will find all persistable variables in all model variables from this :code:`fluid.Program` , e.t. :code:`pserver_startup` , and load them from the specified :code:`path` directory.
doc/fluid/beginners_guide/coding_practice/single_node.rst 0 → 100644
浏览文件 @ 46946b6d
########
单机训练
########
准备工作
########
要进行PaddlePaddle Fluid单机训练，需要先 :ref:`user_guide_prepare_data` 和
:ref:`user_guide_configure_simple_model` 。当\
:ref:`user_guide_configure_simple_model` 完毕后，可以得到两个\
:code:`fluid.Program`， :code:`startup_program` 和 :code:`main_program`。
默认情况下，可以使用 :code:`fluid.default_startup_program()` 与\ :code:`fluid.default_main_program()` 获得全局的 :code:`fluid.Program`。
例如:
.. code-block:: python
import paddle.fluid as fluid
image = fluid.data(name="image", shape=[None, 784], dtype='float32')
label = fluid.data(name="label", shape=[None, 1], dtype='int64')
hidden = fluid.layers.fc(input=image, size=100, act='relu')
prediction = fluid.layers.fc(input=hidden, size=10, act='softmax')
loss = fluid.layers.cross_entropy(input=prediction, label=label)
loss = fluid.layers.mean(loss)
sgd = fluid.optimizer.SGD(learning_rate=0.001)
sgd.minimize(loss)
# Here the fluid.default_startup_program() and fluid.default_main_program()
# has been constructed.
在上述模型配置执行完毕后， :code:`fluid.default_startup_program()` 与\
:code:`fluid.default_main_program()` 配置完毕了。
初始化参数
##########
参数随机初始化
==============
用户配置完模型后，参数初始化操作会被写入到\
:code:`fluid.default_startup_program()` 中。使用 :code:`fluid.Executor()` 运行
这一程序，初始化之后的参数默认被放在全局scope中，即 :code:`fluid.global_scope()` 。例如:
.. code-block:: python
exe = fluid.Executor(fluid.CUDAPlace(0))
exe.run(program=fluid.default_startup_program())
载入预定义参数
==============
在神经网络训练过程中，经常会需要载入预定义模型，进而继续进行训练。\
如何载入预定义参数，请参考 :ref:`user_guide_save_load_vars`。
单卡训练
########
执行单卡训练可以使用 :code:`fluid.Executor()` 中的 :code:`run()` 方法，运行训练\
:code:`fluid.Program` 即可。在运行的时候，用户可以通过 :code:`run(feed=...)`\
参数传入数据；用户可以通过 :code:`run(fetch=...)` 获取输出数据。例如:\
.. code-block:: python
import paddle.fluid as fluid
import numpy
train_program = fluid.Program()
startup_program = fluid.Program()
with fluid.program_guard(train_program, startup_program):
data = fluid.data(name='X', shape=[None, 1], dtype='float32')
hidden = fluid.layers.fc(input=data, size=10)
loss = fluid.layers.mean(hidden)
sgd = fluid.optimizer.SGD(learning_rate=0.001)
sgd.minimize(loss)
use_cuda = True
place = fluid.CUDAPlace(0) if use_cuda else fluid.CPUPlace()
exe = fluid.Executor(place)
# Run the startup program once and only once.
# Not need to optimize/compile the startup program.
startup_program.random_seed=1
exe.run(startup_program)
# Run the main program directly without compile.
x = numpy.random.random(size=(10, 1)).astype('float32')
loss_data, = exe.run(train_program,
feed={"X": x},
fetch_list=[loss.name])
# Or use CompiledProgram:
compiled_prog = fluid.CompiledProgram(train_program)
loss_data, = exe.run(compiled_prog,
feed={"X": x},
fetch_list=[loss.name])
多卡训练
#######################
在多卡训练中，你可以使用 :code:`fluid.CompiledProgram` 来编译 :code:`fluid.Program` ，然后调用 :code:`with_data_parallel` 。例如：
.. code-block:: python
# NOTE: If you use CPU to run the program, you need
# to specify the CPU_NUM, otherwise, fluid will use
# all the number of the logic cores as the CPU_NUM,
# in that case, the batch size of the input should be
# greater than CPU_NUM, if not, the process will be
# failed by an exception.
if not use_cuda:
os.environ['CPU_NUM'] = str(2)
compiled_prog = fluid.CompiledProgram(
train_program).with_data_parallel(
loss_name=loss.name)
loss_data, = exe.run(compiled_prog,
feed={"X": x},
fetch_list=[loss.name])
注释：
1. :ref:`cn_api_fluid_CompiledProgram` 会将传入的 :code:`fluid.Program` 转为计算图，即Graph，因为 :code:`compiled_prog` 与传入的 :code:`train_program` 是完全不同的对象，目前还不能够对 :code:`compiled_prog` 进行保存。
2. 多卡训练也可以使用 :ref:`cn_api_fluid_ParallelExecutor` ，但是现在推荐使用 :ref:`cn_api_fluid_CompiledProgram` .
3. 如果 :code:`exe` 是用CUDAPlace来初始化的，模型会在GPU中运行。在显卡训练模式中，所有的显卡都将被占用。用户可以配置 `CUDA_VISIBLE_DEVICES <http://www.acceleware.com/blog/cudavisibledevices-masking-gpus>`_ 以更改被占用的显卡。
4. 如果 :code:`exe` 是用CPUPlace来初始化的，模型会在CPU中运行。在这种情况下，多线程用于运行模型，同时线程的数目和逻辑核的数目相等。用户可以配置 ``CPU_NUM`` 以更改使用中的线程数目。
进阶使用
###############
.. toctree::
:maxdepth: 2
test_while_training.rst
doc/fluid/beginners_guide/coding_practice/single_node_en.rst 0 → 100644
浏览文件 @ 46946b6d
#####################
Single-node training
#####################
Preparation
############
To perform single-node training in PaddlePaddle Fluid, you need to read :ref:`user_guide_prepare_data_en` and :ref:`user_guide_configure_simple_model_en` . When you have finished reading :ref:`user_guide_configure_simple_model_en` , you can get two :code:`fluid.Program`, namely :code:`startup_program` and :code:`main_program` . By default, you can use :code:`fluid.default_startup_program()` and :code:`fluid.default_main_program()` to get global :code:`fluid.Program` .
For example:
.. code-block:: python
import paddle.fluid as fluid
image = fluid.data(name="image", shape=[None, 784], dtype='float32')
label = fluid.data(name="label", shape=[None, 1], dtype='int64')
hidden = fluid.layers.fc(input=image, size=100, act='relu')
prediction = fluid.layers.fc(input=hidden, size=10, act='softmax')
loss = fluid.layers.cross_entropy(input=prediction, label=label)
loss = fluid.layers.mean(loss)
sgd = fluid.optimizer.SGD(learning_rate=0.001)
sgd.minimize(loss)
# Here the fluid.default_startup_program() and fluid.default_main_program()
# has been constructed.
After the configuration of model, the configurations of :code:`fluid.default_startup_program()` and :code:`fluid.default_main_program()` have been finished.
Initialize Parameters
#######################
Random Initialization of Parameters
====================================
After the configuration of model,the initialization of parameters will be written into :code:`fluid.default_startup_program()` . By running this program in :code:`fluid.Executor()` , the random initialization of parameters will be finished in global scope, i.e. :code:`fluid.global_scope()` .For example:
.. code-block:: python
exe = fluid.Executor(fluid.CUDAPlace(0))
exe.run(program=fluid.default_startup_program())
Load Predefined Parameters
===========================
In the neural network training, predefined models are usually loaded to continue training. For how to load predefined parameters, please refer to :ref:`user_guide_save_load_vars_en`.
Single-card Training
#####################
Single-card training can be performed through calling :code:`run()` of :code:`fluid.Executor()` to run training :code:`fluid.Program` .
In the runtime, users can feed data with :code:`run(feed=...)` and get output data with :code:`run(fetch=...)` . For example:
.. code-block:: python
import paddle.fluid as fluid
import numpy
train_program = fluid.Program()
startup_program = fluid.Program()
with fluid.program_guard(train_program, startup_program):
data = fluid.data(name='X', shape=[None, 1], dtype='float32')
hidden = fluid.layers.fc(input=data, size=10)
loss = fluid.layers.mean(hidden)
sgd = fluid.optimizer.SGD(learning_rate=0.001)
sgd.minimize(loss)
use_cuda = True
place = fluid.CUDAPlace(0) if use_cuda else fluid.CPUPlace()
exe = fluid.Executor(place)
# Run the startup program once and only once.
# Not need to optimize/compile the startup program.
startup_program.random_seed=1
exe.run(startup_program)
# Run the main program directly without compile.
x = numpy.random.random(size=(10, 1)).astype('float32')
loss_data, = exe.run(train_program,
feed={"X": x},
fetch_list=[loss.name])
# Or use CompiledProgram:
compiled_prog = fluid.CompiledProgram(train_program)
loss_data, = exe.run(compiled_prog,
feed={"X": x},
fetch_list=[loss.name])
Multi-card Training
#######################
In multi-card training, you can use :code:`fluid.CompiledProgram` to compile the :code:`fluid.Program`, and then call :code:`with_data_parallel`. For example:
.. code-block:: python
# NOTE: If you use CPU to run the program, you need
# to specify the CPU_NUM, otherwise, fluid will use
# all the number of the logic core as the CPU_NUM,
# in that case, the batch size of the input should be
# greater than CPU_NUM, if not, the process will be
# failed by an exception.
if not use_cuda:
os.environ['CPU_NUM'] = str(2)
compiled_prog = fluid.CompiledProgram(
train_program).with_data_parallel(
loss_name=loss.name)
loss_data, = exe.run(compiled_prog,
feed={"X": x},
fetch_list=[loss.name])
Notes:
1. :ref:`api_fluid_CompiledProgram` will convert the input Program into a computational graph, and :code:`compiled_prog` is a completely different object from the incoming :code:`train_program`. At present, :code:`compiled_prog` can not be saved.
2. Multi-card training can also be used: ref:`api_fluid_ParallelExecutor` , but now it is recommended to use: :ref:`api_fluid_CompiledProgram`.
3. If :code:`exe` is initialized with CUDAPlace, the model will be run in GPU. In the mode of graphics card training, all graphics card will be occupied. Users can configure `CUDA_VISIBLE_DEVICES <http://www.acceleware.com/blog/cudavisibledevices-masking-gpus>`_ to change graphics cards that are being used.
4. If :code:`exe` is initialized with CPUPlace, the model will be run in CPU. In this situation, the multi-threads are used to run the model, and the number of threads is equal to the number of logic cores. Users can configure `CPU_NUM` to change the number of threads that are being used.
Advanced Usage
###############
.. toctree::
:maxdepth: 2
test_while_training_en.rst
doc/fluid/beginners_guide/coding_practice/test_while_training.rst 0 → 100644
浏览文件 @ 46946b6d
.. _user_guide_test_while_training:
##################
训练过程中评测模型
##################
模型的测试评价与训练的 :code:`fluid.Program` 不同。在测试评价中:
1. 测试评价不进行反向传播，不优化更新参数。
2. 测试评价执行的操作可以不同。
* 例如 BatchNorm 操作，在训练和测试时执行不同的算法。
* 测试评价模型与训练模型可以是完全不同的模型。
生成测试 :code:`fluid.Program`
#################################
通过克隆训练 :code:`fluid.Program` 生成测试 :code:`fluid.Program`
=======================================================================
用 :code:`Program.clone()` 方法可以复制出新的 :code:`fluid.Program` 。 通过设置
:code:`Program.clone(for_test=True)` 复制含有用于测试的操作 :code:`fluid.Program` 。简单的使用方法如下:
.. code-block:: python
import paddle.fluid as fluid
image = fluid.data(name="image", shape=[None, 784], dtype='float32')
label = fluid.data(name="label", shape=[None, 1], dtype="int64")
prediction = fluid.layers.fc(
input=fluid.layers.fc(input=image, size=100, act='relu'),
size=10,
act='softmax'
)
loss = fluid.layers.mean(fluid.layers.cross_entropy(input=prediction, label=label))
acc = fluid.layers.accuracy(input=prediction, label=label)
test_program = fluid.default_main_program().clone(for_test=True)
adam = fluid.optimizer.Adam(learning_rate=0.001)
adam.minimize(loss)
在使用 :code:`Optimizer` 之前，将 :code:`fluid.default_main_program()` 复制\
成一个 :code:`test_program` 。之后使用测试数据运行 :code:`test_program`,\
就可以做到运行测试程序，而不影响训练结果。
分别配置训练 :code:`fluid.Program` 和测试 :code:`fluid.Program`
=====================================================================
如果训练程序和测试程序相差较大时，用户也可以通过完全定义两个不同的
:code:`fluid.Program`，分别进行训练和测试。在PaddlePaddle Fluid中，\
所有的参数都有名字。如果两个不同的操作，甚至两个不同的网络使用了同样名字的参数，\
那么他们的值和内存空间都是共享的。
PaddlePaddle Fluid中使用 :code:`fluid.unique_name` 包来随机初始化用户未定义的\
参数名称。通过 :code:`fluid.unique_name.guard` 可以确保多次调用某函数\
参数初始化的名称一致。
例如:
.. code-block:: python
import paddle.fluid as fluid
def network(is_test):
image = fluid.data(name="image", shape=[None, 784], dtype='float32')
label = fluid.data(name="label", shape=[None, 1], dtype="int64")
hidden = fluid.layers.fc(input=image, size=100, act="relu")
hidden = fluid.layers.batch_norm(input=hidden, is_test=is_test)
...
return loss
with fluid.unique_name.guard():
train_loss = network(is_test=False)
sgd = fluid.optimizer.SGD(0.001)
sgd.minimize(train_loss)
test_program = fluid.Program()
with fluid.unique_name.guard():
with fluid.program_guard(test_program, fluid.Program()):
test_loss = network(is_test=True)
# fluid.default_main_program() is the train program
# fluid.test_program is the test program
执行测试 :code:`fluid.Program`
#################################
使用 :code:`Executor` 执行测试 :code:`fluid.Program`
=======================================================
用户可以使用 :code:`Executor.run(program=...)` 来执行测试
:code:`fluid.Program`。
例如
.. code-block:: python
exe = fluid.Executor(fluid.CPUPlace())
test_acc = exe.run(program=test_program, feed=test_data_batch, fetch_list=[acc])
print 'Test accuracy is ', test_acc
使用 :code:`ParallelExecutor` 执行测试 :code:`fluid.Program`
===============================================================
用户可以使用训练用的 :code:`ParallelExecutor` 与测试 :code:`fluid.Program`
一起，新建一个测试的 :code:`ParallelExecutor` ；再使用测试
:code:`ParallelExecutor.run` 来执行测试。
例如:
.. code-block:: python
train_exec = fluid.ParallelExecutor(use_cuda=True, loss_name=loss.name)
test_exec = fluid.ParallelExecutor(use_cuda=True, share_vars_from=train_exec,
main_program=test_program)
test_acc = test_exec.run(fetch_list=[acc], ...)
doc/fluid/beginners_guide/coding_practice/test_while_training_en.rst 0 → 100644
浏览文件 @ 46946b6d
.. _user_guide_test_while_training_en:
##############################
Evaluate model while training
##############################
:code:`fluid.Program` for model test and evaluation is different from the one for training. In the test and evalution phase:
1. There is no back propagation and no process of optimizing and updating parameters in evaluation and test.
2. Operations in model evaluation can be different.
* Take the operator BatchNorm for example, algorithms are different in train and test.
* Evaluation model and training model can be totally different.
Generate :code:`fluid.Program` for test
#######################################
Generate test :code:`fluid.Program` by cloning training :code:`fluid.Program`
============================================================================
:code:`Program.clone()` can generate a copied new :code:`fluid.Program` . You can generate a copy of Program with operations applied for test by setting :code:`Program.clone(for_test=True)` . Simple usage is as follows:
.. code-block:: python
import paddle.fluid as fluid
image = fluid.data(name="image", shape=[None, 784], dtype='float32')
label = fluid.data(name="label", shape=[None, 1], dtype="int64")
prediction = fluid.layers.fc(
input=fluid.layers.fc(input=image, size=100, act='relu'),
size=10,
act='softmax'
)
loss = fluid.layers.mean(fluid.layers.cross_entropy(input=prediction, label=label))
acc = fluid.layers.accuracy(input=prediction, label=label)
test_program = fluid.default_main_program().clone(for_test=True)
adam = fluid.optimizer.Adam(learning_rate=0.001)
adam.minimize(loss)
Before using :code:`Optimizer` , please copy :code:`fluid.default_main_program()` into a :code:`test_program` . Then you can pass test data to :code:`test_program` so that you can run test program without influencing training result.
Configure training :code:`fluid.Program` and test :code:`fluid.Program` individually
=====================================================================================
If the training program is largely different from test program, you can define two totally different :code:`fluid.Program` , and perform training and test individually. In PaddlePaddle Fluid, all parameters are named. If two different operations or even two different networks use parameters with the same name, the value and memory space of these parameters are shared.
Fluid adopts :code:`fluid.unique_name` package to randomly initialize the names of unnamed parameters. :code:`fluid.unique_name.guard` can keep the initialized names consistent across multiple times of calling some function.
For example:
.. code-block:: python
import paddle.fluid as fluid
def network(is_test):
image = fluid.data(name="image", shape=[None, 784], dtype='float32')
label = fluid.data(name="label", shape=[None, 1], dtype="int64")
hidden = fluid.layers.fc(input=image, size=100, act="relu")
hidden = fluid.layers.batch_norm(input=hidden, is_test=is_test)
...
return loss
with fluid.unique_name.guard():
train_loss = network(is_test=False)
sgd = fluid.optimizer.SGD(0.001)
sgd.minimize(train_loss)
test_program = fluid.Program()
with fluid.unique_name.guard():
with fluid.program_guard(test_program, fluid.Program()):
test_loss = network(is_test=True)
# fluid.default_main_program() is the train program
# fluid.test_program is the test program
Perform test :code:`fluid.Program`
###################################
Run test :code:`fluid.Program` with :code:`Executor`
=======================================================
You can run test :code:`fluid.Program` with :code:`Executor.run(program=...)` .
For example:
.. code-block:: python
exe = fluid.Executor(fluid.CPUPlace())
test_acc = exe.run(program=test_program, feed=test_data_batch, fetch_list=[acc])
print 'Test accuracy is ', test_acc
Run test :code:`fluid.Program` with :code:`ParallelExecutor`
=====================================================================
You can use :code:`ParallelExecutor` for training and :code:`fluid.Program` for test to create a new test :code:`ParallelExecutor` ; then use test :code:`ParallelExecutor.run` to run test process.
For example:
.. code-block:: python
train_exec = fluid.ParallelExecutor(use_cuda=True, loss_name=loss.name)
test_exec = fluid.ParallelExecutor(use_cuda=True, share_vars_from=train_exec,
main_program=test_program)
test_acc = test_exec.run(fetch_list=[acc], ...)
doc/fluid/beginners_guide/hapi.md 已删除 100644 → 0
浏览文件 @ 685da70d
此差异已折叠。
doc/fluid/beginners_guide/index_cn.rst
浏览文件 @ 46946b6d
......@@ -20,4 +20,3 @@ PaddlePaddle (PArallel Distributed Deep LEarning)是一个易用、高效、灵
basic_concept/index_cn.rst
coding_practice/index_cn.rst
hapi.md
doc/fluid/beginners_guide/index_en.rst 0 → 100644
浏览文件 @ 46946b6d
################
Beginner's Guide
################
PaddlePaddle (PArallel Distributed Deep LEarning) is a
simple, efficient and extensible deep learning framework.
Please refer to `PaddlePaddle Github <https://github.com/PaddlePaddle/Paddle>`_ for details, and `Release Note <../release_note_en.html>`_ for features incorporated in current version.
Let's start with studying basic concept of PaddlePaddle:
- `Basic Concept <../beginners_guide/basic_concept/index_en.html>`_ ： introduce the basic concept and usage of Paddle
If you have mastered the basic concept of Paddle and you expect to model and build your own network according to the actual problems, you can refer to some details of the use of paddle in the Coding Practice :
- `Coding Practice <../beginners_guide/coding_practice/index_en.html>`_ ： introduce how to model and build your own network for practical problems
.. toctree::
:hidden:
basic_concept/index_en.rst
coding_practice/index_en.rst
doc/fluid/beginners_guide/install/Tables_en.md 0 → 100755
浏览文件 @ 46946b6d
此差异已折叠。
doc/fluid/beginners_guide/quick_start.rst 0 → 100644
浏览文件 @ 46946b6d
此差异已折叠。
doc/fluid/beginners_guide/quick_start_cn.rst 0 → 100644
浏览文件 @ 46946b6d
此差异已折叠。
Markdown is supported
0% 或 .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑！
想要评论请 注册 或