diff --git a/doc/fluid/api_guides/fluid.rst b/doc/fluid/api_guides/fluid.rst new file mode 100755 index 0000000000000000000000000000000000000000..65c21a14f6475509636ee7842bf5a712f954d540 --- /dev/null +++ b/doc/fluid/api_guides/fluid.rst @@ -0,0 +1,172 @@ +###### +fluid +###### +.. _cn_api_fluid_AsyncExecutor: + +AsyncExecutor +>>>>>>>>>>>>>>> + +.. py:class:: paddle.fluid.AsyncExecutor(place=None) + +Python中的异步执行器。AsyncExecutor利用多核处理器和数据排队的强大功能,使数据读取和融合解耦,每个线程并行运行。 + +AsyncExecutor不是在python端读取数据,而是接受一个训练文件列表,该列表将在c++中检索,然后训练输入将被读取、解析并在c++代码中提供给训练网络。 + +AsyncExecutor正在积极开发,API可能在不久的将来会发生变化。 + +参数: + - **place** (fluid.CPUPlace|None) - 指示 executor 将在哪个设备上运行。目前仅支持CPU + +**代码示例:** + +.. code-block:: python + + data_feed = fluid.DataFeedDesc('data.proto') + startup_program = fluid.default_startup_program() + main_program = fluid.default_main_program() + filelist = ["train_data/part-%d" % i for i in range(100)] + thread_num = len(filelist) / 4 + place = fluid.CPUPlace() + async_executor = fluid.AsyncExecutor(place) + async_executor.run_startup_program(startup_program) + epoch = 10 + for i in range(epoch): + async_executor.run(main_program, + data_feed, + filelist, + thread_num, + [acc], + debug=False) + +.. note:: + + 对于并行gpu调试复杂网络,您可以在executor上测试。他们有完全相同的参数,并可以得到相同的结果。 + + 目前仅支持CPU + +.. _cn_api_fluid_DataFeedDesc: + +DataFeedDesc +>>>>>>>>>>>>>> + +.. py:class:: paddle.fluid.DataFeedDesc(proto_file) + +数据描述符,描述输入训练数据格式。 + +这个类目前只用于AsyncExecutor(有关类AsyncExecutor的简要介绍,请参阅注释) + +DataFeedDesc应由来自磁盘的有效protobuf消息初始化: + +.. code-block:: python + + data_feed = fluid.DataFeedDesc('data.proto') + +可以参考 :code:`paddle/fluid/framework/data_feed.proto` 查看我们如何定义message + +一段典型的message可能是这样的: + +.. code-block:: text + + name: "MultiSlotDataFeed" + batch_size: 2 + multi_slot_desc { + slots { + name: "words" + type: "uint64" + is_dense: false + is_used: true + } + slots { + name: "label" + type: "uint64" + is_dense: false + is_used: true + } + } + +但是,用户通常不应该关心消息格式;相反,我们鼓励他们在将原始日志文件转换为AsyncExecutor可以接受的训练文件的过程中,使用 :code:`Data Generator` 生成有效数据描述。 + +DataFeedDesc也可以在运行时更改。一旦你熟悉了每个字段的含义,您可以修改它以更好地满足您的需要。例如: + +.. code-block:: text + + data_feed.set_batch_size(128) + data_feed.set_dense_slots('wd') # The slot named 'wd' will be dense + data_feed.set_use_slots('wd') # The slot named 'wd' will be used + + #Finally, the content can be dumped out for debugging purpose: + + print(data_feed.desc()) + +参数: + - **proto_file** (string) - 包含数据feed中描述的磁盘文件 + + +.. py:method:: set_batch_size(self, batch_size) + +设置batch size,训练期间有效 + + +参数: + - batch_size:batch size + +**代码示例:** + +.. code-block:: python + + data_feed = fluid.DataFeedDesc('data.proto') + data_feed.set_batch_size(128) + +.. py:method:: set_dense_slots(self, dense_slots_name) + +指定slot经过设置后将变成密集的slot,仅在训练期间有效。 + +密集slot的特征将被输入一个Tensor,而稀疏slot的特征将被输入一个lodTensor + + +参数: + - **dense_slots_name** : slot名称的列表,这些slot将被设置为密集的 + +**代码示例:** + +.. code-block:: python + + data_feed = fluid.DataFeedDesc('data.proto') + data_feed.set_dense_slots(['words']) + +.. note:: + + 默认情况下,所有slot都是稀疏的 + +.. py:method:: set_use_slots(self, use_slots_name) + + +设置一个特定的slot是否用于训练。一个数据集包含了很多特征,通过这个函数可以选择哪些特征将用于指定的模型。 + +参数: + - **use_slots_name** :将在训练中使用的slot名列表 + +**代码示例:** + +.. code-block:: python + + data_feed = fluid.DataFeedDesc('data.proto') + data_feed.set_use_slots(['words']) + +.. note:: + + 默认值不用于所有slot + + +.. py:method:: desc(self) + +返回此DataFeedDesc的protobuf信息 + +返回:一个message字符串 + +**代码示例:** + +.. code-block:: python + + data_feed = fluid.DataFeedDesc('data.proto') + print(data_feed.desc()) \ No newline at end of file diff --git a/doc/fluid/api_guides/hy-3.rst b/doc/fluid/api_guides/hy-3.rst new file mode 100755 index 0000000000000000000000000000000000000000..e08cc3d384acaaae8b5581079c4431634eafecbc --- /dev/null +++ b/doc/fluid/api_guides/hy-3.rst @@ -0,0 +1,57 @@ + +.. _cn_api_fluid_layers_linear_chain_crf: + +linear_chain_crf +>>>>>>>>>>>>>>>>>> + +.. py:class:: paddle.fluid.layers.linear_chain_crf(input, label, param_attr=None) + +线性链条件随机场(Linear Chain CRF) + +条件随机场定义间接概率图,节点代表随机变量,边代表两个变量之间的依赖。CRF学习条件概率 :math:`P\left ( Y|X \right )` , :math:`X = \left ( x_{1},x_{2},...,x_{n} \right )` 是结构性输入,:math:`Y = \left ( y_{1},y_{2},...,y_{n} \right )` 为输入标签。 + +线性链条件随机场(Linear Chain CRF)是特殊的条件随机场(CRF),有利于序列标注任务。序列标注任务不为输入设定许多条件依赖。唯一的限制是输入和输出必须是现行序列。因此类似CRF的图是一个简单的链或者线,也就是线性链随机场(linear chain CRF)。 + +该操作符实现了线性链条件随机场(linear chain CRF)的前后向算法。详情请参照http://www.cs.columbia.edu/~mcollins/fb.pdf和http://cseweb.ucsd.edu/~elkan/250Bwinter2012/loglinearCRFs.pdf。 + +1.这里x代表Emission + +2.Transition的第一维度值,代表起始权重,这里用a表示 + +3.Transition的下一维值,代表末尾权重,这里用b表示 + +4.Transition剩下的值,代表转移权重,这里用w表示 + +5.Label用s表示 + +长度为L的序列s的概率定义如下: + +.. math:: + + P(s) = (1/Z)exp(a_{s_{1}}+b_{s_{L}}+sum_{L}^{l=1}x_{s+{l}}+sum_{L}^{l=2}w_{s_{l-1},s_{l}}) + +其中Z是正则化值,所有可能序列的P(s)之和为1,x是线性链条件随机场(linear chain CRF)的发射(emission)特征权重。 + +线性链条件随机场最终输出mini-batch每个训练样本的条件概率的对数 + +注意: + 1.条件随机场(CRF)的特征函数由发射特征(emission feature)和转移特征(transition feature)组成。发射特征(emission feature)权重在调用函数前计算,而不在函数里计算。 + + 2.由于该函数对所有可能序列的进行全局正则化,发射特征(emission feature)权重应是未缩放的。因此如果该函数带有发射特征(emission feature),并且发射特征是任意非线性激活函数的输出,则请勿调用该函数。 + + 3.Emission的第二维度必须和标记数字(tag number)相同 + +参数: + - **input** (Variable,LoDTensor,默认float类型LoDTensor) - 一个二维LoDTensor,shape为[N*D],N是mini-batch的大小,D是总标记数。线性链条件随机场的未缩放发射权重矩阵 + - **input** (Tensor,默认float类型LoDTensor) - 一个二维张量,shape为[(D+2)*D]。linear_chain_crf操作符的可学习参数。更多详情见operator注释 + - **label** (Variable,LoDTensor,默认int64类型LoDTensor) - shape为[N*10的LoDTensor,N是mini-batch的总元素数 + - **param_attr** (ParamAttr) - 可学习参数的属性 + +返回: + output(Variable,Tensor,默认float类型Tensor):shape为[N*D]的二维张量。Emission的指数。这是前向计算中的中间计算结果,在后向计算中还会复用 + + output(Variable,Tensor,默认float类型Tensor):shape为[(D+2)*D]的二维张量。Transition的指数。这是前向计算中的中间计算结果,在后向计算中还会复用 + + output(Variable,Tensor,默认float类型Tensor):mini-batch每个训练样本的条件概率的对数。这是一个shape为[S*1]的二维张量,S是mini-batch的序列数。注:S等同于mini-batch的序列数。输出不再是LoDTensor + +返回类型:output(Variable) diff --git a/doc/fluid/api_guides/hy-second.rst b/doc/fluid/api_guides/hy-second.rst new file mode 100755 index 0000000000000000000000000000000000000000..f34447cfe5e78fb3eb29f8929607e207c46003e1 --- /dev/null +++ b/doc/fluid/api_guides/hy-second.rst @@ -0,0 +1,152 @@ +################ +hy_initializer +################ + +.. _cn_api_fluid_initializer_XavierInitializer: + +XavierInitializer +>>>>>>>>>>>>>>>>>>> +.. py:class:: paddle.fluid.initializer.XavierInitializer(uniform=True, fan_in=None, fan_out=None, seed=0) + +该类实现Xavier权重初始化方法( Xavier weight initializer),Xavier权重初始化方法出自Xavier Glorot和Yoshua Bengio的论文 `Understanding the difficulty of training deep feedforward neural networks `_ + +该初始化函数用于保持所有层的梯度尺度几乎一致。 + +在均匀分布的情况下,取值范围为[-x,x],其中: + +.. math:: + + x = \sqrt{\frac{6.0}{fan\_in+fan\_out}} + +正态分布的情况下,均值为0,标准差为: + +.. math:: + + x = \sqrt{\frac{2.0}{fan\_in+fan\_out}} + +参数: + - **uniform** (bool) - 是否用均匀分布或者正态分布 + - **fan_in** (float) - 用于Xavier初始化的fan_in。如果为None,fan_in沿伸自变量 + - **fan_out** (float) - 用于Xavier初始化的fan_out。如果为None,fan_out沿伸自变量 + - **seed** (int) - 随机种子 + +.. note:: + + 在大多数情况下推荐将fan_in和fan_out设置为None + +**代码示例**: + +.. code-block:: python + + fc = fluid.layers.fc( + input=queries, size=10, + param_attr=fluid.initializer.Xavier(uniform=False)) + + +.. _cn_api_fluid_initializer_MSRAInitializer: + +MSRAInitializer +>>>>>>>>>>>>>>>>> + +.. py:class:: paddle.fluid.initializer.MSRAInitializer(uniform=True, fan_in=None, seed=0) + +实现MSRA初始化(a.k.a. Kaiming初始化) + +该类实现权重初始化方法,方法来自Kaiming He,Xiangyu Zhang,Shaoqing Ren 和 Jian Sun所写的论文: `Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification `_ 。这是一个鲁棒性特别强的初始化方法,并且适应了非线性激活函数(rectifier nonlinearities)。 + + 在均匀分布中,范围为[-x,x],其中: + +.. math:: + + x = \sqrt{\frac{6.0}{fan\_in}} + +在正态分布中,均值为0,标准差为: + +.. math:: + + \sqrt{\frac{2.0}{fan\_in}} + +参数: + - **uniform** (bool) - 是否用均匀分布或正态分布 + - **fan_in** (float) - MSRAInitializer的fan_in。如果为None,fan_in沿伸自变量 + - **seed** (int) - 随机种子 + +.. note:: + + 在大多数情况下推荐设置fan_in为None + +**代码示例**: + +.. code-block:: python + + fc = fluid.layers.fc( + input=queries, size=10, + param_attr=fluid.initializer.MSRA(uniform=False)) + +################## +nets +################## + +.. _cn_api_fluid_nets_glu: +glu +>>>> +.. py:class:: paddle.fluid.nets.glu(input, dim=-1) +The Gated Linear Units(GLU)由切分(split),sigmoid激活函数和按元素相乘组成。沿着给定维将input拆分成两个大小相同的部分,a和b,计算如下: + +.. math:: + + GLU(a,b) = a\bigotimes \sigma (b) + +参考论文: `Language Modeling with Gated Convolutional Networks `_ + +参数: + - **input** (Variable) - 输入变量,张量或者LoDTensor + - **dim** (int) - 拆分的维度。如果 :math: `dim<0`,拆分的维为 :math: `rank(input)+dim`。默认为-1 + +返回:变量 —— 变量的大小为输入的一半 + +返回类型:变量(Variable) + +**代码示例:** + +.. code-block:: python + + data = fluid.layers.data(name="words", shape=[3, 6, 9], dtype="float32") + output = fluid.nets.glu(input=data, dim=1) # shape of output: [3, 3, 9] + + +.. _cn_api_fluid_nets_sequence_conv_pool: +sequence_conv_pool +>>>>>>>>>>>>>>>>>>> + +.. py:class:: paddle.fluid.nets.sequence_conv_pool(input, num_filters, filter_size, param_attr=None, act='sigmoid', pool_type='max') + +sequence_conv_pool由序列卷积和池化组成 + +参数: + - **input** (Variable) - sequence_conv的输入,支持变量时间长度输入序列。当前输入为shape为(T,N)的矩阵,T是mini-batch中的总时间步数,N是input_hidden_size + - **num_filters** (int)- 滤波器数 + - **filter_size** (int)- 滤波器大小 + - **param_attr** (ParamAttr) - Sequence_conv层的参数。默认:None + - **act** (str) - Sequence_conv层的激活函数类型。默认:sigmoid + - **pool_type** (str)- 池化类型。可以是max-pooling的max,average-pooling的average,sum-pooling的sum,sqrt-pooling的sqrt。默认max + +返回:序列卷积(Sequence Convolution)和池化(Pooling)的结果 + + +返回类型:变量(Variable) + +**代码示例**: + +.. code-block:: python + + input_dim = len(word_dict) + emb_dim = 128 + hid_dim = 512 + data = fluid.layers.data( ame="words", shape=[1], dtype="int64", lod_level=1) + emb = fluid.layers.embedding(input=data, size=[input_dim, emb_dim], is_sparse=True) + seq_conv = fluid.nets.sequence_conv_pool(input=emb, + num_filters=hid_dim, + filter_size=3, + act="tanh", + pool_type="sqrt") \ No newline at end of file diff --git a/doc/fluid/book/fit_a_line/README.md b/doc/fluid/book/fit_a_line/README.md new file mode 120000 index 0000000000000000000000000000000000000000..bd2a37a3c4f2897d1f62ec205c5e0f2f51000b11 --- /dev/null +++ b/doc/fluid/book/fit_a_line/README.md @@ -0,0 +1 @@ +../../../../external/book/01.fit_a_line/README.md \ No newline at end of file diff --git a/doc/fluid/book/fit_a_line/image b/doc/fluid/book/fit_a_line/image new file mode 120000 index 0000000000000000000000000000000000000000..d05fb9cdba7453d935f59f0eda3519234ede4459 --- /dev/null +++ b/doc/fluid/book/fit_a_line/image @@ -0,0 +1 @@ +../../../../external/book/01.fit_a_line/image/ \ No newline at end of file diff --git a/doc/fluid/book/image_classification/README.md b/doc/fluid/book/image_classification/README.md new file mode 120000 index 0000000000000000000000000000000000000000..52413a256afad4df447e622495b62ddb7a5eef3f --- /dev/null +++ b/doc/fluid/book/image_classification/README.md @@ -0,0 +1 @@ +../../../../external/book/03.image_classification/README.md \ No newline at end of file diff --git a/doc/fluid/book/image_classification/image b/doc/fluid/book/image_classification/image new file mode 120000 index 0000000000000000000000000000000000000000..afd28996d9fca9c55d5ea34043ac24fcc86c213c --- /dev/null +++ b/doc/fluid/book/image_classification/image @@ -0,0 +1 @@ +../../../../external/book/03.image_classification/image/ \ No newline at end of file diff --git a/doc/fluid/book/index_en.rst b/doc/fluid/book/index_en.rst index d7710a791f77b06b0dcffb5103b6eaaa9e1d5b68..2242e72520a098c75a33ed2ee53df30f698451f5 100644 --- a/doc/fluid/book/index_en.rst +++ b/doc/fluid/book/index_en.rst @@ -2,7 +2,13 @@ Book ====== .. toctree:: - :maxdepth: 1 + :maxdepth: 1 - ../beginners_guide/basics/index.rst - ../beginners_guide/quick_start/index.rst + fit_a_line/README.md + recognize_digits/README.md + image_classification/README.md + word2vec/README.md + recommender_system/README.md + understand_sentiment/README.md + label_semantic_roles/README.md + machine_translation/README.md diff --git a/doc/fluid/book/label_semantic_roles/README.md b/doc/fluid/book/label_semantic_roles/README.md new file mode 120000 index 0000000000000000000000000000000000000000..0b3329adad969993a9ebf770b1fc34363bf900b2 --- /dev/null +++ b/doc/fluid/book/label_semantic_roles/README.md @@ -0,0 +1 @@ +../../../../external/book/07.label_semantic_roles/README.md \ No newline at end of file diff --git a/doc/fluid/book/label_semantic_roles/image b/doc/fluid/book/label_semantic_roles/image new file mode 120000 index 0000000000000000000000000000000000000000..0eb76d55feab0c443e7b1fab3da870b193f166b2 --- /dev/null +++ b/doc/fluid/book/label_semantic_roles/image @@ -0,0 +1 @@ +../../../../external/book/07.label_semantic_roles/image/ \ No newline at end of file diff --git a/doc/fluid/book/machine_translation/README.md b/doc/fluid/book/machine_translation/README.md new file mode 120000 index 0000000000000000000000000000000000000000..a3507c5a5f4181e2279c1e256920b7e6c93b6c05 --- /dev/null +++ b/doc/fluid/book/machine_translation/README.md @@ -0,0 +1 @@ +../../../../external/book/08.machine_translation/README.md \ No newline at end of file diff --git a/doc/fluid/book/machine_translation/image b/doc/fluid/book/machine_translation/image new file mode 120000 index 0000000000000000000000000000000000000000..ed4d2366ec7d3fb3d03e0e4643cf9025bed4f698 --- /dev/null +++ b/doc/fluid/book/machine_translation/image @@ -0,0 +1 @@ +../../../../external/book/08.machine_translation/image/ \ No newline at end of file diff --git a/doc/fluid/book/recognize_digits/README.md b/doc/fluid/book/recognize_digits/README.md new file mode 120000 index 0000000000000000000000000000000000000000..7106e07a423d3a8184b62da2401eb7bb3f307031 --- /dev/null +++ b/doc/fluid/book/recognize_digits/README.md @@ -0,0 +1 @@ +../../../../external/book/02.recognize_digits/README.md \ No newline at end of file diff --git a/doc/fluid/book/recognize_digits/image b/doc/fluid/book/recognize_digits/image new file mode 120000 index 0000000000000000000000000000000000000000..925f1617634b19104d7795071cbaae7b55bd84df --- /dev/null +++ b/doc/fluid/book/recognize_digits/image @@ -0,0 +1 @@ +../../../../external/book/02.recognize_digits/image/ \ No newline at end of file diff --git a/doc/fluid/book/recommender_system/README.md b/doc/fluid/book/recommender_system/README.md new file mode 120000 index 0000000000000000000000000000000000000000..8a1e846978fcb3d96e2d226e1baea1797cd1761c --- /dev/null +++ b/doc/fluid/book/recommender_system/README.md @@ -0,0 +1 @@ +../../../../external/book/05.recommender_system/README.md \ No newline at end of file diff --git a/doc/fluid/book/recommender_system/image b/doc/fluid/book/recommender_system/image new file mode 120000 index 0000000000000000000000000000000000000000..a4e97ee1ae98c330733670be080c86d5aa24fc8d --- /dev/null +++ b/doc/fluid/book/recommender_system/image @@ -0,0 +1 @@ +../../../../external/book/05.recommender_system/image/ \ No newline at end of file diff --git a/doc/fluid/book/understand_sentiment/README.md b/doc/fluid/book/understand_sentiment/README.md new file mode 120000 index 0000000000000000000000000000000000000000..1133fe46ddd0d6c12f16dc3adc2ee7372cfb1f95 --- /dev/null +++ b/doc/fluid/book/understand_sentiment/README.md @@ -0,0 +1 @@ +../../../../external/book/06.understand_sentiment/README.md \ No newline at end of file diff --git a/doc/fluid/book/understand_sentiment/image b/doc/fluid/book/understand_sentiment/image new file mode 120000 index 0000000000000000000000000000000000000000..c519bc94e3b50cc5bb31e7f601f6c274400345a2 --- /dev/null +++ b/doc/fluid/book/understand_sentiment/image @@ -0,0 +1 @@ +../../../../external/book/06.understand_sentiment/image/ \ No newline at end of file diff --git a/doc/fluid/book/word2vec/README.md b/doc/fluid/book/word2vec/README.md new file mode 120000 index 0000000000000000000000000000000000000000..7dfb712d27ab15cb33b2557b1b886d7d369d44f0 --- /dev/null +++ b/doc/fluid/book/word2vec/README.md @@ -0,0 +1 @@ +../../../../external/book/04.word2vec/README.md \ No newline at end of file diff --git a/doc/fluid/book/word2vec/images b/doc/fluid/book/word2vec/images new file mode 120000 index 0000000000000000000000000000000000000000..dd59e54f0c4cf9058d375650889b52a8996d1418 --- /dev/null +++ b/doc/fluid/book/word2vec/images @@ -0,0 +1 @@ +../../../../external/book/04.word2vec/image/ \ No newline at end of file