Skip to content

B
book

PaddlePaddle / book

通知 16
Star 4
Fork 0
B
book
提交 8a826cf0 编写于 作者: Y Yu Yang
浏览文件
操作

Merge branch 'develop' of https://github.com/PaddlePaddle/book into feature/refine_frontpage

上级 4855822b 924961ba
展开全部 显示空白变更内容
内联 并排
Showing with 5947 addition and 4648 deletion
.pre-commit-config.yaml
浏览文件 @ 8a826cf0
- repo: https://github.com/reyoung/mirrors-yapf.git
sha: v0.13.2
- repo: https://github.com/pre-commit/mirrors-yapf.git
sha: v0.16.0
hooks:
- id: yapf
files: (.*\.(py|bzl)|BUILD|.*\.BUILD|WORKSPACE)$ # Bazel BUILD files follow Python syntax.
files: \.py$
- repo: https://github.com/pre-commit/pre-commit-hooks
sha: v0.7.1
sha: a11d9314b22d8f8c7556443875b731ef05965464
hooks:
- id: check-merge-conflict
- id: check-symlinks
......@@ -24,16 +24,17 @@
files: \.md$
- id: remove-tabs
files: \.md$
- repo: git://github.com/reyoung/pre-commit-hooks-jinja-compile.git
sha: 85ad800cbc9c60a64230d60971aa9576fd57e508
hooks:
- id: convert-jinja2-into-html
- repo: local
hooks:
- id: convert-markdown-into-html
name: convert-markdown-into-html
description: "Convert README.md into index.html and README.en.md into index.en.html"
description: Convert README.md into index.html and README.en.md into index.en.html
entry: python pre-commit-hooks/convert_markdown_into_html.py
language: system
files: .+README(\.en)?\.md$
- id: convert-markdown-into-ipynb
name: convert-markdown-into-ipynb
description: Convert README.md into README.ipynb and README.en.md into README.en.ipynb
entry: ./pre-commit-hooks/convert_markdown_into_ipynb.sh
language: system
files: .+README(\.en)?\.md$
.travis.yml
浏览文件 @ 8a826cf0
......@@ -14,8 +14,10 @@ addons:
- python
- python-pip
- python2.7-dev
- golang
before_install:
- pip install -U virtualenv pre-commit pip
- GOPATH=/tmp/go go get -u github.com/wangkuiyi/ipynb/markdown-to-ipynb
script:
- travis/precommit.sh
notifications:
......
README.en.md 0 → 100644
浏览文件 @ 8a826cf0
# Deep Learning with PaddlePaddle
1. [Fit a Line](http://book.paddlepaddle.org/fit_a_line/index.en.html)
1. [Recognize Digits](http://book.paddlepaddle.org/recognize_digits/index.en.html)
1. [Image Classification](http://book.paddlepaddle.org/image_classification/index.en.html)
1. [Word to Vector](http://book.paddlepaddle.org/word2vec/index.en.html)
1. [Understand Sentiment](http://book.paddlepaddle.org/understand_sentiment/index.en.html)
1. [Label Semantic Roles](http://book.paddlepaddle.org/label_semantic_roles/index.en.html)
1. [Machine Translation](http://book.paddlepaddle.org/machine_translation/index.en.html)
1. [Recommender System](http://book.paddlepaddle.org/recommender_system/index.en.html)
This tutorial is contributed by <a xmlns:cc="http://creativecommons.org/ns#" href="http://book.paddlepaddle.org" property="cc:attributionName" rel="cc:attributionURL">PaddlePaddle</a>, and licensed under a <a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/4.0/">Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License</a>.
README.md
浏览文件 @ 8a826cf0
# 深度学习入门
1. [新手入门](fit_a_line/) [[html](http://book.paddlepaddle.org/fit_a_line)]
1. [识别数字](recognize_digits/) [[html](http://book.paddlepaddle.org/recognize_digits)]
1. [图像分类](image_classification/) [[html](http://book.paddlepaddle.org/image_classification)]
1. [词向量](word2vec/) [[html](http://book.paddlepaddle.org/word2vec)]
1. [情感分析](understand_sentiment/) [[html](http://book.paddlepaddle.org/understand_sentiment)]
1. [语义角色标注](label_semantic_roles/) [[html](http://book.paddlepaddle.org/label_semantic_roles)]
1. [机器翻译](machine_translation/) [[html](http://book.paddlepaddle.org/machine_translation)]
1. [个性化推荐](recommender_system/) [[html](http://book.paddlepaddle.org/recommender_system)]
1. [新手入门](http://book.paddlepaddle.org/fit_a_line)
1. [识别数字](http://book.paddlepaddle.org/recognize_digits)
1. [图像分类](http://book.paddlepaddle.org/image_classification)
1. [词向量](http://book.paddlepaddle.org/word2vec)
1. [情感分析](http://book.paddlepaddle.org/understand_sentiment)
1. [语义角色标注](http://book.paddlepaddle.org/label_semantic_roles)
1. [机器翻译](http://book.paddlepaddle.org/machine_translation)
1. [个性化推荐](http://book.paddlepaddle.org/recommender_system)
# Deep Learning Introduction
1. [Fit a Line](fit_a_line/) [[html](http://book.paddlepaddle.org/fit_a_line/index.en.html)]
1. [Recognize Digits](recognize_digits/) [[html](http://book.paddlepaddle.org/recognize_digits/index.en.html)]
1. [Image Classification](image_classification/) [[html](http://book.paddlepaddle.org/image_classification/index.en.html)]
1. [Word to Vector](word2vec/) [[html](http://book.paddlepaddle.org/word2vec/index.en.html)]
1. [Understand Sentiment](understand_sentiment/) [[html](http://book.paddlepaddle.org/understand_sentiment/index.en.html)]
1. [Label Semantic Roles](label_semantic_roles/) [[html](http://book.paddlepaddle.org/label_semantic_roles/index.en.html)]
1. [Machine Translation](machine_translation/) [[html](http://book.paddlepaddle.org/machine_translation/index.en.html)]
1. [Recommender System](recommender_system/) [[html](http://book.paddlepaddle.org/recommender_system/index.en.html)]
<br/>
<a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/4.0/"><img alt="知识共享许可协议" style="border-width:0" src="https://i.creativecommons.org/l/by-nc-sa/4.0/88x31.png" /></a><br /><span xmlns:dct="http://purl.org/dc/terms/" href="http://purl.org/dc/dcmitype/Text" property="dct:title" rel="dct:type">本教程</span> 由 <a xmlns:cc="http://creativecommons.org/ns#" href="http://book.paddlepaddle.org" property="cc:attributionName" rel="cc:attributionURL">PaddlePaddle</a> 创作，采用 <a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/4.0/">知识共享 署名-非商业性使用-相同方式共享 4.0 国际 许可协议</a>进行许可。
This tutorial is contributed by <a xmlns:cc="http://creativecommons.org/ns#" href="http://book.paddlepaddle.org" property="cc:attributionName" rel="cc:attributionURL">PaddlePaddle</a>, and licensed under a <a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/4.0/">Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License</a>.
fit_a_line/README.en.ipynb 0 → 100644
浏览文件 @ 8a826cf0
此差异已折叠。
fit_a_line/README.en.md
浏览文件 @ 8a826cf0
# Linear Regression
Let us begin the tutorial with a classical problem called Linear Regression \[[1](#References)\]. In this chapter, we will train a model from a realistic dataset to predict home prices. Some important concepts in Machine Learning will be covered through this example.
The source code for this tutorial lives on [book/fit_a_line](https://github.com/PaddlePaddle/book/tree/develop/fit_a_line). For instructions on getting started with PaddlePaddle, see [PaddlePaddle installation guide](http://www.paddlepaddle.org/doc_cn/build_and_install/index.html).
The source code for this tutorial lives on [book/fit_a_line](https://github.com/PaddlePaddle/book/tree/develop/fit_a_line). For instructions on getting started with PaddlePaddle, see [PaddlePaddle installation guide](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/getstarted/build_and_install/docker_install_en.rst).
## Problem Setup
Suppose we have a dataset of $n$ real estate properties. These real estate properties will be referred to as *homes* in this chapter for clarity.
......@@ -202,4 +202,4 @@ This chapter introduces *Linear Regression* and how to train and test this model
4. Bishop C M. Pattern recognition[J]. Machine Learning, 2006, 128.
<br/>
<a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/4.0/"><img alt="Common Creative License" style="border-width:0" src="https://i.creativecommons.org/l/by-nc-sa/4.0/88x31.png" /></a> This tutorial was created and published with [Creative Common License 4.0](http://creativecommons.org/licenses/by-nc-sa/4.0/).
This tutorial is contributed by <a xmlns:cc="http://creativecommons.org/ns#" href="http://book.paddlepaddle.org" property="cc:attributionName" rel="cc:attributionURL">PaddlePaddle</a>, and licensed under a <a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/4.0/">Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License</a>.
fit_a_line/README.ipynb 0 → 100644
浏览文件 @ 8a826cf0
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# 线性回归\n",
"让我们从经典的线性回归（Linear Regression \\[[1](#参考文献)\\]）模型开始这份教程。在这一章里，你将使用真实的数据集建立起一个房价预测模型，并且了解到机器学习中的若干重要概念。\n",
"\n",
"本教程源代码目录在[book/fit_a_line](https://github.com/PaddlePaddle/book/tree/develop/fit_a_line)， 初次使用请参考PaddlePaddle[安装教程](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/getstarted/build_and_install/docker_install_cn.rst)。\n",
"\n",
"## 背景介绍\n",
"给定一个大小为$n$的数据集 ${\\{y_{i}, x_{i1}, ..., x_{id}\\}}_{i=1}^{n}$，其中$x_{i1}, \\ldots, x_{id}$是第$i$个样本$d$个属性上的取值，$y_i$是该样本待预测的目标。线性回归模型假设目标$y_i$可以被属性间的线性组合描述，即\n",
"\n",
"$$y_i = \\omega_1x_{i1} + \\omega_2x_{i2} + \\ldots + \\omega_dx_{id} + b, i=1,\\ldots,n$$\n",
"\n",
"例如，在我们将要建模的房价预测问题里，$x_{ij}$是描述房子$i$的各种属性（比如房间的个数、周围学校和医院的个数、交通状况等），而 $y_i$是房屋的价格。\n",
"\n",
"初看起来，这个假设实在过于简单了，变量间的真实关系很难是线性的。但由于线性回归模型有形式简单和易于建模分析的优点，它在实际问题中得到了大量的应用。很多经典的统计学习、机器学习书籍\\[[2,3,4](#参考文献)\\]也选择对线性模型独立成章重点讲解。\n",
"\n",
"## 效果展示\n",
"我们使用从[UCI Housing Data Set](https://archive.ics.uci.edu/ml/datasets/Housing)获得的波士顿房价数据集进行模型的训练和预测。下面的散点图展示了使用模型对部分房屋价格进行的预测。其中，每个点的横坐标表示同一类房屋真实价格的中位数，纵坐标表示线性回归模型根据特征预测的结果，当二者值完全相等的时候就会落在虚线上。所以模型预测得越准确，则点离虚线越近。\n",
"\u003cp align=\"center\"\u003e\n",
" \u003cimg src = \"image/predictions.png\" width=400\u003e\u003cbr/\u003e\n",
" 图1. 预测值 V.S. 真实值\n",
"\u003c/p\u003e\n",
"\n",
"## 模型概览\n",
"\n",
"### 模型定义\n",
"\n",
"在波士顿房价数据集中，和房屋相关的值共有14个：前13个用来描述房屋相关的各种信息，即模型中的 $x_i$；最后一个值为我们要预测的该类房屋价格的中位数，即模型中的 $y_i$。因此，我们的模型就可以表示成：\n",
"\n",
"$$\\hat{Y} = \\omega_1X_{1} + \\omega_2X_{2} + \\ldots + \\omega_{13}X_{13} + b$$\n",
"\n",
"$\\hat{Y}$ 表示模型的预测结果，用来和真实值$Y$区分。模型要学习的参数即：$\\omega_1, \\ldots, \\omega_{13}, b$。\n",
"\n",
"建立模型后，我们需要给模型一个优化目标，使得学到的参数能够让预测值$\\hat{Y}$尽可能地接近真实值$Y$。这里我们引入损失函数（[Loss Function](https://en.wikipedia.org/wiki/Loss_function)，或Cost Function）这个概念。 输入任意一个数据样本的目标值$y_{i}$和模型给出的预测值$\\hat{y_{i}}$，损失函数输出一个非负的实值。这个实质通常用来反映模型误差的大小。\n",
"\n",
"对于线性回归模型来讲，最常见的损失函数就是均方误差（Mean Squared Error， [MSE](https://en.wikipedia.org/wiki/Mean_squared_error)）了，它的形式是：\n",
"\n",
"$$MSE=\\frac{1}{n}\\sum_{i=1}^{n}{(\\hat{Y_i}-Y_i)}^2$$\n",
"\n",
"即对于一个大小为$n$的测试集，$MSE$是$n$个数据预测结果误差平方的均值。\n",
"\n",
"### 训练过程\n",
"\n",
"定义好模型结构之后，我们要通过以下几个步骤进行模型训练\n",
" 1. 初始化参数，其中包括权重$\\omega_i$和偏置$b$，对其进行初始化（如0均值，1方差）。\n",
" 2. 网络正向传播计算网络输出和损失函数。\n",
" 3. 根据损失函数进行反向误差传播 （[backpropagation](https://en.wikipedia.org/wiki/Backpropagation)），将网络误差从输出层依次向前传递, 并更新网络中的参数。\n",
" 4. 重复2~3步骤，直至网络训练误差达到规定的程度或训练轮次达到设定值。\n",
"\n",
"## 数据集\n",
"\n",
"### 数据集接口的封装\n",
"首先加载需要的包\n",
"\n"
]
},
{
"cell_type": "code",
"metadata": {
"editable": true
},
"source": [
"import paddle.v2 as paddle\n",
"import paddle.v2.dataset.uci_housing as uci_housing\n"
],
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n"
]
}
],
"execution_count": 1
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"我们通过uci_housing模块引入了数据集合[UCI Housing Data Set](https://archive.ics.uci.edu/ml/datasets/Housing)\n",
"\n",
"其中，在uci_housing模块中封装了：\n",
"\n",
"1. 数据下载的过程。下载数据保存在~/.cache/paddle/dataset/uci_housing/housing.data。\n",
"2. [数据预处理](#数据预处理)的过程。\n",
"\n",
"\n",
"### 数据集介绍\n",
"这份数据集共506行，每行包含了波士顿郊区的一类房屋的相关信息及该类房屋价格的中位数。其各维属性的意义如下：\n",
"\n",
"| 属性名 | 解释 | 类型 |\n",
"| ------| ------ | ------ |\n",
"| CRIM | 该镇的人均犯罪率 | 连续值 |\n",
"| ZN | 占地面积超过25,000平方呎的住宅用地比例 | 连续值 |\n",
"| INDUS | 非零售商业用地比例 | 连续值 |\n",
"| CHAS | 是否邻近 Charles River | 离散值，1=邻近；0=不邻近 |\n",
"| NOX | 一氧化氮浓度 | 连续值 |\n",
"| RM | 每栋房屋的平均客房数 | 连续值 |\n",
"| AGE | 1940年之前建成的自用单位比例 | 连续值 |\n",
"| DIS | 到波士顿5个就业中心的加权距离 | 连续值 |\n",
"| RAD | 到径向公路的可达性指数 | 连续值 |\n",
"| TAX | 全值财产税率 | 连续值 |\n",
"| PTRATIO | 学生与教师的比例 | 连续值 |\n",
"| B | 1000(BK - 0.63)^2，其中BK为黑人占比 | 连续值 |\n",
"| LSTAT | 低收入人群占比 | 连续值 |\n",
"| MEDV | 同类房屋价格的中位数 | 连续值 |\n",
"\n",
"### 数据预处理\n",
"#### 连续值与离散值\n",
"观察一下数据，我们的第一个发现是：所有的13维属性中，有12维的连续值和1维的离散值（CHAS）。离散值虽然也常使用类似0、1、2这样的数字表示，但是其含义与连续值是不同的，因为这里的差值没有实际意义。例如，我们用0、1、2来分别表示红色、绿色和蓝色的话，我们并不能因此说“蓝色和红色”比“绿色和红色”的距离更远。所以通常对一个有$d$个可能取值的离散属性，我们会将它们转为$d$个取值为0或1的二值属性或者将每个可能取值映射为一个多维向量。不过就这里而言，因为CHAS本身就是一个二值属性，就省去了这个麻烦。\n",
"\n",
"#### 属性的归一化\n",
"另外一个稍加观察即可发现的事实是，各维属性的取值范围差别很大（如图2所示）。例如，属性B的取值范围是[0.32, 396.90]，而属性NOX的取值范围是[0.3850, 0.8170]。这里就要用到一个常见的操作-归一化（normalization）了。归一化的目标是把各位属性的取值范围放缩到差不多的区间，例如[-0.5,0.5]。这里我们使用一种很常见的操作方法：减掉均值，然后除以原取值范围。\n",
"\n",
"做归一化（或 [Feature scaling](https://en.wikipedia.org/wiki/Feature_scaling)）至少有以下3个理由：\n",
"- 过大或过小的数值范围会导致计算时的浮点上溢或下溢。\n",
"- 不同的数值范围会导致不同属性对模型的重要性不同（至少在训练的初始阶段如此），而这个隐含的假设常常是不合理的。这会对优化的过程造成困难，使训练时间大大的加长。\n",
"- 很多的机器学习技巧/模型（例如L1，L2正则项，向量空间模型-Vector Space Model）都基于这样的假设：所有的属性取值都差不多是以0为均值且取值范围相近的。\n",
"\n",
"\u003cp align=\"center\"\u003e\n",
" \u003cimg src = \"image/ranges.png\" width=550\u003e\u003cbr/\u003e\n",
" 图2. 各维属性的取值范围\n",
"\u003c/p\u003e\n",
"\n",
"#### 整理训练集与测试集\n",
"我们将数据集分割为两份：一份用于调整模型的参数，即进行模型的训练，模型在这份数据集上的误差被称为**训练误差**；另外一份被用来测试，模型在这份数据集上的误差被称为**测试误差**。我们训练模型的目的是为了通过从训练数据中找到规律来预测未知的新数据，所以测试误差是更能反映模型表现的指标。分割数据的比例要考虑到两个因素：更多的训练数据会降低参数估计的方差，从而得到更可信的模型；而更多的测试数据会降低测试误差的方差，从而得到更可信的测试误差。我们这个例子中设置的分割比例为$8:2$\n",
"\n",
"\n",
"在更复杂的模型训练过程中，我们往往还会多使用一种数据集：验证集。因为复杂的模型中常常还有一些超参数（[Hyperparameter](https://en.wikipedia.org/wiki/Hyperparameter_optimization)）需要调节，所以我们会尝试多种超参数的组合来分别训练多个模型，然后对比它们在验证集上的表现选择相对最好的一组超参数，最后才使用这组参数下训练的模型在测试集上评估测试误差。由于本章训练的模型比较简单，我们暂且忽略掉这个过程。\n",
"\n",
"## 训练\n",
"\n",
"`fit_a_line/trainer.py`演示了训练的整体过程。\n",
"\n",
"### 初始化PaddlePaddle\n",
"\n"
]
},
{
"cell_type": "code",
"metadata": {
"editable": true
},
"source": [
"paddle.init(use_gpu=False, trainer_count=1)\n"
],
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n"
]
}
],
"execution_count": 1
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"### 模型配置\n",
"\n",
"线性回归的模型其实就是一个采用线性激活函数（linear activation，`LinearActivation`）的全连接层（fully-connected layer，`fc_layer`）：\n",
"\n"
]
},
{
"cell_type": "code",
"metadata": {
"editable": true
},
"source": [
"x = paddle.layer.data(name='x', type=paddle.data_type.dense_vector(13))\n",
"y_predict = paddle.layer.fc(input=x,\n",
" size=1,\n",
" act=paddle.activation.Linear())\n",
"y = paddle.layer.data(name='y', type=paddle.data_type.dense_vector(1))\n",
"cost = paddle.layer.regression_cost(input=y_predict, label=y)\n"
],
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n"
]
}
],
"execution_count": 1
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 创建参数\n",
"\n"
]
},
{
"cell_type": "code",
"metadata": {
"editable": true
},
"source": [
"parameters = paddle.parameters.create(cost)\n"
],
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n"
]
}
],
"execution_count": 1
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"### 创建Trainer\n",
"\n"
]
},
{
"cell_type": "code",
"metadata": {
"editable": true
},
"source": [
"optimizer = paddle.optimizer.Momentum(momentum=0)\n",
"\n",
"trainer = paddle.trainer.SGD(cost=cost,\n",
" parameters=parameters,\n",
" update_equation=optimizer)\n"
],
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n"
]
}
],
"execution_count": 1
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"### 读取数据且打印训练的中间信息\n",
"\n",
"PaddlePaddle提供一个\n",
"[reader机制](https://github.com/PaddlePaddle/Paddle/tree/develop/doc/design/reader)\n",
"来读取数据。 Reader返回的数据可以包括多列，我们需要一个Python dict把列\n",
"序号映射到网络里的数据层。\n",
"\n"
]
},
{
"cell_type": "code",
"metadata": {
"editable": true
},
"source": [
"feeding={'x': 0, 'y': 1}\n"
],
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n"
]
}
],
"execution_count": 1
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"此外，我们还可以提供一个 event handler，来打印训练的进度：\n",
"\n"
]
},
{
"cell_type": "code",
"metadata": {
"editable": true
},
"source": [
"# event_handler to print training and testing info\n",
"def event_handler(event):\n",
" if isinstance(event, paddle.event.EndIteration):\n",
" if event.batch_id % 100 == 0:\n",
" print \"Pass %d, Batch %d, Cost %f\" % (\n",
" event.pass_id, event.batch_id, event.cost)\n",
"\n",
" if isinstance(event, paddle.event.EndPass):\n",
" result = trainer.test(\n",
" reader=paddle.batch(\n",
" uci_housing.test(), batch_size=2),\n",
" feeding=feeding)\n",
" print \"Test %d, Cost %f\" % (event.pass_id, result.cost)\n"
],
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n"
]
}
],
"execution_count": 1
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"### 开始训练\n",
"\n"
]
},
{
"cell_type": "code",
"metadata": {
"editable": true
},
"source": [
"trainer.train(\n",
" reader=paddle.batch(\n",
" paddle.reader.shuffle(\n",
" uci_housing.train(), buf_size=500),\n",
" batch_size=2),\n",
" feeding=feeding,\n",
" event_handler=event_handler,\n",
" num_passes=30)\n"
],
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n"
]
}
],
"execution_count": 1
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"## 总结\n",
"在这章里，我们借助波士顿房价这一数据集，介绍了线性回归模型的基本概念，以及如何使用PaddlePaddle实现训练和测试的过程。很多的模型和技巧都是从简单的线性回归模型演化而来，因此弄清楚线性模型的原理和局限非常重要。\n",
"\n",
"\n",
"## 参考文献\n",
"1. https://en.wikipedia.org/wiki/Linear_regression\n",
"2. Friedman J, Hastie T, Tibshirani R. The elements of statistical learning[M]. Springer, Berlin: Springer series in statistics, 2001.\n",
"3. Murphy K P. Machine learning: a probabilistic perspective[M]. MIT press, 2012.\n",
"4. Bishop C M. Pattern recognition[J]. Machine Learning, 2006, 128.\n",
"\n",
"\u003cbr/\u003e\n",
"\u003ca rel=\"license\" href=\"http://creativecommons.org/licenses/by-nc-sa/4.0/\"\u003e\u003cimg alt=\"知识共享许可协议\" style=\"border-width:0\" src=\"https://i.creativecommons.org/l/by-nc-sa/4.0/88x31.png\" /\u003e\u003c/a\u003e\u003cbr /\u003e\u003cspan xmlns:dct=\"http://purl.org/dc/terms/\" href=\"http://purl.org/dc/dcmitype/Text\" property=\"dct:title\" rel=\"dct:type\"\u003e本教程\u003c/span\u003e 由 \u003ca xmlns:cc=\"http://creativecommons.org/ns#\" href=\"http://book.paddlepaddle.org\" property=\"cc:attributionName\" rel=\"cc:attributionURL\"\u003ePaddlePaddle\u003c/a\u003e 创作，采用 \u003ca rel=\"license\" href=\"http://creativecommons.org/licenses/by-nc-sa/4.0/\"\u003e知识共享 署名-非商业性使用-相同方式共享 4.0 国际 许可协议\u003c/a\u003e进行许可。\n"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.6.0"
}
},
"nbformat": 4,
"nbformat_minor": 0
}
fit_a_line/README.md
浏览文件 @ 8a826cf0
# 线性回归
让我们从经典的线性回归（Linear Regression \[[1](#参考文献)\]）模型开始这份教程。在这一章里，你将使用真实的数据集建立起一个房价预测模型，并且了解到机器学习中的若干重要概念。
本教程源代码目录在[book/fit_a_line](https://github.com/PaddlePaddle/book/tree/develop/fit_a_line)， 初次使用请参考PaddlePaddle[安装教程](http://www.paddlepaddle.org/doc_cn/build_and_install/index.html)。
本教程源代码目录在[book/fit_a_line](https://github.com/PaddlePaddle/book/tree/develop/fit_a_line)， 初次使用请参考PaddlePaddle[安装教程](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/getstarted/build_and_install/docker_install_cn.rst)。
## 背景介绍
给定一个大小为$n$的数据集 ${\{y_{i}, x_{i1}, ..., x_{id}\}}_{i=1}^{n}$，其中$x_{i1}, \ldots, x_{id}$是第$i$个样本$d$个属性上的取值，$y_i$是该样本待预测的目标。线性回归模型假设目标$y_i$可以被属性间的线性组合描述，即
......
fit_a_line/index.en.html
浏览文件 @ 8a826cf0
......@@ -43,7 +43,7 @@
# Linear Regression
Let us begin the tutorial with a classical problem called Linear Regression \[[1](#References)\]. In this chapter, we will train a model from a realistic dataset to predict home prices. Some important concepts in Machine Learning will be covered through this example.
The source code for this tutorial lives on [book/fit_a_line](https://github.com/PaddlePaddle/book/tree/develop/fit_a_line). For instructions on getting started with PaddlePaddle, see [PaddlePaddle installation guide](http://www.paddlepaddle.org/doc_cn/build_and_install/index.html).
The source code for this tutorial lives on [book/fit_a_line](https://github.com/PaddlePaddle/book/tree/develop/fit_a_line). For instructions on getting started with PaddlePaddle, see [PaddlePaddle installation guide](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/getstarted/build_and_install/docker_install_en.rst).
## Problem Setup
Suppose we have a dataset of $n$ real estate properties. These real estate properties will be referred to as *homes* in this chapter for clarity.
......@@ -244,7 +244,7 @@ This chapter introduces *Linear Regression* and how to train and test this model
4. Bishop C M. Pattern recognition[J]. Machine Learning, 2006, 128.
<br/>
<a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/4.0/"><img alt="Common Creative License" style="border-width:0" src="https://i.creativecommons.org/l/by-nc-sa/4.0/88x31.png" /></a> This tutorial was created and published with [Creative Common License 4.0](http://creativecommons.org/licenses/by-nc-sa/4.0/).
This tutorial is contributed by <a xmlns:cc="http://creativecommons.org/ns#" href="http://book.paddlepaddle.org" property="cc:attributionName" rel="cc:attributionURL">PaddlePaddle</a>, and licensed under a <a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/4.0/">Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License</a>.
</div>
<!-- You can change the lines below now. -->
......
fit_a_line/index.html
浏览文件 @ 8a826cf0
......@@ -43,7 +43,7 @@
# 线性回归
让我们从经典的线性回归（Linear Regression \[[1](#参考文献)\]）模型开始这份教程。在这一章里，你将使用真实的数据集建立起一个房价预测模型，并且了解到机器学习中的若干重要概念。
本教程源代码目录在[book/fit_a_line](https://github.com/PaddlePaddle/book/tree/develop/fit_a_line)， 初次使用请参考PaddlePaddle[安装教程](http://www.paddlepaddle.org/doc_cn/build_and_install/index.html)。
本教程源代码目录在[book/fit_a_line](https://github.com/PaddlePaddle/book/tree/develop/fit_a_line)， 初次使用请参考PaddlePaddle[安装教程](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/getstarted/build_and_install/docker_install_cn.rst)。
## 背景介绍
给定一个大小为$n$的数据集 ${\{y_{i}, x_{i1}, ..., x_{id}\}}_{i=1}^{n}$，其中$x_{i1}, \ldots, x_{id}$是第$i$个样本$d$个属性上的取值，$y_i$是该样本待预测的目标。线性回归模型假设目标$y_i$可以被属性间的线性组合描述，即
......
fit_a_line/train.py
浏览文件 @ 8a826cf0
......@@ -18,9 +18,8 @@ def main():
# create optimizer
optimizer = paddle.optimizer.Momentum(momentum=0)
trainer = paddle.trainer.SGD(cost=cost,
parameters=parameters,
update_equation=optimizer)
trainer = paddle.trainer.SGD(
cost=cost, parameters=parameters, update_equation=optimizer)
feeding = {'x': 0, 'y': 1}
......@@ -33,16 +32,14 @@ def main():
if isinstance(event, paddle.event.EndPass):
result = trainer.test(
reader=paddle.batch(
uci_housing.test(), batch_size=2),
reader=paddle.batch(uci_housing.test(), batch_size=2),
feeding=feeding)
print "Test %d, Cost %f" % (event.pass_id, result.cost)
# training
trainer.train(
reader=paddle.batch(
paddle.reader.shuffle(
uci_housing.train(), buf_size=500),
paddle.reader.shuffle(uci_housing.train(), buf_size=500),
batch_size=2),
feeding=feeding,
event_handler=event_handler,
......
image_classification/README.en.md
浏览文件 @ 8a826cf0
此差异已折叠。
image_classification/README.ipynb 0 → 100644
浏览文件 @ 8a826cf0
此差异已折叠。
image_classification/README.md
浏览文件 @ 8a826cf0
图像分类
=======
# 图像分类
本教程源代码目录在[book/image_classification](https://github.com/PaddlePaddle/book/tree/develop/image_classification)， 初次使用请参考PaddlePaddle[安装教程](http://www.paddlepaddle.org/doc_cn/build_and_install/index.html)。
本教程源代码目录在[book/image_classification](https://github.com/PaddlePaddle/book/tree/develop/image_classification)， 初次使用请参考PaddlePaddle[安装教程](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/getstarted/build_and_install/docker_install_cn.rst)。
## 背景介绍
......@@ -252,7 +251,7 @@ paddle.init(use_gpu=False, trainer_count=1)
ResNet模型的第1、3、4步和VGG模型相同，这里不再介绍。主要介绍第2步即CIFAR10数据集上ResNet核心模块。
```python
net = resnet_cifar10(data, depth=56)
net = resnet_cifar10(image, depth=56)
```
先介绍`resnet_cifar10`中的一些基本函数，再介绍网络连接过程。
......@@ -375,7 +374,7 @@ $$ lr = lr_{0} * a^ {\lfloor \frac{n}{ b}\rfloor} $$
cifar.train10()每次产生一条样本，在完成shuffle和batch之后，作为训练的输入。
```python
reader=paddle.reader.batch(
reader=paddle.batch(
paddle.reader.shuffle(
paddle.dataset.cifar.train10(), buf_size=50000),
batch_size=128)
......@@ -402,10 +401,9 @@ def event_handler(event):
sys.stdout.flush()
if isinstance(event, paddle.event.EndPass):
result = trainer.test(
reader=paddle.reader.batch(
reader=paddle.batch(
paddle.dataset.cifar.test10(), batch_size=128),
reader_dict={'image': 0,
'label': 1})
feeding=feeding)
print "\nTest with Pass %d, %s" % (event.pass_id, result.metrics)
```
......
image_classification/deprecated/README.md
浏览文件 @ 8a826cf0
图像分类
=======
本教程源代码目录在[book/image_classification](https://github.com/PaddlePaddle/book/tree/develop/image_classification)， 初次使用请参考PaddlePaddle[安装教程](http://www.paddlepaddle.org/doc_cn/build_and_install/index.html)。
本教程源代码目录在[book/image_classification](https://github.com/PaddlePaddle/book/tree/develop/image_classification)， 初次使用请参考PaddlePaddle[安装教程](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/getstarted/build_and_install/docker_install_cn.rst)。
## 背景介绍
......
image_classification/deprecated/classify.py
浏览文件 @ 8a826cf0
......@@ -44,7 +44,8 @@ def vis_square(data, fname):
(0, 1)) # add some space between filters
+ ((0, 0), ) *
(data.ndim - 3)) # don't pad the last dimension (if there is one)
data = np.pad(data, padding, mode='constant',
data = np.pad(
data, padding, mode='constant',
constant_values=1) # pad with ones (white)
# tile the filters into an image
data = data.reshape((n, n) + data.shape[1:]).transpose((0, 2, 1, 3) + tuple(
......
image_classification/index.en.html
浏览文件 @ 8a826cf0
此差异已折叠。
image_classification/index.html
浏览文件 @ 8a826cf0
......@@ -40,10 +40,9 @@
<!-- This block will be replaced by each markdown file content. Please do not change lines below.-->
<div id="markdown" style='display:none'>
图像分类
=======
# 图像分类
本教程源代码目录在[book/image_classification](https://github.com/PaddlePaddle/book/tree/develop/image_classification)， 初次使用请参考PaddlePaddle[安装教程](http://www.paddlepaddle.org/doc_cn/build_and_install/index.html)。
本教程源代码目录在[book/image_classification](https://github.com/PaddlePaddle/book/tree/develop/image_classification)， 初次使用请参考PaddlePaddle[安装教程](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/getstarted/build_and_install/docker_install_cn.rst)。
## 背景介绍
......@@ -294,7 +293,7 @@ paddle.init(use_gpu=False, trainer_count=1)
ResNet模型的第1、3、4步和VGG模型相同，这里不再介绍。主要介绍第2步即CIFAR10数据集上ResNet核心模块。
```python
net = resnet_cifar10(data, depth=56)
net = resnet_cifar10(image, depth=56)
```
先介绍`resnet_cifar10`中的一些基本函数，再介绍网络连接过程。
......@@ -417,7 +416,7 @@ $$ lr = lr_{0} * a^ {\lfloor \frac{n}{ b}\rfloor} $$
cifar.train10()每次产生一条样本，在完成shuffle和batch之后，作为训练的输入。
```python
reader=paddle.reader.batch(
reader=paddle.batch(
paddle.reader.shuffle(
paddle.dataset.cifar.train10(), buf_size=50000),
batch_size=128)
......@@ -444,10 +443,9 @@ def event_handler(event):
sys.stdout.flush()
if isinstance(event, paddle.event.EndPass):
result = trainer.test(
reader=paddle.reader.batch(
reader=paddle.batch(
paddle.dataset.cifar.test10(), batch_size=128),
reader_dict={'image': 0,
'label': 1})
feeding=feeding)
print "\nTest with Pass %d, %s" % (event.pass_id, result.metrics)
```
......
image_classification/train.py
浏览文件 @ 8a826cf0
......@@ -36,9 +36,8 @@ def main():
# option 2. vgg
net = vgg_bn_drop(image)
out = paddle.layer.fc(input=net,
size=classdim,
act=paddle.activation.Softmax())
out = paddle.layer.fc(
input=net, size=classdim, act=paddle.activation.Softmax())
lbl = paddle.layer.data(
name="label", type=paddle.data_type.integer_value(classdim))
......@@ -75,9 +74,8 @@ def main():
print "\nTest with Pass %d, %s" % (event.pass_id, result.metrics)
# Create trainer
trainer = paddle.trainer.SGD(cost=cost,
parameters=parameters,
update_equation=momentum_optimizer)
trainer = paddle.trainer.SGD(
cost=cost, parameters=parameters, update_equation=momentum_optimizer)
trainer.train(
reader=paddle.batch(
paddle.reader.shuffle(
......
index.en.html 已删除 100644 → 0
浏览文件 @ 4855822b
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<title>Deep Learning 101</title>
<link href="http://cdn.bootcss.com/bootstrap/4.0.0-alpha.6/css/bootstrap.min.css" rel="stylesheet">
<script src="http://cdn.bootcss.com/tether/1.4.0/js/tether.js"></script>
<script src="http://cdn.bootcss.com/jquery/3.1.0/jquery.slim.js"></script>
<script src="http://cdn.bootcss.com/bootstrap/4.0.0-alpha.6/js/bootstrap.min.js"></script>
<style>
* {
font-family:"Roboto","Lato","proxima-nova","Helvetica Neue",Arial,sans-serif;
}
.left-panel {
background: #E5E6EA;
}
.left-panel .card-block a.click_active {
background-color: #597cf1;
color: #fff;
}
.left-panel .card-header {
background: #ecedee;
}
.left-panel .card-block a:not(.click_active) {
background: #e4e6e9;
}
.left-panel .card-block a {
border-radius: 0px;
font-size: 13px;
color: #2F323A;
padding-left: 40px;
border: 1px solid rgba(0,0,0,.125);
}
.left-panel .card-block a.active_color {
color: rgb(70, 74, 76);
font-weight: bolder;
}
.left-panel .list-group-item {
-moz-box-align: center;
align-items: center;
display: flex;
flex-flow: row wrap;
margin-bottom: -1px;
padding: 0.75rem 1.25rem;
position: relative;
color: #2f323a;
}
.navbar img {
height: 90%;
width: 90%;
}
</style>
</head>
<body>
<nav class="navbar navbar-toggleable-md navbar-inverse bg-inverse">
<a class="navbar-brand mr-auto" href="#">
<img alt="PaddlePaddle" src="./.theme/PP_w.png">
</a>
<ul class="nav navbar-nav">
<li class="nav-item">
<a class="nav-link" href="http://paddlepaddle.org">
Home
</a>
</li>
<li class="nav-item ">
<a class="nav-link" href="./index.html">
中文
</a>
</li>
<li class="nav-item active">
<a class="nav-link" href="#">
Deep Learning 101
</a>
</li>
<li class="nav-item">
<a class="nav-link" href="https://github.com/PaddlePaddle/book/">Fork on Github</a>
</li>
</ul>
</nav>
<div class="container-fluid">
<div class="row">
<div class="col-3 pl-1 pr-1 left-panel">
<div id="accordion" role="tablist" aria-multiselectable="true">
<div class="card">
<div class="card-header" role="tab" id="headingOne">
<h5 class="mb-0">
<a aria-expanded="true" aria-controls="collapseOne">
Deep Learning 101<span class="sr-only">(current)</span>
</a>
</h5>
</div>
<div id="collapseOne" class="rounded-0 collapse show" role="tabpanel" aria-labelledby="headingOne">
<div class="card-block pl-0 pr-0 pt-0 pb-0">
<div class="list-group ">
<a href="./fit_a_line/index.en.html" target="content_iframe" class="list-group-item list-group-item-action" style="border: 0px; border-bottom: 2px solid #dddfe3;">
Linear Regression
</a>
<a href="./recognize_digits/index.en.html" target="content_iframe" class="list-group-item list-group-item-action" style="border: 0px; border-bottom: 2px solid #dddfe3;">
Recognize Digits
</a>
<a href="./image_classification/index.en.html" target="content_iframe" class="list-group-item list-group-item-action" style="border: 0px; border-bottom: 2px solid #dddfe3;">
Image Classification
</a>
<a href="./word2vec/index.en.html" target="content_iframe" class="list-group-item list-group-item-action" style="border: 0px; border-bottom: 2px solid #dddfe3;">
Word2Vec
</a>
<a href="./understand_sentiment/index.en.html" target="content_iframe" class="list-group-item list-group-item-action" style="border: 0px; border-bottom: 2px solid #dddfe3;">
Sentiment Analysis
</a>
<a href="./label_semantic_roles/index.en.html" target="content_iframe" class="list-group-item list-group-item-action" style="border: 0px; border-bottom: 2px solid #dddfe3;">
Semantic Role Labeling
</a>
<a href="./machine_translation/index.en.html" target="content_iframe" class="list-group-item list-group-item-action" style="border: 0px; border-bottom: 2px solid #dddfe3;">
Machine Translation
</a>
<a href="./recommender_system/index.en.html" target="content_iframe" class="list-group-item list-group-item-action" style="border: 0px; border-bottom: 2px solid #dddfe3;">
Personalized Recommendation
</a>
</div>
</div>
</div>
</div>
</div>
</div>
<div class="col">
<iframe src="./fit_a_line/index.en.html" style="border: none; overflow-y : hidden" width="100%" height="100%" name="content_iframe" id="content_iframe">
</iframe>
</div>
</div>
</div>
<script>
$('#content_iframe').on('load', function(){
$("#content_iframe").height(200) // trick code to shrink iframe size
var body = $('#content_iframe').contents().find("body")
body.css("overflow-y", "hidden")
$("#content_iframe").height(body.height()+20)
var alllinks = $('#content_iframe').contents().find("a")
for (var i =0; i<alllinks.length; ++i) {
alllinks[i].setAttribute("target", "_blank")
}
});
$(".list-group a").click(function(){
$(".list-group a.click_active").removeClass("click_active");
$(this).addClass("click_active");
})
$($(".list-group a")[0]).addClass("click_active")
</script>
</body>
</html>
\ No newline at end of file
index.html 已删除 100644 → 0
浏览文件 @ 4855822b
<!DOCTYPE html>
<html lang="zh-Hans">
<head>
<meta charset="UTF-8">
<title>深度学习入门</title>
<link href="http://cdn.bootcss.com/bootstrap/4.0.0-alpha.6/css/bootstrap.min.css" rel="stylesheet">
<script src="http://cdn.bootcss.com/tether/1.4.0/js/tether.js"></script>
<script src="http://cdn.bootcss.com/jquery/3.1.0/jquery.slim.js"></script>
<script src="http://cdn.bootcss.com/bootstrap/4.0.0-alpha.6/js/bootstrap.min.js"></script>
<style>
* {
font-family:"Roboto","Lato","proxima-nova","Helvetica Neue",Arial,sans-serif;
}
.left-panel {
background: #E5E6EA;
}
.left-panel .card-block a.click_active {
background-color: #597cf1;
color: #fff;
}
.left-panel .card-header {
background: #ecedee;
}
.left-panel .card-header a {
font-size: 14px;
}
.left-panel .card-block a:not(.click_active) {
background: #e4e6e9;
}
.left-panel .card-block a {
border-radius: 0px;
font-size: 13px;
color: #2F323A;
padding-left: 40px;
border: 1px solid rgba(0,0,0,.125);
}
.left-panel .card-block a.active_color {
color: rgb(70, 74, 76);
font-weight: bolder;
}
.left-panel .list-group-item {
-moz-box-align: center;
align-items: center;
display: flex;
flex-flow: row wrap;
margin-bottom: -1px;
padding: 0.75rem 1.25rem;
position: relative;
color: #2f323a;
}
.navbar img {
height: 90%;
width: 90%;
}
</style>
</head>
<body>
<nav class="navbar navbar-toggleable-md navbar-inverse bg-inverse">
<a class="navbar-brand mr-auto" href="#">
<img alt="PaddlePaddle" src="./.theme/PP_w.png">
</a>
<ul class="nav navbar-nav">
<li class="nav-item">
<a class="nav-link" href="http://paddlepaddle.org">
首页
</a>
</li>
<li class="nav-item active">
<a class="nav-link" href="#">
深度学习入门
</a>
</li>
<li class="nav-item ">
<a class="nav-link" href="./index.en.html">
English
</a>
</li>
<li class="nav-item">
<a class="nav-link" href="https://github.com/PaddlePaddle/book/">Fork on Github</a>
</li>
</ul>
</nav>
<div class="container-fluid">
<div class="row">
<div class="col-3 pl-1 pr-1 left-panel">
<div id="accordion" role="tablist" aria-multiselectable="true">
<div class="card">
<div class="card-header" role="tab" id="headingOne">
<h5 class="mb-0">
<a aria-expanded="true" aria-controls="collapseOne">
深度学习入门
</a>
</h5>
</div>
<div id="collapseOne" class="rounded-0 collapse show" role="tabpanel" aria-labelledby="headingOne">
<div class="card-block pl-0 pr-0 pt-0 pb-0">
<div class="list-group ">
<a href="./fit_a_line/index.html" target="content_iframe" class="list-group-item list-group-item-action" style="border: 0px; border-bottom: 2px solid #dddfe3;">
新手入门
</a>
<a href="./recognize_digits/index.html" target="content_iframe" class="list-group-item list-group-item-action" style="border: 0px; border-bottom: 2px solid #dddfe3;">
识别数字
</a>
<a href="./image_classification/index.html" target="content_iframe" class="list-group-item list-group-item-action" style="border: 0px; border-bottom: 2px solid #dddfe3;">
图像分类
</a>
<a href="./word2vec/index.html" target="content_iframe" class="list-group-item list-group-item-action" style="border: 0px; border-bottom: 2px solid #dddfe3;">
词向量
</a>
<a href="./understand_sentiment/index.html" target="content_iframe" class="list-group-item list-group-item-action" style="border: 0px; border-bottom: 2px solid #dddfe3;">
情感分析
</a>
<a href="./label_semantic_roles/index.html" target="content_iframe" class="list-group-item list-group-item-action" style="border: 0px; border-bottom: 2px solid #dddfe3;">
语义角色标注
</a>
<a href="./machine_translation/index.html" target="content_iframe" class="list-group-item list-group-item-action" style="border: 0px; border-bottom: 2px solid #dddfe3;">
机器翻译
</a>
<a href="./recommender_system/index.html" target="content_iframe" class="list-group-item list-group-item-action" style="border: 0px; border-bottom: 2px solid #dddfe3;">
个性化推荐
</a>
</div>
</div>
</div>
</div>
</div>
</div>
<div class="col">
<iframe src="./fit_a_line/index.html" style="border: none; overflow-y : hidden" width="100%" height="100%" name="content_iframe" id="content_iframe">
</iframe>
</div>
</div>
</div>
<script>
$('#content_iframe').on('load', function(){
$("#content_iframe").height(200) // trick code to shrink iframe size
var body = $('#content_iframe').contents().find("body")
body.css("overflow-y", "hidden")
$("#content_iframe").height(body.height()+20)
var alllinks = $('#content_iframe').contents().find("a")
for (var i =0; i<alllinks.length; ++i) {
alllinks[i].setAttribute("target", "_blank")
}
});
$(".list-group a").click(function(){
$(".list-group a.click_active").removeClass("click_active");
$(this).addClass("click_active");
})
$($(".list-group a")[0]).addClass("click_active")
</script>
</body>
</html>
\ No newline at end of file
label_semantic_roles/README.en.md
浏览文件 @ 8a826cf0
......@@ -2,6 +2,8 @@
Source code of this chapter is in [book/label_semantic_roles](https://github.com/PaddlePaddle/book/tree/develop/label_semantic_roles).
For instructions on getting started with PaddlePaddle, see [PaddlePaddle installation guide](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/getstarted/build_and_install/docker_install_en.rst).
## Background
Natural Language Analysis contains three components: Lexical Analysis, Syntactic Analysis, and Semantic Analysis. Semantic Role Labelling (SRL) is one way for Shallow Semantic Analysis. A predicate of a sentence is a property that a subject possesses or is characterized, such as what it does, what it is or how it is, which mostly corresponds to the core of an event. The noun associated with a predicate is called Argument. Semantic roles express the abstract roles that arguments of a predicate can take in the event, such as Agent, Patient, Theme, Experiencer, Beneficiary, Instrument, Location, Goal and Source, etc.
......@@ -200,6 +202,8 @@ import numpy as np
import paddle.v2 as paddle
import paddle.v2.dataset.conll05 as conll05
paddle.init(use_gpu=False, trainer_count=1)
word_dict, verb_dict, label_dict = conll05.get_dict()
word_dict_len = len(word_dict)
label_dict_len = len(label_dict)
......@@ -470,4 +474,4 @@ Semantic Role Labeling is an important intermediate step in a wide range of natu
10. Zhou J, Xu W. [End-to-end learning of semantic role labeling using recurrent neural networks](http://www.aclweb.org/anthology/P/P15/P15-1109.pdf)[C]//Proceedings of the Annual Meeting of the Association for Computational Linguistics. 2015.
<br/>
<a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/4.0/"><img alt="知识共享许可协议" style="border-width:0" src="https://i.creativecommons.org/l/by-nc-sa/4.0/88x31.png" /></a><br /><span xmlns:dct="http://purl.org/dc/terms/" href="http://purl.org/dc/dcmitype/Text" property="dct:title" rel="dct:type">本教程</span> 由 <a xmlns:cc="http://creativecommons.org/ns#" href="http://book.paddlepaddle.org" property="cc:attributionName" rel="cc:attributionURL">PaddlePaddle</a> 创作，采用 <a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/4.0/">知识共享 署名-非商业性使用-相同方式共享 4.0 国际 许可协议</a>进行许可。
This tutorial is contributed by <a xmlns:cc="http://creativecommons.org/ns#" href="http://book.paddlepaddle.org" property="cc:attributionName" rel="cc:attributionURL">PaddlePaddle</a>, and licensed under a <a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/4.0/">Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License</a>.
label_semantic_roles/README.md
浏览文件 @ 8a826cf0
# 语义角色标注
本教程源代码目录在[book/label_semantic_roles](https://github.com/PaddlePaddle/book/tree/develop/label_semantic_roles)， 初次使用请参考PaddlePaddle[安装教程](http://www.paddlepaddle.org/doc_cn/build_and_install/index.html)。
本教程源代码目录在[book/label_semantic_roles](https://github.com/PaddlePaddle/book/tree/develop/label_semantic_roles)， 初次使用请参考PaddlePaddle[安装教程](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/getstarted/build_and_install/docker_install_cn.rst)。
## 背景介绍
......
label_semantic_roles/db_lstm.py
浏览文件 @ 8a826cf0
......@@ -75,8 +75,7 @@ settings(
learning_method=MomentumOptimizer(momentum=0),
learning_rate=2e-2,
regularization=L2Regularization(8e-4),
model_average=ModelAverage(
average_window=0.5, max_average_window=10000), )
model_average=ModelAverage(average_window=0.5, max_average_window=10000), )
####################################### network ##############################
#8 features and 1 target
......@@ -102,13 +101,12 @@ std_default = ParameterAttribute(initial_std=default_std)
predicate_embedding = embedding_layer(
size=word_dim,
input=predicate,
param_attr=ParameterAttribute(
name='vemb', initial_std=default_std))
param_attr=ParameterAttribute(name='vemb', initial_std=default_std))
word_input = [word, ctx_n2, ctx_n1, ctx_0, ctx_p1, ctx_p2]
emb_layers = [
embedding_layer(
size=word_dim, input=x, param_attr=emb_para) for x in word_input
embedding_layer(size=word_dim, input=x, param_attr=emb_para)
for x in word_input
]
emb_layers.append(predicate_embedding)
mark_embedding = embedding_layer(
......@@ -120,8 +118,8 @@ hidden_0 = mixed_layer(
size=hidden_dim,
bias_attr=std_default,
input=[
full_matrix_projection(
input=emb, param_attr=std_default) for emb in emb_layers
full_matrix_projection(input=emb, param_attr=std_default)
for emb in emb_layers
])
mix_hidden_lr = 1e-3
......@@ -171,10 +169,8 @@ feature_out = mixed_layer(
size=label_dict_len,
bias_attr=std_default,
input=[
full_matrix_projection(
input=input_tmp[0], param_attr=hidden_para_attr),
full_matrix_projection(
input=input_tmp[1], param_attr=lstm_para_attr)
full_matrix_projection(input=input_tmp[0], param_attr=hidden_para_attr),
full_matrix_projection(input=input_tmp[1], param_attr=lstm_para_attr)
], )
if not is_predict:
......
label_semantic_roles/index.en.html
浏览文件 @ 8a826cf0
......@@ -44,6 +44,8 @@
Source code of this chapter is in [book/label_semantic_roles](https://github.com/PaddlePaddle/book/tree/develop/label_semantic_roles).
For instructions on getting started with PaddlePaddle, see [PaddlePaddle installation guide](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/getstarted/build_and_install/docker_install_en.rst).
## Background
Natural Language Analysis contains three components: Lexical Analysis, Syntactic Analysis, and Semantic Analysis. Semantic Role Labelling (SRL) is one way for Shallow Semantic Analysis. A predicate of a sentence is a property that a subject possesses or is characterized, such as what it does, what it is or how it is, which mostly corresponds to the core of an event. The noun associated with a predicate is called Argument. Semantic roles express the abstract roles that arguments of a predicate can take in the event, such as Agent, Patient, Theme, Experiencer, Beneficiary, Instrument, Location, Goal and Source, etc.
......@@ -242,6 +244,8 @@ import numpy as np
import paddle.v2 as paddle
import paddle.v2.dataset.conll05 as conll05
paddle.init(use_gpu=False, trainer_count=1)
word_dict, verb_dict, label_dict = conll05.get_dict()
word_dict_len = len(word_dict)
label_dict_len = len(label_dict)
......@@ -512,7 +516,7 @@ Semantic Role Labeling is an important intermediate step in a wide range of natu
10. Zhou J, Xu W. [End-to-end learning of semantic role labeling using recurrent neural networks](http://www.aclweb.org/anthology/P/P15/P15-1109.pdf)[C]//Proceedings of the Annual Meeting of the Association for Computational Linguistics. 2015.
<br/>
<a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/4.0/"><img alt="知识共享许可协议" style="border-width:0" src="https://i.creativecommons.org/l/by-nc-sa/4.0/88x31.png" /></a><br /><span xmlns:dct="http://purl.org/dc/terms/" href="http://purl.org/dc/dcmitype/Text" property="dct:title" rel="dct:type">本教程</span> 由 <a xmlns:cc="http://creativecommons.org/ns#" href="http://book.paddlepaddle.org" property="cc:attributionName" rel="cc:attributionURL">PaddlePaddle</a> 创作，采用 <a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/4.0/">知识共享 署名-非商业性使用-相同方式共享 4.0 国际 许可协议</a>进行许可。
This tutorial is contributed by <a xmlns:cc="http://creativecommons.org/ns#" href="http://book.paddlepaddle.org" property="cc:attributionName" rel="cc:attributionURL">PaddlePaddle</a>, and licensed under a <a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/4.0/">Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License</a>.
</div>
<!-- You can change the lines below now. -->
......
label_semantic_roles/index.html
浏览文件 @ 8a826cf0
......@@ -42,7 +42,7 @@
<div id="markdown" style='display:none'>
# 语义角色标注
本教程源代码目录在[book/label_semantic_roles](https://github.com/PaddlePaddle/book/tree/develop/label_semantic_roles)， 初次使用请参考PaddlePaddle[安装教程](http://www.paddlepaddle.org/doc_cn/build_and_install/index.html)。
本教程源代码目录在[book/label_semantic_roles](https://github.com/PaddlePaddle/book/tree/develop/label_semantic_roles)， 初次使用请参考PaddlePaddle[安装教程](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/getstarted/build_and_install/docker_install_cn.rst)。
## 背景介绍
......
label_semantic_roles/train.py
浏览文件 @ 8a826cf0
......@@ -40,15 +40,14 @@ def db_lstm():
predicate_embedding = paddle.layer.embedding(
size=word_dim,
input=predicate,
param_attr=paddle.attr.Param(
name='vemb', initial_std=default_std))
param_attr=paddle.attr.Param(name='vemb', initial_std=default_std))
mark_embedding = paddle.layer.embedding(
size=mark_dim, input=mark, param_attr=std_0)
word_input = [word, ctx_n2, ctx_n1, ctx_0, ctx_p1, ctx_p2]
emb_layers = [
paddle.layer.embedding(
size=word_dim, input=x, param_attr=emb_para) for x in word_input
paddle.layer.embedding(size=word_dim, input=x, param_attr=emb_para)
for x in word_input
]
emb_layers.append(predicate_embedding)
emb_layers.append(mark_embedding)
......@@ -109,13 +108,12 @@ def db_lstm():
input=input_tmp[1], param_attr=lstm_para_attr)
], )
crf_cost = paddle.layer.crf(size=label_dict_len,
crf_cost = paddle.layer.crf(
size=label_dict_len,
input=feature_out,
label=target,
param_attr=paddle.attr.Param(
name='crfw',
initial_std=default_std,
learning_rate=mix_hidden_lr))
name='crfw', initial_std=default_std, learning_rate=mix_hidden_lr))
crf_dec = paddle.layer.crf_decoding(
name='crf_dec_l',
......@@ -151,13 +149,11 @@ def main():
model_average=paddle.optimizer.ModelAverage(
average_window=0.5, max_average_window=10000), )
trainer = paddle.trainer.SGD(cost=crf_cost,
parameters=parameters,
update_equation=optimizer)
trainer = paddle.trainer.SGD(
cost=crf_cost, parameters=parameters, update_equation=optimizer)
reader = paddle.batch(
paddle.reader.shuffle(
conll05.test(), buf_size=8192), batch_size=10)
paddle.reader.shuffle(conll05.test(), buf_size=8192), batch_size=10)
feeding = {
'word_data': 0,
......
machine_translation/README.en.md
浏览文件 @ 8a826cf0
此差异已折叠。
machine_translation/README.md
浏览文件 @ 8a826cf0
# 机器翻译
本教程源代码目录在[book/machine_translation](https://github.com/PaddlePaddle/book/tree/develop/machine_translation)， 初次使用请参考PaddlePaddle[安装教程](http://www.paddlepaddle.org/doc_cn/build_and_install/index.html)。
本教程源代码目录在[book/machine_translation](https://github.com/PaddlePaddle/book/tree/develop/machine_translation)， 初次使用请参考PaddlePaddle[安装教程](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/getstarted/build_and_install/docker_install_cn.rst)。
## 背景介绍
......@@ -152,54 +152,8 @@ e_{ij}&=align(z_i,h_j)\\\\
## 数据介绍
### 下载与解压缩
本教程使用[WMT-14](http://www-lium.univ-lemans.fr/~schwenk/cslm_joint_paper/)数据集中的[bitexts(after selection)](http://www-lium.univ-lemans.fr/~schwenk/cslm_joint_paper/data/bitexts.tgz)作为训练集，[dev+test data](http://www-lium.univ-lemans.fr/~schwenk/cslm_joint_paper/data/dev+test.tgz)作为测试集和生成集。
在Linux下，只需简单地运行以下命令：
```bash
cd data
./wmt14_data.sh
```
得到的数据集`data/wmt14`包含如下三个文件夹：
<p align = "center">
<table>
<tr>
<td>文件夹名</td>
<td>法英平行语料文件</td>
<td>文件数</td>
<td>文件大小</td>
</tr>
<tr>
<td>train</td>
<td>ccb2_pc30.src, ccb2_pc30.trg, etc</td>
<td>12</td>
<td>3.55G</td>
</tr>
<tr>
<td>test</td>
<td>ntst1213.src, ntst1213.trg</td>
<td>2</td>
<td>1636k</td>
</tr>
</tr>
<tr>
<td>gen</td>
<td>ntst14.src, ntst14.trg</td>
<td>2</td>
<td>864k</td>
</tr>
</table>
</p>
- `XXX.src`是源法语文件，`XXX.trg`是目标英语文件，文件中的每行存放一个句子
- `XXX.src`和`XXX.trg`的行数一致，且两者任意第$i$行的句子之间都有着一一对应的关系。
### 数据预处理
我们的预处理流程包括两步：
......@@ -220,6 +174,7 @@ cd data
```python
# 加载 paddle的python包
import sys
import paddle.v2 as paddle
# 配置只使用cpu，并且使用一个cpu进行训练
......@@ -256,17 +211,16 @@ wmt14_reader = paddle.batch(
decoder_size = 512 # 解码器中的GRU隐层大小
```
2. 其次，实现编码器框架。分为三步：
1. 其次，实现编码器框架。分为三步：
2.1 将在dataset reader中生成的用每个单词在字典中的索引表示的源语言序列
转换成one-hot vector表示的源语言序列$\mathbf{w}$，其类型为integer_value_sequence。
1 输入是一个文字序列，被表示成整型的序列。序列中每个元素是文字在字典中的索引。所以，我们定义数据层的数据类型为`integer_value_sequence`（整型序列），序列中每个元素的范围是`[0, source_dict_dim)`。
```python
src_word_id = paddle.layer.data(
name='source_language_word',
type=paddle.data_type.integer_value_sequence(source_dict_dim))
```
2.2 将上述编码映射到低维语言空间的词向量$\mathbf{s}$。
1. 将上述编码映射到低维语言空间的词向量$\mathbf{s}$。
```python
src_embedding = paddle.layer.embedding(
......@@ -274,7 +228,7 @@ wmt14_reader = paddle.batch(
size=word_vector_dim,
param_attr=paddle.attr.ParamAttr(name='_source_language_embedding'))
```
2.3 用双向GRU编码源语言序列，拼接两个GRU的编码结果得到$\mathbf{h}$。
1. 用双向GRU编码源语言序列，拼接两个GRU的编码结果得到$\mathbf{h}$。
```python
src_forward = paddle.networks.simple_gru(
......@@ -284,16 +238,17 @@ wmt14_reader = paddle.batch(
encoded_vector = paddle.layer.concat(input=[src_forward, src_backward])
```
3. 接着，定义基于注意力机制的解码器框架。分为三步：
1. 接着，定义基于注意力机制的解码器框架。分为三步：
3.1 对源语言序列编码后的结果（见2.3），过一个前馈神经网络（Feed Forward Neural Network），得到其映射。
1. 对源语言序列编码后的结果（见2.3），过一个前馈神经网络（Feed Forward Neural Network），得到其映射。
```python
with paddle.layer.mixed(size=decoder_size) as encoded_proj:
encoded_proj += paddle.layer.full_matrix_projection(
input=encoded_vector)
```
3.2 构造解码器RNN的初始状态。由于解码器需要预测时序目标序列，但在0时刻并没有初始值，所以我们希望对其进行初始化。这里采用的是将源语言序列逆序编码后的最后一个状态进行非线性映射，作为该初始值，即$c_0=h_T$。
1. 构造解码器RNN的初始状态。由于解码器需要预测时序目标序列，但在0时刻并没有初始值，所以我们希望对其进行初始化。这里采用的是将源语言序列逆序编码后的最后一个状态进行非线性映射，作为该初始值，即$c_0=h_T$。
```python
backward_first = paddle.layer.first_seq(input=src_backward)
......@@ -302,7 +257,8 @@ wmt14_reader = paddle.batch(
decoder_boot += paddle.layer.full_matrix_projection(
input=backward_first)
```
3.3 定义解码阶段每一个时间步的RNN行为，即根据当前时刻的源语言上下文向量$c_i$、解码器隐层状态$z_i$和目标语言中第$i$个词$u_i$，来预测第$i+1$个词的概率$p_{i+1}$。
1. 定义解码阶段每一个时间步的RNN行为，即根据当前时刻的源语言上下文向量$c_i$、解码器隐层状态$z_i$和目标语言中第$i$个词$u_i$，来预测第$i+1$个词的概率$p_{i+1}$。
- decoder_mem记录了前一个时间步的隐层状态$z_i$，其初始状态是decoder_boot。
- context通过调用`simple_attention`函数，实现公式$c_i=\sum {j=1}^{T}a_{ij}h_j$。其中，enc_vec是$h_j$，enc_proj是$h_j$的映射（见3.1），权重$a_{ij}$的计算已经封装在`simple_attention`函数中。
- decoder_inputs融合了$c_i$和当前目标词current_word（即$u_i$）的表示。
......@@ -339,9 +295,7 @@ wmt14_reader = paddle.batch(
return out
```
4. 训练模式与生成模式下的解码器调用区别。
4.1 定义解码器框架名字，和`gru_decoder_with_attention`函数的前两个输入。注意：这两个输入使用`StaticInput`，具体说明可见[StaticInput文档](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/howto/deep_model/rnn/recurrent_group_cn.md#输入)。
1. 定义解码器框架名字，和`gru_decoder_with_attention`函数的前两个输入。注意：这两个输入使用`StaticInput`，具体说明可见[StaticInput文档](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/howto/deep_model/rnn/recurrent_group_cn.md#输入)。
```python
decoder_group_name = "decoder_group"
......@@ -349,7 +303,8 @@ wmt14_reader = paddle.batch(
group_input2 = paddle.layer.StaticInputV2(input=encoded_proj, is_seq=True)
group_inputs = [group_input1, group_input2]
```
4.2 训练模式下的解码器调用：
1. 训练模式下的解码器调用：
- 首先，将目标语言序列的词向量trg_embedding，直接作为训练模式下的current_word传给`gru_decoder_with_attention`函数。
- 其次，使用`recurrent_group`函数循环调用`gru_decoder_with_attention`函数。
......@@ -380,6 +335,7 @@ wmt14_reader = paddle.batch(
type=paddle.data_type.integer_value_sequence(target_dict_dim))
cost = paddle.layer.classification_cost(input=decoder, label=lbl)
```
注意：我们提供的配置在Bahdanau的论文\[[4](#参考文献)\]上做了一些简化，可参考[issue #1133](https://github.com/PaddlePaddle/Paddle/issues/1133)。
### 参数定义
......@@ -387,7 +343,6 @@ wmt14_reader = paddle.batch(
首先依据模型配置的`cost`定义模型参数。
```python
# create parameters
parameters = paddle.parameters.create(cost)
```
......@@ -405,24 +360,30 @@ for param in parameters.keys():
根据优化目标cost,网络拓扑结构和模型参数来构造出trainer用来训练，在构造时还需指定优化方法，这里使用最基本的SGD方法。
```python
optimizer = paddle.optimizer.Adam(learning_rate=1e-4)
optimizer = paddle.optimizer.Adam(
learning_rate=5e-5,
regularization=paddle.optimizer.L2Regularization(rate=1e-3))
trainer = paddle.trainer.SGD(cost=cost,
parameters=parameters,
update_equation=optimizer)
```
2. 构造event_handler
1. 构造event_handler
可以通过自定义回调函数来评估训练过程中的各种状态，比如错误率等。下面的代码通过event.batch_id % 10 == 0 指定没10个batch打印一次日志，包含cost等信息。
```python
def event_handler(event):
if isinstance(event, paddle.event.EndIteration):
if event.batch_id % 10 == 0:
print "Pass %d, Batch %d, Cost %f, %s" % (
print "\nPass %d, Batch %d, Cost %f, %s" % (
event.pass_id, event.batch_id, event.cost, event.metrics)
else:
sys.stdout.write('.')
sys.stdout.flush()
```
3. 启动训练：
1. 启动训练：
```python
trainer.train(
......@@ -431,30 +392,29 @@ for param in parameters.keys():
num_passes=10000,
feeding=feeding)
```
训练开始后，可以观察到event_handler输出的日志如下：
```text
Pass 0, Batch 0, Cost 247.408008, {'classification_error_evaluator': 1.0}
Pass 0, Batch 10, Cost 212.058789, {'classification_error_evaluator': 0.8737863898277283}
...
```
训练开始后，可以观察到event_handler输出的日志如下：
```text
Pass 0, Batch 0, Cost 148.444983, {'classification_error_evaluator': 1.0}
.........
Pass 0, Batch 10, Cost 335.896802, {'classification_error_evaluator': 0.9325153231620789}
.........
```
当`classification_error_evaluator`的值低于0.35的时候，表示训练成功。
## 应用模型
### 下载预训练的模型
由于NMT模型的训练非常耗时，我们在50个物理节点（每节点含有2颗6核CPU）的集群中，花了5天时间训练了16个pass，其中每个pass耗时7个小时。因此，我们提供了一个预先训练好的模型（pass-00012）供大家直接下载使用。该模型大小为205MB，在所有16个模型中有最高的[BLEU评估](#BLEU评估)值26.92。下载并解压模型的命令如下：
```bash
cd pretrained
./wmt14_model.sh
```
### 应用命令与结果
新版api尚未支持机器翻译的翻译过程，尽请期待。
翻译结果请见[效果展示](#效果展示)。
### BLEU评估
BLEU(Bilingual Evaluation understudy)是一种广泛使用的机器翻译自动评测指标，由IBM的watson研究中心于2002年提出\[[5](#参考文献)\]，基本出发点是：机器译文越接近专业翻译人员的翻译结果，翻译系统的性能越好。其中，机器译文与人工参考译文之间的接近程度，采用句子精确度（precision）的计算方法，即比较两者的n元词组相匹配的个数，匹配的个数越多，BLEU得分越好。
......
machine_translation/api_train.py
浏览文件 @ 8a826cf0
......@@ -105,9 +105,8 @@ def main():
# define optimize method and trainer
optimizer = paddle.optimizer.Adam(learning_rate=1e-4)
trainer = paddle.trainer.SGD(cost=cost,
parameters=parameters,
update_equation=optimizer)
trainer = paddle.trainer.SGD(
cost=cost, parameters=parameters, update_equation=optimizer)
# define data reader
feeding = {
......
machine_translation/index.en.html
浏览文件 @ 8a826cf0
此差异已折叠。
machine_translation/index.html
浏览文件 @ 8a826cf0
......@@ -42,7 +42,7 @@
<div id="markdown" style='display:none'>
# 机器翻译
本教程源代码目录在[book/machine_translation](https://github.com/PaddlePaddle/book/tree/develop/machine_translation)， 初次使用请参考PaddlePaddle[安装教程](http://www.paddlepaddle.org/doc_cn/build_and_install/index.html)。
本教程源代码目录在[book/machine_translation](https://github.com/PaddlePaddle/book/tree/develop/machine_translation)， 初次使用请参考PaddlePaddle[安装教程](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/getstarted/build_and_install/docker_install_cn.rst)。
## 背景介绍
......@@ -194,54 +194,8 @@ e_{ij}&=align(z_i,h_j)\\\\
## 数据介绍
### 下载与解压缩
本教程使用[WMT-14](http://www-lium.univ-lemans.fr/~schwenk/cslm_joint_paper/)数据集中的[bitexts(after selection)](http://www-lium.univ-lemans.fr/~schwenk/cslm_joint_paper/data/bitexts.tgz)作为训练集，[dev+test data](http://www-lium.univ-lemans.fr/~schwenk/cslm_joint_paper/data/dev+test.tgz)作为测试集和生成集。
在Linux下，只需简单地运行以下命令：
```bash
cd data
./wmt14_data.sh
```
得到的数据集`data/wmt14`包含如下三个文件夹：
<p align = "center">
<table>
<tr>
<td>文件夹名</td>
<td>法英平行语料文件</td>
<td>文件数</td>
<td>文件大小</td>
</tr>
<tr>
<td>train</td>
<td>ccb2_pc30.src, ccb2_pc30.trg, etc</td>
<td>12</td>
<td>3.55G</td>
</tr>
<tr>
<td>test</td>
<td>ntst1213.src, ntst1213.trg</td>
<td>2</td>
<td>1636k</td>
</tr>
</tr>
<tr>
<td>gen</td>
<td>ntst14.src, ntst14.trg</td>
<td>2</td>
<td>864k</td>
</tr>
</table>
</p>
- `XXX.src`是源法语文件，`XXX.trg`是目标英语文件，文件中的每行存放一个句子
- `XXX.src`和`XXX.trg`的行数一致，且两者任意第$i$行的句子之间都有着一一对应的关系。
### 数据预处理
我们的预处理流程包括两步：
......@@ -262,6 +216,7 @@ cd data
```python
# 加载 paddle的python包
import sys
import paddle.v2 as paddle
# 配置只使用cpu，并且使用一个cpu进行训练
......@@ -298,17 +253,16 @@ wmt14_reader = paddle.batch(
decoder_size = 512 # 解码器中的GRU隐层大小
```
2. 其次，实现编码器框架。分为三步：
1. 其次，实现编码器框架。分为三步：
2.1 将在dataset reader中生成的用每个单词在字典中的索引表示的源语言序列
转换成one-hot vector表示的源语言序列$\mathbf{w}$，其类型为integer_value_sequence。
1 输入是一个文字序列，被表示成整型的序列。序列中每个元素是文字在字典中的索引。所以，我们定义数据层的数据类型为`integer_value_sequence`（整型序列），序列中每个元素的范围是`[0, source_dict_dim)`。
```python
src_word_id = paddle.layer.data(
name='source_language_word',
type=paddle.data_type.integer_value_sequence(source_dict_dim))
```
2.2 将上述编码映射到低维语言空间的词向量$\mathbf{s}$。
1. 将上述编码映射到低维语言空间的词向量$\mathbf{s}$。
```python
src_embedding = paddle.layer.embedding(
......@@ -316,7 +270,7 @@ wmt14_reader = paddle.batch(
size=word_vector_dim,
param_attr=paddle.attr.ParamAttr(name='_source_language_embedding'))
```
2.3 用双向GRU编码源语言序列，拼接两个GRU的编码结果得到$\mathbf{h}$。
1. 用双向GRU编码源语言序列，拼接两个GRU的编码结果得到$\mathbf{h}$。
```python
src_forward = paddle.networks.simple_gru(
......@@ -326,16 +280,17 @@ wmt14_reader = paddle.batch(
encoded_vector = paddle.layer.concat(input=[src_forward, src_backward])
```
3. 接着，定义基于注意力机制的解码器框架。分为三步：
1. 接着，定义基于注意力机制的解码器框架。分为三步：
3.1 对源语言序列编码后的结果（见2.3），过一个前馈神经网络（Feed Forward Neural Network），得到其映射。
1. 对源语言序列编码后的结果（见2.3），过一个前馈神经网络（Feed Forward Neural Network），得到其映射。
```python
with paddle.layer.mixed(size=decoder_size) as encoded_proj:
encoded_proj += paddle.layer.full_matrix_projection(
input=encoded_vector)
```
3.2 构造解码器RNN的初始状态。由于解码器需要预测时序目标序列，但在0时刻并没有初始值，所以我们希望对其进行初始化。这里采用的是将源语言序列逆序编码后的最后一个状态进行非线性映射，作为该初始值，即$c_0=h_T$。
1. 构造解码器RNN的初始状态。由于解码器需要预测时序目标序列，但在0时刻并没有初始值，所以我们希望对其进行初始化。这里采用的是将源语言序列逆序编码后的最后一个状态进行非线性映射，作为该初始值，即$c_0=h_T$。
```python
backward_first = paddle.layer.first_seq(input=src_backward)
......@@ -344,7 +299,8 @@ wmt14_reader = paddle.batch(
decoder_boot += paddle.layer.full_matrix_projection(
input=backward_first)
```
3.3 定义解码阶段每一个时间步的RNN行为，即根据当前时刻的源语言上下文向量$c_i$、解码器隐层状态$z_i$和目标语言中第$i$个词$u_i$，来预测第$i+1$个词的概率$p_{i+1}$。
1. 定义解码阶段每一个时间步的RNN行为，即根据当前时刻的源语言上下文向量$c_i$、解码器隐层状态$z_i$和目标语言中第$i$个词$u_i$，来预测第$i+1$个词的概率$p_{i+1}$。
- decoder_mem记录了前一个时间步的隐层状态$z_i$，其初始状态是decoder_boot。
- context通过调用`simple_attention`函数，实现公式$c_i=\sum {j=1}^{T}a_{ij}h_j$。其中，enc_vec是$h_j$，enc_proj是$h_j$的映射（见3.1），权重$a_{ij}$的计算已经封装在`simple_attention`函数中。
- decoder_inputs融合了$c_i$和当前目标词current_word（即$u_i$）的表示。
......@@ -381,9 +337,7 @@ wmt14_reader = paddle.batch(
return out
```
4. 训练模式与生成模式下的解码器调用区别。
4.1 定义解码器框架名字，和`gru_decoder_with_attention`函数的前两个输入。注意：这两个输入使用`StaticInput`，具体说明可见[StaticInput文档](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/howto/deep_model/rnn/recurrent_group_cn.md#输入)。
1. 定义解码器框架名字，和`gru_decoder_with_attention`函数的前两个输入。注意：这两个输入使用`StaticInput`，具体说明可见[StaticInput文档](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/howto/deep_model/rnn/recurrent_group_cn.md#输入)。
```python
decoder_group_name = "decoder_group"
......@@ -391,7 +345,8 @@ wmt14_reader = paddle.batch(
group_input2 = paddle.layer.StaticInputV2(input=encoded_proj, is_seq=True)
group_inputs = [group_input1, group_input2]
```
4.2 训练模式下的解码器调用：
1. 训练模式下的解码器调用：
- 首先，将目标语言序列的词向量trg_embedding，直接作为训练模式下的current_word传给`gru_decoder_with_attention`函数。
- 其次，使用`recurrent_group`函数循环调用`gru_decoder_with_attention`函数。
......@@ -422,6 +377,7 @@ wmt14_reader = paddle.batch(
type=paddle.data_type.integer_value_sequence(target_dict_dim))
cost = paddle.layer.classification_cost(input=decoder, label=lbl)
```
注意：我们提供的配置在Bahdanau的论文\[[4](#参考文献)\]上做了一些简化，可参考[issue #1133](https://github.com/PaddlePaddle/Paddle/issues/1133)。
### 参数定义
......@@ -429,7 +385,6 @@ wmt14_reader = paddle.batch(
首先依据模型配置的`cost`定义模型参数。
```python
# create parameters
parameters = paddle.parameters.create(cost)
```
......@@ -447,24 +402,30 @@ for param in parameters.keys():
根据优化目标cost,网络拓扑结构和模型参数来构造出trainer用来训练，在构造时还需指定优化方法，这里使用最基本的SGD方法。
```python
optimizer = paddle.optimizer.Adam(learning_rate=1e-4)
optimizer = paddle.optimizer.Adam(
learning_rate=5e-5,
regularization=paddle.optimizer.L2Regularization(rate=1e-3))
trainer = paddle.trainer.SGD(cost=cost,
parameters=parameters,
update_equation=optimizer)
```
2. 构造event_handler
1. 构造event_handler
可以通过自定义回调函数来评估训练过程中的各种状态，比如错误率等。下面的代码通过event.batch_id % 10 == 0 指定没10个batch打印一次日志，包含cost等信息。
```python
def event_handler(event):
if isinstance(event, paddle.event.EndIteration):
if event.batch_id % 10 == 0:
print "Pass %d, Batch %d, Cost %f, %s" % (
print "\nPass %d, Batch %d, Cost %f, %s" % (
event.pass_id, event.batch_id, event.cost, event.metrics)
else:
sys.stdout.write('.')
sys.stdout.flush()
```
3. 启动训练：
1. 启动训练：
```python
trainer.train(
......@@ -473,30 +434,29 @@ for param in parameters.keys():
num_passes=10000,
feeding=feeding)
```
训练开始后，可以观察到event_handler输出的日志如下：
```text
Pass 0, Batch 0, Cost 247.408008, {'classification_error_evaluator': 1.0}
Pass 0, Batch 10, Cost 212.058789, {'classification_error_evaluator': 0.8737863898277283}
...
```
训练开始后，可以观察到event_handler输出的日志如下：
```text
Pass 0, Batch 0, Cost 148.444983, {'classification_error_evaluator': 1.0}
.........
Pass 0, Batch 10, Cost 335.896802, {'classification_error_evaluator': 0.9325153231620789}
.........
```
当`classification_error_evaluator`的值低于0.35的时候，表示训练成功。
## 应用模型
### 下载预训练的模型
由于NMT模型的训练非常耗时，我们在50个物理节点（每节点含有2颗6核CPU）的集群中，花了5天时间训练了16个pass，其中每个pass耗时7个小时。因此，我们提供了一个预先训练好的模型（pass-00012）供大家直接下载使用。该模型大小为205MB，在所有16个模型中有最高的[BLEU评估](#BLEU评估)值26.92。下载并解压模型的命令如下：
```bash
cd pretrained
./wmt14_model.sh
```
### 应用命令与结果
新版api尚未支持机器翻译的翻译过程，尽请期待。
翻译结果请见[效果展示](#效果展示)。
### BLEU评估
BLEU(Bilingual Evaluation understudy)是一种广泛使用的机器翻译自动评测指标，由IBM的watson研究中心于2002年提出\[[5](#参考文献)\]，基本出发点是：机器译文越接近专业翻译人员的翻译结果，翻译系统的性能越好。其中，机器译文与人工参考译文之间的接近程度，采用句子精确度（precision）的计算方法，即比较两者的n元词组相匹配的个数，匹配的个数越多，BLEU得分越好。
......
machine_translation/seqToseq_net.py
浏览文件 @ 8a826cf0
......@@ -110,8 +110,7 @@ group_inputs = [group_input1, group_input2]
if not is_generating:
trg_embedding = embedding_layer(
input=data_layer(
name='target_language_word', size=target_dict_dim),
input=data_layer(name='target_language_word', size=target_dict_dim),
size=word_vector_dim,
param_attr=ParamAttr(name='_target_language_embedding'))
group_inputs.append(trg_embedding)
......@@ -156,8 +155,7 @@ else:
seqtext_printer_evaluator(
input=beam_gen,
id_input=data_layer(
name="sent_id", size=1),
id_input=data_layer(name="sent_id", size=1),
dict_file=trg_lang_dict,
result_file=gen_trans_file)
outputs(beam_gen)
pre-commit-hooks/convert_markdown_into_ipynb.sh 0 → 100755
浏览文件 @ 8a826cf0
#!/bin/sh
for file in $@ ; do
/tmp/go/bin/markdown-to-ipynb < $file > ${file%.*}".ipynb"
if [ $? -ne 0 ]; then
echo >&2 "markdown-to-ipynb $file error"
exit 1
fi
done
recognize_digits/README.en.md
浏览文件 @ 8a826cf0
# Recognize Digits
The source code for this tutorial is under [book/recognize_digits](https://github.com/PaddlePaddle/book/tree/develop/recognize_digits). First-time readers, please refer to PaddlePaddle [installation instructions](http://www.paddlepaddle.org/doc_cn/build_and_install/index.html).
The source code for this tutorial is under [book/recognize_digits](https://github.com/PaddlePaddle/book/tree/develop/recognize_digits). First-time readers, please refer to PaddlePaddle [installation instructions](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/getstarted/build_and_install/docker_install_en.rst).
## Introduction
When we learn a new programming language, the first task is usually to write a program that prints "Hello World." In Machine Learning or Deep Learning, the equivalent task is to train a model to perform handwritten digit recognition with [MNIST](http://yann.lecun.com/exdb/mnist/) dataset. Handwriting recognition is a typical image classification problem. The problem is relatively easy, and MNIST is a complete dataset. As a simple Computer Vision dataset, MNIST contains images of handwritten digits and their corresponding labels (Fig. 1). The input image is a 28x28 matrix, and the label is one of the digits from 0 to 9. Each image is normalized in size and centered.
......@@ -240,7 +240,7 @@ def event_handler(event):
print "Pass %d, Batch %d, Cost %f, %s" % (
event.pass_id, event.batch_id, event.cost, event.metrics)
if isinstance(event, paddle.event.EndPass):
result = trainer.test(reader=paddle.reader.batched(
result = trainer.test(reader=paddle.batch(
paddle.dataset.mnist.test(), batch_size=128))
print "Test with Pass %d, Cost %f, %s\n" % (
event.pass_id, result.cost, result.metrics)
......@@ -248,7 +248,7 @@ def event_handler(event):
result.metrics['classification_error_evaluator']))
trainer.train(
reader=paddle.reader.batched(
reader=paddle.batch(
paddle.reader.shuffle(
paddle.dataset.mnist.train(), buf_size=8192),
batch_size=128),
......@@ -293,7 +293,7 @@ This tutorial describes a few basic Deep Learning models viz. Softmax regression
7. Deng, Li, Michael L. Seltzer, Dong Yu, Alex Acero, Abdel-rahman Mohamed, and Geoffrey E. Hinton. ["Binary coding of speech spectrograms using a deep auto-encoder."](http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.185.1908&rep=rep1&type=pdf) In Interspeech, pp. 1692-1695. 2010.
8. Kégl, Balázs, and Róbert Busa-Fekete. ["Boosting products of base classifiers."](http://dl.acm.org/citation.cfm?id=1553439) In Proceedings of the 26th Annual International Conference on Machine Learning, pp. 497-504. ACM, 2009.
9. Rosenblatt, Frank. ["The perceptron: A probabilistic model for information storage and organization in the brain."](http://psycnet.apa.org/journals/rev/65/6/386/) Psychological review 65, no. 6 (1958): 386.
10. Bishop, Christopher M. ["Pattern recognition."](http://s3.amazonaws.com/academia.edu.documents/30428242/bg0137.pdf?AWSAccessKeyId=AKIAJ56TQJRTWSMTNPEA&Expires=1484816640&Signature=85Ad6%2Fca8T82pmHzxaSXermovIA%3D&response-content-disposition=inline%3B%20filename%3DPattern_recognition_and_machine_learning.pdf) Machine Learning 128 (2006): 1-58.
10. Bishop, Christopher M. ["Pattern recognition."](http://users.isr.ist.utl.pt/~wurmd/Livros/school/Bishop%20-%20Pattern%20Recognition%20And%20Machine%20Learning%20-%20Springer%20%202006.pdf) Machine Learning 128 (2006): 1-58.
<br/>
<a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/4.0/"><img alt="知识共享许可协议" style="border-width:0" src="https://i.creativecommons.org/l/by-nc-sa/4.0/88x31.png" /></a><br /><span xmlns:dct="http://purl.org/dc/terms/" href="http://purl.org/dc/dcmitype/Text" property="dct:title" rel="dct:type">This book</span> is created by <a xmlns:cc="http://creativecommons.org/ns#" href="http://book.paddlepaddle.org" property="cc:attributionName" rel="cc:attributionURL">PaddlePaddle</a>, and uses <a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/4.0/">Shared knowledge signature - non commercial use-Sharing 4.0 International Licensing Protocal</a>.
This tutorial is contributed by <a xmlns:cc="http://creativecommons.org/ns#" href="http://book.paddlepaddle.org" property="cc:attributionName" rel="cc:attributionURL">PaddlePaddle</a>, and licensed under a <a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/4.0/">Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License</a>.
recognize_digits/README.md
浏览文件 @ 8a826cf0
# 识别数字
本教程源代码目录在[book/recognize_digits](https://github.com/PaddlePaddle/book/tree/develop/recognize_digits)， 初次使用请参考PaddlePaddle[安装教程](http://www.paddlepaddle.org/doc_cn/build_and_install/index.html)。
本教程源代码目录在[book/recognize_digits](https://github.com/PaddlePaddle/book/tree/develop/recognize_digits)， 初次使用请参考PaddlePaddle[安装教程](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/getstarted/build_and_install/docker_install_cn.rst)。
## 背景介绍
当我们学习编程的时候，编写的第一个程序一般是实现打印"Hello World"。而机器学习（或深度学习）的入门教程，一般都是 [MNIST](http://yann.lecun.com/exdb/mnist/) 数据库上的手写识别问题。原因是手写识别属于典型的图像分类问题，比较简单，同时MNIST数据集也很完备。MNIST数据集作为一个简单的计算机视觉数据集，包含一系列如图1所示的手写数字图片和对应的标签。图片是28x28的像素矩阵，标签则对应着0~9的10个数字。每张图片都经过了大小归一化和居中处理。
......@@ -245,7 +245,7 @@ def event_handler(event):
print "Pass %d, Batch %d, Cost %f, %s" % (
event.pass_id, event.batch_id, event.cost, event.metrics)
if isinstance(event, paddle.event.EndPass):
result = trainer.test(reader=paddle.reader.batched(
result = trainer.test(reader=paddle.batch(
paddle.dataset.mnist.test(), batch_size=128))
print "Test with Pass %d, Cost %f, %s\n" % (
event.pass_id, result.cost, result.metrics)
......@@ -253,7 +253,7 @@ def event_handler(event):
result.metrics['classification_error_evaluator']))
trainer.train(
reader=paddle.reader.batched(
reader=paddle.batch(
paddle.reader.shuffle(
paddle.dataset.mnist.train(), buf_size=8192),
batch_size=128),
......@@ -289,7 +289,7 @@ trainer.train(
7. Deng, Li, Michael L. Seltzer, Dong Yu, Alex Acero, Abdel-rahman Mohamed, and Geoffrey E. Hinton. ["Binary coding of speech spectrograms using a deep auto-encoder."](http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.185.1908&rep=rep1&type=pdf) In Interspeech, pp. 1692-1695. 2010.
8. Kégl, Balázs, and Róbert Busa-Fekete. ["Boosting products of base classifiers."](http://dl.acm.org/citation.cfm?id=1553439) In Proceedings of the 26th Annual International Conference on Machine Learning, pp. 497-504. ACM, 2009.
9. Rosenblatt, Frank. ["The perceptron: A probabilistic model for information storage and organization in the brain."](http://psycnet.apa.org/journals/rev/65/6/386/) Psychological review 65, no. 6 (1958): 386.
10. Bishop, Christopher M. ["Pattern recognition."](http://s3.amazonaws.com/academia.edu.documents/30428242/bg0137.pdf?AWSAccessKeyId=AKIAJ56TQJRTWSMTNPEA&Expires=1484816640&Signature=85Ad6%2Fca8T82pmHzxaSXermovIA%3D&response-content-disposition=inline%3B%20filename%3DPattern_recognition_and_machine_learning.pdf) Machine Learning 128 (2006): 1-58.
10. Bishop, Christopher M. ["Pattern recognition."](http://users.isr.ist.utl.pt/~wurmd/Livros/school/Bishop%20-%20Pattern%20Recognition%20And%20Machine%20Learning%20-%20Springer%20%202006.pdf) Machine Learning 128 (2006): 1-58.
<br/>
<a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/4.0/"><img alt="知识共享许可协议" style="border-width:0" src="https://i.creativecommons.org/l/by-nc-sa/4.0/88x31.png" /></a><br /><span xmlns:dct="http://purl.org/dc/terms/" href="http://purl.org/dc/dcmitype/Text" property="dct:title" rel="dct:type">本教程</span> 由 <a xmlns:cc="http://creativecommons.org/ns#" href="http://book.paddlepaddle.org" property="cc:attributionName" rel="cc:attributionURL">PaddlePaddle</a> 创作，采用 <a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/4.0/">知识共享 署名-非商业性使用-相同方式共享 4.0 国际 许可协议</a>进行许可。
recognize_digits/index.en.html
浏览文件 @ 8a826cf0
......@@ -42,7 +42,7 @@
<div id="markdown" style='display:none'>
# Recognize Digits
The source code for this tutorial is under [book/recognize_digits](https://github.com/PaddlePaddle/book/tree/develop/recognize_digits). First-time readers, please refer to PaddlePaddle [installation instructions](http://www.paddlepaddle.org/doc_cn/build_and_install/index.html).
The source code for this tutorial is under [book/recognize_digits](https://github.com/PaddlePaddle/book/tree/develop/recognize_digits). First-time readers, please refer to PaddlePaddle [installation instructions](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/getstarted/build_and_install/docker_install_en.rst).
## Introduction
When we learn a new programming language, the first task is usually to write a program that prints "Hello World." In Machine Learning or Deep Learning, the equivalent task is to train a model to perform handwritten digit recognition with [MNIST](http://yann.lecun.com/exdb/mnist/) dataset. Handwriting recognition is a typical image classification problem. The problem is relatively easy, and MNIST is a complete dataset. As a simple Computer Vision dataset, MNIST contains images of handwritten digits and their corresponding labels (Fig. 1). The input image is a 28x28 matrix, and the label is one of the digits from 0 to 9. Each image is normalized in size and centered.
......@@ -282,7 +282,7 @@ def event_handler(event):
print "Pass %d, Batch %d, Cost %f, %s" % (
event.pass_id, event.batch_id, event.cost, event.metrics)
if isinstance(event, paddle.event.EndPass):
result = trainer.test(reader=paddle.reader.batched(
result = trainer.test(reader=paddle.batch(
paddle.dataset.mnist.test(), batch_size=128))
print "Test with Pass %d, Cost %f, %s\n" % (
event.pass_id, result.cost, result.metrics)
......@@ -290,7 +290,7 @@ def event_handler(event):
result.metrics['classification_error_evaluator']))
trainer.train(
reader=paddle.reader.batched(
reader=paddle.batch(
paddle.reader.shuffle(
paddle.dataset.mnist.train(), buf_size=8192),
batch_size=128),
......@@ -335,10 +335,10 @@ This tutorial describes a few basic Deep Learning models viz. Softmax regression
7. Deng, Li, Michael L. Seltzer, Dong Yu, Alex Acero, Abdel-rahman Mohamed, and Geoffrey E. Hinton. ["Binary coding of speech spectrograms using a deep auto-encoder."](http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.185.1908&rep=rep1&type=pdf) In Interspeech, pp. 1692-1695. 2010.
8. Kégl, Balázs, and Róbert Busa-Fekete. ["Boosting products of base classifiers."](http://dl.acm.org/citation.cfm?id=1553439) In Proceedings of the 26th Annual International Conference on Machine Learning, pp. 497-504. ACM, 2009.
9. Rosenblatt, Frank. ["The perceptron: A probabilistic model for information storage and organization in the brain."](http://psycnet.apa.org/journals/rev/65/6/386/) Psychological review 65, no. 6 (1958): 386.
10. Bishop, Christopher M. ["Pattern recognition."](http://s3.amazonaws.com/academia.edu.documents/30428242/bg0137.pdf?AWSAccessKeyId=AKIAJ56TQJRTWSMTNPEA&Expires=1484816640&Signature=85Ad6%2Fca8T82pmHzxaSXermovIA%3D&response-content-disposition=inline%3B%20filename%3DPattern_recognition_and_machine_learning.pdf) Machine Learning 128 (2006): 1-58.
10. Bishop, Christopher M. ["Pattern recognition."](http://users.isr.ist.utl.pt/~wurmd/Livros/school/Bishop%20-%20Pattern%20Recognition%20And%20Machine%20Learning%20-%20Springer%20%202006.pdf) Machine Learning 128 (2006): 1-58.
<br/>
<a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/4.0/"><img alt="知识共享许可协议" style="border-width:0" src="https://i.creativecommons.org/l/by-nc-sa/4.0/88x31.png" /></a><br /><span xmlns:dct="http://purl.org/dc/terms/" href="http://purl.org/dc/dcmitype/Text" property="dct:title" rel="dct:type">This book</span> is created by <a xmlns:cc="http://creativecommons.org/ns#" href="http://book.paddlepaddle.org" property="cc:attributionName" rel="cc:attributionURL">PaddlePaddle</a>, and uses <a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/4.0/">Shared knowledge signature - non commercial use-Sharing 4.0 International Licensing Protocal</a>.
This tutorial is contributed by <a xmlns:cc="http://creativecommons.org/ns#" href="http://book.paddlepaddle.org" property="cc:attributionName" rel="cc:attributionURL">PaddlePaddle</a>, and licensed under a <a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/4.0/">Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License</a>.
</div>
<!-- You can change the lines below now. -->
......
recognize_digits/index.html
浏览文件 @ 8a826cf0
......@@ -42,7 +42,7 @@
<div id="markdown" style='display:none'>
# 识别数字
本教程源代码目录在[book/recognize_digits](https://github.com/PaddlePaddle/book/tree/develop/recognize_digits)， 初次使用请参考PaddlePaddle[安装教程](http://www.paddlepaddle.org/doc_cn/build_and_install/index.html)。
本教程源代码目录在[book/recognize_digits](https://github.com/PaddlePaddle/book/tree/develop/recognize_digits)， 初次使用请参考PaddlePaddle[安装教程](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/getstarted/build_and_install/docker_install_cn.rst)。
## 背景介绍
当我们学习编程的时候，编写的第一个程序一般是实现打印"Hello World"。而机器学习（或深度学习）的入门教程，一般都是 [MNIST](http://yann.lecun.com/exdb/mnist/) 数据库上的手写识别问题。原因是手写识别属于典型的图像分类问题，比较简单，同时MNIST数据集也很完备。MNIST数据集作为一个简单的计算机视觉数据集，包含一系列如图1所示的手写数字图片和对应的标签。图片是28x28的像素矩阵，标签则对应着0~9的10个数字。每张图片都经过了大小归一化和居中处理。
......@@ -287,7 +287,7 @@ def event_handler(event):
print "Pass %d, Batch %d, Cost %f, %s" % (
event.pass_id, event.batch_id, event.cost, event.metrics)
if isinstance(event, paddle.event.EndPass):
result = trainer.test(reader=paddle.reader.batched(
result = trainer.test(reader=paddle.batch(
paddle.dataset.mnist.test(), batch_size=128))
print "Test with Pass %d, Cost %f, %s\n" % (
event.pass_id, result.cost, result.metrics)
......@@ -295,7 +295,7 @@ def event_handler(event):
result.metrics['classification_error_evaluator']))
trainer.train(
reader=paddle.reader.batched(
reader=paddle.batch(
paddle.reader.shuffle(
paddle.dataset.mnist.train(), buf_size=8192),
batch_size=128),
......@@ -331,7 +331,7 @@ trainer.train(
7. Deng, Li, Michael L. Seltzer, Dong Yu, Alex Acero, Abdel-rahman Mohamed, and Geoffrey E. Hinton. ["Binary coding of speech spectrograms using a deep auto-encoder."](http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.185.1908&rep=rep1&type=pdf) In Interspeech, pp. 1692-1695. 2010.
8. Kégl, Balázs, and Róbert Busa-Fekete. ["Boosting products of base classifiers."](http://dl.acm.org/citation.cfm?id=1553439) In Proceedings of the 26th Annual International Conference on Machine Learning, pp. 497-504. ACM, 2009.
9. Rosenblatt, Frank. ["The perceptron: A probabilistic model for information storage and organization in the brain."](http://psycnet.apa.org/journals/rev/65/6/386/) Psychological review 65, no. 6 (1958): 386.
10. Bishop, Christopher M. ["Pattern recognition."](http://s3.amazonaws.com/academia.edu.documents/30428242/bg0137.pdf?AWSAccessKeyId=AKIAJ56TQJRTWSMTNPEA&Expires=1484816640&Signature=85Ad6%2Fca8T82pmHzxaSXermovIA%3D&response-content-disposition=inline%3B%20filename%3DPattern_recognition_and_machine_learning.pdf) Machine Learning 128 (2006): 1-58.
10. Bishop, Christopher M. ["Pattern recognition."](http://users.isr.ist.utl.pt/~wurmd/Livros/school/Bishop%20-%20Pattern%20Recognition%20And%20Machine%20Learning%20-%20Springer%20%202006.pdf) Machine Learning 128 (2006): 1-58.
<br/>
<a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/4.0/"><img alt="知识共享许可协议" style="border-width:0" src="https://i.creativecommons.org/l/by-nc-sa/4.0/88x31.png" /></a><br /><span xmlns:dct="http://purl.org/dc/terms/" href="http://purl.org/dc/dcmitype/Text" property="dct:title" rel="dct:type">本教程</span> 由 <a xmlns:cc="http://creativecommons.org/ns#" href="http://book.paddlepaddle.org" property="cc:attributionName" rel="cc:attributionURL">PaddlePaddle</a> 创作，采用 <a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/4.0/">知识共享 署名-非商业性使用-相同方式共享 4.0 国际 许可协议</a>进行许可。
......
recognize_digits/train.py
浏览文件 @ 8a826cf0
......@@ -2,9 +2,8 @@ import paddle.v2 as paddle
def softmax_regression(img):
predict = paddle.layer.fc(input=img,
size=10,
act=paddle.activation.Softmax())
predict = paddle.layer.fc(
input=img, size=10, act=paddle.activation.Softmax())
return predict
......@@ -12,14 +11,12 @@ def multilayer_perceptron(img):
# The first fully-connected layer
hidden1 = paddle.layer.fc(input=img, size=128, act=paddle.activation.Relu())
# The second fully-connected layer and the according activation function
hidden2 = paddle.layer.fc(input=hidden1,
size=64,
act=paddle.activation.Relu())
hidden2 = paddle.layer.fc(
input=hidden1, size=64, act=paddle.activation.Relu())
# The thrid fully-connected layer, note that the hidden size should be 10,
# which is the number of unique digits
predict = paddle.layer.fc(input=hidden2,
size=10,
act=paddle.activation.Softmax())
predict = paddle.layer.fc(
input=hidden2, size=10, act=paddle.activation.Softmax())
return predict
......@@ -43,14 +40,12 @@ def convolutional_neural_network(img):
pool_stride=2,
act=paddle.activation.Tanh())
# The first fully-connected layer
fc1 = paddle.layer.fc(input=conv_pool_2,
size=128,
act=paddle.activation.Tanh())
fc1 = paddle.layer.fc(
input=conv_pool_2, size=128, act=paddle.activation.Tanh())
# The softmax layer, note that the hidden size should be 10,
# which is the number of unique digits
predict = paddle.layer.fc(input=fc1,
size=10,
act=paddle.activation.Softmax())
predict = paddle.layer.fc(
input=fc1, size=10, act=paddle.activation.Softmax())
return predict
......@@ -76,9 +71,8 @@ optimizer = paddle.optimizer.Momentum(
momentum=0.9,
regularization=paddle.optimizer.L2Regularization(rate=0.0005 * 128))
trainer = paddle.trainer.SGD(cost=cost,
parameters=parameters,
update_equation=optimizer)
trainer = paddle.trainer.SGD(
cost=cost, parameters=parameters, update_equation=optimizer)
lists = []
......@@ -89,7 +83,7 @@ def event_handler(event):
print "Pass %d, Batch %d, Cost %f, %s" % (
event.pass_id, event.batch_id, event.cost, event.metrics)
if isinstance(event, paddle.event.EndPass):
result = trainer.test(reader=paddle.reader.batched(
result = trainer.test(reader=paddle.batch(
paddle.dataset.mnist.test(), batch_size=128))
print "Test with Pass %d, Cost %f, %s\n" % (event.pass_id, result.cost,
result.metrics)
......@@ -98,9 +92,8 @@ def event_handler(event):
trainer.train(
reader=paddle.reader.batched(
paddle.reader.shuffle(
paddle.dataset.mnist.train(), buf_size=8192),
reader=paddle.batch(
paddle.reader.shuffle(paddle.dataset.mnist.train(), buf_size=8192),
batch_size=128),
event_handler=event_handler,
num_passes=100)
......
recommender_system/README.en.ipynb 0 → 100644
浏览文件 @ 8a826cf0
此差异已折叠。
recommender_system/README.en.md
浏览文件 @ 8a826cf0
此差异已折叠。
recommender_system/README.ipynb
浏览文件 @ 8a826cf0
此差异已折叠。
recommender_system/README.md
浏览文件 @ 8a826cf0
# 个性化推荐
本教程源代码目录在[book/recommender_system](https://github.com/PaddlePaddle/book/tree/develop/recommender_system)， 初次使用请参考PaddlePaddle[安装教程](http://www.paddlepaddle.org/doc_cn/build_and_install/index.html)。
本教程源代码目录在[book/recommender_system](https://github.com/PaddlePaddle/book/tree/develop/recommender_system)， 初次使用请参考PaddlePaddle[安装教程](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/getstarted/build_and_install/docker_install_cn.rst)。
## 背景介绍
......
recommender_system/common_utils.py 已删除 100755 → 0
浏览文件 @ 4855822b
# Copyright (c) 2016 PaddlePaddle Authors. All Rights Reserved
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
from paddle.trainer.PyDataProvider2 import *
def meta_to_header(meta, name):
metas = meta[name]['__meta__']['raw_meta']
for each_meta in metas:
slot_name = each_meta.get('name', '%s_id' % name)
if each_meta['type'] == 'id':
yield slot_name, integer_value(each_meta['max'])
elif each_meta['type'] == 'embedding':
is_seq = each_meta['seq'] == 'sequence'
yield slot_name, integer_value(
len(each_meta['dict']),
seq_type=SequenceType.SEQUENCE
if is_seq else SequenceType.NO_SEQUENCE)
elif each_meta['type'] == 'one_hot_dense':
yield slot_name, dense_vector(len(each_meta['dict']))
recommender_system/data/config.json 已删除 100644 → 0
浏览文件 @ 4855822b
{
"user": {
"file": {
"name": "users.dat",
"delimiter": "::"
},
"fields": ["id", "gender", "age", "occupation"]
},
"movie": {
"file": {
"name": "movies.dat",
"delimiter": "::"
},
"fields": ["id", "title", "genres"]
}
}
recommender_system/data/config_generator.py 已删除 100644 → 0
浏览文件 @ 4855822b
此差异已折叠。
recommender_system/data/getdata.sh 已删除 100755 → 0
浏览文件 @ 4855822b
#!/bin/bash
# Copyright (c) 2016 PaddlePaddle Authors. All Rights Reserved
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
set -ex
cd "$(dirname "$0")"
# download the dataset
wget http://files.grouplens.org/datasets/movielens/ml-1m.zip
# unzip the dataset
unzip ml-1m.zip
# remove the unused zip file
rm ml-1m.zip
recommender_system/data/meta_generator.py 已删除 100644 → 0
浏览文件 @ 4855822b
此差异已折叠。
recommender_system/data/split.py 已删除 100644 → 0
浏览文件 @ 4855822b
#!/bin/env python2
# Copyright (c) 2016 PaddlePaddle Authors. All Rights Reserved
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""
Separate movielens 1m dataset to train/test file.
Usage:
./separate.py <input_file> [--test_ratio=<test_ratio>] [--delimiter=<delimiter>]
./separate.py -h | --help
Options:
-h --help Show this screen.
--version Show version.
--test_ratio=<test_ratio> Test ratio for separate [default: 0.1].
--delimiter=<delimiter> File delimiter [default: ,].
"""
import docopt
import collections
import random
def process(test_ratio, input_file, delimiter, **kwargs):
test_ratio = float(test_ratio)
rating_dict = collections.defaultdict(list)
with open(input_file, 'r') as f:
for line in f:
user_id = int(line.split(delimiter)[0])
rating_dict[user_id].append(line.strip())
with open(input_file + ".train", 'w') as train_file:
with open(input_file + ".test", 'w') as test_file:
for k in rating_dict.keys():
lines = rating_dict[k]
assert isinstance(lines, list)
random.shuffle(lines)
test_len = int(len(lines) * test_ratio)
for line in lines[:test_len]:
print >> test_file, line
for line in lines[test_len:]:
print >> train_file, line
if __name__ == '__main__':
args = docopt.docopt(__doc__, version='0.1.0')
kwargs = dict()
for key in args.keys():
if key != '--help':
param_name = key
assert isinstance(param_name, str)
param_name = param_name.replace('<', '')
param_name = param_name.replace('>', '')
param_name = param_name.replace('--', '')
kwargs[param_name] = args[key]
process(**kwargs)
recommender_system/dataprovider.py 已删除 100755 → 0
浏览文件 @ 4855822b
此差异已折叠。
recommender_system/evaluate.py 已删除 100755 → 0
浏览文件 @ 4855822b
此差异已折叠。
recommender_system/index.en.html
浏览文件 @ 8a826cf0
此差异已折叠。
recommender_system/index.html
浏览文件 @ 8a826cf0
此差异已折叠。
recommender_system/prediction.py 已删除 100755 → 0
浏览文件 @ 4855822b
此差异已折叠。
recommender_system/preprocess.sh 已删除 100755 → 0
浏览文件 @ 4855822b
此差异已折叠。
recommender_system/train.py 0 → 100644
浏览文件 @ 8a826cf0
此差异已折叠。
recommender_system/train.sh 已删除 100755 → 0
浏览文件 @ 4855822b
此差异已折叠。
recommender_system/trainer_config.py 已删除 100755 → 0
浏览文件 @ 4855822b
此差异已折叠。
tools/convert-markdown-into-ipynb-and-test.sh 0 → 100755
浏览文件 @ 8a826cf0
此差异已折叠。
understand_sentiment/README.en.md
浏览文件 @ 8a826cf0
此差异已折叠。
understand_sentiment/README.md
浏览文件 @ 8a826cf0
此差异已折叠。
understand_sentiment/index.en.html
浏览文件 @ 8a826cf0
此差异已折叠。
understand_sentiment/index.html
浏览文件 @ 8a826cf0
此差异已折叠。
understand_sentiment/train.py
浏览文件 @ 8a826cf0
此差异已折叠。
word2vec/README.en.md
浏览文件 @ 8a826cf0
此差异已折叠。
word2vec/README.md
浏览文件 @ 8a826cf0
此差异已折叠。
word2vec/index.en.html
浏览文件 @ 8a826cf0
此差异已折叠。
word2vec/index.html
浏览文件 @ 8a826cf0
此差异已折叠。
word2vec/train.py
浏览文件 @ 8a826cf0
此差异已折叠。
Markdown is supported
0% 或 .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑！
想要评论请 注册 或