diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index 40c5fec3519e79aeed7e47187e62b149a2da03ca..655b823a76dd4b9b5be10c1a2b3d9c9cb11dc799 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -32,3 +32,11 @@ entry: python pre-commit-hooks/convert_markdown_into_html.py language: system files: \.md$ +- repo: local + hooks: + - id: convert-markdown-into-ipynb + name: convert-markdown-into-ipynb + description: Convert README.md into README.ipynb and README.en.md into README.en.ipynb + entry: ./pre-commit-hooks/convert_markdown_into_ipynb.sh + language: system + files: \.md$ diff --git a/.travis.yml b/.travis.yml index f5409798d0baed1d518ff8c85c84fcad0d0fa701..3cf90df9582abf65371ddad5ca5eab8cb687dc77 100644 --- a/.travis.yml +++ b/.travis.yml @@ -14,8 +14,10 @@ addons: - python - python-pip - python2.7-dev + - golang before_install: - pip install virtualenv pre-commit + - GOPATH=/tmp/go go get -u github.com/wangkuiyi/ipynb/markdown-to-ipynb script: - travis/precommit.sh notifications: diff --git a/fit_a_line/README.en.ipynb b/fit_a_line/README.en.ipynb new file mode 100644 index 0000000000000000000000000000000000000000..110b3031026f1fd0d65cbb1b3a7d45c43c86ee7a --- /dev/null +++ b/fit_a_line/README.en.ipynb @@ -0,0 +1,412 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Linear Regression\n", + "Let us begin the tutorial with a classical problem called Linear Regression \\[[1](#References)\\]. In this chapter, we will train a model from a realistic dataset to predict home prices. Some important concepts in Machine Learning will be covered through this example.\n", + "\n", + "The source code for this tutorial lives on [book/fit_a_line](https://github.com/PaddlePaddle/book/tree/develop/fit_a_line). For instructions on getting started with PaddlePaddle, see [PaddlePaddle installation guide](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/getstarted/build_and_install/docker_install_en.rst).\n", + "\n", + "## Problem Setup\n", + "Suppose we have a dataset of $n$ real estate properties. These real estate properties will be referred to as *homes* in this chapter for clarity.\n", + "\n", + "Each home is associated with $d$ attributes. The attributes describe characteristics such the number of rooms in the home, the number of schools or hospitals in the neighborhood, and the traffic condition nearby.\n", + "\n", + "In our problem setup, the attribute $x_{i,j}$ denotes the $j$th characteristic of the $i$th home. In addition, $y_i$ denotes the price of the $i$th home. Our task is to predict $y_i$ given a set of attributes $\\{x_{i,1}, ..., x_{i,d}\\}$. We assume that the price of a home is a linear combination of all of its attributes, namely,\n", + "\n", + "$$y_i = \\omega_1x_{i,1} + \\omega_2x_{i,2} + \\ldots + \\omega_dx_{i,d} + b, i=1,\\ldots,n$$\n", + "\n", + "where $\\vec{\\omega}$ and $b$ are the model parameters we want to estimate. Once they are learned, we will be able to predict the price of a home, given the attributes associated with it. We call this model **Linear Regression**. In other words, we want to regress a value against several values linearly. In practice, a linear model is often too simplistic to capture the real relationships between the variables. Yet, because Linear Regression is easy to train and analyze, it has been applied to a large number of real problems. As a result, it is an important topic in many classic Statistical Learning and Machine Learning textbooks \\[[2,3,4](#References)\\].\n", + "\n", + "## Results Demonstration\n", + "We first show the result of our model. The dataset [UCI Housing Data Set](https://archive.ics.uci.edu/ml/datasets/Housing) is used to train a linear model to predict the home prices in Boston. The figure below shows the predictions the model makes for some home prices. The $X$-axis represents the median value of the prices of simlilar homes within a bin, while the $Y$-axis represents the home value our linear model predicts. The dotted line represents points where $X=Y$. When reading the diagram, the more precise the model predicts, the closer the point is to the dotted line.\n", + "\u003cp align=\"center\"\u003e\n", + " \u003cimg src = \"image/predictions_en.png\" width=400\u003e\u003cbr/\u003e\n", + " Figure 1. Predicted Value V.S. Actual Value\n", + "\u003c/p\u003e\n", + "\n", + "## Model Overview\n", + "\n", + "### Model Definition\n", + "\n", + "In the UCI Housing Data Set, there are 13 home attributes $\\{x_{i,j}\\}$ that are related to the median home price $y_i$, which we aim to predict. Thus, our model can be written as:\n", + "\n", + "$$\\hat{Y} = \\omega_1X_{1} + \\omega_2X_{2} + \\ldots + \\omega_{13}X_{13} + b$$\n", + "\n", + "where $\\hat{Y}$ is the predicted value used to differentiate from actual value $Y$. The model learns parameters $\\omega_1, \\ldots, \\omega_{13}, b$, where the entries of $\\vec{\\omega}$ are **weights** and $b$ is **bias**.\n", + "\n", + "Now we need an objective to optimize, so that the learned parameters can make $\\hat{Y}$ as close to $Y$ as possible. Let's refer to the concept of [Loss Function (Cost Function)](https://en.wikipedia.org/wiki/Loss_function). A loss function must output a non-negative value, given any pair of the actual value $y_i$ and the predicted value $\\hat{y_i}$. This value reflects the magnitutude of the model error.\n", + "\n", + "For Linear Regression, the most common loss function is [Mean Square Error (MSE)](https://en.wikipedia.org/wiki/Mean_squared_error) which has the following form:\n", + "\n", + "$$MSE=\\frac{1}{n}\\sum_{i=1}^{n}{(\\hat{Y_i}-Y_i)}^2$$\n", + "\n", + "That is, for a dataset of size $n$, MSE is the average value of the the prediction sqaure errors.\n", + "\n", + "### Training\n", + "\n", + "After setting up our model, there are several major steps to go through to train it:\n", + "1. Initialize the parameters including the weights $\\vec{\\omega}$ and the bias $b$. For example, we can set their mean values as $0$s, and their standard deviations as $1$s.\n", + "2. Feedforward. Evaluate the network output and compute the corresponding loss.\n", + "3. [Backpropagate](https://en.wikipedia.org/wiki/Backpropagation) the errors. The errors will be propagated from the output layer back to the input layer, during which the model parameters will be updated with the corresponding errors.\n", + "4. Repeat steps 2~3, until the loss is below a predefined threshold or the maximum number of repeats is reached.\n", + "\n", + "## Dataset\n", + "\n", + "### Python Dataset Modules\n", + "\n", + "Our program starts with importing necessary packages:\n", + "\n" + ] + }, + { + "cell_type": "code", + "metadata": { + "editable": true + }, + "source": [ + "import paddle.v2 as paddle\n", + "import paddle.v2.dataset.uci_housing as uci_housing\n" + ], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n" + ] + } + ], + "execution_count": 1 + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n", + "We encapsulated the [UCI Housing Data Set](https://archive.ics.uci.edu/ml/datasets/Housing) in our Python module `uci_housing`. This module can\n", + "\n", + "1. download the dataset to `~/.cache/paddle/dataset/uci_housing/housing.data`, if not yet, and\n", + "2. [preprocesses](#preprocessing) the dataset.\n", + "\n", + "### An Introduction of the Dataset\n", + "\n", + "The UCI housing dataset has 506 instances. Each instance describes the attributes of a house in surburban Boston. The attributes are explained below:\n", + "\n", + "| Attribute Name | Characteristic | Data Type |\n", + "| ------| ------ | ------ |\n", + "| CRIM | per capita crime rate by town | Continuous|\n", + "| ZN | proportion of residential land zoned for lots over 25,000 sq.ft. | Continuous |\n", + "| INDUS | proportion of non-retail business acres per town | Continuous |\n", + "| CHAS | Charles River dummy variable | Discrete, 1 if tract bounds river; 0 otherwise|\n", + "| NOX | nitric oxides concentration (parts per 10 million) | Continuous |\n", + "| RM | average number of rooms per dwelling | Continuous |\n", + "| AGE | proportion of owner-occupied units built prior to 1940 | Continuous |\n", + "| DIS | weighted distances to five Boston employment centres | Continuous |\n", + "| RAD | index of accessibility to radial highways | Continuous |\n", + "| TAX | full-value property-tax rate per $10,000 | Continuous |\n", + "| PTRATIO | pupil-teacher ratio by town | Continuous |\n", + "| B | 1000(Bk - 0.63)^2 where Bk is the proportion of blacks by town | Continuous |\n", + "| LSTAT | % lower status of the population | Continuous |\n", + "| MEDV | Median value of owner-occupied homes in $1000's | Continuous |\n", + "\n", + "The last entry is the median home price.\n", + "\n", + "### Preprocessing\n", + "#### Continuous and Discrete Data\n", + "We define a feature vector of length 13 for each home, where each entry corresponds to an attribute. Our first observation is that, among the 13 dimensions, there are 12 continuous dimensions and 1 discrete dimension.\n", + "\n", + "Note that although a discrete value is also written as numeric values such as 0, 1, or 2, its meaning differs from a continuous value drastically. The linear difference between two discrete values has no meaning. For example, suppose $0$, $1$, and $2$ are used to represent colors *Red*, *Green*, and *Blue* respectively. Judging from the numeric representation of these colors, *Red* differs more from *Blue* than it does from *Green*. Yet in actuality, it is not true that extent to which the color *Blue* is different from *Red* is greater than the extent to which *Green* is different from *Red*. Therefore, when handling a discrete feature that has $d$ possible values, we usually convert it to $d$ new features where each feature takes a binary value, $0$ or $1$, indicating whether the original value is absent or present. Alternatively, the discrete features can be mapped onto a continuous multi-dimensional vector through an embedding table. For our problem here, because CHAS itself is a binary discrete value, we do not need to do any preprocessing.\n", + "\n", + "#### Feature Normalization\n", + "We also observe a huge difference among the value ranges of the 13 features (Figure 2). For instance, the values of feature *B* fall in $[0.32, 396.90]$, whereas those of feature *NOX* has a range of $[0.3850, 0.8170]$. An effective optimization would require data normalization. The goal of data normalization is to scale te values of each feature into roughly the same range, perhaps $[-0.5, 0.5]$. Here, we adopt a popular normalization technique where we substract the mean value from the feature value and divide the result by the width of the original range.\n", + "\n", + "There are at least three reasons for [Feature Normalization](https://en.wikipedia.org/wiki/Feature_scaling) (Feature Scaling):\n", + "- A value range that is too large or too small might cause floating number overflow or underflow during computation.\n", + "- Different value ranges might result in varying *importances* of different features to the model (at least in the beginning of the training process). This assumption about the data is often unreasonable, making the optimization difficult, which in turn results in increased training time.\n", + "- Many machine learning techniques or models (e.g., *L1/L2 regularization* and *Vector Space Model*) assumes that all the features have roughly zero means and their value ranges are similar.\n", + "\n", + "\u003cp align=\"center\"\u003e\n", + " \u003cimg src = \"image/ranges_en.png\" width=550\u003e\u003cbr/\u003e\n", + " Figure 2. The value ranges of the features\n", + "\u003c/p\u003e\n", + "\n", + "#### Prepare Training and Test Sets\n", + "We split the dataset in two, one for adjusting the model parameters, namely, for model training, and the other for model testing. The model error on the former is called the **training error**, and the error on the latter is called the **test error**. Our goal in training a model is to find the statistical dependency between the outputs and the inputs, so that we can predict new outputs given new inputs. As a result, the test error reflects the performance of the model better than the training error does. We consider two things when deciding the ratio of the training set to the test set: 1) More training data will decrease the variance of the parameter estimation, yielding more reliable models; 2) More test data will decrease the variance of the test error, yielding more reliable test errors. One standard split ratio is $8:2$.\n", + "\n", + "\n", + "When training complex models, we usually have one more split: the validation set. Complex models usually have [Hyperparameters](https://en.wikipedia.org/wiki/Hyperparameter_optimization) that need to be set before the training process, such as the number of layers in the network. Because hyperparameters are not part of the model parameters, they cannot be trained using the same loss function. Thus we will try several sets of hyperparameters to train several models and cross-validate them on the validation set to pick the best one; finally, the selected trained model is tested on the test set. Because our model is relatively simple, we will omit this validation process.\n", + "\n", + "\n", + "## Training\n", + "\n", + "`fit_a_line/trainer.py` demonstrates the training using [PaddlePaddle](http://paddlepaddle.org).\n", + "\n", + "### Initialize PaddlePaddle\n", + "\n" + ] + }, + { + "cell_type": "code", + "metadata": { + "editable": true + }, + "source": [ + "paddle.init(use_gpu=False, trainer_count=1)\n" + ], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n" + ] + } + ], + "execution_count": 1 + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n", + "### Model Configuration\n", + "\n", + "Logistic regression is essentially a fully-connected layer with linear activation:\n", + "\n" + ] + }, + { + "cell_type": "code", + "metadata": { + "editable": true + }, + "source": [ + "x = paddle.layer.data(name='x', type=paddle.data_type.dense_vector(13))\n", + "y_predict = paddle.layer.fc(input=x,\n", + " size=1,\n", + " act=paddle.activation.Linear())\n", + "y = paddle.layer.data(name='y', type=paddle.data_type.dense_vector(1))\n", + "cost = paddle.layer.regression_cost(input=y_predict, label=y)\n" + ], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n" + ] + } + ], + "execution_count": 1 + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Create Parameters\n", + "\n" + ] + }, + { + "cell_type": "code", + "metadata": { + "editable": true + }, + "source": [ + "parameters = paddle.parameters.create(cost)\n" + ], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n" + ] + } + ], + "execution_count": 1 + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n", + "### Create Trainer\n", + "\n" + ] + }, + { + "cell_type": "code", + "metadata": { + "editable": true + }, + "source": [ + "optimizer = paddle.optimizer.Momentum(momentum=0)\n", + "\n", + "trainer = paddle.trainer.SGD(cost=cost,\n", + " parameters=parameters,\n", + " update_equation=optimizer)\n" + ], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n" + ] + } + ], + "execution_count": 1 + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n", + "### Feeding Data\n", + "\n", + "PaddlePaddle provides the\n", + "[reader mechanism](https://github.com/PaddlePaddle/Paddle/tree/develop/doc/design/reader)\n", + "for loadinng training data. A reader may return multiple columns, and we need a Python dictionary to specify the mapping from column index to data layers.\n", + "\n" + ] + }, + { + "cell_type": "code", + "metadata": { + "editable": true + }, + "source": [ + "feeding={'x': 0, 'y': 1}\n" + ], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n" + ] + } + ], + "execution_count": 1 + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n", + "Moreover, an event handler is provided to print the training progress:\n", + "\n" + ] + }, + { + "cell_type": "code", + "metadata": { + "editable": true + }, + "source": [ + "# event_handler to print training and testing info\n", + "def event_handler(event):\n", + " if isinstance(event, paddle.event.EndIteration):\n", + " if event.batch_id % 100 == 0:\n", + " print \"Pass %d, Batch %d, Cost %f\" % (\n", + " event.pass_id, event.batch_id, event.cost)\n", + "\n", + " if isinstance(event, paddle.event.EndPass):\n", + " result = trainer.test(\n", + " reader=paddle.batch(\n", + " uci_housing.test(), batch_size=2),\n", + " feeding=feeding)\n", + " print \"Test %d, Cost %f\" % (event.pass_id, result.cost)\n" + ], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n" + ] + } + ], + "execution_count": 1 + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n", + "### Start Training\n", + "\n" + ] + }, + { + "cell_type": "code", + "metadata": { + "editable": true + }, + "source": [ + "trainer.train(\n", + " reader=paddle.batch(\n", + " paddle.reader.shuffle(\n", + " uci_housing.train(), buf_size=500),\n", + " batch_size=2),\n", + " feeding=feeding,\n", + " event_handler=event_handler,\n", + " num_passes=30)\n" + ], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n" + ] + } + ], + "execution_count": 1 + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n", + "## Summary\n", + "This chapter introduces *Linear Regression* and how to train and test this model with PaddlePaddle, using the UCI Housing Data Set. Because a large number of more complex models and techniques are derived from linear regression, it is important to understand its underlying theory and limitation.\n", + "\n", + "\n", + "## References\n", + "1. https://en.wikipedia.org/wiki/Linear_regression\n", + "2. Friedman J, Hastie T, Tibshirani R. The elements of statistical learning[M]. Springer, Berlin: Springer series in statistics, 2001.\n", + "3. Murphy K P. Machine learning: a probabilistic perspective[M]. MIT press, 2012.\n", + "4. Bishop C M. Pattern recognition[J]. Machine Learning, 2006, 128.\n", + "\n", + "\u003cbr/\u003e\n", + "This tutorial is contributed by \u003ca xmlns:cc=\"http://creativecommons.org/ns#\" href=\"http://book.paddlepaddle.org\" property=\"cc:attributionName\" rel=\"cc:attributionURL\"\u003ePaddlePaddle\u003c/a\u003e, and licensed under a \u003ca rel=\"license\" href=\"http://creativecommons.org/licenses/by-nc-sa/4.0/\"\u003eCreative Commons Attribution-NonCommercial-ShareAlike 4.0 International License\u003c/a\u003e.\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.6.0" + } + }, + "nbformat": 4, + "nbformat_minor": 0 +} diff --git a/fit_a_line/README.en.md b/fit_a_line/README.en.md index 455eca6a2bc69482926b60099ebe267e4011aa7c..eeef844d24c4ceb25028d7ec85a5335f82d5c51d 100644 --- a/fit_a_line/README.en.md +++ b/fit_a_line/README.en.md @@ -1,7 +1,7 @@ # Linear Regression Let us begin the tutorial with a classical problem called Linear Regression \[[1](#References)\]. In this chapter, we will train a model from a realistic dataset to predict home prices. Some important concepts in Machine Learning will be covered through this example. -The source code for this tutorial lives on [book/fit_a_line](https://github.com/PaddlePaddle/book/tree/develop/fit_a_line). For instructions on getting started with PaddlePaddle, see [PaddlePaddle installation guide](http://www.paddlepaddle.org/doc_cn/build_and_install/index.html). +The source code for this tutorial lives on [book/fit_a_line](https://github.com/PaddlePaddle/book/tree/develop/fit_a_line). For instructions on getting started with PaddlePaddle, see [PaddlePaddle installation guide](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/getstarted/build_and_install/docker_install_en.rst). ## Problem Setup Suppose we have a dataset of $n$ real estate properties. These real estate properties will be referred to as *homes* in this chapter for clarity. diff --git a/fit_a_line/README.ipynb b/fit_a_line/README.ipynb new file mode 100644 index 0000000000000000000000000000000000000000..c690570a8a4ef048f2e3460a4b095919bf041f60 --- /dev/null +++ b/fit_a_line/README.ipynb @@ -0,0 +1,407 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# 线性回归\n", + "让我们从经典的线性回归（Linear Regression \\[[1](#参考文献)\\]）模型开始这份教程。在这一章里，你将使用真实的数据集建立起一个房价预测模型，并且了解到机器学习中的若干重要概念。\n", + "\n", + "本教程源代码目录在[book/fit_a_line](https://github.com/PaddlePaddle/book/tree/develop/fit_a_line)， 初次使用请参考PaddlePaddle[安装教程](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/getstarted/build_and_install/docker_install_cn.rst)。\n", + "\n", + "## 背景介绍\n", + "给定一个大小为$n$的数据集 ${\\{y_{i}, x_{i1}, ..., x_{id}\\}}_{i=1}^{n}$，其中$x_{i1}, \\ldots, x_{id}$是第$i$个样本$d$个属性上的取值，$y_i$是该样本待预测的目标。线性回归模型假设目标$y_i$可以被属性间的线性组合描述，即\n", + "\n", + "$$y_i = \\omega_1x_{i1} + \\omega_2x_{i2} + \\ldots + \\omega_dx_{id} + b, i=1,\\ldots,n$$\n", + "\n", + "例如，在我们将要建模的房价预测问题里，$x_{ij}$是描述房子$i$的各种属性（比如房间的个数、周围学校和医院的个数、交通状况等），而 $y_i$是房屋的价格。\n", + "\n", + "初看起来，这个假设实在过于简单了，变量间的真实关系很难是线性的。但由于线性回归模型有形式简单和易于建模分析的优点，它在实际问题中得到了大量的应用。很多经典的统计学习、机器学习书籍\\[[2,3,4](#参考文献)\\]也选择对线性模型独立成章重点讲解。\n", + "\n", + "## 效果展示\n", + "我们使用从[UCI Housing Data Set](https://archive.ics.uci.edu/ml/datasets/Housing)获得的波士顿房价数据集进行模型的训练和预测。下面的散点图展示了使用模型对部分房屋价格进行的预测。其中，每个点的横坐标表示同一类房屋真实价格的中位数，纵坐标表示线性回归模型根据特征预测的结果，当二者值完全相等的时候就会落在虚线上。所以模型预测得越准确，则点离虚线越近。\n", + "\u003cp align=\"center\"\u003e\n", + " \u003cimg src = \"image/predictions.png\" width=400\u003e\u003cbr/\u003e\n", + " 图1. 预测值 V.S. 真实值\n", + "\u003c/p\u003e\n", + "\n", + "## 模型概览\n", + "\n", + "### 模型定义\n", + "\n", + "在波士顿房价数据集中，和房屋相关的值共有14个：前13个用来描述房屋相关的各种信息，即模型中的 $x_i$；最后一个值为我们要预测的该类房屋价格的中位数，即模型中的 $y_i$。因此，我们的模型就可以表示成：\n", + "\n", + "$$\\hat{Y} = \\omega_1X_{1} + \\omega_2X_{2} + \\ldots + \\omega_{13}X_{13} + b$$\n", + "\n", + "$\\hat{Y}$ 表示模型的预测结果，用来和真实值$Y$区分。模型要学习的参数即：$\\omega_1, \\ldots, \\omega_{13}, b$。\n", + "\n", + "建立模型后，我们需要给模型一个优化目标，使得学到的参数能够让预测值$\\hat{Y}$尽可能地接近真实值$Y$。这里我们引入损失函数（[Loss Function](https://en.wikipedia.org/wiki/Loss_function)，或Cost Function）这个概念。 输入任意一个数据样本的目标值$y_{i}$和模型给出的预测值$\\hat{y_{i}}$，损失函数输出一个非负的实值。这个实质通常用来反映模型误差的大小。\n", + "\n", + "对于线性回归模型来讲，最常见的损失函数就是均方误差（Mean Squared Error， [MSE](https://en.wikipedia.org/wiki/Mean_squared_error)）了，它的形式是：\n", + "\n", + "$$MSE=\\frac{1}{n}\\sum_{i=1}^{n}{(\\hat{Y_i}-Y_i)}^2$$\n", + "\n", + "即对于一个大小为$n$的测试集，$MSE$是$n$个数据预测结果误差平方的均值。\n", + "\n", + "### 训练过程\n", + "\n", + "定义好模型结构之后，我们要通过以下几个步骤进行模型训练\n", + " 1. 初始化参数，其中包括权重$\\omega_i$和偏置$b$，对其进行初始化（如0均值，1方差）。\n", + " 2. 网络正向传播计算网络输出和损失函数。\n", + " 3. 根据损失函数进行反向误差传播 （[backpropagation](https://en.wikipedia.org/wiki/Backpropagation)），将网络误差从输出层依次向前传递, 并更新网络中的参数。\n", + " 4. 重复2~3步骤，直至网络训练误差达到规定的程度或训练轮次达到设定值。\n", + "\n", + "## 数据集\n", + "\n", + "### 数据集接口的封装\n", + "首先加载需要的包\n", + "\n" + ] + }, + { + "cell_type": "code", + "metadata": { + "editable": true + }, + "source": [ + "import paddle.v2 as paddle\n", + "import paddle.v2.dataset.uci_housing as uci_housing\n" + ], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n" + ] + } + ], + "execution_count": 1 + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n", + "我们通过uci_housing模块引入了数据集合[UCI Housing Data Set](https://archive.ics.uci.edu/ml/datasets/Housing)\n", + "\n", + "其中，在uci_housing模块中封装了：\n", + "\n", + "1. 数据下载的过程。下载数据保存在~/.cache/paddle/dataset/uci_housing/housing.data。\n", + "2. [数据预处理](#数据预处理)的过程。\n", + "\n", + "\n", + "### 数据集介绍\n", + "这份数据集共506行，每行包含了波士顿郊区的一类房屋的相关信息及该类房屋价格的中位数。其各维属性的意义如下：\n", + "\n", + "| 属性名 | 解释 | 类型 |\n", + "| ------| ------ | ------ |\n", + "| CRIM | 该镇的人均犯罪率 | 连续值 |\n", + "| ZN | 占地面积超过25,000平方呎的住宅用地比例 | 连续值 |\n", + "| INDUS | 非零售商业用地比例 | 连续值 |\n", + "| CHAS | 是否邻近 Charles River | 离散值，1=邻近；0=不邻近 |\n", + "| NOX | 一氧化氮浓度 | 连续值 |\n", + "| RM | 每栋房屋的平均客房数 | 连续值 |\n", + "| AGE | 1940年之前建成的自用单位比例 | 连续值 |\n", + "| DIS | 到波士顿5个就业中心的加权距离 | 连续值 |\n", + "| RAD | 到径向公路的可达性指数 | 连续值 |\n", + "| TAX | 全值财产税率 | 连续值 |\n", + "| PTRATIO | 学生与教师的比例 | 连续值 |\n", + "| B | 1000(BK - 0.63)^2，其中BK为黑人占比 | 连续值 |\n", + "| LSTAT | 低收入人群占比 | 连续值 |\n", + "| MEDV | 同类房屋价格的中位数 | 连续值 |\n", + "\n", + "### 数据预处理\n", + "#### 连续值与离散值\n", + "观察一下数据，我们的第一个发现是：所有的13维属性中，有12维的连续值和1维的离散值（CHAS）。离散值虽然也常使用类似0、1、2这样的数字表示，但是其含义与连续值是不同的，因为这里的差值没有实际意义。例如，我们用0、1、2来分别表示红色、绿色和蓝色的话，我们并不能因此说“蓝色和红色”比“绿色和红色”的距离更远。所以通常对一个有$d$个可能取值的离散属性，我们会将它们转为$d$个取值为0或1的二值属性或者将每个可能取值映射为一个多维向量。不过就这里而言，因为CHAS本身就是一个二值属性，就省去了这个麻烦。\n", + "\n", + "#### 属性的归一化\n", + "另外一个稍加观察即可发现的事实是，各维属性的取值范围差别很大（如图2所示）。例如，属性B的取值范围是[0.32, 396.90]，而属性NOX的取值范围是[0.3850, 0.8170]。这里就要用到一个常见的操作-归一化（normalization）了。归一化的目标是把各位属性的取值范围放缩到差不多的区间，例如[-0.5,0.5]。这里我们使用一种很常见的操作方法：减掉均值，然后除以原取值范围。\n", + "\n", + "做归一化（或 [Feature scaling](https://en.wikipedia.org/wiki/Feature_scaling)）至少有以下3个理由：\n", + "- 过大或过小的数值范围会导致计算时的浮点上溢或下溢。\n", + "- 不同的数值范围会导致不同属性对模型的重要性不同（至少在训练的初始阶段如此），而这个隐含的假设常常是不合理的。这会对优化的过程造成困难，使训练时间大大的加长。\n", + "- 很多的机器学习技巧/模型（例如L1，L2正则项，向量空间模型-Vector Space Model）都基于这样的假设：所有的属性取值都差不多是以0为均值且取值范围相近的。\n", + "\n", + "\u003cp align=\"center\"\u003e\n", + " \u003cimg src = \"image/ranges.png\" width=550\u003e\u003cbr/\u003e\n", + " 图2. 各维属性的取值范围\n", + "\u003c/p\u003e\n", + "\n", + "#### 整理训练集与测试集\n", + "我们将数据集分割为两份：一份用于调整模型的参数，即进行模型的训练，模型在这份数据集上的误差被称为**训练误差**；另外一份被用来测试，模型在这份数据集上的误差被称为**测试误差**。我们训练模型的目的是为了通过从训练数据中找到规律来预测未知的新数据，所以测试误差是更能反映模型表现的指标。分割数据的比例要考虑到两个因素：更多的训练数据会降低参数估计的方差，从而得到更可信的模型；而更多的测试数据会降低测试误差的方差，从而得到更可信的测试误差。我们这个例子中设置的分割比例为$8:2$\n", + "\n", + "\n", + "在更复杂的模型训练过程中，我们往往还会多使用一种数据集：验证集。因为复杂的模型中常常还有一些超参数（[Hyperparameter](https://en.wikipedia.org/wiki/Hyperparameter_optimization)）需要调节，所以我们会尝试多种超参数的组合来分别训练多个模型，然后对比它们在验证集上的表现选择相对最好的一组超参数，最后才使用这组参数下训练的模型在测试集上评估测试误差。由于本章训练的模型比较简单，我们暂且忽略掉这个过程。\n", + "\n", + "## 训练\n", + "\n", + "`fit_a_line/trainer.py`演示了训练的整体过程。\n", + "\n", + "### 初始化PaddlePaddle\n", + "\n" + ] + }, + { + "cell_type": "code", + "metadata": { + "editable": true + }, + "source": [ + "paddle.init(use_gpu=False, trainer_count=1)\n" + ], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n" + ] + } + ], + "execution_count": 1 + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n", + "### 模型配置\n", + "\n", + "线性回归的模型其实就是一个采用线性激活函数（linear activation，`LinearActivation`）的全连接层（fully-connected layer，`fc_layer`）：\n", + "\n" + ] + }, + { + "cell_type": "code", + "metadata": { + "editable": true + }, + "source": [ + "x = paddle.layer.data(name='x', type=paddle.data_type.dense_vector(13))\n", + "y_predict = paddle.layer.fc(input=x,\n", + " size=1,\n", + " act=paddle.activation.Linear())\n", + "y = paddle.layer.data(name='y', type=paddle.data_type.dense_vector(1))\n", + "cost = paddle.layer.regression_cost(input=y_predict, label=y)\n" + ], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n" + ] + } + ], + "execution_count": 1 + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### 创建参数\n", + "\n" + ] + }, + { + "cell_type": "code", + "metadata": { + "editable": true + }, + "source": [ + "parameters = paddle.parameters.create(cost)\n" + ], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n" + ] + } + ], + "execution_count": 1 + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n", + "### 创建Trainer\n", + "\n" + ] + }, + { + "cell_type": "code", + "metadata": { + "editable": true + }, + "source": [ + "optimizer = paddle.optimizer.Momentum(momentum=0)\n", + "\n", + "trainer = paddle.trainer.SGD(cost=cost,\n", + " parameters=parameters,\n", + " update_equation=optimizer)\n" + ], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n" + ] + } + ], + "execution_count": 1 + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n", + "### 读取数据且打印训练的中间信息\n", + "\n", + "PaddlePaddle提供一个\n", + "[reader机制](https://github.com/PaddlePaddle/Paddle/tree/develop/doc/design/reader)\n", + "来读取数据。 Reader返回的数据可以包括多列，我们需要一个Python dict把列\n", + "序号映射到网络里的数据层。\n", + "\n" + ] + }, + { + "cell_type": "code", + "metadata": { + "editable": true + }, + "source": [ + "feeding={'x': 0, 'y': 1}\n" + ], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n" + ] + } + ], + "execution_count": 1 + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n", + "此外，我们还可以提供一个 event handler，来打印训练的进度：\n", + "\n" + ] + }, + { + "cell_type": "code", + "metadata": { + "editable": true + }, + "source": [ + "# event_handler to print training and testing info\n", + "def event_handler(event):\n", + " if isinstance(event, paddle.event.EndIteration):\n", + " if event.batch_id % 100 == 0:\n", + " print \"Pass %d, Batch %d, Cost %f\" % (\n", + " event.pass_id, event.batch_id, event.cost)\n", + "\n", + " if isinstance(event, paddle.event.EndPass):\n", + " result = trainer.test(\n", + " reader=paddle.batch(\n", + " uci_housing.test(), batch_size=2),\n", + " feeding=feeding)\n", + " print \"Test %d, Cost %f\" % (event.pass_id, result.cost)\n" + ], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n" + ] + } + ], + "execution_count": 1 + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n", + "### 开始训练\n", + "\n" + ] + }, + { + "cell_type": "code", + "metadata": { + "editable": true + }, + "source": [ + "trainer.train(\n", + " reader=paddle.batch(\n", + " paddle.reader.shuffle(\n", + " uci_housing.train(), buf_size=500),\n", + " batch_size=2),\n", + " feeding=feeding,\n", + " event_handler=event_handler,\n", + " num_passes=30)\n" + ], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n" + ] + } + ], + "execution_count": 1 + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n", + "## 总结\n", + "在这章里，我们借助波士顿房价这一数据集，介绍了线性回归模型的基本概念，以及如何使用PaddlePaddle实现训练和测试的过程。很多的模型和技巧都是从简单的线性回归模型演化而来，因此弄清楚线性模型的原理和局限非常重要。\n", + "\n", + "\n", + "## 参考文献\n", + "1. https://en.wikipedia.org/wiki/Linear_regression\n", + "2. Friedman J, Hastie T, Tibshirani R. The elements of statistical learning[M]. Springer, Berlin: Springer series in statistics, 2001.\n", + "3. Murphy K P. Machine learning: a probabilistic perspective[M]. MIT press, 2012.\n", + "4. Bishop C M. Pattern recognition[J]. Machine Learning, 2006, 128.\n", + "\n", + "\u003cbr/\u003e\n", + "\u003ca rel=\"license\" href=\"http://creativecommons.org/licenses/by-nc-sa/4.0/\"\u003e\u003cimg alt=\"知识共享许可协议\" style=\"border-width:0\" src=\"https://i.creativecommons.org/l/by-nc-sa/4.0/88x31.png\" /\u003e\u003c/a\u003e\u003cbr /\u003e\u003cspan xmlns:dct=\"http://purl.org/dc/terms/\" href=\"http://purl.org/dc/dcmitype/Text\" property=\"dct:title\" rel=\"dct:type\"\u003e本教程\u003c/span\u003e 由 \u003ca xmlns:cc=\"http://creativecommons.org/ns#\" href=\"http://book.paddlepaddle.org\" property=\"cc:attributionName\" rel=\"cc:attributionURL\"\u003ePaddlePaddle\u003c/a\u003e 创作，采用 \u003ca rel=\"license\" href=\"http://creativecommons.org/licenses/by-nc-sa/4.0/\"\u003e知识共享 署名-非商业性使用-相同方式共享 4.0 国际 许可协议\u003c/a\u003e进行许可。\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.6.0" + } + }, + "nbformat": 4, + "nbformat_minor": 0 +} diff --git a/fit_a_line/README.md b/fit_a_line/README.md index 266c6e91cb4d5249997203cada7ef920d3744386..6ae43bcc591e51b9ff91c61f2c8ae7e6ff407c25 100644 --- a/fit_a_line/README.md +++ b/fit_a_line/README.md @@ -1,7 +1,7 @@ # 线性回归 让我们从经典的线性回归（Linear Regression \[[1](#参考文献)\]）模型开始这份教程。在这一章里，你将使用真实的数据集建立起一个房价预测模型，并且了解到机器学习中的若干重要概念。 -本教程源代码目录在[book/fit_a_line](https://github.com/PaddlePaddle/book/tree/develop/fit_a_line)， 初次使用请参考PaddlePaddle[安装教程](http://www.paddlepaddle.org/doc_cn/build_and_install/index.html)。 +本教程源代码目录在[book/fit_a_line](https://github.com/PaddlePaddle/book/tree/develop/fit_a_line)， 初次使用请参考PaddlePaddle[安装教程](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/getstarted/build_and_install/docker_install_cn.rst)。 ## 背景介绍 给定一个大小为$n$的数据集 ${\{y_{i}, x_{i1}, ..., x_{id}\}}_{i=1}^{n}$，其中$x_{i1}, \ldots, x_{id}$是第$i$个样本$d$个属性上的取值，$y_i$是该样本待预测的目标。线性回归模型假设目标$y_i$可以被属性间的线性组合描述，即 @@ -15,8 +15,8 @@ $$y_i = \omega_1x_{i1} + \omega_2x_{i2} + \ldots + \omega_dx_{id} + b, i=1,\ldo ## 效果展示 我们使用从[UCI Housing Data Set](https://archive.ics.uci.edu/ml/datasets/Housing)获得的波士顿房价数据集进行模型的训练和预测。下面的散点图展示了使用模型对部分房屋价格进行的预测。其中，每个点的横坐标表示同一类房屋真实价格的中位数，纵坐标表示线性回归模型根据特征预测的结果，当二者值完全相等的时候就会落在虚线上。所以模型预测得越准确，则点离虚线越近。

-
- 图1. 预测值 V.S. 真实值 +
+ 图1. 预测值 V.S. 真实值

## 模型概览 @@ -96,8 +96,8 @@ import paddle.v2.dataset.uci_housing as uci_housing - 很多的机器学习技巧/模型（例如L1，L2正则项，向量空间模型-Vector Space Model）都基于这样的假设：所有的属性取值都差不多是以0为均值且取值范围相近的。

-
- 图2. 各维属性的取值范围 +
+ 图2. 各维属性的取值范围

#### 整理训练集与测试集 diff --git a/fit_a_line/index.en.html b/fit_a_line/index.en.html index ab68d50d51198eb2e0dff363a033b76885637f38..32f4146197d87c7feb459ebfcb784ee1f92f1c11 100644 --- a/fit_a_line/index.en.html +++ b/fit_a_line/index.en.html @@ -43,7 +43,7 @@ # Linear Regression Let us begin the tutorial with a classical problem called Linear Regression \[[1](#References)\]. In this chapter, we will train a model from a realistic dataset to predict home prices. Some important concepts in Machine Learning will be covered through this example. -The source code for this tutorial lives on [book/fit_a_line](https://github.com/PaddlePaddle/book/tree/develop/fit_a_line). For instructions on getting started with PaddlePaddle, see [PaddlePaddle installation guide](http://www.paddlepaddle.org/doc_cn/build_and_install/index.html). +The source code for this tutorial lives on [book/fit_a_line](https://github.com/PaddlePaddle/book/tree/develop/fit_a_line). For instructions on getting started with PaddlePaddle, see [PaddlePaddle installation guide](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/getstarted/build_and_install/docker_install_en.rst). ## Problem Setup Suppose we have a dataset of $n$ real estate properties. These real estate properties will be referred to as *homes* in this chapter for clarity. diff --git a/fit_a_line/index.html b/fit_a_line/index.html index 7bcd415d8d7563a228303ed0679b5f3c3599a14d..10185b9d9b1a9c15d748401f449cfcfbf44235b4 100644 --- a/fit_a_line/index.html +++ b/fit_a_line/index.html @@ -43,7 +43,7 @@ # 线性回归 让我们从经典的线性回归（Linear Regression \[[1](#参考文献)\]）模型开始这份教程。在这一章里，你将使用真实的数据集建立起一个房价预测模型，并且了解到机器学习中的若干重要概念。 -本教程源代码目录在[book/fit_a_line](https://github.com/PaddlePaddle/book/tree/develop/fit_a_line)， 初次使用请参考PaddlePaddle[安装教程](http://www.paddlepaddle.org/doc_cn/build_and_install/index.html)。 +本教程源代码目录在[book/fit_a_line](https://github.com/PaddlePaddle/book/tree/develop/fit_a_line)， 初次使用请参考PaddlePaddle[安装教程](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/getstarted/build_and_install/docker_install_cn.rst)。 ## 背景介绍 给定一个大小为$n$的数据集 ${\{y_{i}, x_{i1}, ..., x_{id}\}}_{i=1}^{n}$，其中$x_{i1}, \ldots, x_{id}$是第$i$个样本$d$个属性上的取值，$y_i$是该样本待预测的目标。线性回归模型假设目标$y_i$可以被属性间的线性组合描述，即 @@ -57,8 +57,8 @@ $$y_i = \omega_1x_{i1} + \omega_2x_{i2} + \ldots + \omega_dx_{id} + b, i=1,\ldo ## 效果展示 我们使用从[UCI Housing Data Set](https://archive.ics.uci.edu/ml/datasets/Housing)获得的波士顿房价数据集进行模型的训练和预测。下面的散点图展示了使用模型对部分房屋价格进行的预测。其中，每个点的横坐标表示同一类房屋真实价格的中位数，纵坐标表示线性回归模型根据特征预测的结果，当二者值完全相等的时候就会落在虚线上。所以模型预测得越准确，则点离虚线越近。

-
- 图1. 预测值 V.S. 真实值 +
+ 图1. 预测值 V.S. 真实值

## 模型概览 @@ -138,8 +138,8 @@ import paddle.v2.dataset.uci_housing as uci_housing - 很多的机器学习技巧/模型（例如L1，L2正则项，向量空间模型-Vector Space Model）都基于这样的假设：所有的属性取值都差不多是以0为均值且取值范围相近的。

-
- 图2. 各维属性的取值范围 +
+ 图2. 各维属性的取值范围

#### 整理训练集与测试集 diff --git a/image_classification/README.en.md b/image_classification/README.en.md index 1d7ad33fb5be7f1ea074dbfe143fe4ba45de43f8..57aac0935b94faf11a33c3291d0c6991075cd698 100644 --- a/image_classification/README.en.md +++ b/image_classification/README.en.md @@ -1,7 +1,7 @@ Image Classification ======================= -The source code of this chapter is in [book/image_classification](https://github.com/PaddlePaddle/book/tree/develop/image_classification). For the first-time users, please refer to PaddlePaddle [Installation Tutorial](http://www.paddlepaddle.org/doc_cn/build_and_install/index.html) for installation instructions. +The source code of this chapter is in [book/image_classification](https://github.com/PaddlePaddle/book/tree/develop/image_classification). For the first-time users, please refer to PaddlePaddle [Installation Tutorial](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/getstarted/build_and_install/docker_install_en.rst) for installation instructions. ## Background diff --git a/image_classification/README.md b/image_classification/README.md index 7ab465b6971958ffcd4db653251ceca0a4b98e7b..dd09f7994fa06739601aa7efbfb4a4699d7cea52 100644 --- a/image_classification/README.md +++ b/image_classification/README.md @@ -1,7 +1,7 @@ 图像分类 ======= -本教程源代码目录在[book/image_classification](https://github.com/PaddlePaddle/book/tree/develop/image_classification)， 初次使用请参考PaddlePaddle[安装教程](http://www.paddlepaddle.org/doc_cn/build_and_install/index.html)。 +本教程源代码目录在[book/image_classification](https://github.com/PaddlePaddle/book/tree/develop/image_classification)， 初次使用请参考PaddlePaddle[安装教程](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/getstarted/build_and_install/docker_install_cn.rst)。 ## 背景介绍 @@ -173,24 +173,24 @@ paddle.init(use_gpu=False, trainer_count=1) 1. 定义数据输入及其维度 - 网络输入定义为 `data_layer` (数据层)，在图像分类中即为图像像素信息。CIFRAR10是RGB 3通道32x32大小的彩色图，因此输入数据大小为3072(3x32x32)，类别大小为10，即10分类。 + 网络输入定义为 `data_layer` (数据层)，在图像分类中即为图像像素信息。CIFRAR10是RGB 3通道32x32大小的彩色图，因此输入数据大小为3072(3x32x32)，类别大小为10，即10分类。 - ```python + ```python datadim = 3 * 32 * 32 classdim = 10 image = paddle.layer.data( name="image", type=paddle.data_type.dense_vector(datadim)) - ``` + ``` 2. 定义VGG网络核心模块 - ```python - net = vgg_bn_drop(image) - ``` - VGG核心模块的输入是数据层，`vgg_bn_drop` 定义了16层VGG结构，每层卷积后面引入BN层和Dropout层，详细的定义如下： + ```python + net = vgg_bn_drop(image) + ``` + VGG核心模块的输入是数据层，`vgg_bn_drop` 定义了16层VGG结构，每层卷积后面引入BN层和Dropout层，详细的定义如下： - ```python + ```python def vgg_bn_drop(input): def conv_block(ipt, num_filter, groups, dropouts, num_channels=None): return paddle.networks.img_conv_group( @@ -219,33 +219,33 @@ paddle.init(use_gpu=False, trainer_count=1) layer_attr=paddle.attr.Extra(drop_rate=0.5)) fc2 = paddle.layer.fc(input=bn, size=512, act=paddle.activation.Linear()) return fc2 - ``` + ``` - 2.1. 首先定义了一组卷积网络，即conv_block。卷积核大小为3x3，池化窗口大小为2x2，窗口滑动大小为2，groups决定每组VGG模块是几次连续的卷积操作，dropouts指定Dropout操作的概率。所使用的`img_conv_group`是在`paddle.networks`中预定义的模块，由若干组 `Conv->BN->ReLu->Dropout` 和 一组 `Pooling` 组成， + 2.1. 首先定义了一组卷积网络，即conv_block。卷积核大小为3x3，池化窗口大小为2x2，窗口滑动大小为2，groups决定每组VGG模块是几次连续的卷积操作，dropouts指定Dropout操作的概率。所使用的`img_conv_group`是在`paddle.networks`中预定义的模块，由若干组 `Conv->BN->ReLu->Dropout` 和 一组 `Pooling` 组成， - 2.2. 五组卷积操作，即 5个conv_block。 第一、二组采用两次连续的卷积操作。第三、四、五组采用三次连续的卷积操作。每组最后一个卷积后面Dropout概率为0，即不使用Dropout操作。 + 2.2. 五组卷积操作，即 5个conv_block。 第一、二组采用两次连续的卷积操作。第三、四、五组采用三次连续的卷积操作。每组最后一个卷积后面Dropout概率为0，即不使用Dropout操作。 - 2.3. 最后接两层512维的全连接。 + 2.3. 最后接两层512维的全连接。 3. 定义分类器 - 通过上面VGG网络提取高层特征，然后经过全连接层映射到类别维度大小的向量，再通过Softmax归一化得到每个类别的概率，也可称作分类器。 + 通过上面VGG网络提取高层特征，然后经过全连接层映射到类别维度大小的向量，再通过Softmax归一化得到每个类别的概率，也可称作分类器。 - ```python + ```python out = paddle.layer.fc(input=net, size=classdim, act=paddle.activation.Softmax()) - ``` + ``` 4. 定义损失函数和网络输出 - 在有监督训练中需要输入图像对应的类别信息，同样通过`paddle.layer.data`来定义。训练中采用多类交叉熵作为损失函数，并作为网络的输出，预测阶段定义网络的输出为分类器得到的概率信息。 + 在有监督训练中需要输入图像对应的类别信息，同样通过`paddle.layer.data`来定义。训练中采用多类交叉熵作为损失函数，并作为网络的输出，预测阶段定义网络的输出为分类器得到的概率信息。 - ```python + ```python lbl = paddle.layer.data( name="label", type=paddle.data_type.integer_value(classdim)) cost = paddle.layer.classification_cost(input=out, label=lbl) - ``` + ``` ### ResNet diff --git a/image_classification/deprecated/README.md b/image_classification/deprecated/README.md index a82a32c8a84cedd5da05e2a66f791819f09f65cb..84125f4ded5e46dfbe15e536ee8de51099469c2b 100644 --- a/image_classification/deprecated/README.md +++ b/image_classification/deprecated/README.md @@ -1,7 +1,7 @@ 图像分类 ======= -本教程源代码目录在[book/image_classification](https://github.com/PaddlePaddle/book/tree/develop/image_classification)， 初次使用请参考PaddlePaddle[安装教程](http://www.paddlepaddle.org/doc_cn/build_and_install/index.html)。 +本教程源代码目录在[book/image_classification](https://github.com/PaddlePaddle/book/tree/develop/image_classification)， 初次使用请参考PaddlePaddle[安装教程](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/getstarted/build_and_install/docker_install_cn.rst)。 ## 背景介绍 @@ -244,77 +244,77 @@ $$ lr = lr_{0} * a^ {\lfloor \frac{n}{ b}\rfloor} $$ 1. 定义数据输入及其维度 - 网络输入定义为 `data_layer` (数据层)，在图像分类中即为图像像素信息。CIFRAR10是RGB 3通道32x32大小的彩色图，因此输入数据大小为3072(3x32x32)，类别大小为10，即10分类。 + 网络输入定义为 `data_layer` (数据层)，在图像分类中即为图像像素信息。CIFRAR10是RGB 3通道32x32大小的彩色图，因此输入数据大小为3072(3x32x32)，类别大小为10，即10分类。 - ```python - datadim = 3 * 32 * 32 - classdim = 10 - data = data_layer(name='image', size=datadim) - ``` + ```python + datadim = 3 * 32 * 32 + classdim = 10 + data = data_layer(name='image', size=datadim) + ``` 2. 定义VGG网络核心模块 - ```python - net = vgg_bn_drop(data) - ``` - VGG核心模块的输入是数据层，`vgg_bn_drop` 定义了16层VGG结构，每层卷积后面引入BN层和Dropout层，详细的定义如下： - - ```python - def vgg_bn_drop(input, num_channels): - def conv_block(ipt, num_filter, groups, dropouts, num_channels_=None): - return img_conv_group( - input=ipt, - num_channels=num_channels_, - pool_size=2, - pool_stride=2, - conv_num_filter=[num_filter] * groups, - conv_filter_size=3, - conv_act=ReluActivation(), - conv_with_batchnorm=True, - conv_batchnorm_drop_rate=dropouts, - pool_type=MaxPooling()) - - conv1 = conv_block(input, 64, 2, [0.3, 0], 3) - conv2 = conv_block(conv1, 128, 2, [0.4, 0]) - conv3 = conv_block(conv2, 256, 3, [0.4, 0.4, 0]) - conv4 = conv_block(conv3, 512, 3, [0.4, 0.4, 0]) - conv5 = conv_block(conv4, 512, 3, [0.4, 0.4, 0]) - - drop = dropout_layer(input=conv5, dropout_rate=0.5) - fc1 = fc_layer(input=drop, size=512, act=LinearActivation()) - bn = batch_norm_layer( - input=fc1, act=ReluActivation(), layer_attr=ExtraAttr(drop_rate=0.5)) - fc2 = fc_layer(input=bn, size=512, act=LinearActivation()) - return fc2 - - ``` - - 2.1. 首先定义了一组卷积网络，即conv_block。卷积核大小为3x3，池化窗口大小为2x2，窗口滑动大小为2，groups决定每组VGG模块是几次连续的卷积操作，dropouts指定Dropout操作的概率。所使用的`img_conv_group`是在`paddle.trainer_config_helpers`中预定义的模块，由若干组 `Conv->BN->ReLu->Dropout` 和 一组 `Pooling` 组成， - - 2.2. 五组卷积操作，即 5个conv_block。 第一、二组采用两次连续的卷积操作。第三、四、五组采用三次连续的卷积操作。每组最后一个卷积后面Dropout概率为0，即不使用Dropout操作。 - - 2.3. 最后接两层512维的全连接。 + ```python + net = vgg_bn_drop(data) + ``` + VGG核心模块的输入是数据层，`vgg_bn_drop` 定义了16层VGG结构，每层卷积后面引入BN层和Dropout层，详细的定义如下： + + ```python + def vgg_bn_drop(input, num_channels): + def conv_block(ipt, num_filter, groups, dropouts, num_channels_=None): + return img_conv_group( + input=ipt, + num_channels=num_channels_, + pool_size=2, + pool_stride=2, + conv_num_filter=[num_filter] * groups, + conv_filter_size=3, + conv_act=ReluActivation(), + conv_with_batchnorm=True, + conv_batchnorm_drop_rate=dropouts, + pool_type=MaxPooling()) + + conv1 = conv_block(input, 64, 2, [0.3, 0], 3) + conv2 = conv_block(conv1, 128, 2, [0.4, 0]) + conv3 = conv_block(conv2, 256, 3, [0.4, 0.4, 0]) + conv4 = conv_block(conv3, 512, 3, [0.4, 0.4, 0]) + conv5 = conv_block(conv4, 512, 3, [0.4, 0.4, 0]) + + drop = dropout_layer(input=conv5, dropout_rate=0.5) + fc1 = fc_layer(input=drop, size=512, act=LinearActivation()) + bn = batch_norm_layer( + input=fc1, act=ReluActivation(), layer_attr=ExtraAttr(drop_rate=0.5)) + fc2 = fc_layer(input=bn, size=512, act=LinearActivation()) + return fc2 + + ``` + + 2.1. 首先定义了一组卷积网络，即conv_block。卷积核大小为3x3，池化窗口大小为2x2，窗口滑动大小为2，groups决定每组VGG模块是几次连续的卷积操作，dropouts指定Dropout操作的概率。所使用的`img_conv_group`是在`paddle.trainer_config_helpers`中预定义的模块，由若干组 `Conv->BN->ReLu->Dropout` 和 一组 `Pooling` 组成， + + 2.2. 五组卷积操作，即 5个conv_block。 第一、二组采用两次连续的卷积操作。第三、四、五组采用三次连续的卷积操作。每组最后一个卷积后面Dropout概率为0，即不使用Dropout操作。 + + 2.3. 最后接两层512维的全连接。 3. 定义分类器 - 通过上面VGG网络提取高层特征，然后经过全连接层映射到类别维度大小的向量，再通过Softmax归一化得到每个类别的概率，也可称作分类器。 + 通过上面VGG网络提取高层特征，然后经过全连接层映射到类别维度大小的向量，再通过Softmax归一化得到每个类别的概率，也可称作分类器。 - ```python - out = fc_layer(input=net, size=class_num, act=SoftmaxActivation()) - ``` + ```python + out = fc_layer(input=net, size=class_num, act=SoftmaxActivation()) + ``` 4. 定义损失函数和网络输出 - 在有监督训练中需要输入图像对应的类别信息，同样通过`data_layer`来定义。训练中采用多类交叉熵作为损失函数，并作为网络的输出，预测阶段定义网络的输出为分类器得到的概率信息。 + 在有监督训练中需要输入图像对应的类别信息，同样通过`data_layer`来定义。训练中采用多类交叉熵作为损失函数，并作为网络的输出，预测阶段定义网络的输出为分类器得到的概率信息。 - ```python - if not is_predict: - lbl = data_layer(name="label", size=class_num) - cost = classification_cost(input=out, label=lbl) - outputs(cost) - else: - outputs(out) - ``` + ```python + if not is_predict: + lbl = data_layer(name="label", size=class_num) + cost = classification_cost(input=out, label=lbl) + outputs(cost) + else: + outputs(out) + ``` ### ResNet diff --git a/image_classification/index.en.html b/image_classification/index.en.html index 929f3365c9be2d773766261470de5cb94f7ebb1d..457b54851ef970f228662ff5a7e52485c64d8647 100644 --- a/image_classification/index.en.html +++ b/image_classification/index.en.html @@ -43,7 +43,7 @@ Image Classification ======================= -The source code of this chapter is in [book/image_classification](https://github.com/PaddlePaddle/book/tree/develop/image_classification). For the first-time users, please refer to PaddlePaddle [Installation Tutorial](http://www.paddlepaddle.org/doc_cn/build_and_install/index.html) for installation instructions. +The source code of this chapter is in [book/image_classification](https://github.com/PaddlePaddle/book/tree/develop/image_classification). For the first-time users, please refer to PaddlePaddle [Installation Tutorial](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/getstarted/build_and_install/docker_install_en.rst) for installation instructions. ## Background diff --git a/image_classification/index.html b/image_classification/index.html index 4d799536b2e13042294b40a81e0cdbcb52b1cda2..21446e9413af4cddf98fed5f392c5aaa406479be 100644 --- a/image_classification/index.html +++ b/image_classification/index.html @@ -43,7 +43,7 @@ 图像分类 ======= -本教程源代码目录在[book/image_classification](https://github.com/PaddlePaddle/book/tree/develop/image_classification)， 初次使用请参考PaddlePaddle[安装教程](http://www.paddlepaddle.org/doc_cn/build_and_install/index.html)。 +本教程源代码目录在[book/image_classification](https://github.com/PaddlePaddle/book/tree/develop/image_classification)， 初次使用请参考PaddlePaddle[安装教程](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/getstarted/build_and_install/docker_install_cn.rst)。 ## 背景介绍 @@ -215,24 +215,24 @@ paddle.init(use_gpu=False, trainer_count=1) 1. 定义数据输入及其维度 - 网络输入定义为 `data_layer` (数据层)，在图像分类中即为图像像素信息。CIFRAR10是RGB 3通道32x32大小的彩色图，因此输入数据大小为3072(3x32x32)，类别大小为10，即10分类。 + 网络输入定义为 `data_layer` (数据层)，在图像分类中即为图像像素信息。CIFRAR10是RGB 3通道32x32大小的彩色图，因此输入数据大小为3072(3x32x32)，类别大小为10，即10分类。 - ```python + ```python datadim = 3 * 32 * 32 classdim = 10 image = paddle.layer.data( name="image", type=paddle.data_type.dense_vector(datadim)) - ``` + ``` 2. 定义VGG网络核心模块 - ```python - net = vgg_bn_drop(image) - ``` - VGG核心模块的输入是数据层，`vgg_bn_drop` 定义了16层VGG结构，每层卷积后面引入BN层和Dropout层，详细的定义如下： + ```python + net = vgg_bn_drop(image) + ``` + VGG核心模块的输入是数据层，`vgg_bn_drop` 定义了16层VGG结构，每层卷积后面引入BN层和Dropout层，详细的定义如下： - ```python + ```python def vgg_bn_drop(input): def conv_block(ipt, num_filter, groups, dropouts, num_channels=None): return paddle.networks.img_conv_group( @@ -261,33 +261,33 @@ paddle.init(use_gpu=False, trainer_count=1) layer_attr=paddle.attr.Extra(drop_rate=0.5)) fc2 = paddle.layer.fc(input=bn, size=512, act=paddle.activation.Linear()) return fc2 - ``` + ``` - 2.1. 首先定义了一组卷积网络，即conv_block。卷积核大小为3x3，池化窗口大小为2x2，窗口滑动大小为2，groups决定每组VGG模块是几次连续的卷积操作，dropouts指定Dropout操作的概率。所使用的`img_conv_group`是在`paddle.networks`中预定义的模块，由若干组 `Conv->BN->ReLu->Dropout` 和 一组 `Pooling` 组成， + 2.1. 首先定义了一组卷积网络，即conv_block。卷积核大小为3x3，池化窗口大小为2x2，窗口滑动大小为2，groups决定每组VGG模块是几次连续的卷积操作，dropouts指定Dropout操作的概率。所使用的`img_conv_group`是在`paddle.networks`中预定义的模块，由若干组 `Conv->BN->ReLu->Dropout` 和 一组 `Pooling` 组成， - 2.2. 五组卷积操作，即 5个conv_block。 第一、二组采用两次连续的卷积操作。第三、四、五组采用三次连续的卷积操作。每组最后一个卷积后面Dropout概率为0，即不使用Dropout操作。 + 2.2. 五组卷积操作，即 5个conv_block。 第一、二组采用两次连续的卷积操作。第三、四、五组采用三次连续的卷积操作。每组最后一个卷积后面Dropout概率为0，即不使用Dropout操作。 - 2.3. 最后接两层512维的全连接。 + 2.3. 最后接两层512维的全连接。 3. 定义分类器 - 通过上面VGG网络提取高层特征，然后经过全连接层映射到类别维度大小的向量，再通过Softmax归一化得到每个类别的概率，也可称作分类器。 + 通过上面VGG网络提取高层特征，然后经过全连接层映射到类别维度大小的向量，再通过Softmax归一化得到每个类别的概率，也可称作分类器。 - ```python + ```python out = paddle.layer.fc(input=net, size=classdim, act=paddle.activation.Softmax()) - ``` + ``` 4. 定义损失函数和网络输出 - 在有监督训练中需要输入图像对应的类别信息，同样通过`paddle.layer.data`来定义。训练中采用多类交叉熵作为损失函数，并作为网络的输出，预测阶段定义网络的输出为分类器得到的概率信息。 + 在有监督训练中需要输入图像对应的类别信息，同样通过`paddle.layer.data`来定义。训练中采用多类交叉熵作为损失函数，并作为网络的输出，预测阶段定义网络的输出为分类器得到的概率信息。 - ```python + ```python lbl = paddle.layer.data( name="label", type=paddle.data_type.integer_value(classdim)) cost = paddle.layer.classification_cost(input=out, label=lbl) - ``` + ``` ### ResNet diff --git a/label_semantic_roles/README.en.md b/label_semantic_roles/README.en.md index 4edc9b3f8996ca0f9f3ac81e51a48dbc01334bf5..6fd83b71891fbcbc5fc736b47fbaf1be61a22f60 100644 --- a/label_semantic_roles/README.en.md +++ b/label_semantic_roles/README.en.md @@ -2,6 +2,8 @@ Source code of this chapter is in [book/label_semantic_roles](https://github.com/PaddlePaddle/book/tree/develop/label_semantic_roles). +For instructions on getting started with PaddlePaddle, see [PaddlePaddle installation guide](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/getstarted/build_and_install/docker_install_en.rst). + ## Background Natural Language Analysis contains three components: Lexical Analysis, Syntactic Analysis, and Semantic Analysis. Semantic Role Labelling (SRL) is one way for Shallow Semantic Analysis. A predicate of a sentence is a property that a subject possesses or is characterized, such as what it does, what it is or how it is, which mostly corresponds to the core of an event. The noun associated with a predicate is called Argument. Semantic roles express the abstract roles that arguments of a predicate can take in the event, such as Agent, Patient, Theme, Experiencer, Beneficiary, Instrument, Location, Goal and Source, etc. diff --git a/label_semantic_roles/README.md b/label_semantic_roles/README.md index e0413c3f6d17e3ee173dad86e4863e710dfe08ba..63e8a615be887ac835530c22690cf484b70feec4 100644 --- a/label_semantic_roles/README.md +++ b/label_semantic_roles/README.md @@ -1,6 +1,6 @@ # 语义角色标注 -本教程源代码目录在[book/label_semantic_roles](https://github.com/PaddlePaddle/book/tree/develop/label_semantic_roles)， 初次使用请参考PaddlePaddle[安装教程](http://www.paddlepaddle.org/doc_cn/build_and_install/index.html)。 +本教程源代码目录在[book/label_semantic_roles](https://github.com/PaddlePaddle/book/tree/develop/label_semantic_roles)， 初次使用请参考PaddlePaddle[安装教程](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/getstarted/build_and_install/docker_install_cn.rst)。 ## 背景介绍 diff --git a/label_semantic_roles/index.en.html b/label_semantic_roles/index.en.html index 9fbeb75d5c3a99282b27f34286af83b6438c42e5..1d2bfa167af73ad2458d764da18c74d67ae1aa58 100644 --- a/label_semantic_roles/index.en.html +++ b/label_semantic_roles/index.en.html @@ -44,6 +44,8 @@ Source code of this chapter is in [book/label_semantic_roles](https://github.com/PaddlePaddle/book/tree/develop/label_semantic_roles). +For instructions on getting started with PaddlePaddle, see [PaddlePaddle installation guide](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/getstarted/build_and_install/docker_install_en.rst). + ## Background Natural Language Analysis contains three components: Lexical Analysis, Syntactic Analysis, and Semantic Analysis. Semantic Role Labelling (SRL) is one way for Shallow Semantic Analysis. A predicate of a sentence is a property that a subject possesses or is characterized, such as what it does, what it is or how it is, which mostly corresponds to the core of an event. The noun associated with a predicate is called Argument. Semantic roles express the abstract roles that arguments of a predicate can take in the event, such as Agent, Patient, Theme, Experiencer, Beneficiary, Instrument, Location, Goal and Source, etc. diff --git a/label_semantic_roles/index.html b/label_semantic_roles/index.html index ed6f21dcbcffcc71a06a12963366e4e8b69669d4..ce683978f880135c2b8ad70c5bcad29e9f525a87 100644 --- a/label_semantic_roles/index.html +++ b/label_semantic_roles/index.html @@ -42,7 +42,7 @@