Merge branch 'develop' of https://github.com/PaddlePaddle/book into feature/refine_frontpage

.pre-commit-config.yaml

--   repo: https://github.com/reyoung/mirrors-yapf.git
-    sha: v0.13.2
+-   repo: https://github.com/pre-commit/mirrors-yapf.git
+    sha: v0.16.0
     hooks:
-    - id: yapf
-      files: (.*\.(py|bzl)|BUILD|.*\.BUILD|WORKSPACE)$  # Bazel BUILD files follow Python syntax.
+    -   id: yapf
+        files: \.py$
 -   repo: https://github.com/pre-commit/pre-commit-hooks
-    sha: v0.7.1
+    sha: a11d9314b22d8f8c7556443875b731ef05965464
     hooks:
     -   id: check-merge-conflict
     -   id: check-symlinks
@@ -24,16 +24,17 @@
         files: \.md$
     -   id: remove-tabs
         files: \.md$
-- repo: git://github.com/reyoung/pre-commit-hooks-jinja-compile.git
-  sha: 85ad800cbc9c60a64230d60971aa9576fd57e508
-  hooks:
-  -   id: convert-jinja2-into-html
-- repo: local
-  hooks:
-    -  id: convert-markdown-into-html
-       name: convert-markdown-into-html
-       description: "Convert README.md into index.html and README.en.md into index.en.html"
-       entry: python pre-commit-hooks/convert_markdown_into_html.py
-       language: system
-       files: .+README(\.en)?\.md$
-
+-   repo: local
+    hooks:
+    -   id: convert-markdown-into-html
+        name: convert-markdown-into-html
+        description: Convert README.md into index.html and README.en.md into index.en.html
+        entry: python pre-commit-hooks/convert_markdown_into_html.py
+        language: system
+        files: .+README(\.en)?\.md$
+    -   id: convert-markdown-into-ipynb
+        name: convert-markdown-into-ipynb
+        description: Convert README.md into README.ipynb and README.en.md into README.en.ipynb
+        entry: ./pre-commit-hooks/convert_markdown_into_ipynb.sh
+        language: system
+        files: .+README(\.en)?\.md$

.travis.yml

@@ -14,8 +14,10 @@ addons:
       - python
       - python-pip
       - python2.7-dev
+      - golang
 before_install:
   -  pip install -U virtualenv pre-commit pip
+  -  GOPATH=/tmp/go go get -u github.com/wangkuiyi/ipynb/markdown-to-ipynb
 script:
   - travis/precommit.sh
 notifications:

README.en.md

+# Deep Learning with PaddlePaddle
+
+1. [Fit a Line](http://book.paddlepaddle.org/fit_a_line/index.en.html)
+1. [Recognize Digits](http://book.paddlepaddle.org/recognize_digits/index.en.html)
+1. [Image Classification](http://book.paddlepaddle.org/image_classification/index.en.html)
+1. [Word to Vector](http://book.paddlepaddle.org/word2vec/index.en.html)
+1. [Understand Sentiment](http://book.paddlepaddle.org/understand_sentiment/index.en.html)
+1. [Label Semantic Roles](http://book.paddlepaddle.org/label_semantic_roles/index.en.html)
+1. [Machine Translation](http://book.paddlepaddle.org/machine_translation/index.en.html)
+1. [Recommender System](http://book.paddlepaddle.org/recommender_system/index.en.html)
+
+This tutorial is contributed by <a xmlns:cc="http://creativecommons.org/ns#" href="http://book.paddlepaddle.org" property="cc:attributionName" rel="cc:attributionURL">PaddlePaddle</a>, and licensed under a <a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/4.0/">Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License</a>.

README.md

 # 深度学习入门
 
-1. [新手入门](fit_a_line/) [[html](http://book.paddlepaddle.org/fit_a_line)]
-1. [识别数字](recognize_digits/) [[html](http://book.paddlepaddle.org/recognize_digits)]
-1. [图像分类](image_classification/) [[html](http://book.paddlepaddle.org/image_classification)]
-1. [词向量](word2vec/) [[html](http://book.paddlepaddle.org/word2vec)]
-1. [情感分析](understand_sentiment/) [[html](http://book.paddlepaddle.org/understand_sentiment)]
-1. [语义角色标注](label_semantic_roles/) [[html](http://book.paddlepaddle.org/label_semantic_roles)]
-1. [机器翻译](machine_translation/) [[html](http://book.paddlepaddle.org/machine_translation)]
-1. [个性化推荐](recommender_system/) [[html](http://book.paddlepaddle.org/recommender_system)]
+1. [新手入门](http://book.paddlepaddle.org/fit_a_line)
+1. [识别数字](http://book.paddlepaddle.org/recognize_digits)
+1. [图像分类](http://book.paddlepaddle.org/image_classification)
+1. [词向量](http://book.paddlepaddle.org/word2vec)
+1. [情感分析](http://book.paddlepaddle.org/understand_sentiment)
+1. [语义角色标注](http://book.paddlepaddle.org/label_semantic_roles)
+1. [机器翻译](http://book.paddlepaddle.org/machine_translation)
+1. [个性化推荐](http://book.paddlepaddle.org/recommender_system)
 
 
-
-# Deep Learning Introduction
-
-1. [Fit a Line](fit_a_line/) [[html](http://book.paddlepaddle.org/fit_a_line/index.en.html)]
-1. [Recognize Digits](recognize_digits/) [[html](http://book.paddlepaddle.org/recognize_digits/index.en.html)]
-1. [Image Classification](image_classification/) [[html](http://book.paddlepaddle.org/image_classification/index.en.html)]
-1. [Word to Vector](word2vec/) [[html](http://book.paddlepaddle.org/word2vec/index.en.html)]
-1. [Understand Sentiment](understand_sentiment/) [[html](http://book.paddlepaddle.org/understand_sentiment/index.en.html)]
-1. [Label Semantic Roles](label_semantic_roles/) [[html](http://book.paddlepaddle.org/label_semantic_roles/index.en.html)]
-1. [Machine Translation](machine_translation/) [[html](http://book.paddlepaddle.org/machine_translation/index.en.html)]
-1. [Recommender System](recommender_system/) [[html](http://book.paddlepaddle.org/recommender_system/index.en.html)]
-
-<br/>
 <a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/4.0/"><img alt="知识共享许可协议" style="border-width:0" src="https://i.creativecommons.org/l/by-nc-sa/4.0/88x31.png" /></a><br /><span xmlns:dct="http://purl.org/dc/terms/" href="http://purl.org/dc/dcmitype/Text" property="dct:title" rel="dct:type">本教程</span> 由 <a xmlns:cc="http://creativecommons.org/ns#" href="http://book.paddlepaddle.org" property="cc:attributionName" rel="cc:attributionURL">PaddlePaddle</a> 创作，采用 <a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/4.0/">知识共享 署名-非商业性使用-相同方式共享 4.0 国际 许可协议</a>进行许可。
-This tutorial is contributed by <a xmlns:cc="http://creativecommons.org/ns#" href="http://book.paddlepaddle.org" property="cc:attributionName" rel="cc:attributionURL">PaddlePaddle</a>, and licensed under a <a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/4.0/">Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License</a>.

fit_a_line/README.en.ipynb

+{
+  "cells": [
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "# Linear Regression\n",
+        "Let us begin the tutorial with a classical problem called Linear Regression \\[[1](#References)\\]. In this chapter, we will train a model from a realistic dataset to predict home prices. Some important concepts in Machine Learning will be covered through this example.\n",
+        "\n",
+        "The source code for this tutorial lives on [book/fit_a_line](https://github.com/PaddlePaddle/book/tree/develop/fit_a_line). For instructions on getting started with PaddlePaddle, see [PaddlePaddle installation guide](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/getstarted/build_and_install/docker_install_en.rst).\n",
+        "\n",
+        "## Problem Setup\n",
+        "Suppose we have a dataset of $n$ real estate properties. These real estate properties will be referred to as *homes* in this chapter for clarity.\n",
+        "\n",
+        "Each home is associated with $d$ attributes. The attributes describe characteristics such the number of rooms in the home, the number of schools or hospitals in the neighborhood, and the traffic condition nearby.\n",
+        "\n",
+        "In our problem setup, the attribute $x_{i,j}$ denotes the $j$th characteristic of the $i$th home. In addition, $y_i$ denotes the price of the $i$th home. Our task is to predict $y_i$ given a set of attributes $\\{x_{i,1}, ..., x_{i,d}\\}$. We assume that the price of a home is a linear combination of all of its attributes, namely,\n",
+        "\n",
+        "$$y_i = \\omega_1x_{i,1} + \\omega_2x_{i,2} + \\ldots + \\omega_dx_{i,d} + b,  i=1,\\ldots,n$$\n",
+        "\n",
+        "where $\\vec{\\omega}$ and $b$ are the model parameters we want to estimate. Once they are learned, we will be able to predict the price of a home, given the attributes associated with it. We call this model **Linear Regression**. In other words, we want to regress a value against several values linearly. In practice, a linear model is often too simplistic to capture the real relationships between the variables. Yet, because Linear Regression is easy to train and analyze, it has been applied to a large number of real problems. As a result, it is an important topic in many classic Statistical Learning and Machine Learning textbooks \\[[2,3,4](#References)\\].\n",
+        "\n",
+        "## Results Demonstration\n",
+        "We first show the result of our model. The dataset [UCI Housing Data Set](https://archive.ics.uci.edu/ml/datasets/Housing) is used to train a linear model to predict the home prices in Boston. The figure below shows the predictions the model makes for some home prices. The $X$-axis represents the median value of the prices of simlilar homes within a bin, while the $Y$-axis represents the home value our linear model predicts. The dotted line represents points where $X=Y$. When reading the diagram, the more precise the model predicts, the closer the point is to the dotted line.\n",
+        "\u003cp align=\"center\"\u003e\n",
+        "    \u003cimg src = \"image/predictions_en.png\" width=400\u003e\u003cbr/\u003e\n",
+        "    Figure 1. Predicted Value V.S. Actual Value\n",
+        "\u003c/p\u003e\n",
+        "\n",
+        "## Model Overview\n",
+        "\n",
+        "### Model Definition\n",
+        "\n",
+        "In the UCI Housing Data Set, there are 13 home attributes $\\{x_{i,j}\\}$ that are related to the median home price $y_i$, which we aim to predict. Thus, our model can be written as:\n",
+        "\n",
+        "$$\\hat{Y} = \\omega_1X_{1} + \\omega_2X_{2} + \\ldots + \\omega_{13}X_{13} + b$$\n",
+        "\n",
+        "where $\\hat{Y}$ is the predicted value used to differentiate from actual value $Y$. The model learns parameters $\\omega_1, \\ldots, \\omega_{13}, b$, where the entries of $\\vec{\\omega}$ are **weights** and $b$ is **bias**.\n",
+        "\n",
+        "Now we need an objective to optimize, so that the learned parameters can make $\\hat{Y}$ as close to $Y$ as possible. Let's refer to the concept of [Loss Function (Cost Function)](https://en.wikipedia.org/wiki/Loss_function). A loss function must output a non-negative value, given any pair of the actual value $y_i$ and the predicted value $\\hat{y_i}$. This value reflects the magnitutude of the model error.\n",
+        "\n",
+        "For Linear Regression, the most common loss function is [Mean Square Error (MSE)](https://en.wikipedia.org/wiki/Mean_squared_error) which has the following form:\n",
+        "\n",
+        "$$MSE=\\frac{1}{n}\\sum_{i=1}^{n}{(\\hat{Y_i}-Y_i)}^2$$\n",
+        "\n",
+        "That is, for a dataset of size $n$, MSE is the average value of the the prediction sqaure errors.\n",
+        "\n",
+        "### Training\n",
+        "\n",
+        "After setting up our model, there are several major steps to go through to train it:\n",
+        "1. Initialize the parameters including the weights $\\vec{\\omega}$ and the bias $b$. For example, we can set their mean values as $0$s, and their standard deviations as $1$s.\n",
+        "2. Feedforward. Evaluate the network output and compute the corresponding loss.\n",
+        "3. [Backpropagate](https://en.wikipedia.org/wiki/Backpropagation) the errors. The errors will be propagated from the output layer back to the input layer, during which the model parameters will be updated with the corresponding errors.\n",
+        "4. Repeat steps 2~3, until the loss is below a predefined threshold or the maximum number of repeats is reached.\n",
+        "\n",
+        "## Dataset\n",
+        "\n",
+        "### Python Dataset Modules\n",
+        "\n",
+        "Our program starts with importing necessary packages:\n",
+        "\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "metadata": {
+        "editable": true
+      },
+      "source": [
+        "import paddle.v2 as paddle\n",
+        "import paddle.v2.dataset.uci_housing as uci_housing\n"
+      ],
+      "outputs": [
+        {
+          "name": "stdout",
+          "output_type": "stream",
+          "text": [
+            "\n"
+          ]
+        }
+      ],
+      "execution_count": 1
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "\n",
+        "We encapsulated the [UCI Housing Data Set](https://archive.ics.uci.edu/ml/datasets/Housing) in our Python module `uci_housing`.  This module can\n",
+        "\n",
+        "1. download the dataset to `~/.cache/paddle/dataset/uci_housing/housing.data`, if not yet, and\n",
+        "2.  [preprocesses](#preprocessing) the dataset.\n",
+        "\n",
+        "### An Introduction of the Dataset\n",
+        "\n",
+        "The UCI housing dataset has 506 instances. Each instance describes the attributes of a house in surburban Boston.  The attributes are explained below:\n",
+        "\n",
+        "| Attribute Name | Characteristic | Data Type |\n",
+        "| ------| ------ | ------ |\n",
+        "| CRIM | per capita crime rate by town | Continuous|\n",
+        "| ZN | proportion of residential land zoned for lots over 25,000 sq.ft. | Continuous |\n",
+        "| INDUS | proportion of non-retail business acres per town | Continuous |\n",
+        "| CHAS | Charles River dummy variable | Discrete, 1 if tract bounds river; 0 otherwise|\n",
+        "| NOX | nitric oxides concentration (parts per 10 million) | Continuous |\n",
+        "| RM | average number of rooms per dwelling | Continuous |\n",
+        "| AGE | proportion of owner-occupied units built prior to 1940 | Continuous |\n",
+        "| DIS | weighted distances to five Boston employment centres | Continuous |\n",
+        "| RAD | index of accessibility to radial highways | Continuous |\n",
+        "| TAX | full-value property-tax rate per $10,000 | Continuous |\n",
+        "| PTRATIO | pupil-teacher ratio by town | Continuous |\n",
+        "| B | 1000(Bk - 0.63)^2 where Bk is the proportion of blacks by town | Continuous |\n",
+        "| LSTAT | % lower status of the population | Continuous |\n",
+        "| MEDV | Median value of owner-occupied homes in $1000's | Continuous |\n",
+        "\n",
+        "The last entry is the median home price.\n",
+        "\n",
+        "### Preprocessing\n",
+        "#### Continuous and Discrete Data\n",
+        "We define a feature vector of length 13 for each home, where each entry corresponds to an attribute. Our first observation is that, among the 13 dimensions, there are 12 continuous dimensions and 1 discrete dimension.\n",
+        "\n",
+        "Note that although a discrete value is also written as numeric values such as 0, 1, or 2, its meaning differs from a continuous value drastically.  The linear difference between two discrete values has no meaning. For example, suppose $0$, $1$, and $2$ are used to represent colors *Red*, *Green*, and *Blue* respectively. Judging from the numeric representation of these colors, *Red* differs more from *Blue* than it does from *Green*. Yet in actuality, it is not true that extent to which the color *Blue* is different from *Red* is greater than the extent to which *Green* is different from *Red*. Therefore, when handling a discrete feature that has $d$ possible values, we usually convert it to $d$ new features where each feature takes a binary value, $0$ or $1$, indicating whether the original value is absent or present. Alternatively, the discrete features can be mapped onto a continuous multi-dimensional vector through an embedding table. For our problem here, because CHAS itself is a binary discrete value, we do not need to do any preprocessing.\n",
+        "\n",
+        "#### Feature Normalization\n",
+        "We also observe a huge difference among the value ranges of the 13 features (Figure 2). For instance, the values of feature *B* fall in $[0.32, 396.90]$, whereas those of feature *NOX* has a range of $[0.3850, 0.8170]$. An effective optimization would require data normalization. The goal of data normalization is to scale te values of each feature into roughly the same range, perhaps $[-0.5, 0.5]$. Here, we adopt a popular normalization technique where we substract the mean value from the feature value and divide the result by the width of the original range.\n",
+        "\n",
+        "There are at least three reasons for [Feature Normalization](https://en.wikipedia.org/wiki/Feature_scaling) (Feature Scaling):\n",
+        "- A value range that is too large or too small might cause floating number overflow or underflow during computation.\n",
+        "- Different value ranges might result in varying *importances* of different features to the model (at least in the beginning of the training process). This assumption about the data is often unreasonable, making the optimization difficult, which in turn results in increased training time.\n",
+        "- Many machine learning techniques or models (e.g., *L1/L2 regularization* and *Vector Space Model*) assumes that all the features have roughly zero means and their value ranges are similar.\n",
+        "\n",
+        "\u003cp align=\"center\"\u003e\n",
+        "    \u003cimg src = \"image/ranges_en.png\" width=550\u003e\u003cbr/\u003e\n",
+        "    Figure 2. The value ranges of the features\n",
+        "\u003c/p\u003e\n",
+        "\n",
+        "#### Prepare Training and Test Sets\n",
+        "We split the dataset in two, one for adjusting the model parameters, namely, for model training, and the other for model testing. The model error on the former is called the **training error**, and the error on the latter is called the **test error**. Our goal in training a model is to find the statistical dependency between the outputs and the inputs, so that we can predict new outputs given new inputs. As a result, the test error reflects the performance of the model better than the training error does. We consider two things when deciding the ratio of the training set to the test set: 1) More training data will decrease the variance of the parameter estimation, yielding more reliable models; 2) More test data will decrease the variance of the test error, yielding more reliable test errors. One standard split ratio is $8:2$.\n",
+        "\n",
+        "\n",
+        "When training complex models, we usually have one more split: the validation set. Complex models usually have [Hyperparameters](https://en.wikipedia.org/wiki/Hyperparameter_optimization) that need to be set before the training process, such as the number of layers in the network. Because hyperparameters are not part of the model parameters, they cannot be trained using the same loss function. Thus we will try several sets of hyperparameters to train several models and cross-validate them on the validation set to pick the best one; finally, the selected trained model is tested on the test set. Because our model is relatively simple, we will omit this validation process.\n",
+        "\n",
+        "\n",
+        "## Training\n",
+        "\n",
+        "`fit_a_line/trainer.py` demonstrates the training using [PaddlePaddle](http://paddlepaddle.org).\n",
+        "\n",
+        "### Initialize PaddlePaddle\n",
+        "\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "metadata": {
+        "editable": true
+      },
+      "source": [
+        "paddle.init(use_gpu=False, trainer_count=1)\n"
+      ],
+      "outputs": [
+        {
+          "name": "stdout",
+          "output_type": "stream",
+          "text": [
+            "\n"
+          ]
+        }
+      ],
+      "execution_count": 1
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "\n",
+        "### Model Configuration\n",
+        "\n",
+        "Logistic regression is essentially a fully-connected layer with linear activation:\n",
+        "\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "metadata": {
+        "editable": true
+      },
+      "source": [
+        "x = paddle.layer.data(name='x', type=paddle.data_type.dense_vector(13))\n",
+        "y_predict = paddle.layer.fc(input=x,\n",
+        "                                size=1,\n",
+        "                                act=paddle.activation.Linear())\n",
+        "y = paddle.layer.data(name='y', type=paddle.data_type.dense_vector(1))\n",
+        "cost = paddle.layer.regression_cost(input=y_predict, label=y)\n"
+      ],
+      "outputs": [
+        {
+          "name": "stdout",
+          "output_type": "stream",
+          "text": [
+            "\n"
+          ]
+        }
+      ],
+      "execution_count": 1
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "### Create Parameters\n",
+        "\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "metadata": {
+        "editable": true
+      },
+      "source": [
+        "parameters = paddle.parameters.create(cost)\n"
+      ],
+      "outputs": [
+        {
+          "name": "stdout",
+          "output_type": "stream",
+          "text": [
+            "\n"
+          ]
+        }
+      ],
+      "execution_count": 1
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "\n",
+        "### Create Trainer\n",
+        "\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "metadata": {
+        "editable": true
+      },
+      "source": [
+        "optimizer = paddle.optimizer.Momentum(momentum=0)\n",
+        "\n",
+        "trainer = paddle.trainer.SGD(cost=cost,\n",
+        "                             parameters=parameters,\n",
+        "                             update_equation=optimizer)\n"
+      ],
+      "outputs": [
+        {
+          "name": "stdout",
+          "output_type": "stream",
+          "text": [
+            "\n"
+          ]
+        }
+      ],
+      "execution_count": 1
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "\n",
+        "### Feeding Data\n",
+        "\n",
+        "PaddlePaddle provides the\n",
+        "[reader mechanism](https://github.com/PaddlePaddle/Paddle/tree/develop/doc/design/reader)\n",
+        "for loadinng training data. A reader may return multiple columns, and we need a Python dictionary to specify the mapping from column index to data layers.\n",
+        "\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "metadata": {
+        "editable": true
+      },
+      "source": [
+        "feeding={'x': 0, 'y': 1}\n"
+      ],
+      "outputs": [
+        {
+          "name": "stdout",
+          "output_type": "stream",
+          "text": [
+            "\n"
+          ]
+        }
+      ],
+      "execution_count": 1
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "\n",
+        "Moreover, an event handler is provided to print the training progress:\n",
+        "\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "metadata": {
+        "editable": true
+      },
+      "source": [
+        "# event_handler to print training and testing info\n",
+        "def event_handler(event):\n",
+        "    if isinstance(event, paddle.event.EndIteration):\n",
+        "        if event.batch_id % 100 == 0:\n",
+        "            print \"Pass %d, Batch %d, Cost %f\" % (\n",
+        "                event.pass_id, event.batch_id, event.cost)\n",
+        "\n",
+        "    if isinstance(event, paddle.event.EndPass):\n",
+        "        result = trainer.test(\n",
+        "            reader=paddle.batch(\n",
+        "                uci_housing.test(), batch_size=2),\n",
+        "            feeding=feeding)\n",
+        "        print \"Test %d, Cost %f\" % (event.pass_id, result.cost)\n"
+      ],
+      "outputs": [
+        {
+          "name": "stdout",
+          "output_type": "stream",
+          "text": [
+            "\n"
+          ]
+        }
+      ],
+      "execution_count": 1
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "\n",
+        "### Start Training\n",
+        "\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "metadata": {
+        "editable": true
+      },
+      "source": [
+        "trainer.train(\n",
+        "    reader=paddle.batch(\n",
+        "        paddle.reader.shuffle(\n",
+        "            uci_housing.train(), buf_size=500),\n",
+        "        batch_size=2),\n",
+        "    feeding=feeding,\n",
+        "    event_handler=event_handler,\n",
+        "    num_passes=30)\n"
+      ],
+      "outputs": [
+        {
+          "name": "stdout",
+          "output_type": "stream",
+          "text": [
+            "\n"
+          ]
+        }
+      ],
+      "execution_count": 1
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "\n",
+        "## Summary\n",
+        "This chapter introduces *Linear Regression* and how to train and test this model with PaddlePaddle, using the UCI Housing Data Set. Because a large number of more complex models and techniques are derived from linear regression, it is important to understand its underlying theory and limitation.\n",
+        "\n",
+        "\n",
+        "## References\n",
+        "1. https://en.wikipedia.org/wiki/Linear_regression\n",
+        "2. Friedman J, Hastie T, Tibshirani R. The elements of statistical learning[M]. Springer, Berlin: Springer series in statistics, 2001.\n",
+        "3. Murphy K P. Machine learning: a probabilistic perspective[M]. MIT press, 2012.\n",
+        "4. Bishop C M. Pattern recognition[J]. Machine Learning, 2006, 128.\n",
+        "\n",
+        "\u003cbr/\u003e\n",
+        "This tutorial is contributed by \u003ca xmlns:cc=\"http://creativecommons.org/ns#\" href=\"http://book.paddlepaddle.org\" property=\"cc:attributionName\" rel=\"cc:attributionURL\"\u003ePaddlePaddle\u003c/a\u003e, and licensed under a \u003ca rel=\"license\" href=\"http://creativecommons.org/licenses/by-nc-sa/4.0/\"\u003eCreative Commons Attribution-NonCommercial-ShareAlike 4.0 International License\u003c/a\u003e.\n"
+      ]
+    }
+  ],
+  "metadata": {
+    "kernelspec": {
+      "display_name": "Python 3",
+      "language": "python",
+      "name": "python3"
+    },
+    "language_info": {
+      "codemirror_mode": {
+        "name": "ipython",
+        "version": 3
+      },
+      "file_extension": ".py",
+      "mimetype": "text/x-python",
+      "name": "python",
+      "nbconvert_exporter": "python",
+      "pygments_lexer": "ipython3",
+      "version": "3.6.0"
+    }
+  },
+  "nbformat": 4,
+  "nbformat_minor": 0
+}

fit_a_line/README.en.md

 # Linear Regression
 Let us begin the tutorial with a classical problem called Linear Regression \[[1](#References)\]. In this chapter, we will train a model from a realistic dataset to predict home prices. Some important concepts in Machine Learning will be covered through this example.
 
-The source code for this tutorial lives on [book/fit_a_line](https://github.com/PaddlePaddle/book/tree/develop/fit_a_line). For instructions on getting started with PaddlePaddle, see [PaddlePaddle installation guide](http://www.paddlepaddle.org/doc_cn/build_and_install/index.html).
+The source code for this tutorial lives on [book/fit_a_line](https://github.com/PaddlePaddle/book/tree/develop/fit_a_line). For instructions on getting started with PaddlePaddle, see [PaddlePaddle installation guide](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/getstarted/build_and_install/docker_install_en.rst).
 
 ## Problem Setup
 Suppose we have a dataset of $n$ real estate properties. These real estate properties will be referred to as *homes* in this chapter for clarity.
@@ -202,4 +202,4 @@ This chapter introduces *Linear Regression* and how to train and test this model
 4. Bishop C M. Pattern recognition[J]. Machine Learning, 2006, 128.
 
 <br/>
-<a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/4.0/"><img alt="Common Creative License" style="border-width:0" src="https://i.creativecommons.org/l/by-nc-sa/4.0/88x31.png" /></a> This tutorial was created and published with [Creative Common License 4.0](http://creativecommons.org/licenses/by-nc-sa/4.0/).
+This tutorial is contributed by <a xmlns:cc="http://creativecommons.org/ns#" href="http://book.paddlepaddle.org" property="cc:attributionName" rel="cc:attributionURL">PaddlePaddle</a>, and licensed under a <a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/4.0/">Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License</a>.

fit_a_line/README.ipynb

+{
+  "cells": [
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "# 线性回归\n",
+        "让我们从经典的线性回归（Linear Regression \\[[1](#参考文献)\\]）模型开始这份教程。在这一章里，你将使用真实的数据集建立起一个房价预测模型，并且了解到机器学习中的若干重要概念。\n",
+        "\n",
+        "本教程源代码目录在[book/fit_a_line](https://github.com/PaddlePaddle/book/tree/develop/fit_a_line)， 初次使用请参考PaddlePaddle[安装教程](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/getstarted/build_and_install/docker_install_cn.rst)。\n",
+        "\n",
+        "## 背景介绍\n",
+        "给定一个大小为$n$的数据集  ${\\{y_{i}, x_{i1}, ..., x_{id}\\}}_{i=1}^{n}$，其中$x_{i1}, \\ldots, x_{id}$是第$i$个样本$d$个属性上的取值，$y_i$是该样本待预测的目标。线性回归模型假设目标$y_i$可以被属性间的线性组合描述，即\n",
+        "\n",
+        "$$y_i = \\omega_1x_{i1} + \\omega_2x_{i2} + \\ldots + \\omega_dx_{id} + b,  i=1,\\ldots,n$$\n",
+        "\n",
+        "例如，在我们将要建模的房价预测问题里，$x_{ij}$是描述房子$i$的各种属性（比如房间的个数、周围学校和医院的个数、交通状况等），而 $y_i$是房屋的价格。\n",
+        "\n",
+        "初看起来，这个假设实在过于简单了，变量间的真实关系很难是线性的。但由于线性回归模型有形式简单和易于建模分析的优点，它在实际问题中得到了大量的应用。很多经典的统计学习、机器学习书籍\\[[2,3,4](#参考文献)\\]也选择对线性模型独立成章重点讲解。\n",
+        "\n",
+        "## 效果展示\n",
+        "我们使用从[UCI Housing Data Set](https://archive.ics.uci.edu/ml/datasets/Housing)获得的波士顿房价数据集进行模型的训练和预测。下面的散点图展示了使用模型对部分房屋价格进行的预测。其中，每个点的横坐标表示同一类房屋真实价格的中位数，纵坐标表示线性回归模型根据特征预测的结果，当二者值完全相等的时候就会落在虚线上。所以模型预测得越准确，则点离虚线越近。\n",
+        "\u003cp align=\"center\"\u003e\n",
+        "    \u003cimg src = \"image/predictions.png\" width=400\u003e\u003cbr/\u003e\n",
+        "    图1. 预测值 V.S. 真实值\n",
+        "\u003c/p\u003e\n",
+        "\n",
+        "## 模型概览\n",
+        "\n",
+        "### 模型定义\n",
+        "\n",
+        "在波士顿房价数据集中，和房屋相关的值共有14个：前13个用来描述房屋相关的各种信息，即模型中的 $x_i$；最后一个值为我们要预测的该类房屋价格的中位数，即模型中的 $y_i$。因此，我们的模型就可以表示成：\n",
+        "\n",
+        "$$\\hat{Y} = \\omega_1X_{1} + \\omega_2X_{2} + \\ldots + \\omega_{13}X_{13} + b$$\n",
+        "\n",
+        "$\\hat{Y}$ 表示模型的预测结果，用来和真实值$Y$区分。模型要学习的参数即：$\\omega_1, \\ldots, \\omega_{13}, b$。\n",
+        "\n",
+        "建立模型后，我们需要给模型一个优化目标，使得学到的参数能够让预测值$\\hat{Y}$尽可能地接近真实值$Y$。这里我们引入损失函数（[Loss Function](https://en.wikipedia.org/wiki/Loss_function)，或Cost Function）这个概念。 输入任意一个数据样本的目标值$y_{i}$和模型给出的预测值$\\hat{y_{i}}$，损失函数输出一个非负的实值。这个实质通常用来反映模型误差的大小。\n",
+        "\n",
+        "对于线性回归模型来讲，最常见的损失函数就是均方误差（Mean Squared Error， [MSE](https://en.wikipedia.org/wiki/Mean_squared_error)）了，它的形式是：\n",
+        "\n",
+        "$$MSE=\\frac{1}{n}\\sum_{i=1}^{n}{(\\hat{Y_i}-Y_i)}^2$$\n",
+        "\n",
+        "即对于一个大小为$n$的测试集，$MSE$是$n$个数据预测结果误差平方的均值。\n",
+        "\n",
+        "### 训练过程\n",
+        "\n",
+        "定义好模型结构之后，我们要通过以下几个步骤进行模型训练\n",
+        " 1. 初始化参数，其中包括权重$\\omega_i$和偏置$b$，对其进行初始化（如0均值，1方差）。\n",
+        " 2. 网络正向传播计算网络输出和损失函数。\n",
+        " 3. 根据损失函数进行反向误差传播 （[backpropagation](https://en.wikipedia.org/wiki/Backpropagation)），将网络误差从输出层依次向前传递, 并更新网络中的参数。\n",
+        " 4. 重复2~3步骤，直至网络训练误差达到规定的程度或训练轮次达到设定值。\n",
+        "\n",
+        "## 数据集\n",
+        "\n",
+        "### 数据集接口的封装\n",
+        "首先加载需要的包\n",
+        "\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "metadata": {
+        "editable": true
+      },
+      "source": [
+        "import paddle.v2 as paddle\n",
+        "import paddle.v2.dataset.uci_housing as uci_housing\n"
+      ],
+      "outputs": [
+        {
+          "name": "stdout",
+          "output_type": "stream",
+          "text": [
+            "\n"
+          ]
+        }
+      ],
+      "execution_count": 1
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "\n",
+        "我们通过uci_housing模块引入了数据集合[UCI Housing Data Set](https://archive.ics.uci.edu/ml/datasets/Housing)\n",
+        "\n",
+        "其中，在uci_housing模块中封装了：\n",
+        "\n",
+        "1. 数据下载的过程。下载数据保存在~/.cache/paddle/dataset/uci_housing/housing.data。\n",
+        "2. [数据预处理](#数据预处理)的过程。\n",
+        "\n",
+        "\n",
+        "### 数据集介绍\n",
+        "这份数据集共506行，每行包含了波士顿郊区的一类房屋的相关信息及该类房屋价格的中位数。其各维属性的意义如下：\n",
+        "\n",
+        "| 属性名 | 解释 | 类型 |\n",
+        "| ------| ------ | ------ |\n",
+        "| CRIM | 该镇的人均犯罪率 | 连续值 |\n",
+        "| ZN | 占地面积超过25,000平方呎的住宅用地比例 | 连续值 |\n",
+        "| INDUS | 非零售商业用地比例 | 连续值 |\n",
+        "| CHAS | 是否邻近 Charles River  | 离散值，1=邻近；0=不邻近 |\n",
+        "| NOX | 一氧化氮浓度 | 连续值 |\n",
+        "| RM | 每栋房屋的平均客房数 | 连续值 |\n",
+        "| AGE | 1940年之前建成的自用单位比例 | 连续值 |\n",
+        "| DIS | 到波士顿5个就业中心的加权距离 | 连续值 |\n",
+        "| RAD | 到径向公路的可达性指数 | 连续值 |\n",
+        "| TAX | 全值财产税率 | 连续值 |\n",
+        "| PTRATIO | 学生与教师的比例 | 连续值 |\n",
+        "| B | 1000(BK - 0.63)^2，其中BK为黑人占比 | 连续值 |\n",
+        "| LSTAT | 低收入人群占比 | 连续值 |\n",
+        "| MEDV | 同类房屋价格的中位数 | 连续值 |\n",
+        "\n",
+        "### 数据预处理\n",
+        "#### 连续值与离散值\n",
+        "观察一下数据，我们的第一个发现是：所有的13维属性中，有12维的连续值和1维的离散值（CHAS）。离散值虽然也常使用类似0、1、2这样的数字表示，但是其含义与连续值是不同的，因为这里的差值没有实际意义。例如，我们用0、1、2来分别表示红色、绿色和蓝色的话，我们并不能因此说“蓝色和红色”比“绿色和红色”的距离更远。所以通常对一个有$d$个可能取值的离散属性，我们会将它们转为$d$个取值为0或1的二值属性或者将每个可能取值映射为一个多维向量。不过就这里而言，因为CHAS本身就是一个二值属性，就省去了这个麻烦。\n",
+        "\n",
+        "#### 属性的归一化\n",
+        "另外一个稍加观察即可发现的事实是，各维属性的取值范围差别很大（如图2所示）。例如，属性B的取值范围是[0.32, 396.90]，而属性NOX的取值范围是[0.3850, 0.8170]。这里就要用到一个常见的操作-归一化（normalization）了。归一化的目标是把各位属性的取值范围放缩到差不多的区间，例如[-0.5,0.5]。这里我们使用一种很常见的操作方法：减掉均值，然后除以原取值范围。\n",
+        "\n",
+        "做归一化（或 [Feature scaling](https://en.wikipedia.org/wiki/Feature_scaling)）至少有以下3个理由：\n",
+        "- 过大或过小的数值范围会导致计算时的浮点上溢或下溢。\n",
+        "- 不同的数值范围会导致不同属性对模型的重要性不同（至少在训练的初始阶段如此），而这个隐含的假设常常是不合理的。这会对优化的过程造成困难，使训练时间大大的加长。\n",
+        "- 很多的机器学习技巧/模型（例如L1，L2正则项，向量空间模型-Vector Space Model）都基于这样的假设：所有的属性取值都差不多是以0为均值且取值范围相近的。\n",
+        "\n",
+        "\u003cp align=\"center\"\u003e\n",
+        "    \u003cimg src = \"image/ranges.png\" width=550\u003e\u003cbr/\u003e\n",
+        "    图2. 各维属性的取值范围\n",
+        "\u003c/p\u003e\n",
+        "\n",
+        "#### 整理训练集与测试集\n",
+        "我们将数据集分割为两份：一份用于调整模型的参数，即进行模型的训练，模型在这份数据集上的误差被称为**训练误差**；另外一份被用来测试，模型在这份数据集上的误差被称为**测试误差**。我们训练模型的目的是为了通过从训练数据中找到规律来预测未知的新数据，所以测试误差是更能反映模型表现的指标。分割数据的比例要考虑到两个因素：更多的训练数据会降低参数估计的方差，从而得到更可信的模型；而更多的测试数据会降低测试误差的方差，从而得到更可信的测试误差。我们这个例子中设置的分割比例为$8:2$\n",
+        "\n",
+        "\n",
+        "在更复杂的模型训练过程中，我们往往还会多使用一种数据集：验证集。因为复杂的模型中常常还有一些超参数（[Hyperparameter](https://en.wikipedia.org/wiki/Hyperparameter_optimization)）需要调节，所以我们会尝试多种超参数的组合来分别训练多个模型，然后对比它们在验证集上的表现选择相对最好的一组超参数，最后才使用这组参数下训练的模型在测试集上评估测试误差。由于本章训练的模型比较简单，我们暂且忽略掉这个过程。\n",
+        "\n",
+        "## 训练\n",
+        "\n",
+        "`fit_a_line/trainer.py`演示了训练的整体过程。\n",
+        "\n",
+        "### 初始化PaddlePaddle\n",
+        "\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "metadata": {
+        "editable": true
+      },
+      "source": [
+        "paddle.init(use_gpu=False, trainer_count=1)\n"
+      ],
+      "outputs": [
+        {
+          "name": "stdout",
+          "output_type": "stream",
+          "text": [
+            "\n"
+          ]
+        }
+      ],
+      "execution_count": 1
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "\n",
+        "### 模型配置\n",
+        "\n",
+        "线性回归的模型其实就是一个采用线性激活函数（linear activation，`LinearActivation`）的全连接层（fully-connected layer，`fc_layer`）：\n",
+        "\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "metadata": {
+        "editable": true
+      },
+      "source": [
+        "x = paddle.layer.data(name='x', type=paddle.data_type.dense_vector(13))\n",
+        "y_predict = paddle.layer.fc(input=x,\n",
+        "                                size=1,\n",
+        "                                act=paddle.activation.Linear())\n",
+        "y = paddle.layer.data(name='y', type=paddle.data_type.dense_vector(1))\n",
+        "cost = paddle.layer.regression_cost(input=y_predict, label=y)\n"
+      ],
+      "outputs": [
+        {
+          "name": "stdout",
+          "output_type": "stream",
+          "text": [
+            "\n"
+          ]
+        }
+      ],
+      "execution_count": 1
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "### 创建参数\n",
+        "\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "metadata": {
+        "editable": true
+      },
+      "source": [
+        "parameters = paddle.parameters.create(cost)\n"
+      ],
+      "outputs": [
+        {
+          "name": "stdout",
+          "output_type": "stream",
+          "text": [
+            "\n"
+          ]
+        }
+      ],
+      "execution_count": 1
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "\n",
+        "### 创建Trainer\n",
+        "\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "metadata": {
+        "editable": true
+      },
+      "source": [
+        "optimizer = paddle.optimizer.Momentum(momentum=0)\n",
+        "\n",
+        "trainer = paddle.trainer.SGD(cost=cost,\n",
+        "                             parameters=parameters,\n",
+        "                             update_equation=optimizer)\n"
+      ],
+      "outputs": [
+        {
+          "name": "stdout",
+          "output_type": "stream",
+          "text": [
+            "\n"
+          ]
+        }
+      ],
+      "execution_count": 1
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "\n",
+        "### 读取数据且打印训练的中间信息\n",
+        "\n",
+        "PaddlePaddle提供一个\n",
+        "[reader机制](https://github.com/PaddlePaddle/Paddle/tree/develop/doc/design/reader)\n",
+        "来读取数据。 Reader返回的数据可以包括多列，我们需要一个Python dict把列\n",
+        "序号映射到网络里的数据层。\n",
+        "\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "metadata": {
+        "editable": true
+      },
+      "source": [
+        "feeding={'x': 0, 'y': 1}\n"
+      ],
+      "outputs": [
+        {
+          "name": "stdout",
+          "output_type": "stream",
+          "text": [
+            "\n"
+          ]
+        }
+      ],
+      "execution_count": 1
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "\n",
+        "此外，我们还可以提供一个 event handler，来打印训练的进度：\n",
+        "\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "metadata": {
+        "editable": true
+      },
+      "source": [
+        "# event_handler to print training and testing info\n",
+        "def event_handler(event):\n",
+        "    if isinstance(event, paddle.event.EndIteration):\n",
+        "        if event.batch_id % 100 == 0:\n",
+        "            print \"Pass %d, Batch %d, Cost %f\" % (\n",
+        "                event.pass_id, event.batch_id, event.cost)\n",
+        "\n",
+        "    if isinstance(event, paddle.event.EndPass):\n",
+        "        result = trainer.test(\n",
+        "            reader=paddle.batch(\n",
+        "                uci_housing.test(), batch_size=2),\n",
+        "            feeding=feeding)\n",
+        "        print \"Test %d, Cost %f\" % (event.pass_id, result.cost)\n"
+      ],
+      "outputs": [
+        {
+          "name": "stdout",
+          "output_type": "stream",
+          "text": [
+            "\n"
+          ]
+        }
+      ],
+      "execution_count": 1
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "\n",
+        "### 开始训练\n",
+        "\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "metadata": {
+        "editable": true
+      },
+      "source": [
+        "trainer.train(\n",
+        "    reader=paddle.batch(\n",
+        "        paddle.reader.shuffle(\n",
+        "            uci_housing.train(), buf_size=500),\n",
+        "        batch_size=2),\n",
+        "    feeding=feeding,\n",
+        "    event_handler=event_handler,\n",
+        "    num_passes=30)\n"
+      ],
+      "outputs": [
+        {
+          "name": "stdout",
+          "output_type": "stream",
+          "text": [
+            "\n"
+          ]
+        }
+      ],
+      "execution_count": 1
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "\n",
+        "## 总结\n",
+        "在这章里，我们借助波士顿房价这一数据集，介绍了线性回归模型的基本概念，以及如何使用PaddlePaddle实现训练和测试的过程。很多的模型和技巧都是从简单的线性回归模型演化而来，因此弄清楚线性模型的原理和局限非常重要。\n",
+        "\n",
+        "\n",
+        "## 参考文献\n",
+        "1. https://en.wikipedia.org/wiki/Linear_regression\n",
+        "2. Friedman J, Hastie T, Tibshirani R. The elements of statistical learning[M]. Springer, Berlin: Springer series in statistics, 2001.\n",
+        "3. Murphy K P. Machine learning: a probabilistic perspective[M]. MIT press, 2012.\n",
+        "4. Bishop C M. Pattern recognition[J]. Machine Learning, 2006, 128.\n",
+        "\n",
+        "\u003cbr/\u003e\n",
+        "\u003ca rel=\"license\" href=\"http://creativecommons.org/licenses/by-nc-sa/4.0/\"\u003e\u003cimg alt=\"知识共享许可协议\" style=\"border-width:0\" src=\"https://i.creativecommons.org/l/by-nc-sa/4.0/88x31.png\" /\u003e\u003c/a\u003e\u003cbr /\u003e\u003cspan xmlns:dct=\"http://purl.org/dc/terms/\" href=\"http://purl.org/dc/dcmitype/Text\" property=\"dct:title\" rel=\"dct:type\"\u003e本教程\u003c/span\u003e 由 \u003ca xmlns:cc=\"http://creativecommons.org/ns#\" href=\"http://book.paddlepaddle.org\" property=\"cc:attributionName\" rel=\"cc:attributionURL\"\u003ePaddlePaddle\u003c/a\u003e 创作，采用 \u003ca rel=\"license\" href=\"http://creativecommons.org/licenses/by-nc-sa/4.0/\"\u003e知识共享 署名-非商业性使用-相同方式共享 4.0 国际 许可协议\u003c/a\u003e进行许可。\n"
+      ]
+    }
+  ],
+  "metadata": {
+    "kernelspec": {
+      "display_name": "Python 3",
+      "language": "python",
+      "name": "python3"
+    },
+    "language_info": {
+      "codemirror_mode": {
+        "name": "ipython",
+        "version": 3
+      },
+      "file_extension": ".py",
+      "mimetype": "text/x-python",
+      "name": "python",
+      "nbconvert_exporter": "python",
+      "pygments_lexer": "ipython3",
+      "version": "3.6.0"
+    }
+  },
+  "nbformat": 4,
+  "nbformat_minor": 0
+}

fit_a_line/README.md

 # 线性回归
 让我们从经典的线性回归（Linear Regression \[[1](#参考文献)\]）模型开始这份教程。在这一章里，你将使用真实的数据集建立起一个房价预测模型，并且了解到机器学习中的若干重要概念。
 
-本教程源代码目录在[book/fit_a_line](https://github.com/PaddlePaddle/book/tree/develop/fit_a_line)， 初次使用请参考PaddlePaddle[安装教程](http://www.paddlepaddle.org/doc_cn/build_and_install/index.html)。
+本教程源代码目录在[book/fit_a_line](https://github.com/PaddlePaddle/book/tree/develop/fit_a_line)， 初次使用请参考PaddlePaddle[安装教程](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/getstarted/build_and_install/docker_install_cn.rst)。
 
 ## 背景介绍
 给定一个大小为$n$的数据集  ${\{y_{i}, x_{i1}, ..., x_{id}\}}_{i=1}^{n}$，其中$x_{i1}, \ldots, x_{id}$是第$i$个样本$d$个属性上的取值，$y_i$是该样本待预测的目标。线性回归模型假设目标$y_i$可以被属性间的线性组合描述，即
@@ -15,8 +15,8 @@ $$y_i = \omega_1x_{i1} + \omega_2x_{i2} + \ldots + \omega_dx_{id} + b,  i=1,\ldo
 ## 效果展示
 我们使用从[UCI Housing Data Set](https://archive.ics.uci.edu/ml/datasets/Housing)获得的波士顿房价数据集进行模型的训练和预测。下面的散点图展示了使用模型对部分房屋价格进行的预测。其中，每个点的横坐标表示同一类房屋真实价格的中位数，纵坐标表示线性回归模型根据特征预测的结果，当二者值完全相等的时候就会落在虚线上。所以模型预测得越准确，则点离虚线越近。
 <p align="center">
-	<img src = "image/predictions.png" width=400><br/>
-	图1. 预测值 V.S. 真实值
+    <img src = "image/predictions.png" width=400><br/>
+    图1. 预测值 V.S. 真实值
 </p>
 
 ## 模型概览
@@ -96,8 +96,8 @@ import paddle.v2.dataset.uci_housing as uci_housing
 - 很多的机器学习技巧/模型（例如L1，L2正则项，向量空间模型-Vector Space Model）都基于这样的假设：所有的属性取值都差不多是以0为均值且取值范围相近的。
 
 <p align="center">
-	<img src = "image/ranges.png" width=550><br/>
-	图2. 各维属性的取值范围
+    <img src = "image/ranges.png" width=550><br/>
+    图2. 各维属性的取值范围
 </p>
 
 #### 整理训练集与测试集

fit_a_line/index.en.html

@@ -43,7 +43,7 @@
 # Linear Regression
 Let us begin the tutorial with a classical problem called Linear Regression \[[1](#References)\]. In this chapter, we will train a model from a realistic dataset to predict home prices. Some important concepts in Machine Learning will be covered through this example.
 
-The source code for this tutorial lives on [book/fit_a_line](https://github.com/PaddlePaddle/book/tree/develop/fit_a_line). For instructions on getting started with PaddlePaddle, see [PaddlePaddle installation guide](http://www.paddlepaddle.org/doc_cn/build_and_install/index.html).
+The source code for this tutorial lives on [book/fit_a_line](https://github.com/PaddlePaddle/book/tree/develop/fit_a_line). For instructions on getting started with PaddlePaddle, see [PaddlePaddle installation guide](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/getstarted/build_and_install/docker_install_en.rst).
 
 ## Problem Setup
 Suppose we have a dataset of $n$ real estate properties. These real estate properties will be referred to as *homes* in this chapter for clarity.
@@ -244,7 +244,7 @@ This chapter introduces *Linear Regression* and how to train and test this model
 4. Bishop C M. Pattern recognition[J]. Machine Learning, 2006, 128.
 
 <br/>
-<a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/4.0/"><img alt="Common Creative License" style="border-width:0" src="https://i.creativecommons.org/l/by-nc-sa/4.0/88x31.png" /></a> This tutorial was created and published with [Creative Common License 4.0](http://creativecommons.org/licenses/by-nc-sa/4.0/).
+This tutorial is contributed by <a xmlns:cc="http://creativecommons.org/ns#" href="http://book.paddlepaddle.org" property="cc:attributionName" rel="cc:attributionURL">PaddlePaddle</a>, and licensed under a <a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/4.0/">Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License</a>.
 
 </div>
 <!-- You can change the lines below now. -->

fit_a_line/index.html

@@ -43,7 +43,7 @@
 # 线性回归
 让我们从经典的线性回归（Linear Regression \[[1](#参考文献)\]）模型开始这份教程。在这一章里，你将使用真实的数据集建立起一个房价预测模型，并且了解到机器学习中的若干重要概念。
 
-本教程源代码目录在[book/fit_a_line](https://github.com/PaddlePaddle/book/tree/develop/fit_a_line)， 初次使用请参考PaddlePaddle[安装教程](http://www.paddlepaddle.org/doc_cn/build_and_install/index.html)。
+本教程源代码目录在[book/fit_a_line](https://github.com/PaddlePaddle/book/tree/develop/fit_a_line)， 初次使用请参考PaddlePaddle[安装教程](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/getstarted/build_and_install/docker_install_cn.rst)。
 
 ## 背景介绍
 给定一个大小为$n$的数据集  ${\{y_{i}, x_{i1}, ..., x_{id}\}}_{i=1}^{n}$，其中$x_{i1}, \ldots, x_{id}$是第$i$个样本$d$个属性上的取值，$y_i$是该样本待预测的目标。线性回归模型假设目标$y_i$可以被属性间的线性组合描述，即
@@ -57,8 +57,8 @@ $$y_i = \omega_1x_{i1} + \omega_2x_{i2} + \ldots + \omega_dx_{id} + b,  i=1,\ldo
 ## 效果展示
 我们使用从[UCI Housing Data Set](https://archive.ics.uci.edu/ml/datasets/Housing)获得的波士顿房价数据集进行模型的训练和预测。下面的散点图展示了使用模型对部分房屋价格进行的预测。其中，每个点的横坐标表示同一类房屋真实价格的中位数，纵坐标表示线性回归模型根据特征预测的结果，当二者值完全相等的时候就会落在虚线上。所以模型预测得越准确，则点离虚线越近。
 <p align="center">
-	<img src = "image/predictions.png" width=400><br/>
-	图1. 预测值 V.S. 真实值
+    <img src = "image/predictions.png" width=400><br/>
+    图1. 预测值 V.S. 真实值
 </p>
 
 ## 模型概览
@@ -138,8 +138,8 @@ import paddle.v2.dataset.uci_housing as uci_housing
 - 很多的机器学习技巧/模型（例如L1，L2正则项，向量空间模型-Vector Space Model）都基于这样的假设：所有的属性取值都差不多是以0为均值且取值范围相近的。
 
 <p align="center">
-	<img src = "image/ranges.png" width=550><br/>
-	图2. 各维属性的取值范围
+    <img src = "image/ranges.png" width=550><br/>
+    图2. 各维属性的取值范围
 </p>
 
 #### 整理训练集与测试集

fit_a_line/train.py

@@ -18,9 +18,8 @@ def main():
     # create optimizer
     optimizer = paddle.optimizer.Momentum(momentum=0)
 
-    trainer = paddle.trainer.SGD(cost=cost,
-                                 parameters=parameters,
-                                 update_equation=optimizer)
+    trainer = paddle.trainer.SGD(
+        cost=cost, parameters=parameters, update_equation=optimizer)
 
     feeding = {'x': 0, 'y': 1}
 
@@ -33,16 +32,14 @@ def main():
 
         if isinstance(event, paddle.event.EndPass):
             result = trainer.test(
-                reader=paddle.batch(
-                    uci_housing.test(), batch_size=2),
+                reader=paddle.batch(uci_housing.test(), batch_size=2),
                 feeding=feeding)
             print "Test %d, Cost %f" % (event.pass_id, result.cost)
 
     # training
     trainer.train(
         reader=paddle.batch(
-            paddle.reader.shuffle(
-                uci_housing.train(), buf_size=500),
+            paddle.reader.shuffle(uci_housing.train(), buf_size=500),
             batch_size=2),
         feeding=feeding,
         event_handler=event_handler,

image_classification/README.en.md

 Image Classification
 =======================
 
-The source code of this chapter is in [book/image_classification](https://github.com/PaddlePaddle/book/tree/develop/image_classification). For the first-time users, please refer to PaddlePaddle[Installation Tutorial](http://www.paddlepaddle.org/doc_cn/build_and_install/index.html) for installation instructions.
+The source code of this chapter is in [book/image_classification](https://github.com/PaddlePaddle/book/tree/develop/image_classification). For the first-time users, please refer to PaddlePaddle [Installation Tutorial](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/getstarted/build_and_install/docker_install_en.rst) for installation instructions.
 
 ## Background
 
@@ -135,146 +135,73 @@ Figure 10. ResNet model for ImageNet
 </p>
 
 
-## Data Preparation
-
-### Data description and downloading
+## Dataset
 
 Commonly used public datasets for image classification are CIFAR(https://www.cs.toronto.edu/~kriz/cifar.html), ImageNet(http://image-net.org/), COCO(http://mscoco.org/), etc. Those used for fine-grained image classification are CUB-200-2011(http://www.vision.caltech.edu/visipedia/CUB-200-2011.html), Stanford Dog(http://vision.stanford.edu/aditya86/ImageNetDogs/), Oxford-flowers(http://www.robots.ox.ac.uk/~vgg/data/flowers/), etc. Among them, ImageNet are the largest and most research results are reported on ImageNet as mentioned in Model Overview section. Since 2010, the data of Imagenet has gone through some changes. The commonly used ImageNet-2012 dataset contains 1000 categories. There are 1,281,167 training images, ranging from 732 to 1200 images per category, and 50,000 validation images with 50 images per category in average.
 
-Since ImageNet is too large to be downloaded and trained efficiently, we use CIFAR10 (https://www.cs.toronto.edu/~kriz/cifar.html) in this tutorial. The CIFAR-10 dataset consists of 60000 32x32 color images in 10 classes, with 6000 images per class. There are 50000 training images and 10000 test images. Figure 11 shows all the classes in CIFAR10 as well as 10 images randomly sampled from each category.
+Since ImageNet is too large to be downloaded and trained efficiently, we use CIFAR-10 (https://www.cs.toronto.edu/~kriz/cifar.html) in this tutorial. The CIFAR-10 dataset consists of 60000 32x32 color images in 10 classes, with 6000 images per class. There are 50000 training images and 10000 test images. Figure 11 shows all the classes in CIFAR-10 as well as 10 images randomly sampled from each category.
 
 <p align="center">
 <img src="image/cifar.png" width="350"><br/>
 Figure 11. CIFAR10 dataset[21]
 </p>
 
-The following command is used for downloading data and calculating the mean image used for data preprocessing.
-
-```bash
-./data/get_data.sh
-```
+ `paddle.datasets` package encapsulates multiple public datasets, including `cifar`, `imdb`, `mnist`, `moivelens` and `wmt14`, etc. There's no need for us to manually download and preprocess CIFAR-10.
 
-###  Data provider for PaddlePaddle
+After issuing a command `python train.py`, training will starting immediately. The details will be unpacked by the following sessions to see how it works.
 
-We use Python interface for providing data to PaddlePaddle. The following file dataprovider.py is a complete example for CIFAR10.
+## Model Structure
 
-- 'initializer' function performs initialization of dataprovider: loading the mean image, defining two input types -- image and label.
+### Initialize PaddlePaddle
 
-- 'process' function sends preprocessed data to PaddlePaddle. Data preprocessing performed in this function includes data perturbation, random horizontal flipping, deducting mean image from the raw image.
+We must import and initialize PaddlePaddle (enable/disable GPU, set the number of trainers, etc).
 
 ```python
-import numpy as np
-import cPickle
-from paddle.trainer.PyDataProvider2 import *
-
-def initializer(settings, mean_path, is_train, **kwargs):
-    settings.is_train = is_train
-    settings.input_size = 3 * 32 * 32
-    settings.mean = np.load(mean_path)['mean']
-    settings.input_types = {
-        'image': dense_vector(settings.input_size),
-        'label': integer_value(10)
-    }
-
-
-@provider(init_hook=initializer, pool_size=50000)
-def process(settings, file_list):
-    with open(file_list, 'r') as fdata:
-        for fname in fdata:
-            fo = open(fname.strip(), 'rb')
-            batch = cPickle.load(fo)
-            fo.close()
-            images = batch['data']
-            labels = batch['labels']
-            for im, lab in zip(images, labels):
-                if settings.is_train and np.random.randint(2):
-                    im = im.reshape(3, 32, 32)
-                    im = im[:,:,::-1]
-                    im = im.flatten()
-                im = im - settings.mean
-                yield {
-                    'image': im.astype('float32'),
-                    'label': int(lab)
-                }
-```
+import sys
+import paddle.v2 as paddle
 
-## Model Config
-
-### Data Definition
-
-In model config file, function `define_py_data_sources2` sets argument 'module' to dataprovider file for loading data, 'args' to mean image file. If the config file is used for prediction, then there is no need to set argument 'train_list'.
-
-```python
-from paddle.trainer_config_helpers import *
-
-is_predict = get_config_arg("is_predict", bool, False)
-if not is_predict:
-    define_py_data_sources2(
-        train_list='data/train.list',
-        test_list='data/test.list',
-        module='dataprovider',
-        obj='process',
-        args={'mean_path': 'data/mean.meta'})
-```
-
-### Algorithm Settings
-
-In model config file, function 'settings' specifies optimization algorithm, batch size, learning rate, momentum and L2 regularization.
-
-```python
-settings(
-    batch_size=128,
-    learning_rate=0.1 / 128.0,
-    learning_rate_decay_a=0.1,
-    learning_rate_decay_b=50000 * 100,
-    learning_rate_schedule='discexp',
-    learning_method=MomentumOptimizer(0.9),
-    regularization=L2Regularization(0.0005 * 128),)
+# PaddlePaddle init
+paddle.init(use_gpu=False, trainer_count=1)
 ```
 
-The learning rate adjustment policy can be defined with variables `learning_rate_decay_a`($a$), `learning_rate_decay_b`($b$) and `learning_rate_schedule`. In this example, discrete exponential method is used for adjusting learning rate. The formula is as follows,
-$$  lr = lr_{0} * a^ {\lfloor \frac{n}{ b}\rfloor} $$
-where $n$ is the number of processed samples, $lr_{0}$ is the learning_rate set in 'settings'.
-
-### Model Architecture
-
-Here we provide the cofig files for VGG and ResNet models.
+As alluded to in section [Model Overview](#model-overview), here we provide the implementations of both VGG and ResNet models.
 
-#### VGG
+### VGG
 
-First we define VGG network. Since the image size and amount of CIFAR10 are relatively small comparing to ImageNet, we uses a small version of VGG network for CIFAR10. Convolution groups incorporate BN and dropout operations.
+First, we use a VGG network. Since the image size and amount of CIFAR10 are relatively small comparing to ImageNet, we uses a small version of VGG network for CIFAR10. Convolution groups incorporate BN and dropout operations.
 
 1. Define input data and its dimension
 
-        The input to the network is defined as `data_layer`, or image pixels in the context of image classification. The images in CIFAR10 are 32x32 color images of three channels. Therefore, the size of the input data is 3072 (3x32x32), and the number of categories is 10.
+        The input to the network is defined as `paddle.layer.data`, or image pixels in the context of image classification. The images in CIFAR10 are 32x32 color images of three channels. Therefore, the size of the input data is 3072 (3x32x32), and the number of categories is 10.
 
     ```python
     datadim = 3 * 32 * 32
     classdim = 10
-    data = data_layer(name='image', size=datadim)
+    image = paddle.layer.data(
+        name="image", type=paddle.data_type.dense_vector(datadim))
     ```
 
 2. Define VGG main module
 
     ```python
-    net = vgg_bn_drop(data)
+    net = vgg_bn_drop(image)
     ```
-        The input to VGG main module is from data layer. `vgg_bn_drop` defines a 16-layer VGG network, with each convolutional layer followed by BN and dropout layers. Here is the definition in detail:
+        The input to VGG main module is from the data layer. `vgg_bn_drop` defines a 16-layer VGG network, with each convolutional layer followed by BN and dropout layers. Here is the definition in detail:
 
     ```python
-    def vgg_bn_drop(input, num_channels):
-        def conv_block(ipt, num_filter, groups, dropouts, num_channels_=None):
-            return img_conv_group(
+    def vgg_bn_drop(input):
+        def conv_block(ipt, num_filter, groups, dropouts, num_channels=None):
+            return paddle.networks.img_conv_group(
                 input=ipt,
-                num_channels=num_channels_,
+                num_channels=num_channels,
                 pool_size=2,
                 pool_stride=2,
                 conv_num_filter=[num_filter] * groups,
                 conv_filter_size=3,
-                conv_act=ReluActivation(),
+                conv_act=paddle.activation.Relu(),
                 conv_with_batchnorm=True,
                 conv_batchnorm_drop_rate=dropouts,
-                pool_type=MaxPooling())
+                pool_type=paddle.pooling.Max())
 
         conv1 = conv_block(input, 64, 2, [0.3, 0], 3)
         conv2 = conv_block(conv1, 128, 2, [0.4, 0])
@@ -282,16 +209,17 @@ First we define VGG network. Since the image size and amount of CIFAR10 are rela
         conv4 = conv_block(conv3, 512, 3, [0.4, 0.4, 0])
         conv5 = conv_block(conv4, 512, 3, [0.4, 0.4, 0])
 
-        drop = dropout_layer(input=conv5, dropout_rate=0.5)
-        fc1 = fc_layer(input=drop, size=512, act=LinearActivation())
-        bn = batch_norm_layer(
-            input=fc1, act=ReluActivation(), layer_attr=ExtraAttr(drop_rate=0.5))
-        fc2 = fc_layer(input=bn, size=512, act=LinearActivation())
+        drop = paddle.layer.dropout(input=conv5, dropout_rate=0.5)
+        fc1 = paddle.layer.fc(input=drop, size=512, act=paddle.activation.Linear())
+        bn = paddle.layer.batch_norm(
+            input=fc1,
+            act=paddle.activation.Relu(),
+            layer_attr=paddle.attr.Extra(drop_rate=0.5))
+        fc2 = paddle.layer.fc(input=bn, size=512, act=paddle.activation.Linear())
         return fc2
-
     ```
 
-        2.1. First defines a convolution block or conv_block. The default convolution kernel is 3x3, and the default pooling size is 2x2 with stride 2. Dropout specifies the probability in dropout operation. Function `img_conv_group` is defined in `paddle.trainer_config_helpers` consisting of a series of `Conv->BN->ReLu->Dropout` and a `Pooling`.
+        2.1. First defines a convolution block or conv_block. The default convolution kernel is 3x3, and the default pooling size is 2x2 with stride 2. Dropout specifies the probability in dropout operation. Function `img_conv_group` is defined in `paddle.networks` consisting of a series of `Conv->BN->ReLu->Dropout` and a `Pooling`.
 
 
         2.2. Five groups of convolutions. The first two groups perform two convolutions, while the last three groups perform three convolutions. The dropout rate of the last convolution in each group is set to 0, which means there is no dropout for this layer.
@@ -309,15 +237,12 @@ First we define VGG network. Since the image size and amount of CIFAR10 are rela
 
 4. Define Loss Function and Outputs
 
-        In the context of supervised learning, labels of training images are defined in `data_layer`, too. During training, cross-entropy is used as loss function and as the output of the network; During testing, the outputs are the probabilities calculated in the classifier.
+        In the context of supervised learning, labels of training images are defined in `paddle.layer.data`, too. During training, cross-entropy is used as loss function and as the output of the network; During testing, the outputs are the probabilities calculated in the classifier.
 
     ```python
-    if not is_predict:
-        lbl = data_layer(name="label", size=class_num)
-        cost = classification_cost(input=out, label=lbl)
-        outputs(cost)
-    else:
-        outputs(out)
+    lbl = paddle.layer.data(
+        name="label", type=paddle.data_type.integer_value(classdim))
+    cost = paddle.layer.classification_cost(input=out, label=lbl)
     ```
 
 ### ResNet
@@ -325,13 +250,13 @@ First we define VGG network. Since the image size and amount of CIFAR10 are rela
 The first, third and forth steps of a ResNet are the same as a VGG. The second one is the main module.
 
 ```python
-net = resnet_cifar10(data, depth=56)
+net = resnet_cifar10(data, depth=32)
 ```
 
 Here are some basic functions used in `resnet_cifar10`:
 
   - `conv_bn_layer` : convolutional layer followed by BN.
-  - `shortcut` : the shortcut branch in a residual block. There are two kinds of shortcuts: 1x1 convolution used when the number of channels between input and output are different; direct connection used otherwise.
+  - `shortcut` : the shortcut branch in a residual block. There are two kinds of shortcuts: 1x1 convolution used when the number of channels between input and output is different; direct connection used otherwise.
 
   - `basicblock` : a basic residual module as shown in the left of Figure 9, consisting of two sequential 3x3 convolutions and one "shortcut" branch.
   - `bottleneck` : a bottleneck module as shown in the right of Figure 9, consisting of a two 1x1 convolutions with one 3x3 convolution in between branch and a "shortcut" branch.
@@ -343,47 +268,38 @@ def conv_bn_layer(input,
                   filter_size,
                   stride,
                   padding,
-                  active_type=ReluActivation(),
+                  active_type=paddle.activation.Relu(),
                   ch_in=None):
-    tmp = img_conv_layer(
+    tmp = paddle.layer.img_conv(
         input=input,
         filter_size=filter_size,
         num_channels=ch_in,
         num_filters=ch_out,
         stride=stride,
         padding=padding,
-        act=LinearActivation(),
+        act=paddle.activation.Linear(),
         bias_attr=False)
-    return batch_norm_layer(input=tmp, act=active_type)
-
+    return paddle.layer.batch_norm(input=tmp, act=active_type)
 
 def shortcut(ipt, n_in, n_out, stride):
     if n_in != n_out:
-        return conv_bn_layer(ipt, n_out, 1, stride, 0, LinearActivation())
+        return conv_bn_layer(ipt, n_out, 1, stride, 0,
+                             paddle.activation.Linear())
     else:
         return ipt
 
 def basicblock(ipt, ch_out, stride):
-    ch_in = ipt.num_filters
+    ch_in = ch_out * 2
     tmp = conv_bn_layer(ipt, ch_out, 3, stride, 1)
-    tmp = conv_bn_layer(tmp, ch_out, 3, 1, 1, LinearActivation())
-    short = shortcut(ipt, ch_in, ch_out, stride)
-    return addto_layer(input=[ipt, short], act=ReluActivation())
-
-def bottleneck(ipt, ch_out, stride):
-    ch_in = ipt.num_filter
-    tmp = conv_bn_layer(ipt, ch_out, 1, stride, 0)
-    tmp = conv_bn_layer(tmp, ch_out, 3, 1, 1)
-    tmp = conv_bn_layer(tmp, ch_out * 4, 1, 1, 0, LinearActivation())
+    tmp = conv_bn_layer(tmp, ch_out, 3, 1, 1, paddle.activation.Linear())
     short = shortcut(ipt, ch_in, ch_out, stride)
-    return addto_layer(input=[ipt, short], act=ReluActivation())
+    return paddle.layer.addto(input=[tmp, short], act=paddle.activation.Relu())
 
 def layer_warp(block_func, ipt, features, count, stride):
     tmp = block_func(ipt, features, stride)
     for i in range(1, count):
         tmp = block_func(tmp, features, 1)
     return tmp
-
 ```
 
 The following are the components of `resnet_cifar10`:
@@ -395,106 +311,131 @@ The following are the components of `resnet_cifar10`:
 Note: besides the first convolutional layer and the last fully-connected layer, the total number of layers in three `layer_warp` should be dividable by 6, that is the depth of `resnet_cifar10` should satisfy $(depth - 2) % 6 == 0$.
 
 ```python
-def resnet_cifar10(ipt, depth=56):
+def resnet_cifar10(ipt, depth=32):
     # depth should be one of 20, 32, 44, 56, 110, 1202
     assert (depth - 2) % 6 == 0
     n = (depth - 2) / 6
     nStages = {16, 64, 128}
-    conv1 = conv_bn_layer(ipt,
-        ch_in=3,
-        ch_out=16,
-        filter_size=3,
-        stride=1,
-        padding=1)
+    conv1 = conv_bn_layer(
+        ipt, ch_in=3, ch_out=16, filter_size=3, stride=1, padding=1)
     res1 = layer_warp(basicblock, conv1, 16, n, 1)
     res2 = layer_warp(basicblock, res1, 32, n, 2)
     res3 = layer_warp(basicblock, res2, 64, n, 2)
-    pool = img_pool_layer(input=res3,
-                         pool_size=8,
-                         stride=1,
-                         pool_type=AvgPooling())
+    pool = paddle.layer.img_pool(
+        input=res3, pool_size=8, stride=1, pool_type=paddle.pooling.Avg())
     return pool
 ```
 
 ## Model Training
 
-We can train the model by running the script train.sh, which specifies config file, device type, number of threads, number of passes, path to the trained models, etc,
+### Define Parameters
 
-``` bash
-sh train.sh
-```
+First, we create the model parameters according to the previous model configuration `cost`.
 
-Here is an example script `train.sh`:
-
-```bash
-#cfg=models/resnet.py
-cfg=models/vgg.py
-output=output
-log=train.log
-
-paddle train \
-    --config=$cfg \
-    --use_gpu=true \
-    --trainer_count=1 \
-    --log_period=100 \
-    --num_passes=300 \
-    --save_dir=$output \
-    2>&1 | tee $log
+```python
+# Create parameters
+parameters = paddle.parameters.create(cost)
 ```
 
-- `--config=$cfg` : specifies config file. The default is `models/vgg.py`.
-- `--use_gpu=true` : uses GPU for training. If use CPU，set it to be false.
-- `--trainer_count=1` : specifies the number of threads or GPUs.
-- `--log_period=100` : specifies the number of batches between two logs.
-- `--save_dir=$output` : specifies the path for saving trained models.
+### Create Trainer
 
-Here is an example log after training for one pass. The average error rates are 0.79958 on training set and 0.7858 on validation set.
+Before jumping into creating a training module, algorithm setting is also necessary.
+Here we specified `Momentum` optimization algorithm via `paddle.optimizer`.
 
-```text
-TrainerInternal.cpp:165]  Batch=300 samples=38400 AvgCost=2.07708 CurrentCost=1.96158 Eval: classification_error_evaluator=0.81151  CurrentEval: classification_error_evaluator=0.789297
-TrainerInternal.cpp:181]  Pass=0 Batch=391 samples=50000 AvgCost=2.03348 Eval: classification_error_evaluator=0.79958
-Tester.cpp:115]  Test samples=10000 cost=1.99246 Eval: classification_error_evaluator=0.7858
+```python
+# Create optimizer
+momentum_optimizer = paddle.optimizer.Momentum(
+    momentum=0.9,
+    regularization=paddle.optimizer.L2Regularization(rate=0.0002 * 128),
+    learning_rate=0.1 / 128.0,
+    learning_rate_decay_a=0.1,
+    learning_rate_decay_b=50000 * 100,
+    learning_rate_schedule='discexp',
+    batch_size=128)
+
+# Create trainer
+trainer = paddle.trainer.SGD(cost=cost,
+                             parameters=parameters,
+                             update_equation=momentum_optimizer)
 ```
 
-Figure 12 shows the curve of training error rate, which indicates it converges at Pass 200 with error rate 8.54%.
+The learning rate adjustment policy can be defined with variables `learning_rate_decay_a`($a$), `learning_rate_decay_b`($b$) and `learning_rate_schedule`. In this example, discrete exponential method is used for adjusting learning rate. The formula is as follows,
+$$  lr = lr_{0} * a^ {\lfloor \frac{n}{ b}\rfloor} $$
+where $n$ is the number of processed samples, $lr_{0}$ is the learning_rate.
 
-<p align="center">
-<img src="image/plot_en.png" width="400" ><br/>
-Figure 12. The error rate of VGG model on CIFAR10
-</p>
+### Training
 
-## Model Application
+`cifar.train10()` will yield records during each pass, after shuffling, a batch input is generated for training.
 
-After training is done, the model from each pass is saved in `output/pass-%05d`. For example, the model of Pass 300 is saved in `output/pass-00299`. The script `classify.py` can be used to extract features and to classify an image. The default config file of this script is `models/vgg.py`.
+```python
+reader=paddle.batch(
+    paddle.reader.shuffle(
+        paddle.dataset.cifar.train10(), buf_size=50000),
+        batch_size=128)
+```
+
+`feeding` is devoted to specifying the correspondence between each yield record and `paddle.layer.data`. For instance,
+ the first column of data generated by `cifar.train10()` corresponds to image layer's feature.
 
+```python
+feeding={'image': 0,
+         'label': 1}
+```
 
-### Prediction
+Callback function `event_handler` will be called during training when a pre-defined event happens.
 
-We can run the following script to predict the category of an image. The default device is GPU. If to use CPU, set `-c`.
 
-```bash
-python classify.py --job=predict --model=output/pass-00299 --data=image/dog.png # -c
+```python
+# event handler to track training and testing process
+def event_handler(event):
+    if isinstance(event, paddle.event.EndIteration):
+        if event.batch_id % 100 == 0:
+            print "\nPass %d, Batch %d, Cost %f, %s" % (
+                event.pass_id, event.batch_id, event.cost, event.metrics)
+        else:
+            sys.stdout.write('.')
+            sys.stdout.flush()
+    if isinstance(event, paddle.event.EndPass):
+        result = trainer.test(
+            reader=paddle.batch(
+                paddle.dataset.cifar.test10(), batch_size=128),
+            feeding=feeding)
+        print "\nTest with Pass %d, %s" % (event.pass_id, result.metrics)
 ```
 
-Here is the result：
+Finally, we can invoke `trainer.train` to start training:
 
-```text
-Label of image/dog.png is: 5
+```python
+trainer.train(
+    reader=reader,
+    num_passes=200,
+    event_handler=event_handler,
+    feeding=feeding)
 ```
 
-### Feature Extraction
+Here is an example log after training for one pass. The average error rates are 0.6875 on the training set and 0.8852 on the validation set.
 
-We can run the following command to extract features from an image. Here `job` should be `extract` and the default layer is the first convolutional layer. Figure 13 shows the 64 feature maps output from the first convolutional layer of the VGG model.
-
-```bash
-python classify.py --job=extract --model=output/pass-00299 --data=image/dog.png # -c
+```text
+Pass 0, Batch 0, Cost 2.473182, {'classification_error_evaluator': 0.9140625}
+...................................................................................................
+Pass 0, Batch 100, Cost 1.913076, {'classification_error_evaluator': 0.78125}
+...................................................................................................
+Pass 0, Batch 200, Cost 1.783041, {'classification_error_evaluator': 0.7421875}
+...................................................................................................
+Pass 0, Batch 300, Cost 1.668833, {'classification_error_evaluator': 0.6875}
+..........................................................................................
+Test with Pass 0, {'classification_error_evaluator': 0.885200023651123}
 ```
 
+Figure 12 shows the curve of training error rate, which indicates it converges at Pass 200 with error rate 8.54%.
 <p align="center">
-<img src="image/fea_conv0.png" width="500"><br/>
-Figre 13. Visualization of convolution layer feature maps
+<img src="image/plot_en.png" width="400" ><br/>
+Figure 12. The error rate of VGG model on CIFAR10
 </p>
 
+
+After training is done, the model from each pass is saved in `output/pass-%05d`. For example, the model of Pass 300 is saved in `output/pass-00299`.
+
 ## Conclusion
 
 Traditional image classification methods involve multiple stages of processing and the framework is very complicated. In contrast, CNN models can be trained end-to-end with significant increase of classification accuracy. In this chapter, we introduce three models -- VGG, GoogleNet, ResNet, provide PaddlePaddle config files for training VGG and ResNet on CIFAR10, and explain how to perform prediction and feature extraction using PaddlePaddle API. For other datasets such as ImageNet, the procedure for config and training are the same and you are welcome to give it a try.
@@ -547,4 +488,4 @@ Traditional image classification methods involve multiple stages of processing a
 [22] http://cs231n.github.io/classification/
 
 <br/>
-<a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/4.0/"><img alt="知识共享许可协议" style="border-width:0" src="https://i.creativecommons.org/l/by-nc-sa/4.0/88x31.png" /></a><br /><span xmlns:dct="http://purl.org/dc/terms/" href="http://purl.org/dc/dcmitype/Text" property="dct:title" rel="dct:type">本教程</span> 由 <a xmlns:cc="http://creativecommons.org/ns#" href="http://book.paddlepaddle.org" property="cc:attributionName" rel="cc:attributionURL">PaddlePaddle</a> 创作，采用 <a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/4.0/">知识共享 署名-非商业性使用-相同方式共享 4.0 国际 许可协议</a>进行许可。
+This tutorial is contributed by <a xmlns:cc="http://creativecommons.org/ns#" href="http://book.paddlepaddle.org" property="cc:attributionName" rel="cc:attributionURL">PaddlePaddle</a>, and licensed under a <a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/4.0/">Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License</a>.

image_classification/README.ipynb

+{
+  "cells": [
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "# 图像分类\n",
+        "\n",
+        "本教程源代码目录在[book/image_classification](https://github.com/PaddlePaddle/book/tree/develop/image_classification)， 初次使用请参考PaddlePaddle[安装教程](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/getstarted/build_and_install/docker_install_cn.rst)。\n",
+        "\n",
+        "## 背景介绍\n",
+        "\n",
+        "图像相比文字能够提供更加生动、容易理解及更具艺术感的信息，是人们转递与交换信息的重要来源。在本教程中，我们专注于图像识别领域的一个重要问题，即图像分类。\n",
+        "\n",
+        "图像分类是根据图像的语义信息将不同类别图像区分开来，是计算机视觉中重要的基本问题，也是图像检测、图像分割、物体跟踪、行为分析等其他高层视觉任务的基础。图像分类在很多领域有广泛应用，包括安防领域的人脸识别和智能视频分析等，交通领域的交通场景识别，互联网领域基于内容的图像检索和相册自动归类，医学领域的图像识别等。\n",
+        "\n",
+        "\n",
+        "一般来说，图像分类通过手工特征或特征学习方法对整个图像进行全部描述，然后使用分类器判别物体类别，因此如何提取图像的特征至关重要。在深度学习算法之前使用较多的是基于词袋(Bag of Words)模型的物体分类方法。词袋方法从自然语言处理中引入，即一句话可以用一个装了词的袋子表示其特征，袋子中的词为句子中的单词、短语或字。对于图像而言，词袋方法需要构建字典。最简单的词袋模型框架可以设计为**底层特征抽取**、**特征编码**、**分类器设计**三个过程。\n",
+        "\n",
+        "而基于深度学习的图像分类方法，可以通过有监督或无监督的方式**学习**层次化的特征描述，从而取代了手工设计或选择图像特征的工作。深度学习模型中的卷积神经网络(Convolution Neural Network, CNN)近年来在图像领域取得了惊人的成绩，CNN直接利用图像像素信息作为输入，最大程度上保留了输入图像的所有信息，通过卷积操作进行特征的提取和高层抽象，模型输出直接是图像识别的结果。这种基于\"输入-输出\"直接端到端的学习方法取得了非常好的效果，得到了广泛的应用。\n",
+        "\n",
+        "本教程主要介绍图像分类的深度学习模型，以及如何使用PaddlePaddle训练CNN模型。\n",
+        "\n",
+        "## 效果展示\n",
+        "\n",
+        "图像分类包括通用图像分类、细粒度图像分类等。图1展示了通用图像分类效果，即模型可以正确识别图像上的主要物体。\n",
+        "\n",
+        "\u003cp align=\"center\"\u003e\n",
+        "\u003cimg src=\"image/dog_cat.png \"  width=\"350\" \u003e\u003cbr/\u003e\n",
+        "图1. 通用图像分类展示\n",
+        "\u003c/p\u003e\n",
+        "\n",
+        "\n",
+        "图2展示了细粒度图像分类-花卉识别的效果，要求模型可以正确识别花的类别。\n",
+        "\n",
+        "\n",
+        "\u003cp align=\"center\"\u003e\n",
+        "\u003cimg src=\"image/flowers.png\" width=\"400\" \u003e\u003cbr/\u003e\n",
+        "图2. 细粒度图像分类展示\n",
+        "\u003c/p\u003e\n",
+        "\n",
+        "\n",
+        "一个好的模型既要对不同类别识别正确，同时也应该能够对不同视角、光照、背景、变形或部分遮挡的图像正确识别(这里我们统一称作图像扰动)。图3展示了一些图像的扰动，较好的模型会像聪明的人类一样能够正确识别。\n",
+        "\n",
+        "\u003cp align=\"center\"\u003e\n",
+        "\u003cimg src=\"image/variations.png\" width=\"550\" \u003e\u003cbr/\u003e\n",
+        "图3. 扰动图片展示[22]\n",
+        "\u003c/p\u003e\n",
+        "\n",
+        "## 模型概览\n",
+        "\n",
+        "图像识别领域大量的研究成果都是建立在[PASCAL VOC](http://host.robots.ox.ac.uk/pascal/VOC/)、[ImageNet](http://image-net.org/)等公开的数据集上，很多图像识别算法通常在这些数据集上进行测试和比较。PASCAL VOC是2005年发起的一个视觉挑战赛，ImageNet是2010年发起的大规模视觉识别竞赛(ILSVRC)的数据集，在本章中我们基于这些竞赛的一些论文介绍图像分类模型。\n",
+        "\n",
+        "在2012年之前的传统图像分类方法可以用背景描述中提到的三步完成，但通常完整建立图像识别模型一般包括底层特征学习、特征编码、空间约束、分类器设计、模型融合等几个阶段。\n",
+        "  1). **底层特征提取**: 通常从图像中按照固定步长、尺度提取大量局部特征描述。常用的局部特征包括SIFT(Scale-Invariant Feature Transform, 尺度不变特征转换) \\[[1](#参考文献)\\]、HOG(Histogram of Oriented Gradient, 方向梯度直方图) \\[[2](#参考文献)\\]、LBP(Local Bianray Pattern, 局部二值模式) \\[[3](#参考文献)\\] 等，一般也采用多种特征描述子，防止丢失过多的有用信息。\n",
+        "  2). **特征编码**: 底层特征中包含了大量冗余与噪声，为了提高特征表达的鲁棒性，需要使用一种特征变换算法对底层特征进行编码，称作特征编码。常用的特征编码包括向量量化编码 \\[[4](#参考文献)\\]、稀疏编码 \\[[5](#参考文献)\\]、局部线性约束编码 \\[[6](#参考文献)\\]、Fisher向量编码 \\[[7](#参考文献)\\] 等。\n",
+        "  3). **空间特征约束**: 特征编码之后一般会经过空间特征约束，也称作**特征汇聚**。特征汇聚是指在一个空间范围内，对每一维特征取最大值或者平均值，可以获得一定特征不变形的特征表达。金字塔特征匹配是一种常用的特征聚会方法，这种方法提出将图像均匀分块，在分块内做特征汇聚。\n",
+        "  4). **通过分类器分类**: 经过前面步骤之后一张图像可以用一个固定维度的向量进行描述，接下来就是经过分类器对图像进行分类。通常使用的分类器包括SVM(Support Vector Machine, 支持向量机)、随机森林等。而使用核方法的SVM是最为广泛的分类器，在传统图像分类任务上性能很好。\n",
+        "\n",
+        "这种方法在PASCAL VOC竞赛中的图像分类算法中被广泛使用 \\[[18](#参考文献)\\]。[NEC实验室](http://www.nec-labs.com/)在ILSVRC2010中采用SIFT和LBP特征，两个非线性编码器以及SVM分类器获得图像分类的冠军 \\[[8](#参考文献)\\]。\n",
+        "\n",
+        "Alex Krizhevsky在2012年ILSVRC提出的CNN模型 \\[[9](#参考文献)\\] 取得了历史性的突破，效果大幅度超越传统方法，获得了ILSVRC2012冠军，该模型被称作AlexNet。这也是首次将深度学习用于大规模图像分类中。从AlexNet之后，涌现了一系列CNN模型，不断地在ImageNet上刷新成绩，如图4展示。随着模型变得越来越深以及精妙的结构设计，Top-5的错误率也越来越低，降到了3.5%附近。而在同样的ImageNet数据集上，人眼的辨识错误率大概在5.1%，也就是目前的深度学习模型的识别能力已经超过了人眼。\n",
+        "\n",
+        "\u003cp align=\"center\"\u003e\n",
+        "\u003cimg src=\"image/ilsvrc.png\" width=\"500\" \u003e\u003cbr/\u003e\n",
+        "图4. ILSVRC图像分类Top-5错误率\n",
+        "\u003c/p\u003e\n",
+        "\n",
+        "### CNN\n",
+        "\n",
+        "传统CNN包含卷积层、全连接层等组件，并采用softmax多类别分类器和多类交叉熵损失函数，一个典型的卷积神经网络如图5所示，我们先介绍用来构造CNN的常见组件。\n",
+        "\n",
+        "\u003cp align=\"center\"\u003e\n",
+        "\u003cimg src=\"image/lenet.png\"\u003e\u003cbr/\u003e\n",
+        "图5. CNN网络示例[20]\n",
+        "\u003c/p\u003e\n",
+        "\n",
+        "- 卷积层(convolution layer): 执行卷积操作提取底层到高层的特征，发掘出图片局部关联性质和空间不变性质。\n",
+        "- 池化层(pooling layer): 执行降采样操作。通过取卷积输出特征图中局部区块的最大值(max-pooling)或者均值(avg-pooling)。降采样也是图像处理中常见的一种操作，可以过滤掉一些不重要的高频信息。\n",
+        "- 全连接层(fully-connected layer，或者fc layer): 输入层到隐藏层的神经元是全部连接的。\n",
+        "- 非线性变化: 卷积层、全连接层后面一般都会接非线性变化层，例如Sigmoid、Tanh、ReLu等来增强网络的表达能力，在CNN里最常使用的为ReLu激活函数。\n",
+        "- Dropout \\[[10](#参考文献)\\] : 在模型训练阶段随机让一些隐层节点权重不工作，提高网络的泛化能力，一定程度上防止过拟合。\n",
+        "\n",
+        "另外，在训练过程中由于每层参数不断更新，会导致下一次输入分布发生变化，这样导致训练过程需要精心设计超参数。如2015年Sergey Ioffe和Christian Szegedy提出了Batch Normalization (BN)算法 \\[[14](#参考文献)\\] 中，每个batch对网络中的每一层特征都做归一化，使得每层分布相对稳定。BN算法不仅起到一定的正则作用，而且弱化了一些超参数的设计。经过实验证明，BN算法加速了模型收敛过程，在后来较深的模型中被广泛使用。\n",
+        "\n",
+        "接下来我们主要介绍VGG，GoogleNet和ResNet网络结构。\n",
+        "\n",
+        "### VGG\n",
+        "\n",
+        "牛津大学VGG(Visual Geometry Group)组在2014年ILSVRC提出的模型被称作VGG模型 \\[[11](#参考文献)\\] 。该模型相比以往模型进一步加宽和加深了网络结构，它的核心是五组卷积操作，每两组之间做Max-Pooling空间降维。同一组内采用多次连续的3X3卷积，卷积核的数目由较浅组的64增多到最深组的512，同一组内的卷积核数目是一样的。卷积之后接两层全连接层，之后是分类层。由于每组内卷积层的不同，有11、13、16、19层这几种模型，下图展示一个16层的网络结构。VGG模型结构相对简洁，提出之后也有很多文章基于此模型进行研究，如在ImageNet上首次公开超过人眼识别的模型\\[[19](#参考文献)\\]就是借鉴VGG模型的结构。\n",
+        "\n",
+        "\u003cp align=\"center\"\u003e\n",
+        "\u003cimg src=\"image/vgg16.png\" width=\"750\" \u003e\u003cbr/\u003e\n",
+        "图6. 基于ImageNet的VGG16模型\n",
+        "\u003c/p\u003e\n",
+        "\n",
+        "### GoogleNet\n",
+        "\n",
+        "GoogleNet \\[[12](#参考文献)\\] 在2014年ILSVRC的获得了冠军，在介绍该模型之前我们先来了解NIN(Network in Network)模型 \\[[13](#参考文献)\\] 和Inception模块，因为GoogleNet模型由多组Inception模块组成，模型设计借鉴了NIN的一些思想。\n",
+        "\n",
+        "NIN模型主要有两个特点：1) 引入了多层感知卷积网络(Multi-Layer Perceptron Convolution, MLPconv)代替一层线性卷积网络。MLPconv是一个微小的多层卷积网络，即在线性卷积后面增加若干层1x1的卷积，这样可以提取出高度非线性特征。2) 传统的CNN最后几层一般都是全连接层，参数较多。而NIN模型设计最后一层卷积层包含类别维度大小的特征图，然后采用全局均值池化(Avg-Pooling)替代全连接层，得到类别维度大小的向量，再进行分类。这种替代全连接层的方式有利于减少参数。\n",
+        "\n",
+        "Inception模块如下图7所示，图(a)是最简单的设计，输出是3个卷积层和一个池化层的特征拼接。这种设计的缺点是池化层不会改变特征通道数，拼接后会导致特征的通道数较大，经过几层这样的模块堆积后，通道数会越来越大，导致参数和计算量也随之增大。为了改善这个缺点，图(b)引入3个1x1卷积层进行降维，所谓的降维就是减少通道数，同时如NIN模型中提到的1x1卷积也可以修正线性特征。\n",
+        "\n",
+        "\u003cp align=\"center\"\u003e\n",
+        "\u003cimg src=\"image/inception.png\" width=\"800\" \u003e\u003cbr/\u003e\n",
+        "图7. Inception模块\n",
+        "\u003c/p\u003e\n",
+        "\n",
+        "GoogleNet由多组Inception模块堆积而成。另外，在网络最后也没有采用传统的多层全连接层，而是像NIN网络一样采用了均值池化层；但与NIN不同的是，池化层后面接了一层到类别数映射的全连接层。除了这两个特点之外，由于网络中间层特征也很有判别性，GoogleNet在中间层添加了两个辅助分类器，在后向传播中增强梯度并且增强正则化，而整个网络的损失函数是这个三个分类器的损失加权求和。\n",
+        "\n",
+        "GoogleNet整体网络结构如图8所示，总共22层网络：开始由3层普通的卷积组成；接下来由三组子网络组成，第一组子网络包含2个Inception模块，第二组包含5个Inception模块，第三组包含2个Inception模块；然后接均值池化层、全连接层。\n",
+        "\n",
+        "\u003cp align=\"center\"\u003e\n",
+        "\u003cimg src=\"image/googlenet.jpeg\" \u003e\u003cbr/\u003e\n",
+        "图8. GoogleNet[12]\n",
+        "\u003c/p\u003e\n",
+        "\n",
+        "\n",
+        "上面介绍的是GoogleNet第一版模型(称作GoogleNet-v1)。GoogleNet-v2 \\[[14](#参考文献)\\] 引入BN层；GoogleNet-v3 \\[[16](#参考文献)\\] 对一些卷积层做了分解，进一步提高网络非线性能力和加深网络；GoogleNet-v4 \\[[17](#参考文献)\\] 引入下面要讲的ResNet设计思路。从v1到v4每一版的改进都会带来准确度的提升，介于篇幅，这里不再详细介绍v2到v4的结构。\n",
+        "\n",
+        "\n",
+        "### ResNet\n",
+        "\n",
+        "ResNet(Residual Network) \\[[15](#参考文献)\\] 是2015年ImageNet图像分类、图像物体定位和图像物体检测比赛的冠军。针对训练卷积神经网络时加深网络导致准确度下降的问题，ResNet提出了采用残差学习。在已有设计思路(BN, 小卷积核，全卷积网络)的基础上，引入了残差模块。每个残差模块包含两条路径，其中一条路径是输入特征的直连通路，另一条路径对该特征做两到三次卷积操作得到该特征的残差，最后再将两条路径上的特征相加。\n",
+        "\n",
+        "残差模块如图9所示，左边是基本模块连接方式，由两个输出通道数相同的3x3卷积组成。右边是瓶颈模块(Bottleneck)连接方式，之所以称为瓶颈，是因为上面的1x1卷积用来降维(图示例即256-\u003e64)，下面的1x1卷积用来升维(图示例即64-\u003e256)，这样中间3x3卷积的输入和输出通道数都较小(图示例即64-\u003e64)。\n",
+        "\n",
+        "\u003cp align=\"center\"\u003e\n",
+        "\u003cimg src=\"image/resnet_block.jpg\" width=\"400\"\u003e\u003cbr/\u003e\n",
+        "图9. 残差模块\n",
+        "\u003c/p\u003e\n",
+        "\n",
+        "图10展示了50、101、152层网络连接示意图，使用的是瓶颈模块。这三个模型的区别在于每组中残差模块的重复次数不同(见图右上角)。ResNet训练收敛较快，成功的训练了上百乃至近千层的卷积神经网络。\n",
+        "\n",
+        "\u003cp align=\"center\"\u003e\n",
+        "\u003cimg src=\"image/resnet.png\"\u003e\u003cbr/\u003e\n",
+        "图10. 基于ImageNet的ResNet模型\n",
+        "\u003c/p\u003e\n",
+        "\n",
+        "\n",
+        "## 数据准备\n",
+        "\n",
+        "通用图像分类公开的标准数据集常用的有[CIFAR](\u003chttps://www.cs.toronto.edu/~kriz/cifar.html)、[ImageNet](http://image-net.org/)、[COCO](http://mscoco.org/)等，常用的细粒度图像分类数据集包括[CUB-200-2011](http://www.vision.caltech.edu/visipedia/CUB-200-2011.html)、[Stanford Dog](http://vision.stanford.edu/aditya86/ImageNetDogs/)、[Oxford-flowers](http://www.robots.ox.ac.uk/~vgg/data/flowers/)等。其中ImageNet数据集规模相对较大，如[模型概览](#模型概览)一章所讲，大量研究成果基于ImageNet。ImageNet数据从2010年来稍有变化，常用的是ImageNet-2012数据集，该数据集包含1000个类别：训练集包含1,281,167张图片，每个类别数据732至1300张不等，验证集包含50,000张图片，平均每个类别50张图片。\n",
+        "\n",
+        "由于ImageNet数据集较大，下载和训练较慢，为了方便大家学习，我们使用[CIFAR10](\u003chttps://www.cs.toronto.edu/~kriz/cifar.html\u003e)数据集。CIFAR10数据集包含60,000张32x32的彩色图片，10个类别，每个类包含6,000张。其中50,000张图片作为训练集，10000张作为测试集。图11从每个类别中随机抽取了10张图片，展示了所有的类别。\n",
+        "\n",
+        "\u003cp align=\"center\"\u003e\n",
+        "\u003cimg src=\"image/cifar.png\" width=\"350\"\u003e\u003cbr/\u003e\n",
+        "图11. CIFAR10数据集[21]\n",
+        "\u003c/p\u003e\n",
+        "\n",
+        "Paddle API提供了自动加载cifar数据集模块 `paddle.dataset.cifar`。\n",
+        "\n",
+        "通过输入`python train.py`，就可以开始训练模型了，以下小节将详细介绍`train.py`的相关内容。\n",
+        "\n",
+        "### 模型结构\n",
+        "\n",
+        "#### Paddle 初始化\n",
+        "\n",
+        "通过 `paddle.init`，初始化Paddle是否使用GPU，trainer的数目等等。\n",
+        "\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "metadata": {
+        "editable": true
+      },
+      "source": [
+        "import sys\n",
+        "import paddle.v2 as paddle\n",
+        "from vgg import vgg_bn_drop\n",
+        "from resnet import resnet_cifar10\n",
+        "\n",
+        "# PaddlePaddle init\n",
+        "paddle.init(use_gpu=False, trainer_count=1)\n"
+      ],
+      "outputs": [
+        {
+          "name": "stdout",
+          "output_type": "stream",
+          "text": [
+            "\n"
+          ]
+        }
+      ],
+      "execution_count": 1
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "\n",
+        "本教程中我们提供了VGG和ResNet两个模型的配置。\n",
+        "\n",
+        "#### VGG\n",
+        "\n",
+        "首先介绍VGG模型结构，由于CIFAR10图片大小和数量相比ImageNet数据小很多，因此这里的模型针对CIFAR10数据做了一定的适配。卷积部分引入了BN和Dropout操作。\n",
+        "\n",
+        "1. 定义数据输入及其维度\n",
+        "\n",
+        "    网络输入定义为 `data_layer` (数据层)，在图像分类中即为图像像素信息。CIFRAR10是RGB 3通道32x32大小的彩色图，因此输入数据大小为3072(3x32x32)，类别大小为10，即10分类。\n",
+        "\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "metadata": {
+        "editable": true
+      },
+      "source": [
+        "    datadim = 3 * 32 * 32\n",
+        "    classdim = 10\n",
+        "\n",
+        "    image = paddle.layer.data(\n",
+        "        name=\"image\", type=paddle.data_type.dense_vector(datadim))\n"
+      ],
+      "outputs": [
+        {
+          "name": "stdout",
+          "output_type": "stream",
+          "text": [
+            "\n"
+          ]
+        }
+      ],
+      "execution_count": 1
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "\n",
+        "2. 定义VGG网络核心模块\n",
+        "\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "metadata": {
+        "editable": true
+      },
+      "source": [
+        "    net = vgg_bn_drop(image)\n"
+      ],
+      "outputs": [
+        {
+          "name": "stdout",
+          "output_type": "stream",
+          "text": [
+            "\n"
+          ]
+        }
+      ],
+      "execution_count": 1
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "    VGG核心模块的输入是数据层，`vgg_bn_drop` 定义了16层VGG结构，每层卷积后面引入BN层和Dropout层，详细的定义如下：\n",
+        "\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "metadata": {
+        "editable": true
+      },
+      "source": [
+        "    def vgg_bn_drop(input):\n",
+        "        def conv_block(ipt, num_filter, groups, dropouts, num_channels=None):\n",
+        "            return paddle.networks.img_conv_group(\n",
+        "                input=ipt,\n",
+        "                num_channels=num_channels,\n",
+        "                pool_size=2,\n",
+        "                pool_stride=2,\n",
+        "                conv_num_filter=[num_filter] * groups,\n",
+        "                conv_filter_size=3,\n",
+        "                conv_act=paddle.activation.Relu(),\n",
+        "                conv_with_batchnorm=True,\n",
+        "                conv_batchnorm_drop_rate=dropouts,\n",
+        "                pool_type=paddle.pooling.Max())\n",
+        "\n",
+        "        conv1 = conv_block(input, 64, 2, [0.3, 0], 3)\n",
+        "        conv2 = conv_block(conv1, 128, 2, [0.4, 0])\n",
+        "        conv3 = conv_block(conv2, 256, 3, [0.4, 0.4, 0])\n",
+        "        conv4 = conv_block(conv3, 512, 3, [0.4, 0.4, 0])\n",
+        "        conv5 = conv_block(conv4, 512, 3, [0.4, 0.4, 0])\n",
+        "\n",
+        "        drop = paddle.layer.dropout(input=conv5, dropout_rate=0.5)\n",
+        "        fc1 = paddle.layer.fc(input=drop, size=512, act=paddle.activation.Linear())\n",
+        "        bn = paddle.layer.batch_norm(\n",
+        "            input=fc1,\n",
+        "            act=paddle.activation.Relu(),\n",
+        "            layer_attr=paddle.attr.Extra(drop_rate=0.5))\n",
+        "        fc2 = paddle.layer.fc(input=bn, size=512, act=paddle.activation.Linear())\n",
+        "        return fc2\n"
+      ],
+      "outputs": [
+        {
+          "name": "stdout",
+          "output_type": "stream",
+          "text": [
+            "\n"
+          ]
+        }
+      ],
+      "execution_count": 1
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "\n",
+        "    2.1. 首先定义了一组卷积网络，即conv_block。卷积核大小为3x3，池化窗口大小为2x2，窗口滑动大小为2，groups决定每组VGG模块是几次连续的卷积操作，dropouts指定Dropout操作的概率。所使用的`img_conv_group`是在`paddle.networks`中预定义的模块，由若干组 `Conv-\u003eBN-\u003eReLu-\u003eDropout` 和 一组 `Pooling` 组成，\n",
+        "\n",
+        "    2.2. 五组卷积操作，即 5个conv_block。 第一、二组采用两次连续的卷积操作。第三、四、五组采用三次连续的卷积操作。每组最后一个卷积后面Dropout概率为0，即不使用Dropout操作。\n",
+        "\n",
+        "    2.3. 最后接两层512维的全连接。\n",
+        "\n",
+        "3. 定义分类器\n",
+        "\n",
+        "    通过上面VGG网络提取高层特征，然后经过全连接层映射到类别维度大小的向量，再通过Softmax归一化得到每个类别的概率，也可称作分类器。\n",
+        "\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "metadata": {
+        "editable": true
+      },
+      "source": [
+        "    out = paddle.layer.fc(input=net,\n",
+        "                          size=classdim,\n",
+        "                          act=paddle.activation.Softmax())\n"
+      ],
+      "outputs": [
+        {
+          "name": "stdout",
+          "output_type": "stream",
+          "text": [
+            "\n"
+          ]
+        }
+      ],
+      "execution_count": 1
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "\n",
+        "4. 定义损失函数和网络输出\n",
+        "\n",
+        "    在有监督训练中需要输入图像对应的类别信息，同样通过`paddle.layer.data`来定义。训练中采用多类交叉熵作为损失函数，并作为网络的输出，预测阶段定义网络的输出为分类器得到的概率信息。\n",
+        "\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "metadata": {
+        "editable": true
+      },
+      "source": [
+        "    lbl = paddle.layer.data(\n",
+        "        name=\"label\", type=paddle.data_type.integer_value(classdim))\n",
+        "    cost = paddle.layer.classification_cost(input=out, label=lbl)\n"
+      ],
+      "outputs": [
+        {
+          "name": "stdout",
+          "output_type": "stream",
+          "text": [
+            "\n"
+          ]
+        }
+      ],
+      "execution_count": 1
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "\n",
+        "### ResNet\n",
+        "\n",
+        "ResNet模型的第1、3、4步和VGG模型相同，这里不再介绍。主要介绍第2步即CIFAR10数据集上ResNet核心模块。\n",
+        "\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "metadata": {
+        "editable": true
+      },
+      "source": [
+        "net = resnet_cifar10(image, depth=56)\n"
+      ],
+      "outputs": [
+        {
+          "name": "stdout",
+          "output_type": "stream",
+          "text": [
+            "\n"
+          ]
+        }
+      ],
+      "execution_count": 1
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "\n",
+        "先介绍`resnet_cifar10`中的一些基本函数，再介绍网络连接过程。\n",
+        "\n",
+        "  - `conv_bn_layer` : 带BN的卷积层。\n",
+        "  - `shortcut` : 残差模块的\"直连\"路径，\"直连\"实际分两种形式：残差模块输入和输出特征通道数不等时，采用1x1卷积的升维操作；残差模块输入和输出通道相等时，采用直连操作。\n",
+        "  - `basicblock` : 一个基础残差模块，即图9左边所示，由两组3x3卷积组成的路径和一条\"直连\"路径组成。\n",
+        "  - `bottleneck` : 一个瓶颈残差模块，即图9右边所示，由上下1x1卷积和中间3x3卷积组成的路径和一条\"直连\"路径组成。\n",
+        "  - `layer_warp` : 一组残差模块，由若干个残差模块堆积而成。每组中第一个残差模块滑动窗口大小与其他可以不同，以用来减少特征图在垂直和水平方向的大小。\n",
+        "\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "metadata": {
+        "editable": true
+      },
+      "source": [
+        "def conv_bn_layer(input,\n",
+        "                  ch_out,\n",
+        "                  filter_size,\n",
+        "                  stride,\n",
+        "                  padding,\n",
+        "                  active_type=paddle.activation.Relu(),\n",
+        "                  ch_in=None):\n",
+        "    tmp = paddle.layer.img_conv(\n",
+        "        input=input,\n",
+        "        filter_size=filter_size,\n",
+        "        num_channels=ch_in,\n",
+        "        num_filters=ch_out,\n",
+        "        stride=stride,\n",
+        "        padding=padding,\n",
+        "        act=paddle.activation.Linear(),\n",
+        "        bias_attr=False)\n",
+        "    return paddle.layer.batch_norm(input=tmp, act=active_type)\n",
+        "\n",
+        "def shortcut(ipt, n_in, n_out, stride):\n",
+        "    if n_in != n_out:\n",
+        "        return conv_bn_layer(ipt, n_out, 1, stride, 0,\n",
+        "                             paddle.activation.Linear())\n",
+        "    else:\n",
+        "        return ipt\n",
+        "\n",
+        "def basicblock(ipt, ch_out, stride):\n",
+        "    ch_in = ch_out * 2\n",
+        "    tmp = conv_bn_layer(ipt, ch_out, 3, stride, 1)\n",
+        "    tmp = conv_bn_layer(tmp, ch_out, 3, 1, 1, paddle.activation.Linear())\n",
+        "    short = shortcut(ipt, ch_in, ch_out, stride)\n",
+        "    return paddle.layer.addto(input=[tmp, short], act=paddle.activation.Relu())\n",
+        "\n",
+        "def layer_warp(block_func, ipt, features, count, stride):\n",
+        "    tmp = block_func(ipt, features, stride)\n",
+        "    for i in range(1, count):\n",
+        "        tmp = block_func(tmp, features, 1)\n",
+        "    return tmp\n"
+      ],
+      "outputs": [
+        {
+          "name": "stdout",
+          "output_type": "stream",
+          "text": [
+            "\n"
+          ]
+        }
+      ],
+      "execution_count": 1
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "\n",
+        "`resnet_cifar10` 的连接结构主要有以下几个过程。\n",
+        "\n",
+        "1. 底层输入连接一层 `conv_bn_layer`，即带BN的卷积层。\n",
+        "2. 然后连接3组残差模块即下面配置3组 `layer_warp` ，每组采用图 10 左边残差模块组成。\n",
+        "3. 最后对网络做均值池化并返回该层。\n",
+        "\n",
+        "注意：除过第一层卷积层和最后一层全连接层之外，要求三组 `layer_warp` 总的含参层数能够被6整除，即 `resnet_cifar10` 的 depth 要满足 $(depth - 2) % 6 == 0$ 。\n",
+        "\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "metadata": {
+        "editable": true
+      },
+      "source": [
+        "def resnet_cifar10(ipt, depth=32):\n",
+        "    # depth should be one of 20, 32, 44, 56, 110, 1202\n",
+        "    assert (depth - 2) % 6 == 0\n",
+        "    n = (depth - 2) / 6\n",
+        "    nStages = {16, 64, 128}\n",
+        "    conv1 = conv_bn_layer(\n",
+        "        ipt, ch_in=3, ch_out=16, filter_size=3, stride=1, padding=1)\n",
+        "    res1 = layer_warp(basicblock, conv1, 16, n, 1)\n",
+        "    res2 = layer_warp(basicblock, res1, 32, n, 2)\n",
+        "    res3 = layer_warp(basicblock, res2, 64, n, 2)\n",
+        "    pool = paddle.layer.img_pool(\n",
+        "        input=res3, pool_size=8, stride=1, pool_type=paddle.pooling.Avg())\n",
+        "    return pool\n"
+      ],
+      "outputs": [
+        {
+          "name": "stdout",
+          "output_type": "stream",
+          "text": [
+            "\n"
+          ]
+        }
+      ],
+      "execution_count": 1
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "\n",
+        "## 训练模型\n",
+        "\n",
+        "### 定义参数\n",
+        "\n",
+        "首先依据模型配置的`cost`定义模型参数。\n",
+        "\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "metadata": {
+        "editable": true
+      },
+      "source": [
+        "# Create parameters\n",
+        "parameters = paddle.parameters.create(cost)\n"
+      ],
+      "outputs": [
+        {
+          "name": "stdout",
+          "output_type": "stream",
+          "text": [
+            "\n"
+          ]
+        }
+      ],
+      "execution_count": 1
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "\n",
+        "可以打印参数名字，如果在网络配置中没有指定名字，则默认生成。\n",
+        "\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "metadata": {
+        "editable": true
+      },
+      "source": [
+        "print parameters.keys()\n"
+      ],
+      "outputs": [
+        {
+          "name": "stdout",
+          "output_type": "stream",
+          "text": [
+            "\n"
+          ]
+        }
+      ],
+      "execution_count": 1
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "\n",
+        "### 构造训练(Trainer)\n",
+        "\n",
+        "根据网络拓扑结构和模型参数来构造出trainer用来训练，在构造时还需指定优化方法，这里使用最基本的Momentum方法，同时设定了学习率、正则等。\n",
+        "\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "metadata": {
+        "editable": true
+      },
+      "source": [
+        "# Create optimizer\n",
+        "momentum_optimizer = paddle.optimizer.Momentum(\n",
+        "    momentum=0.9,\n",
+        "    regularization=paddle.optimizer.L2Regularization(rate=0.0002 * 128),\n",
+        "    learning_rate=0.1 / 128.0,\n",
+        "    learning_rate_decay_a=0.1,\n",
+        "    learning_rate_decay_b=50000 * 100,\n",
+        "    learning_rate_schedule='discexp',\n",
+        "    batch_size=128)\n",
+        "\n",
+        "# Create trainer\n",
+        "trainer = paddle.trainer.SGD(cost=cost,\n",
+        "                             parameters=parameters,\n",
+        "                             update_equation=momentum_optimizer)\n"
+      ],
+      "outputs": [
+        {
+          "name": "stdout",
+          "output_type": "stream",
+          "text": [
+            "\n"
+          ]
+        }
+      ],
+      "execution_count": 1
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "\n",
+        "通过 `learning_rate_decay_a` (简写$a$） 、`learning_rate_decay_b` (简写$b$) 和 `learning_rate_schedule` 指定学习率调整策略，这里采用离散指数的方式调节学习率，计算公式如下， $n$ 代表已经处理过的累计总样本数，$lr_{0}$ 即为 `settings` 里设置的 `learning_rate`。\n",
+        "\n",
+        "$$  lr = lr_{0} * a^ {\\lfloor \\frac{n}{ b}\\rfloor} $$\n",
+        "\n",
+        "\n",
+        "### 训练\n",
+        "\n",
+        "cifar.train10()每次产生一条样本，在完成shuffle和batch之后，作为训练的输入。\n",
+        "\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "metadata": {
+        "editable": true
+      },
+      "source": [
+        "reader=paddle.batch(\n",
+        "    paddle.reader.shuffle(\n",
+        "        paddle.dataset.cifar.train10(), buf_size=50000),\n",
+        "        batch_size=128)\n"
+      ],
+      "outputs": [
+        {
+          "name": "stdout",
+          "output_type": "stream",
+          "text": [
+            "\n"
+          ]
+        }
+      ],
+      "execution_count": 1
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "\n",
+        "通过`feeding`来指定每一个数据和`paddle.layer.data`的对应关系。例如: `cifar.train10()`产生数据的第0列对应image层的特征。\n",
+        "\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "metadata": {
+        "editable": true
+      },
+      "source": [
+        "feeding={'image': 0,\n",
+        "         'label': 1}\n"
+      ],
+      "outputs": [
+        {
+          "name": "stdout",
+          "output_type": "stream",
+          "text": [
+            "\n"
+          ]
+        }
+      ],
+      "execution_count": 1
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "\n",
+        "可以使用`event_handler`回调函数来观察训练过程，或进行测试等, 该回调函数是`trainer.train`函数里设定。\n",
+        "\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "metadata": {
+        "editable": true
+      },
+      "source": [
+        "# End batch and end pass event handler\n",
+        "def event_handler(event):\n",
+        "    if isinstance(event, paddle.event.EndIteration):\n",
+        "        if event.batch_id % 100 == 0:\n",
+        "            print \"\\nPass %d, Batch %d, Cost %f, %s\" % (\n",
+        "                event.pass_id, event.batch_id, event.cost, event.metrics)\n",
+        "        else:\n",
+        "            sys.stdout.write('.')\n",
+        "            sys.stdout.flush()\n",
+        "    if isinstance(event, paddle.event.EndPass):\n",
+        "        result = trainer.test(\n",
+        "            reader=paddle.batch(\n",
+        "                paddle.dataset.cifar.test10(), batch_size=128),\n",
+        "            feeding=feeding)\n",
+        "        print \"\\nTest with Pass %d, %s\" % (event.pass_id, result.metrics)\n"
+      ],
+      "outputs": [
+        {
+          "name": "stdout",
+          "output_type": "stream",
+          "text": [
+            "\n"
+          ]
+        }
+      ],
+      "execution_count": 1
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "\n",
+        "通过`trainer.train`函数训练:\n",
+        "\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "metadata": {
+        "editable": true
+      },
+      "source": [
+        "trainer.train(\n",
+        "    reader=reader,\n",
+        "    num_passes=200,\n",
+        "    event_handler=event_handler,\n",
+        "    feeding=feeding)\n"
+      ],
+      "outputs": [
+        {
+          "name": "stdout",
+          "output_type": "stream",
+          "text": [
+            "\n"
+          ]
+        }
+      ],
+      "execution_count": 1
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "\n",
+        "一轮训练log示例如下所示，经过1个pass， 训练集上平均error为0.6875 ，测试集上平均error为0.8852 。\n",
+        "\n",
+        "```text\n",
+        "Pass 0, Batch 0, Cost 2.473182, {'classification_error_evaluator': 0.9140625}\n",
+        "...................................................................................................\n",
+        "Pass 0, Batch 100, Cost 1.913076, {'classification_error_evaluator': 0.78125}\n",
+        "...................................................................................................\n",
+        "Pass 0, Batch 200, Cost 1.783041, {'classification_error_evaluator': 0.7421875}\n",
+        "...................................................................................................\n",
+        "Pass 0, Batch 300, Cost 1.668833, {'classification_error_evaluator': 0.6875}\n",
+        "..........................................................................................\n",
+        "Test with Pass 0, {'classification_error_evaluator': 0.885200023651123}\n",
+        "```\n",
+        "\n",
+        "图12是训练的分类错误率曲线图，运行到第200个pass后基本收敛，最终得到测试集上分类错误率为8.54%。\n",
+        "\n",
+        "\u003cp align=\"center\"\u003e\n",
+        "\u003cimg src=\"image/plot.png\" width=\"400\" \u003e\u003cbr/\u003e\n",
+        "图12. CIFAR10数据集上VGG模型的分类错误率\n",
+        "\u003c/p\u003e\n",
+        "\n",
+        "\n",
+        "## 总结\n",
+        "\n",
+        "传统图像分类方法由多个阶段构成，框架较为复杂，而端到端的CNN模型结构可一步到位，而且大幅度提升了分类准确率。本文我们首先介绍VGG、GoogleNet、ResNet三个经典的模型；然后基于CIFAR10数据集，介绍如何使用PaddlePaddle配置和训练CNN模型，尤其是VGG和ResNet模型；最后介绍如何使用PaddlePaddle的API接口对图片进行预测和特征提取。对于其他数据集比如ImageNet，配置和训练流程是同样的，大家可以自行进行实验。\n",
+        "\n",
+        "\n",
+        "## 参考文献\n",
+        "\n",
+        "[1] D. G. Lowe, [Distinctive image features from scale-invariant keypoints](http://www.cs.ubc.ca/~lowe/papers/ijcv04.pdf). IJCV, 60(2):91-110, 2004.\n",
+        "\n",
+        "[2] N. Dalal, B. Triggs, [Histograms of Oriented Gradients for Human Detection](http://vision.stanford.edu/teaching/cs231b_spring1213/papers/CVPR05_DalalTriggs.pdf), Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2005.\n",
+        "\n",
+        "[3] Ahonen, T., Hadid, A., and Pietikinen, M. (2006). [Face description with local binary patterns: Application to face recognition](http://ieeexplore.ieee.org/document/1717463/). PAMI, 28.\n",
+        "\n",
+        "[4] J. Sivic, A. Zisserman, [Video Google: A Text Retrieval Approach to Object Matching in Videos](http://www.robots.ox.ac.uk/~vgg/publications/papers/sivic03.pdf), Proc. Ninth Int'l Conf. Computer Vision, pp. 1470-1478, 2003.\n",
+        "\n",
+        "[5] B. Olshausen, D. Field, [Sparse Coding with an Overcomplete Basis Set: A Strategy Employed by V1?](http://redwood.psych.cornell.edu/papers/olshausen_field_1997.pdf), Vision Research, vol. 37, pp. 3311-3325, 1997.\n",
+        "\n",
+        "[6] Wang, J., Yang, J., Yu, K., Lv, F., Huang, T., and Gong, Y. (2010). [Locality-constrained Linear Coding for image classification](http://ieeexplore.ieee.org/abstract/document/5540018/). In CVPR.\n",
+        "\n",
+        "[7] Perronnin, F., Sánchez, J., \u0026 Mensink, T. (2010). [Improving the fisher kernel for large-scale image classification](http://dl.acm.org/citation.cfm?id=1888101). In ECCV (4).\n",
+        "\n",
+        "[8] Lin, Y., Lv, F., Cao, L., Zhu, S., Yang, M., Cour, T., Yu, K., and Huang, T. (2011). [Large-scale image clas- sification: Fast feature extraction and SVM training](http://ieeexplore.ieee.org/document/5995477/). In CVPR.\n",
+        "\n",
+        "[9] Krizhevsky, A., Sutskever, I., and Hinton, G. (2012). [ImageNet classification with deep convolutional neu- ral networks](http://www.cs.toronto.edu/~kriz/imagenet_classification_with_deep_convolutional.pdf). In NIPS.\n",
+        "\n",
+        "[10] G.E. Hinton, N. Srivastava, A. Krizhevsky, I. Sutskever, and R.R. Salakhutdinov. [Improving neural networks by preventing co-adaptation of feature detectors](https://arxiv.org/abs/1207.0580). arXiv preprint arXiv:1207.0580, 2012.\n",
+        "\n",
+        "[11] K. Chatfield, K. Simonyan, A. Vedaldi, A. Zisserman. [Return of the Devil in the Details: Delving Deep into Convolutional Nets](https://arxiv.org/abs/1405.3531). BMVC, 2014。\n",
+        "\n",
+        "[12] Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., Rabinovich, A., [Going deeper with convolutions](https://arxiv.org/abs/1409.4842). In: CVPR. (2015)\n",
+        "\n",
+        "[13] Lin, M., Chen, Q., and Yan, S. [Network in network](https://arxiv.org/abs/1312.4400). In Proc. ICLR, 2014.\n",
+        "\n",
+        "[14] S. Ioffe and C. Szegedy. [Batch normalization: Accelerating deep network training by reducing internal covariate shift](https://arxiv.org/abs/1502.03167). In ICML, 2015.\n",
+        "\n",
+        "[15] K. He, X. Zhang, S. Ren, J. Sun. [Deep Residual Learning for Image Recognition](https://arxiv.org/abs/1512.03385). CVPR 2016.\n",
+        "\n",
+        "[16] Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z. [Rethinking the incep-tion architecture for computer vision](https://arxiv.org/abs/1512.00567). In: CVPR. (2016).\n",
+        "\n",
+        "[17] Szegedy, C., Ioffe, S., Vanhoucke, V. [Inception-v4, inception-resnet and the impact of residual connections on learning](https://arxiv.org/abs/1602.07261). arXiv:1602.07261 (2016).\n",
+        "\n",
+        "[18] Everingham, M., Eslami, S. M. A., Van Gool, L., Williams, C. K. I., Winn, J. and Zisserman, A. [The Pascal Visual Object Classes Challenge: A Retrospective]((http://link.springer.com/article/10.1007/s11263-014-0733-5)). International Journal of Computer Vision, 111(1), 98-136, 2015.\n",
+        "\n",
+        "[19] He, K., Zhang, X., Ren, S., and Sun, J. [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://arxiv.org/abs/1502.01852). ArXiv e-prints, February 2015.\n",
+        "\n",
+        "[20] http://deeplearning.net/tutorial/lenet.html\n",
+        "\n",
+        "[21] https://www.cs.toronto.edu/~kriz/cifar.html\n",
+        "\n",
+        "[22] http://cs231n.github.io/classification/\n",
+        "\n",
+        "\u003cbr/\u003e\n",
+        "\u003ca rel=\"license\" href=\"http://creativecommons.org/licenses/by-nc-sa/4.0/\"\u003e\u003cimg alt=\"知识共享许可协议\" style=\"border-width:0\" src=\"https://i.creativecommons.org/l/by-nc-sa/4.0/88x31.png\" /\u003e\u003c/a\u003e\u003cbr /\u003e\u003cspan xmlns:dct=\"http://purl.org/dc/terms/\" href=\"http://purl.org/dc/dcmitype/Text\" property=\"dct:title\" rel=\"dct:type\"\u003e本教程\u003c/span\u003e 由 \u003ca xmlns:cc=\"http://creativecommons.org/ns#\" href=\"http://book.paddlepaddle.org\" property=\"cc:attributionName\" rel=\"cc:attributionURL\"\u003ePaddlePaddle\u003c/a\u003e 创作，采用 \u003ca rel=\"license\" href=\"http://creativecommons.org/licenses/by-nc-sa/4.0/\"\u003e知识共享 署名-非商业性使用-相同方式共享 4.0 国际 许可协议\u003c/a\u003e进行许可。\n"
+      ]
+    }
+  ],
+  "metadata": {
+    "kernelspec": {
+      "display_name": "Python 3",
+      "language": "python",
+      "name": "python3"
+    },
+    "language_info": {
+      "codemirror_mode": {
+        "name": "ipython",
+        "version": 3
+      },
+      "file_extension": ".py",
+      "mimetype": "text/x-python",
+      "name": "python",
+      "nbconvert_exporter": "python",
+      "pygments_lexer": "ipython3",
+      "version": "3.6.0"
+    }
+  },
+  "nbformat": 4,
+  "nbformat_minor": 0
+}

image_classification/README.md

-图像分类
-=======
+# 图像分类
 
-本教程源代码目录在[book/image_classification](https://github.com/PaddlePaddle/book/tree/develop/image_classification)， 初次使用请参考PaddlePaddle[安装教程](http://www.paddlepaddle.org/doc_cn/build_and_install/index.html)。
+本教程源代码目录在[book/image_classification](https://github.com/PaddlePaddle/book/tree/develop/image_classification)， 初次使用请参考PaddlePaddle[安装教程](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/getstarted/build_and_install/docker_install_cn.rst)。
 
 ## 背景介绍
 
@@ -173,24 +172,24 @@ paddle.init(use_gpu=False, trainer_count=1)
 
 1. 定义数据输入及其维度
 
-	网络输入定义为 `data_layer` (数据层)，在图像分类中即为图像像素信息。CIFRAR10是RGB 3通道32x32大小的彩色图，因此输入数据大小为3072(3x32x32)，类别大小为10，即10分类。
+    网络输入定义为 `data_layer` (数据层)，在图像分类中即为图像像素信息。CIFRAR10是RGB 3通道32x32大小的彩色图，因此输入数据大小为3072(3x32x32)，类别大小为10，即10分类。
 
-	```python
+    ```python
     datadim = 3 * 32 * 32
     classdim = 10
 
     image = paddle.layer.data(
         name="image", type=paddle.data_type.dense_vector(datadim))
-	```
+    ```
 
 2. 定义VGG网络核心模块
 
-	```python
-	net = vgg_bn_drop(image)
-	```
-	VGG核心模块的输入是数据层，`vgg_bn_drop` 定义了16层VGG结构，每层卷积后面引入BN层和Dropout层，详细的定义如下：
+    ```python
+    net = vgg_bn_drop(image)
+    ```
+    VGG核心模块的输入是数据层，`vgg_bn_drop` 定义了16层VGG结构，每层卷积后面引入BN层和Dropout层，详细的定义如下：
 
-	```python
+    ```python
     def vgg_bn_drop(input):
         def conv_block(ipt, num_filter, groups, dropouts, num_channels=None):
             return paddle.networks.img_conv_group(
@@ -219,40 +218,40 @@ paddle.init(use_gpu=False, trainer_count=1)
             layer_attr=paddle.attr.Extra(drop_rate=0.5))
         fc2 = paddle.layer.fc(input=bn, size=512, act=paddle.activation.Linear())
         return fc2
-	```
+    ```
 
-	2.1. 首先定义了一组卷积网络，即conv_block。卷积核大小为3x3，池化窗口大小为2x2，窗口滑动大小为2，groups决定每组VGG模块是几次连续的卷积操作，dropouts指定Dropout操作的概率。所使用的`img_conv_group`是在`paddle.networks`中预定义的模块，由若干组 `Conv->BN->ReLu->Dropout` 和 一组 `Pooling` 组成，
+    2.1. 首先定义了一组卷积网络，即conv_block。卷积核大小为3x3，池化窗口大小为2x2，窗口滑动大小为2，groups决定每组VGG模块是几次连续的卷积操作，dropouts指定Dropout操作的概率。所使用的`img_conv_group`是在`paddle.networks`中预定义的模块，由若干组 `Conv->BN->ReLu->Dropout` 和 一组 `Pooling` 组成，
 
-	2.2. 五组卷积操作，即 5个conv_block。 第一、二组采用两次连续的卷积操作。第三、四、五组采用三次连续的卷积操作。每组最后一个卷积后面Dropout概率为0，即不使用Dropout操作。
+    2.2. 五组卷积操作，即 5个conv_block。 第一、二组采用两次连续的卷积操作。第三、四、五组采用三次连续的卷积操作。每组最后一个卷积后面Dropout概率为0，即不使用Dropout操作。
 
-	2.3. 最后接两层512维的全连接。
+    2.3. 最后接两层512维的全连接。
 
 3. 定义分类器
 
-	通过上面VGG网络提取高层特征，然后经过全连接层映射到类别维度大小的向量，再通过Softmax归一化得到每个类别的概率，也可称作分类器。
+    通过上面VGG网络提取高层特征，然后经过全连接层映射到类别维度大小的向量，再通过Softmax归一化得到每个类别的概率，也可称作分类器。
 
-	```python
+    ```python
     out = paddle.layer.fc(input=net,
                           size=classdim,
                           act=paddle.activation.Softmax())
-	```
+    ```
 
 4. 定义损失函数和网络输出
 
-	在有监督训练中需要输入图像对应的类别信息，同样通过`paddle.layer.data`来定义。训练中采用多类交叉熵作为损失函数，并作为网络的输出，预测阶段定义网络的输出为分类器得到的概率信息。
+    在有监督训练中需要输入图像对应的类别信息，同样通过`paddle.layer.data`来定义。训练中采用多类交叉熵作为损失函数，并作为网络的输出，预测阶段定义网络的输出为分类器得到的概率信息。
 
-	```python
+    ```python
     lbl = paddle.layer.data(
         name="label", type=paddle.data_type.integer_value(classdim))
     cost = paddle.layer.classification_cost(input=out, label=lbl)
-	```
+    ```
 
 ### ResNet
 
 ResNet模型的第1、3、4步和VGG模型相同，这里不再介绍。主要介绍第2步即CIFAR10数据集上ResNet核心模块。
 
 ```python
-net = resnet_cifar10(data, depth=56)
+net = resnet_cifar10(image, depth=56)
 ```
 
 先介绍`resnet_cifar10`中的一些基本函数，再介绍网络连接过程。
@@ -375,7 +374,7 @@ $$  lr = lr_{0} * a^ {\lfloor \frac{n}{ b}\rfloor} $$
 cifar.train10()每次产生一条样本，在完成shuffle和batch之后，作为训练的输入。
 
 ```python
-reader=paddle.reader.batch(
+reader=paddle.batch(
     paddle.reader.shuffle(
         paddle.dataset.cifar.train10(), buf_size=50000),
         batch_size=128)
@@ -402,10 +401,9 @@ def event_handler(event):
             sys.stdout.flush()
     if isinstance(event, paddle.event.EndPass):
         result = trainer.test(
-            reader=paddle.reader.batch(
+            reader=paddle.batch(
                 paddle.dataset.cifar.test10(), batch_size=128),
-            reader_dict={'image': 0,
-                         'label': 1})
+            feeding=feeding)
         print "\nTest with Pass %d, %s" % (event.pass_id, result.metrics)
 ```
 

image_classification/deprecated/README.md

 图像分类
 =======
 
-本教程源代码目录在[book/image_classification](https://github.com/PaddlePaddle/book/tree/develop/image_classification)， 初次使用请参考PaddlePaddle[安装教程](http://www.paddlepaddle.org/doc_cn/build_and_install/index.html)。
+本教程源代码目录在[book/image_classification](https://github.com/PaddlePaddle/book/tree/develop/image_classification)， 初次使用请参考PaddlePaddle[安装教程](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/getstarted/build_and_install/docker_install_cn.rst)。
 
 ## 背景介绍
 
@@ -244,77 +244,77 @@ $$  lr = lr_{0} * a^ {\lfloor \frac{n}{ b}\rfloor} $$
 
 1. 定义数据输入及其维度
 
-	网络输入定义为 `data_layer` (数据层)，在图像分类中即为图像像素信息。CIFRAR10是RGB 3通道32x32大小的彩色图，因此输入数据大小为3072(3x32x32)，类别大小为10，即10分类。
+    网络输入定义为 `data_layer` (数据层)，在图像分类中即为图像像素信息。CIFRAR10是RGB 3通道32x32大小的彩色图，因此输入数据大小为3072(3x32x32)，类别大小为10，即10分类。
 
-	```python
-	datadim = 3 * 32 * 32
-	classdim = 10
-	data = data_layer(name='image', size=datadim)
-	```
+    ```python
+    datadim = 3 * 32 * 32
+    classdim = 10
+    data = data_layer(name='image', size=datadim)
+    ```
 
 2. 定义VGG网络核心模块
 
-	```python
-	net = vgg_bn_drop(data)
-	```
-	VGG核心模块的输入是数据层，`vgg_bn_drop` 定义了16层VGG结构，每层卷积后面引入BN层和Dropout层，详细的定义如下：
-
-	```python
-	def vgg_bn_drop(input, num_channels):
-	    def conv_block(ipt, num_filter, groups, dropouts, num_channels_=None):
-	        return img_conv_group(
-	            input=ipt,
-	            num_channels=num_channels_,
-	            pool_size=2,
-	            pool_stride=2,
-	            conv_num_filter=[num_filter] * groups,
-	            conv_filter_size=3,
-	            conv_act=ReluActivation(),
-	            conv_with_batchnorm=True,
-	            conv_batchnorm_drop_rate=dropouts,
-	            pool_type=MaxPooling())
-
-	    conv1 = conv_block(input, 64, 2, [0.3, 0], 3)
-	    conv2 = conv_block(conv1, 128, 2, [0.4, 0])
-	    conv3 = conv_block(conv2, 256, 3, [0.4, 0.4, 0])
-	    conv4 = conv_block(conv3, 512, 3, [0.4, 0.4, 0])
-	    conv5 = conv_block(conv4, 512, 3, [0.4, 0.4, 0])
-
-	    drop = dropout_layer(input=conv5, dropout_rate=0.5)
-	    fc1 = fc_layer(input=drop, size=512, act=LinearActivation())
-	    bn = batch_norm_layer(
-	        input=fc1, act=ReluActivation(), layer_attr=ExtraAttr(drop_rate=0.5))
-	    fc2 = fc_layer(input=bn, size=512, act=LinearActivation())
-	    return fc2
-
-	```
-
-	2.1. 首先定义了一组卷积网络，即conv_block。卷积核大小为3x3，池化窗口大小为2x2，窗口滑动大小为2，groups决定每组VGG模块是几次连续的卷积操作，dropouts指定Dropout操作的概率。所使用的`img_conv_group`是在`paddle.trainer_config_helpers`中预定义的模块，由若干组 `Conv->BN->ReLu->Dropout` 和 一组 `Pooling` 组成，
-
-	2.2. 五组卷积操作，即 5个conv_block。 第一、二组采用两次连续的卷积操作。第三、四、五组采用三次连续的卷积操作。每组最后一个卷积后面Dropout概率为0，即不使用Dropout操作。
-
-	2.3. 最后接两层512维的全连接。
+    ```python
+    net = vgg_bn_drop(data)
+    ```
+    VGG核心模块的输入是数据层，`vgg_bn_drop` 定义了16层VGG结构，每层卷积后面引入BN层和Dropout层，详细的定义如下：
+
+    ```python
+    def vgg_bn_drop(input, num_channels):
+        def conv_block(ipt, num_filter, groups, dropouts, num_channels_=None):
+            return img_conv_group(
+                input=ipt,
+                num_channels=num_channels_,
+                pool_size=2,
+                pool_stride=2,
+                conv_num_filter=[num_filter] * groups,
+                conv_filter_size=3,
+                conv_act=ReluActivation(),
+                conv_with_batchnorm=True,
+                conv_batchnorm_drop_rate=dropouts,
+                pool_type=MaxPooling())
+
+        conv1 = conv_block(input, 64, 2, [0.3, 0], 3)
+        conv2 = conv_block(conv1, 128, 2, [0.4, 0])
+        conv3 = conv_block(conv2, 256, 3, [0.4, 0.4, 0])
+        conv4 = conv_block(conv3, 512, 3, [0.4, 0.4, 0])
+        conv5 = conv_block(conv4, 512, 3, [0.4, 0.4, 0])
+
+        drop = dropout_layer(input=conv5, dropout_rate=0.5)
+        fc1 = fc_layer(input=drop, size=512, act=LinearActivation())
+        bn = batch_norm_layer(
+            input=fc1, act=ReluActivation(), layer_attr=ExtraAttr(drop_rate=0.5))
+        fc2 = fc_layer(input=bn, size=512, act=LinearActivation())
+        return fc2
+
+    ```
+
+    2.1. 首先定义了一组卷积网络，即conv_block。卷积核大小为3x3，池化窗口大小为2x2，窗口滑动大小为2，groups决定每组VGG模块是几次连续的卷积操作，dropouts指定Dropout操作的概率。所使用的`img_conv_group`是在`paddle.trainer_config_helpers`中预定义的模块，由若干组 `Conv->BN->ReLu->Dropout` 和 一组 `Pooling` 组成，
+
+    2.2. 五组卷积操作，即 5个conv_block。 第一、二组采用两次连续的卷积操作。第三、四、五组采用三次连续的卷积操作。每组最后一个卷积后面Dropout概率为0，即不使用Dropout操作。
+
+    2.3. 最后接两层512维的全连接。
 
 3. 定义分类器
 
-	通过上面VGG网络提取高层特征，然后经过全连接层映射到类别维度大小的向量，再通过Softmax归一化得到每个类别的概率，也可称作分类器。
+    通过上面VGG网络提取高层特征，然后经过全连接层映射到类别维度大小的向量，再通过Softmax归一化得到每个类别的概率，也可称作分类器。
 
-	```python
-	out = fc_layer(input=net, size=class_num, act=SoftmaxActivation())
-	```
+    ```python
+    out = fc_layer(input=net, size=class_num, act=SoftmaxActivation())
+    ```
 
 4. 定义损失函数和网络输出
 
-	在有监督训练中需要输入图像对应的类别信息，同样通过`data_layer`来定义。训练中采用多类交叉熵作为损失函数，并作为网络的输出，预测阶段定义网络的输出为分类器得到的概率信息。
+    在有监督训练中需要输入图像对应的类别信息，同样通过`data_layer`来定义。训练中采用多类交叉熵作为损失函数，并作为网络的输出，预测阶段定义网络的输出为分类器得到的概率信息。
 
-	```python
-	if not is_predict:
-	    lbl = data_layer(name="label", size=class_num)
-	    cost = classification_cost(input=out, label=lbl)
-	    outputs(cost)
-	else:
-	    outputs(out)
-	```
+    ```python
+    if not is_predict:
+        lbl = data_layer(name="label", size=class_num)
+        cost = classification_cost(input=out, label=lbl)
+        outputs(cost)
+    else:
+        outputs(out)
+    ```
 
 ### ResNet
 

image_classification/deprecated/classify.py

@@ -44,8 +44,9 @@ def vis_square(data, fname):
          (0, 1))  # add some space between filters
         + ((0, 0), ) *
         (data.ndim - 3))  # don't pad the last dimension (if there is one)
-    data = np.pad(data, padding, mode='constant',
-                  constant_values=1)  # pad with ones (white)
+    data = np.pad(
+        data, padding, mode='constant',
+        constant_values=1)  # pad with ones (white)
     # tile the filters into an image
     data = data.reshape((n, n) + data.shape[1:]).transpose((0, 2, 1, 3) + tuple(
         range(4, data.ndim + 1)))

image_classification/index.en.html

@@ -43,7 +43,7 @@
 Image Classification
 =======================
 
-The source code of this chapter is in [book/image_classification](https://github.com/PaddlePaddle/book/tree/develop/image_classification). For the first-time users, please refer to PaddlePaddle[Installation Tutorial](http://www.paddlepaddle.org/doc_cn/build_and_install/index.html) for installation instructions.
+The source code of this chapter is in [book/image_classification](https://github.com/PaddlePaddle/book/tree/develop/image_classification). For the first-time users, please refer to PaddlePaddle [Installation Tutorial](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/getstarted/build_and_install/docker_install_en.rst) for installation instructions.
 
 ## Background
 
@@ -177,146 +177,73 @@ Figure 10. ResNet model for ImageNet
 </p>
 
 
-## Data Preparation
-
-### Data description and downloading
+## Dataset
 
 Commonly used public datasets for image classification are CIFAR(https://www.cs.toronto.edu/~kriz/cifar.html), ImageNet(http://image-net.org/), COCO(http://mscoco.org/), etc. Those used for fine-grained image classification are CUB-200-2011(http://www.vision.caltech.edu/visipedia/CUB-200-2011.html), Stanford Dog(http://vision.stanford.edu/aditya86/ImageNetDogs/), Oxford-flowers(http://www.robots.ox.ac.uk/~vgg/data/flowers/), etc. Among them, ImageNet are the largest and most research results are reported on ImageNet as mentioned in Model Overview section. Since 2010, the data of Imagenet has gone through some changes. The commonly used ImageNet-2012 dataset contains 1000 categories. There are 1,281,167 training images, ranging from 732 to 1200 images per category, and 50,000 validation images with 50 images per category in average.
 
-Since ImageNet is too large to be downloaded and trained efficiently, we use CIFAR10 (https://www.cs.toronto.edu/~kriz/cifar.html) in this tutorial. The CIFAR-10 dataset consists of 60000 32x32 color images in 10 classes, with 6000 images per class. There are 50000 training images and 10000 test images. Figure 11 shows all the classes in CIFAR10 as well as 10 images randomly sampled from each category.
+Since ImageNet is too large to be downloaded and trained efficiently, we use CIFAR-10 (https://www.cs.toronto.edu/~kriz/cifar.html) in this tutorial. The CIFAR-10 dataset consists of 60000 32x32 color images in 10 classes, with 6000 images per class. There are 50000 training images and 10000 test images. Figure 11 shows all the classes in CIFAR-10 as well as 10 images randomly sampled from each category.
 
 <p align="center">
 <img src="image/cifar.png" width="350"><br/>
 Figure 11. CIFAR10 dataset[21]
 </p>
 
-The following command is used for downloading data and calculating the mean image used for data preprocessing.
-
-```bash
-./data/get_data.sh
-```
+ `paddle.datasets` package encapsulates multiple public datasets, including `cifar`, `imdb`, `mnist`, `moivelens` and `wmt14`, etc. There's no need for us to manually download and preprocess CIFAR-10.
 
-###  Data provider for PaddlePaddle
+After issuing a command `python train.py`, training will starting immediately. The details will be unpacked by the following sessions to see how it works.
 
-We use Python interface for providing data to PaddlePaddle. The following file dataprovider.py is a complete example for CIFAR10.
+## Model Structure
 
-- 'initializer' function performs initialization of dataprovider: loading the mean image, defining two input types -- image and label.
+### Initialize PaddlePaddle
 
-- 'process' function sends preprocessed data to PaddlePaddle. Data preprocessing performed in this function includes data perturbation, random horizontal flipping, deducting mean image from the raw image.
+We must import and initialize PaddlePaddle (enable/disable GPU, set the number of trainers, etc).
 
 ```python
-import numpy as np
-import cPickle
-from paddle.trainer.PyDataProvider2 import *
-
-def initializer(settings, mean_path, is_train, **kwargs):
-    settings.is_train = is_train
-    settings.input_size = 3 * 32 * 32
-    settings.mean = np.load(mean_path)['mean']
-    settings.input_types = {
-        'image': dense_vector(settings.input_size),
-        'label': integer_value(10)
-    }
-
-
-@provider(init_hook=initializer, pool_size=50000)
-def process(settings, file_list):
-    with open(file_list, 'r') as fdata:
-        for fname in fdata:
-            fo = open(fname.strip(), 'rb')
-            batch = cPickle.load(fo)
-            fo.close()
-            images = batch['data']
-            labels = batch['labels']
-            for im, lab in zip(images, labels):
-                if settings.is_train and np.random.randint(2):
-                    im = im.reshape(3, 32, 32)
-                    im = im[:,:,::-1]
-                    im = im.flatten()
-                im = im - settings.mean
-                yield {
-                    'image': im.astype('float32'),
-                    'label': int(lab)
-                }
-```
+import sys
+import paddle.v2 as paddle
 
-## Model Config
-
-### Data Definition
-
-In model config file, function `define_py_data_sources2` sets argument 'module' to dataprovider file for loading data, 'args' to mean image file. If the config file is used for prediction, then there is no need to set argument 'train_list'.
-
-```python
-from paddle.trainer_config_helpers import *
-
-is_predict = get_config_arg("is_predict", bool, False)
-if not is_predict:
-    define_py_data_sources2(
-        train_list='data/train.list',
-        test_list='data/test.list',
-        module='dataprovider',
-        obj='process',
-        args={'mean_path': 'data/mean.meta'})
-```
-
-### Algorithm Settings
-
-In model config file, function 'settings' specifies optimization algorithm, batch size, learning rate, momentum and L2 regularization.
-
-```python
-settings(
-    batch_size=128,
-    learning_rate=0.1 / 128.0,
-    learning_rate_decay_a=0.1,
-    learning_rate_decay_b=50000 * 100,
-    learning_rate_schedule='discexp',
-    learning_method=MomentumOptimizer(0.9),
-    regularization=L2Regularization(0.0005 * 128),)
+# PaddlePaddle init
+paddle.init(use_gpu=False, trainer_count=1)
 ```
 
-The learning rate adjustment policy can be defined with variables `learning_rate_decay_a`($a$), `learning_rate_decay_b`($b$) and `learning_rate_schedule`. In this example, discrete exponential method is used for adjusting learning rate. The formula is as follows,
-$$  lr = lr_{0} * a^ {\lfloor \frac{n}{ b}\rfloor} $$
-where $n$ is the number of processed samples, $lr_{0}$ is the learning_rate set in 'settings'.
-
-### Model Architecture
-
-Here we provide the cofig files for VGG and ResNet models.
+As alluded to in section [Model Overview](#model-overview), here we provide the implementations of both VGG and ResNet models.
 
-#### VGG
+### VGG
 
-First we define VGG network. Since the image size and amount of CIFAR10 are relatively small comparing to ImageNet, we uses a small version of VGG network for CIFAR10. Convolution groups incorporate BN and dropout operations.
+First, we use a VGG network. Since the image size and amount of CIFAR10 are relatively small comparing to ImageNet, we uses a small version of VGG network for CIFAR10. Convolution groups incorporate BN and dropout operations.
 
 1. Define input data and its dimension
 
-        The input to the network is defined as `data_layer`, or image pixels in the context of image classification. The images in CIFAR10 are 32x32 color images of three channels. Therefore, the size of the input data is 3072 (3x32x32), and the number of categories is 10.
+        The input to the network is defined as `paddle.layer.data`, or image pixels in the context of image classification. The images in CIFAR10 are 32x32 color images of three channels. Therefore, the size of the input data is 3072 (3x32x32), and the number of categories is 10.
 
     ```python
     datadim = 3 * 32 * 32
     classdim = 10
-    data = data_layer(name='image', size=datadim)
+    image = paddle.layer.data(
+        name="image", type=paddle.data_type.dense_vector(datadim))
     ```
 
 2. Define VGG main module
 
     ```python
-    net = vgg_bn_drop(data)
+    net = vgg_bn_drop(image)
     ```
-        The input to VGG main module is from data layer. `vgg_bn_drop` defines a 16-layer VGG network, with each convolutional layer followed by BN and dropout layers. Here is the definition in detail:
+        The input to VGG main module is from the data layer. `vgg_bn_drop` defines a 16-layer VGG network, with each convolutional layer followed by BN and dropout layers. Here is the definition in detail:
 
     ```python
-    def vgg_bn_drop(input, num_channels):
-        def conv_block(ipt, num_filter, groups, dropouts, num_channels_=None):
-            return img_conv_group(
+    def vgg_bn_drop(input):
+        def conv_block(ipt, num_filter, groups, dropouts, num_channels=None):
+            return paddle.networks.img_conv_group(
                 input=ipt,
-                num_channels=num_channels_,
+                num_channels=num_channels,
                 pool_size=2,
                 pool_stride=2,
                 conv_num_filter=[num_filter] * groups,
                 conv_filter_size=3,
-                conv_act=ReluActivation(),
+                conv_act=paddle.activation.Relu(),
                 conv_with_batchnorm=True,
                 conv_batchnorm_drop_rate=dropouts,
-                pool_type=MaxPooling())
+                pool_type=paddle.pooling.Max())
 
         conv1 = conv_block(input, 64, 2, [0.3, 0], 3)
         conv2 = conv_block(conv1, 128, 2, [0.4, 0])
@@ -324,16 +251,17 @@ First we define VGG network. Since the image size and amount of CIFAR10 are rela
         conv4 = conv_block(conv3, 512, 3, [0.4, 0.4, 0])
         conv5 = conv_block(conv4, 512, 3, [0.4, 0.4, 0])
 
-        drop = dropout_layer(input=conv5, dropout_rate=0.5)
-        fc1 = fc_layer(input=drop, size=512, act=LinearActivation())
-        bn = batch_norm_layer(
-            input=fc1, act=ReluActivation(), layer_attr=ExtraAttr(drop_rate=0.5))
-        fc2 = fc_layer(input=bn, size=512, act=LinearActivation())
+        drop = paddle.layer.dropout(input=conv5, dropout_rate=0.5)
+        fc1 = paddle.layer.fc(input=drop, size=512, act=paddle.activation.Linear())
+        bn = paddle.layer.batch_norm(
+            input=fc1,
+            act=paddle.activation.Relu(),
+            layer_attr=paddle.attr.Extra(drop_rate=0.5))
+        fc2 = paddle.layer.fc(input=bn, size=512, act=paddle.activation.Linear())
         return fc2
-
     ```
 
-        2.1. First defines a convolution block or conv_block. The default convolution kernel is 3x3, and the default pooling size is 2x2 with stride 2. Dropout specifies the probability in dropout operation. Function `img_conv_group` is defined in `paddle.trainer_config_helpers` consisting of a series of `Conv->BN->ReLu->Dropout` and a `Pooling`.
+        2.1. First defines a convolution block or conv_block. The default convolution kernel is 3x3, and the default pooling size is 2x2 with stride 2. Dropout specifies the probability in dropout operation. Function `img_conv_group` is defined in `paddle.networks` consisting of a series of `Conv->BN->ReLu->Dropout` and a `Pooling`.
 
 
         2.2. Five groups of convolutions. The first two groups perform two convolutions, while the last three groups perform three convolutions. The dropout rate of the last convolution in each group is set to 0, which means there is no dropout for this layer.
@@ -351,15 +279,12 @@ First we define VGG network. Since the image size and amount of CIFAR10 are rela
 
 4. Define Loss Function and Outputs
 
-        In the context of supervised learning, labels of training images are defined in `data_layer`, too. During training, cross-entropy is used as loss function and as the output of the network; During testing, the outputs are the probabilities calculated in the classifier.
+        In the context of supervised learning, labels of training images are defined in `paddle.layer.data`, too. During training, cross-entropy is used as loss function and as the output of the network; During testing, the outputs are the probabilities calculated in the classifier.
 
     ```python
-    if not is_predict:
-        lbl = data_layer(name="label", size=class_num)
-        cost = classification_cost(input=out, label=lbl)
-        outputs(cost)
-    else:
-        outputs(out)
+    lbl = paddle.layer.data(
+        name="label", type=paddle.data_type.integer_value(classdim))
+    cost = paddle.layer.classification_cost(input=out, label=lbl)
     ```
 
 ### ResNet
@@ -367,13 +292,13 @@ First we define VGG network. Since the image size and amount of CIFAR10 are rela
 The first, third and forth steps of a ResNet are the same as a VGG. The second one is the main module.
 
 ```python
-net = resnet_cifar10(data, depth=56)
+net = resnet_cifar10(data, depth=32)
 ```
 
 Here are some basic functions used in `resnet_cifar10`:
 
   - `conv_bn_layer` : convolutional layer followed by BN.
-  - `shortcut` : the shortcut branch in a residual block. There are two kinds of shortcuts: 1x1 convolution used when the number of channels between input and output are different; direct connection used otherwise.
+  - `shortcut` : the shortcut branch in a residual block. There are two kinds of shortcuts: 1x1 convolution used when the number of channels between input and output is different; direct connection used otherwise.
 
   - `basicblock` : a basic residual module as shown in the left of Figure 9, consisting of two sequential 3x3 convolutions and one "shortcut" branch.
   - `bottleneck` : a bottleneck module as shown in the right of Figure 9, consisting of a two 1x1 convolutions with one 3x3 convolution in between branch and a "shortcut" branch.
@@ -385,47 +310,38 @@ def conv_bn_layer(input,
                   filter_size,
                   stride,
                   padding,
-                  active_type=ReluActivation(),
+                  active_type=paddle.activation.Relu(),
                   ch_in=None):
-    tmp = img_conv_layer(
+    tmp = paddle.layer.img_conv(
         input=input,
         filter_size=filter_size,
         num_channels=ch_in,
         num_filters=ch_out,
         stride=stride,
         padding=padding,
-        act=LinearActivation(),
+        act=paddle.activation.Linear(),
         bias_attr=False)
-    return batch_norm_layer(input=tmp, act=active_type)
-
+    return paddle.layer.batch_norm(input=tmp, act=active_type)
 
 def shortcut(ipt, n_in, n_out, stride):
     if n_in != n_out:
-        return conv_bn_layer(ipt, n_out, 1, stride, 0, LinearActivation())
+        return conv_bn_layer(ipt, n_out, 1, stride, 0,
+                             paddle.activation.Linear())
     else:
         return ipt
 
 def basicblock(ipt, ch_out, stride):
-    ch_in = ipt.num_filters
+    ch_in = ch_out * 2
     tmp = conv_bn_layer(ipt, ch_out, 3, stride, 1)
-    tmp = conv_bn_layer(tmp, ch_out, 3, 1, 1, LinearActivation())
-    short = shortcut(ipt, ch_in, ch_out, stride)
-    return addto_layer(input=[ipt, short], act=ReluActivation())
-
-def bottleneck(ipt, ch_out, stride):
-    ch_in = ipt.num_filter
-    tmp = conv_bn_layer(ipt, ch_out, 1, stride, 0)
-    tmp = conv_bn_layer(tmp, ch_out, 3, 1, 1)
-    tmp = conv_bn_layer(tmp, ch_out * 4, 1, 1, 0, LinearActivation())
+    tmp = conv_bn_layer(tmp, ch_out, 3, 1, 1, paddle.activation.Linear())
     short = shortcut(ipt, ch_in, ch_out, stride)
-    return addto_layer(input=[ipt, short], act=ReluActivation())
+    return paddle.layer.addto(input=[tmp, short], act=paddle.activation.Relu())
 
 def layer_warp(block_func, ipt, features, count, stride):
     tmp = block_func(ipt, features, stride)
     for i in range(1, count):
         tmp = block_func(tmp, features, 1)
     return tmp
-
 ```
 
 The following are the components of `resnet_cifar10`:
@@ -437,106 +353,131 @@ The following are the components of `resnet_cifar10`:
 Note: besides the first convolutional layer and the last fully-connected layer, the total number of layers in three `layer_warp` should be dividable by 6, that is the depth of `resnet_cifar10` should satisfy $(depth - 2) % 6 == 0$.
 
 ```python
-def resnet_cifar10(ipt, depth=56):
+def resnet_cifar10(ipt, depth=32):
     # depth should be one of 20, 32, 44, 56, 110, 1202
     assert (depth - 2) % 6 == 0
     n = (depth - 2) / 6
     nStages = {16, 64, 128}
-    conv1 = conv_bn_layer(ipt,
-        ch_in=3,
-        ch_out=16,
-        filter_size=3,
-        stride=1,
-        padding=1)
+    conv1 = conv_bn_layer(
+        ipt, ch_in=3, ch_out=16, filter_size=3, stride=1, padding=1)
     res1 = layer_warp(basicblock, conv1, 16, n, 1)
     res2 = layer_warp(basicblock, res1, 32, n, 2)
     res3 = layer_warp(basicblock, res2, 64, n, 2)
-    pool = img_pool_layer(input=res3,
-                         pool_size=8,
-                         stride=1,
-                         pool_type=AvgPooling())
+    pool = paddle.layer.img_pool(
+        input=res3, pool_size=8, stride=1, pool_type=paddle.pooling.Avg())
     return pool
 ```
 
 ## Model Training
 
-We can train the model by running the script train.sh, which specifies config file, device type, number of threads, number of passes, path to the trained models, etc,
+### Define Parameters
 
-``` bash
-sh train.sh
-```
+First, we create the model parameters according to the previous model configuration `cost`.
 
-Here is an example script `train.sh`:
-
-```bash
-#cfg=models/resnet.py
-cfg=models/vgg.py
-output=output
-log=train.log
-
-paddle train \
-    --config=$cfg \
-    --use_gpu=true \
-    --trainer_count=1 \
-    --log_period=100 \
-    --num_passes=300 \
-    --save_dir=$output \
-    2>&1 | tee $log
+```python
+# Create parameters
+parameters = paddle.parameters.create(cost)
 ```
 
-- `--config=$cfg` : specifies config file. The default is `models/vgg.py`.
-- `--use_gpu=true` : uses GPU for training. If use CPU，set it to be false.
-- `--trainer_count=1` : specifies the number of threads or GPUs.
-- `--log_period=100` : specifies the number of batches between two logs.
-- `--save_dir=$output` : specifies the path for saving trained models.
+### Create Trainer
 
-Here is an example log after training for one pass. The average error rates are 0.79958 on training set and 0.7858 on validation set.
+Before jumping into creating a training module, algorithm setting is also necessary.
+Here we specified `Momentum` optimization algorithm via `paddle.optimizer`.
 
-```text
-TrainerInternal.cpp:165]  Batch=300 samples=38400 AvgCost=2.07708 CurrentCost=1.96158 Eval: classification_error_evaluator=0.81151  CurrentEval: classification_error_evaluator=0.789297
-TrainerInternal.cpp:181]  Pass=0 Batch=391 samples=50000 AvgCost=2.03348 Eval: classification_error_evaluator=0.79958
-Tester.cpp:115]  Test samples=10000 cost=1.99246 Eval: classification_error_evaluator=0.7858
+```python
+# Create optimizer
+momentum_optimizer = paddle.optimizer.Momentum(
+    momentum=0.9,
+    regularization=paddle.optimizer.L2Regularization(rate=0.0002 * 128),
+    learning_rate=0.1 / 128.0,
+    learning_rate_decay_a=0.1,
+    learning_rate_decay_b=50000 * 100,
+    learning_rate_schedule='discexp',
+    batch_size=128)
+
+# Create trainer
+trainer = paddle.trainer.SGD(cost=cost,
+                             parameters=parameters,
+                             update_equation=momentum_optimizer)
 ```
 
-Figure 12 shows the curve of training error rate, which indicates it converges at Pass 200 with error rate 8.54%.
+The learning rate adjustment policy can be defined with variables `learning_rate_decay_a`($a$), `learning_rate_decay_b`($b$) and `learning_rate_schedule`. In this example, discrete exponential method is used for adjusting learning rate. The formula is as follows,
+$$  lr = lr_{0} * a^ {\lfloor \frac{n}{ b}\rfloor} $$
+where $n$ is the number of processed samples, $lr_{0}$ is the learning_rate.
 
-<p align="center">
-<img src="image/plot_en.png" width="400" ><br/>
-Figure 12. The error rate of VGG model on CIFAR10
-</p>
+### Training
 
-## Model Application
+`cifar.train10()` will yield records during each pass, after shuffling, a batch input is generated for training.
 
-After training is done, the model from each pass is saved in `output/pass-%05d`. For example, the model of Pass 300 is saved in `output/pass-00299`. The script `classify.py` can be used to extract features and to classify an image. The default config file of this script is `models/vgg.py`.
+```python
+reader=paddle.batch(
+    paddle.reader.shuffle(
+        paddle.dataset.cifar.train10(), buf_size=50000),
+        batch_size=128)
+```
+
+`feeding` is devoted to specifying the correspondence between each yield record and `paddle.layer.data`. For instance,
+ the first column of data generated by `cifar.train10()` corresponds to image layer's feature.
 
+```python
+feeding={'image': 0,
+         'label': 1}
+```
 
-### Prediction
+Callback function `event_handler` will be called during training when a pre-defined event happens.
 
-We can run the following script to predict the category of an image. The default device is GPU. If to use CPU, set `-c`.
 
-```bash
-python classify.py --job=predict --model=output/pass-00299 --data=image/dog.png # -c
+```python
+# event handler to track training and testing process
+def event_handler(event):
+    if isinstance(event, paddle.event.EndIteration):
+        if event.batch_id % 100 == 0:
+            print "\nPass %d, Batch %d, Cost %f, %s" % (
+                event.pass_id, event.batch_id, event.cost, event.metrics)
+        else:
+            sys.stdout.write('.')
+            sys.stdout.flush()
+    if isinstance(event, paddle.event.EndPass):
+        result = trainer.test(
+            reader=paddle.batch(
+                paddle.dataset.cifar.test10(), batch_size=128),
+            feeding=feeding)
+        print "\nTest with Pass %d, %s" % (event.pass_id, result.metrics)
 ```
 
-Here is the result：
+Finally, we can invoke `trainer.train` to start training:
 
-```text
-Label of image/dog.png is: 5
+```python
+trainer.train(
+    reader=reader,
+    num_passes=200,
+    event_handler=event_handler,
+    feeding=feeding)
 ```
 
-### Feature Extraction
+Here is an example log after training for one pass. The average error rates are 0.6875 on the training set and 0.8852 on the validation set.
 
-We can run the following command to extract features from an image. Here `job` should be `extract` and the default layer is the first convolutional layer. Figure 13 shows the 64 feature maps output from the first convolutional layer of the VGG model.
-
-```bash
-python classify.py --job=extract --model=output/pass-00299 --data=image/dog.png # -c
+```text
+Pass 0, Batch 0, Cost 2.473182, {'classification_error_evaluator': 0.9140625}
+...................................................................................................
+Pass 0, Batch 100, Cost 1.913076, {'classification_error_evaluator': 0.78125}
+...................................................................................................
+Pass 0, Batch 200, Cost 1.783041, {'classification_error_evaluator': 0.7421875}
+...................................................................................................
+Pass 0, Batch 300, Cost 1.668833, {'classification_error_evaluator': 0.6875}
+..........................................................................................
+Test with Pass 0, {'classification_error_evaluator': 0.885200023651123}
 ```
 
+Figure 12 shows the curve of training error rate, which indicates it converges at Pass 200 with error rate 8.54%.
 <p align="center">
-<img src="image/fea_conv0.png" width="500"><br/>
-Figre 13. Visualization of convolution layer feature maps
+<img src="image/plot_en.png" width="400" ><br/>
+Figure 12. The error rate of VGG model on CIFAR10
 </p>
 
+
+After training is done, the model from each pass is saved in `output/pass-%05d`. For example, the model of Pass 300 is saved in `output/pass-00299`.
+
 ## Conclusion
 
 Traditional image classification methods involve multiple stages of processing and the framework is very complicated. In contrast, CNN models can be trained end-to-end with significant increase of classification accuracy. In this chapter, we introduce three models -- VGG, GoogleNet, ResNet, provide PaddlePaddle config files for training VGG and ResNet on CIFAR10, and explain how to perform prediction and feature extraction using PaddlePaddle API. For other datasets such as ImageNet, the procedure for config and training are the same and you are welcome to give it a try.
@@ -589,7 +530,7 @@ Traditional image classification methods involve multiple stages of processing a
 [22] http://cs231n.github.io/classification/
 
 <br/>
-<a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/4.0/"><img alt="知识共享许可协议" style="border-width:0" src="https://i.creativecommons.org/l/by-nc-sa/4.0/88x31.png" /></a><br /><span xmlns:dct="http://purl.org/dc/terms/" href="http://purl.org/dc/dcmitype/Text" property="dct:title" rel="dct:type">本教程</span> 由 <a xmlns:cc="http://creativecommons.org/ns#" href="http://book.paddlepaddle.org" property="cc:attributionName" rel="cc:attributionURL">PaddlePaddle</a> 创作，采用 <a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/4.0/">知识共享 署名-非商业性使用-相同方式共享 4.0 国际 许可协议</a>进行许可。
+This tutorial is contributed by <a xmlns:cc="http://creativecommons.org/ns#" href="http://book.paddlepaddle.org" property="cc:attributionName" rel="cc:attributionURL">PaddlePaddle</a>, and licensed under a <a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/4.0/">Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License</a>.
 
 </div>
 <!-- You can change the lines below now. -->

image_classification/index.html

@@ -40,10 +40,9 @@
 
 <!-- This block will be replaced by each markdown file content. Please do not change lines below.-->
 <div id="markdown" style='display:none'>
-图像分类
-=======
+# 图像分类
 
-本教程源代码目录在[book/image_classification](https://github.com/PaddlePaddle/book/tree/develop/image_classification)， 初次使用请参考PaddlePaddle[安装教程](http://www.paddlepaddle.org/doc_cn/build_and_install/index.html)。
+本教程源代码目录在[book/image_classification](https://github.com/PaddlePaddle/book/tree/develop/image_classification)， 初次使用请参考PaddlePaddle[安装教程](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/getstarted/build_and_install/docker_install_cn.rst)。
 
 ## 背景介绍
 
@@ -215,24 +214,24 @@ paddle.init(use_gpu=False, trainer_count=1)
 
 1. 定义数据输入及其维度
 
-	网络输入定义为 `data_layer` (数据层)，在图像分类中即为图像像素信息。CIFRAR10是RGB 3通道32x32大小的彩色图，因此输入数据大小为3072(3x32x32)，类别大小为10，即10分类。
+    网络输入定义为 `data_layer` (数据层)，在图像分类中即为图像像素信息。CIFRAR10是RGB 3通道32x32大小的彩色图，因此输入数据大小为3072(3x32x32)，类别大小为10，即10分类。
 
-	```python
+    ```python
     datadim = 3 * 32 * 32
     classdim = 10
 
     image = paddle.layer.data(
         name="image", type=paddle.data_type.dense_vector(datadim))
-	```
+    ```
 
 2. 定义VGG网络核心模块
 
-	```python
-	net = vgg_bn_drop(image)
-	```
-	VGG核心模块的输入是数据层，`vgg_bn_drop` 定义了16层VGG结构，每层卷积后面引入BN层和Dropout层，详细的定义如下：
+    ```python
+    net = vgg_bn_drop(image)
+    ```
+    VGG核心模块的输入是数据层，`vgg_bn_drop` 定义了16层VGG结构，每层卷积后面引入BN层和Dropout层，详细的定义如下：
 
-	```python
+    ```python
     def vgg_bn_drop(input):
         def conv_block(ipt, num_filter, groups, dropouts, num_channels=None):
             return paddle.networks.img_conv_group(
@@ -261,40 +260,40 @@ paddle.init(use_gpu=False, trainer_count=1)
             layer_attr=paddle.attr.Extra(drop_rate=0.5))
         fc2 = paddle.layer.fc(input=bn, size=512, act=paddle.activation.Linear())
         return fc2
-	```
+    ```
 
-	2.1. 首先定义了一组卷积网络，即conv_block。卷积核大小为3x3，池化窗口大小为2x2，窗口滑动大小为2，groups决定每组VGG模块是几次连续的卷积操作，dropouts指定Dropout操作的概率。所使用的`img_conv_group`是在`paddle.networks`中预定义的模块，由若干组 `Conv->BN->ReLu->Dropout` 和 一组 `Pooling` 组成，
+    2.1. 首先定义了一组卷积网络，即conv_block。卷积核大小为3x3，池化窗口大小为2x2，窗口滑动大小为2，groups决定每组VGG模块是几次连续的卷积操作，dropouts指定Dropout操作的概率。所使用的`img_conv_group`是在`paddle.networks`中预定义的模块，由若干组 `Conv->BN->ReLu->Dropout` 和 一组 `Pooling` 组成，
 
-	2.2. 五组卷积操作，即 5个conv_block。 第一、二组采用两次连续的卷积操作。第三、四、五组采用三次连续的卷积操作。每组最后一个卷积后面Dropout概率为0，即不使用Dropout操作。
+    2.2. 五组卷积操作，即 5个conv_block。 第一、二组采用两次连续的卷积操作。第三、四、五组采用三次连续的卷积操作。每组最后一个卷积后面Dropout概率为0，即不使用Dropout操作。
 
-	2.3. 最后接两层512维的全连接。
+    2.3. 最后接两层512维的全连接。
 
 3. 定义分类器
 
-	通过上面VGG网络提取高层特征，然后经过全连接层映射到类别维度大小的向量，再通过Softmax归一化得到每个类别的概率，也可称作分类器。
+    通过上面VGG网络提取高层特征，然后经过全连接层映射到类别维度大小的向量，再通过Softmax归一化得到每个类别的概率，也可称作分类器。
 
-	```python
+    ```python
     out = paddle.layer.fc(input=net,
                           size=classdim,
                           act=paddle.activation.Softmax())
-	```
+    ```
 
 4. 定义损失函数和网络输出
 
-	在有监督训练中需要输入图像对应的类别信息，同样通过`paddle.layer.data`来定义。训练中采用多类交叉熵作为损失函数，并作为网络的输出，预测阶段定义网络的输出为分类器得到的概率信息。
+    在有监督训练中需要输入图像对应的类别信息，同样通过`paddle.layer.data`来定义。训练中采用多类交叉熵作为损失函数，并作为网络的输出，预测阶段定义网络的输出为分类器得到的概率信息。
 
-	```python
+    ```python
     lbl = paddle.layer.data(
         name="label", type=paddle.data_type.integer_value(classdim))
     cost = paddle.layer.classification_cost(input=out, label=lbl)
-	```
+    ```
 
 ### ResNet
 
 ResNet模型的第1、3、4步和VGG模型相同，这里不再介绍。主要介绍第2步即CIFAR10数据集上ResNet核心模块。
 
 ```python
-net = resnet_cifar10(data, depth=56)
+net = resnet_cifar10(image, depth=56)
 ```
 
 先介绍`resnet_cifar10`中的一些基本函数，再介绍网络连接过程。
@@ -417,7 +416,7 @@ $$  lr = lr_{0} * a^ {\lfloor \frac{n}{ b}\rfloor} $$
 cifar.train10()每次产生一条样本，在完成shuffle和batch之后，作为训练的输入。
 
 ```python
-reader=paddle.reader.batch(
+reader=paddle.batch(
     paddle.reader.shuffle(
         paddle.dataset.cifar.train10(), buf_size=50000),
         batch_size=128)
@@ -444,10 +443,9 @@ def event_handler(event):
             sys.stdout.flush()
     if isinstance(event, paddle.event.EndPass):
         result = trainer.test(
-            reader=paddle.reader.batch(
+            reader=paddle.batch(
                 paddle.dataset.cifar.test10(), batch_size=128),
-            reader_dict={'image': 0,
-                         'label': 1})
+            feeding=feeding)
         print "\nTest with Pass %d, %s" % (event.pass_id, result.metrics)
 ```
 

image_classification/train.py

@@ -36,9 +36,8 @@ def main():
     # option 2. vgg
     net = vgg_bn_drop(image)
 
-    out = paddle.layer.fc(input=net,
-                          size=classdim,
-                          act=paddle.activation.Softmax())
+    out = paddle.layer.fc(
+        input=net, size=classdim, act=paddle.activation.Softmax())
 
     lbl = paddle.layer.data(
         name="label", type=paddle.data_type.integer_value(classdim))
@@ -75,9 +74,8 @@ def main():
             print "\nTest with Pass %d, %s" % (event.pass_id, result.metrics)
 
     # Create trainer
-    trainer = paddle.trainer.SGD(cost=cost,
-                                 parameters=parameters,
-                                 update_equation=momentum_optimizer)
+    trainer = paddle.trainer.SGD(
+        cost=cost, parameters=parameters, update_equation=momentum_optimizer)
     trainer.train(
         reader=paddle.batch(
             paddle.reader.shuffle(

index.en.html

-<!DOCTYPE html>
-<html lang="en">
-<head>
-    <meta charset="UTF-8">
-    <title>Deep Learning 101</title>
-    <link href="http://cdn.bootcss.com/bootstrap/4.0.0-alpha.6/css/bootstrap.min.css" rel="stylesheet">
-    <script src="http://cdn.bootcss.com/tether/1.4.0/js/tether.js"></script>
-    <script src="http://cdn.bootcss.com/jquery/3.1.0/jquery.slim.js"></script>
-    <script src="http://cdn.bootcss.com/bootstrap/4.0.0-alpha.6/js/bootstrap.min.js"></script>
-    <style>
-* {
-    font-family:"Roboto","Lato","proxima-nova","Helvetica Neue",Arial,sans-serif;
-}
-.left-panel {
-    background: #E5E6EA;
-}
-
-.left-panel .card-block a.click_active {
-    background-color: #597cf1;
-    color: #fff;
-}
- 
-.left-panel .card-header {
-    background: #ecedee;
-}
-
-
-
-.left-panel .card-block a:not(.click_active) {
-    background: #e4e6e9;
-}
-
-.left-panel .card-block a {
-    border-radius: 0px;
-    font-size: 13px;
-    color: #2F323A;
-    padding-left: 40px;
-    border: 1px solid rgba(0,0,0,.125);
-}
-
-.left-panel .card-block a.active_color {
-    color:  rgb(70, 74, 76);
-    font-weight: bolder;
-}
-
-.left-panel .list-group-item {
-    -moz-box-align: center;
-    align-items: center;
-    display: flex;
-    flex-flow: row wrap;
-    margin-bottom: -1px;
-    padding: 0.75rem 1.25rem;
-    position: relative;
-    color: #2f323a;
-}
-
-.navbar img {
-    height: 90%;
-    width: 90%;
-}
-
-   </style>
-</head>
-<body>
-    <nav class="navbar navbar-toggleable-md navbar-inverse bg-inverse">
-        <a class="navbar-brand mr-auto" href="#">
-            <img alt="PaddlePaddle" src="./.theme/PP_w.png">
-        </a>
-        <ul class="nav navbar-nav">
-          <li class="nav-item">
-            <a class="nav-link" href="http://paddlepaddle.org">
-                Home
-            </a>
-          </li>
-          <li class="nav-item ">
-            <a class="nav-link" href="./index.html">
-                中文
-            </a>
-          </li>
-          <li class="nav-item active">
-            <a class="nav-link" href="#">
-                Deep Learning 101
-            </a>
-          </li>
-          <li class="nav-item">
-            <a class="nav-link" href="https://github.com/PaddlePaddle/book/">Fork on Github</a>
-          </li>
-        </ul>
-    </nav>
-    <div class="container-fluid">
-        <div class="row">
-            <div class="col-3 pl-1 pr-1 left-panel">
-                <div id="accordion" role="tablist" aria-multiselectable="true">
-                  <div class="card">
-                    <div class="card-header" role="tab" id="headingOne">
-                      <h5 class="mb-0">
-                        <a aria-expanded="true" aria-controls="collapseOne">
-                            Deep Learning 101<span class="sr-only">(current)</span>
-                        </a>
-                      </h5>
-                    </div>
-                    <div id="collapseOne" class="rounded-0 collapse show" role="tabpanel" aria-labelledby="headingOne">
-                      <div class="card-block pl-0 pr-0 pt-0 pb-0">
-                        <div class="list-group ">
-                          
-                            <a href="./fit_a_line/index.en.html" target="content_iframe" class="list-group-item list-group-item-action" style="border: 0px; border-bottom: 2px solid #dddfe3;">
-                                Linear Regression
-                            </a>
-                          
-                            <a href="./recognize_digits/index.en.html" target="content_iframe" class="list-group-item list-group-item-action" style="border: 0px; border-bottom: 2px solid #dddfe3;">
-                                Recognize Digits
-                            </a>
-                          
-                            <a href="./image_classification/index.en.html" target="content_iframe" class="list-group-item list-group-item-action" style="border: 0px; border-bottom: 2px solid #dddfe3;">
-                                Image Classification
-                            </a>
-                          
-                            <a href="./word2vec/index.en.html" target="content_iframe" class="list-group-item list-group-item-action" style="border: 0px; border-bottom: 2px solid #dddfe3;">
-                                Word2Vec
-                            </a>
-                          
-                            <a href="./understand_sentiment/index.en.html" target="content_iframe" class="list-group-item list-group-item-action" style="border: 0px; border-bottom: 2px solid #dddfe3;">
-                                Sentiment Analysis
-                            </a>
-                          
-                            <a href="./label_semantic_roles/index.en.html" target="content_iframe" class="list-group-item list-group-item-action" style="border: 0px; border-bottom: 2px solid #dddfe3;">
-                                Semantic Role Labeling
-                            </a>
-                          
-                            <a href="./machine_translation/index.en.html" target="content_iframe" class="list-group-item list-group-item-action" style="border: 0px; border-bottom: 2px solid #dddfe3;">
-                                Machine Translation
-                            </a>
-                          
-                            <a href="./recommender_system/index.en.html" target="content_iframe" class="list-group-item list-group-item-action" style="border: 0px; border-bottom: 2px solid #dddfe3;">
-                                Personalized Recommendation
-                            </a>
-                          
-                        </div>
-                      </div>
-                    </div>
-                  </div>
-                </div>
-            </div>
-            <div class="col">
-                <iframe src="./fit_a_line/index.en.html" style="border: none; overflow-y : hidden" width="100%" height="100%" name="content_iframe" id="content_iframe">
-                </iframe>
-            </div>
-        </div>
-    </div>
-    <script>
-    $('#content_iframe').on('load', function(){
-        $("#content_iframe").height(200)  // trick code to shrink iframe size
-        var body = $('#content_iframe').contents().find("body")
-        body.css("overflow-y", "hidden")
-        $("#content_iframe").height(body.height()+20)
-        var alllinks = $('#content_iframe').contents().find("a")
-        for (var i =0; i<alllinks.length; ++i) {
-            alllinks[i].setAttribute("target", "_blank")
-        }
-    });
-    $(".list-group a").click(function(){ 
-       $(".list-group a.click_active").removeClass("click_active"); 
-       $(this).addClass("click_active"); 
-    })
-    $($(".list-group a")[0]).addClass("click_active") 
-    </script>
-</body>
-</html>
\ No newline at end of file

index.html

-<!DOCTYPE html>
-<html lang="zh-Hans">
-<head>
-    <meta charset="UTF-8">
-    <title>深度学习入门</title>
-    <link href="http://cdn.bootcss.com/bootstrap/4.0.0-alpha.6/css/bootstrap.min.css" rel="stylesheet">
-    <script src="http://cdn.bootcss.com/tether/1.4.0/js/tether.js"></script>
-    <script src="http://cdn.bootcss.com/jquery/3.1.0/jquery.slim.js"></script>
-    <script src="http://cdn.bootcss.com/bootstrap/4.0.0-alpha.6/js/bootstrap.min.js"></script>
-    <style>
-* {
-    font-family:"Roboto","Lato","proxima-nova","Helvetica Neue",Arial,sans-serif;
-}
-.left-panel {
-    background: #E5E6EA;
-}
-
-.left-panel .card-block a.click_active {
-    background-color: #597cf1;
-    color: #fff;
-}
- 
-.left-panel .card-header {
-    background: #ecedee;
-}
-
-
-.left-panel .card-header a {
-    font-size: 14px;
-}
-
-
-.left-panel .card-block a:not(.click_active) {
-    background: #e4e6e9;
-}
-
-.left-panel .card-block a {
-    border-radius: 0px;
-    font-size: 13px;
-    color: #2F323A;
-    padding-left: 40px;
-    border: 1px solid rgba(0,0,0,.125);
-}
-
-.left-panel .card-block a.active_color {
-    color:  rgb(70, 74, 76);
-    font-weight: bolder;
-}
-
-.left-panel .list-group-item {
-    -moz-box-align: center;
-    align-items: center;
-    display: flex;
-    flex-flow: row wrap;
-    margin-bottom: -1px;
-    padding: 0.75rem 1.25rem;
-    position: relative;
-    color: #2f323a;
-}
-
-.navbar img {
-    height: 90%;
-    width: 90%;
-}
-
-   </style>
-</head>
-<body>
-    <nav class="navbar navbar-toggleable-md navbar-inverse bg-inverse">
-        <a class="navbar-brand mr-auto" href="#">
-            <img alt="PaddlePaddle" src="./.theme/PP_w.png">
-        </a>
-        <ul class="nav navbar-nav">
-          <li class="nav-item">
-            <a class="nav-link" href="http://paddlepaddle.org">
-                首页
-            </a>
-          </li>
-          <li class="nav-item active">
-            <a class="nav-link" href="#">
-                深度学习入门
-            </a>
-          </li>
-          <li class="nav-item ">
-            <a class="nav-link" href="./index.en.html">
-                English
-            </a>
-          </li>
-          <li class="nav-item">
-            <a class="nav-link" href="https://github.com/PaddlePaddle/book/">Fork on Github</a>
-          </li>
-        </ul>
-    </nav>
-    <div class="container-fluid">
-        <div class="row">
-            <div class="col-3 pl-1 pr-1 left-panel">
-                <div id="accordion" role="tablist" aria-multiselectable="true">
-                  <div class="card">
-                    <div class="card-header" role="tab" id="headingOne">
-                      <h5 class="mb-0">
-                        <a aria-expanded="true" aria-controls="collapseOne">
-                            深度学习入门
-                        </a>
-                      </h5>
-                    </div>
-                    <div id="collapseOne" class="rounded-0 collapse show" role="tabpanel" aria-labelledby="headingOne">
-                      <div class="card-block pl-0 pr-0 pt-0 pb-0">
-                        <div class="list-group ">
-                          
-                            <a href="./fit_a_line/index.html" target="content_iframe" class="list-group-item list-group-item-action" style="border: 0px; border-bottom: 2px solid #dddfe3;">
-                                新手入门
-                            </a>
-                          
-                            <a href="./recognize_digits/index.html" target="content_iframe" class="list-group-item list-group-item-action" style="border: 0px; border-bottom: 2px solid #dddfe3;">
-                                识别数字
-                            </a>
-                          
-                            <a href="./image_classification/index.html" target="content_iframe" class="list-group-item list-group-item-action" style="border: 0px; border-bottom: 2px solid #dddfe3;">
-                                图像分类
-                            </a>
-                          
-                            <a href="./word2vec/index.html" target="content_iframe" class="list-group-item list-group-item-action" style="border: 0px; border-bottom: 2px solid #dddfe3;">
-                                词向量
-                            </a>
-                          
-                            <a href="./understand_sentiment/index.html" target="content_iframe" class="list-group-item list-group-item-action" style="border: 0px; border-bottom: 2px solid #dddfe3;">
-                                情感分析
-                            </a>
-                          
-                            <a href="./label_semantic_roles/index.html" target="content_iframe" class="list-group-item list-group-item-action" style="border: 0px; border-bottom: 2px solid #dddfe3;">
-                                语义角色标注
-                            </a>
-                          
-                            <a href="./machine_translation/index.html" target="content_iframe" class="list-group-item list-group-item-action" style="border: 0px; border-bottom: 2px solid #dddfe3;">
-                                机器翻译
-                            </a>
-                          
-                            <a href="./recommender_system/index.html" target="content_iframe" class="list-group-item list-group-item-action" style="border: 0px; border-bottom: 2px solid #dddfe3;">
-                                个性化推荐
-                            </a>
-                          
-                        </div>
-                      </div>
-                    </div>
-                  </div>
-                </div>
-            </div>
-            <div class="col">
-                <iframe src="./fit_a_line/index.html" style="border: none; overflow-y : hidden" width="100%" height="100%" name="content_iframe" id="content_iframe">
-                </iframe>
-            </div>
-        </div>
-    </div>
-    <script>
-    $('#content_iframe').on('load', function(){
-        $("#content_iframe").height(200)  // trick code to shrink iframe size
-        var body = $('#content_iframe').contents().find("body")
-        body.css("overflow-y", "hidden")
-        $("#content_iframe").height(body.height()+20)
-        var alllinks = $('#content_iframe').contents().find("a")
-        for (var i =0; i<alllinks.length; ++i) {
-            alllinks[i].setAttribute("target", "_blank")
-        }
-    });
-    $(".list-group a").click(function(){ 
-       $(".list-group a.click_active").removeClass("click_active"); 
-       $(this).addClass("click_active"); 
-    })
-    $($(".list-group a")[0]).addClass("click_active") 
-    </script>
-</body>
-</html>
\ No newline at end of file

label_semantic_roles/README.en.md

@@ -2,6 +2,8 @@
 
 Source code of this chapter is in [book/label_semantic_roles](https://github.com/PaddlePaddle/book/tree/develop/label_semantic_roles).
 
+For instructions on getting started with PaddlePaddle, see [PaddlePaddle installation guide](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/getstarted/build_and_install/docker_install_en.rst).
+
 ## Background
 
 Natural Language Analysis contains three components: Lexical Analysis, Syntactic Analysis, and Semantic Analysis. Semantic Role Labelling (SRL) is one way for Shallow Semantic Analysis. A predicate of a sentence is a property that a subject possesses or is characterized, such as what it does, what it is or how it is, which mostly corresponds to the core of an event. The noun associated with a predicate is called Argument. Semantic roles express the abstract roles that arguments of a predicate can take in the event, such as Agent, Patient, Theme, Experiencer, Beneficiary, Instrument, Location, Goal and Source, etc.
@@ -200,6 +202,8 @@ import numpy as np
 import paddle.v2 as paddle
 import paddle.v2.dataset.conll05 as conll05
 
+paddle.init(use_gpu=False, trainer_count=1)
+
 word_dict, verb_dict, label_dict = conll05.get_dict()
 word_dict_len = len(word_dict)
 label_dict_len = len(label_dict)
@@ -470,4 +474,4 @@ Semantic Role Labeling is an important intermediate step in a wide range of natu
 10. Zhou J, Xu W. [End-to-end learning of semantic role labeling using recurrent neural networks](http://www.aclweb.org/anthology/P/P15/P15-1109.pdf)[C]//Proceedings of the Annual Meeting of the Association for Computational Linguistics. 2015.
 
 <br/>
-<a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/4.0/"><img alt="知识共享许可协议" style="border-width:0" src="https://i.creativecommons.org/l/by-nc-sa/4.0/88x31.png" /></a><br /><span xmlns:dct="http://purl.org/dc/terms/" href="http://purl.org/dc/dcmitype/Text" property="dct:title" rel="dct:type">本教程</span> 由 <a xmlns:cc="http://creativecommons.org/ns#" href="http://book.paddlepaddle.org" property="cc:attributionName" rel="cc:attributionURL">PaddlePaddle</a> 创作，采用 <a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/4.0/">知识共享 署名-非商业性使用-相同方式共享 4.0 国际 许可协议</a>进行许可。
+This tutorial is contributed by <a xmlns:cc="http://creativecommons.org/ns#" href="http://book.paddlepaddle.org" property="cc:attributionName" rel="cc:attributionURL">PaddlePaddle</a>, and licensed under a <a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/4.0/">Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License</a>.

label_semantic_roles/README.md

 # 语义角色标注
 
-本教程源代码目录在[book/label_semantic_roles](https://github.com/PaddlePaddle/book/tree/develop/label_semantic_roles)， 初次使用请参考PaddlePaddle[安装教程](http://www.paddlepaddle.org/doc_cn/build_and_install/index.html)。
+本教程源代码目录在[book/label_semantic_roles](https://github.com/PaddlePaddle/book/tree/develop/label_semantic_roles)， 初次使用请参考PaddlePaddle[安装教程](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/getstarted/build_and_install/docker_install_cn.rst)。
 
 ## 背景介绍
 

label_semantic_roles/db_lstm.py

@@ -75,8 +75,7 @@ settings(
     learning_method=MomentumOptimizer(momentum=0),
     learning_rate=2e-2,
     regularization=L2Regularization(8e-4),
-    model_average=ModelAverage(
-        average_window=0.5, max_average_window=10000), )
+    model_average=ModelAverage(average_window=0.5, max_average_window=10000), )
 
 ####################################### network ##############################
 #8 features and 1 target
@@ -102,13 +101,12 @@ std_default = ParameterAttribute(initial_std=default_std)
 predicate_embedding = embedding_layer(
     size=word_dim,
     input=predicate,
-    param_attr=ParameterAttribute(
-        name='vemb', initial_std=default_std))
+    param_attr=ParameterAttribute(name='vemb', initial_std=default_std))
 
 word_input = [word, ctx_n2, ctx_n1, ctx_0, ctx_p1, ctx_p2]
 emb_layers = [
-    embedding_layer(
-        size=word_dim, input=x, param_attr=emb_para) for x in word_input
+    embedding_layer(size=word_dim, input=x, param_attr=emb_para)
+    for x in word_input
 ]
 emb_layers.append(predicate_embedding)
 mark_embedding = embedding_layer(
@@ -120,8 +118,8 @@ hidden_0 = mixed_layer(
     size=hidden_dim,
     bias_attr=std_default,
     input=[
-        full_matrix_projection(
-            input=emb, param_attr=std_default) for emb in emb_layers
+        full_matrix_projection(input=emb, param_attr=std_default)
+        for emb in emb_layers
     ])
 
 mix_hidden_lr = 1e-3
@@ -171,10 +169,8 @@ feature_out = mixed_layer(
     size=label_dict_len,
     bias_attr=std_default,
     input=[
-        full_matrix_projection(
-            input=input_tmp[0], param_attr=hidden_para_attr),
-        full_matrix_projection(
-            input=input_tmp[1], param_attr=lstm_para_attr)
+        full_matrix_projection(input=input_tmp[0], param_attr=hidden_para_attr),
+        full_matrix_projection(input=input_tmp[1], param_attr=lstm_para_attr)
     ], )
 
 if not is_predict:

label_semantic_roles/index.en.html

@@ -44,6 +44,8 @@
 
 Source code of this chapter is in [book/label_semantic_roles](https://github.com/PaddlePaddle/book/tree/develop/label_semantic_roles).
 
+For instructions on getting started with PaddlePaddle, see [PaddlePaddle installation guide](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/getstarted/build_and_install/docker_install_en.rst).
+
 ## Background
 
 Natural Language Analysis contains three components: Lexical Analysis, Syntactic Analysis, and Semantic Analysis. Semantic Role Labelling (SRL) is one way for Shallow Semantic Analysis. A predicate of a sentence is a property that a subject possesses or is characterized, such as what it does, what it is or how it is, which mostly corresponds to the core of an event. The noun associated with a predicate is called Argument. Semantic roles express the abstract roles that arguments of a predicate can take in the event, such as Agent, Patient, Theme, Experiencer, Beneficiary, Instrument, Location, Goal and Source, etc.
@@ -242,6 +244,8 @@ import numpy as np
 import paddle.v2 as paddle
 import paddle.v2.dataset.conll05 as conll05
 
+paddle.init(use_gpu=False, trainer_count=1)
+
 word_dict, verb_dict, label_dict = conll05.get_dict()
 word_dict_len = len(word_dict)
 label_dict_len = len(label_dict)
@@ -512,7 +516,7 @@ Semantic Role Labeling is an important intermediate step in a wide range of natu
 10. Zhou J, Xu W. [End-to-end learning of semantic role labeling using recurrent neural networks](http://www.aclweb.org/anthology/P/P15/P15-1109.pdf)[C]//Proceedings of the Annual Meeting of the Association for Computational Linguistics. 2015.
 
 <br/>
-<a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/4.0/"><img alt="知识共享许可协议" style="border-width:0" src="https://i.creativecommons.org/l/by-nc-sa/4.0/88x31.png" /></a><br /><span xmlns:dct="http://purl.org/dc/terms/" href="http://purl.org/dc/dcmitype/Text" property="dct:title" rel="dct:type">本教程</span> 由 <a xmlns:cc="http://creativecommons.org/ns#" href="http://book.paddlepaddle.org" property="cc:attributionName" rel="cc:attributionURL">PaddlePaddle</a> 创作，采用 <a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/4.0/">知识共享 署名-非商业性使用-相同方式共享 4.0 国际 许可协议</a>进行许可。
+This tutorial is contributed by <a xmlns:cc="http://creativecommons.org/ns#" href="http://book.paddlepaddle.org" property="cc:attributionName" rel="cc:attributionURL">PaddlePaddle</a>, and licensed under a <a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/4.0/">Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License</a>.
 
 </div>
 <!-- You can change the lines below now. -->

label_semantic_roles/index.html

@@ -42,7 +42,7 @@
 <div id="markdown" style='display:none'>
 # 语义角色标注
 
-本教程源代码目录在[book/label_semantic_roles](https://github.com/PaddlePaddle/book/tree/develop/label_semantic_roles)， 初次使用请参考PaddlePaddle[安装教程](http://www.paddlepaddle.org/doc_cn/build_and_install/index.html)。
+本教程源代码目录在[book/label_semantic_roles](https://github.com/PaddlePaddle/book/tree/develop/label_semantic_roles)， 初次使用请参考PaddlePaddle[安装教程](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/getstarted/build_and_install/docker_install_cn.rst)。
 
 ## 背景介绍
 

label_semantic_roles/train.py

@@ -40,15 +40,14 @@ def db_lstm():
     predicate_embedding = paddle.layer.embedding(
         size=word_dim,
         input=predicate,
-        param_attr=paddle.attr.Param(
-            name='vemb', initial_std=default_std))
+        param_attr=paddle.attr.Param(name='vemb', initial_std=default_std))
     mark_embedding = paddle.layer.embedding(
         size=mark_dim, input=mark, param_attr=std_0)
 
     word_input = [word, ctx_n2, ctx_n1, ctx_0, ctx_p1, ctx_p2]
     emb_layers = [
-        paddle.layer.embedding(
-            size=word_dim, input=x, param_attr=emb_para) for x in word_input
+        paddle.layer.embedding(size=word_dim, input=x, param_attr=emb_para)
+        for x in word_input
     ]
     emb_layers.append(predicate_embedding)
     emb_layers.append(mark_embedding)
@@ -109,13 +108,12 @@ def db_lstm():
                 input=input_tmp[1], param_attr=lstm_para_attr)
         ], )
 
-    crf_cost = paddle.layer.crf(size=label_dict_len,
-                                input=feature_out,
-                                label=target,
-                                param_attr=paddle.attr.Param(
-                                    name='crfw',
-                                    initial_std=default_std,
-                                    learning_rate=mix_hidden_lr))
+    crf_cost = paddle.layer.crf(
+        size=label_dict_len,
+        input=feature_out,
+        label=target,
+        param_attr=paddle.attr.Param(
+            name='crfw', initial_std=default_std, learning_rate=mix_hidden_lr))
 
     crf_dec = paddle.layer.crf_decoding(
         name='crf_dec_l',
@@ -151,13 +149,11 @@ def main():
         model_average=paddle.optimizer.ModelAverage(
             average_window=0.5, max_average_window=10000), )
 
-    trainer = paddle.trainer.SGD(cost=crf_cost,
-                                 parameters=parameters,
-                                 update_equation=optimizer)
+    trainer = paddle.trainer.SGD(
+        cost=crf_cost, parameters=parameters, update_equation=optimizer)
 
     reader = paddle.batch(
-        paddle.reader.shuffle(
-            conll05.test(), buf_size=8192), batch_size=10)
+        paddle.reader.shuffle(conll05.test(), buf_size=8192), batch_size=10)
 
     feeding = {
         'word_data': 0,

machine_translation/README.en.md

 # Machine Translation
 
-The source codes is located at [book/machine_translation](https://github.com/PaddlePaddle/book/tree/develop/machine_translation). Please refer to the PaddlePaddle [installation tutorial](http://www.paddlepaddle.org/doc_cn/build_and_install/index.html) if you are a first time user.
+The source codes is located at [book/machine_translation](https://github.com/PaddlePaddle/book/tree/develop/machine_translation). Please refer to the PaddlePaddle [installation tutorial](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/getstarted/build_and_install/docker_install_en.rst) if you are a first time user.
 
 ## Background
 
@@ -185,77 +185,10 @@ Note: $z_{i+1}$ and $p_{i+1}$ are computed the same way as in [Decoder](#Decoder
 
 ## Data Preparation
 
-### Download and Uncompression
-
 This tutorial uses a dataset from [WMT-14](http://www-lium.univ-lemans.fr/~schwenk/cslm_joint_paper/), where [bitexts (after selection)](http://www-lium.univ-lemans.fr/~schwenk/cslm_joint_paper/data/bitexts.tgz) is used as the training set, and [dev+test data](http://www-lium.univ-lemans.fr/~schwenk/cslm_joint_paper/data/dev+test.tgz) is used as test and generation set.
 
-Run the following command in Linux to obtain the data:
-```bash
-cd data
-./wmt14_data.sh
-```
-There are three folders in the downloaded dataset `data/wmt14`:
-<p align = "center">
-<table>
-<tr>
-<td>Folder Name</td>
-<td>French-English Parallel Corpus</td>
-<td>Number of Files</td>
-<td>Size of Files</td>
-</tr>
-
-<tr>
-<td>train</td>
-<td>ccb2_pc30.src, ccb2_pc30.trg, etc</td>
-<td>12</td>
-<td>3.55G</td>
-</tr>
-
-<tr>
-<td>test</td>
-<td>ntst1213.src, ntst1213.trg</td>
-<td>2</td>
-<td>1636k</td>
-</tr>
-
-</tr>
-<tr>
-<td>gen</td>
-<td>ntst14.src, ntst14.trg</td>
-<td>2</td>
-<td>864k</td>
-</tr>
-</table>
-</p>
-
-- `XXX.src` is the source file in French and `XXX.trg`is the target file in English. Each row of the file contains one sentence.
-- `XXX.src` and `XXX.trg` has the same number of rows and there is a one-to-one correspondance between the sentences at any row from the two files.
 
-### User Defined Dataset (Optional)
-
-To use your own dataset, just put it under the `data` folder and organize it as follows
-```text
-user_dataset
-├── train
-│   ├── train_file1.src
-│   ├── train_file1.trg
-│   └── ...
-├── test
-│   ├── test_file1.src
-│   ├── test_file1.trg
-│   └── ...
-├── gen
-│   ├── gen_file1.src
-│   ├── gen_file1.trg
-│   └── ...
-```
-
-Explanation of the directories:
-- First level: `user_dataset`: the name of the user defined dataset.
-- Second level: `train`、`test` and `gen`: these names should not be changed.
-- Third level: Parallel corpus in source language and target language, each with a postfix of `.src` and `.trg`.
-
-### Data Pre-processing
+### Data Preprocessing
 
 There are two steps for pre-processing:
 - Merge the source and target parallel corpus files into one file
@@ -264,248 +197,104 @@ There are two steps for pre-processing:
 
 - Create source dictionary and target dictionary, each containing **DICTSIZE** number of words, including the most frequent (DICTSIZE - 3) fo word from the corpus and 3 special token `<s>` (begin of sequence), `<e>` (end of sequence)  and `<unk>` (unknown words that are not in the vocabulary).
 
-`preprocess.py` is used for pre-processing:
-```python
-python preprocess.py -i INPUT [-d DICTSIZE] [-m]
-```
-- `-i INPUT`: path to the original dataset.
-- `-d DICTSIZE`: number of words in the dictionary. If unspecified, the dictionary will contain all the words appeared in the input dataset.
-- `-m --mergeDict`: merge the source dictionary with target dictionary, making the two dictionaries have the same content.
-
-The specific command to run the script is as follows:
-```python
-python preprocess.py -i data/wmt14 -d 30000
-```
-You will see the following messages after a few minutes:
-```text
-concat parallel corpora for dataset
-build source dictionary for train data
-build target dictionary for train data
-dictionary size is 30000
-```
-The pre-processed data is located at `data/pre-wmt14`:
-```text
-pre-wmt14
-├── train
-│   └── train
-├── test
-│   └── test
-├── gen
-│   └── gen
-├── train.list
-├── test.list
-├── gen.list
-├── src.dict
-└── trg.dict
-```
-- `train`, `test` and `gen`: contains French-English parallel corpus for training, testing and generation. Each row from each file is separated into two columns with a "\t", where the first column is the sequence in French and the second one is in English.
-- `train.list`, `test.list` and `gen.list`: record respectively the path to `train`, `test` and `gen` folders.
-- `src.dict` and `trg.dict`: source (French) and target (English) dictionary. Each dictionary contains 30000 words (29997 most frequent words and 3 special tokens).
-
-### Providing Data to PaddlePaddle
-
-We use `dataprovider.py` to provide data to PaddlePaddle as follows:
+### A Subset of Dataset
 
-1. Import PyDataProvider2 package from PaddlePaddle and define three special tokens:
+Because the full dataset is very big, to reduce the time for downloading the full dataset. PadddlePaddle package `paddle.dataset.wmt14` provides a preprocessed `subset of dataset`(http://paddlepaddle.bj.bcebos.com/demo/wmt_shrinked_data/wmt14.tgz).
 
-   ```python
-   from paddle.trainer.PyDataProvider2 import *
-   UNK_IDX = 2    #out of vocabulary word
-   START = "<s>"  #begin of sequence
-   END = "<e>"    #end of sequence
-   ```
-2. Use initialization function `hook` to define the input data types (`input_types`) for training and generation:
-   - Training: there are three input sequences, where "source language sequence" and "target language sequence" are input and the "target language next word sequence" is the label.
-   - Generation: there are two input sequences, where the "source language sequence" is the input and “source language sequence id” are the ids for the input data (optional).
-
-  `src_dict_path` in the `hook` function is the path to the source language dictionary, while `trg_dict_path` the path to target language dictionary. `is_generating` is passed from model config file. For more details on the usage of the `hook` function please refer to [Model Config](#Model Config).
-
-   ```python
-   def hook(settings, src_dict_path, trg_dict_path, is_generating, file_list,
-            **kwargs):
-       # job_mode = 1: training 0: generation
-       settings.job_mode = not is_generating
-
-       def fun(dict_path): # load dictionary according to the path
-           out_dict = dict()
-           with open(dict_path, "r") as fin:
-               out_dict = {
-                   line.strip(): line_count
-                   for line_count, line in enumerate(fin)
-               }
-           return out_dict
-
-       settings.src_dict = fun(src_dict_path)
-       settings.trg_dict = fun(trg_dict_path)
-
-       if settings.job_mode:                                  #training
-           settings.input_types = {
-               'source_language_word':                        #source language sequence
-               integer_value_sequence(len(settings.src_dict)),
-               'target_language_word':                        #target language sequence
-               integer_value_sequence(len(settings.trg_dict)),
-               'target_language_next_word':                   #target language next word sequence
-               integer_value_sequence(len(settings.trg_dict))
-           }
-       else:                                                  #generation
-           settings.input_types = {
-               'source_language_word':                        #source language sequence
-               integer_value_sequence(len(settings.src_dict)),
-               'sent_id':                                     #source language sequence id
-               integer_value_sequence(len(open(file_list[0], "r").readlines()))
-           }
-   ```
-3. Use `process` function to open the file `file_name`, read each row of the file, convert the data to be compatible with `input_types`, and then use `yield` to return to PaddlePaddle process. More specifically
-
-   - add `<s>` to the beginning of each source language sequence and add `<e>` to the end, producing "source_language_word".
-   - add `<s>` to the beginning of each target language senquence, producing "target_language_word".
-   - add `<e>` to the end of each target language senquence, producing "target_language_next_word".
-
-   ```python
-   def _get_ids(s, dictionary): # get the location of each word from the source language sequence in the dictionary
-       words = s.strip().split()
-       return [dictionary[START]] + \
-              [dictionary.get(w, UNK_IDX) for w in words] + \
-              [dictionary[END]]
-
-   @provider(init_hook=hook, pool_size=50000)
-   def process(settings, file_name):
-       with open(file_name, 'r') as f:
-           for line_count, line in enumerate(f):
-               line_split = line.strip().split('\t')
-               if settings.job_mode and len(line_split) != 2:
-                   continue
-               src_seq = line_split[0]
-               src_ids = _get_ids(src_seq, settings.src_dict)
-
-               if settings.job_mode:
-                   trg_seq = line_split[1]
-                   trg_words = trg_seq.split()
-                   trg_ids = [settings.trg_dict.get(w, UNK_IDX) for w in trg_words]
-
-                   # sequence with length longer than 80 with be removed during training to avoid an overly deep RNN.
-                   if len(src_ids) > 80 or len(trg_ids) > 80:
-                       continue
-                   trg_ids_next = trg_ids + [settings.trg_dict[END]]
-                   trg_ids = [settings.trg_dict[START]] + trg_ids
-                   yield {
-                       'source_language_word': src_ids,
-                       'target_language_word': trg_ids,
-                       'target_language_next_word': trg_ids_next
-                   }
-               else:
-                   yield {'source_language_word': src_ids, 'sent_id': [line_count]}
-   ```
-Note: The size of the training data is 3.55G. For machines with limited memories, it is recommended to use `pool_size` to set the number of data samples stored in memory.
-
-## Model Config
+This subset has 193319 instances of training data and 6003 instances of test data. Dictionary size is 30000. Because of the limitation of size of the subset, the effectiveness of trained model from this subset is not guaranteed.
 
-### Data Definition
+## Training Instructions
 
-1. Specify the path to data and source/target dictionaries. `is_generating` accepts argument passed from command lines and is used to denote whether the current configuration is for training (default) or generation. See [Usage and Resutls](#Usage and Results).
+### Initialize PaddlePaddle
 
-   ```python
-   import os
-   from paddle.trainer_config_helpers import *
+```python
+import sys
+import paddle.v2 as paddle
 
-   data_dir = "./data/pre-wmt14" # data path
-   src_lang_dict = os.path.join(data_dir, 'src.dict') # path to the source language dictionary
-   trg_lang_dict = os.path.join(data_dir, 'trg.dict') # path to the target language dictionary
-   is_generating = get_config_arg("is_generating", bool, False) # config mode
-   ```
-2. Use `define_py_data_sources2` to get data from `dataprovider.py`, and use `args` variable to input the source/target language dicitonary path and config mode.
+# train with a single CPU
+paddle.init(use_gpu=False, trainer_count=1)
+```
 
-   ```python
-   if not is_generating:
-       train_list = os.path.join(data_dir, 'train.list')
-       test_list = os.path.join(data_dir, 'test.list')
-   else:
-       train_list = None
-       test_list = os.path.join(data_dir, 'gen.list')
-
-   define_py_data_sources2(
-       train_list,
-       test_list,
-       module="dataprovider",
-       obj="process",
-       args={
-           "src_dict_path": src_lang_dict, # source language dictionary path
-           "trg_dict_path": trg_lang_dict, # target language dictionary path
-           "is_generating": is_generating  # config mode
-       })
-   ```
+### Define DataSet
 
-### Algorithm Configuration
+We will define dictionary size, and create [**data reader**](https://github.com/PaddlePaddle/Paddle/tree/develop/doc/design/reader#python-data-reader-design-doc) for WMT-14 dataset.
 
 ```python
-settings(
-    learning_method = AdamOptimizer(),
-    batch_size = 50,
-    learning_rate = 5e-4)
+# source and target dict dim.
+dict_size = 30000
+
+feeding = {
+    'source_language_word': 0,
+    'target_language_word': 1,
+    'target_language_next_word': 2
+}
+wmt14_reader = paddle.batch(
+    paddle.reader.shuffle(
+        paddle.dataset.wmt14.train(dict_size=dict_size), buf_size=8192),
+    batch_size=5)
 ```
-This tutorial will use the default SGD and Adam learning algorithm, with a learning rate of 5e-4. Note that the `batch_size = 50` denotes generating 50 sequence each time.
 
-### Model Structure
+### Model Configuration
 
 1. Define some global variables
 
    ```python
-   source_dict_dim = len(open(src_lang_dict, "r").readlines()) # size of the source language dictionary
-   target_dict_dim = len(open(trg_lang_dict, "r").readlines()) # size of target language dictionary
-   word_vector_dim = 512 # dimensionality of word vector
-   encoder_size = 512      # dimensionality of the hidden state of encoder GRU
-   decoder_size = 512    # dimentionality of the hidden state of decoder GRU
-
-   if is_generating:
-       beam_size=3    # beam size for the beam search algorithm
-       max_length=250 # maximum length for the generated sentence
-       gen_trans_file = get_config_arg("gen_trans_file", str, None) # generate file
+   source_dict_dim = dict_size # source language dictionary size
+   target_dict_dim = dict_size # destination language dictionary size
+   word_vector_dim = 512 # word embedding dimension
+   encoder_size = 512 # hidden layer size of GRU in encoder
+   decoder_size = 512 # hidden layer size of GRU in decoder
   ```
 
-2. Implement Encoder as follows:
-
-   2.1 Input one-hot vector representations $\mathbf{w}$ converted with `dataprovider.py` from the source language sentence
+1. Implement Encoder as follows:
+   1. Input is a sequence of words represented by an integer word index sequence. So we define data layer of data type `integer_value_sequence`. The value range of each element in the sequence is `[0, source_dict_dim)`
 
    ```python
-   src_word_id = data_layer(name='source_language_word', size=source_dict_dim)
+    src_word_id = paddle.layer.data(
+        name='source_language_word',
+        type=paddle.data_type.integer_value_sequence(source_dict_dim))
    ```
-   2.2 Map the one-hot vector into a word vector $\mathbf{s}$ in a low-dimensional semantic space
+
+   1. Map the one-hot vector (represented by word index) into a word vector $\mathbf{s}$ in a low-dimensional semantic space
 
    ```python
-   src_embedding = embedding_layer(
-       input=src_word_id,
-       size=word_vector_dim,
-       param_attr=ParamAttr(name='_source_language_embedding'))
+    src_embedding = paddle.layer.embedding(
+        input=src_word_id,
+        size=word_vector_dim,
+        param_attr=paddle.attr.ParamAttr(name='_source_language_embedding'))
    ```
-   2.3 Use bi-direcitonal GRU to encode the source language sequence, and concatenate the encoding outputs from the two GRUs to get $\mathbf{h}$
+
+   1. Use bi-direcitonal GRU to encode the source language sequence, and concatenate the encoding outputs from the two GRUs to get $\mathbf{h}$
 
    ```python
-   src_forward = simple_gru(input=src_embedding, size=encoder_size)
-   src_backward = simple_gru(
-       input=src_embedding, size=encoder_size, reverse=True)
-   encoded_vector = concat_layer(input=[src_forward, src_backward])
+    src_forward = paddle.networks.simple_gru(
+        input=src_embedding, size=encoder_size)
+    src_backward = paddle.networks.simple_gru(
+        input=src_embedding, size=encoder_size, reverse=True)
+    encoded_vector = paddle.layer.concat(input=[src_forward, src_backward])
    ```
 
-3. Implement Attention-based Decoder as follows:
+1. Implement Attention-based Decoder as follows:
 
-   3.1 Get a projection of the encoding (c.f. 2.3) of the source language sequence by passing it into a feed forward neural network
+   1. Get a projection of the encoding (c.f. 2.3) of the source language sequence by passing it into a feed forward neural network
 
    ```python
-   with mixed_layer(size=decoder_size) as encoded_proj:
-       encoded_proj += full_matrix_projection(input=encoded_vector)
+    with paddle.layer.mixed(size=decoder_size) as encoded_proj:
+        encoded_proj += paddle.layer.full_matrix_projection(
+            input=encoded_vector)
    ```
 
-   3.2 Use a non-linear transformation of the last hidden state of the backward GRU on the source language sentence as the initial state of the decoder RNN $c_0=h_T$
+   1. Use a non-linear transformation of the last hidden state of the backward GRU on the source language sentence as the initial state of the decoder RNN $c_0=h_T$
 
    ```python
-   backward_first = first_seq(input=src_backward)
-   with mixed_layer(
-           size=decoder_size,
-           act=TanhActivation(), ) as decoder_boot:
-       decoder_boot += full_matrix_projection(input=backward_first)
+    backward_first = paddle.layer.first_seq(input=src_backward)
+    with paddle.layer.mixed(
+            size=decoder_size, act=paddle.activation.Tanh()) as decoder_boot:
+        decoder_boot += paddle.layer.full_matrix_projection(
+            input=backward_first)
    ```
 
-   3.3 Define the computation in each time step for the decoder RNN, i.e., according to the current context vector $c_i$, hidden state for the decoder $z_i$ and the $i$-th word $u_i$ in the target language to predict the probability $p_{i+1}$ for the $i+1$-th word.
+   1. Define the computation in each time step for the decoder RNN, i.e., according to the current context vector $c_i$, hidden state for the decoder $z_i$ and the $i$-th word $u_i$ in the target language to predict the probability $p_{i+1}$ for the $i+1$-th word.
 
       - decoder_mem records the hidden state $z_i$ from the previous time step, with an initial state as decoder_boot.
       - context is computed via `simple_attention` as $c_i=\sum {j=1}^{T}a_{ij}h_j$, where enc_vec is the projection of $h_j$ and enc_proj is the projection of $h_j$ (c.f. 3.1). $a_{ij}$ is calculated within `simple_attention`.
@@ -514,184 +303,148 @@ This tutorial will use the default SGD and Adam learning algorithm, with a learn
       - Softmax normalization is used in the end to computed the probability of words, i.e., $p\left ( u_i|u_{&lt;i},\mathbf{x} \right )=softmax(W_sz_i+b_z)$. The output is returned.
 
    ```python
-   def gru_decoder_with_attention(enc_vec, enc_proj, current_word):
-       decoder_mem = memory(
-           name='gru_decoder', size=decoder_size, boot_layer=decoder_boot)
-
-       context = simple_attention(
-           encoded_sequence=enc_vec,
-           encoded_proj=enc_proj,
-           decoder_state=decoder_mem, )
-
-       with mixed_layer(size=decoder_size * 3) as decoder_inputs:
-           decoder_inputs += full_matrix_projection(input=context)
-           decoder_inputs += full_matrix_projection(input=current_word)
-
-       gru_step = gru_step_layer(
-           name='gru_decoder',
-           input=decoder_inputs,
-           output_mem=decoder_mem,
-           size=decoder_size)
-
-       with mixed_layer(
-               size=target_dict_dim, bias_attr=True,
-               act=SoftmaxActivation()) as out:
-           out += full_matrix_projection(input=gru_step)
-       return out
+    def gru_decoder_with_attention(enc_vec, enc_proj, current_word):
+
+        decoder_mem = paddle.layer.memory(
+            name='gru_decoder', size=decoder_size, boot_layer=decoder_boot)
+
+        context = paddle.networks.simple_attention(
+            encoded_sequence=enc_vec,
+            encoded_proj=enc_proj,
+            decoder_state=decoder_mem)
+
+        with paddle.layer.mixed(size=decoder_size * 3) as decoder_inputs:
+            decoder_inputs += paddle.layer.full_matrix_projection(input=context)
+            decoder_inputs += paddle.layer.full_matrix_projection(
+                input=current_word)
+
+        gru_step = paddle.layer.gru_step(
+            name='gru_decoder',
+            input=decoder_inputs,
+            output_mem=decoder_mem,
+            size=decoder_size)
+
+        with paddle.layer.mixed(
+                size=target_dict_dim,
+                bias_attr=True,
+                act=paddle.activation.Softmax()) as out:
+            out += paddle.layer.full_matrix_projection(input=gru_step)
+        return out
     ```
 
-4. Decoder differences between the training and generation
-
-   4.1 Define the name for the decoder and the first two input for `gru_decoder_with_attention`. Note that `StaticInput` is used for the two inputs. Please refer to [StaticInput Document](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/howto/deep_model/rnn/recurrent_group_cn.md#输入) for more details.
+1. Define the name for the decoder and the first two input for `gru_decoder_with_attention`. Note that `StaticInput` is used for the two inputs. Please refer to [StaticInput Document](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/howto/deep_model/rnn/recurrent_group_cn.md#输入) for more details.
 
-   ```python
-   decoder_group_name = "decoder_group"
-   group_input1 = StaticInput(input=encoded_vector, is_seq=True)
-   group_input2 =  StaticInput(input=encoded_proj, is_seq=True)
-   group_inputs = [group_input1, group_input2]
-   ```
+    ```python
+    decoder_group_name = "decoder_group"
+    group_input1 = paddle.layer.StaticInputV2(input=encoded_vector, is_seq=True)
+    group_input2 = paddle.layer.StaticInputV2(input=encoded_proj, is_seq=True)
+    group_inputs = [group_input1, group_input2]
+    ```
 
-   4.2 In training mode:
+1. Training mode:
 
-      - word embedding from the target langauge trg_embedding is passed to `gru_decoder_with_attention` as current_word.
+      - word embedding from the target language trg_embedding is passed to `gru_decoder_with_attention` as current_word.
       - `recurrent_group` calls `gru_decoder_with_attention` in a recurrent way
       - the sequence of next words from the target language is used as label (lbl)
       - multi-class cross-entropy (`classification_cost`) is used to calculate the cost
 
-   ```python
-   if not is_generating:
-       trg_embedding = embedding_layer(
-           input=data_layer(
-               name='target_language_word', size=target_dict_dim),
-           size=word_vector_dim,
-           param_attr=ParamAttr(name='_target_language_embedding'))
-       group_inputs.append(trg_embedding)
-
-       decoder = recurrent_group(
-           name=decoder_group_name,
-           step=gru_decoder_with_attention,
-           input=group_inputs)
-
-       lbl = data_layer(name='target_language_next_word', size=target_dict_dim)
-       cost = classification_cost(input=decoder, label=lbl)
-       outputs(cost)
-   ```
+    ```python
+    trg_embedding = paddle.layer.embedding(
+        input=paddle.layer.data(
+            name='target_language_word',
+            type=paddle.data_type.integer_value_sequence(target_dict_dim)),
+        size=word_vector_dim,
+        param_attr=paddle.attr.ParamAttr(name='_target_language_embedding'))
+    group_inputs.append(trg_embedding)
+
+    # For decoder equipped with attention mechanism, in training,
+    # target embeding (the groudtruth) is the data input,
+    # while encoded source sequence is accessed to as an unbounded memory.
+    # Here, the StaticInput defines a read-only memory
+    # for the recurrent_group.
+    decoder = paddle.layer.recurrent_group(
+        name=decoder_group_name,
+        step=gru_decoder_with_attention,
+        input=group_inputs)
+
+    lbl = paddle.layer.data(
+        name='target_language_next_word',
+        type=paddle.data_type.integer_value_sequence(target_dict_dim))
+    cost = paddle.layer.classification_cost(input=decoder, label=lbl)
+    ```
 
-   4.3 In generation mode:
+Note: Our configuration is based on Bahdanau et al. \[[4](#Reference)\] but with a few simplifications. Please refer to [issue #1133](https://github.com/PaddlePaddle/Paddle/issues/1133) for more details.
 
-      - during generation, as the decoder RNN will take the word vector generated from the previous time step as input, `GeneratedInput` is used to implement this automatically. Please refer to [GeneratedInput Document](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/howto/deep_model/rnn/recurrent_group_cn.md#输入) for details.
-      - `beam_search` will call `gru_decoder_with_attention` to generate id
-      - `seqtext_printer_evaluator` outputs the generated sentence to `gen_trans_file` according to `trg_lang_dict`
+### Create Parameters
 
-   ```python
-   else:
-       trg_embedding = GeneratedInput(
-           size=target_dict_dim,
-           embedding_name='_target_language_embedding',
-           embedding_size=word_vector_dim)
-       group_inputs.append(trg_embedding)
-
-       beam_gen = beam_search(
-           name=decoder_group_name,
-           step=gru_decoder_with_attention,
-           input=group_inputs,
-           bos_id=0,
-           eos_id=1,
-           beam_size=beam_size,
-           max_length=max_length)
-
-       seqtext_printer_evaluator(
-           input=beam_gen,
-           id_input=data_layer(
-               name="sent_id", size=1),
-           dict_file=trg_lang_dict,
-           result_file=gen_trans_file)
-       outputs(beam_gen)
-   ```
-Note: Our configuration is based on Bahdanau et al. \[[4](#Reference)\] but with a few simplifications. Please refer to [issue #1133](https://github.com/PaddlePaddle/Paddle/issues/1133) for more details.
+Create every parameter that `cost` layer needs.
 
+```python
+parameters = paddle.parameters.create(cost)
+```
+
+We can get parameter names. If the parameter name is not specified during model configuration, it will be generated.
+
+```python
+for param in parameters.keys():
+    print param
+```
 
 ## Model Training
 
-Training can be started with the following command:
+1. Create trainer
 
-```bash
-./train.sh
-```
-where `train.sh` contains
+    We need to tell trainer what to optimize, and how to optimize. Here trainer will optimize `cost` layer using stochastic gradient descent (SDG).
 
-```bash
-paddle train \
---config='seqToseq_net.py' \
---save_dir='model' \
---use_gpu=false \
---num_passes=16 \
---show_parameter_stats_period=100 \
---trainer_count=4 \
---log_period=10 \
---dot_period=5 \
-2>&1 | tee 'train.log'
-```
-- config: configuration file for the network
-- save_dir: path to save the trained model
-- use_gpu: whether to use GPU for training; CPU is used here
-- num_passes: number of passes for training. In PaddlePaddle, one pass meansing one pass of complete training pass using all the data in the training set
-- show_parameter_stats_period: here we show the statistics of parameters every 100 batches
-- trainer_count: the number of CPU processes or GPU devices
-- log_period: here we print log every 10 batches
-- dot_period: we print one "." every 5 batches
-
-The training loss will the printed every 10 batches, and you will see messages like those below:
-```text
-I0719 19:16:45.952062 15563 TrainerInternal.cpp:160]  Batch=10 samples=500 AvgCost=198.475 CurrentCost=198.475 Eval: classification_error_evaluator=0.737155  CurrentEval: classification_error_evaluator=0.737155
-I0719 19:17:56.707319 15563 TrainerInternal.cpp:160]  Batch=20 samples=1000 AvgCost=157.479 CurrentCost=116.483 Eval: classification_error_evaluator=0.698392  CurrentEval: classification_error_evaluator=0.659065
-.....
-```
-- AvgCost: average cost from batch-0 to the current batch.
-- CurrentCost: the cost for the current batch
-- classification\_error\_evaluator (Eval): average error rate from evaluator-0 to the current evaluator for each word
-- classification\_error\_evaluator (CurrentEval): error rate for the current evaluator for each word
+    ```python
+    optimizer = paddle.optimizer.Adam(
+        learning_rate=5e-5,
+        regularization=paddle.optimizer.L2Regularization(rate=1e-3))
+    trainer = paddle.trainer.SGD(cost=cost,
+                                 parameters=parameters,
+                                 update_equation=optimizer)
+    ```
 
-The model training is successful when the classification\_error\_evaluator is lower than 0.35.
+1. Define event handler
 
-## Model Usage
+    The event handler is a callback function invoked by trainer when an event happens. Here we will print log in event handler.
 
-### Download Pre-trained Model
+    ```python
+    def event_handler(event):
+        if isinstance(event, paddle.event.EndIteration):
+            if event.batch_id % 10 == 0:
+                print "\nPass %d, Batch %d, Cost %f, %s" % (
+                    event.pass_id, event.batch_id, event.cost, event.metrics)
+    ```
 
-As the training of an NMT model is very time consuming, we provide a pre-trained model (pass-00012, ~205M). The model is trained with a cluster of 50 physical nodes (each node has two 6-core CPU). We trained 16 passes (taking about 5 days) with each pass taking about 7 hours. The provided model (pass-00012) has the highest [BLEU Score](#BLEU Score) of 26.92. Run the following command to down load the model:
-```bash
-cd pretrained
-./wmt14_model.sh
-```
+1. Start training
 
-### Usage and Results
+    ```python
+    trainer.train(
+        reader=wmt14_reader,
+        event_handler=event_handler,
+        num_passes=10000,
+        feeding=feeding)
+    ```
 
-Run the following command to perform translation from French to English:
+    ```text
+    Pass 0, Batch 0, Cost 247.408008, {'classification_error_evaluator': 1.0}
+    Pass 0, Batch 10, Cost 212.058789, {'classification_error_evaluator': 0.8737863898277283}
+    ...
+    ```
 
-```bash
-./gen.sh
-```
-where `gen.sh` contains:
+The model training is successful when the `classification_error_evaluator` is lower than 0.35.
+
+## Model Usage
+
+### Download Pre-trained Model
+
+As the training of an NMT model is very time consuming, we provide a pre-trained model (pass-00012, ~205M). The model is trained with a cluster of 50 physical nodes (each node has two 6-core CPU). We trained 16 passes (taking about 5 days) with each pass taking about 7 hours. The provided model (pass-00012) has the highest [BLEU Score](#BLEU Score) of 26.92. Run the following command to download the model:
 
 ```bash
-paddle train \
---job=test \
---config='seqToseq_net.py' \
---save_dir='pretrained/wmt14_model' \
---use_gpu=true \
---num_passes=13 \
---test_pass=12 \
---trainer_count=1 \
---config_args=is_generating=1,gen_trans_file="gen_result" \
-2>&1 | tee 'translation/gen.log'
+cd pretrained
+./wmt14_model.sh
 ```
-Parameters different training are listed as follows:
-- job: set the mode as testing.
-- save_dir: path to the pre-trained model.
-- num_passes and test_pass: load the model parameters from pass $i\epsilon \left [ test\\_pass,num\\_passes-1 \right ]$. Here we only load `data/wmt14_model/pass-00012`.
-- config_args: pass the self-defined command line parameters to model configuration. `is_generating=1` indicates generation mode and `gen_trans_file="gen_result"` represents the file generated.
-
-For translation results please refer to [Illustrative Results](#Illustrative Results).
 
 ### BLEU Evaluation
 
@@ -717,7 +470,7 @@ BLEU = 26.92
 
 ## Summary
 
-End-to-end neural machine translation is a recently developed way to perform machine translations. In this chapter, we introduced the typical "Encoder-Decoder" framework and "attention" mechanism. Since NMT is a typical Sequence-to-Sequence (Seq2Seq) learning problem, tasks such as query rewriting, abstraction generation and single-turn dialogues can all be solved with the model presented in this chapter.
+End-to-end neural machine translation is a recently developed way to perform machine translations. In this chapter, we introduced the typical "Encoder-Decoder" framework and "attention" mechanism. Since NMT is a typical Sequence-to-Sequence (Seq2Seq) learning problem, tasks such as query rewriting, abstraction generation, and single-turn dialogues can all be solved with the model presented in this chapter.
 
 ## References
 
@@ -728,4 +481,4 @@ End-to-end neural machine translation is a recently developed way to perform mac
 5. Papineni K, Roukos S, Ward T, et al. [BLEU: a method for automatic evaluation of machine translation](http://dl.acm.org/citation.cfm?id=1073135)[C]//Proceedings of the 40th annual meeting on association for computational linguistics. Association for Computational Linguistics, 2002: 311-318.
 
 <br/>
-<a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/4.0/"><img alt="知识共享许可协议" style="border-width:0" src="https://i.creativecommons.org/l/by-nc-sa/4.0/88x31.png" /></a><br /><span xmlns:dct="http://purl.org/dc/terms/" href="http://purl.org/dc/dcmitype/Text" property="dct:title" rel="dct:type">本教程</span> 由 <a xmlns:cc="http://creativecommons.org/ns#" href="http://book.paddlepaddle.org" property="cc:attributionName" rel="cc:attributionURL">PaddlePaddle</a> 创作，采用 <a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/4.0/">知识共享 署名-非商业性使用-相同方式共享 4.0 国际 许可协议</a>进行许可。
+This tutorial is contributed by <a xmlns:cc="http://creativecommons.org/ns#" href="http://book.paddlepaddle.org" property="cc:attributionName" rel="cc:attributionURL">PaddlePaddle</a>, and licensed under a <a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/4.0/">Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License</a>.

machine_translation/README.md

 # 机器翻译
 
-本教程源代码目录在[book/machine_translation](https://github.com/PaddlePaddle/book/tree/develop/machine_translation)， 初次使用请参考PaddlePaddle[安装教程](http://www.paddlepaddle.org/doc_cn/build_and_install/index.html)。
+本教程源代码目录在[book/machine_translation](https://github.com/PaddlePaddle/book/tree/develop/machine_translation)， 初次使用请参考PaddlePaddle[安装教程](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/getstarted/build_and_install/docker_install_cn.rst)。
 
 ## 背景介绍
 
@@ -152,54 +152,8 @@ e_{ij}&=align(z_i,h_j)\\\\
 
 ## 数据介绍
 
-### 下载与解压缩
-
 本教程使用[WMT-14](http://www-lium.univ-lemans.fr/~schwenk/cslm_joint_paper/)数据集中的[bitexts(after selection)](http://www-lium.univ-lemans.fr/~schwenk/cslm_joint_paper/data/bitexts.tgz)作为训练集，[dev+test data](http://www-lium.univ-lemans.fr/~schwenk/cslm_joint_paper/data/dev+test.tgz)作为测试集和生成集。
 
-在Linux下，只需简单地运行以下命令：
-
-```bash
-cd data
-./wmt14_data.sh
-```
-
-得到的数据集`data/wmt14`包含如下三个文件夹：
-<p align = "center">
-<table>
-<tr>
-<td>文件夹名</td>
-<td>法英平行语料文件</td>
-<td>文件数</td>
-<td>文件大小</td>
-</tr>
-
-<tr>
-<td>train</td>
-<td>ccb2_pc30.src, ccb2_pc30.trg, etc</td>
-<td>12</td>
-<td>3.55G</td>
-</tr>
-
-<tr>
-<td>test</td>
-<td>ntst1213.src, ntst1213.trg</td>
-<td>2</td>
-<td>1636k</td>
-</tr>
-
-</tr>
-<tr>
-<td>gen</td>
-<td>ntst14.src, ntst14.trg</td>
-<td>2</td>
-<td>864k</td>
-</tr>
-</table>
-</p>
-
-- `XXX.src`是源法语文件，`XXX.trg`是目标英语文件，文件中的每行存放一个句子
-- `XXX.src`和`XXX.trg`的行数一致，且两者任意第$i$行的句子之间都有着一一对应的关系。
-
 ### 数据预处理
 
 我们的预处理流程包括两步：
@@ -220,6 +174,7 @@ cd data
 
 ```python
 # 加载 paddle的python包
+import sys
 import paddle.v2 as paddle
 
 # 配置只使用cpu，并且使用一个cpu进行训练
@@ -256,17 +211,16 @@ wmt14_reader = paddle.batch(
    decoder_size = 512 # 解码器中的GRU隐层大小
   ```
 
-2. 其次，实现编码器框架。分为三步：
+1. 其次，实现编码器框架。分为三步：
 
-   2.1 将在dataset reader中生成的用每个单词在字典中的索引表示的源语言序列
-   转换成one-hot vector表示的源语言序列$\mathbf{w}$，其类型为integer_value_sequence。
+   1 输入是一个文字序列，被表示成整型的序列。序列中每个元素是文字在字典中的索引。所以，我们定义数据层的数据类型为`integer_value_sequence`（整型序列），序列中每个元素的范围是`[0, source_dict_dim)`。
 
    ```python
     src_word_id = paddle.layer.data(
         name='source_language_word',
         type=paddle.data_type.integer_value_sequence(source_dict_dim))
    ```
-   2.2 将上述编码映射到低维语言空间的词向量$\mathbf{s}$。
+   1. 将上述编码映射到低维语言空间的词向量$\mathbf{s}$。
 
    ```python
     src_embedding = paddle.layer.embedding(
@@ -274,7 +228,7 @@ wmt14_reader = paddle.batch(
         size=word_vector_dim,
         param_attr=paddle.attr.ParamAttr(name='_source_language_embedding'))
    ```
-   2.3 用双向GRU编码源语言序列，拼接两个GRU的编码结果得到$\mathbf{h}$。
+   1. 用双向GRU编码源语言序列，拼接两个GRU的编码结果得到$\mathbf{h}$。
 
    ```python
     src_forward = paddle.networks.simple_gru(
@@ -284,16 +238,17 @@ wmt14_reader = paddle.batch(
     encoded_vector = paddle.layer.concat(input=[src_forward, src_backward])
    ```
 
-3. 接着，定义基于注意力机制的解码器框架。分为三步：
+1. 接着，定义基于注意力机制的解码器框架。分为三步：
 
-   3.1 对源语言序列编码后的结果（见2.3），过一个前馈神经网络（Feed Forward Neural Network），得到其映射。
+   1. 对源语言序列编码后的结果（见2.3），过一个前馈神经网络（Feed Forward Neural Network），得到其映射。
 
    ```python
     with paddle.layer.mixed(size=decoder_size) as encoded_proj:
         encoded_proj += paddle.layer.full_matrix_projection(
             input=encoded_vector)
    ```
-   3.2 构造解码器RNN的初始状态。由于解码器需要预测时序目标序列，但在0时刻并没有初始值，所以我们希望对其进行初始化。这里采用的是将源语言序列逆序编码后的最后一个状态进行非线性映射，作为该初始值，即$c_0=h_T$。
+
+   1. 构造解码器RNN的初始状态。由于解码器需要预测时序目标序列，但在0时刻并没有初始值，所以我们希望对其进行初始化。这里采用的是将源语言序列逆序编码后的最后一个状态进行非线性映射，作为该初始值，即$c_0=h_T$。
 
    ```python
     backward_first = paddle.layer.first_seq(input=src_backward)
@@ -302,7 +257,8 @@ wmt14_reader = paddle.batch(
         decoder_boot += paddle.layer.full_matrix_projection(
             input=backward_first)
    ```
-   3.3 定义解码阶段每一个时间步的RNN行为，即根据当前时刻的源语言上下文向量$c_i$、解码器隐层状态$z_i$和目标语言中第$i$个词$u_i$，来预测第$i+1$个词的概率$p_{i+1}$。
+
+   1. 定义解码阶段每一个时间步的RNN行为，即根据当前时刻的源语言上下文向量$c_i$、解码器隐层状态$z_i$和目标语言中第$i$个词$u_i$，来预测第$i+1$个词的概率$p_{i+1}$。
       - decoder_mem记录了前一个时间步的隐层状态$z_i$，其初始状态是decoder_boot。
       - context通过调用`simple_attention`函数，实现公式$c_i=\sum {j=1}^{T}a_{ij}h_j$。其中，enc_vec是$h_j$，enc_proj是$h_j$的映射（见3.1），权重$a_{ij}$的计算已经封装在`simple_attention`函数中。
       - decoder_inputs融合了$c_i$和当前目标词current_word（即$u_i$）的表示。
@@ -339,24 +295,23 @@ wmt14_reader = paddle.batch(
         return out
     ```
 
-4. 训练模式与生成模式下的解码器调用区别。
+1. 定义解码器框架名字，和`gru_decoder_with_attention`函数的前两个输入。注意：这两个输入使用`StaticInput`，具体说明可见[StaticInput文档](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/howto/deep_model/rnn/recurrent_group_cn.md#输入)。
 
-   4.1 定义解码器框架名字，和`gru_decoder_with_attention`函数的前两个输入。注意：这两个输入使用`StaticInput`，具体说明可见[StaticInput文档](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/howto/deep_model/rnn/recurrent_group_cn.md#输入)。
-
-   ```python
+    ```python
     decoder_group_name = "decoder_group"
     group_input1 = paddle.layer.StaticInputV2(input=encoded_vector, is_seq=True)
     group_input2 = paddle.layer.StaticInputV2(input=encoded_proj, is_seq=True)
     group_inputs = [group_input1, group_input2]
-   ```
-   4.2 训练模式下的解码器调用：
+    ```
 
-      - 首先，将目标语言序列的词向量trg_embedding，直接作为训练模式下的current_word传给`gru_decoder_with_attention`函数。
-      - 其次，使用`recurrent_group`函数循环调用`gru_decoder_with_attention`函数。
-      - 接着，使用目标语言的下一个词序列作为标签层lbl，即预测目标词。
-      - 最后，用多类交叉熵损失函数`classification_cost`来计算损失值。
+1. 训练模式下的解码器调用：
 
-   ```python
+   - 首先，将目标语言序列的词向量trg_embedding，直接作为训练模式下的current_word传给`gru_decoder_with_attention`函数。
+   - 其次，使用`recurrent_group`函数循环调用`gru_decoder_with_attention`函数。
+   - 接着，使用目标语言的下一个词序列作为标签层lbl，即预测目标词。
+   - 最后，用多类交叉熵损失函数`classification_cost`来计算损失值。
+
+    ```python
     trg_embedding = paddle.layer.embedding(
         input=paddle.layer.data(
             name='target_language_word',
@@ -379,7 +334,8 @@ wmt14_reader = paddle.batch(
         name='target_language_next_word',
         type=paddle.data_type.integer_value_sequence(target_dict_dim))
     cost = paddle.layer.classification_cost(input=decoder, label=lbl)
-   ```
+    ```
+
 注意：我们提供的配置在Bahdanau的论文\[[4](#参考文献)\]上做了一些简化，可参考[issue #1133](https://github.com/PaddlePaddle/Paddle/issues/1133)。
 
 ### 参数定义
@@ -387,7 +343,6 @@ wmt14_reader = paddle.batch(
 首先依据模型配置的`cost`定义模型参数。
 
 ```python
-# create parameters
 parameters = paddle.parameters.create(cost)
 ```
 
@@ -405,24 +360,30 @@ for param in parameters.keys():
     根据优化目标cost,网络拓扑结构和模型参数来构造出trainer用来训练，在构造时还需指定优化方法，这里使用最基本的SGD方法。
 
     ```python
-    optimizer = paddle.optimizer.Adam(learning_rate=1e-4)
+    optimizer = paddle.optimizer.Adam(
+        learning_rate=5e-5,
+        regularization=paddle.optimizer.L2Regularization(rate=1e-3))
     trainer = paddle.trainer.SGD(cost=cost,
                                  parameters=parameters,
                                  update_equation=optimizer)
     ```
 
-2. 构造event_handler
+1. 构造event_handler
 
     可以通过自定义回调函数来评估训练过程中的各种状态，比如错误率等。下面的代码通过event.batch_id % 10 == 0 指定没10个batch打印一次日志，包含cost等信息。
+
     ```python
     def event_handler(event):
         if isinstance(event, paddle.event.EndIteration):
             if event.batch_id % 10 == 0:
-                print "Pass %d, Batch %d, Cost %f, %s" % (
+                print "\nPass %d, Batch %d, Cost %f, %s" % (
                     event.pass_id, event.batch_id, event.cost, event.metrics)
+            else:
+                sys.stdout.write('.')
+                sys.stdout.flush()
     ```
 
-3. 启动训练：
+1. 启动训练：
 
     ```python
     trainer.train(
@@ -431,30 +392,29 @@ for param in parameters.keys():
         num_passes=10000,
         feeding=feeding)
     ```
-    训练开始后，可以观察到event_handler输出的日志如下：
-    ```text
-    Pass 0, Batch 0, Cost 247.408008, {'classification_error_evaluator': 1.0}
-    Pass 0, Batch 10, Cost 212.058789, {'classification_error_evaluator': 0.8737863898277283}
-    ...
-    ```
 
+训练开始后，可以观察到event_handler输出的日志如下：
+
+```text
+Pass 0, Batch 0, Cost 148.444983, {'classification_error_evaluator': 1.0}
+.........
+Pass 0, Batch 10, Cost 335.896802, {'classification_error_evaluator': 0.9325153231620789}
+.........
+```
+
+    当`classification_error_evaluator`的值低于0.35的时候，表示训练成功。
 
 ## 应用模型
 
 ### 下载预训练的模型
 
 由于NMT模型的训练非常耗时，我们在50个物理节点（每节点含有2颗6核CPU）的集群中，花了5天时间训练了16个pass，其中每个pass耗时7个小时。因此，我们提供了一个预先训练好的模型（pass-00012）供大家直接下载使用。该模型大小为205MB，在所有16个模型中有最高的[BLEU评估](#BLEU评估)值26.92。下载并解压模型的命令如下：
+
 ```bash
 cd pretrained
 ./wmt14_model.sh
 ```
 
-### 应用命令与结果
-
-新版api尚未支持机器翻译的翻译过程，尽请期待。
-
-翻译结果请见[效果展示](#效果展示)。
-
 ### BLEU评估
 
 BLEU(Bilingual Evaluation understudy)是一种广泛使用的机器翻译自动评测指标，由IBM的watson研究中心于2002年提出\[[5](#参考文献)\]，基本出发点是：机器译文越接近专业翻译人员的翻译结果，翻译系统的性能越好。其中，机器译文与人工参考译文之间的接近程度，采用句子精确度（precision）的计算方法，即比较两者的n元词组相匹配的个数，匹配的个数越多，BLEU得分越好。

machine_translation/api_train.py

@@ -105,9 +105,8 @@ def main():
 
     # define optimize method and trainer
     optimizer = paddle.optimizer.Adam(learning_rate=1e-4)
-    trainer = paddle.trainer.SGD(cost=cost,
-                                 parameters=parameters,
-                                 update_equation=optimizer)
+    trainer = paddle.trainer.SGD(
+        cost=cost, parameters=parameters, update_equation=optimizer)
 
     # define data reader
     feeding = {

machine_translation/index.en.html

@@ -42,7 +42,7 @@
 <div id="markdown" style='display:none'>
 # Machine Translation
 
-The source codes is located at [book/machine_translation](https://github.com/PaddlePaddle/book/tree/develop/machine_translation). Please refer to the PaddlePaddle [installation tutorial](http://www.paddlepaddle.org/doc_cn/build_and_install/index.html) if you are a first time user.
+The source codes is located at [book/machine_translation](https://github.com/PaddlePaddle/book/tree/develop/machine_translation). Please refer to the PaddlePaddle [installation tutorial](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/getstarted/build_and_install/docker_install_en.rst) if you are a first time user.
 
 ## Background
 
@@ -227,77 +227,10 @@ Note: $z_{i+1}$ and $p_{i+1}$ are computed the same way as in [Decoder](#Decoder
 
 ## Data Preparation
 
-### Download and Uncompression
-
 This tutorial uses a dataset from [WMT-14](http://www-lium.univ-lemans.fr/~schwenk/cslm_joint_paper/), where [bitexts (after selection)](http://www-lium.univ-lemans.fr/~schwenk/cslm_joint_paper/data/bitexts.tgz) is used as the training set, and [dev+test data](http://www-lium.univ-lemans.fr/~schwenk/cslm_joint_paper/data/dev+test.tgz) is used as test and generation set.
 
-Run the following command in Linux to obtain the data:
-```bash
-cd data
-./wmt14_data.sh
-```
-There are three folders in the downloaded dataset `data/wmt14`:
-<p align = "center">
-<table>
-<tr>
-<td>Folder Name</td>
-<td>French-English Parallel Corpus</td>
-<td>Number of Files</td>
-<td>Size of Files</td>
-</tr>
-
-<tr>
-<td>train</td>
-<td>ccb2_pc30.src, ccb2_pc30.trg, etc</td>
-<td>12</td>
-<td>3.55G</td>
-</tr>
-
-<tr>
-<td>test</td>
-<td>ntst1213.src, ntst1213.trg</td>
-<td>2</td>
-<td>1636k</td>
-</tr>
-
-</tr>
-<tr>
-<td>gen</td>
-<td>ntst14.src, ntst14.trg</td>
-<td>2</td>
-<td>864k</td>
-</tr>
-</table>
-</p>
-
-- `XXX.src` is the source file in French and `XXX.trg`is the target file in English. Each row of the file contains one sentence.
-- `XXX.src` and `XXX.trg` has the same number of rows and there is a one-to-one correspondance between the sentences at any row from the two files.
 
-### User Defined Dataset (Optional)
-
-To use your own dataset, just put it under the `data` folder and organize it as follows
-```text
-user_dataset
-├── train
-│   ├── train_file1.src
-│   ├── train_file1.trg
-│   └── ...
-├── test
-│   ├── test_file1.src
-│   ├── test_file1.trg
-│   └── ...
-├── gen
-│   ├── gen_file1.src
-│   ├── gen_file1.trg
-│   └── ...
-```
-
-Explanation of the directories:
-- First level: `user_dataset`: the name of the user defined dataset.
-- Second level: `train`、`test` and `gen`: these names should not be changed.
-- Third level: Parallel corpus in source language and target language, each with a postfix of `.src` and `.trg`.
-
-### Data Pre-processing
+### Data Preprocessing
 
 There are two steps for pre-processing:
 - Merge the source and target parallel corpus files into one file
@@ -306,248 +239,104 @@ There are two steps for pre-processing:
 
 - Create source dictionary and target dictionary, each containing **DICTSIZE** number of words, including the most frequent (DICTSIZE - 3) fo word from the corpus and 3 special token `<s>` (begin of sequence), `<e>` (end of sequence)  and `<unk>` (unknown words that are not in the vocabulary).
 
-`preprocess.py` is used for pre-processing:
-```python
-python preprocess.py -i INPUT [-d DICTSIZE] [-m]
-```
-- `-i INPUT`: path to the original dataset.
-- `-d DICTSIZE`: number of words in the dictionary. If unspecified, the dictionary will contain all the words appeared in the input dataset.
-- `-m --mergeDict`: merge the source dictionary with target dictionary, making the two dictionaries have the same content.
-
-The specific command to run the script is as follows:
-```python
-python preprocess.py -i data/wmt14 -d 30000
-```
-You will see the following messages after a few minutes:
-```text
-concat parallel corpora for dataset
-build source dictionary for train data
-build target dictionary for train data
-dictionary size is 30000
-```
-The pre-processed data is located at `data/pre-wmt14`:
-```text
-pre-wmt14
-├── train
-│   └── train
-├── test
-│   └── test
-├── gen
-│   └── gen
-├── train.list
-├── test.list
-├── gen.list
-├── src.dict
-└── trg.dict
-```
-- `train`, `test` and `gen`: contains French-English parallel corpus for training, testing and generation. Each row from each file is separated into two columns with a "\t", where the first column is the sequence in French and the second one is in English.
-- `train.list`, `test.list` and `gen.list`: record respectively the path to `train`, `test` and `gen` folders.
-- `src.dict` and `trg.dict`: source (French) and target (English) dictionary. Each dictionary contains 30000 words (29997 most frequent words and 3 special tokens).
-
-### Providing Data to PaddlePaddle
+### A Subset of Dataset
 
-We use `dataprovider.py` to provide data to PaddlePaddle as follows:
+Because the full dataset is very big, to reduce the time for downloading the full dataset. PadddlePaddle package `paddle.dataset.wmt14` provides a preprocessed `subset of dataset`(http://paddlepaddle.bj.bcebos.com/demo/wmt_shrinked_data/wmt14.tgz).
 
-1. Import PyDataProvider2 package from PaddlePaddle and define three special tokens:
+This subset has 193319 instances of training data and 6003 instances of test data. Dictionary size is 30000. Because of the limitation of size of the subset, the effectiveness of trained model from this subset is not guaranteed.
 
-   ```python
-   from paddle.trainer.PyDataProvider2 import *
-   UNK_IDX = 2    #out of vocabulary word
-   START = "<s>"  #begin of sequence
-   END = "<e>"    #end of sequence
-   ```
-2. Use initialization function `hook` to define the input data types (`input_types`) for training and generation:
-   - Training: there are three input sequences, where "source language sequence" and "target language sequence" are input and the "target language next word sequence" is the label.
-   - Generation: there are two input sequences, where the "source language sequence" is the input and “source language sequence id” are the ids for the input data (optional).
+## Training Instructions
 
-  `src_dict_path` in the `hook` function is the path to the source language dictionary, while `trg_dict_path` the path to target language dictionary. `is_generating` is passed from model config file. For more details on the usage of the `hook` function please refer to [Model Config](#Model Config).
+### Initialize PaddlePaddle
 
-   ```python
-   def hook(settings, src_dict_path, trg_dict_path, is_generating, file_list,
-            **kwargs):
-       # job_mode = 1: training 0: generation
-       settings.job_mode = not is_generating
-
-       def fun(dict_path): # load dictionary according to the path
-           out_dict = dict()
-           with open(dict_path, "r") as fin:
-               out_dict = {
-                   line.strip(): line_count
-                   for line_count, line in enumerate(fin)
-               }
-           return out_dict
-
-       settings.src_dict = fun(src_dict_path)
-       settings.trg_dict = fun(trg_dict_path)
-
-       if settings.job_mode:                                  #training
-           settings.input_types = {
-               'source_language_word':                        #source language sequence
-               integer_value_sequence(len(settings.src_dict)),
-               'target_language_word':                        #target language sequence
-               integer_value_sequence(len(settings.trg_dict)),
-               'target_language_next_word':                   #target language next word sequence
-               integer_value_sequence(len(settings.trg_dict))
-           }
-       else:                                                  #generation
-           settings.input_types = {
-               'source_language_word':                        #source language sequence
-               integer_value_sequence(len(settings.src_dict)),
-               'sent_id':                                     #source language sequence id
-               integer_value_sequence(len(open(file_list[0], "r").readlines()))
-           }
-   ```
-3. Use `process` function to open the file `file_name`, read each row of the file, convert the data to be compatible with `input_types`, and then use `yield` to return to PaddlePaddle process. More specifically
-
-   - add `<s>` to the beginning of each source language sequence and add `<e>` to the end, producing "source_language_word".
-   - add `<s>` to the beginning of each target language senquence, producing "target_language_word".
-   - add `<e>` to the end of each target language senquence, producing "target_language_next_word".
-
-   ```python
-   def _get_ids(s, dictionary): # get the location of each word from the source language sequence in the dictionary
-       words = s.strip().split()
-       return [dictionary[START]] + \
-              [dictionary.get(w, UNK_IDX) for w in words] + \
-              [dictionary[END]]
-
-   @provider(init_hook=hook, pool_size=50000)
-   def process(settings, file_name):
-       with open(file_name, 'r') as f:
-           for line_count, line in enumerate(f):
-               line_split = line.strip().split('\t')
-               if settings.job_mode and len(line_split) != 2:
-                   continue
-               src_seq = line_split[0]
-               src_ids = _get_ids(src_seq, settings.src_dict)
-
-               if settings.job_mode:
-                   trg_seq = line_split[1]
-                   trg_words = trg_seq.split()
-                   trg_ids = [settings.trg_dict.get(w, UNK_IDX) for w in trg_words]
-
-                   # sequence with length longer than 80 with be removed during training to avoid an overly deep RNN.
-                   if len(src_ids) > 80 or len(trg_ids) > 80:
-                       continue
-                   trg_ids_next = trg_ids + [settings.trg_dict[END]]
-                   trg_ids = [settings.trg_dict[START]] + trg_ids
-                   yield {
-                       'source_language_word': src_ids,
-                       'target_language_word': trg_ids,
-                       'target_language_next_word': trg_ids_next
-                   }
-               else:
-                   yield {'source_language_word': src_ids, 'sent_id': [line_count]}
-   ```
-Note: The size of the training data is 3.55G. For machines with limited memories, it is recommended to use `pool_size` to set the number of data samples stored in memory.
-
-## Model Config
-
-### Data Definition
-
-1. Specify the path to data and source/target dictionaries. `is_generating` accepts argument passed from command lines and is used to denote whether the current configuration is for training (default) or generation. See [Usage and Resutls](#Usage and Results).
-
-   ```python
-   import os
-   from paddle.trainer_config_helpers import *
+```python
+import sys
+import paddle.v2 as paddle
 
-   data_dir = "./data/pre-wmt14" # data path
-   src_lang_dict = os.path.join(data_dir, 'src.dict') # path to the source language dictionary
-   trg_lang_dict = os.path.join(data_dir, 'trg.dict') # path to the target language dictionary
-   is_generating = get_config_arg("is_generating", bool, False) # config mode
-   ```
-2. Use `define_py_data_sources2` to get data from `dataprovider.py`, and use `args` variable to input the source/target language dicitonary path and config mode.
+# train with a single CPU
+paddle.init(use_gpu=False, trainer_count=1)
+```
 
-   ```python
-   if not is_generating:
-       train_list = os.path.join(data_dir, 'train.list')
-       test_list = os.path.join(data_dir, 'test.list')
-   else:
-       train_list = None
-       test_list = os.path.join(data_dir, 'gen.list')
-
-   define_py_data_sources2(
-       train_list,
-       test_list,
-       module="dataprovider",
-       obj="process",
-       args={
-           "src_dict_path": src_lang_dict, # source language dictionary path
-           "trg_dict_path": trg_lang_dict, # target language dictionary path
-           "is_generating": is_generating  # config mode
-       })
-   ```
+### Define DataSet
 
-### Algorithm Configuration
+We will define dictionary size, and create [**data reader**](https://github.com/PaddlePaddle/Paddle/tree/develop/doc/design/reader#python-data-reader-design-doc) for WMT-14 dataset.
 
 ```python
-settings(
-    learning_method = AdamOptimizer(),
-    batch_size = 50,
-    learning_rate = 5e-4)
+# source and target dict dim.
+dict_size = 30000
+
+feeding = {
+    'source_language_word': 0,
+    'target_language_word': 1,
+    'target_language_next_word': 2
+}
+wmt14_reader = paddle.batch(
+    paddle.reader.shuffle(
+        paddle.dataset.wmt14.train(dict_size=dict_size), buf_size=8192),
+    batch_size=5)
 ```
-This tutorial will use the default SGD and Adam learning algorithm, with a learning rate of 5e-4. Note that the `batch_size = 50` denotes generating 50 sequence each time.
 
-### Model Structure
+### Model Configuration
 
 1. Define some global variables
 
    ```python
-   source_dict_dim = len(open(src_lang_dict, "r").readlines()) # size of the source language dictionary
-   target_dict_dim = len(open(trg_lang_dict, "r").readlines()) # size of target language dictionary
-   word_vector_dim = 512 # dimensionality of word vector
-   encoder_size = 512      # dimensionality of the hidden state of encoder GRU
-   decoder_size = 512    # dimentionality of the hidden state of decoder GRU
-
-   if is_generating:
-       beam_size=3    # beam size for the beam search algorithm
-       max_length=250 # maximum length for the generated sentence
-       gen_trans_file = get_config_arg("gen_trans_file", str, None) # generate file
+   source_dict_dim = dict_size # source language dictionary size
+   target_dict_dim = dict_size # destination language dictionary size
+   word_vector_dim = 512 # word embedding dimension
+   encoder_size = 512 # hidden layer size of GRU in encoder
+   decoder_size = 512 # hidden layer size of GRU in decoder
   ```
 
-2. Implement Encoder as follows:
-
-   2.1 Input one-hot vector representations $\mathbf{w}$ converted with `dataprovider.py` from the source language sentence
+1. Implement Encoder as follows:
+   1. Input is a sequence of words represented by an integer word index sequence. So we define data layer of data type `integer_value_sequence`. The value range of each element in the sequence is `[0, source_dict_dim)`
 
    ```python
-   src_word_id = data_layer(name='source_language_word', size=source_dict_dim)
+    src_word_id = paddle.layer.data(
+        name='source_language_word',
+        type=paddle.data_type.integer_value_sequence(source_dict_dim))
    ```
-   2.2 Map the one-hot vector into a word vector $\mathbf{s}$ in a low-dimensional semantic space
+
+   1. Map the one-hot vector (represented by word index) into a word vector $\mathbf{s}$ in a low-dimensional semantic space
 
    ```python
-   src_embedding = embedding_layer(
-       input=src_word_id,
-       size=word_vector_dim,
-       param_attr=ParamAttr(name='_source_language_embedding'))
+    src_embedding = paddle.layer.embedding(
+        input=src_word_id,
+        size=word_vector_dim,
+        param_attr=paddle.attr.ParamAttr(name='_source_language_embedding'))
    ```
-   2.3 Use bi-direcitonal GRU to encode the source language sequence, and concatenate the encoding outputs from the two GRUs to get $\mathbf{h}$
+
+   1. Use bi-direcitonal GRU to encode the source language sequence, and concatenate the encoding outputs from the two GRUs to get $\mathbf{h}$
 
    ```python
-   src_forward = simple_gru(input=src_embedding, size=encoder_size)
-   src_backward = simple_gru(
-       input=src_embedding, size=encoder_size, reverse=True)
-   encoded_vector = concat_layer(input=[src_forward, src_backward])
+    src_forward = paddle.networks.simple_gru(
+        input=src_embedding, size=encoder_size)
+    src_backward = paddle.networks.simple_gru(
+        input=src_embedding, size=encoder_size, reverse=True)
+    encoded_vector = paddle.layer.concat(input=[src_forward, src_backward])
    ```
 
-3. Implement Attention-based Decoder as follows:
+1. Implement Attention-based Decoder as follows:
 
-   3.1 Get a projection of the encoding (c.f. 2.3) of the source language sequence by passing it into a feed forward neural network
+   1. Get a projection of the encoding (c.f. 2.3) of the source language sequence by passing it into a feed forward neural network
 
    ```python
-   with mixed_layer(size=decoder_size) as encoded_proj:
-       encoded_proj += full_matrix_projection(input=encoded_vector)
+    with paddle.layer.mixed(size=decoder_size) as encoded_proj:
+        encoded_proj += paddle.layer.full_matrix_projection(
+            input=encoded_vector)
    ```
 
-   3.2 Use a non-linear transformation of the last hidden state of the backward GRU on the source language sentence as the initial state of the decoder RNN $c_0=h_T$
+   1. Use a non-linear transformation of the last hidden state of the backward GRU on the source language sentence as the initial state of the decoder RNN $c_0=h_T$
 
    ```python
-   backward_first = first_seq(input=src_backward)
-   with mixed_layer(
-           size=decoder_size,
-           act=TanhActivation(), ) as decoder_boot:
-       decoder_boot += full_matrix_projection(input=backward_first)
+    backward_first = paddle.layer.first_seq(input=src_backward)
+    with paddle.layer.mixed(
+            size=decoder_size, act=paddle.activation.Tanh()) as decoder_boot:
+        decoder_boot += paddle.layer.full_matrix_projection(
+            input=backward_first)
    ```
 
-   3.3 Define the computation in each time step for the decoder RNN, i.e., according to the current context vector $c_i$, hidden state for the decoder $z_i$ and the $i$-th word $u_i$ in the target language to predict the probability $p_{i+1}$ for the $i+1$-th word.
+   1. Define the computation in each time step for the decoder RNN, i.e., according to the current context vector $c_i$, hidden state for the decoder $z_i$ and the $i$-th word $u_i$ in the target language to predict the probability $p_{i+1}$ for the $i+1$-th word.
 
       - decoder_mem records the hidden state $z_i$ from the previous time step, with an initial state as decoder_boot.
       - context is computed via `simple_attention` as $c_i=\sum {j=1}^{T}a_{ij}h_j$, where enc_vec is the projection of $h_j$ and enc_proj is the projection of $h_j$ (c.f. 3.1). $a_{ij}$ is calculated within `simple_attention`.
@@ -556,184 +345,148 @@ This tutorial will use the default SGD and Adam learning algorithm, with a learn
       - Softmax normalization is used in the end to computed the probability of words, i.e., $p\left ( u_i|u_{&lt;i},\mathbf{x} \right )=softmax(W_sz_i+b_z)$. The output is returned.
 
    ```python
-   def gru_decoder_with_attention(enc_vec, enc_proj, current_word):
-       decoder_mem = memory(
-           name='gru_decoder', size=decoder_size, boot_layer=decoder_boot)
-
-       context = simple_attention(
-           encoded_sequence=enc_vec,
-           encoded_proj=enc_proj,
-           decoder_state=decoder_mem, )
-
-       with mixed_layer(size=decoder_size * 3) as decoder_inputs:
-           decoder_inputs += full_matrix_projection(input=context)
-           decoder_inputs += full_matrix_projection(input=current_word)
-
-       gru_step = gru_step_layer(
-           name='gru_decoder',
-           input=decoder_inputs,
-           output_mem=decoder_mem,
-           size=decoder_size)
-
-       with mixed_layer(
-               size=target_dict_dim, bias_attr=True,
-               act=SoftmaxActivation()) as out:
-           out += full_matrix_projection(input=gru_step)
-       return out
+    def gru_decoder_with_attention(enc_vec, enc_proj, current_word):
+
+        decoder_mem = paddle.layer.memory(
+            name='gru_decoder', size=decoder_size, boot_layer=decoder_boot)
+
+        context = paddle.networks.simple_attention(
+            encoded_sequence=enc_vec,
+            encoded_proj=enc_proj,
+            decoder_state=decoder_mem)
+
+        with paddle.layer.mixed(size=decoder_size * 3) as decoder_inputs:
+            decoder_inputs += paddle.layer.full_matrix_projection(input=context)
+            decoder_inputs += paddle.layer.full_matrix_projection(
+                input=current_word)
+
+        gru_step = paddle.layer.gru_step(
+            name='gru_decoder',
+            input=decoder_inputs,
+            output_mem=decoder_mem,
+            size=decoder_size)
+
+        with paddle.layer.mixed(
+                size=target_dict_dim,
+                bias_attr=True,
+                act=paddle.activation.Softmax()) as out:
+            out += paddle.layer.full_matrix_projection(input=gru_step)
+        return out
     ```
 
-4. Decoder differences between the training and generation
+1. Define the name for the decoder and the first two input for `gru_decoder_with_attention`. Note that `StaticInput` is used for the two inputs. Please refer to [StaticInput Document](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/howto/deep_model/rnn/recurrent_group_cn.md#输入) for more details.
 
-   4.1 Define the name for the decoder and the first two input for `gru_decoder_with_attention`. Note that `StaticInput` is used for the two inputs. Please refer to [StaticInput Document](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/howto/deep_model/rnn/recurrent_group_cn.md#输入) for more details.
-
-   ```python
-   decoder_group_name = "decoder_group"
-   group_input1 = StaticInput(input=encoded_vector, is_seq=True)
-   group_input2 =  StaticInput(input=encoded_proj, is_seq=True)
-   group_inputs = [group_input1, group_input2]
-   ```
+    ```python
+    decoder_group_name = "decoder_group"
+    group_input1 = paddle.layer.StaticInputV2(input=encoded_vector, is_seq=True)
+    group_input2 = paddle.layer.StaticInputV2(input=encoded_proj, is_seq=True)
+    group_inputs = [group_input1, group_input2]
+    ```
 
-   4.2 In training mode:
+1. Training mode:
 
-      - word embedding from the target langauge trg_embedding is passed to `gru_decoder_with_attention` as current_word.
+      - word embedding from the target language trg_embedding is passed to `gru_decoder_with_attention` as current_word.
       - `recurrent_group` calls `gru_decoder_with_attention` in a recurrent way
       - the sequence of next words from the target language is used as label (lbl)
       - multi-class cross-entropy (`classification_cost`) is used to calculate the cost
 
-   ```python
-   if not is_generating:
-       trg_embedding = embedding_layer(
-           input=data_layer(
-               name='target_language_word', size=target_dict_dim),
-           size=word_vector_dim,
-           param_attr=ParamAttr(name='_target_language_embedding'))
-       group_inputs.append(trg_embedding)
-
-       decoder = recurrent_group(
-           name=decoder_group_name,
-           step=gru_decoder_with_attention,
-           input=group_inputs)
-
-       lbl = data_layer(name='target_language_next_word', size=target_dict_dim)
-       cost = classification_cost(input=decoder, label=lbl)
-       outputs(cost)
-   ```
+    ```python
+    trg_embedding = paddle.layer.embedding(
+        input=paddle.layer.data(
+            name='target_language_word',
+            type=paddle.data_type.integer_value_sequence(target_dict_dim)),
+        size=word_vector_dim,
+        param_attr=paddle.attr.ParamAttr(name='_target_language_embedding'))
+    group_inputs.append(trg_embedding)
+
+    # For decoder equipped with attention mechanism, in training,
+    # target embeding (the groudtruth) is the data input,
+    # while encoded source sequence is accessed to as an unbounded memory.
+    # Here, the StaticInput defines a read-only memory
+    # for the recurrent_group.
+    decoder = paddle.layer.recurrent_group(
+        name=decoder_group_name,
+        step=gru_decoder_with_attention,
+        input=group_inputs)
+
+    lbl = paddle.layer.data(
+        name='target_language_next_word',
+        type=paddle.data_type.integer_value_sequence(target_dict_dim))
+    cost = paddle.layer.classification_cost(input=decoder, label=lbl)
+    ```
 
-   4.3 In generation mode:
+Note: Our configuration is based on Bahdanau et al. \[[4](#Reference)\] but with a few simplifications. Please refer to [issue #1133](https://github.com/PaddlePaddle/Paddle/issues/1133) for more details.
 
-      - during generation, as the decoder RNN will take the word vector generated from the previous time step as input, `GeneratedInput` is used to implement this automatically. Please refer to [GeneratedInput Document](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/howto/deep_model/rnn/recurrent_group_cn.md#输入) for details.
-      - `beam_search` will call `gru_decoder_with_attention` to generate id
-      - `seqtext_printer_evaluator` outputs the generated sentence to `gen_trans_file` according to `trg_lang_dict`
+### Create Parameters
 
-   ```python
-   else:
-       trg_embedding = GeneratedInput(
-           size=target_dict_dim,
-           embedding_name='_target_language_embedding',
-           embedding_size=word_vector_dim)
-       group_inputs.append(trg_embedding)
-
-       beam_gen = beam_search(
-           name=decoder_group_name,
-           step=gru_decoder_with_attention,
-           input=group_inputs,
-           bos_id=0,
-           eos_id=1,
-           beam_size=beam_size,
-           max_length=max_length)
-
-       seqtext_printer_evaluator(
-           input=beam_gen,
-           id_input=data_layer(
-               name="sent_id", size=1),
-           dict_file=trg_lang_dict,
-           result_file=gen_trans_file)
-       outputs(beam_gen)
-   ```
-Note: Our configuration is based on Bahdanau et al. \[[4](#Reference)\] but with a few simplifications. Please refer to [issue #1133](https://github.com/PaddlePaddle/Paddle/issues/1133) for more details.
+Create every parameter that `cost` layer needs.
 
+```python
+parameters = paddle.parameters.create(cost)
+```
+
+We can get parameter names. If the parameter name is not specified during model configuration, it will be generated.
+
+```python
+for param in parameters.keys():
+    print param
+```
 
 ## Model Training
 
-Training can be started with the following command:
+1. Create trainer
 
-```bash
-./train.sh
-```
-where `train.sh` contains
+    We need to tell trainer what to optimize, and how to optimize. Here trainer will optimize `cost` layer using stochastic gradient descent (SDG).
 
-```bash
-paddle train \
---config='seqToseq_net.py' \
---save_dir='model' \
---use_gpu=false \
---num_passes=16 \
---show_parameter_stats_period=100 \
---trainer_count=4 \
---log_period=10 \
---dot_period=5 \
-2>&1 | tee 'train.log'
-```
-- config: configuration file for the network
-- save_dir: path to save the trained model
-- use_gpu: whether to use GPU for training; CPU is used here
-- num_passes: number of passes for training. In PaddlePaddle, one pass meansing one pass of complete training pass using all the data in the training set
-- show_parameter_stats_period: here we show the statistics of parameters every 100 batches
-- trainer_count: the number of CPU processes or GPU devices
-- log_period: here we print log every 10 batches
-- dot_period: we print one "." every 5 batches
-
-The training loss will the printed every 10 batches, and you will see messages like those below:
-```text
-I0719 19:16:45.952062 15563 TrainerInternal.cpp:160]  Batch=10 samples=500 AvgCost=198.475 CurrentCost=198.475 Eval: classification_error_evaluator=0.737155  CurrentEval: classification_error_evaluator=0.737155
-I0719 19:17:56.707319 15563 TrainerInternal.cpp:160]  Batch=20 samples=1000 AvgCost=157.479 CurrentCost=116.483 Eval: classification_error_evaluator=0.698392  CurrentEval: classification_error_evaluator=0.659065
-.....
-```
-- AvgCost: average cost from batch-0 to the current batch.
-- CurrentCost: the cost for the current batch
-- classification\_error\_evaluator (Eval): average error rate from evaluator-0 to the current evaluator for each word
-- classification\_error\_evaluator (CurrentEval): error rate for the current evaluator for each word
+    ```python
+    optimizer = paddle.optimizer.Adam(
+        learning_rate=5e-5,
+        regularization=paddle.optimizer.L2Regularization(rate=1e-3))
+    trainer = paddle.trainer.SGD(cost=cost,
+                                 parameters=parameters,
+                                 update_equation=optimizer)
+    ```
 
-The model training is successful when the classification\_error\_evaluator is lower than 0.35.
+1. Define event handler
 
-## Model Usage
+    The event handler is a callback function invoked by trainer when an event happens. Here we will print log in event handler.
 
-### Download Pre-trained Model
+    ```python
+    def event_handler(event):
+        if isinstance(event, paddle.event.EndIteration):
+            if event.batch_id % 10 == 0:
+                print "\nPass %d, Batch %d, Cost %f, %s" % (
+                    event.pass_id, event.batch_id, event.cost, event.metrics)
+    ```
 
-As the training of an NMT model is very time consuming, we provide a pre-trained model (pass-00012, ~205M). The model is trained with a cluster of 50 physical nodes (each node has two 6-core CPU). We trained 16 passes (taking about 5 days) with each pass taking about 7 hours. The provided model (pass-00012) has the highest [BLEU Score](#BLEU Score) of 26.92. Run the following command to down load the model:
-```bash
-cd pretrained
-./wmt14_model.sh
-```
+1. Start training
 
-### Usage and Results
+    ```python
+    trainer.train(
+        reader=wmt14_reader,
+        event_handler=event_handler,
+        num_passes=10000,
+        feeding=feeding)
+    ```
 
-Run the following command to perform translation from French to English:
+    ```text
+    Pass 0, Batch 0, Cost 247.408008, {'classification_error_evaluator': 1.0}
+    Pass 0, Batch 10, Cost 212.058789, {'classification_error_evaluator': 0.8737863898277283}
+    ...
+    ```
 
-```bash
-./gen.sh
-```
-where `gen.sh` contains:
+The model training is successful when the `classification_error_evaluator` is lower than 0.35.
+
+## Model Usage
+
+### Download Pre-trained Model
+
+As the training of an NMT model is very time consuming, we provide a pre-trained model (pass-00012, ~205M). The model is trained with a cluster of 50 physical nodes (each node has two 6-core CPU). We trained 16 passes (taking about 5 days) with each pass taking about 7 hours. The provided model (pass-00012) has the highest [BLEU Score](#BLEU Score) of 26.92. Run the following command to download the model:
 
 ```bash
-paddle train \
---job=test \
---config='seqToseq_net.py' \
---save_dir='pretrained/wmt14_model' \
---use_gpu=true \
---num_passes=13 \
---test_pass=12 \
---trainer_count=1 \
---config_args=is_generating=1,gen_trans_file="gen_result" \
-2>&1 | tee 'translation/gen.log'
+cd pretrained
+./wmt14_model.sh
 ```
-Parameters different training are listed as follows:
-- job: set the mode as testing.
-- save_dir: path to the pre-trained model.
-- num_passes and test_pass: load the model parameters from pass $i\epsilon \left [ test\\_pass,num\\_passes-1 \right ]$. Here we only load `data/wmt14_model/pass-00012`.
-- config_args: pass the self-defined command line parameters to model configuration. `is_generating=1` indicates generation mode and `gen_trans_file="gen_result"` represents the file generated.
-
-For translation results please refer to [Illustrative Results](#Illustrative Results).
 
 ### BLEU Evaluation
 
@@ -759,7 +512,7 @@ BLEU = 26.92
 
 ## Summary
 
-End-to-end neural machine translation is a recently developed way to perform machine translations. In this chapter, we introduced the typical "Encoder-Decoder" framework and "attention" mechanism. Since NMT is a typical Sequence-to-Sequence (Seq2Seq) learning problem, tasks such as query rewriting, abstraction generation and single-turn dialogues can all be solved with the model presented in this chapter.
+End-to-end neural machine translation is a recently developed way to perform machine translations. In this chapter, we introduced the typical "Encoder-Decoder" framework and "attention" mechanism. Since NMT is a typical Sequence-to-Sequence (Seq2Seq) learning problem, tasks such as query rewriting, abstraction generation, and single-turn dialogues can all be solved with the model presented in this chapter.
 
 ## References
 
@@ -770,7 +523,7 @@ End-to-end neural machine translation is a recently developed way to perform mac
 5. Papineni K, Roukos S, Ward T, et al. [BLEU: a method for automatic evaluation of machine translation](http://dl.acm.org/citation.cfm?id=1073135)[C]//Proceedings of the 40th annual meeting on association for computational linguistics. Association for Computational Linguistics, 2002: 311-318.
 
 <br/>
-<a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/4.0/"><img alt="知识共享许可协议" style="border-width:0" src="https://i.creativecommons.org/l/by-nc-sa/4.0/88x31.png" /></a><br /><span xmlns:dct="http://purl.org/dc/terms/" href="http://purl.org/dc/dcmitype/Text" property="dct:title" rel="dct:type">本教程</span> 由 <a xmlns:cc="http://creativecommons.org/ns#" href="http://book.paddlepaddle.org" property="cc:attributionName" rel="cc:attributionURL">PaddlePaddle</a> 创作，采用 <a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/4.0/">知识共享 署名-非商业性使用-相同方式共享 4.0 国际 许可协议</a>进行许可。
+This tutorial is contributed by <a xmlns:cc="http://creativecommons.org/ns#" href="http://book.paddlepaddle.org" property="cc:attributionName" rel="cc:attributionURL">PaddlePaddle</a>, and licensed under a <a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/4.0/">Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License</a>.
 
 </div>
 <!-- You can change the lines below now. -->

machine_translation/index.html

@@ -42,7 +42,7 @@
 <div id="markdown" style='display:none'>
 # 机器翻译
 
-本教程源代码目录在[book/machine_translation](https://github.com/PaddlePaddle/book/tree/develop/machine_translation)， 初次使用请参考PaddlePaddle[安装教程](http://www.paddlepaddle.org/doc_cn/build_and_install/index.html)。
+本教程源代码目录在[book/machine_translation](https://github.com/PaddlePaddle/book/tree/develop/machine_translation)， 初次使用请参考PaddlePaddle[安装教程](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/getstarted/build_and_install/docker_install_cn.rst)。
 
 ## 背景介绍
 
@@ -194,54 +194,8 @@ e_{ij}&=align(z_i,h_j)\\\\
 
 ## 数据介绍
 
-### 下载与解压缩
-
 本教程使用[WMT-14](http://www-lium.univ-lemans.fr/~schwenk/cslm_joint_paper/)数据集中的[bitexts(after selection)](http://www-lium.univ-lemans.fr/~schwenk/cslm_joint_paper/data/bitexts.tgz)作为训练集，[dev+test data](http://www-lium.univ-lemans.fr/~schwenk/cslm_joint_paper/data/dev+test.tgz)作为测试集和生成集。
 
-在Linux下，只需简单地运行以下命令：
-
-```bash
-cd data
-./wmt14_data.sh
-```
-
-得到的数据集`data/wmt14`包含如下三个文件夹：
-<p align = "center">
-<table>
-<tr>
-<td>文件夹名</td>
-<td>法英平行语料文件</td>
-<td>文件数</td>
-<td>文件大小</td>
-</tr>
-
-<tr>
-<td>train</td>
-<td>ccb2_pc30.src, ccb2_pc30.trg, etc</td>
-<td>12</td>
-<td>3.55G</td>
-</tr>
-
-<tr>
-<td>test</td>
-<td>ntst1213.src, ntst1213.trg</td>
-<td>2</td>
-<td>1636k</td>
-</tr>
-
-</tr>
-<tr>
-<td>gen</td>
-<td>ntst14.src, ntst14.trg</td>
-<td>2</td>
-<td>864k</td>
-</tr>
-</table>
-</p>
-
-- `XXX.src`是源法语文件，`XXX.trg`是目标英语文件，文件中的每行存放一个句子
-- `XXX.src`和`XXX.trg`的行数一致，且两者任意第$i$行的句子之间都有着一一对应的关系。
-
 ### 数据预处理
 
 我们的预处理流程包括两步：
@@ -262,6 +216,7 @@ cd data
 
 ```python
 # 加载 paddle的python包
+import sys
 import paddle.v2 as paddle
 
 # 配置只使用cpu，并且使用一个cpu进行训练
@@ -298,17 +253,16 @@ wmt14_reader = paddle.batch(
    decoder_size = 512 # 解码器中的GRU隐层大小
   ```
 
-2. 其次，实现编码器框架。分为三步：
+1. 其次，实现编码器框架。分为三步：
 
-   2.1 将在dataset reader中生成的用每个单词在字典中的索引表示的源语言序列
-   转换成one-hot vector表示的源语言序列$\mathbf{w}$，其类型为integer_value_sequence。
+   1 输入是一个文字序列，被表示成整型的序列。序列中每个元素是文字在字典中的索引。所以，我们定义数据层的数据类型为`integer_value_sequence`（整型序列），序列中每个元素的范围是`[0, source_dict_dim)`。
 
    ```python
     src_word_id = paddle.layer.data(
         name='source_language_word',
         type=paddle.data_type.integer_value_sequence(source_dict_dim))
    ```
-   2.2 将上述编码映射到低维语言空间的词向量$\mathbf{s}$。
+   1. 将上述编码映射到低维语言空间的词向量$\mathbf{s}$。
 
    ```python
     src_embedding = paddle.layer.embedding(
@@ -316,7 +270,7 @@ wmt14_reader = paddle.batch(
         size=word_vector_dim,
         param_attr=paddle.attr.ParamAttr(name='_source_language_embedding'))
    ```
-   2.3 用双向GRU编码源语言序列，拼接两个GRU的编码结果得到$\mathbf{h}$。
+   1. 用双向GRU编码源语言序列，拼接两个GRU的编码结果得到$\mathbf{h}$。
 
    ```python
     src_forward = paddle.networks.simple_gru(
@@ -326,16 +280,17 @@ wmt14_reader = paddle.batch(
     encoded_vector = paddle.layer.concat(input=[src_forward, src_backward])
    ```
 
-3. 接着，定义基于注意力机制的解码器框架。分为三步：
+1. 接着，定义基于注意力机制的解码器框架。分为三步：
 
-   3.1 对源语言序列编码后的结果（见2.3），过一个前馈神经网络（Feed Forward Neural Network），得到其映射。
+   1. 对源语言序列编码后的结果（见2.3），过一个前馈神经网络（Feed Forward Neural Network），得到其映射。
 
    ```python
     with paddle.layer.mixed(size=decoder_size) as encoded_proj:
         encoded_proj += paddle.layer.full_matrix_projection(
             input=encoded_vector)
    ```
-   3.2 构造解码器RNN的初始状态。由于解码器需要预测时序目标序列，但在0时刻并没有初始值，所以我们希望对其进行初始化。这里采用的是将源语言序列逆序编码后的最后一个状态进行非线性映射，作为该初始值，即$c_0=h_T$。
+
+   1. 构造解码器RNN的初始状态。由于解码器需要预测时序目标序列，但在0时刻并没有初始值，所以我们希望对其进行初始化。这里采用的是将源语言序列逆序编码后的最后一个状态进行非线性映射，作为该初始值，即$c_0=h_T$。
 
    ```python
     backward_first = paddle.layer.first_seq(input=src_backward)
@@ -344,7 +299,8 @@ wmt14_reader = paddle.batch(
         decoder_boot += paddle.layer.full_matrix_projection(
             input=backward_first)
    ```
-   3.3 定义解码阶段每一个时间步的RNN行为，即根据当前时刻的源语言上下文向量$c_i$、解码器隐层状态$z_i$和目标语言中第$i$个词$u_i$，来预测第$i+1$个词的概率$p_{i+1}$。
+
+   1. 定义解码阶段每一个时间步的RNN行为，即根据当前时刻的源语言上下文向量$c_i$、解码器隐层状态$z_i$和目标语言中第$i$个词$u_i$，来预测第$i+1$个词的概率$p_{i+1}$。
       - decoder_mem记录了前一个时间步的隐层状态$z_i$，其初始状态是decoder_boot。
       - context通过调用`simple_attention`函数，实现公式$c_i=\sum {j=1}^{T}a_{ij}h_j$。其中，enc_vec是$h_j$，enc_proj是$h_j$的映射（见3.1），权重$a_{ij}$的计算已经封装在`simple_attention`函数中。
       - decoder_inputs融合了$c_i$和当前目标词current_word（即$u_i$）的表示。
@@ -381,24 +337,23 @@ wmt14_reader = paddle.batch(
         return out
     ```
 
-4. 训练模式与生成模式下的解码器调用区别。
+1. 定义解码器框架名字，和`gru_decoder_with_attention`函数的前两个输入。注意：这两个输入使用`StaticInput`，具体说明可见[StaticInput文档](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/howto/deep_model/rnn/recurrent_group_cn.md#输入)。
 
-   4.1 定义解码器框架名字，和`gru_decoder_with_attention`函数的前两个输入。注意：这两个输入使用`StaticInput`，具体说明可见[StaticInput文档](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/howto/deep_model/rnn/recurrent_group_cn.md#输入)。
-
-   ```python
+    ```python
     decoder_group_name = "decoder_group"
     group_input1 = paddle.layer.StaticInputV2(input=encoded_vector, is_seq=True)
     group_input2 = paddle.layer.StaticInputV2(input=encoded_proj, is_seq=True)
     group_inputs = [group_input1, group_input2]
-   ```
-   4.2 训练模式下的解码器调用：
+    ```
 
-      - 首先，将目标语言序列的词向量trg_embedding，直接作为训练模式下的current_word传给`gru_decoder_with_attention`函数。
-      - 其次，使用`recurrent_group`函数循环调用`gru_decoder_with_attention`函数。
-      - 接着，使用目标语言的下一个词序列作为标签层lbl，即预测目标词。
-      - 最后，用多类交叉熵损失函数`classification_cost`来计算损失值。
+1. 训练模式下的解码器调用：
 
-   ```python
+   - 首先，将目标语言序列的词向量trg_embedding，直接作为训练模式下的current_word传给`gru_decoder_with_attention`函数。
+   - 其次，使用`recurrent_group`函数循环调用`gru_decoder_with_attention`函数。
+   - 接着，使用目标语言的下一个词序列作为标签层lbl，即预测目标词。
+   - 最后，用多类交叉熵损失函数`classification_cost`来计算损失值。
+
+    ```python
     trg_embedding = paddle.layer.embedding(
         input=paddle.layer.data(
             name='target_language_word',
@@ -421,7 +376,8 @@ wmt14_reader = paddle.batch(
         name='target_language_next_word',
         type=paddle.data_type.integer_value_sequence(target_dict_dim))
     cost = paddle.layer.classification_cost(input=decoder, label=lbl)
-   ```
+    ```
+
 注意：我们提供的配置在Bahdanau的论文\[[4](#参考文献)\]上做了一些简化，可参考[issue #1133](https://github.com/PaddlePaddle/Paddle/issues/1133)。
 
 ### 参数定义
@@ -429,7 +385,6 @@ wmt14_reader = paddle.batch(
 首先依据模型配置的`cost`定义模型参数。
 
 ```python
-# create parameters
 parameters = paddle.parameters.create(cost)
 ```
 
@@ -447,24 +402,30 @@ for param in parameters.keys():
     根据优化目标cost,网络拓扑结构和模型参数来构造出trainer用来训练，在构造时还需指定优化方法，这里使用最基本的SGD方法。
 
     ```python
-    optimizer = paddle.optimizer.Adam(learning_rate=1e-4)
+    optimizer = paddle.optimizer.Adam(
+        learning_rate=5e-5,
+        regularization=paddle.optimizer.L2Regularization(rate=1e-3))
     trainer = paddle.trainer.SGD(cost=cost,
                                  parameters=parameters,
                                  update_equation=optimizer)
     ```
 
-2. 构造event_handler
+1. 构造event_handler
 
     可以通过自定义回调函数来评估训练过程中的各种状态，比如错误率等。下面的代码通过event.batch_id % 10 == 0 指定没10个batch打印一次日志，包含cost等信息。
+
     ```python
     def event_handler(event):
         if isinstance(event, paddle.event.EndIteration):
             if event.batch_id % 10 == 0:
-                print "Pass %d, Batch %d, Cost %f, %s" % (
+                print "\nPass %d, Batch %d, Cost %f, %s" % (
                     event.pass_id, event.batch_id, event.cost, event.metrics)
+            else:
+                sys.stdout.write('.')
+                sys.stdout.flush()
     ```
 
-3. 启动训练：
+1. 启动训练：
 
     ```python
     trainer.train(
@@ -473,30 +434,29 @@ for param in parameters.keys():
         num_passes=10000,
         feeding=feeding)
     ```
-    训练开始后，可以观察到event_handler输出的日志如下：
-    ```text
-    Pass 0, Batch 0, Cost 247.408008, {'classification_error_evaluator': 1.0}
-    Pass 0, Batch 10, Cost 212.058789, {'classification_error_evaluator': 0.8737863898277283}
-    ...
-    ```
 
+训练开始后，可以观察到event_handler输出的日志如下：
+
+```text
+Pass 0, Batch 0, Cost 148.444983, {'classification_error_evaluator': 1.0}
+.........
+Pass 0, Batch 10, Cost 335.896802, {'classification_error_evaluator': 0.9325153231620789}
+.........
+```
+
+    当`classification_error_evaluator`的值低于0.35的时候，表示训练成功。
 
 ## 应用模型
 
 ### 下载预训练的模型
 
 由于NMT模型的训练非常耗时，我们在50个物理节点（每节点含有2颗6核CPU）的集群中，花了5天时间训练了16个pass，其中每个pass耗时7个小时。因此，我们提供了一个预先训练好的模型（pass-00012）供大家直接下载使用。该模型大小为205MB，在所有16个模型中有最高的[BLEU评估](#BLEU评估)值26.92。下载并解压模型的命令如下：
+
 ```bash
 cd pretrained
 ./wmt14_model.sh
 ```
 
-### 应用命令与结果
-
-新版api尚未支持机器翻译的翻译过程，尽请期待。
-
-翻译结果请见[效果展示](#效果展示)。
-
 ### BLEU评估
 
 BLEU(Bilingual Evaluation understudy)是一种广泛使用的机器翻译自动评测指标，由IBM的watson研究中心于2002年提出\[[5](#参考文献)\]，基本出发点是：机器译文越接近专业翻译人员的翻译结果，翻译系统的性能越好。其中，机器译文与人工参考译文之间的接近程度，采用句子精确度（precision）的计算方法，即比较两者的n元词组相匹配的个数，匹配的个数越多，BLEU得分越好。

machine_translation/seqToseq_net.py

@@ -110,8 +110,7 @@ group_inputs = [group_input1, group_input2]
 
 if not is_generating:
     trg_embedding = embedding_layer(
-        input=data_layer(
-            name='target_language_word', size=target_dict_dim),
+        input=data_layer(name='target_language_word', size=target_dict_dim),
         size=word_vector_dim,
         param_attr=ParamAttr(name='_target_language_embedding'))
     group_inputs.append(trg_embedding)
@@ -156,8 +155,7 @@ else:
 
     seqtext_printer_evaluator(
         input=beam_gen,
-        id_input=data_layer(
-            name="sent_id", size=1),
+        id_input=data_layer(name="sent_id", size=1),
         dict_file=trg_lang_dict,
         result_file=gen_trans_file)
     outputs(beam_gen)

pre-commit-hooks/convert_markdown_into_ipynb.sh

+#!/bin/sh
+for file in $@ ; do
+	/tmp/go/bin/markdown-to-ipynb < $file > ${file%.*}".ipynb"
+    if [ $? -ne 0 ]; then
+        echo >&2 "markdown-to-ipynb $file error"
+        exit 1
+    fi
+done
+

recognize_digits/README.en.md

 # Recognize Digits
 
-The source code for this tutorial is under [book/recognize_digits](https://github.com/PaddlePaddle/book/tree/develop/recognize_digits). First-time readers, please refer to PaddlePaddle [installation instructions](http://www.paddlepaddle.org/doc_cn/build_and_install/index.html).
+The source code for this tutorial is under [book/recognize_digits](https://github.com/PaddlePaddle/book/tree/develop/recognize_digits). First-time readers, please refer to PaddlePaddle [installation instructions](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/getstarted/build_and_install/docker_install_en.rst).
 
 ## Introduction
 When we learn a new programming language, the first task is usually to write a program that prints "Hello World." In Machine Learning or Deep Learning, the equivalent task is to train a model to perform handwritten digit recognition with [MNIST](http://yann.lecun.com/exdb/mnist/) dataset. Handwriting recognition is a typical image classification problem. The problem is relatively easy, and MNIST is a complete dataset. As a simple Computer Vision dataset, MNIST contains images of handwritten digits and their corresponding labels (Fig. 1). The input image is a 28x28 matrix, and the label is one of the digits from 0 to 9. Each image is normalized in size and centered.
@@ -240,7 +240,7 @@ def event_handler(event):
             print "Pass %d, Batch %d, Cost %f, %s" % (
                 event.pass_id, event.batch_id, event.cost, event.metrics)
     if isinstance(event, paddle.event.EndPass):
-        result = trainer.test(reader=paddle.reader.batched(
+        result = trainer.test(reader=paddle.batch(
             paddle.dataset.mnist.test(), batch_size=128))
         print "Test with Pass %d, Cost %f, %s\n" % (
             event.pass_id, result.cost, result.metrics)
@@ -248,7 +248,7 @@ def event_handler(event):
                       result.metrics['classification_error_evaluator']))
 
 trainer.train(
-    reader=paddle.reader.batched(
+    reader=paddle.batch(
         paddle.reader.shuffle(
             paddle.dataset.mnist.train(), buf_size=8192),
         batch_size=128),
@@ -293,7 +293,7 @@ This tutorial describes a few basic Deep Learning models viz. Softmax regression
 7. Deng, Li, Michael L. Seltzer, Dong Yu, Alex Acero, Abdel-rahman Mohamed, and Geoffrey E. Hinton. ["Binary coding of speech spectrograms using a deep auto-encoder."](http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.185.1908&rep=rep1&type=pdf) In Interspeech, pp. 1692-1695. 2010.
 8. Kégl, Balázs, and Róbert Busa-Fekete. ["Boosting products of base classifiers."](http://dl.acm.org/citation.cfm?id=1553439) In Proceedings of the 26th Annual International Conference on Machine Learning, pp. 497-504. ACM, 2009.
 9. Rosenblatt, Frank. ["The perceptron: A probabilistic model for information storage and organization in the brain."](http://psycnet.apa.org/journals/rev/65/6/386/) Psychological review 65, no. 6 (1958): 386.
-10. Bishop, Christopher M. ["Pattern recognition."](http://s3.amazonaws.com/academia.edu.documents/30428242/bg0137.pdf?AWSAccessKeyId=AKIAJ56TQJRTWSMTNPEA&Expires=1484816640&Signature=85Ad6%2Fca8T82pmHzxaSXermovIA%3D&response-content-disposition=inline%3B%20filename%3DPattern_recognition_and_machine_learning.pdf) Machine Learning 128 (2006): 1-58.
+10. Bishop, Christopher M. ["Pattern recognition."](http://users.isr.ist.utl.pt/~wurmd/Livros/school/Bishop%20-%20Pattern%20Recognition%20And%20Machine%20Learning%20-%20Springer%20%202006.pdf) Machine Learning 128 (2006): 1-58.
 
 <br/>
-<a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/4.0/"><img alt="知识共享许可协议" style="border-width:0" src="https://i.creativecommons.org/l/by-nc-sa/4.0/88x31.png" /></a><br /><span xmlns:dct="http://purl.org/dc/terms/" href="http://purl.org/dc/dcmitype/Text" property="dct:title" rel="dct:type">This book</span> is created by <a xmlns:cc="http://creativecommons.org/ns#" href="http://book.paddlepaddle.org" property="cc:attributionName" rel="cc:attributionURL">PaddlePaddle</a>, and uses <a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/4.0/">Shared knowledge signature - non commercial use-Sharing 4.0 International Licensing Protocal</a>.
+This tutorial is contributed by <a xmlns:cc="http://creativecommons.org/ns#" href="http://book.paddlepaddle.org" property="cc:attributionName" rel="cc:attributionURL">PaddlePaddle</a>, and licensed under a <a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/4.0/">Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License</a>.

recognize_digits/README.md

 # 识别数字
 
-本教程源代码目录在[book/recognize_digits](https://github.com/PaddlePaddle/book/tree/develop/recognize_digits)， 初次使用请参考PaddlePaddle[安装教程](http://www.paddlepaddle.org/doc_cn/build_and_install/index.html)。
+本教程源代码目录在[book/recognize_digits](https://github.com/PaddlePaddle/book/tree/develop/recognize_digits)， 初次使用请参考PaddlePaddle[安装教程](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/getstarted/build_and_install/docker_install_cn.rst)。
 
 ## 背景介绍
 当我们学习编程的时候，编写的第一个程序一般是实现打印"Hello World"。而机器学习（或深度学习）的入门教程，一般都是 [MNIST](http://yann.lecun.com/exdb/mnist/) 数据库上的手写识别问题。原因是手写识别属于典型的图像分类问题，比较简单，同时MNIST数据集也很完备。MNIST数据集作为一个简单的计算机视觉数据集，包含一系列如图1所示的手写数字图片和对应的标签。图片是28x28的像素矩阵，标签则对应着0~9的10个数字。每张图片都经过了大小归一化和居中处理。
@@ -245,7 +245,7 @@ def event_handler(event):
             print "Pass %d, Batch %d, Cost %f, %s" % (
                 event.pass_id, event.batch_id, event.cost, event.metrics)
     if isinstance(event, paddle.event.EndPass):
-        result = trainer.test(reader=paddle.reader.batched(
+        result = trainer.test(reader=paddle.batch(
             paddle.dataset.mnist.test(), batch_size=128))
         print "Test with Pass %d, Cost %f, %s\n" % (
             event.pass_id, result.cost, result.metrics)
@@ -253,7 +253,7 @@ def event_handler(event):
                       result.metrics['classification_error_evaluator']))
 
 trainer.train(
-    reader=paddle.reader.batched(
+    reader=paddle.batch(
         paddle.reader.shuffle(
             paddle.dataset.mnist.train(), buf_size=8192),
         batch_size=128),
@@ -289,7 +289,7 @@ trainer.train(
 7. Deng, Li, Michael L. Seltzer, Dong Yu, Alex Acero, Abdel-rahman Mohamed, and Geoffrey E. Hinton. ["Binary coding of speech spectrograms using a deep auto-encoder."](http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.185.1908&rep=rep1&type=pdf) In Interspeech, pp. 1692-1695. 2010.
 8. Kégl, Balázs, and Róbert Busa-Fekete. ["Boosting products of base classifiers."](http://dl.acm.org/citation.cfm?id=1553439) In Proceedings of the 26th Annual International Conference on Machine Learning, pp. 497-504. ACM, 2009.
 9. Rosenblatt, Frank. ["The perceptron: A probabilistic model for information storage and organization in the brain."](http://psycnet.apa.org/journals/rev/65/6/386/) Psychological review 65, no. 6 (1958): 386.
-10. Bishop, Christopher M. ["Pattern recognition."](http://s3.amazonaws.com/academia.edu.documents/30428242/bg0137.pdf?AWSAccessKeyId=AKIAJ56TQJRTWSMTNPEA&Expires=1484816640&Signature=85Ad6%2Fca8T82pmHzxaSXermovIA%3D&response-content-disposition=inline%3B%20filename%3DPattern_recognition_and_machine_learning.pdf) Machine Learning 128 (2006): 1-58.
+10. Bishop, Christopher M. ["Pattern recognition."](http://users.isr.ist.utl.pt/~wurmd/Livros/school/Bishop%20-%20Pattern%20Recognition%20And%20Machine%20Learning%20-%20Springer%20%202006.pdf) Machine Learning 128 (2006): 1-58.
 
 <br/>
 <a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/4.0/"><img alt="知识共享许可协议" style="border-width:0" src="https://i.creativecommons.org/l/by-nc-sa/4.0/88x31.png" /></a><br /><span xmlns:dct="http://purl.org/dc/terms/" href="http://purl.org/dc/dcmitype/Text" property="dct:title" rel="dct:type">本教程</span> 由 <a xmlns:cc="http://creativecommons.org/ns#" href="http://book.paddlepaddle.org" property="cc:attributionName" rel="cc:attributionURL">PaddlePaddle</a> 创作，采用 <a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/4.0/">知识共享 署名-非商业性使用-相同方式共享 4.0 国际 许可协议</a>进行许可。

recognize_digits/index.en.html

@@ -42,7 +42,7 @@
 <div id="markdown" style='display:none'>
 # Recognize Digits
 
-The source code for this tutorial is under [book/recognize_digits](https://github.com/PaddlePaddle/book/tree/develop/recognize_digits). First-time readers, please refer to PaddlePaddle [installation instructions](http://www.paddlepaddle.org/doc_cn/build_and_install/index.html).
+The source code for this tutorial is under [book/recognize_digits](https://github.com/PaddlePaddle/book/tree/develop/recognize_digits). First-time readers, please refer to PaddlePaddle [installation instructions](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/getstarted/build_and_install/docker_install_en.rst).
 
 ## Introduction
 When we learn a new programming language, the first task is usually to write a program that prints "Hello World." In Machine Learning or Deep Learning, the equivalent task is to train a model to perform handwritten digit recognition with [MNIST](http://yann.lecun.com/exdb/mnist/) dataset. Handwriting recognition is a typical image classification problem. The problem is relatively easy, and MNIST is a complete dataset. As a simple Computer Vision dataset, MNIST contains images of handwritten digits and their corresponding labels (Fig. 1). The input image is a 28x28 matrix, and the label is one of the digits from 0 to 9. Each image is normalized in size and centered.
@@ -282,7 +282,7 @@ def event_handler(event):
             print "Pass %d, Batch %d, Cost %f, %s" % (
                 event.pass_id, event.batch_id, event.cost, event.metrics)
     if isinstance(event, paddle.event.EndPass):
-        result = trainer.test(reader=paddle.reader.batched(
+        result = trainer.test(reader=paddle.batch(
             paddle.dataset.mnist.test(), batch_size=128))
         print "Test with Pass %d, Cost %f, %s\n" % (
             event.pass_id, result.cost, result.metrics)
@@ -290,7 +290,7 @@ def event_handler(event):
                       result.metrics['classification_error_evaluator']))
 
 trainer.train(
-    reader=paddle.reader.batched(
+    reader=paddle.batch(
         paddle.reader.shuffle(
             paddle.dataset.mnist.train(), buf_size=8192),
         batch_size=128),
@@ -335,10 +335,10 @@ This tutorial describes a few basic Deep Learning models viz. Softmax regression
 7. Deng, Li, Michael L. Seltzer, Dong Yu, Alex Acero, Abdel-rahman Mohamed, and Geoffrey E. Hinton. ["Binary coding of speech spectrograms using a deep auto-encoder."](http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.185.1908&rep=rep1&type=pdf) In Interspeech, pp. 1692-1695. 2010.
 8. Kégl, Balázs, and Róbert Busa-Fekete. ["Boosting products of base classifiers."](http://dl.acm.org/citation.cfm?id=1553439) In Proceedings of the 26th Annual International Conference on Machine Learning, pp. 497-504. ACM, 2009.
 9. Rosenblatt, Frank. ["The perceptron: A probabilistic model for information storage and organization in the brain."](http://psycnet.apa.org/journals/rev/65/6/386/) Psychological review 65, no. 6 (1958): 386.
-10. Bishop, Christopher M. ["Pattern recognition."](http://s3.amazonaws.com/academia.edu.documents/30428242/bg0137.pdf?AWSAccessKeyId=AKIAJ56TQJRTWSMTNPEA&Expires=1484816640&Signature=85Ad6%2Fca8T82pmHzxaSXermovIA%3D&response-content-disposition=inline%3B%20filename%3DPattern_recognition_and_machine_learning.pdf) Machine Learning 128 (2006): 1-58.
+10. Bishop, Christopher M. ["Pattern recognition."](http://users.isr.ist.utl.pt/~wurmd/Livros/school/Bishop%20-%20Pattern%20Recognition%20And%20Machine%20Learning%20-%20Springer%20%202006.pdf) Machine Learning 128 (2006): 1-58.
 
 <br/>
-<a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/4.0/"><img alt="知识共享许可协议" style="border-width:0" src="https://i.creativecommons.org/l/by-nc-sa/4.0/88x31.png" /></a><br /><span xmlns:dct="http://purl.org/dc/terms/" href="http://purl.org/dc/dcmitype/Text" property="dct:title" rel="dct:type">This book</span> is created by <a xmlns:cc="http://creativecommons.org/ns#" href="http://book.paddlepaddle.org" property="cc:attributionName" rel="cc:attributionURL">PaddlePaddle</a>, and uses <a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/4.0/">Shared knowledge signature - non commercial use-Sharing 4.0 International Licensing Protocal</a>.
+This tutorial is contributed by <a xmlns:cc="http://creativecommons.org/ns#" href="http://book.paddlepaddle.org" property="cc:attributionName" rel="cc:attributionURL">PaddlePaddle</a>, and licensed under a <a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/4.0/">Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License</a>.
 
 </div>
 <!-- You can change the lines below now. -->

recognize_digits/index.html

@@ -42,7 +42,7 @@
 <div id="markdown" style='display:none'>
 # 识别数字
 
-本教程源代码目录在[book/recognize_digits](https://github.com/PaddlePaddle/book/tree/develop/recognize_digits)， 初次使用请参考PaddlePaddle[安装教程](http://www.paddlepaddle.org/doc_cn/build_and_install/index.html)。
+本教程源代码目录在[book/recognize_digits](https://github.com/PaddlePaddle/book/tree/develop/recognize_digits)， 初次使用请参考PaddlePaddle[安装教程](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/getstarted/build_and_install/docker_install_cn.rst)。
 
 ## 背景介绍
 当我们学习编程的时候，编写的第一个程序一般是实现打印"Hello World"。而机器学习（或深度学习）的入门教程，一般都是 [MNIST](http://yann.lecun.com/exdb/mnist/) 数据库上的手写识别问题。原因是手写识别属于典型的图像分类问题，比较简单，同时MNIST数据集也很完备。MNIST数据集作为一个简单的计算机视觉数据集，包含一系列如图1所示的手写数字图片和对应的标签。图片是28x28的像素矩阵，标签则对应着0~9的10个数字。每张图片都经过了大小归一化和居中处理。
@@ -287,7 +287,7 @@ def event_handler(event):
             print "Pass %d, Batch %d, Cost %f, %s" % (
                 event.pass_id, event.batch_id, event.cost, event.metrics)
     if isinstance(event, paddle.event.EndPass):
-        result = trainer.test(reader=paddle.reader.batched(
+        result = trainer.test(reader=paddle.batch(
             paddle.dataset.mnist.test(), batch_size=128))
         print "Test with Pass %d, Cost %f, %s\n" % (
             event.pass_id, result.cost, result.metrics)
@@ -295,7 +295,7 @@ def event_handler(event):
                       result.metrics['classification_error_evaluator']))
 
 trainer.train(
-    reader=paddle.reader.batched(
+    reader=paddle.batch(
         paddle.reader.shuffle(
             paddle.dataset.mnist.train(), buf_size=8192),
         batch_size=128),
@@ -331,7 +331,7 @@ trainer.train(
 7. Deng, Li, Michael L. Seltzer, Dong Yu, Alex Acero, Abdel-rahman Mohamed, and Geoffrey E. Hinton. ["Binary coding of speech spectrograms using a deep auto-encoder."](http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.185.1908&rep=rep1&type=pdf) In Interspeech, pp. 1692-1695. 2010.
 8. Kégl, Balázs, and Róbert Busa-Fekete. ["Boosting products of base classifiers."](http://dl.acm.org/citation.cfm?id=1553439) In Proceedings of the 26th Annual International Conference on Machine Learning, pp. 497-504. ACM, 2009.
 9. Rosenblatt, Frank. ["The perceptron: A probabilistic model for information storage and organization in the brain."](http://psycnet.apa.org/journals/rev/65/6/386/) Psychological review 65, no. 6 (1958): 386.
-10. Bishop, Christopher M. ["Pattern recognition."](http://s3.amazonaws.com/academia.edu.documents/30428242/bg0137.pdf?AWSAccessKeyId=AKIAJ56TQJRTWSMTNPEA&Expires=1484816640&Signature=85Ad6%2Fca8T82pmHzxaSXermovIA%3D&response-content-disposition=inline%3B%20filename%3DPattern_recognition_and_machine_learning.pdf) Machine Learning 128 (2006): 1-58.
+10. Bishop, Christopher M. ["Pattern recognition."](http://users.isr.ist.utl.pt/~wurmd/Livros/school/Bishop%20-%20Pattern%20Recognition%20And%20Machine%20Learning%20-%20Springer%20%202006.pdf) Machine Learning 128 (2006): 1-58.
 
 <br/>
 <a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/4.0/"><img alt="知识共享许可协议" style="border-width:0" src="https://i.creativecommons.org/l/by-nc-sa/4.0/88x31.png" /></a><br /><span xmlns:dct="http://purl.org/dc/terms/" href="http://purl.org/dc/dcmitype/Text" property="dct:title" rel="dct:type">本教程</span> 由 <a xmlns:cc="http://creativecommons.org/ns#" href="http://book.paddlepaddle.org" property="cc:attributionName" rel="cc:attributionURL">PaddlePaddle</a> 创作，采用 <a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/4.0/">知识共享 署名-非商业性使用-相同方式共享 4.0 国际 许可协议</a>进行许可。

recognize_digits/train.py

@@ -2,9 +2,8 @@ import paddle.v2 as paddle
 
 
 def softmax_regression(img):
-    predict = paddle.layer.fc(input=img,
-                              size=10,
-                              act=paddle.activation.Softmax())
+    predict = paddle.layer.fc(
+        input=img, size=10, act=paddle.activation.Softmax())
     return predict
 
 
@@ -12,14 +11,12 @@ def multilayer_perceptron(img):
     # The first fully-connected layer
     hidden1 = paddle.layer.fc(input=img, size=128, act=paddle.activation.Relu())
     # The second fully-connected layer and the according activation function
-    hidden2 = paddle.layer.fc(input=hidden1,
-                              size=64,
-                              act=paddle.activation.Relu())
+    hidden2 = paddle.layer.fc(
+        input=hidden1, size=64, act=paddle.activation.Relu())
     # The thrid fully-connected layer, note that the hidden size should be 10,
     # which is the number of unique digits
-    predict = paddle.layer.fc(input=hidden2,
-                              size=10,
-                              act=paddle.activation.Softmax())
+    predict = paddle.layer.fc(
+        input=hidden2, size=10, act=paddle.activation.Softmax())
     return predict
 
 
@@ -43,14 +40,12 @@ def convolutional_neural_network(img):
         pool_stride=2,
         act=paddle.activation.Tanh())
     # The first fully-connected layer
-    fc1 = paddle.layer.fc(input=conv_pool_2,
-                          size=128,
-                          act=paddle.activation.Tanh())
+    fc1 = paddle.layer.fc(
+        input=conv_pool_2, size=128, act=paddle.activation.Tanh())
     # The softmax layer, note that the hidden size should be 10,
     # which is the number of unique digits
-    predict = paddle.layer.fc(input=fc1,
-                              size=10,
-                              act=paddle.activation.Softmax())
+    predict = paddle.layer.fc(
+        input=fc1, size=10, act=paddle.activation.Softmax())
     return predict
 
 
@@ -76,9 +71,8 @@ optimizer = paddle.optimizer.Momentum(
     momentum=0.9,
     regularization=paddle.optimizer.L2Regularization(rate=0.0005 * 128))
 
-trainer = paddle.trainer.SGD(cost=cost,
-                             parameters=parameters,
-                             update_equation=optimizer)
+trainer = paddle.trainer.SGD(
+    cost=cost, parameters=parameters, update_equation=optimizer)
 
 lists = []
 
@@ -89,7 +83,7 @@ def event_handler(event):
             print "Pass %d, Batch %d, Cost %f, %s" % (
                 event.pass_id, event.batch_id, event.cost, event.metrics)
     if isinstance(event, paddle.event.EndPass):
-        result = trainer.test(reader=paddle.reader.batched(
+        result = trainer.test(reader=paddle.batch(
             paddle.dataset.mnist.test(), batch_size=128))
         print "Test with Pass %d, Cost %f, %s\n" % (event.pass_id, result.cost,
                                                     result.metrics)
@@ -98,9 +92,8 @@ def event_handler(event):
 
 
 trainer.train(
-    reader=paddle.reader.batched(
-        paddle.reader.shuffle(
-            paddle.dataset.mnist.train(), buf_size=8192),
+    reader=paddle.batch(
+        paddle.reader.shuffle(paddle.dataset.mnist.train(), buf_size=8192),
         batch_size=128),
     event_handler=event_handler,
     num_passes=100)

recommender_system/README.en.ipynb

+{
+  "cells": [
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "# Personalized Recommendation\n",
+        "\n",
+        "The source code of this tutorial is in [book/recommender_system](https://github.com/PaddlePaddle/book/tree/develop/recommender_system).\n",
+        "\n",
+        "For instructions on getting started with PaddlePaddle, see [PaddlePaddle installation guide](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/getstarted/build_and_install/docker_install_en.rst).\n",
+        "\n",
+        "\n",
+        "## Background\n",
+        "\n",
+        "With the fast growth of e-commerce, online videos, and online reading business, users have to rely on recommender systems to avoid manually browsing tremendous volume of choices.  Recommender systems understand users' interest by mining user behavior and other properties of users and products.\n",
+        "\n",
+        "Some well know approaches include:\n",
+        "\n",
+        "- User behavior-based approach.  A well-known method is collaborative filtering. The underlying assumption is that if a person A has the same opinion as a person B on an issue, A is more likely to have B's opinion on a different issue than that of a randomly chosen person.\n",
+        "\n",
+        "- Content-based recommendation[[1](#reference)]. This approach infers feature vectors that represent products from their descriptions.  It also infers feature vectors that represent users' interests.  Then it measures the relevance of users and products by some distances between these feature vectors.\n",
+        "\n",
+        "- Hybrid approach[[2](#reference)]: This approach uses the content-based information to help address the cold start problem[[6](#reference)] in behavior-based approach.\n",
+        "\n",
+        "Among these options, collaborative filtering might be the most studied one.  Some of its variants include user-based[[3](#reference)], item-based [[4](#reference)], social network based[[5](#reference)], and model-based.\n",
+        "\n",
+        "This tutorial explains a deep learning based approach and how to implement it using PaddlePaddle.  We will train a model using a dataset that includes user information, movie information, and ratings.  Once we train the model, we will be able to get a predicted rating given a pair of user and movie IDs.\n",
+        "\n",
+        "\n",
+        "## Model Overview\n",
+        "\n",
+        "To know more about deep learning based recommendation, let us start from going over the Youtube recommender system[[7](#参考文献)] before introducing our hybrid model.\n",
+        "\n",
+        "\n",
+        "### YouTube's Deep Learning Recommendation Model\n",
+        "\n",
+        "YouTube is a video-sharing Web site with one of the largest user base in the world.  Its recommender system serves more than a billion users.  This system is composed of two major parts: candidate generation and ranking.  The former selects few hundreds of candidates from millions of videos, and the latter ranks and outputs the top 10.\n",
+        "\n",
+        "\u003cp align=\"center\"\u003e\n",
+        "\u003cimg src=\"image/YouTube_Overview.en.png\" width=\"70%\" \u003e\u003cbr/\u003e\n",
+        "Figure 1. YouTube recommender system overview.\n",
+        "\u003c/p\u003e\n",
+        "\n",
+        "#### Candidate Generation Network\n",
+        "\n",
+        "Youtube models candidate generation as a multiclass classification problem with a huge number of classes equal to the number of videos.  The architecture of the model is as follows:\n",
+        "\n",
+        "\u003cp align=\"center\"\u003e\n",
+        "\u003cimg src=\"image/Deep_candidate_generation_model_architecture.en.png\" width=\"70%\" \u003e\u003cbr/\u003e\n",
+        "Figure. Deep candidate geeration model.\n",
+        "\u003c/p\u003e\n",
+        "\n",
+        "The first stage of this model maps watching history and search queries into fixed-length representative features.  Then, an MLP (multi-layer perceptron, as described in the [Recognize Digits](https://github.com/PaddlePaddle/book/blob/develop/recognize_digits/README.md) tutorial) takes the concatenation of all representative vectors.  The output of the MLP represents the user' *intrinsic interests*.  At training time, it is used together with a softmax output layer for minimizing the classification error.   At serving time, it is used to compute the relevance of the user with all movies.\n",
+        "\n",
+        "For a user $U$, the predicted watching probability of video $i$ is\n",
+        "\n",
+        "$$P(\\omega=i|u)=\\frac{e^{v_{i}u}}{\\sum_{j \\in V}e^{v_{j}u}}$$\n",
+        "\n",
+        "where $u$ is the representative vector of user $U$, $V$ is the corpus of all videos, $v_i$ is the representative vector of the $i$-th video. $u$ and $v_i$ are vectors of the same length, so we can compute their dot product using a fully connected layer.\n",
+        "\n",
+        "This model could have a performance issue as the softmax output covers millions of classification labels.  To optimize performance, at the training time, the authors down-sample negative samples, so the actual number of classes is reduced to thousands.  At serving time, the authors ignore the normalization of the softmax outputs, because the results are just for ranking.\n",
+        "\n",
+        "\n",
+        "#### Ranking Network\n",
+        "\n",
+        "The architecture of the ranking network is similar to that of the candidate generation network.  Similar to ranking models widely used in online advertising, it uses rich features like video ID, last watching time, etc.  The output layer of the ranking network is a weighted logistic regression, which rates all candidate videos.\n",
+        "\n",
+        "\n",
+        "### Hybrid Model\n",
+        "\n",
+        "In the section, let us introduce our movie recommendation system.\n",
+        "\n",
+        "In our network, the input includes features of users and movies.  The user feature includes four properties: user ID, gender, occupation, and age.  Movie features include their IDs, genres, and titles.\n",
+        "\n",
+        "We use fully-connected layers to map user features into representative feature vectors and concatenate them.  The process of movie features is similar, except that for movie titles -- we feed titles into a text convolution network as described in the [sentiment analysis tutorial](https://github.com/PaddlePaddle/book/blob/develop/understand_sentiment/README.md)）to get a fixed-length representative feature vector.\n",
+        "\n",
+        "Given the feature vectors of users and movies, we compute the relevance using cosine similarity.  We minimize the squared error at training time.\n",
+        "\n",
+        "\u003cp align=\"center\"\u003e\n",
+        "\n",
+        "\u003cimg src=\"image/rec_regression_network_en.png\" width=\"90%\" \u003e\u003cbr/\u003e\n",
+        "Figure 3. A hybrid recommendation model.\n",
+        "\u003c/p\u003e\n",
+        "\n",
+        "## Dataset\n",
+        "\n",
+        "We use the [MovieLens ml-1m](http://files.grouplens.org/datasets/movielens/ml-1m.zip) to train our model.  This dataset includes 10,000 ratings of 4,000 movies from 6,000 users to 4,000 movies.  Each rate is in the range of 1~5.  Thanks to GroupLens Research for collecting, processing and publishing the dataset.\n",
+        "\n",
+        "`paddle.v2.datasets` package encapsulates multiple public datasets, including `cifar`, `imdb`, `mnist`, `moivelens` and `wmt14`, etc. There's no need for us to manually download and preprocess `MovieLens` dataset.\n",
+        "\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "metadata": {
+        "editable": true
+      },
+      "source": [
+        "# Run this block to show dataset's documentation\n",
+        "help(paddle.v2.dataset.movielens)\n"
+      ],
+      "outputs": [
+        {
+          "name": "stdout",
+          "output_type": "stream",
+          "text": [
+            "\n"
+          ]
+        }
+      ],
+      "execution_count": 1
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "\n",
+        "The raw `MoiveLens` contains movie ratings, relevant features from both movies and users.\n",
+        "For instance, one movie's feature could be:\n",
+        "\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "metadata": {
+        "editable": true
+      },
+      "source": [
+        "movie_info = paddle.dataset.movielens.movie_info()\n",
+        "print movie_info.values()[0]\n"
+      ],
+      "outputs": [
+        {
+          "name": "stdout",
+          "output_type": "stream",
+          "text": [
+            "\n"
+          ]
+        }
+      ],
+      "execution_count": 1
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "\n",
+        "```text\n",
+        "\u003cMovieInfo id(1), title(Toy Story), categories(['Animation', \"Children's\", 'Comedy'])\u003e\n",
+        "```\n",
+        "\n",
+        "One user's feature could be:\n",
+        "\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "metadata": {
+        "editable": true
+      },
+      "source": [
+        "user_info = paddle.dataset.movielens.user_info()\n",
+        "print user_info.values()[0]\n"
+      ],
+      "outputs": [
+        {
+          "name": "stdout",
+          "output_type": "stream",
+          "text": [
+            "\n"
+          ]
+        }
+      ],
+      "execution_count": 1
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "\n",
+        "```text\n",
+        "\u003cUserInfo id(1), gender(F), age(1), job(10)\u003e\n",
+        "```\n",
+        "\n",
+        "In this dateset, the distribution of age is shown as follows:\n",
+        "\n",
+        "```text\n",
+        "1: \"Under 18\"\n",
+        "18: \"18-24\"\n",
+        "25: \"25-34\"\n",
+        "35: \"35-44\"\n",
+        "45: \"45-49\"\n",
+        "50: \"50-55\"\n",
+        "56: \"56+\"\n",
+        "```\n",
+        "\n",
+        "User's occupation is selected from the following options:\n",
+        "\n",
+        "```text\n",
+        "0: \"other\" or not specified\n",
+        "1: \"academic/educator\"\n",
+        "2: \"artist\"\n",
+        "3: \"clerical/admin\"\n",
+        "4: \"college/grad student\"\n",
+        "5: \"customer service\"\n",
+        "6: \"doctor/health care\"\n",
+        "7: \"executive/managerial\"\n",
+        "8: \"farmer\"\n",
+        "9: \"homemaker\"\n",
+        "10: \"K-12 student\"\n",
+        "11: \"lawyer\"\n",
+        "12: \"programmer\"\n",
+        "13: \"retired\"\n",
+        "14: \"sales/marketing\"\n",
+        "15: \"scientist\"\n",
+        "16: \"self-employed\"\n",
+        "17: \"technician/engineer\"\n",
+        "18: \"tradesman/craftsman\"\n",
+        "19: \"unemployed\"\n",
+        "20: \"writer\"\n",
+        "```\n",
+        "\n",
+        "Each record consists of three main components: user features, movie features and movie ratings.\n",
+        "Likewise, as a simple example, consider the following:\n",
+        "\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "metadata": {
+        "editable": true
+      },
+      "source": [
+        "train_set_creator = paddle.dataset.movielens.train()\n",
+        "train_sample = next(train_set_creator())\n",
+        "uid = train_sample[0]\n",
+        "mov_id = train_sample[len(user_info[uid].value())]\n",
+        "print \"User %s rates Movie %s with Score %s\"%(user_info[uid], movie_info[mov_id], train_sample[-1])\n"
+      ],
+      "outputs": [
+        {
+          "name": "stdout",
+          "output_type": "stream",
+          "text": [
+            "\n"
+          ]
+        }
+      ],
+      "execution_count": 1
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "\n",
+        "```text\n",
+        "User \u003cUserInfo id(1), gender(F), age(1), job(10)\u003e rates Movie \u003cMovieInfo id(1193), title(One Flew Over the Cuckoo's Nest), categories(['Drama'])\u003e with Score [5.0]\n",
+        "```\n",
+        "\n",
+        "The output shows that user 1 gave movie `1193` a rating of 5.\n",
+        "\n",
+        "After issuing a command `python train.py`, training will start immediately. The details will be unpacked by the following sessions to see how it works.\n",
+        "\n",
+        "## Model Architecture\n",
+        "\n",
+        "### Initialize PaddlePaddle\n",
+        "\n",
+        "First, we must import and initialize PaddlePaddle (enable/disable GPU, set the number of trainers, etc).\n",
+        "\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "metadata": {
+        "editable": true
+      },
+      "source": [
+        "%matplotlib inline\n",
+        "\n",
+        "import matplotlib.pyplot as plt\n",
+        "from IPython import display\n",
+        "import cPickle\n",
+        "\n",
+        "import paddle.v2 as paddle\n",
+        "\n",
+        "paddle.init(use_gpu=False)\n"
+      ],
+      "outputs": [
+        {
+          "name": "stdout",
+          "output_type": "stream",
+          "text": [
+            "\n"
+          ]
+        }
+      ],
+      "execution_count": 1
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "\n",
+        "### Model Configuration\n",
+        "\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "metadata": {
+        "editable": true
+      },
+      "source": [
+        "uid = paddle.layer.data(\n",
+        "        name='user_id',\n",
+        "        type=paddle.data_type.integer_value(\n",
+        "            paddle.dataset.movielens.max_user_id() + 1))\n",
+        "usr_emb = paddle.layer.embedding(input=uid, size=32)\n",
+        "\n",
+        "usr_gender_id = paddle.layer.data(\n",
+        "        name='gender_id', type=paddle.data_type.integer_value(2))\n",
+        "usr_gender_emb = paddle.layer.embedding(input=usr_gender_id, size=16)\n",
+        "\n",
+        "usr_age_id = paddle.layer.data(\n",
+        "        name='age_id',\n",
+        "        type=paddle.data_type.integer_value(\n",
+        "            len(paddle.dataset.movielens.age_table)))\n",
+        "usr_age_emb = paddle.layer.embedding(input=usr_age_id, size=16)\n",
+        "\n",
+        "usr_job_id = paddle.layer.data(\n",
+        "        name='job_id',\n",
+        "        type=paddle.data_type.integer_value(paddle.dataset.movielens.max_job_id(\n",
+        "        ) + 1))\n",
+        "usr_job_emb = paddle.layer.embedding(input=usr_job_id, size=16)\n"
+      ],
+      "outputs": [
+        {
+          "name": "stdout",
+          "output_type": "stream",
+          "text": [
+            "\n"
+          ]
+        }
+      ],
+      "execution_count": 1
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "\n",
+        "As shown in the above code, the input is four dimension integers for each user, that is,  `user_id`,`gender_id`, `age_id` and `job_id`. In order to deal with these features conveniently, we use the language model in NLP to transform these discrete values into embedding vaules `usr_emb`, `usr_gender_emb`, `usr_age_emb` and `usr_job_emb`.\n",
+        "\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "metadata": {
+        "editable": true
+      },
+      "source": [
+        "usr_combined_features = paddle.layer.fc(\n",
+        "        input=[usr_emb, usr_gender_emb, usr_age_emb, usr_job_emb],\n",
+        "        size=200,\n",
+        "        act=paddle.activation.Tanh())\n"
+      ],
+      "outputs": [
+        {
+          "name": "stdout",
+          "output_type": "stream",
+          "text": [
+            "\n"
+          ]
+        }
+      ],
+      "execution_count": 1
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "\n",
+        "Then, employing user features as input, directly connecting to a fully-connected layer, which is used to reduce dimension to 200.\n",
+        "\n",
+        "Furthermore, we do a similar transformation for each movie feature. The model configuration is:\n",
+        "\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "metadata": {
+        "editable": true
+      },
+      "source": [
+        "mov_id = paddle.layer.data(\n",
+        "    name='movie_id',\n",
+        "    type=paddle.data_type.integer_value(\n",
+        "        paddle.dataset.movielens.max_movie_id() + 1))\n",
+        "mov_emb = paddle.layer.embedding(input=mov_id, size=32)\n",
+        "\n",
+        "mov_categories = paddle.layer.data(\n",
+        "    name='category_id',\n",
+        "    type=paddle.data_type.sparse_binary_vector(\n",
+        "        len(paddle.dataset.movielens.movie_categories())))\n",
+        "\n",
+        "mov_categories_hidden = paddle.layer.fc(input=mov_categories, size=32)\n",
+        "\n",
+        "\n",
+        "movie_title_dict = paddle.dataset.movielens.get_movie_title_dict()\n",
+        "mov_title_id = paddle.layer.data(\n",
+        "    name='movie_title',\n",
+        "    type=paddle.data_type.integer_value_sequence(len(movie_title_dict)))\n",
+        "mov_title_emb = paddle.layer.embedding(input=mov_title_id, size=32)\n",
+        "mov_title_conv = paddle.networks.sequence_conv_pool(\n",
+        "    input=mov_title_emb, hidden_size=32, context_len=3)\n",
+        "\n",
+        "mov_combined_features = paddle.layer.fc(\n",
+        "    input=[mov_emb, mov_categories_hidden, mov_title_conv],\n",
+        "    size=200,\n",
+        "    act=paddle.activation.Tanh())\n"
+      ],
+      "outputs": [
+        {
+          "name": "stdout",
+          "output_type": "stream",
+          "text": [
+            "\n"
+          ]
+        }
+      ],
+      "execution_count": 1
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "\n",
+        "Movie title, a sequence of words represented by an integer word index sequence, will be feed into a `sequence_conv_pool` layer, which will apply convolution and pooling on time dimension. Because pooling is done on time dimension, the output will be a fixed-length vector regardless the length of the input sequence.\n",
+        "\n",
+        "Finally, we can use cosine similarity to calculate the similarity between user characteristics and movie features.\n",
+        "\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "metadata": {
+        "editable": true
+      },
+      "source": [
+        "inference = paddle.layer.cos_sim(a=usr_combined_features, b=mov_combined_features, size=1, scale=5)\n",
+        "cost = paddle.layer.regression_cost(\n",
+        "        input=inference,\n",
+        "        label=paddle.layer.data(\n",
+        "        name='score', type=paddle.data_type.dense_vector(1)))\n"
+      ],
+      "outputs": [
+        {
+          "name": "stdout",
+          "output_type": "stream",
+          "text": [
+            "\n"
+          ]
+        }
+      ],
+      "execution_count": 1
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "\n",
+        "## Model Training\n",
+        "\n",
+        "### Define Parameters\n",
+        "\n",
+        "First, we define the model parameters according to the previous model configuration `cost`.\n",
+        "\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "metadata": {
+        "editable": true
+      },
+      "source": [
+        "# Create parameters\n",
+        "parameters = paddle.parameters.create(cost)\n"
+      ],
+      "outputs": [
+        {
+          "name": "stdout",
+          "output_type": "stream",
+          "text": [
+            "\n"
+          ]
+        }
+      ],
+      "execution_count": 1
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "\n",
+        "### Create Trainer\n",
+        "\n",
+        "Before jumping into creating a training module, algorithm setting is also necessary. Here we specified Adam optimization algorithm via `paddle.optimizer`.\n",
+        "\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "metadata": {
+        "editable": true
+      },
+      "source": [
+        "trainer = paddle.trainer.SGD(cost=cost, parameters=parameters,\n",
+        "                             update_equation=paddle.optimizer.Adam(learning_rate=1e-4))\n"
+      ],
+      "outputs": [
+        {
+          "name": "stdout",
+          "output_type": "stream",
+          "text": [
+            "\n"
+          ]
+        }
+      ],
+      "execution_count": 1
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "\n",
+        "```text\n",
+        "[INFO 2017-03-06 17:12:13,378 networks.py:1472] The input order is [user_id, gender_id, age_id, job_id, movie_id, category_id, movie_title, score]\n",
+        "[INFO 2017-03-06 17:12:13,379 networks.py:1478] The output order is [__regression_cost_0__]\n",
+        "```\n",
+        "\n",
+        "### Training\n",
+        "\n",
+        "`paddle.dataset.movielens.train` will yield records during each pass, after shuffling, a batch input is generated for training.\n",
+        "\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "metadata": {
+        "editable": true
+      },
+      "source": [
+        "reader=paddle.reader.batch(\n",
+        "    paddle.reader.shuffle(\n",
+        "        paddle.dataset.movielens.trai(), buf_size=8192),\n",
+        "        batch_size=256)\n"
+      ],
+      "outputs": [
+        {
+          "name": "stdout",
+          "output_type": "stream",
+          "text": [
+            "\n"
+          ]
+        }
+      ],
+      "execution_count": 1
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "\n",
+        "`feeding` is devoted to specifying the correspondence between each yield record and `paddle.layer.data`. For instance, the first column of data generated by `movielens.train` corresponds to `user_id` feature.\n",
+        "\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "metadata": {
+        "editable": true
+      },
+      "source": [
+        "feeding = {\n",
+        "    'user_id': 0,\n",
+        "    'gender_id': 1,\n",
+        "    'age_id': 2,\n",
+        "    'job_id': 3,\n",
+        "    'movie_id': 4,\n",
+        "    'category_id': 5,\n",
+        "    'movie_title': 6,\n",
+        "    'score': 7\n",
+        "}\n"
+      ],
+      "outputs": [
+        {
+          "name": "stdout",
+          "output_type": "stream",
+          "text": [
+            "\n"
+          ]
+        }
+      ],
+      "execution_count": 1
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "\n",
+        "Callback function `event_handler` will be called during training when a pre-defined event happens.\n",
+        "\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "metadata": {
+        "editable": true
+      },
+      "source": [
+        "step=0\n",
+        "\n",
+        "train_costs=[],[]\n",
+        "test_costs=[],[]\n",
+        "\n",
+        "def event_handler(event):\n",
+        "    global step\n",
+        "    global train_costs\n",
+        "    global test_costs\n",
+        "    if isinstance(event, paddle.event.EndIteration):\n",
+        "        need_plot = False\n",
+        "        if step % 10 == 0:  # every 10 batches, record a train cost\n",
+        "            train_costs[0].append(step)\n",
+        "            train_costs[1].append(event.cost)\n",
+        "\n",
+        "        if step % 1000 == 0: # every 1000 batches, record a test cost\n",
+        "            result = trainer.test(reader=paddle.batch(\n",
+        "                  paddle.dataset.movielens.test(), batch_size=256))\n",
+        "            test_costs[0].append(step)\n",
+        "            test_costs[1].append(result.cost)\n",
+        "\n",
+        "        if step % 100 == 0: # every 100 batches, update cost plot\n",
+        "            plt.plot(*train_costs)\n",
+        "            plt.plot(*test_costs)\n",
+        "            plt.legend(['Train Cost', 'Test Cost'], loc='upper left')\n",
+        "            display.clear_output(wait=True)\n",
+        "            display.display(plt.gcf())\n",
+        "            plt.gcf().clear()\n",
+        "        step += 1\n"
+      ],
+      "outputs": [
+        {
+          "name": "stdout",
+          "output_type": "stream",
+          "text": [
+            "\n"
+          ]
+        }
+      ],
+      "execution_count": 1
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "\n",
+        "Finally, we can invoke `trainer.train` to start training:\n",
+        "\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "metadata": {
+        "editable": true
+      },
+      "source": [
+        "trainer.train(\n",
+        "    reader=reader,\n",
+        "    event_handler=event_handler,\n",
+        "    feeding=feeding,\n",
+        "    num_passes=200)\n"
+      ],
+      "outputs": [
+        {
+          "name": "stdout",
+          "output_type": "stream",
+          "text": [
+            "\n"
+          ]
+        }
+      ],
+      "execution_count": 1
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "\n",
+        "## Conclusion\n",
+        "\n",
+        "This tutorial goes over traditional approaches in recommender system and a deep learning based approach.  We also show that how to train and use the model with PaddlePaddle.  Deep learning has been well used in computer vision and NLP, we look forward to its new successes in recommender systems.\n",
+        "\n",
+        "## Reference\n",
+        "\n",
+        "1. [Peter Brusilovsky](https://en.wikipedia.org/wiki/Peter_Brusilovsky) (2007). *The Adaptive Web*. p. 325.\n",
+        "2. Robin Burke , [Hybrid Web Recommender Systems](http://www.dcs.warwick.ac.uk/~acristea/courses/CS411/2010/Book%20-%20The%20Adaptive%20Web/HybridWebRecommenderSystems.pdf), pp. 377-408, The Adaptive Web, Peter Brusilovsky, Alfred Kobsa, Wolfgang Nejdl (Ed.), Lecture Notes in Computer Science, Springer-Verlag, Berlin, Germany, Lecture Notes in Computer Science, Vol. 4321, May 2007, 978-3-540-72078-2.\n",
+        "3. P. Resnick, N. Iacovou, etc. “[GroupLens: An Open Architecture for Collaborative Filtering of Netnews](http://ccs.mit.edu/papers/CCSWP165.html)”, Proceedings of ACM Conference on Computer Supported Cooperative Work, CSCW 1994. pp.175-186.\n",
+        "4. Sarwar, Badrul, et al. \"[Item-based collaborative filtering recommendation algorithms.](http://files.grouplens.org/papers/www10_sarwar.pdf)\" *Proceedings of the 10th International Conference on World Wide Web*. ACM, 2001.\n",
+        "5. Kautz, Henry, Bart Selman, and Mehul Shah. \"[Referral Web: Combining Social networks and collaborative filtering.](http://www.cs.cornell.edu/selman/papers/pdf/97.cacm.refweb.pdf)\" Communications of the ACM 40.3 (1997): 63-65. APA\n",
+        "6. Yuan, Jianbo, et al. [\"Solving Cold-Start Problem in Large-scale Recommendation Engines: A Deep Learning Approach.\"](https://arxiv.org/pdf/1611.05480v1.pdf) *arXiv preprint arXiv:1611.05480* (2016).\n",
+        "7. Covington P, Adams J, Sargin E. [Deep neural networks for youtube recommendations](https://static.googleusercontent.com/media/research.google.com/zh-CN//pubs/archive/45530.pdf)[C]//Proceedings of the 10th ACM Conference on Recommender Systems. ACM, 2016: 191-198.\n",
+        "\n",
+        "\u003cbr/\u003e\n",
+        "This tutorial is contributed by \u003ca xmlns:cc=\"http://creativecommons.org/ns#\" href=\"http://book.paddlepaddle.org\" property=\"cc:attributionName\" rel=\"cc:attributionURL\"\u003ePaddlePaddle\u003c/a\u003e, and licensed under a \u003ca rel=\"license\" href=\"http://creativecommons.org/licenses/by-nc-sa/4.0/\"\u003eCreative Commons Attribution-NonCommercial-ShareAlike 4.0 International License\u003c/a\u003e.\n"
+      ]
+    }
+  ],
+  "metadata": {
+    "kernelspec": {
+      "display_name": "Python 3",
+      "language": "python",
+      "name": "python3"
+    },
+    "language_info": {
+      "codemirror_mode": {
+        "name": "ipython",
+        "version": 3
+      },
+      "file_extension": ".py",
+      "mimetype": "text/x-python",
+      "name": "python",
+      "nbconvert_exporter": "python",
+      "pygments_lexer": "ipython3",
+      "version": "3.6.0"
+    }
+  },
+  "nbformat": 4,
+  "nbformat_minor": 0
+}

recommender_system/README.en.md

@@ -2,6 +2,9 @@
 
 The source code of this tutorial is in [book/recommender_system](https://github.com/PaddlePaddle/book/tree/develop/recommender_system).
 
+For instructions on getting started with PaddlePaddle, see [PaddlePaddle installation guide](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/getstarted/build_and_install/docker_install_en.rst).
+
+
 ## Background
 
 With the fast growth of e-commerce, online videos, and online reading business, users have to rely on recommender systems to avoid manually browsing tremendous volume of choices.  Recommender systems understand users' interest by mining user behavior and other properties of users and products.
@@ -76,22 +79,287 @@ Figure 3. A hybrid recommendation model.
 
 ## Dataset
 
-We use the [MovieLens ml-1m](http://files.grouplens.org/datasets/movielens/ml-1m.zip) to train our model.  This dataset includes 10,000 ratings of 4,000 movies from 6,000 users to 4,000 movies.  Each rate is in the range of 1~5.  Thanks to GroupLens Research for collecting, processing and publishing the dataset.  
-
-We don't have to download and preprocess the data.  Instead, we can use PaddlePaddle's dataset module `paddle.v2.dataset.movielens`.
-
-
-## Model Specification
-
-
-
-## Training
-
-
-
-## Inference
-
+We use the [MovieLens ml-1m](http://files.grouplens.org/datasets/movielens/ml-1m.zip) to train our model.  This dataset includes 10,000 ratings of 4,000 movies from 6,000 users to 4,000 movies.  Each rate is in the range of 1~5.  Thanks to GroupLens Research for collecting, processing and publishing the dataset.
+
+`paddle.v2.datasets` package encapsulates multiple public datasets, including `cifar`, `imdb`, `mnist`, `moivelens` and `wmt14`, etc. There's no need for us to manually download and preprocess `MovieLens` dataset.
+
+```python
+# Run this block to show dataset's documentation
+help(paddle.v2.dataset.movielens)
+```
 
+The raw `MoiveLens` contains movie ratings, relevant features from both movies and users.
+For instance, one movie's feature could be:
+
+```python
+movie_info = paddle.dataset.movielens.movie_info()
+print movie_info.values()[0]
+```
+
+```text
+<MovieInfo id(1), title(Toy Story), categories(['Animation', "Children's", 'Comedy'])>
+```
+
+One user's feature could be:
+
+```python
+user_info = paddle.dataset.movielens.user_info()
+print user_info.values()[0]
+```
+
+```text
+<UserInfo id(1), gender(F), age(1), job(10)>
+```
+
+In this dateset, the distribution of age is shown as follows:
+
+```text
+1: "Under 18"
+18: "18-24"
+25: "25-34"
+35: "35-44"
+45: "45-49"
+50: "50-55"
+56: "56+"
+```
+
+User's occupation is selected from the following options:
+
+```text
+0: "other" or not specified
+1: "academic/educator"
+2: "artist"
+3: "clerical/admin"
+4: "college/grad student"
+5: "customer service"
+6: "doctor/health care"
+7: "executive/managerial"
+8: "farmer"
+9: "homemaker"
+10: "K-12 student"
+11: "lawyer"
+12: "programmer"
+13: "retired"
+14: "sales/marketing"
+15: "scientist"
+16: "self-employed"
+17: "technician/engineer"
+18: "tradesman/craftsman"
+19: "unemployed"
+20: "writer"
+```
+
+Each record consists of three main components: user features, movie features and movie ratings.
+Likewise, as a simple example, consider the following:
+
+```python
+train_set_creator = paddle.dataset.movielens.train()
+train_sample = next(train_set_creator())
+uid = train_sample[0]
+mov_id = train_sample[len(user_info[uid].value())]
+print "User %s rates Movie %s with Score %s"%(user_info[uid], movie_info[mov_id], train_sample[-1])
+```
+
+```text
+User <UserInfo id(1), gender(F), age(1), job(10)> rates Movie <MovieInfo id(1193), title(One Flew Over the Cuckoo's Nest), categories(['Drama'])> with Score [5.0]
+```
+
+The output shows that user 1 gave movie `1193` a rating of 5.
+
+After issuing a command `python train.py`, training will start immediately. The details will be unpacked by the following sessions to see how it works.
+
+## Model Architecture
+
+### Initialize PaddlePaddle
+
+First, we must import and initialize PaddlePaddle (enable/disable GPU, set the number of trainers, etc).
+
+```python
+%matplotlib inline
+
+import matplotlib.pyplot as plt
+from IPython import display
+import cPickle
+
+import paddle.v2 as paddle
+
+paddle.init(use_gpu=False)
+```
+
+### Model Configuration
+
+```python
+uid = paddle.layer.data(
+        name='user_id',
+        type=paddle.data_type.integer_value(
+            paddle.dataset.movielens.max_user_id() + 1))
+usr_emb = paddle.layer.embedding(input=uid, size=32)
+
+usr_gender_id = paddle.layer.data(
+        name='gender_id', type=paddle.data_type.integer_value(2))
+usr_gender_emb = paddle.layer.embedding(input=usr_gender_id, size=16)
+
+usr_age_id = paddle.layer.data(
+        name='age_id',
+        type=paddle.data_type.integer_value(
+            len(paddle.dataset.movielens.age_table)))
+usr_age_emb = paddle.layer.embedding(input=usr_age_id, size=16)
+
+usr_job_id = paddle.layer.data(
+        name='job_id',
+        type=paddle.data_type.integer_value(paddle.dataset.movielens.max_job_id(
+        ) + 1))
+usr_job_emb = paddle.layer.embedding(input=usr_job_id, size=16)
+```
+
+As shown in the above code, the input is four dimension integers for each user, that is,  `user_id`,`gender_id`, `age_id` and `job_id`. In order to deal with these features conveniently, we use the language model in NLP to transform these discrete values into embedding vaules `usr_emb`, `usr_gender_emb`, `usr_age_emb` and `usr_job_emb`.
+
+```python
+usr_combined_features = paddle.layer.fc(
+        input=[usr_emb, usr_gender_emb, usr_age_emb, usr_job_emb],
+        size=200,
+        act=paddle.activation.Tanh())
+```
+
+Then, employing user features as input, directly connecting to a fully-connected layer, which is used to reduce dimension to 200.
+
+Furthermore, we do a similar transformation for each movie feature. The model configuration is:
+
+```python
+mov_id = paddle.layer.data(
+    name='movie_id',
+    type=paddle.data_type.integer_value(
+        paddle.dataset.movielens.max_movie_id() + 1))
+mov_emb = paddle.layer.embedding(input=mov_id, size=32)
+
+mov_categories = paddle.layer.data(
+    name='category_id',
+    type=paddle.data_type.sparse_binary_vector(
+        len(paddle.dataset.movielens.movie_categories())))
+
+mov_categories_hidden = paddle.layer.fc(input=mov_categories, size=32)
+
+
+movie_title_dict = paddle.dataset.movielens.get_movie_title_dict()
+mov_title_id = paddle.layer.data(
+    name='movie_title',
+    type=paddle.data_type.integer_value_sequence(len(movie_title_dict)))
+mov_title_emb = paddle.layer.embedding(input=mov_title_id, size=32)
+mov_title_conv = paddle.networks.sequence_conv_pool(
+    input=mov_title_emb, hidden_size=32, context_len=3)
+
+mov_combined_features = paddle.layer.fc(
+    input=[mov_emb, mov_categories_hidden, mov_title_conv],
+    size=200,
+    act=paddle.activation.Tanh())
+```
+
+Movie title, a sequence of words represented by an integer word index sequence, will be feed into a `sequence_conv_pool` layer, which will apply convolution and pooling on time dimension. Because pooling is done on time dimension, the output will be a fixed-length vector regardless the length of the input sequence.
+
+Finally, we can use cosine similarity to calculate the similarity between user characteristics and movie features.
+
+```python
+inference = paddle.layer.cos_sim(a=usr_combined_features, b=mov_combined_features, size=1, scale=5)
+cost = paddle.layer.regression_cost(
+        input=inference,
+        label=paddle.layer.data(
+        name='score', type=paddle.data_type.dense_vector(1)))
+```
+
+## Model Training
+
+### Define Parameters
+
+First, we define the model parameters according to the previous model configuration `cost`.
+
+```python
+# Create parameters
+parameters = paddle.parameters.create(cost)
+```
+
+### Create Trainer
+
+Before jumping into creating a training module, algorithm setting is also necessary. Here we specified Adam optimization algorithm via `paddle.optimizer`.
+
+```python
+trainer = paddle.trainer.SGD(cost=cost, parameters=parameters,
+                             update_equation=paddle.optimizer.Adam(learning_rate=1e-4))
+```
+
+```text
+[INFO 2017-03-06 17:12:13,378 networks.py:1472] The input order is [user_id, gender_id, age_id, job_id, movie_id, category_id, movie_title, score]
+[INFO 2017-03-06 17:12:13,379 networks.py:1478] The output order is [__regression_cost_0__]
+```
+
+### Training
+
+`paddle.dataset.movielens.train` will yield records during each pass, after shuffling, a batch input is generated for training.
+
+```python
+reader=paddle.reader.batch(
+    paddle.reader.shuffle(
+        paddle.dataset.movielens.trai(), buf_size=8192),
+        batch_size=256)
+```
+
+`feeding` is devoted to specifying the correspondence between each yield record and `paddle.layer.data`. For instance, the first column of data generated by `movielens.train` corresponds to `user_id` feature.
+
+```python
+feeding = {
+    'user_id': 0,
+    'gender_id': 1,
+    'age_id': 2,
+    'job_id': 3,
+    'movie_id': 4,
+    'category_id': 5,
+    'movie_title': 6,
+    'score': 7
+}
+```
+
+Callback function `event_handler` will be called during training when a pre-defined event happens.
+
+```python
+step=0
+
+train_costs=[],[]
+test_costs=[],[]
+
+def event_handler(event):
+    global step
+    global train_costs
+    global test_costs
+    if isinstance(event, paddle.event.EndIteration):
+        need_plot = False
+        if step % 10 == 0:  # every 10 batches, record a train cost
+            train_costs[0].append(step)
+            train_costs[1].append(event.cost)
+
+        if step % 1000 == 0: # every 1000 batches, record a test cost
+            result = trainer.test(reader=paddle.batch(
+                  paddle.dataset.movielens.test(), batch_size=256))
+            test_costs[0].append(step)
+            test_costs[1].append(result.cost)
+
+        if step % 100 == 0: # every 100 batches, update cost plot
+            plt.plot(*train_costs)
+            plt.plot(*test_costs)
+            plt.legend(['Train Cost', 'Test Cost'], loc='upper left')
+            display.clear_output(wait=True)
+            display.display(plt.gcf())
+            plt.gcf().clear()
+        step += 1
+```
+
+Finally, we can invoke `trainer.train` to start training:
+
+```python
+trainer.train(
+    reader=reader,
+    event_handler=event_handler,
+    feeding=feeding,
+    num_passes=200)
+```
 
 ## Conclusion
 
@@ -99,13 +367,13 @@ This tutorial goes over traditional approaches in recommender system and a deep
 
 ## Reference
 
-1. [Peter Brusilovsky](https://en.wikipedia.org/wiki/Peter_Brusilovsky) (2007). *The Adaptive Web*. p. 325.
-2. Robin Burke , [Hybrid Web Recommender Systems](http://www.dcs.warwick.ac.uk/~acristea/courses/CS411/2010/Book%20-%20The%20Adaptive%20Web/HybridWebRecommenderSystems.pdf), pp. 377-408, The Adaptive Web, Peter Brusilovsky, Alfred Kobsa, Wolfgang Nejdl (Ed.), Lecture Notes in Computer Science, Springer-Verlag, Berlin, Germany, Lecture Notes in Computer Science, Vol. 4321, May 2007, 978-3-540-72078-2.
+1. [Peter Brusilovsky](https://en.wikipedia.org/wiki/Peter_Brusilovsky) (2007). *The Adaptive Web*. p. 325.
+2. Robin Burke , [Hybrid Web Recommender Systems](http://www.dcs.warwick.ac.uk/~acristea/courses/CS411/2010/Book%20-%20The%20Adaptive%20Web/HybridWebRecommenderSystems.pdf), pp. 377-408, The Adaptive Web, Peter Brusilovsky, Alfred Kobsa, Wolfgang Nejdl (Ed.), Lecture Notes in Computer Science, Springer-Verlag, Berlin, Germany, Lecture Notes in Computer Science, Vol. 4321, May 2007, 978-3-540-72078-2.
 3. P. Resnick, N. Iacovou, etc. “[GroupLens: An Open Architecture for Collaborative Filtering of Netnews](http://ccs.mit.edu/papers/CCSWP165.html)”, Proceedings of ACM Conference on Computer Supported Cooperative Work, CSCW 1994. pp.175-186.
-4. Sarwar, Badrul, et al. "[Item-based collaborative filtering recommendation algorithms.](http://files.grouplens.org/papers/www10_sarwar.pdf)" *Proceedings of the 10th International Conference on World Wide Web*. ACM, 2001.
+4. Sarwar, Badrul, et al. "[Item-based collaborative filtering recommendation algorithms.](http://files.grouplens.org/papers/www10_sarwar.pdf)" *Proceedings of the 10th International Conference on World Wide Web*. ACM, 2001.
 5. Kautz, Henry, Bart Selman, and Mehul Shah. "[Referral Web: Combining Social networks and collaborative filtering.](http://www.cs.cornell.edu/selman/papers/pdf/97.cacm.refweb.pdf)" Communications of the ACM 40.3 (1997): 63-65. APA
-6. Yuan, Jianbo, et al. ["Solving Cold-Start Problem in Large-scale Recommendation Engines: A Deep Learning Approach."](https://arxiv.org/pdf/1611.05480v1.pdf) *arXiv preprint arXiv:1611.05480* (2016).
+6. Yuan, Jianbo, et al. ["Solving Cold-Start Problem in Large-scale Recommendation Engines: A Deep Learning Approach."](https://arxiv.org/pdf/1611.05480v1.pdf) *arXiv preprint arXiv:1611.05480* (2016).
 7. Covington P, Adams J, Sargin E. [Deep neural networks for youtube recommendations](https://static.googleusercontent.com/media/research.google.com/zh-CN//pubs/archive/45530.pdf)[C]//Proceedings of the 10th ACM Conference on Recommender Systems. ACM, 2016: 191-198.
 
 <br/>
-<a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/4.0/"><img alt="Creative Commons" style="border-width:0" src="https://i.creativecommons.org/l/by-nc-sa/4.0/88x31.png" /></a><br /><span xmlns:dct="http://purl.org/dc/terms/" href="http://purl.org/dc/dcmitype/Text" property="dct:title" rel="dct:type">This tutorial</span> was created by <a xmlns:cc="http://creativecommons.org/ns#" href="http://book.paddlepaddle.org" property="cc:attributionName" rel="cc:attributionURL">the PaddlePaddle community</a> and published under <a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/4.0/">Common Creative 4.0 License</a>。
+This tutorial is contributed by <a xmlns:cc="http://creativecommons.org/ns#" href="http://book.paddlepaddle.org" property="cc:attributionName" rel="cc:attributionURL">PaddlePaddle</a>, and licensed under a <a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/4.0/">Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License</a>.

recommender_system/README.ipynb

 {
- "cells": [
-  {
-   "cell_type": "markdown",
-   "metadata": {
-    "collapsed": true,
-    "deletable": true,
-    "editable": true
-   },
-   "source": [
-    "# 个性化推荐\n",
-    "\n",
-    "本教程源代码目录在[book/recommender_system](https://github.com/PaddlePaddle/book/tree/develop/recommender_system)， 初次使用请参考PaddlePaddle[安装教程](http://www.paddlepaddle.org/doc_cn/build_and_install/index.html)。\n",
-    "\n",
-    "## 背景介绍\n",
-    "\n",
-    "在网络技术不断发展和电子商务规模不断扩大的背景下，商品数量和种类快速增长，用户需要花费大量时间才能找到自己想买的商品，这就是信息超载问题。为了解决这个难题，推荐系统（Recommender System）应运而生。\n",
-    "\n",
-    "个性化推荐系统是信息过滤系统（Information Filtering System）的子集，它可以用在很多领域，如电影、音乐、电商和 Feed 流推荐等。推荐系统通过分析、挖掘用户行为，发现用户的个性化需求与兴趣特点，将用户可能感兴趣的信息或商品推荐给用户。与搜索引擎不同，推荐系统不需要用户准确地描述出自己的需求，而是根据分析历史行为建模，主动提供满足用户兴趣和需求的信息。\n",
-    "\n",
-    "传统的推荐系统方法主要有：\n",
-    "\n",
-    "- 协同过滤推荐（Collaborative Filtering Recommendation）：该方法收集分析用户历史行为、活动、偏好，计算一个用户与其他用户的相似度，利用目标用户的相似用户对商品评价的加权评价值，来预测目标用户对特定商品的喜好程度。优点是可以给用户推荐未浏览过的新产品；缺点是对于没有任何行为的新用户存在冷启动的问题，同时也存在用户与商品之间的交互数据不够多造成的稀疏问题，会导致模型难以找到相近用户。\n",
-    "- 基于内容过滤推荐[[1](#参考文献)]（Content-based Filtering Recommendation）：该方法利用商品的内容描述，抽象出有意义的特征，通过计算用户的兴趣和商品描述之间的相似度，来给用户做推荐。优点是简单直接，不需要依据其他用户对商品的评价，而是通过商品属性进行商品相似度度量，从而推荐给用户所感兴趣商品的相似商品；缺点是对于没有任何行为的新用户同样存在冷启动的问题。\n",
-    "- 组合推荐[[2](#参考文献)]（Hybrid Recommendation）：运用不同的输入和技术共同进行推荐，以弥补各自推荐技术的缺点。\n",
-    "\n",
-    "其中协同过滤是应用最广泛的技术之一，它又可以分为多个子类：基于用户 （User-Based）的推荐[[3](#参考文献)] 、基于物品（Item-Based）的推荐[[4](#参考文献)]、基于社交网络关系（Social-Based）的推荐[[5](#参考文献)]、基于模型（Model-based）的推荐等。1994年明尼苏达大学推出的GroupLens系统[[3](#参考文献)]一般被认为是推荐系统成为一个相对独立的研究方向的标志。该系统首次提出了基于协同过滤来完成推荐任务的思想，此后，基于该模型的协同过滤推荐引领了推荐系统十几年的发展方向。\n",
-    "\n",
-    "深度学习具有优秀的自动提取特征的能力，能够学习多层次的抽象特征表示，并对异质或跨域的内容信息进行学习，可以一定程度上处理推荐系统冷启动问题[[6](#参考文献)]。本教程主要介绍个性化推荐的深度学习模型，以及如何使用PaddlePaddle实现模型。\n",
-    "\n",
-    "## 效果展示\n",
-    "\n",
-    "我们使用包含用户信息、电影信息与电影评分的数据集作为个性化推荐的应用场景。当我们训练好模型后，只需要输入对应的用户ID和电影ID，就可以得出一个匹配的分数（范围[1,5]，分数越高视为兴趣越大），然后根据所有电影的推荐得分排序，推荐给用户可能感兴趣的电影。\n",
-    "\n",
-    "```\n",
-    "Input movie_id: 1962\n",
-    "Input user_id: 1\n",
-    "Prediction Score is 4.25\n",
-    "```\n",
-    "\n",
-    "## 模型概览\n",
-    "\n",
-    "本章中，我们首先介绍YouTube的视频推荐系统[[7](#参考文献)]，然后介绍我们实现的融合推荐模型。\n",
-    "\n",
-    "### YouTube的深度神经网络推荐系统\n",
-    "\n",
-    "YouTube是世界上最大的视频上传、分享和发现网站，YouTube推荐系统为超过10亿用户从不断增长的视频库中推荐个性化的内容。整个系统由两个神经网络组成：候选生成网络和排序网络。候选生成网络从百万量级的视频库中生成上百个候选，排序网络对候选进行打分排序，输出排名最高的数十个结果。系统结构如图1所示：\n",
-    "\n",
-    "<p align=\"center\">\n",
-    "<img src=\"image/YouTube_Overview.png\" width=\"70%\" ><br/>\n",
-    "图1. YouTube 推荐系统结构\n",
-    "</p>\n",
-    "\n",
-    "#### 候选生成网络（Candidate Generation Network）\n",
-    "\n",
-    "候选生成网络将推荐问题建模为一个类别数极大的多类分类问题：对于一个Youtube用户，使用其观看历史（视频ID）、搜索词记录（search tokens）、人口学信息（如地理位置、用户登录设备）、二值特征（如性别，是否登录）和连续特征（如用户年龄）等，对视频库中所有视频进行多分类，得到每一类别的分类结果（即每一个视频的推荐概率），最终输出概率较高的几百个视频。\n",
-    "\n",
-    "首先，将观看历史及搜索词记录这类历史信息，映射为向量后取平均值得到定长表示；同时，输入人口学特征以优化新用户的推荐效果，并将二值特征和连续特征归一化处理到[0, 1]范围。接下来，将所有特征表示拼接为一个向量，并输入给非线形多层感知器（MLP，详见[识别数字](https://github.com/PaddlePaddle/book/blob/develop/recognize_digits/README.md)教程）处理。最后，训练时将MLP的输出给softmax做分类，预测时计算用户的综合特征（MLP的输出）与所有视频的相似度，取得分最高的$k$个作为候选生成网络的筛选结果。图2显示了候选生成网络结构。\n",
-    "\n",
-    "<p align=\"center\">\n",
-    "<img src=\"image/Deep_candidate_generation_model_architecture.png\" width=\"70%\" ><br/>\n",
-    "图2. 候选生成网络结构\n",
-    "</p>\n",
-    "\n",
-    "对于一个用户$U$，预测此刻用户要观看的视频$\\omega$为视频$i$的概率公式为：\n",
-    "\n",
-    "$$P(\\omega=i|u)=\\frac{e^{v_{i}u}}{\\sum_{j \\in V}e^{v_{j}u}}$$\n",
-    "\n",
-    "其中$u$为用户$U$的特征表示，$V$为视频库集合，$v_i$为视频库中第$i$个视频的特征表示。$u$和$v_i$为长度相等的向量，两者点积可以通过全连接层实现。\n",
-    "\n",
-    "考虑到softmax分类的类别数非常多，为了保证一定的计算效率：1）训练阶段，使用负样本类别采样将实际计算的类别数缩小至数千；2）推荐（预测）阶段，忽略softmax的归一化计算（不影响结果），将类别打分问题简化为点积（dot product）空间中的最近邻（nearest neighbor）搜索问题，取与$u$最近的$k$个视频作为生成的候选。\n",
-    "\n",
-    "#### 排序网络（Ranking Network）\n",
-    "排序网络的结构类似于候选生成网络，但是它的目标是对候选进行更细致的打分排序。和传统广告排序中的特征抽取方法类似，这里也构造了大量的用于视频排序的相关特征（如视频 ID、上次观看时间等）。这些特征的处理方式和候选生成网络类似，不同之处是排序网络的顶部是一个加权逻辑回归（weighted logistic regression），它对所有候选视频进行打分，从高到底排序后将分数较高的一些视频返回给用户。\n",
-    "\n",
-    "### 融合推荐模型\n",
-    "\n",
-    "在下文的电影推荐系统中：\n",
-    "\n",
-    "1. 首先，使用用户特征和电影特征作为神经网络的输入，其中：\n",
-    "\n",
-    "   - 用户特征融合了四个属性信息，分别是用户ID、性别、职业和年龄。\n",
-    "\n",
-    "   - 电影特征融合了三个属性信息，分别是电影ID、电影类型ID和电影名称。\n",
-    "\n",
-    "2. 对用户特征，将用户ID映射为维度大小为256的向量表示，输入全连接层，并对其他三个属性也做类似的处理。然后将四个属性的特征表示分别全连接并相加。\n",
-    "\n",
-    "3. 对电影特征，将电影ID以类似用户ID的方式进行处理，电影类型ID以向量的形式直接输入全连接层，电影名称用文本卷积神经网络（详见[第5章](https://github.com/PaddlePaddle/book/blob/develop/understand_sentiment/README.md)）得到其定长向量表示。然后将三个属性的特征表示分别全连接并相加。\n",
-    "\n",
-    "4. 得到用户和电影的向量表示后，计算二者的余弦相似度作为推荐系统的打分。最后，用该相似度打分和用户真实打分的差异的平方作为该回归模型的损失函数。\n",
-    "\n",
-    "<p align=\"center\">\n",
-    "\n",
-    "<img src=\"image/rec_regression_network.png\" width=\"90%\" ><br/>\n",
-    "图3. 融合推荐模型 \n",
-    "</p> "
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
-   "source": [
-    "## 数据准备\n",
-    "\n",
-    "### 数据介绍与下载\n",
-    "\n",
-    "我们以 [MovieLens 百万数据集（ml-1m）](http://files.grouplens.org/datasets/movielens/ml-1m.zip)为例进行介绍。ml-1m 数据集包含了 6,000 位用户对 4,000 部电影的 1,000,000 条评价（评分范围 1~5 分，均为整数），由 GroupLens Research 实验室搜集整理。\n",
-    "\n",
-    "Paddle在API中提供了自动加载数据的模块。数据模块为 `paddle.dataset.movielens`"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 1,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
-   "outputs": [],
-   "source": [
-    "import paddle.v2 as paddle\n",
-    "paddle.init(use_gpu=False)"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 2,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
-   "outputs": [],
-   "source": [
-    "# Run this block to show dataset's documentation\n",
-    "# help(paddle.dataset.movielens)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
-   "source": [
-    "在原始数据中包含电影的特征数据，用户的特征数据，和用户对电影的评分。\n",
-    "\n",
-    "例如，其中某一个电影特征为:"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 3,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
-   "outputs": [
+  "cells": [
     {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "<MovieInfo id(1), title(Toy Story ), categories(['Animation', \"Children's\", 'Comedy'])>\n"
-     ]
-    }
-   ],
-   "source": [
-    "movie_info = paddle.dataset.movielens.movie_info()\n",
-    "print movie_info.values()[0]"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
-   "source": [
-    "这表示，电影的id是1，标题是《Toy Story》，该电影被分为到三个类别中。这三个类别是动画，儿童，喜剧。"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 4,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
-   "outputs": [
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "# 个性化推荐\n",
+        "\n",
+        "本教程源代码目录在[book/recommender_system](https://github.com/PaddlePaddle/book/tree/develop/recommender_system)， 初次使用请参考PaddlePaddle[安装教程](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/getstarted/build_and_install/docker_install_cn.rst)。\n",
+        "\n",
+        "## 背景介绍\n",
+        "\n",
+        "在网络技术不断发展和电子商务规模不断扩大的背景下，商品数量和种类快速增长，用户需要花费大量时间才能找到自己想买的商品，这就是信息超载问题。为了解决这个难题，推荐系统（Recommender System）应运而生。\n",
+        "\n",
+        "个性化推荐系统是信息过滤系统（Information Filtering System）的子集，它可以用在很多领域，如电影、音乐、电商和 Feed 流推荐等。推荐系统通过分析、挖掘用户行为，发现用户的个性化需求与兴趣特点，将用户可能感兴趣的信息或商品推荐给用户。与搜索引擎不同，推荐系统不需要用户准确地描述出自己的需求，而是根据分析历史行为建模，主动提供满足用户兴趣和需求的信息。\n",
+        "\n",
+        "传统的推荐系统方法主要有：\n",
+        "\n",
+        "- 协同过滤推荐（Collaborative Filtering Recommendation）：该方法收集分析用户历史行为、活动、偏好，计算一个用户与其他用户的相似度，利用目标用户的相似用户对商品评价的加权评价值，来预测目标用户对特定商品的喜好程度。优点是可以给用户推荐未浏览过的新产品；缺点是对于没有任何行为的新用户存在冷启动的问题，同时也存在用户与商品之间的交互数据不够多造成的稀疏问题，会导致模型难以找到相近用户。\n",
+        "- 基于内容过滤推荐[[1](#参考文献)]（Content-based Filtering Recommendation）：该方法利用商品的内容描述，抽象出有意义的特征，通过计算用户的兴趣和商品描述之间的相似度，来给用户做推荐。优点是简单直接，不需要依据其他用户对商品的评价，而是通过商品属性进行商品相似度度量，从而推荐给用户所感兴趣商品的相似商品；缺点是对于没有任何行为的新用户同样存在冷启动的问题。\n",
+        "- 组合推荐[[2](#参考文献)]（Hybrid Recommendation）：运用不同的输入和技术共同进行推荐，以弥补各自推荐技术的缺点。\n",
+        "\n",
+        "其中协同过滤是应用最广泛的技术之一，它又可以分为多个子类：基于用户 （User-Based）的推荐[[3](#参考文献)] 、基于物品（Item-Based）的推荐[[4](#参考文献)]、基于社交网络关系（Social-Based）的推荐[[5](#参考文献)]、基于模型（Model-based）的推荐等。1994年明尼苏达大学推出的GroupLens系统[[3](#参考文献)]一般被认为是推荐系统成为一个相对独立的研究方向的标志。该系统首次提出了基于协同过滤来完成推荐任务的思想，此后，基于该模型的协同过滤推荐引领了推荐系统十几年的发展方向。\n",
+        "\n",
+        "深度学习具有优秀的自动提取特征的能力，能够学习多层次的抽象特征表示，并对异质或跨域的内容信息进行学习，可以一定程度上处理推荐系统冷启动问题[[6](#参考文献)]。本教程主要介绍个性化推荐的深度学习模型，以及如何使用PaddlePaddle实现模型。\n",
+        "\n",
+        "## 效果展示\n",
+        "\n",
+        "我们使用包含用户信息、电影信息与电影评分的数据集作为个性化推荐的应用场景。当我们训练好模型后，只需要输入对应的用户ID和电影ID，就可以得出一个匹配的分数（范围[1,5]，分数越高视为兴趣越大），然后根据所有电影的推荐得分排序，推荐给用户可能感兴趣的电影。\n",
+        "\n",
+        "```\n",
+        "Input movie_id: 1962\n",
+        "Input user_id: 1\n",
+        "Prediction Score is 4.25\n",
+        "```\n",
+        "\n",
+        "## 模型概览\n",
+        "\n",
+        "本章中，我们首先介绍YouTube的视频推荐系统[[7](#参考文献)]，然后介绍我们实现的融合推荐模型。\n",
+        "\n",
+        "### YouTube的深度神经网络推荐系统\n",
+        "\n",
+        "YouTube是世界上最大的视频上传、分享和发现网站，YouTube推荐系统为超过10亿用户从不断增长的视频库中推荐个性化的内容。整个系统由两个神经网络组成：候选生成网络和排序网络。候选生成网络从百万量级的视频库中生成上百个候选，排序网络对候选进行打分排序，输出排名最高的数十个结果。系统结构如图1所示：\n",
+        "\n",
+        "\u003cp align=\"center\"\u003e\n",
+        "\u003cimg src=\"image/YouTube_Overview.png\" width=\"70%\" \u003e\u003cbr/\u003e\n",
+        "图1. YouTube 推荐系统结构\n",
+        "\u003c/p\u003e\n",
+        "\n",
+        "#### 候选生成网络（Candidate Generation Network）\n",
+        "\n",
+        "候选生成网络将推荐问题建模为一个类别数极大的多类分类问题：对于一个Youtube用户，使用其观看历史（视频ID）、搜索词记录（search tokens）、人口学信息（如地理位置、用户登录设备）、二值特征（如性别，是否登录）和连续特征（如用户年龄）等，对视频库中所有视频进行多分类，得到每一类别的分类结果（即每一个视频的推荐概率），最终输出概率较高的几百个视频。\n",
+        "\n",
+        "首先，将观看历史及搜索词记录这类历史信息，映射为向量后取平均值得到定长表示；同时，输入人口学特征以优化新用户的推荐效果，并将二值特征和连续特征归一化处理到[0, 1]范围。接下来，将所有特征表示拼接为一个向量，并输入给非线形多层感知器（MLP，详见[识别数字](https://github.com/PaddlePaddle/book/blob/develop/recognize_digits/README.md)教程）处理。最后，训练时将MLP的输出给softmax做分类，预测时计算用户的综合特征（MLP的输出）与所有视频的相似度，取得分最高的$k$个作为候选生成网络的筛选结果。图2显示了候选生成网络结构。\n",
+        "\n",
+        "\u003cp align=\"center\"\u003e\n",
+        "\u003cimg src=\"image/Deep_candidate_generation_model_architecture.png\" width=\"70%\" \u003e\u003cbr/\u003e\n",
+        "图2. 候选生成网络结构\n",
+        "\u003c/p\u003e\n",
+        "\n",
+        "对于一个用户$U$，预测此刻用户要观看的视频$\\omega$为视频$i$的概率公式为：\n",
+        "\n",
+        "$$P(\\omega=i|u)=\\frac{e^{v_{i}u}}{\\sum_{j \\in V}e^{v_{j}u}}$$\n",
+        "\n",
+        "其中$u$为用户$U$的特征表示，$V$为视频库集合，$v_i$为视频库中第$i$个视频的特征表示。$u$和$v_i$为长度相等的向量，两者点积可以通过全连接层实现。\n",
+        "\n",
+        "考虑到softmax分类的类别数非常多，为了保证一定的计算效率：1）训练阶段，使用负样本类别采样将实际计算的类别数缩小至数千；2）推荐（预测）阶段，忽略softmax的归一化计算（不影响结果），将类别打分问题简化为点积（dot product）空间中的最近邻（nearest neighbor）搜索问题，取与$u$最近的$k$个视频作为生成的候选。\n",
+        "\n",
+        "#### 排序网络（Ranking Network）\n",
+        "排序网络的结构类似于候选生成网络，但是它的目标是对候选进行更细致的打分排序。和传统广告排序中的特征抽取方法类似，这里也构造了大量的用于视频排序的相关特征（如视频 ID、上次观看时间等）。这些特征的处理方式和候选生成网络类似，不同之处是排序网络的顶部是一个加权逻辑回归（weighted logistic regression），它对所有候选视频进行打分，从高到底排序后将分数较高的一些视频返回给用户。\n",
+        "\n",
+        "### 融合推荐模型\n",
+        "\n",
+        "在下文的电影推荐系统中：\n",
+        "\n",
+        "1. 首先，使用用户特征和电影特征作为神经网络的输入，其中：\n",
+        "\n",
+        "   - 用户特征融合了四个属性信息，分别是用户ID、性别、职业和年龄。\n",
+        "\n",
+        "   - 电影特征融合了三个属性信息，分别是电影ID、电影类型ID和电影名称。\n",
+        "\n",
+        "2. 对用户特征，将用户ID映射为维度大小为256的向量表示，输入全连接层，并对其他三个属性也做类似的处理。然后将四个属性的特征表示分别全连接并相加。\n",
+        "\n",
+        "3. 对电影特征，将电影ID以类似用户ID的方式进行处理，电影类型ID以向量的形式直接输入全连接层，电影名称用文本卷积神经网络（详见[第5章](https://github.com/PaddlePaddle/book/blob/develop/understand_sentiment/README.md)）得到其定长向量表示。然后将三个属性的特征表示分别全连接并相加。\n",
+        "\n",
+        "4. 得到用户和电影的向量表示后，计算二者的余弦相似度作为推荐系统的打分。最后，用该相似度打分和用户真实打分的差异的平方作为该回归模型的损失函数。\n",
+        "\n",
+        "\u003cp align=\"center\"\u003e\n",
+        "\n",
+        "\u003cimg src=\"image/rec_regression_network.png\" width=\"90%\" \u003e\u003cbr/\u003e\n",
+        "图3. 融合推荐模型\n",
+        "\u003c/p\u003e\n",
+        "\n",
+        "## 数据准备\n",
+        "\n",
+        "### 数据介绍与下载\n",
+        "\n",
+        "我们以 [MovieLens 百万数据集（ml-1m）](http://files.grouplens.org/datasets/movielens/ml-1m.zip)为例进行介绍。ml-1m 数据集包含了 6,000 位用户对 4,000 部电影的 1,000,000 条评价（评分范围 1~5 分，均为整数），由 GroupLens Research 实验室搜集整理。\n",
+        "\n",
+        "Paddle在API中提供了自动加载数据的模块。数据模块为 `paddle.dataset.movielens`\n",
+        "\n",
+        "\n"
+      ]
+    },
     {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "<UserInfo id(1), gender(F), age(1), job(10)>\n"
-     ]
-    }
-   ],
-   "source": [
-    "user_info = paddle.dataset.movielens.user_info()\n",
-    "print user_info.values()[0]"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
-   "source": [
-    "这表示，该用户ID是1，女性，年龄比18岁还年轻。职业ID是10。\n",
-    "\n",
-    "\n",
-    "其中，年龄使用下列分布\n",
-    "*  1:  \"Under 18\"\n",
-    "* 18:  \"18-24\"\n",
-    "* 25:  \"25-34\"\n",
-    "* 35:  \"35-44\"\n",
-    "* 45:  \"45-49\"\n",
-    "* 50:  \"50-55\"\n",
-    "* 56:  \"56+\"\n",
-    "\n",
-    "职业是从下面几种选项里面选则得出:\n",
-    "*  0:  \"other\" or not specified\n",
-    "*  1:  \"academic/educator\"\n",
-    "*  2:  \"artist\"\n",
-    "*  3:  \"clerical/admin\"\n",
-    "*  4:  \"college/grad student\"\n",
-    "*  5:  \"customer service\"\n",
-    "*  6:  \"doctor/health care\"\n",
-    "*  7:  \"executive/managerial\"\n",
-    "*  8:  \"farmer\"\n",
-    "*  9:  \"homemaker\"\n",
-    "* 10:  \"K-12 student\"\n",
-    "* 11:  \"lawyer\"\n",
-    "* 12:  \"programmer\"\n",
-    "* 13:  \"retired\"\n",
-    "* 14:  \"sales/marketing\"\n",
-    "* 15:  \"scientist\"\n",
-    "* 16:  \"self-employed\"\n",
-    "* 17:  \"technician/engineer\"\n",
-    "* 18:  \"tradesman/craftsman\"\n",
-    "* 19:  \"unemployed\"\n",
-    "* 20:  \"writer\""
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
-   "source": [
-    "而对于每一条训练/测试数据，均为 <用户特征> + <电影特征> + 评分。\n",
-    "\n",
-    "例如，我们获得第一条训练数据:"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 5,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
-   "outputs": [
+      "cell_type": "code",
+      "metadata": {
+        "editable": true
+      },
+      "source": [
+        "import paddle.v2 as paddle\n",
+        "paddle.init(use_gpu=False)\n"
+      ],
+      "outputs": [
+        {
+          "name": "stdout",
+          "output_type": "stream",
+          "text": [
+            "\n"
+          ]
+        }
+      ],
+      "execution_count": 1
+    },
     {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "User <UserInfo id(1), gender(F), age(1), job(10)> rates Movie <MovieInfo id(1193), title(One Flew Over the Cuckoo's Nest ), categories(['Drama'])> with Score [5.0]\n"
-     ]
-    }
-   ],
-   "source": [
-    "train_set_creator = paddle.dataset.movielens.train()\n",
-    "train_sample = next(train_set_creator())\n",
-    "uid = train_sample[0]\n",
-    "mov_id = train_sample[len(user_info[uid].value())]\n",
-    "print \"User %s rates Movie %s with Score %s\"%(user_info[uid], movie_info[mov_id], train_sample[-1])"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
-   "source": [
-    "即用户1对电影1193的评价为5分。"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
-   "source": [
-    "## 模型配置说明\n",
-    "\n",
-    "下面我们开始根据输入数据的形式配置模型。"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 6,
-   "metadata": {
-    "collapsed": true,
-    "deletable": true,
-    "editable": true
-   },
-   "outputs": [],
-   "source": [
-    "uid = paddle.layer.data(\n",
-    "        name='user_id',\n",
-    "        type=paddle.data_type.integer_value(\n",
-    "            paddle.dataset.movielens.max_user_id() + 1))\n",
-    "usr_emb = paddle.layer.embedding(input=uid, size=32)\n",
-    "\n",
-    "usr_gender_id = paddle.layer.data(\n",
-    "        name='gender_id', type=paddle.data_type.integer_value(2))\n",
-    "usr_gender_emb = paddle.layer.embedding(input=usr_gender_id, size=16)\n",
-    "\n",
-    "usr_age_id = paddle.layer.data(\n",
-    "        name='age_id',\n",
-    "        type=paddle.data_type.integer_value(\n",
-    "            len(paddle.dataset.movielens.age_table)))\n",
-    "usr_age_emb = paddle.layer.embedding(input=usr_age_id, size=16)\n",
-    "\n",
-    "usr_job_id = paddle.layer.data(\n",
-    "        name='job_id',\n",
-    "        type=paddle.data_type.integer_value(paddle.dataset.movielens.max_job_id(\n",
-    "        ) + 1))\n",
-    "usr_job_emb = paddle.layer.embedding(input=usr_job_id, size=16)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
-   "source": [
-    "如上述代码所示，对于每个用户，我们输入4维特征。其中包括`user_id`,`gender_id`,`age_id`,`job_id`。这几维特征均是简单的整数值。为了后续神经网络处理这些特征方便，我们借鉴NLP中的语言模型，将这几维离散的整数值，变换成embedding取出。分别形成`usr_emb`, `usr_gender_emb`, `usr_age_emb`, `usr_job_emb`。"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 7,
-   "metadata": {
-    "collapsed": true,
-    "deletable": true,
-    "editable": true
-   },
-   "outputs": [],
-   "source": [
-    "usr_combined_features = paddle.layer.fc(\n",
-    "        input=[usr_emb, usr_gender_emb, usr_age_emb, usr_job_emb],\n",
-    "        size=200,\n",
-    "        act=paddle.activation.Tanh())"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
-   "source": [
-    "然后，我们对于所有的用户特征，均输入到一个全连接层(fc)中。将所有特征融合为一个200维度的特征。"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
-   "source": [
-    "进而，我们对每一个电影特征做类似的变换，网络配置为:"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 8,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
-   "outputs": [],
-   "source": [
-    "mov_id = paddle.layer.data(\n",
-    "    name='movie_id',\n",
-    "    type=paddle.data_type.integer_value(\n",
-    "        paddle.dataset.movielens.max_movie_id() + 1))\n",
-    "mov_emb = paddle.layer.embedding(input=mov_id, size=32)\n",
-    "\n",
-    "mov_categories = paddle.layer.data(\n",
-    "    name='category_id',\n",
-    "    type=paddle.data_type.sparse_binary_vector(\n",
-    "        len(paddle.dataset.movielens.movie_categories())))\n",
-    "\n",
-    "mov_categories_hidden = paddle.layer.fc(input=mov_categories, size=32)\n",
-    "\n",
-    "\n",
-    "movie_title_dict = paddle.dataset.movielens.get_movie_title_dict()\n",
-    "mov_title_id = paddle.layer.data(\n",
-    "    name='movie_title',\n",
-    "    type=paddle.data_type.integer_value_sequence(len(movie_title_dict)))\n",
-    "mov_title_emb = paddle.layer.embedding(input=mov_title_id, size=32)\n",
-    "mov_title_conv = paddle.networks.sequence_conv_pool(\n",
-    "    input=mov_title_emb, hidden_size=32, context_len=3)\n",
-    "\n",
-    "mov_combined_features = paddle.layer.fc(\n",
-    "    input=[mov_emb, mov_categories_hidden, mov_title_conv],\n",
-    "    size=200,\n",
-    "    act=paddle.activation.Tanh())"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
-   "source": [
-    "电影ID和电影类型分别映射到其对应的特征隐层。对于电影标题名称(title)，一个ID序列表示的词语序列，在输入卷积层后，将得到每个时间窗口的特征（序列特征），然后通过在时间维度降采样得到固定维度的特征，整个过程在text_conv_pool实现。\n",
-    "\n",
-    "最后再将电影的特征融合进`mov_combined_features`中。"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 9,
-   "metadata": {
-    "collapsed": true,
-    "deletable": true,
-    "editable": true
-   },
-   "outputs": [],
-   "source": [
-    "inference = paddle.layer.cos_sim(a=usr_combined_features, b=mov_combined_features, size=1, scale=5)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
-   "source": [
-    "进而，我们使用余弦相似度计算用户特征与电影特征的相似性。并将这个相似性拟合(回归)到用户评分上。"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 10,
-   "metadata": {
-    "collapsed": true,
-    "deletable": true,
-    "editable": true
-   },
-   "outputs": [],
-   "source": [
-    "cost = paddle.layer.regression_cost(\n",
-    "        input=inference,\n",
-    "        label=paddle.layer.data(\n",
-    "            name='score', type=paddle.data_type.dense_vector(1)))"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
-   "source": [
-    "至此，我们的优化目标就是这个网络配置中的`cost`了。"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
-   "source": [
-    "## 训练模型\n",
-    "\n",
-    "### 定义参数\n",
-    "神经网络的模型，我们可以简单的理解为网络拓朴结构+参数。之前一节，我们定义出了优化目标`cost`。这个`cost`即为网络模型的拓扑结构。我们开始训练模型，需要先定义出参数。定义方法为:"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 11,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
-   "outputs": [
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "\n",
+        "\n"
+      ]
+    },
     {
-     "name": "stderr",
-     "output_type": "stream",
-     "text": [
-      "[INFO 2017-03-06 17:12:13,284 networks.py:1472] The input order is [user_id, gender_id, age_id, job_id, movie_id, category_id, movie_title, score]\n",
-      "[INFO 2017-03-06 17:12:13,287 networks.py:1478] The output order is [__regression_cost_0__]\n"
-     ]
-    }
-   ],
-   "source": [
-    "parameters = paddle.parameters.create(cost)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
-   "source": [
-    "`parameters`是模型的所有参数集合。他是一个python的dict。我们可以查看到这个网络中的所有参数名称。因为之前定义模型的时候，我们没有指定参数名称，这里参数名称是自动生成的。当然，我们也可以指定每一个参数名称，方便日后维护。"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 12,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
-   "outputs": [
+      "cell_type": "code",
+      "metadata": {
+        "editable": true
+      },
+      "source": [
+        "# Run this block to show dataset's documentation\n",
+        "# help(paddle.dataset.movielens)\n"
+      ],
+      "outputs": [
+        {
+          "name": "stdout",
+          "output_type": "stream",
+          "text": [
+            "\n"
+          ]
+        }
+      ],
+      "execution_count": 1
+    },
     {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "[u'___fc_layer_2__.wbias', u'___fc_layer_2__.w2', u'___embedding_layer_3__.w0', u'___embedding_layer_5__.w0', u'___embedding_layer_2__.w0', u'___embedding_layer_1__.w0', u'___fc_layer_1__.wbias', u'___fc_layer_0__.wbias', u'___fc_layer_1__.w0', u'___fc_layer_0__.w2', u'___fc_layer_0__.w3', u'___fc_layer_0__.w0', u'___fc_layer_0__.w1', u'___fc_layer_2__.w1', u'___fc_layer_2__.w0', u'___embedding_layer_4__.w0', u'___sequence_conv_pool_0___conv_fc.w0', u'___embedding_layer_0__.w0', u'___sequence_conv_pool_0___conv_fc.wbias']\n"
-     ]
-    }
-   ],
-   "source": [
-    "print parameters.keys()"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
-   "source": [
-    "### 构造训练(trainer)\n",
-    "\n",
-    "下面，我们根据网络拓扑结构和模型参数来构造出一个本地训练(trainer)。在构造本地训练的时候，我们还需要指定这个训练的优化方法。这里我们使用Adam来作为优化算法。"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 13,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
-   "outputs": [
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "\n",
+        "在原始数据中包含电影的特征数据，用户的特征数据，和用户对电影的评分。\n",
+        "\n",
+        "例如，其中某一个电影特征为:\n",
+        "\n",
+        "\n"
+      ]
+    },
     {
-     "name": "stderr",
-     "output_type": "stream",
-     "text": [
-      "[INFO 2017-03-06 17:12:13,378 networks.py:1472] The input order is [user_id, gender_id, age_id, job_id, movie_id, category_id, movie_title, score]\n",
-      "[INFO 2017-03-06 17:12:13,379 networks.py:1478] The output order is [__regression_cost_0__]\n"
-     ]
-    }
-   ],
-   "source": [
-    "trainer = paddle.trainer.SGD(cost=cost, parameters=parameters, \n",
-    "                            update_equation=paddle.optimizer.Adam(learning_rate=1e-4))"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
-   "source": [
-    "### 训练\n",
-    "\n",
-    "下面我们开始训练过程。\n",
-    "\n",
-    "我们直接使用Paddle提供的数据集读取程序。`paddle.dataset.movielens.train()`和`paddle.dataset.movielens.test()`分别做训练和预测数据集。并且通过`reader_dict`来指定每一个数据和data_layer的对应关系。\n",
-    "\n",
-    "例如，这里的reader_dict表示的是，对于数据层 `user_id`，使用了reader中每一条数据的第0个元素。`gender_id`数据层使用了第1个元素。以此类推。\n",
-    "\n",
-    "训练过程是完全自动的。我们可以使用event_handler来观察训练过程，或进行测试等。这里我们在event_handler里面绘制了训练误差曲线和测试误差曲线。并且保存了模型。"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 14,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
-   "outputs": [
+      "cell_type": "code",
+      "metadata": {
+        "editable": true
+      },
+      "source": [
+        "movie_info = paddle.dataset.movielens.movie_info()\n",
+        "print movie_info.values()[0]\n"
+      ],
+      "outputs": [
+        {
+          "name": "stdout",
+          "output_type": "stream",
+          "text": [
+            "\n"
+          ]
+        }
+      ],
+      "execution_count": 1
+    },
     {
-     "data": {
-      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD8CAYAAABn919SAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzsnXd4HNX1v987s0VarbptuVvucpcLYGOaKQm2CRACKQQI\nEEpCEgL8EkIKIUC+hEBCEkIIMSWVTmgBjI0pxmCwccW9N7mrd22b3x+zMzu7Oyutula+7/P4sXbq\n3dndzz333HPOFZqmIZFIJJLUR+nuBkgkEomkY5CCLpFIJL0EKegSiUTSS5CCLpFIJL0EKegSiUTS\nS5CCLpFIJL0EKegSiUTSS5CCLpFIJL0EKegSiUTSS3B05c369OmjFRYWduUtJRKJJOVZvXp1qaZp\nfVs6rksFvbCwkFWrVnXlLSUSiSTlEULsS+Y46XKRSCSSXoIUdIlEIuklSEGXSCSSXkKX+tDt8Pv9\nlJSU0NjY2N1N6TWkpaUxePBgnE5ndzdFIpF0Id0u6CUlJWRmZlJYWIgQorubk/JomkZZWRklJSUM\nHz68u5sjkUi6kG53uTQ2NpKfny/FvIMQQpCfny9HPBLJCUi3CzogxbyDkc9TIjkx6RGC3hIVdT7K\napu6uxkSiUTSo0kJQa9q8FNe5+uUa5eVlVFcXExxcTH9+/dn0KBB5mufL7l7XnPNNWzbtq1V933z\nzTeZPn06EyZMoLi4mJ/85CetbvuaNWt4++23W32eRCLpnXT7pGgyCAGhTlrLOj8/n3Xr1gHwq1/9\nCq/Xy49+9KOoYzRNQ9M0FMW+//v73//eqnuuX7+eW265hTfffJMxY8YQDAZZsGBBq9u+Zs0aNm7c\nyPnnn9/qcyUSSe8jJSx0RQhCWicpegJ27tzJ+PHj+eY3v8mECRM4fPgwN9xwAzNmzGDChAncc889\n5rGnnXYa69atIxAIkJOTwx133MGUKVOYNWsWx44di7v2b3/7W+68807GjBkDgKqqfPe73wVgz549\nzJkzh8mTJ3PeeedRUlICwHPPPcfEiROZMmUKc+bMoaGhgXvuuYenn36a4uJiXnrppS54KhKJpCfT\noyz0u/+3ic2HquO2+wIhAqEQHlfrmzt+YBZ3fWlCm9qzdetW/vWvfzFjxgwA7r//fvLy8ggEAsyZ\nM4dLL72U8ePHR51TVVXFmWeeyf33389tt93GU089xR133BF1zMaNG/n5z39ue8+bbrqJ6667jm9+\n85ssWLCAW265hZdeeom7776bDz74gIKCAiorK0lPT+eXv/wlGzdu5I9//GOb3p9EIuldpISFjoCu\ntc91Ro4caYo5wLPPPsu0adOYNm0aW7ZsYfPmzXHnpKenM3fuXACmT5/O3r17W3XPFStW8PWvfx2A\nq666imXLlgEwe/ZsrrrqKp544glCoVAb35FEIunN9CgLPZElfbS6kaPVjUwalN2lIXkZGRnm3zt2\n7OBPf/oTK1euJCcnhyuuuMI21tvlcpl/q6pKIBCIO2bChAmsXr2aCROSHzk8/vjjrFixgjfeeINp\n06axdu3aVr4biUTS20kJC10Ja3hnTYwmQ3V1NZmZmWRlZXH48GEWLVrU5mvdfvvt3HvvvezcuROA\nYDDIY489BsDMmTN54YUXAPjPf/7DGWecAcDu3buZOXMm9957L7m5uRw8eJDMzExqamra+c4kEklv\noUdZ6IkwrPKQpqHSPUkz06ZNY/z48RQVFTFs2DBmz57d5mtNnTqV3//+93z1q181rfyLLroIgL/8\n5S9ce+21/OY3v6GgoMCMoLn11lvZs2cPmqbxhS98gYkTJ1JQUMCDDz7I1KlT+fnPf86ll17a/jcq\nkUhSFqF1YfTIjBkztNgFLrZs2cK4ceOaPa+8zkdJRT1F/TNxOdTObGKvIZnnKpFIUgMhxGpN02a0\ndJx0uUgkEkkvIUUEPeJykUgkEok9qSHoYRM9JE10iUQiSUhKCLoattCDUtAlEokkIakh6GELPShd\nLhKJRJKQ1BJ0aaFLJBJJQlJC0BUBAtEpgt4R5XMBnnrqKY4cOWK7T9M0HnjgAcaOHUtxcTEnnXQS\nTz/9dKvb+vLLL7N169ZWnyeRSE4MUiaxSFU6R9CTKZ+bDE899RTTpk2jf//+cfv+8pe/8P7777Nq\n1SoyMzOpqqritddea/U9Xn75ZRRFoaioqNXnSiSS3k9KWOhAWNC79p7//Oc/OfnkkykuLuamm24i\nFAoRCAS48sormTRpEhMnTuThhx/m+eefZ926dXzta1+ztezvu+8+HnvsMTIzMwHIzs7mqquuAmDx\n4sUUFxczadIkrr/+evPcH//4x4wfP57Jkyfzk5/8hGXLlvHWW29x6623Ulxc3OqiXxKJpPfTsyz0\nhXfAkQ22u4b6A4AAZyszRftPgrn3t7opGzdu5JVXXmH58uU4HA5uuOEGnnvuOUaOHElpaSkbNujt\nrKysJCcnhz//+c888sgjFBcXR12nvLwcv9/PsGHD4u5RX1/Ptddey9KlSxk5cqRZMveyyy7jrbfe\nYtOmTQghzHvMmzePSy+9lIsvvrjV70cikfR+UsNC14IodK15vmTJEj777DNmzJhBcXExS5cuZdeu\nXYwaNYpt27Zx8803s2jRIrKzs9t8jy1btjBmzBhGjhwJ6OVyP/zwQ/Ly8lAUheuvv55XXnklquqj\nRCKRJKJnWeiJLOnSnSj+JvaKoYztn9klTdE0jWuvvZZ77703bt/nn3/OwoUL+ctf/sJ///vfZpeP\ny8vLw+l0sn//foYOHZrUvZ1OJ6tWreKdd97hxRdf5K9//SuLFy9u83uRSCQnBqlhoadl49R8qKGm\nLrvlueeeywsvvEBpaSmgR8Ps37+f48ePo2kal112Gffccw9r1qwBaLaU7R133MFNN91k7q+urubf\n//4348aNY8eOHezevRvQy+WeeeaZ1NTUUF1dzQUXXMAf/vAHs/a5LJcrkUiao2dZ6IlIy4bqErxa\nHZqW3yWLXEyaNIm77rqLc889l1AohNPp5LHHHkNVVb797W+jaRpCCH77298CcM0113DdddeRnp7O\nypUroxa6+MEPfkBdXR3Tp0/H5XLhdDq5/fbb8Xg8PPnkk1xyySUEg0FOOeUUrr/+eo4dO8Yll1xC\nU1MToVCIhx56CIBvfOMb3Hjjjfz+97/n1VdfpbCwsNOfg0QiSR1aLJ8rhHgKuAA4pmnaxPC2POB5\noBDYC3xV07SKlm7W1vK5AP4jW/AFNdIGjDMTjSSJkeVzJZLeQ0eWz/0HcH7MtjuAdzVNGw28G37d\nqfidmXhoIhRIPtlHIpFITiRaFHRN0z4EymM2XwT8M/z3P4FOj6MLuLIQAmiq7uxbSSQSSUrS1knR\nAk3TDof/PgIUtKcRSa2a5EjHpzkQjVXtudUJQVeuQiWRSHoO7Y5y0XT1SKggQogbhBCrhBCrjh8/\nHrc/LS2NsrKyFkVIUQTVeFD9tRAKtrfZvRZN0ygrKyMtLa27myKRSLqYtka5HBVCDNA07bAQYgBw\nLNGBmqYtABaAPikau3/w4MGUlJRgJ/ZWfIEQVTU19BVVUBoEp6eNTe/9pKWlMXjw4O5uhkQi6WLa\nKuivA98C7g//3/pKU2GcTifDhw9v8bh9ZXV85cElbM78Pq7xF8CX/9rWW0okEkmvpEWXixDiWeAT\nYKwQokQI8W10IT9PCLEDODf8ulPJcDsI4OBAn9Ng+9sQDHT2LSUSiSSlaNFC1zTtGwl2ndPBbWkW\nr1tv6o7cMxh5+C04sAIKZ3dlEyQSiaRHkxqp/4DboeBQBFszTgbVBdve6u4mSSQSSY8iZQRdCIHL\noVCrpcPwM2DrmyDD8yQSicQkZQQdwKkq+IMhGDsPKvbAsS3d3SSJRCLpMaScoPuCmi7oANve7N4G\nSSQSSQ8ipQTdpQrdQs8aAIOmw1bpR5dIJBKDlBJ0p0MhYCwsOnYeHFoD1YebP0kikUhOEFJL0FUF\nfzA8EVo0X/9fRrtIJBIJkIKC7jMs9L5FkDtcCrpEIpGESSlBN33oAELoVvqeD6FJLssmkUgkKSXo\nZtiiwdh5EPTBziXd1yiJRCLpIaSeoAcsyURDToH0PBntIpFIJKSaoDssPnQA1QFjzocdiyDo776G\nSSQSSQ8gpQQ9yoduUDQPGqtg3/LuaZREIpH0EFJK0ON86AAjzwZHmox2kUgkJzwpKOgxBblcGTDi\nLN2PLot1SSSSE5iUE3RfIBS/Y+w8qNoPRzZ0faMkEomkh5BSgu5yCAIhO0GfCwjpdpFIJCc0KSXo\nti4XAG8/GHKyXiNdIpFITlBST9DtXC6gu12OfA6VB7q2URKJRNJDSClBdzkUGgNB+51msa6FXdcg\niUQi6UGklKB73Q78QY0mO1HvMxryR8tFLyQSyQlLSgl6ZpoDgJrGgP0BRfNg70fQUNmFrZJIJJKe\nQe8S9LHzIRSQxbokEskJSWoJutsJQE1jgrotg2dARl8Z7SKRSE5IUkvQW7LQFVUv1rVzCQR8Xdgy\niUQi6X5STNBbsNBBj3Zpqoa9y7qoVRKJRNIzSDFB1y306kQWOuh1XZwemTUqkUhOOFJS0GubE3Rn\nul6BURbrkkgkJxgpJehedws+dIOx86DmEBxa2wWtkkgkkp5BSgm6Q1XwuNTmfeigT4wKRbpdJBLJ\nCUVKCTroVnqLFnpGPgydJdcalUgkJxQpJ+iZaQ5qmpJYP3TsPDi2CSr2dnqbJBKJpCeQgoLubNlC\nB70MAEgrXSKRnDCkoKAn4XIByBsBfcdJP7pEIjlhSElBr21KQtBBt9L3LYf68s5tlEQikfQAUk7Q\n3Q7VvnyuHWPngxaEHYs7t1ESiUTSA2iXoAshbhVCbBJCbBRCPCuESOuohiXClWihaDsGTgVvf1ms\nSyKRnBC0WdCFEIOAm4EZmqZNBFTg6x3VsES4HK0QdEXRF5De+S74Gzu3YRKJRNLNtNfl4gDShRAO\nwAMcan+TmqdVgg56sS5/Hez5sPMaJZFIJD2ANgu6pmkHgd8B+4HDQJWmaXHOaiHEDUKIVUKIVceP\nH297S8O4HAq+YCsEffgZ4PLKpekkEkmvpz0ul1zgImA4MBDIEEJcEXucpmkLNE2boWnajL59+7a9\npWHcDgV/UCMUSrLwlsMNo87RF48OtaIjkEgkkhSjPS6Xc4E9mqYd1zTND7wMnNoxzUqMy6E3uVVW\netEFUHsUDq7upFZJJBJJ99MeQd8PzBRCeIQQAjgH2NIxzUqMS9Wb3NQaP/ro80Co0u0ikUh6Ne3x\noa8AXgLWABvC11rQQe1KiNuw0AMhqhv9yble0nOhcLYsAyCRSHo17Ypy0TTtLk3TijRNm6hp2pWa\npjV1VMMS4XaoAFTU+5j8q8Xc//bW5E4cOx9Kt0HZrk5snUQikXQfKZcpavjQS2v1vuP1dUlGSprF\nuqTbRSKR9E5SVtD9Qd3VEkp2mbmcoVAwSRbrkkgkvZbUE/TwpKg/PCnaqlVDi+bBgRVQV9rxDZNI\nJJJuJuUE3e3Um7z5cDXQynWgx84DLQTb3+6ElkkkEkn3knKC7lD0Jj/0zvbwllYo+oApkDVYRrtI\nJJJeScoJ+uTB2QCkhS31VlnoQujFuna9B776TmidRCKRdB8pJ+gZbgcnD88jJ90FtNKHDrofPdAA\nuz/o6KZJJBJJt5Jygg7gVIW5yIXWKhMdGHYauLNk1qhEIul1pKSgOxTFTP1PtkZX5GQXjP4CbHsb\nQkmufCSRSCQpQEoKum6hh8MWW2uhg+52qS+FAys7uGUSiUTSfaSkoDsUhWDYNG+DnMOo80BxSreL\nRCLpVaSmoKsi8qItip6WBcNP18MX22LhSyQSSQ8kJQXdqUaa3WY5HjsPyndB6faWj5VIJJIUICUF\n3aFELPQ2+dBBF3SQxbokEkmvITUFvSMs9OxBMKBYFuuSSCS9hpQUdKfFh550tUU7iuZDySqoOdoB\nrZJIJJLuJSUF3ajnAu2c0xw7D9Bg+8J2t0kikUi6m5QUdKuF3q4YlYIJep10WaxLIpH0AlJS0Nsd\ntmgghL403e4PoKm2vc2SSCSSbiU1Bd3qcmmfja5njQab9AqMEolEksKkpKBHuVzamxc09FRIy5HR\nLhKJJOVJSUHvkLBFA9UBY87XVzEKBtp7NYlEIuk2UlPQOyKxyErRPGiogP2ftP9aEolE0k2kpKC7\nHJFmt7p8rh0jzwHVLd0uEokkpUlJQVeEaPmg1uD2wogz9TIAsliXRCJJUVJS0Hcfr+v4i46dB5X7\n4Njmjr+2RCKRdAEpKegnFeZ2/EXHztX/l0lGEokkRUlJQZ87aUDU60Aw1P6LZvaHQTPkohcSiSRl\nSUlBj6W6sYPCDYvmwaG1UH2oY64nkUgkXUivEPTKel/HXGjsfP1/Ge0ikUhSkF4h6C+sKumYC/Ud\nC3kjpB9dIpGkJL1C0B9buqtjLiSEHu2y50NorO6Ya0okEkkX0SsEvV+mu+MuVjQfQn7YuaTjrimR\nSCRdQMoL+swReQzISe+4Cw45BTz50o8ukUhSjpQVdMMqdzlUQh2S/x9GUWHMXNi+GIL+jruuRCKR\ndDLtEnQhRI4Q4iUhxFYhxBYhxKyOalhL/Pe7p/LApZNxqQqBjhR00MMXm6pg70cde12JRCLpRNpr\nof8JeFvTtCJgCrCl/U1KjiF5Hr46YwgORXSshQ4wYg440qXbRSKRpBRtFnQhRDZwBvAkgKZpPk3T\nKjuqYcmiqoJAqAMyRa24PDByjh6+KIt1SSSSFKE9Fvpw4DjwdyHEWiHEE0KIjA5qV9KoQhDsaAsd\n9PDF6hI48nnHX1sikUg6gfYIugOYBvxV07SpQB1wR+xBQogbhBCrhBCrjh8/3o7bJWiEIgh2hhU9\n5nxAyCQjiUSSMrRH0EuAEk3TVoRfv4Qu8FFomrZA07QZmqbN6Nu3bztuZ4+qCILBThB0b189hFEW\n65JIJClCmwVd07QjwAEhxNjwpnOALi8mriqi46NcDIrmwZENULm/c64vkUgkHUh7o1x+ADwthPgc\nKAbua3+TWoeqCEKdNXFpFuta2DnXl0gkkg6kXYKuadq6sDtlsqZpF2uaVtFRDUsWR2da6H1GQZ8x\n+tJ0EolE0sNJ2UxRA6WzfOgGY+fBvo+hocsjMiUSiaRVpLygd1qUi0HRfAgFYMc7nXcPiUQi6QBS\nXtBVpf2p/8dqGimvS7BIxqAZkNGPmvWvUXjHm6zZ3+VeJYlEIkmKXiDotDux6OT/e5dp9yawwBUF\nxs7Fvfc9XPh5Y/3hdt1LIpFIOoteIOgKwZCG1sluF1ewjlnK5o4vMyCRSCQdRMoLukMRAHRWoAsA\nw8/Er6ZznrIKf2dOwEokEkk7SHlBV8OC3qmWszONI31nc666hlAw0Hn3kUgkknbQawQ9FIKbn13L\n5Y9/anvc/rJ6bn52LfW+tgny4f5n019U0L9ua5vbKpFIJJ1Jygu6w2Khv77+EMt3lQHgD4ai/Opf\nW/AJr68/xJbDNQmvdeerG9ly2H5x6NKBZxHQFMZVy0UvJBJJzyTlBV0RuqDf91ZkbY3S2iZG/3wh\n/1y+19x2uKoRAF8gsWvm35/u4+q/r7Tdp6XlsUoby6Tajzug1RKJRNLxpLygO1Rd0J9decDctr+8\nHoBX1h2KO74ll8vR6ibqmuKPURXBO8HpDPLtgfI97WmypAto9Af5dHdZdzdDIulSUl7QDQvdSiAc\nieJQBOsOVHLP/zZjHFbvC7Z4zQcXbbPZqrE4NF3/Uy5N1+N5cNE2vr7gUzYfsnehSSS9kZQX9EAw\n3oXiD29TFcFX//YJT328x1xJriEJQa9u8MdtC2lwQCvggKNQLnqRAhyt1l1sieZEJJLeSMoLer0/\nXqAr63VBdqoCNcaCf339oSh/ux12keZGNupnaTNh/3KoL29bgyVdwpA8DwD7wu43ieREIOUF3c7i\nPl6jW2eqophRMAYf7SxlwYe7m72mXdapUXN9hWsWaCHYvqitTe6xNPqDbQ7r7Gk4Vf2rXW8zHyKR\n9FZSXtDtfOKltXqhLaciUNV4H3tbMAR9qzIKMgf2yqXp5vzuA8b/snd0VE3hkVsgpPHP5XubjW6y\ncqiygVfWlnRm0ySSTsPR3Q1oL3aCfrymCdAjYGIt9GSwd7no/wdCGoydC+ufA38DONNbff2eihHa\n2RtoDAv6858doMEfpLLezw/PHd3ieZc//il7y+qZO3EAaU61s5spkXQoKW+hn1SYG7ft+VV6CKND\nUcxM0lhaW8zLsND9wZC+1qi/DnYvbWVrJV1FU9gibwgLe3ldU1LnHQlPpra3gqdE0h2kvKB/eeog\nHv7GVNt9GhoOxf4ttraGeih8fFWDHwpPB1dmr3S7QO8Qs8aYyfJkP29BOPNYFmGTpCApL+hCCAZm\np5mvf3fZFPNvXyBkJh4BWI315n6wdsa7sSpSRb0fHG4YfS5se1svItPLsAvbtEPTOrlscTto9Ed/\nLq0VaH8v/FwlvZ+UF3QAlyPyNpwWAW8KhKJ86F+ZNtj822cTv27w+vpDvLwmemLMMPDMybWx86Hu\nGBxc1Z6m90gqkxT04T99i5+9srGTW9M2GgNttNDDXxdpoUtSkV4h6EaIGhDlYmkKhKL2zSjM5YLJ\nAwD7hCQrt72wXnevhAlZBKHeF4DR54HigK29w+1iJOJA8v5mgGdX7u+M5rSbplgLPUmL2+j+/S18\nPySSnkivEPTmLHS3ZZ/LoXDqyD5AxGJrzmVgjcm2+pXLan2QngPDZsO2tyipqOf8P37IzmOJKzn2\nVKrq/VTV+7nk0eXmth1Ha1s8r6tdLcdrmthQUpX08W230I3qndJCl6QevUPQVaugR/5ef6CS7RZx\nqm0MmD51w3XSnC5Zk5ZCmtVCD28vmg+l29m2aS1bj9Twk/9uaNf76A5O+c0S5vz+Aw5WNpjbNiVR\n/6Sr9W7un5bxpUeSL10c70NvncXd2uMlkp5ArxB0qxXuiEkkavAHOXdcAbedN4YLiweZFvzd/9tM\noz9oTnYa/OOak7j3oglAdIy7VdCbDOtv7FwAhh7/AIDV+yo65g21g0AwxCWPfsyH24/H7dtfVs8b\nn0cqUJZU1NPoD1Fe54s6zup+SURXuyRKa5N3A0F8+5KJ3Fm1N1LOIdWXGgyGNKobk5sLkfQeeoWg\nJ/KhG2SlO7j5nNFkpzvN/Uu2HOU/n+6L+6FPHZpLYZ8MIBLDDNEWqWn95QyF/pPoU7LE3NfZlt35\nf/yQL/7hw4T7y+t9rNlfya3Pr4vbd8Gfl/H9Z9aaryvqIj/4AeFIoYIsd7MTxgY9PbQx9nNoyYWy\neNMRLn3sE2rDpQJSfTHwO1/byORfLe5xI41dx2spvONN1uzvfuOnN9IrBN0V5SePTyRyO+xdMvW+\nYJTlDbr7xuPSMwQve+wTc7tVwJqs/tmx88kpW0s+un/3UGXEuv3zuzu44okVSb+Pel+gxS/61iM1\nbDua2FdvxFHbiXJ1Y1iswvusotXoDzJ5cDZDcj1xafKfl1Tyv/XRteW7y8ecbEcS276Wolb2xxTx\nSnUL/fnP9OS6njYXYIwcX1t7sJtb0jvpVYI+ZXC2rYXudkRSuK2TpoFgKM4X7FAF6c5IRQRj8s8a\n5fL/XljPpkPhCbqi+Qg0zlZ1y/d4bUTQf//Odj7aWdpi+49UNbLpUBW3PLeOSx5dHucCaQ2GSDfn\nEjGyKK0/9tqmAG6HgsuhmPsNLnzkY37w7FpCIY3CO97kNwu3dJvll2xNllgBt7O4//7xHh5arNe+\nj62r39Ms29ZidHwdMZLaeayGrz72ie3CL63FeMo9q5tpO1c9tTJqZbTuplcIulNVeO6Gmfzz2pPj\nfOgA/bLc5t8Oi4X+0uoSfvisLsQ3njGCv105HafFQoeI28X6uzhW08T8hz9i+c5S6D+JuvSBfEFZ\nDUTqyLSGG/+9ivkPf8SyHbr4x2Y5tgZDyJqzMI3rW0XfH9RwO1TcDiWhaBrJNk8s29NtLpekBT0J\nC/3u/23m4fd2EgxpxFaI6GmWbVvpiPdx/8KtrNxbbq7X2x4UY1H3HpqQ1hqWbD7Kh9uPc9frm7q7\nKSa9QtABZo7IJ8fjinKpGPTPimSSWi30Q1WNvLv1mH5MdhpfnNAfIErQjVj02MlTgMufWAFCcKjg\nLE5TNpBGU1KC/p9P97H1SDVLtx+n8I432RxehMHoPGwWYUoaQ6SDIY2PdtiPDgwLPFaUDQs9kWga\n20Oa1qxQaJrGFU+sYMnmo61ufywHKxsoqYi4Q5Lx7wMEQ8370KvqI/MHO47VmOGKBr0lDj3UIR1T\nx4mw8Zx7gZ5z3b/ikwrrmgKc99BS1h2o7IYW9SJBN7Crrhgt6PZv2VrEK91G0JuLuy7pN4d04eMM\n5XOOVscLeuy5v3h1I+f/cRkvhP2csdZ0e36DVuG64slo/73xFg0LPdZqTXOquBxq9ByBBaOdmhZ9\nbuyIwhcM8dHOUtsvfGuZff97nPbb96OunQwtuVzK6yNurdIaX1wn2lsyRTvCQjeeTUfkHhjfwVQf\nACV6FusPVLLjWC33L2x+EZ3OotcJup1gF1hqvSQqp2u10DyuiA/dsOSaczEcyplGleZhvmutaW1b\nWWoJIbR+EXI8TtvrBROIibUNiSyv5ixLo9MyLPTYY90OpVmXi1XorQK5PWaSNlm3SFswrv3Z3nLe\n/PxwwuNihexgRUP0fst7r/cF4iz0VI9yMegI15hiCnq7L2VO2vfUGkDJUt1gP59gvCtBO4bZ7eDE\nEPRkLHTLD1pVBI9fNQNo3uViEMDB+6FizlbWsmG/7md89IOd5v6r//4ZwZDGQ4u3RblkEgm6tTDU\n+1uP8cwKPb3eGkYZmwlpntuMZWkIumFRx7lcnPaTogZ1TVZBj5y77UjbBd0fDPGr1zdxuKqh5YPD\n1/7dom1c9tgnfO+ZNSzfZe9WihXkino/Ryz13q3tb/AH435+3RHlomkaL6w6kNS6t8nS3Pc2WYwJ\n4454IhELPbUF3ZqIZ8V4W+1xm7aHXifodpOiXnfE4k7scol+PW1oDqoi+CBsXWsapDkTl+J9JziD\nzFAVI5oGVn3cAAAgAElEQVQ2A/qEq5Xlu0p5+L2d/PTlSDapdSQQdT2LmFzzj8/42Sv6OdZSBHYL\ne+jnNmOhixgLPc6H3vykqDXKwdrGmsZoayVRh2DHsh3H+cfyvdz9+uakjvcHQzzyfqSzvPxx+7BQ\nO1eDGZlEdPvrmoLxLpdusNA3HKzi9pc+5/+9GJ9DYPDoBzv1yfgkSTTaaw2iA0VYaaUPfUNJFff8\nb3OPs+gb/IksdL2dsVFTXUWvE3Rngvrn5v4ES9LFDrnzvW6umjWM51bu50B5PcGQFrfgNOhWVTAU\nYmloMkHh4ByxilBIY1z/rKjjjGJRNRZRjBVCg0RiYrXcEllxzVmWSoyFHiv+poWeoFOwCnp0XH70\n8a2x0I1nYNcR25FMZxEMabaCsdUykrA+Y7t1VLvDQjdE4K0NRxIK2ANvb9Mn45OkIzom47fRgfOr\nSV/ror98xFMf74kr5dDdJJpjCUkLvWNx2iQWWelv8adbsRPr88YVENL04VUwpJmCaMUXDBEIadTi\n4UD2DM5TVlHvC8StlGQMfa0/1LIE6ex2XxZN06JcHgkt9GZ+wI4YH3qsFetWFdyqbqHbCUqtRdCt\nbqHYSdTmJi41TeNfn+zlUGUDoZDGD5/TrVGv28EvX9tI4R3NV69sSiKk0+4ZpDmVqGFylMvFF4xz\nP3XHpKi1TTuOtVwgLRk6xoceFvQ2Xsuubn6yFrdxy+4YMdU2BTiWoAxGR7iyOoN2C7oQQhVCrBVC\nvNERDWov1sSiN28+jSW3nRG1P5Gbw26pOmc4Yckf1AXO7pj6pqA5rN3X9yyGK0e5+O6neD0ms9L4\nYVl/E4nqk9h9eRv8wahhXlldE2W1TWiaxpMf7THdCclMiv7ohfUs31UaJ1r5Xjfu8DqadqJcl6D6\nZKz1FFu61sq+snp++dombn1+XVStkQy3g399si/heQaJOjIrdiKW4XLwya4yM7IoyuXiC8ZZ5O0J\nW/zjku0tdkx2WNtdlWRN+hav2RGhhuH/k40wiuXiR5cz5e7FQKRTaG2rumPE9JVHl3Pyfe/a7kvU\nURodVeyIv6voCAv9h0D3xOjYYHWpTBiYzah+mXHHXH1qId84eUjUNjvr2/C3+4Mhgppm6xer9wdN\ny+pwwRwAM8nISkTQLRZ6goxQO+uwpjEQJWYr95Qz/ddLePKjPdz7xmbmP/xRuK3R51q/eEb7a5oC\nfP+ZtXEdx9B8j1m50s61UWsZIVgFL95Cj7yubvTzajjN+2h1I9f84zNAH5JWWmLBMyyhos1ZgnvL\n6uK2xVp7dj/+dJfKntI6bv/v50B0p9ngC8S5n5IR9JdWl8RZcBsPVvHHJTtaPNcO63ejPROj1o6y\nI0Yaxte+rdFL6w9URspOJFG22o72dLB/fncHq/eVt3jcE8t2mx0PYJbYsGtrYkHX/+8mj0v7BF0I\nMRiYDzzRMc1pP8n0jL+6cAK/uWQyV84cZm6zi2Z0mqV2NZr8IdtjXl93iGBIw6EI/N4BrA+N4Dw1\nXtAjSTmRbaUJkpDsJvRqGv1Rgm5UdnxzQyR0LxjS4n7Axn1LKuqjBGNIbnrcscPyPLjDE792VnZz\nPvSXVpew8WCV+drgpy9v4Jbn17HlcDU/f2UDe0p1QR6U46HCEguuWkZWVnfO4Nz0qDb8+s1426E2\nJiXd7sdmTRaD6Gdc7wvGPfOWfPXldT5+9OJ6rv77Z1HbL/hzpMTvna9uNEsLJIP182iwcS0l6/Iw\nPgfoWJdLR4Sj2o1Uk6E99/79O9v5yl8/afG4X7+5haoGf5yA24UoJpogNjqeVPWh/xG4HehRMxbf\nmjWMp687pcXj7r14ovm3nQ/dKOp1rKaRF1eXUFobb1F/XlJJIKS7Y1yqYHFwBlOVnfxAfZnL1XeZ\nq6xglrIJV/kWCijHEYqI+OEE/jnjS2EtY1vTGIiy2oy2WIfmD7y9Nc7qNiZAT/vt+5TW+jh3XD9O\nH90HIUTcsQNz0unr1cskHKiILlYFMVEuUS6XID96cb0pZlYxNEIFqxv8+CyC1TfTHWWhW9tiFbZk\nBCnWPWEX6ZPujBH0YLSgx1qAdhOlVox2HWom3PLfn+7j4fd2JtwfS0sW+r8+2ZvUdezCM9/ZfDSp\nssh2mBZ6O7Nnj1Y38sE2PTO7tREzbU2QakuHVlrri/pOldqs4JVo5GOMDrvLQrd3KCeBEOIC4Jim\naauFEGc1c9wNwA0AQ4cObevtWsXdF01s+aAwqiISTngaLpdjNtmfBrVNAYIhfe1Sh6LwRmgm39be\n4v85X4o+8GP4UhpQBg1uFxV4qdQyqdC84b+9VJBJpeal764DhLRCvvPPrRQKLxVaJpV1TaYPO8Ol\nUtukf+GsCzp/sO04owuiXUyHqxr5zn8iIwaHooBDUBrwxbkm0pwq04flArB6bwXThubGvVeDf3y8\n1/z75TXRlfOs1pQxyqmoj7Z8NDQqGyIdpLUt1h9LMj/k2ExV45w0p0JjeGTltgj6Ux/t4Z439DDJ\nDJdKZYMv7gdqTEA3+oMcqmygf3Za1PyL0QFYz2vJjaBpGk0Bvf78gg9386Mvjo0KqY2NjY/lV/9L\nLrTT2qGGNI1gSOP6f61ieJ8M3v/RWUldw4qRJONvp4U+90/LzMJzyei59Xk253KpawqQ4baXskRZ\nz81x0v8tiXpdVutjZN/oYxJ1SIZh0l0+9DYLOjAbuFAIMQ9IA7KEEP/RNO0K60Gapi0AFgDMmDGj\nx00Nq0IQxN4/bgh6cxNLVQ1+00J3OhT2af2Z1rQANz7OGuJkb8kBckUtV07O5KMN2xmb6cdXW0Yu\nNeSIWvootfTX9pOj1JJDLarQYCWwEl6J1BQj9LyCz5nFTFc6jY4syuq9HHNmUNPkpVT1UokXt68P\neUePMk4cD3cUmTz87g5W7In4Dx2qQFUFTYH4yA6Afllp5GW4bH3Vy3dGijMttVlAwyBa0PVn+J3/\nrCYrzSJeQS3KQo8qFGax1pOxsBr9IR5buosrZg7D63aY57hUXdBVRUSVUP7zexEfd99MNxV1/qh7\nQqTzOv2B981ksL33z49rr3VkcfFfPm62nY9+sIsHF22jj9dNaW0TXxhfwKmj+pj7rS6V9vjQrZFA\ngaBmiprh7mqJJZuPsmJPGVfPHs6gnHRTWNtroVuriGpJTItaO6ZELpfV+8r5yl8/4e9Xn8Scon5x\n++3CHf+2dBfnT+zPsPyMZJrNsZr4kU0iQ8MwTAz78OZn17JqbznLf3pOUvdqL20WdE3Tfgr8FCBs\nof8oVsxTAVUREIxPLIKIGDX346pq8Os+dFXBabHym3AxbWIRQW9/Vu8r58WGHD4IDmGEM4PdgTr6\neF2U1vooyHKb9V8EITKp548XDmWQq4HfvLycXGrJFbWcOUQlV9Sxr+QAwxxN5DWWMULZRy41ZKjh\nEUQDsBLOtnQEDTtd/NIdGQ1kHO5LnSOLnQ0uhh0YzCVKI9VkUEcaHBwA7izGeuqorqrUzShLR9dc\nHXYr1h+itdRCdaM1MSlEbfh1ptsRJehRFrple7pTtbVcX1t3kMeX7WFfWT2/unC8eS23U4XGAIqI\nFnRrO/pmujlY0WBjoesTpYmKrRkC0+gPcd9bW/jZvHGsb2HN0xdW6RE2RnRTrJ++JQs9WazXDYY0\ncz4kGaNR0zSzBs+SLcd4/0dnmQlobfFjJ/L7JxOFaP3dJbLQPwlXgFyxpzyBoEc/x/I6H79ZuJWn\nV+znw9vntNwI4J7/beb7z6xl491fNEdUVkNjRJ9IxxBpp/6wjWi33y3axo++ODap+7WH9ljovQKH\nKsBvn9llRHw0VwfaaqE7YnqF7HQng3PT+XS3xgfbjkddyygAluFyAPoPXEOhGi/V6UNpUBU+CEX8\ns/vS+jG6IJOnDuzh7CH9eHvTEXOfGx/Z1DExN8Cl4zy8/ukmckUNudSSI2rN0UCuqGVg02489dXM\nDFaj7g1xlsvS4Mf/D4BnAWqAuwW4vHzqdlCrpVNLGnVaOnWkUUs6tcbf4f/rSINNfvIONzJNHKaW\ndLZvO042+v6A5evmD2nU+gK4HArpLjWmlK+9hd4vy82+Mt23n+l2mElaZeH5hGdX7uf1dQd59oaZ\nQGQiVBd01faafbxuNhysMjuOr580hHUHKqltCsQJ7s5jNWbUlHXfgg9387N542iJ2CzlWNG2tuvB\nRdt4YdUBlv44OdGxEiXommZa1sk4Ae6wrItr9MXGs2lNBrBBXYK5iGTCKWfdHwkZTGQRG4ECjy3d\nxW3njYla7AbiBd1wlewvr+eR93bw/bNHt9iOY+FOvbzWFyfow/I9ZKbrJTye+mgPHyfI4n3k/Z2p\nI+iapn0AfNAR1+pqjEzFrPT4uipGklJ9Amspx+OkusGPP6D70GOzULPSnTgUEfWlrg/7Zj3hRTQy\nw26IkX0z2HVcHxIHQhrlddHDvNqmAPW+AB6XSlZ69MfWhItjuHivAt5bDnBywvf7jeKhuB0Kr6zZ\nz1XT8ln02WZcgRq8opHnvjURmmp4cfkWjpWW8r1T+1NbU8nSz7aTIRrw0kiGaCCPGjJowKs0kEEj\nbmH50b64gLOJHiUYNGpOakmnTkvDuS0Ln+rhNIeDpqAH7/4cRjqC1JGOZ9UW/Nm5qOlZnK5to0Kk\nUUsa/UQf6ghRgZdRBTms3a+XKLU+3zpfkM/26hFAQ/M87Curp8EfjLLQrfTNdNPoD1HTFKCP18X9\nX5nMNxZ8yrIdpXELjZz70Icsu30OGW4Hv317a8JnnAiXmpzYGBidF7RuwQ2r3zgYClksdMHOY7U8\n+v5OHrh0cpwBAvB8eBQBMCTPA0Q62JYmiu2wJsNZsbO4j1U3cvNza3nk8mn08bqj3CWJ/PfWyK+F\nGw9zUfGgqP3Wa+gJTpF9v1u8PSlBN7DafFa3nlGq2ZiXAb0z7I5yBSe8hT6mwMv2o7VMGZwTty/W\n5ZLjcZp+379dOZ09pXXcv3ArJRUNug/d8gPJz3Axa0Q+60sqo75ENTEW+oDsdK47fQSnDM8zkxju\nem0jF0weGNWWRn+Qel8Qj1MlM82+qFcyOFXd/dAYgFqRwWFlACMGj2bK8DwYq1uZuw5u5ckDuxnR\ndyrfXbwGOKv5axLQBV408tEPZ/Dqim28smIbGTSQIRrx0oDX8neGaGS0A9yhevqKKjzaEbJrmpio\n1uEVjbD8ZfPaj6mAYVzXos/WALXHvRxzeSknC3EwnzkONxVkUqZl4diwnjlKiNMyxrJX1FCuZeFO\nkEFsRPUcr2kyk9I+2a0P4//24a644yvqffxm4RZzMZLW4HQ0b6HbWaGh8IR9rFGxaNMRs35/LNaQ\nU6sPXQDX/uMz9pfXc9OcUYzq5222vb5AiAZL0lWZTZRXSyTqBOwSxJ76eC+f7i7n+c8O8L05o6Lb\nkqBDs7pl7KJ4rEXs/EGt1VEvTlXY5jUYna/LoWDXNCFo18pjbeWEF/TnbphFgz9omwVq+H/fCy+C\n8ejl08w6GqeOzDdL636yu4zCfI8p6H28blb94lwgcW0Zwx3gcih8aUq0eNf5glGWklMV1PuCNPiC\npLtU06pvCw4lXCI3GCIQ1HCogte+NzvqmKL+mfiDGt99ek3c+TeeMYK/fbjbfH3vRRO487VNVJJJ\npZYJ/SdywOtmaci+xIJJaeReIU1jZF8vCzceQRDinvMLeeTtdRSk+RG+Wq6YmscXRnmpqijn8XfX\nk0cN43L8+KuPkUsNg/xHOUutII9qXCIIR+FbLmArfDs8UghsdHGzW48YKtMyTfGfcWQkR9UG0sv6\nkYMXjg5giKuGQz4P//l0v23TrYtrWzF+/D85vyjOgj9a3cj6mEUPYifs7PzNDf4gGW5HnNvvF69u\nTCzosT70QMSHbqydmqimkZXlu8oY98u3mT0qH4DjCTKbmyPRXEC1TSasYdFuP1qTVLIYRI/O7OY7\nrKOgQCjU6hICuR6X6XKJmucJRQS9vinITptSDYer2hYm2h5OeEHPy3Al3BcbemQdojpVJcpNo/vQ\n9eOtE4F2HQVEC3pLZKe7qPcFwy4XR1SoW2txqgK3U0XT9DhpOyYOyk54/qTBkX3Lbp/DgOw07nwt\nsgRXTaM/ztc6bkAWW2zqxIOe8t/oj/wgNBTufHs/kMfAfrpb5ezcMWRPHc2xozX8e3EBALNz8/m4\nTLekx+TpoyzQ8NJAcX6Q2vKj3H1eAf9+dw251HD2UIX9B0rIE9XkiRoGc5w8pYbsHYuY5AQM78Zf\n72SZAqRBpZZBuZZJOVlmR9B/5TK+WO1noOKgjEx9O1k01VUyJCedCYNzKMz3xL1PY9FmK/e+sZmv\nnTTE/DwNkXjoq1O47YX1gG7J2gm6Xd6EgS/Gh25Y6FZRbE0qvdHxxArmZ3vLKav1MbrAy8i+9tZ+\noqJatoIe/v+1dYf44TnRrpBEk6JWCz22fRV1PkrKI/NQ/mB84p1BovLNeRkRQbeOoIzO16kqBLUA\n5z60NOo8gTAFffLg7LhFyDuLE17QW4NTFcwelc/HO8twKCKqnnlIi/hIrZUDreI+f/IAc1GG9HBM\ns1Xvn7p6Btf+I36Vn+x0B5X1eqZoukuN8wf/8JzR/Ond5NLN05xqnC83lpF9M+ib6ba1eMZY4tzz\nva44P+z3n1nL2P6RY9KdKs9efwrvbD7Kj1/6PO56GeEww02H4gXf8JEbz9M6sZmTHumII35aQS0e\nNje6KNcyqR82k5eCelvSho/iz3vik3xe++7JXPfXxeSJaqbkBXlg7kCoL+Oh1z41xT+PagaJUiYp\nu+mz8WOuDvkh1g548BYW4qRhVw7K0T7826mao4D6JRs4+H4p5yhZlGnZlJJNqZZFI26++finvPq9\n2QghTJGwukIMC7M2xhedwE4AYn3omm3Wr2Gp+gIhTvq/Jdxz0QQujBkpGhjiW1rbZLqAjlY3ctlj\nn5jtXXLbmYC+8PbEQdmcVJgX1f5Y7GrVWEU7duWvRIJudakci/m+nnLfu1GumkAw3kLfeLCKDQer\nEsaV53sjH7Sdhe52KLYjq5CmmZ3EqH7eqEqfnYkU9FbgVBUWXDmD/eX1OFSFbIuF3ugPmsJjtcpV\ni7h/afJAU9A9zkgEhsGsEZGYZCvZ6U4OVTZS7wvSx+uKsupvO28MN58zmu+fPYrRP1/Y4nvwuFQz\nvR/gipnxyV5CCK6cOYyH3tlubrv61EJuPHNElP/ertDZ0u3Ho2b605wKOR6XbUgZ6Ik9jS3EXBvP\n01qP/vJThtI/O40nP9pDjaV2SabbYfourW6FtJhMUYO+2V6Ok8NxLYe09GyYeBoAj7wy0ExPv/OC\n8dwbnvC698Lx/Pb1VeSKGvKp4aopXpZv2MbPzuzHG59uYFymn+GeBjzlBxjMcfKVGjwfLeK3NtMe\ntVoapceyOfC7vmypTqNo4BBudcDQnXuYpxylVMsmcHQgftcwth+OHtIbiXB1TQF++domfnD2KArD\n4XNNgZCZVKX70G0EPWypVtbrWZH3vrGFeZMGAHDVrGHsK6s3cw2MzFN/UKOqwU9uhiuq9LP1+d8d\nTn7ae/98bnthXcKQ3zpfEF8gxDMr9jGmfyZDcj1ReQlldbGCHhHNYzWN9PW6EUJE+cRjDZBYv3sg\nFL8WrpHdHDsiMMj1RATdLoPZpSq2cx+BkMbR6kZURTA0z4MvECIQDNlORHckUtBbgVNVyHA7GDdA\nr3UeK+jGMNgq6FYL3WpZG5OiVrdOIvdMjsdFgz9InS/AUJcnStDnTuxvtu2SaYPisjaH98mISihJ\nd6lRP8bzxtv7YWPbMn1YLgOy022PjcX6BTfmFaw/DCtet6PFyoLGM7Ra6G6Hwk1njeTJj/ZExZX3\ny3JTc1x/3S8z4sdP5FqzWmBW0V/9i/OYeu87AORYPuc7X98MeKjVPByggAsHj+eldf25esJp/OHT\nFXypcCBfGN8/aj3Xu+eO5LGFK+kjqrh4tJOtu3bRh2ryRRV9RBX51dUMFUcZVLqT76tVqB+8wqNG\ns56/F4CLNZXT3dmUaVmUatk0+PNg8fu8s8NP6CCsZhyFp08Fbz98Ph8ZLgeNfl/Yhx4vqoa1acSY\nq0pEpPpnp3HIUmq4pilgjtiO1zbx6e4yc37FiEZZs78iqlBZeZ0v6rvoUESc8C3ceDgq+3Xa0Ehg\nQkVY3H998UR+8epG/vPpPi6dPpj9ZfWc8eD7/HRuETeeOTKqJkxLPn5/eN7IDjuXGOjBDQaBUIja\npgAOJdKROFXF1p0SCOnlrjNcqulSq/cHyZKC3nOIXYTBKuhuh2rG1kb70BXLMYqZSGQcY3WFJpqo\nMu5TUecj3aVGRdOoCToMgHOK+vHwN6Yy4a5F5rY0p0p+RiSmMNsmXBOIyuq878uToiZuh/fJSFjL\nPRajrYk6q76Z7rihcixGp2cdWaiKsJ1/KMhKY9fxOoSIrn2fSNDdDn2SuaYxECXouRku3OHl+BIt\nFQiR59fg1y1Op6rEfU/uWrgLyOewls/MguG8uH2IzZXg/NH9WbzpEKtvm8a+/ft44OVl3HlWX15c\nukYXfqr1/0UVRcFDaCuWc3GwiYtdwObwP+BJoFzzUurKpuz1bEIZfbnLkUZp2N1TpmXx3CtH6Pel\n2TS69QnPo9VNpjVtlLGwMmFgFh9sO87xmiYeWBQpODYs38PqfRVc8ujyqOPX7q+Iep2Z5jBF2iB2\ngZfPLYlZxpyB8XzXhSeUDTfGki1HufHMkVGTp5X1fpoCwaiO38pr6w5x6sh8231HEtS5ybP8VvxB\njYl3LWJQTjrfDI9sE82BBcPi73U7TOOtwRckqx0RaskgBb0VxEaseFwqUwZns76kigy3aoYnWt0S\nVnF3ORTuuWgitz6/jj6Z+hfFqnOJ6j+Ygl7vx+OK9oE7ojqM6C9yXoYrrsZFulNl3qSIVZ5I0L92\n0lBzsjPDHX3dRbeckVTqNrS8EtGofl7bCAErZuanI3pS2u7H1C/8XPtluqP25zcz+V2QlUZNY23c\nEoMORdBE4rVfIfL86sPhfU5VaTaCJCdmpJLrcZpCpyqCEAqKtx+iIIPloQquXuXmaLAg7jqF+R7e\nuvk0Tr7rVfqIKq6Y6OG6qV427tjFOys3kC908e8jquhXv51JahVZwmJJVgD/0v/c5HZTqmVT80g+\nL7qCDFnj5aymEFc6/QRR0BAMrvJyhbMBnnPxa6FQ6QyioTCwIYN9zkZCKIQ0QRCFEIJhn7zKPY4K\n83yP4qLKESSEYh4zcftSblbLzGOCKJw9sT+LtxxnzJ5VXKWWMXrfNi5XSwghCKwuo6CikS8ruxlS\n54WNhzmpfie5SgMBVIKoNG1zsXB7GQPyvEwVO8Pb9Xu+vPggZ3xjOoPFMYKaSgCFIJH/g5bXWrhu\nYWZMuQrQF7yxTorGMjA7DX9Q04MY3A4zACKZWv7tRQp6C4zom8HucMJP7GpIQgjuvGA8lz72CV63\ng6L+mdxy7mi+dlLEAlNjBP2LE/qz+Z7z+efyvfo1ksjfs0bTpLvUKKGy+uiTiZjxuNSojiMngaC7\nHAqnj+7Dsh2lcR1FMvcxsH7hH7tielShsB99YQwXFw/i3XBYaCKsiTEGqiJsQ0KLh+Tw6rpDZke2\n+NYzqKjz2SaOGfTLdLPzWG1cRUbjs0vU6QFkh8W+wRfAFwzhcihRo7K448PXUgQ8dfVJ/Or1Taag\nGxNzqhoZfcRODoJeUri6MYAvqK+UVat5WK8MZAkD2Zo5jj8Fx3DB5AG88fnhqPNc+Mm3uHqKvI18\nc6KHRSs/J19UM9hXT5PmQxMqCkGcIkAaIRQ08kWIelGL6tPwOAV9hV/f7nOQJ5pQCSEUDZUQKiG8\nhxUuUH1haQzhCIBQgyhCP0ZoIdTdGsWxj3YnnOIE9sLZTmAN3Gcc8z8oBP7gQs9kfkkv9Ro1Qf0i\nXBz+8xWb5DZeho/stscQ0gQBFJT3nFzm1jsbz0tuVrpDBFDIWJHGfFeQjF1pXO8KEUQhgIrqcOLU\nHPhLFUS5g6aQYMzaXJYMDJDtGweMaPnm7UAKegu89//OMlefiR2GQmSyJjfDhRCCW84dE7XfOntu\nFUZje3PRCgZDLDXBM1yOKEFN5KNPhHHuNbML+fvHe5OKaXcnWBzb4Bfzx9nWKYdoQT9/YrS//trT\nhqMoosWoG+uCGZHrCtsKmV+eOpiFG49wx9wiIBKVY3URPXbFNL7zn0iMfUGW7pqJnTg12h67/Rsn\nD+XZlXqMuiHQhh/fpYqozyQW41oXFQ/irLH9yE6PTDwbk5eqEHF14K143Q6OVjdGhQSW1zWZNVgA\nfvzFsXGC7sPJ4bDrBw12KOnMKprEr5evBCBLdVDtD/DrUyayZn9FlA/8lS+fyoOLtrF8VxmzBueb\nyVd/+PIUbn1+fVwbvzNzJI8tjSRmnTI8jxV7yrn61ELOLurHVU+t5MbTh7Ng2S7UcKcxZZCXX8wr\n4orHP+ErUwfy2tr9PHfdyZSU1/Lzl9fz72tPorSmnp+8tJ7RfdJ58qppfOdfKzhQWqN3GgT5w2UT\nueOltagEyXELGpp8fPOkgbz02T5UQnz/rEIe/2AHqgjiIIRKEAdBVELceNpQ/v7RLn270Ld/YXQf\nlm49jEqIcwbl8dH2I6hoTMzysKO+kpHpaZTU15jXKsr1UFFbj6YFUUNNZBLC6w8xyhmEdHtXUEci\nBb0V2A2lTyrM5epTC/nOmSNtz7EOs6xCbK5sEuNmeeArk81VdQysceEZbkeUcEeNAGKE0ehKJg3K\nZkN40QPj+Dvnj+f2LxYlNeveUp9z3ekjeHzZbltrsjn3Q1q4g2tR0G2iNBJZwdkeJ8/fOCtue77X\nzXM3zGT8wKy4hUX6ZekmW2yHaDyrWP//5MHZPLsyfL+woD+zQhd4l0OxHYYbGM/DmCD0WjpUI8RP\nUSDT5eSGM0awwJLEZWCEwVknkw9WRCYxhWg+v8LAFwxFRaEYnZLuQ49+z5lpTk4ensfyXWVRGa+D\nc+o8XYAAABQSSURBVONj7oEoMVcVEZV3YTyf6ibdbRMIuzc8GVmoaZnU4KE0mE4FWahZ/chQcjhC\nCceUvlS7/OzXjuASXug7lu3aYXZrkUn/43nT+DS85kCOcHLhKQNRivqxcIW+EMlXhp3Ef0PRi5KA\nPiFbf3Ixjy79IGr7yOLp/HqjPqp0j53EzzbrtW5uGjmSBQd3c8WIYfzjyF7z+PtPmcQ7m49ypLqR\nkAaDctJ54lszbJ9RZ9DrFonuTOx+qA5V4VcXTki4+LQ1IcQqGIaFHus2/+pJ8RNmRk0NgAHZabjU\nSE8f66O3438/OM2MIDDeg6IIc7KmJZLxlhuz/pdMja6l0Zy4GRZ2Swt724XdtXZkAjBzRD5Zac64\nNhWEo2Fi72OE8cWGZ55vydA0LG5j0s5uUtSK0TkYtVmsk2TG/Y2RYFbM6MmlKrx9y+nMHKFP7BlV\nG8cNyGKvpe5LdrozqeQzfzAUtU6ttY2xHX1WmiNcSC66RonHpfLFCfE+fivpTtW8ntuh4Ap/3tZw\nR9DnE4zP0vjduFTVnP8orW0yM08DwRC+QCguwuQPllDbJn8oLu8imSX5Mi3PLtcyf/KzVyKFy6ob\n/SiKiOvsHapCmkul0R+krimA1935VrkVKeitoKUJPjusxYnsLHS7Ko9PfmsGv79sSuQ8yxdyUE56\ntA+9GWGz5koYgtucO6A9GBbnD8+NjueNvd8v5sdXJWxO9MHeQjeewZLbzmDlz85tVVtjP0djgro6\nRmB+MX8cK392TpwP3TrRHOt3h8TlHiAi1oar7teWVbOM92k8stiaPWlOhaL+WaabzBD06cOi6xDl\nelxxIz87F44vELJ1lzhs3EaZaU7TALC6EV2qws0JYrgNzhwbWR3CpSqmQRK7dGCOx2V+F+rCIwen\nQ5AfrrdTXuczBd0f1PjmE5/GhUIu3xWp2W8UZbO+l0QJSkII8/cyMCfyrPpl2RtqR6qabEcyAj23\nwszsbkdWd1uQgt4KmvuhJsJanMgqXEaUiJ28njOugK9MH2w5L3LUwJz0qNdWv74rZvLSGoliWJuD\ncpKLJYdIxmKiGHIrRocRK3BThkSLzXWnx08KfTls1b9z6xk8cvnUuP1WMXrmulP49cUTzWiWUf0y\nzYnJZImdCzEEOzYe3qEq5g/6sSumm9udzUxE7z5e12xp2IjLRRcWQ6wgsgC2IcaxVTWNVZeMCV4j\nkSY2P+AKy1q5BvleNzv+by7Xzh5ubksUdaEqSpyhkeZUzGgna7ihQ1UY2dfLMJtyBwPDo9biwTmm\nVe92KuaIrDYmbDEvI5I0Z1joTlUhJ92JIvTiYIaLKBAKmVU1m8PtUKJGG82tbTAs38Mdc4t46pqT\nzG3G9yyWfWV1qELEdZwa+oiurilgFtPrSqSgtwK7SbiWuOXcMZw+ug/P3zAzytK7qHgQw/I9XDWr\nMOG5U8NuEiEEz1x/ChcXD4zLFG3OQrdywxkj2Hj3FxNaHHbcMbeIf1xzEsVD4itRxhI0U6EjX+D/\nfvdUfmxTA3rhD0/n8asifsXJg3PYe/98RhdkxlWZHNXPGzU/ceqoPlwxc1jcDynT7Ug6+ibWZ28M\n6e0mvQ2sE7rNLS92zezCZj8HI2yxIDP+c4iNy46NWTbanWVa6HpGbKzonDI8L+7a7rDv+tunD4/b\nF4tDEXHFsYQQpuvJuri3UxWkOVUW3XJG3HWM+QGPJaRXt9D19xH7fnM9EXeY0dk4VQVFEeRluCir\n85nzDK2pRWPtgBOVwhbh9/idM0dGGT2JlrbbW1aHqgoaYqpJappGhlu30JsCoRYDCjoaOSmaBN89\nayR//SC+lGoy9M9O49/fjl+wuiArrcXFC/797VPM7LtTR/bh1JF6aYBEUS7NRUYIIVpd1MvtUDlr\nrH3KfizpTv1L7HIozBqRz8wR+eb6pLGMG5BlZtvasfTHZ/HG54d5cNE2RvfzJjVxu+rOc5NapxLi\nXS4TBmbx07lFXBzj/48l0+3g2tOaF8QR4SJVr31vNoX5Gfzrk7383uLXnT4slz99vZhzx0X8zm/d\nfDrzHl4Wd63YUEujozBcMYbLpW+MoBsug+V3nM0fl2znhVUl5qSkM8YoMfIorChCRC3obWD40Cst\n1SZdFt94LMZozet2mKMWlyOSGBe7AlaOJ2Ks1Jo+dP11XoaLI1UN5GUYORnxpWmvPrWQf4TDgQ0O\nVzVGddSbDja/qpQdsXWNFKF3KJX1fg6GM2ozXCp1viCaplvoZjVGVVroPY6fnF8UtZ5kV+F1O0yB\nsOK2fEmsowZrJUQgudnMDuL5G2dyx9wi0l0qz94wM86X3hqG5WeYE3/JVqlzO9SE9VpiiRV0IQQ3\nnjnSDF9MxIa7v8it50XCUo0ELbvswylDcsj2OLlsRmSSe3L487moeFCU5ZcodDTWQjdGKsZ2o6RD\nH2+0oBsTeQNz0s2wzSHhaJTYztHu++VQhDlp+4OzR/H2LacDkXIVNU3xbkS7UYuxz+NymCn6boeS\ncAST63GZAm4IqGFd98tM4/1tx1m4QV+py67zvtUSMnz1qYWWdkTa9lyCFP/mlud79/+dGfXa6rYv\nCUcXGYELGhoZlmCD1uRsdATSQk9BEn1JPC4Hd31pPDuP1fL0Cvta3p3FqH6Z5vJsHcGYAl1oLi5u\n3mpuC8ZcyDkJCoYlw+775pki8MS3ZjD+l4tsjzPEa8qQnLi68waJyiJYhd5qUBjbV+/TfcjWjuyp\nq2dEiasxKT86/DxjOzO7kFFVFebk4fA+GRT110dTdiO82EU7rBh1YjJcqunCsQvrzEpzUN0YICs9\n3m1mPJtrZhfy0c5SdpfW4QlPOsZidW9cOn0wI/t5uXDyQMptrPlYYhP8nrn+FLMTtM6dffSTOZz2\n2/fN18P7ZLD1SA3D8j1sPVKjW+iW55RM3fmORAp6CtJcr3/N7OH8d3UJT6/Y35UGeoeTmeZk133z\nEopde1AUwbLb58S5Klp7DYN0p8o5Rf3M+h5WcjNcPPyNqcwaYV9DBBJHTyXKbvW4VFRLgSirxRub\n1fvt04ejCD0ZCuIn9h2qYMltZ7LpUBU/fG6dvk0RZqVC63fNboLQLmrqri+Nx6kqZmy+x+2I+NAd\nSlwn0MfrproxYL4vK0bnZJ3HuXjqIPPaVqydk9ft4Mrw5HAygh4bnWC4NyH684mNu3/wsil896yR\nPG1ZDMXa8SUbUttRSJdLCtKSyNmV8U1FOrP9Q/I8SbtoWkIIwZNXn8TZRfbx2BdOGdhs55Eoeioj\nQZ6AECLKerdaprEC4nU7+ME5o02rOLbzGDcgi1H9vHzJMhmtKgJfwAhzjVwvx+OMMybsQk5PC09c\nG1Z+mlMxQx3dDiVKeN+8+TQe+lox540vYFh+hu37hehEqZkxneOkQdk8c/0pUZ2sxxL/HWzlKkWx\nGJ2WMe9h/V563Q4mD85hRqE+XzS6wGvOV4B0uUg6gLkTB7D21MqENZ4lPYtEFnpz0TRWUbRa5S0J\niNWifuk7s8yJa6sYOhTFFGOXJelLCEFBlpsDllWA7Dpdw58cCEU6BWv9cKsbYsJAfV7BGvX0wo2z\n+OrfPom6phCCa2cPZ1Q/LyP7RoT/kcunMnfigLh2ZLisyUEth902hxCCT356ttmpvPzdU7nkr8uj\nFgS5dPpgZo3MZ3Cuh61HIou1SEGXJMUjl09lVYI4XJdDz16VpAZWK/edW+PD/+ww4sQvnT6Y7HSn\nuYB5otKxBtZOYkZhfHgj6CJtxMnbZdVaBd3KvRdN0DMlwyOfmSPy2VNaR67HGVXqormOCuBkm7BL\ngF9+aXzctsL8DNtOxZoPke9188jlU/n+M2sT3rOlsaA11n/KkBx23Tcv+nwhTHfMKMtEc1dHuUhB\nT1EumDwwLmZbkppYBWl0QfTE8k1njYwq/RB7juGbz07XBT1ZN1WihBnQrXh/wL48bHOlhK+Myam4\n+8IJXH/6cPK9bjPJraPcaINz0ympaEjoo47NGenfQgRTC31Mq7BGEkkLXSI5wWiuHMPt5xfZbs/x\nODlY2WAm75w2qg/7yvYnNQn38k2nMsymkzBQLevlxka2GOGWXxhfwPzJA5q9j8uhmGGRhsulo4Tz\n6lML+fWbW8hNoggZkHARa4Nkyli3BpeqmOWUuxIp6BJJN9OSC8IOw59rLHv4qwsn8NUZQ2yt+Vim\nDbVP+DJQFcEDl07mzA2HmTAwOgHMsNinDs3lolaElBqx20bfde9FExg/MHFy2bLb5zS7NOG3TxvO\n5acMtV3X1o7cDBf3XjSBrHSnGc3z2vdms+1oDbfbLF7eXtyOsKB38pJzscgoF4kkBZkaDuMzLECn\nqsTVzWkrDkWQ43HxzVPiSywYr1pbRfCscIGu/mFf9JWzCpk+zN5XDvrEqrVsdCzWUgTJcuWsQi4q\nHvT/2zvfGKmuMg4/P5ZdsLTuQkHcAC2g2xqsCoQgWCS1jasQU/uhiUtMJFXTRE1aUqOBNmli/KQf\njDUxto3/+kFrtVolREVsm/jnA3X5Vyi4slVMIdBdbdoaP9Ht64f7zjI7zu7szN6ZOXfyPslkzj33\nzr3PbM6+c+57zz2Xfp9j5n2r+rjlhszrE1VmOZ0LpZFHPTVmEs2b6KEHQQG557YBblrRywcHltbe\nuE5mk+euN5Vwz60D7Np8Xc27cVvBb+/dPtn7f9tbFzblLvAFk3P9x0XRIAhqML9rHoPvfnvtDeug\nu0tcnrAZJylrlHnz1JJg/ocvfWjK9L7V6L2qu+4ZOuuldC2j1lz/eRMBPQgS4KGh9QzkOHVCI3R3\nzePyxMSU59RWkudokGZwXZVpfNtB6Qxm4s3W3q8dOfQgSICPr18x40XCVjB5N+kMKZetPhHZDcvb\n++OTOu/x/H9edyPPluihB0EA1H5yFGQP4f7AO5YmkQtPma/ecRN3bFhRc7hk3kQPPQgCAHr9CUmV\nj3WrJIJ5bRZ2d3HzO/O/YF2L6KEHQQDAD+/azFPHLkw+Oi4oHhHQgyAAsrHftR74HKRNpFyCIAg6\nhAjoQRAEHULDAV3SKknPSjot6QVJ9+YpFgRBENTHXHLobwBfNLOjkq4Bjkg6ZGanc3ILgiAI6qDh\nHrqZXTSzo17+D3AGyP+JvkEQBMGsyCWHLmk1sAE4XGXd3ZKGJQ2Pj4/ncbggCIKgCnMO6JKuBn4O\n7DGz1yvXm9mjZrbJzDYtW7ZsrocLgiAIpmFOAV1SN1kw/5GZ/SIfpSAIgqARZDWmmpz2g9nM948B\nr5jZnll+Zhz4Z0MHhKXAvxr8bDsokm+4No8i+RbJFYrlO1fX682sZopjLgF9G/BH4CTwplffb2a/\nbmiHtY83bGabmrHvZlAk33BtHkXyLZIrFMu3Va4ND1s0sz9Bzk9WDYIgCBom7hQNgiDoEIoU0B9t\nt0CdFMk3XJtHkXyL5ArF8m2Ja8M59CAIgiAtitRDD4IgCGagEAFd0kcljUgalbS3TQ7flzQm6VRZ\n3RJJhySd9ffFXi9J33Lf5yVtLPvMbt/+rKTdTXKtOnFawr4LJT0n6YT7fsXr10g67F5PSOrx+gW+\nPOrrV5fta5/Xj0j6SDN8/Thdko5JOlAA13OSTko6LmnY61JtC32SnpT0V0lnJG1N2PVG/5uWXq9L\n2tNWXzNL+gV0AS8Ca4Ee4ASwrg0e24GNwKmyuq8De728F/ial3cCvyEbBbQFOOz1S4C/+/tiLy9u\ngms/sNHL1wB/A9Yl7Cvgai93k00hsQX4KTDk9Q8Dn/Py54GHvTwEPOHldd4+FgBrvN10Nak93Af8\nGDjgyym7ngOWVtSl2hYeAz7r5R6gL1XXCu8u4BJwfTt9m/YFc/xDbQUOli3vA/a1yWU1UwP6CNDv\n5X5gxMuPALsqtwN2AY+U1U/ZronevwI+XARf4CrgKPB+shsx5le2A+AgsNXL8307VbaN8u1ydlwJ\nPA3cChzwYyfp6vs+x/8H9OTaAtAL/AO/tpeyaxX3QeDP7fYtQsplBfBS2fJ50pnVcbmZXfTyJWC5\nl6dzbvl30dSJ05L19RTGcWAMOETWY33VzN6ocuxJL1//GnBtC32/CXyZKzfUXZuwK4ABv5N0RNLd\nXpdiW1gDjAM/8HTWdyUtStS1kiHgcS+3zbcIAb0QWPbTmtSQIc0wcVpqvmY2YWbryXq/m4F3tVmp\nKpI+BoyZ2ZF2u9TBNjPbCOwAviBpe/nKhNrCfLK05nfMbAPwX7KUxSQJuU7i10tuB35Wua7VvkUI\n6BeAVWXLK70uBV6W1A/g72NeP51zy76Lqk+clqxvCTN7FXiWLG3RJ6l0N3P5sSe9fH0v8O8W+d4M\n3C7pHPATsrTLQ4m6AmBmF/x9DHiK7AczxbZwHjhvZqVpuJ8kC/ApupazAzhqZi/7ctt8ixDQ/wIM\n+CiCHrJTm/1tdiqxHyhdkd5Nlqsu1X/Kr2pvAV7zU7CDwKCkxX7le9DrckWSgO8BZ8zsGwXwXSap\nz8tvIcv3nyEL7HdO41v6HncCz3hPaD8w5CNL1gADwHN5uprZPjNbaWarydriM2b2yRRdASQtUvZE\nMTx9MQicIsG2YGaXgJck3ehVtwGnU3StYBdX0i0lr/b4NvNCQY4XHHaSjdR4EXigTQ6PAxeBy2Q9\nic+Q5UKfBs4CvweW+LYCvu2+J4FNZfv5NDDqr7ua5LqN7DTveeC4v3Ym7Pte4Jj7ngIe9Pq1ZEFu\nlOx0doHXL/TlUV+/tmxfD/j3GAF2NLlN3MKVUS5JurrXCX+9UPr/SbgtrAeGvS38kmzUR5KufpxF\nZGdcvWV1bfONO0WDIAg6hCKkXIIgCIJZEAE9CIKgQ4iAHgRB0CFEQA+CIOgQIqAHQRB0CBHQgyAI\nOoQI6EEQBB1CBPQgCIIO4X9iGnorp+WAJQAAAABJRU5ErkJggg==\n",
-      "text/plain": [
-       "<matplotlib.figure.Figure at 0x119bae4d0>"
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "\n",
+        "    \u003cMovieInfo id(1), title(Toy Story ), categories(['Animation', \"Children's\", 'Comedy'])\u003e\n",
+        "\n",
+        "\n",
+        "这表示，电影的id是1，标题是《Toy Story》，该电影被分为到三个类别中。这三个类别是动画，儿童，喜剧。\n",
+        "\n",
+        "\n"
       ]
-     },
-     "metadata": {},
-     "output_type": "display_data"
     },
     {
-     "data": {
-      "text/plain": [
-       "<matplotlib.figure.Figure at 0x119bae4d0>"
+      "cell_type": "code",
+      "metadata": {
+        "editable": true
+      },
+      "source": [
+        "user_info = paddle.dataset.movielens.user_info()\n",
+        "print user_info.values()[0]\n"
+      ],
+      "outputs": [
+        {
+          "name": "stdout",
+          "output_type": "stream",
+          "text": [
+            "\n"
+          ]
+        }
+      ],
+      "execution_count": 1
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "\n",
+        "    \u003cUserInfo id(1), gender(F), age(1), job(10)\u003e\n",
+        "\n",
+        "\n",
+        "这表示，该用户ID是1，女性，年龄比18岁还年轻。职业ID是10。\n",
+        "\n",
+        "\n",
+        "其中，年龄使用下列分布\n",
+        "*  1:  \"Under 18\"\n",
+        "* 18:  \"18-24\"\n",
+        "* 25:  \"25-34\"\n",
+        "* 35:  \"35-44\"\n",
+        "* 45:  \"45-49\"\n",
+        "* 50:  \"50-55\"\n",
+        "* 56:  \"56+\"\n",
+        "\n",
+        "职业是从下面几种选项里面选则得出:\n",
+        "*  0:  \"other\" or not specified\n",
+        "*  1:  \"academic/educator\"\n",
+        "*  2:  \"artist\"\n",
+        "*  3:  \"clerical/admin\"\n",
+        "*  4:  \"college/grad student\"\n",
+        "*  5:  \"customer service\"\n",
+        "*  6:  \"doctor/health care\"\n",
+        "*  7:  \"executive/managerial\"\n",
+        "*  8:  \"farmer\"\n",
+        "*  9:  \"homemaker\"\n",
+        "* 10:  \"K-12 student\"\n",
+        "* 11:  \"lawyer\"\n",
+        "* 12:  \"programmer\"\n",
+        "* 13:  \"retired\"\n",
+        "* 14:  \"sales/marketing\"\n",
+        "* 15:  \"scientist\"\n",
+        "* 16:  \"self-employed\"\n",
+        "* 17:  \"technician/engineer\"\n",
+        "* 18:  \"tradesman/craftsman\"\n",
+        "* 19:  \"unemployed\"\n",
+        "* 20:  \"writer\"\n",
+        "\n",
+        "而对于每一条训练/测试数据，均为 \u003c用户特征\u003e + \u003c电影特征\u003e + 评分。\n",
+        "\n",
+        "例如，我们获得第一条训练数据:\n",
+        "\n",
+        "\n"
       ]
-     },
-     "metadata": {},
-     "output_type": "display_data"
-    }
-   ],
-   "source": [
-    "%matplotlib inline\n",
-    "\n",
-    "import matplotlib.pyplot as plt\n",
-    "from IPython import display\n",
-    "import cPickle\n",
-    "\n",
-    "feeding = {\n",
-    "    'user_id': 0,\n",
-    "    'gender_id': 1,\n",
-    "    'age_id': 2,\n",
-    "    'job_id': 3,\n",
-    "    'movie_id': 4,\n",
-    "    'category_id': 5,\n",
-    "    'movie_title': 6,\n",
-    "    'score': 7\n",
-    "}\n",
-    "\n",
-    "step=0\n",
-    "\n",
-    "train_costs=[],[]\n",
-    "test_costs=[],[]\n",
-    "\n",
-    "def event_handler(event):\n",
-    "    global step\n",
-    "    global train_costs\n",
-    "    global test_costs\n",
-    "    if isinstance(event, paddle.event.EndIteration):\n",
-    "        need_plot = False\n",
-    "        if step % 10 == 0:  # every 10 batches, record a train cost\n",
-    "            train_costs[0].append(step)\n",
-    "            train_costs[1].append(event.cost)\n",
-    "            \n",
-    "        if step % 1000 == 0: # every 1000 batches, record a test cost\n",
-    "            result = trainer.test(reader=paddle.batch(\n",
-    "                  paddle.dataset.movielens.test(), batch_size=256))\n",
-    "            test_costs[0].append(step)\n",
-    "            test_costs[1].append(result.cost)\n",
-    "        \n",
-    "        if step % 100 == 0: # every 100 batches, update cost plot\n",
-    "            plt.plot(*train_costs)\n",
-    "            plt.plot(*test_costs)\n",
-    "            plt.legend(['Train Cost', 'Test Cost'], loc='upper left')\n",
-    "            display.clear_output(wait=True)\n",
-    "            display.display(plt.gcf())\n",
-    "            plt.gcf().clear()\n",
-    "        step += 1\n",
-    "\n",
-    "trainer.train(\n",
-    "    reader=paddle.batch(\n",
-    "            paddle.reader.shuffle(\n",
-    "            paddle.dataset.movielens.train(), buf_size=8192),\n",
-    "                            batch_size=256),\n",
-    "    event_handler=event_handler,\n",
-    "    feeding=feeding,\n",
-    "    num_passes=2)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
-   "source": [
-    "## 应用模型\n",
-    "\n",
-    "在训练了几轮以后，您可以对模型进行推断。我们可以使用任意一个用户ID和电影ID，来预测该用户对该电影的评分。示例程序为:"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 15,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
-   "outputs": [
+    },
+    {
+      "cell_type": "code",
+      "metadata": {
+        "editable": true
+      },
+      "source": [
+        "train_set_creator = paddle.dataset.movielens.train()\n",
+        "train_sample = next(train_set_creator())\n",
+        "uid = train_sample[0]\n",
+        "mov_id = train_sample[len(user_info[uid].value())]\n",
+        "print \"User %s rates Movie %s with Score %s\"%(user_info[uid], movie_info[mov_id], train_sample[-1])\n"
+      ],
+      "outputs": [
+        {
+          "name": "stdout",
+          "output_type": "stream",
+          "text": [
+            "\n"
+          ]
+        }
+      ],
+      "execution_count": 1
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "\n",
+        "    User \u003cUserInfo id(1), gender(F), age(1), job(10)\u003e rates Movie \u003cMovieInfo id(1193), title(One Flew Over the Cuckoo's Nest ), categories(['Drama'])\u003e with Score [5.0]\n",
+        "\n",
+        "\n",
+        "即用户1对电影1193的评价为5分。\n",
+        "\n",
+        "## 模型配置说明\n",
+        "\n",
+        "下面我们开始根据输入数据的形式配置模型。\n",
+        "\n",
+        "\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "metadata": {
+        "editable": true
+      },
+      "source": [
+        "uid = paddle.layer.data(\n",
+        "        name='user_id',\n",
+        "        type=paddle.data_type.integer_value(\n",
+        "            paddle.dataset.movielens.max_user_id() + 1))\n",
+        "usr_emb = paddle.layer.embedding(input=uid, size=32)\n",
+        "\n",
+        "usr_gender_id = paddle.layer.data(\n",
+        "        name='gender_id', type=paddle.data_type.integer_value(2))\n",
+        "usr_gender_emb = paddle.layer.embedding(input=usr_gender_id, size=16)\n",
+        "\n",
+        "usr_age_id = paddle.layer.data(\n",
+        "        name='age_id',\n",
+        "        type=paddle.data_type.integer_value(\n",
+        "            len(paddle.dataset.movielens.age_table)))\n",
+        "usr_age_emb = paddle.layer.embedding(input=usr_age_id, size=16)\n",
+        "\n",
+        "usr_job_id = paddle.layer.data(\n",
+        "        name='job_id',\n",
+        "        type=paddle.data_type.integer_value(paddle.dataset.movielens.max_job_id(\n",
+        "        ) + 1))\n",
+        "usr_job_emb = paddle.layer.embedding(input=usr_job_id, size=16)\n"
+      ],
+      "outputs": [
+        {
+          "name": "stdout",
+          "output_type": "stream",
+          "text": [
+            "\n"
+          ]
+        }
+      ],
+      "execution_count": 1
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "\n",
+        "如上述代码所示，对于每个用户，我们输入4维特征。其中包括`user_id`,`gender_id`,`age_id`,`job_id`。这几维特征均是简单的整数值。为了后续神经网络处理这些特征方便，我们借鉴NLP中的语言模型，将这几维离散的整数值，变换成embedding取出。分别形成`usr_emb`, `usr_gender_emb`, `usr_age_emb`, `usr_job_emb`。\n",
+        "\n",
+        "\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "metadata": {
+        "editable": true
+      },
+      "source": [
+        "usr_combined_features = paddle.layer.fc(\n",
+        "        input=[usr_emb, usr_gender_emb, usr_age_emb, usr_job_emb],\n",
+        "        size=200,\n",
+        "        act=paddle.activation.Tanh())\n"
+      ],
+      "outputs": [
+        {
+          "name": "stdout",
+          "output_type": "stream",
+          "text": [
+            "\n"
+          ]
+        }
+      ],
+      "execution_count": 1
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "\n",
+        "然后，我们对于所有的用户特征，均输入到一个全连接层(fc)中。将所有特征融合为一个200维度的特征。\n",
+        "\n",
+        "进而，我们对每一个电影特征做类似的变换，网络配置为:\n",
+        "\n",
+        "\n"
+      ]
+    },
     {
-     "name": "stderr",
-     "output_type": "stream",
-     "text": [
-      "[INFO 2017-03-06 17:17:08,132 networks.py:1472] The input order is [user_id, gender_id, age_id, job_id, movie_id, category_id, movie_title]\n",
-      "[INFO 2017-03-06 17:17:08,134 networks.py:1478] The output order is [__cos_sim_0__]\n"
-     ]
+      "cell_type": "code",
+      "metadata": {
+        "editable": true
+      },
+      "source": [
+        "mov_id = paddle.layer.data(\n",
+        "    name='movie_id',\n",
+        "    type=paddle.data_type.integer_value(\n",
+        "        paddle.dataset.movielens.max_movie_id() + 1))\n",
+        "mov_emb = paddle.layer.embedding(input=mov_id, size=32)\n",
+        "\n",
+        "mov_categories = paddle.layer.data(\n",
+        "    name='category_id',\n",
+        "    type=paddle.data_type.sparse_binary_vector(\n",
+        "        len(paddle.dataset.movielens.movie_categories())))\n",
+        "\n",
+        "mov_categories_hidden = paddle.layer.fc(input=mov_categories, size=32)\n",
+        "\n",
+        "\n",
+        "movie_title_dict = paddle.dataset.movielens.get_movie_title_dict()\n",
+        "mov_title_id = paddle.layer.data(\n",
+        "    name='movie_title',\n",
+        "    type=paddle.data_type.integer_value_sequence(len(movie_title_dict)))\n",
+        "mov_title_emb = paddle.layer.embedding(input=mov_title_id, size=32)\n",
+        "mov_title_conv = paddle.networks.sequence_conv_pool(\n",
+        "    input=mov_title_emb, hidden_size=32, context_len=3)\n",
+        "\n",
+        "mov_combined_features = paddle.layer.fc(\n",
+        "    input=[mov_emb, mov_categories_hidden, mov_title_conv],\n",
+        "    size=200,\n",
+        "    act=paddle.activation.Tanh())\n"
+      ],
+      "outputs": [
+        {
+          "name": "stdout",
+          "output_type": "stream",
+          "text": [
+            "\n"
+          ]
+        }
+      ],
+      "execution_count": 1
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "\n",
+        "电影ID和电影类型分别映射到其对应的特征隐层。对于电影标题名称(title)，一个ID序列表示的词语序列，在输入卷积层后，将得到每个时间窗口的特征（序列特征），然后通过在时间维度降采样得到固定维度的特征，整个过程在text_conv_pool实现。\n",
+        "\n",
+        "最后再将电影的特征融合进`mov_combined_features`中。\n",
+        "\n",
+        "\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "metadata": {
+        "editable": true
+      },
+      "source": [
+        "inference = paddle.layer.cos_sim(a=usr_combined_features, b=mov_combined_features, size=1, scale=5)\n"
+      ],
+      "outputs": [
+        {
+          "name": "stdout",
+          "output_type": "stream",
+          "text": [
+            "\n"
+          ]
+        }
+      ],
+      "execution_count": 1
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "\n",
+        "进而，我们使用余弦相似度计算用户特征与电影特征的相似性。并将这个相似性拟合(回归)到用户评分上。\n",
+        "\n",
+        "\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "metadata": {
+        "editable": true
+      },
+      "source": [
+        "cost = paddle.layer.regression_cost(\n",
+        "        input=inference,\n",
+        "        label=paddle.layer.data(\n",
+        "            name='score', type=paddle.data_type.dense_vector(1)))\n"
+      ],
+      "outputs": [
+        {
+          "name": "stdout",
+          "output_type": "stream",
+          "text": [
+            "\n"
+          ]
+        }
+      ],
+      "execution_count": 1
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "\n",
+        "至此，我们的优化目标就是这个网络配置中的`cost`了。\n",
+        "\n",
+        "## 训练模型\n",
+        "\n",
+        "### 定义参数\n",
+        "神经网络的模型，我们可以简单的理解为网络拓朴结构+参数。之前一节，我们定义出了优化目标`cost`。这个`cost`即为网络模型的拓扑结构。我们开始训练模型，需要先定义出参数。定义方法为:\n",
+        "\n",
+        "\n"
+      ]
     },
     {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "[Predict] User 234 Rating Movie 345 With Score 4.16\n"
-     ]
+      "cell_type": "code",
+      "metadata": {
+        "editable": true
+      },
+      "source": [
+        "parameters = paddle.parameters.create(cost)\n"
+      ],
+      "outputs": [
+        {
+          "name": "stdout",
+          "output_type": "stream",
+          "text": [
+            "\n"
+          ]
+        }
+      ],
+      "execution_count": 1
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "\n",
+        "    [INFO 2017-03-06 17:12:13,284 networks.py:1472] The input order is [user_id, gender_id, age_id, job_id, movie_id, category_id, movie_title, score]\n",
+        "    [INFO 2017-03-06 17:12:13,287 networks.py:1478] The output order is [__regression_cost_0__]\n",
+        "\n",
+        "\n",
+        "`parameters`是模型的所有参数集合。他是一个python的dict。我们可以查看到这个网络中的所有参数名称。因为之前定义模型的时候，我们没有指定参数名称，这里参数名称是自动生成的。当然，我们也可以指定每一个参数名称，方便日后维护。\n",
+        "\n",
+        "\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "metadata": {
+        "editable": true
+      },
+      "source": [
+        "print parameters.keys()\n"
+      ],
+      "outputs": [
+        {
+          "name": "stdout",
+          "output_type": "stream",
+          "text": [
+            "\n"
+          ]
+        }
+      ],
+      "execution_count": 1
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "\n",
+        "    [u'___fc_layer_2__.wbias', u'___fc_layer_2__.w2', u'___embedding_layer_3__.w0', u'___embedding_layer_5__.w0', u'___embedding_layer_2__.w0', u'___embedding_layer_1__.w0', u'___fc_layer_1__.wbias', u'___fc_layer_0__.wbias', u'___fc_layer_1__.w0', u'___fc_layer_0__.w2', u'___fc_layer_0__.w3', u'___fc_layer_0__.w0', u'___fc_layer_0__.w1', u'___fc_layer_2__.w1', u'___fc_layer_2__.w0', u'___embedding_layer_4__.w0', u'___sequence_conv_pool_0___conv_fc.w0', u'___embedding_layer_0__.w0', u'___sequence_conv_pool_0___conv_fc.wbias']\n",
+        "\n",
+        "\n",
+        "### 构造训练(trainer)\n",
+        "\n",
+        "下面，我们根据网络拓扑结构和模型参数来构造出一个本地训练(trainer)。在构造本地训练的时候，我们还需要指定这个训练的优化方法。这里我们使用Adam来作为优化算法。\n",
+        "\n",
+        "\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "metadata": {
+        "editable": true
+      },
+      "source": [
+        "trainer = paddle.trainer.SGD(cost=cost, parameters=parameters,\n",
+        "                            update_equation=paddle.optimizer.Adam(learning_rate=1e-4))\n"
+      ],
+      "outputs": [
+        {
+          "name": "stdout",
+          "output_type": "stream",
+          "text": [
+            "\n"
+          ]
+        }
+      ],
+      "execution_count": 1
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "\n",
+        "    [INFO 2017-03-06 17:12:13,378 networks.py:1472] The input order is [user_id, gender_id, age_id, job_id, movie_id, category_id, movie_title, score]\n",
+        "    [INFO 2017-03-06 17:12:13,379 networks.py:1478] The output order is [__regression_cost_0__]\n",
+        "\n",
+        "\n",
+        "### 训练\n",
+        "\n",
+        "下面我们开始训练过程。\n",
+        "\n",
+        "我们直接使用Paddle提供的数据集读取程序。`paddle.dataset.movielens.train()`和`paddle.dataset.movielens.test()`分别做训练和预测数据集。并且通过`reader_dict`来指定每一个数据和data_layer的对应关系。\n",
+        "\n",
+        "例如，这里的reader_dict表示的是，对于数据层 `user_id`，使用了reader中每一条数据的第0个元素。`gender_id`数据层使用了第1个元素。以此类推。\n",
+        "\n",
+        "训练过程是完全自动的。我们可以使用event_handler来观察训练过程，或进行测试等。这里我们在event_handler里面绘制了训练误差曲线和测试误差曲线。并且保存了模型。\n",
+        "\n",
+        "\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "metadata": {
+        "editable": true
+      },
+      "source": [
+        "%matplotlib inline\n",
+        "\n",
+        "import matplotlib.pyplot as plt\n",
+        "from IPython import display\n",
+        "import cPickle\n",
+        "\n",
+        "feeding = {\n",
+        "    'user_id': 0,\n",
+        "    'gender_id': 1,\n",
+        "    'age_id': 2,\n",
+        "    'job_id': 3,\n",
+        "    'movie_id': 4,\n",
+        "    'category_id': 5,\n",
+        "    'movie_title': 6,\n",
+        "    'score': 7\n",
+        "}\n",
+        "\n",
+        "step=0\n",
+        "\n",
+        "train_costs=[],[]\n",
+        "test_costs=[],[]\n",
+        "\n",
+        "def event_handler(event):\n",
+        "    global step\n",
+        "    global train_costs\n",
+        "    global test_costs\n",
+        "    if isinstance(event, paddle.event.EndIteration):\n",
+        "        need_plot = False\n",
+        "        if step % 10 == 0:  # every 10 batches, record a train cost\n",
+        "            train_costs[0].append(step)\n",
+        "            train_costs[1].append(event.cost)\n",
+        "\n",
+        "        if step % 1000 == 0: # every 1000 batches, record a test cost\n",
+        "            result = trainer.test(reader=paddle.batch(\n",
+        "                  paddle.dataset.movielens.test(), batch_size=256))\n",
+        "            test_costs[0].append(step)\n",
+        "            test_costs[1].append(result.cost)\n",
+        "\n",
+        "        if step % 100 == 0: # every 100 batches, update cost plot\n",
+        "            plt.plot(*train_costs)\n",
+        "            plt.plot(*test_costs)\n",
+        "            plt.legend(['Train Cost', 'Test Cost'], loc='upper left')\n",
+        "            display.clear_output(wait=True)\n",
+        "            display.display(plt.gcf())\n",
+        "            plt.gcf().clear()\n",
+        "        step += 1\n",
+        "\n",
+        "trainer.train(\n",
+        "    reader=paddle.batch(\n",
+        "            paddle.reader.shuffle(\n",
+        "            paddle.dataset.movielens.train(), buf_size=8192),\n",
+        "                            batch_size=256),\n",
+        "    event_handler=event_handler,\n",
+        "    feeding=feeding,\n",
+        "    num_passes=2)\n"
+      ],
+      "outputs": [
+        {
+          "name": "stdout",
+          "output_type": "stream",
+          "text": [
+            "\n"
+          ]
+        }
+      ],
+      "execution_count": 1
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "\n",
+        "\n",
+        "![png](./image/output_32_0.png)\n",
+        "\n",
+        "## 应用模型\n",
+        "\n",
+        "在训练了几轮以后，您可以对模型进行推断。我们可以使用任意一个用户ID和电影ID，来预测该用户对该电影的评分。示例程序为:\n",
+        "\n",
+        "\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "metadata": {
+        "editable": true
+      },
+      "source": [
+        "import copy\n",
+        "user_id = 234\n",
+        "movie_id = 345\n",
+        "\n",
+        "user = user_info[user_id]\n",
+        "movie = movie_info[movie_id]\n",
+        "\n",
+        "feature = user.value() + movie.value()\n",
+        "\n",
+        "infer_dict = copy.copy(feeding)\n",
+        "del infer_dict['score']\n",
+        "\n",
+        "prediction = paddle.infer(output=inference, parameters=parameters, input=[feature], feeding=infer_dict)\n",
+        "score = (prediction[0][0] + 5.0) / 2\n",
+        "print \"[Predict] User %d Rating Movie %d With Score %.2f\"%(user_id, movie_id, score)\n"
+      ],
+      "outputs": [
+        {
+          "name": "stdout",
+          "output_type": "stream",
+          "text": [
+            "\n"
+          ]
+        }
+      ],
+      "execution_count": 1
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "\n",
+        "    [INFO 2017-03-06 17:17:08,132 networks.py:1472] The input order is [user_id, gender_id, age_id, job_id, movie_id, category_id, movie_title]\n",
+        "    [INFO 2017-03-06 17:17:08,134 networks.py:1478] The output order is [__cos_sim_0__]\n",
+        "\n",
+        "\n",
+        "    [Predict] User 234 Rating Movie 345 With Score 4.16\n",
+        "\n",
+        "\n",
+        "## 总结\n",
+        "\n",
+        "本章介绍了传统的推荐系统方法和YouTube的深度神经网络推荐系统，并以电影推荐为例，使用PaddlePaddle训练了一个个性化推荐神经网络模型。推荐系统几乎涵盖了电商系统、社交网络、广告推荐、搜索引擎等领域的方方面面，而在图像处理、自然语言处理等领域已经发挥重要作用的深度学习技术，也将会在推荐系统领域大放异彩。\n",
+        "\n",
+        "## 参考文献\n",
+        "\n",
+        "1. [Peter Brusilovsky](https://en.wikipedia.org/wiki/Peter_Brusilovsky) (2007). *The Adaptive Web*. p. 325.\n",
+        "2. Robin Burke , [Hybrid Web Recommender Systems](http://www.dcs.warwick.ac.uk/~acristea/courses/CS411/2010/Book%20-%20The%20Adaptive%20Web/HybridWebRecommenderSystems.pdf), pp. 377-408, The Adaptive Web, Peter Brusilovsky, Alfred Kobsa, Wolfgang Nejdl (Ed.), Lecture Notes in Computer Science, Springer-Verlag, Berlin, Germany, Lecture Notes in Computer Science, Vol. 4321, May 2007, 978-3-540-72078-2.\n",
+        "3. P. Resnick, N. Iacovou, etc. “[GroupLens: An Open Architecture for Collaborative Filtering of Netnews](http://ccs.mit.edu/papers/CCSWP165.html)”, Proceedings of ACM Conference on Computer Supported Cooperative Work, CSCW 1994. pp.175-186.\n",
+        "4. Sarwar, Badrul, et al. \"[Item-based collaborative filtering recommendation algorithms.](http://files.grouplens.org/papers/www10_sarwar.pdf)\" *Proceedings of the 10th international conference on World Wide Web*. ACM, 2001.\n",
+        "5. Kautz, Henry, Bart Selman, and Mehul Shah. \"[Referral Web: combining social networks and collaborative filtering.](http://www.cs.cornell.edu/selman/papers/pdf/97.cacm.refweb.pdf)\" Communications of the ACM 40.3 (1997): 63-65. APA\n",
+        "6. Yuan, Jianbo, et al. [\"Solving Cold-Start Problem in Large-scale Recommendation Engines: A Deep Learning Approach.\"](https://arxiv.org/pdf/1611.05480v1.pdf) *arXiv preprint arXiv:1611.05480* (2016).\n",
+        "7. Covington P, Adams J, Sargin E. [Deep neural networks for youtube recommendations](https://static.googleusercontent.com/media/research.google.com/zh-CN//pubs/archive/45530.pdf)[C]//Proceedings of the 10th ACM Conference on Recommender Systems. ACM, 2016: 191-198.\n",
+        "\n",
+        "\u003cbr/\u003e\n",
+        "\u003ca rel=\"license\" href=\"http://creativecommons.org/licenses/by-nc-sa/4.0/\"\u003e\u003cimg alt=\"知识共享许可协议\" style=\"border-width:0\" src=\"https://i.creativecommons.org/l/by-nc-sa/4.0/88x31.png\" /\u003e\u003c/a\u003e\u003cbr /\u003e\u003cspan xmlns:dct=\"http://purl.org/dc/terms/\" href=\"http://purl.org/dc/dcmitype/Text\" property=\"dct:title\" rel=\"dct:type\"\u003e本教程\u003c/span\u003e 由 \u003ca xmlns:cc=\"http://creativecommons.org/ns#\" href=\"http://book.paddlepaddle.org\" property=\"cc:attributionName\" rel=\"cc:attributionURL\"\u003ePaddlePaddle\u003c/a\u003e 创作，采用 \u003ca rel=\"license\" href=\"http://creativecommons.org/licenses/by-nc-sa/4.0/\"\u003e知识共享 署名-非商业性使用-相同方式共享 4.0 国际 许可协议\u003c/a\u003e进行许可。\n"
+      ]
+    }
+  ],
+  "metadata": {
+    "kernelspec": {
+      "display_name": "Python 3",
+      "language": "python",
+      "name": "python3"
+    },
+    "language_info": {
+      "codemirror_mode": {
+        "name": "ipython",
+        "version": 3
+      },
+      "file_extension": ".py",
+      "mimetype": "text/x-python",
+      "name": "python",
+      "nbconvert_exporter": "python",
+      "pygments_lexer": "ipython3",
+      "version": "3.6.0"
     }
-   ],
-   "source": [
-    "import copy\n",
-    "user_id = 234\n",
-    "movie_id = 345\n",
-    "\n",
-    "user = user_info[user_id]\n",
-    "movie = movie_info[movie_id]\n",
-    "\n",
-    "feature = user.value() + movie.value()\n",
-    "\n",
-    "infer_dict = copy.copy(feeding)\n",
-    "del infer_dict['score']\n",
-    "\n",
-    "prediction = paddle.infer(output=inference, parameters=parameters, input=[feature], feeding=infer_dict)\n",
-    "score = (prediction[0][0] + 5.0) / 2\n",
-    "print \"[Predict] User %d Rating Movie %d With Score %.2f\"%(user_id, movie_id, score)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
-   "source": [
-    "## 总结\n",
-    "\n",
-    "本章介绍了传统的推荐系统方法和YouTube的深度神经网络推荐系统，并以电影推荐为例，使用PaddlePaddle训练了一个个性化推荐神经网络模型。推荐系统几乎涵盖了电商系统、社交网络、广告推荐、搜索引擎等领域的方方面面，而在图像处理、自然语言处理等领域已经发挥重要作用的深度学习技术，也将会在推荐系统领域大放异彩。\n",
-    "\n",
-    "## 参考文献\n",
-    "\n",
-    "1. [Peter Brusilovsky](https://en.wikipedia.org/wiki/Peter_Brusilovsky) (2007). *The Adaptive Web*. p. 325.\n",
-    "2. Robin Burke , [Hybrid Web Recommender Systems](http://www.dcs.warwick.ac.uk/~acristea/courses/CS411/2010/Book%20-%20The%20Adaptive%20Web/HybridWebRecommenderSystems.pdf), pp. 377-408, The Adaptive Web, Peter Brusilovsky, Alfred Kobsa, Wolfgang Nejdl (Ed.), Lecture Notes in Computer Science, Springer-Verlag, Berlin, Germany, Lecture Notes in Computer Science, Vol. 4321, May 2007, 978-3-540-72078-2.\n",
-    "3. P. Resnick, N. Iacovou, etc. “[GroupLens: An Open Architecture for Collaborative Filtering of Netnews](http://ccs.mit.edu/papers/CCSWP165.html)”, Proceedings of ACM Conference on Computer Supported Cooperative Work, CSCW 1994. pp.175-186.\n",
-    "4. Sarwar, Badrul, et al. \"[Item-based collaborative filtering recommendation algorithms.](http://files.grouplens.org/papers/www10_sarwar.pdf)\" *Proceedings of the 10th international conference on World Wide Web*. ACM, 2001.\n",
-    "5. Kautz, Henry, Bart Selman, and Mehul Shah. \"[Referral Web: combining social networks and collaborative filtering.](http://www.cs.cornell.edu/selman/papers/pdf/97.cacm.refweb.pdf)\" Communications of the ACM 40.3 (1997): 63-65. APA\n",
-    "6. Yuan, Jianbo, et al. [\"Solving Cold-Start Problem in Large-scale Recommendation Engines: A Deep Learning Approach.\"](https://arxiv.org/pdf/1611.05480v1.pdf) *arXiv preprint arXiv:1611.05480* (2016).\n",
-    "7. Covington P, Adams J, Sargin E. [Deep neural networks for youtube recommendations](https://static.googleusercontent.com/media/research.google.com/zh-CN//pubs/archive/45530.pdf)[C]//Proceedings of the 10th ACM Conference on Recommender Systems. ACM, 2016: 191-198.\n",
-    "\n",
-    "<br/>\n",
-    "<a rel=\"license\" href=\"http://creativecommons.org/licenses/by-nc-sa/4.0/\"><img alt=\"知识共享许可协议\" style=\"border-width:0\" src=\"https://i.creativecommons.org/l/by-nc-sa/4.0/88x31.png\" /></a><br /><span xmlns:dct=\"http://purl.org/dc/terms/\" href=\"http://purl.org/dc/dcmitype/Text\" property=\"dct:title\" rel=\"dct:type\">本教程</span> 由 <a xmlns:cc=\"http://creativecommons.org/ns#\" href=\"http://book.paddlepaddle.org\" property=\"cc:attributionName\" rel=\"cc:attributionURL\">PaddlePaddle</a> 创作，采用 <a rel=\"license\" href=\"http://creativecommons.org/licenses/by-nc-sa/4.0/\">知识共享 署名-非商业性使用-相同方式共享 4.0 国际 许可协议</a>进行许可。\n"
-   ]
-  }
- ],
- "metadata": {
-  "kernelspec": {
-   "display_name": "Python 2",
-   "language": "python",
-   "name": "python2"
   },
-  "language_info": {
-   "codemirror_mode": {
-    "name": "ipython",
-    "version": 2
-   },
-   "file_extension": ".py",
-   "mimetype": "text/x-python",
-   "name": "python",
-   "nbconvert_exporter": "python",
-   "pygments_lexer": "ipython2",
-   "version": "2.7.13"
-  }
- },
- "nbformat": 4,
- "nbformat_minor": 0
+  "nbformat": 4,
+  "nbformat_minor": 0
 }

recommender_system/README.md

 # 个性化推荐
 
-本教程源代码目录在[book/recommender_system](https://github.com/PaddlePaddle/book/tree/develop/recommender_system)， 初次使用请参考PaddlePaddle[安装教程](http://www.paddlepaddle.org/doc_cn/build_and_install/index.html)。
+本教程源代码目录在[book/recommender_system](https://github.com/PaddlePaddle/book/tree/develop/recommender_system)， 初次使用请参考PaddlePaddle[安装教程](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/getstarted/build_and_install/docker_install_cn.rst)。
 
 ## 背景介绍
 

recommender_system/common_utils.py

-# Copyright (c) 2016 PaddlePaddle Authors. All Rights Reserved
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-from paddle.trainer.PyDataProvider2 import *
-
-
-def meta_to_header(meta, name):
-    metas = meta[name]['__meta__']['raw_meta']
-    for each_meta in metas:
-        slot_name = each_meta.get('name', '%s_id' % name)
-        if each_meta['type'] == 'id':
-            yield slot_name, integer_value(each_meta['max'])
-        elif each_meta['type'] == 'embedding':
-            is_seq = each_meta['seq'] == 'sequence'
-            yield slot_name, integer_value(
-                len(each_meta['dict']),
-                seq_type=SequenceType.SEQUENCE
-                if is_seq else SequenceType.NO_SEQUENCE)
-        elif each_meta['type'] == 'one_hot_dense':
-            yield slot_name, dense_vector(len(each_meta['dict']))

recommender_system/data/config.json

-{
-  "user": {
-    "file": {
-      "name": "users.dat",
-      "delimiter": "::"
-    },
-    "fields": ["id", "gender", "age", "occupation"]
-  },
-  "movie": {
-    "file": {
-      "name": "movies.dat",
-      "delimiter": "::"
-    },
-    "fields": ["id", "title", "genres"]
-  }
-}

recommender_system/data/config_generator.py

-#!/bin/env python2
-# Copyright (c) 2016 PaddlePaddle Authors. All Rights Reserved
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-"""
-config_generator.py
-
-Usage:
-    ./config_generator.py <config_file> [--output_format=<output_format>]
-    ./config_generator.py -h | --help
-
-Options:
-    -h --help                           Show this screen.
-    --output_format=<output_format>     Output Config format(json or yaml) [default: json].
-"""
-
-import json
-import docopt
-import copy
-
-DEFAULT_FILE = {"type": "split", "delimiter": ","}
-
-DEFAULT_FIELD = {
-    "id": {
-        "type": "id"
-    },
-    "gender": {
-        "name": "gender",
-        "type": "embedding",
-        "dict": {
-            "type": "char_based"
-        }
-    },
-    "age": {
-        "name": "age",
-        "type": "embedding",
-        "dict": {
-            "type": "whole_content",
-            "sort": True
-        }
-    },
-    "occupation": {
-        "name": "occupation",
-        "type": "embedding",
-        "dict": {
-            "type": "whole_content",
-            "sort": "true"
-        }
-    },
-    "title": {
-        "regex": {
-            "pattern": r"^(.*)\((\d+)\)$",
-            "group_id": 1,
-            "strip": True
-        },
-        "name": "title",
-        "type": {
-            "name": "embedding",
-            "seq_type": "sequence",
-        },
-        "dict": {
-            "type": "char_based"
-        }
-    },
-    "genres": {
-        "type": "one_hot_dense",
-        "dict": {
-            "type": "split",
-            "delimiter": "|"
-        },
-        "name": "genres"
-    }
-}
-
-
-def merge_dict(master_dict, slave_dict):
-    return dict(((k, master_dict.get(k) or slave_dict.get(k))
-                 for k in set(slave_dict) | set(master_dict)))
-
-
-def main(filename, fmt):
-    with open(filename, 'r') as f:
-        conf = json.load(f)
-        obj = dict()
-        for k in conf:
-            val = conf[k]
-            file_dict = val['file']
-            file_dict = merge_dict(file_dict, DEFAULT_FILE)
-
-            fields = []
-            for pos, field_key in enumerate(val['fields']):
-                assert isinstance(field_key, basestring)
-                field = copy.deepcopy(DEFAULT_FIELD[field_key])
-                field['pos'] = pos
-                fields.append(field)
-            obj[k] = {"file": file_dict, "fields": fields}
-    meta = {"meta": obj}
-    # print meta
-    if fmt == 'json':
-
-        def formatter(x):
-            import json
-            return json.dumps(x, indent=2)
-    elif fmt == 'yaml':
-
-        def formatter(x):
-            import yaml
-            return yaml.safe_dump(x, default_flow_style=False)
-    else:
-        raise NotImplementedError("Dump format %s is not implemented" % fmt)
-
-    print formatter(meta)
-
-
-if __name__ == '__main__':
-    args = docopt.docopt(__doc__, version="0.1.0")
-    main(args["<config_file>"], args["--output_format"])

recommender_system/data/getdata.sh

-#!/bin/bash
-# Copyright (c) 2016 PaddlePaddle Authors. All Rights Reserved
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-set -ex
-cd "$(dirname "$0")"
-# download the dataset
-wget http://files.grouplens.org/datasets/movielens/ml-1m.zip
-# unzip the dataset
-unzip ml-1m.zip
-# remove the unused zip file
-rm ml-1m.zip

recommender_system/data/meta_generator.py

-#!/bin/env python2
-# Copyright (c) 2016 PaddlePaddle Authors. All Rights Reserved
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-"""
-Preprocess Movielens dataset, to get movie/user object.
-
-Usage:
-    ./preprocess.py <dataset_dir> <binary_filename> [--config=<config_file>]
-    ./preprocess.py -h | --help
-
-Options:
-    -h --help               Show this screen.
-    --version               Show version.
-    --config=<config_file>  Get MetaData config file [default: config.json].
-"""
-import docopt
-import os
-import sys
-import re
-import collections
-
-try:
-    import cPickle as pickle
-except ImportError:
-    import pickle
-
-
-class UniqueIDGenerator(object):
-    def __init__(self):
-        self.pool = collections.defaultdict(self.__next_id__)
-        self.next_id = 0
-
-    def __next_id__(self):
-        tmp = self.next_id
-        self.next_id += 1
-        return tmp
-
-    def __call__(self, k):
-        return self.pool[k]
-
-    def to_list(self):
-        ret_val = [None] * len(self.pool)
-        for k in self.pool.keys():
-            ret_val[self.pool[k]] = k
-        return ret_val
-
-
-class SortedIDGenerator(object):
-    def __init__(self):
-        self.__key_set__ = set()
-        self.dict = None
-
-    def scan(self, key):
-        self.__key_set__.add(key)
-
-    def finish_scan(self, compare=None, key=None, reverse=False):
-        self.__key_set__ = sorted(
-            list(self.__key_set__), cmp=compare, key=key, reverse=reverse)
-        self.dict = dict()
-        for idx, each_key in enumerate(self.__key_set__):
-            self.dict[each_key] = idx
-
-    def __call__(self, key):
-        return self.dict[key]
-
-    def to_list(self):
-        return self.__key_set__
-
-
-class SplitFileReader(object):
-    def __init__(self, work_dir, config):
-        assert isinstance(config, dict)
-        self.filename = config['name']
-        self.delimiter = config.get('delimiter', ',')
-        self.work_dir = work_dir
-
-    def read(self):
-        with open(os.path.join(self.work_dir, self.filename), 'r') as f:
-            for line in f:
-                line = line.strip()
-                if isinstance(self.delimiter, unicode):
-                    self.delimiter = str(self.delimiter)
-                yield line.split(self.delimiter)
-
-    @staticmethod
-    def create(work_dir, config):
-        assert isinstance(config, dict)
-        if config['type'] == 'split':
-            return SplitFileReader(work_dir, config)
-
-
-class IFileReader(object):
-    READERS = [SplitFileReader]
-
-    def read(self):
-        raise NotImplementedError()
-
-    @staticmethod
-    def create(work_dir, config):
-        for reader_cls in IFileReader.READERS:
-            val = reader_cls.create(work_dir, config)
-            if val is not None:
-                return val
-
-
-class IDFieldParser(object):
-    TYPE = 'id'
-
-    def __init__(self, config):
-        self.__max_id__ = -sys.maxint - 1
-        self.__min_id__ = sys.maxint
-        self.__id_count__ = 0
-
-    def scan(self, line):
-        idx = int(line)
-        self.__max_id__ = max(self.__max_id__, idx)
-        self.__min_id__ = min(self.__min_id__, idx)
-        self.__id_count__ += 1
-
-    def parse(self, line):
-        return int(line)
-
-    def meta_field(self):
-        return {
-            "is_key": True,
-            'max': self.__max_id__,
-            'min': self.__min_id__,
-            'count': self.__id_count__,
-            'type': 'id'
-        }
-
-
-class SplitEmbeddingDict(object):
-    def __init__(self, delimiter):
-        self.__id__ = UniqueIDGenerator()
-        self.delimiter = delimiter
-
-    def scan(self, multi):
-        for val in multi.split(self.delimiter):
-            self.__id__(val)
-
-    def parse(self, multi):
-        return map(self.__id__, multi.split(self.delimiter))
-
-    def meta_field(self):
-        return self.__id__.to_list()
-
-
-class EmbeddingFieldParser(object):
-    TYPE = 'embedding'
-
-    NO_SEQUENCE = "no_sequence"
-    SEQUENCE = "sequence"
-
-    class CharBasedEmbeddingDict(object):
-        def __init__(self, is_seq=True):
-            self.__id__ = UniqueIDGenerator()
-            self.is_seq = is_seq
-
-        def scan(self, s):
-            for ch in s:
-                self.__id__(ch)
-
-        def parse(self, s):
-            return map(self.__id__, s) if self.is_seq else self.__id__(s[0])
-
-        def meta_field(self):
-            return self.__id__.to_list()
-
-    class WholeContentDict(object):
-        def __init__(self, need_sort=True):
-            assert need_sort
-            self.__id__ = SortedIDGenerator()
-            self.__has_finished__ = False
-
-        def scan(self, txt):
-            self.__id__.scan(txt)
-
-        def meta_field(self):
-            if not self.__has_finished__:
-                self.__id__.finish_scan()
-                self.__has_finished__ = True
-            return self.__id__.to_list()
-
-        def parse(self, txt):
-            return self.__id__(txt)
-
-    def __init__(self, config):
-        try:
-            self.seq_type = config['type']['seq_type']
-        except TypeError:
-            self.seq_type = EmbeddingFieldParser.NO_SEQUENCE
-
-        if config['dict']['type'] == 'char_based':
-            self.dict = EmbeddingFieldParser.CharBasedEmbeddingDict(
-                self.seq_type == EmbeddingFieldParser.SEQUENCE)
-        elif config['dict']['type'] == 'split':
-            self.dict = SplitEmbeddingDict(config['dict'].get('delimiter', ','))
-        elif config['dict']['type'] == 'whole_content':
-            self.dict = EmbeddingFieldParser.WholeContentDict(config['dict'][
-                'sort'])
-        else:
-            print config
-            assert False
-
-        self.name = config['name']
-
-    def scan(self, s):
-        self.dict.scan(s)
-
-    def meta_field(self):
-        return {
-            'name': self.name,
-            'dict': self.dict.meta_field(),
-            'type': 'embedding',
-            'seq': self.seq_type
-        }
-
-    def parse(self, s):
-        return self.dict.parse(s)
-
-
-class OneHotDenseFieldParser(object):
-    TYPE = 'one_hot_dense'
-
-    def __init__(self, config):
-        if config['dict']['type'] == 'split':
-            self.dict = SplitEmbeddingDict(config['dict']['delimiter'])
-        self.name = config['name']
-
-    def scan(self, s):
-        self.dict.scan(s)
-
-    def meta_field(self):
-        # print self.dict.meta_field()
-        return {
-            'dict': self.dict.meta_field(),
-            'name': self.name,
-            'type': 'one_hot_dense'
-        }
-
-    def parse(self, s):
-        ids = self.dict.parse(s)
-        retv = [0.0] * len(self.dict.meta_field())
-        for idx in ids:
-            retv[idx] = 1.0
-        # print retv
-        return retv
-
-
-class FieldParserFactory(object):
-    PARSERS = [IDFieldParser, EmbeddingFieldParser, OneHotDenseFieldParser]
-
-    @staticmethod
-    def create(config):
-        if isinstance(config['type'], basestring):
-            config_type = config['type']
-        elif isinstance(config['type'], dict):
-            config_type = config['type']['name']
-
-        assert config_type is not None
-
-        for each_parser_cls in FieldParserFactory.PARSERS:
-            if config_type == each_parser_cls.TYPE:
-                return each_parser_cls(config)
-        print config
-
-
-class CompositeFieldParser(object):
-    def __init__(self, parser, extractor):
-        self.extractor = extractor
-        self.parser = parser
-
-    def scan(self, *args, **kwargs):
-        self.parser.scan(self.extractor.extract(*args, **kwargs))
-
-    def parse(self, *args, **kwargs):
-        return self.parser.parse(self.extractor.extract(*args, **kwargs))
-
-    def meta_field(self):
-        return self.parser.meta_field()
-
-
-class PositionContentExtractor(object):
-    def __init__(self, pos):
-        self.pos = pos
-
-    def extract(self, line):
-        assert isinstance(line, list)
-        return line[self.pos]
-
-
-class RegexPositionContentExtractor(PositionContentExtractor):
-    def __init__(self, pos, pattern, group_id, strip=True):
-        PositionContentExtractor.__init__(self, pos)
-        pattern = pattern.strip()
-        self.pattern = re.compile(pattern)
-        self.group_id = group_id
-        self.strip = strip
-
-    def extract(self, line):
-        line = PositionContentExtractor.extract(self, line)
-        match = self.pattern.match(line)
-        # print line, self.pattern.pattern, match
-        assert match is not None
-        txt = match.group(self.group_id)
-        if self.strip:
-            txt.strip()
-        return txt
-
-
-class ContentExtractorFactory(object):
-    def extract(self, line):
-        pass
-
-    @staticmethod
-    def create(config):
-        if 'pos' in config:
-            if 'regex' not in config:
-                return PositionContentExtractor(config['pos'])
-            else:
-                extra_args = config['regex']
-                return RegexPositionContentExtractor(
-                    pos=config['pos'], **extra_args)
-
-
-class MetaFile(object):
-    def __init__(self, work_dir):
-        self.work_dir = work_dir
-        self.obj = dict()
-
-    def parse(self, config):
-        config = config['meta']
-
-        ret_obj = dict()
-        for key in config.keys():
-            val = config[key]
-            assert 'file' in val
-            reader = IFileReader.create(self.work_dir, val['file'])
-            assert reader is not None
-            assert 'fields' in val and isinstance(val['fields'], list)
-            fields_config = val['fields']
-            field_parsers = map(MetaFile.__field_config_mapper__, fields_config)
-
-            for each_parser in field_parsers:
-                assert each_parser is not None
-
-            for each_block in reader.read():
-                for each_parser in field_parsers:
-                    each_parser.scan(each_block)
-
-            metas = map(lambda x: x.meta_field(), field_parsers)
-            # print metas
-            key_index = filter(
-                lambda x: x is not None,
-                map(lambda (idx, meta): idx if 'is_key' in meta and meta['is_key'] else None,
-                    enumerate(metas)))[0]
-
-            key_map = []
-            for i in range(min(key_index, len(metas))):
-                key_map.append(i)
-            for i in range(key_index + 1, len(metas)):
-                key_map.append(i)
-
-            obj = {'__meta__': {'raw_meta': metas, 'feature_map': key_map}}
-
-            for each_block in reader.read():
-                idx = field_parsers[key_index].parse(each_block)
-                val = []
-                for i, each_parser in enumerate(field_parsers):
-                    if i != key_index:
-                        val.append(each_parser.parse(each_block))
-                obj[idx] = val
-            ret_obj[key] = obj
-        self.obj = ret_obj
-        return ret_obj
-
-    @staticmethod
-    def __field_config_mapper__(conf):
-        assert isinstance(conf, dict)
-        extrator = ContentExtractorFactory.create(conf)
-        field_parser = FieldParserFactory.create(conf)
-        assert extrator is not None
-        assert field_parser is not None
-        return CompositeFieldParser(field_parser, extrator)
-
-    def dump(self, fp):
-        pickle.dump(self.obj, fp, pickle.HIGHEST_PROTOCOL)
-
-
-def preprocess(binary_filename, dataset_dir, config, **kwargs):
-    assert isinstance(config, str)
-    with open(config, 'r') as config_file:
-        file_loader = None
-        if config.lower().endswith('.yaml'):
-            import yaml
-            file_loader = yaml
-        elif config.lower().endswith('.json'):
-            import json
-            file_loader = json
-        config = file_loader.load(config_file)
-    meta = MetaFile(dataset_dir)
-    meta.parse(config)
-    with open(binary_filename, 'wb') as outf:
-        meta.dump(outf)
-
-
-if __name__ == '__main__':
-    args = docopt.docopt(__doc__, version='0.1.0')
-    kwargs = dict()
-    for key in args.keys():
-        if key != '--help':
-            param_name = key
-            assert isinstance(param_name, str)
-            param_name = param_name.replace('<', '')
-            param_name = param_name.replace('>', '')
-            param_name = param_name.replace('--', '')
-            kwargs[param_name] = args[key]
-    preprocess(**kwargs)

recommender_system/data/requirements.txt

-PyYAML
-docopt

recommender_system/data/split.py

-#!/bin/env python2
-# Copyright (c) 2016 PaddlePaddle Authors. All Rights Reserved
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-"""
-Separate movielens 1m dataset to train/test file.
-
-Usage:
-    ./separate.py <input_file> [--test_ratio=<test_ratio>] [--delimiter=<delimiter>]
-    ./separate.py -h | --help
-
-Options:
-    -h --help                       Show this screen.
-    --version                       Show version.
-    --test_ratio=<test_ratio>       Test ratio for separate [default: 0.1].
-    --delimiter=<delimiter>         File delimiter [default: ,].
-"""
-import docopt
-import collections
-import random
-
-
-def process(test_ratio, input_file, delimiter, **kwargs):
-    test_ratio = float(test_ratio)
-    rating_dict = collections.defaultdict(list)
-    with open(input_file, 'r') as f:
-        for line in f:
-            user_id = int(line.split(delimiter)[0])
-            rating_dict[user_id].append(line.strip())
-
-    with open(input_file + ".train", 'w') as train_file:
-        with open(input_file + ".test", 'w') as test_file:
-            for k in rating_dict.keys():
-                lines = rating_dict[k]
-                assert isinstance(lines, list)
-                random.shuffle(lines)
-                test_len = int(len(lines) * test_ratio)
-                for line in lines[:test_len]:
-                    print >> test_file, line
-
-                for line in lines[test_len:]:
-                    print >> train_file, line
-
-
-if __name__ == '__main__':
-    args = docopt.docopt(__doc__, version='0.1.0')
-    kwargs = dict()
-    for key in args.keys():
-        if key != '--help':
-            param_name = key
-            assert isinstance(param_name, str)
-            param_name = param_name.replace('<', '')
-            param_name = param_name.replace('>', '')
-            param_name = param_name.replace('--', '')
-            kwargs[param_name] = args[key]
-    process(**kwargs)

recommender_system/dataprovider.py

-# Copyright (c) 2016 PaddlePaddle Authors. All Rights Reserved
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-from paddle.trainer.PyDataProvider2 import *
-from common_utils import meta_to_header
-
-
-def __list_to_map__(lst):
-    ret_val = dict()
-    for each in lst:
-        k, v = each
-        ret_val[k] = v
-    return ret_val
-
-
-def hook(settings, meta, **kwargs):
-    """
-    Init hook is invoked before process data. It will set obj.slots and store
-    data meta.
-
-    :param obj: global object. It will passed to process routine.
-    :type obj: object
-    :param meta: the meta file object, which passed from trainer_config. Meta
-                 file record movie/user features.
-    :param kwargs: unused other arguments.
-    """
-
-    # Header define slots that used for paddle.
-    #    first part is movie features.
-    #    second part is user features.
-    #    final part is rating score.
-    # header is a list of [USE_SEQ_OR_NOT?, SlotType]
-    movie_headers = list(meta_to_header(meta, 'movie'))
-    settings.movie_names = [h[0] for h in movie_headers]
-    headers = movie_headers
-    user_headers = list(meta_to_header(meta, 'user'))
-    settings.user_names = [h[0] for h in user_headers]
-    headers.extend(user_headers)
-    headers.append(("rating", dense_vector(1)))  # Score
-
-    # slot types.
-    settings.input_types = __list_to_map__(headers)
-    settings.meta = meta
-
-
-@provider(init_hook=hook, cache=CacheType.CACHE_PASS_IN_MEM)
-def process(settings, filename):
-    with open(filename, 'r') as f:
-        for line in f:
-            # Get a rating from file.
-            user_id, movie_id, score = map(int, line.split('::')[:-1])
-
-            # Scale score to [-2, +2]
-            score = float(score - 3)
-
-            # Get movie/user features by movie_id, user_id
-            movie_meta = settings.meta['movie'][movie_id]
-            user_meta = settings.meta['user'][user_id]
-
-            outputs = [('movie_id', movie_id - 1)]
-
-            # Then add movie features
-            for i, each_meta in enumerate(movie_meta):
-                outputs.append((settings.movie_names[i + 1], each_meta))
-
-            # Then add user id.
-            outputs.append(('user_id', user_id - 1))
-
-            # Then add user features.
-            for i, each_meta in enumerate(user_meta):
-                outputs.append((settings.user_names[i + 1], each_meta))
-
-            # Finally, add score
-            outputs.append(('rating', [score]))
-            # Return data to paddle
-            yield __list_to_map__(outputs)

recommender_system/evaluate.py

-#!/usr/bin/python
-# Copyright (c) 2016 PaddlePaddle Authors. All Rights Reserved
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-import sys
-import re
-import math
-
-
-def get_best_pass(log_filename):
-    with open(log_filename, 'r') as f:
-        text = f.read()
-        pattern = re.compile('Test.*? cost=([0-9]+\.[0-9]+).*?pass-([0-9]+)',
-                             re.S)
-        results = re.findall(pattern, text)
-        sorted_results = sorted(results, key=lambda result: float(result[0]))
-        return sorted_results[0]
-
-
-log_filename = sys.argv[1]
-log = get_best_pass(log_filename)
-predict_error = math.sqrt(float(log[0])) / 2
-print 'Best pass is %s, error is %s, which means predict get error as %f' % (
-    log[1], log[0], predict_error)
-
-evaluate_pass = "output/pass-%s" % log[1]
-print "evaluating from pass %s" % evaluate_pass

recommender_system/index.en.html

@@ -44,6 +44,9 @@
 
 The source code of this tutorial is in [book/recommender_system](https://github.com/PaddlePaddle/book/tree/develop/recommender_system).
 
+For instructions on getting started with PaddlePaddle, see [PaddlePaddle installation guide](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/getstarted/build_and_install/docker_install_en.rst).
+
+
 ## Background
 
 With the fast growth of e-commerce, online videos, and online reading business, users have to rely on recommender systems to avoid manually browsing tremendous volume of choices.  Recommender systems understand users' interest by mining user behavior and other properties of users and products.
@@ -118,22 +121,287 @@ Figure 3. A hybrid recommendation model.
 
 ## Dataset
 
-We use the [MovieLens ml-1m](http://files.grouplens.org/datasets/movielens/ml-1m.zip) to train our model.  This dataset includes 10,000 ratings of 4,000 movies from 6,000 users to 4,000 movies.  Each rate is in the range of 1~5.  Thanks to GroupLens Research for collecting, processing and publishing the dataset.  
+We use the [MovieLens ml-1m](http://files.grouplens.org/datasets/movielens/ml-1m.zip) to train our model.  This dataset includes 10,000 ratings of 4,000 movies from 6,000 users to 4,000 movies.  Each rate is in the range of 1~5.  Thanks to GroupLens Research for collecting, processing and publishing the dataset.
+
+`paddle.v2.datasets` package encapsulates multiple public datasets, including `cifar`, `imdb`, `mnist`, `moivelens` and `wmt14`, etc. There's no need for us to manually download and preprocess `MovieLens` dataset.
+
+```python
+# Run this block to show dataset's documentation
+help(paddle.v2.dataset.movielens)
+```
+
+The raw `MoiveLens` contains movie ratings, relevant features from both movies and users.
+For instance, one movie's feature could be:
+
+```python
+movie_info = paddle.dataset.movielens.movie_info()
+print movie_info.values()[0]
+```
+
+```text
+<MovieInfo id(1), title(Toy Story), categories(['Animation', "Children's", 'Comedy'])>
+```
+
+One user's feature could be:
+
+```python
+user_info = paddle.dataset.movielens.user_info()
+print user_info.values()[0]
+```
+
+```text
+<UserInfo id(1), gender(F), age(1), job(10)>
+```
+
+In this dateset, the distribution of age is shown as follows:
+
+```text
+1: "Under 18"
+18: "18-24"
+25: "25-34"
+35: "35-44"
+45: "45-49"
+50: "50-55"
+56: "56+"
+```
+
+User's occupation is selected from the following options:
+
+```text
+0: "other" or not specified
+1: "academic/educator"
+2: "artist"
+3: "clerical/admin"
+4: "college/grad student"
+5: "customer service"
+6: "doctor/health care"
+7: "executive/managerial"
+8: "farmer"
+9: "homemaker"
+10: "K-12 student"
+11: "lawyer"
+12: "programmer"
+13: "retired"
+14: "sales/marketing"
+15: "scientist"
+16: "self-employed"
+17: "technician/engineer"
+18: "tradesman/craftsman"
+19: "unemployed"
+20: "writer"
+```
+
+Each record consists of three main components: user features, movie features and movie ratings.
+Likewise, as a simple example, consider the following:
+
+```python
+train_set_creator = paddle.dataset.movielens.train()
+train_sample = next(train_set_creator())
+uid = train_sample[0]
+mov_id = train_sample[len(user_info[uid].value())]
+print "User %s rates Movie %s with Score %s"%(user_info[uid], movie_info[mov_id], train_sample[-1])
+```
+
+```text
+User <UserInfo id(1), gender(F), age(1), job(10)> rates Movie <MovieInfo id(1193), title(One Flew Over the Cuckoo's Nest), categories(['Drama'])> with Score [5.0]
+```
+
+The output shows that user 1 gave movie `1193` a rating of 5.
+
+After issuing a command `python train.py`, training will start immediately. The details will be unpacked by the following sessions to see how it works.
+
+## Model Architecture
+
+### Initialize PaddlePaddle
+
+First, we must import and initialize PaddlePaddle (enable/disable GPU, set the number of trainers, etc).
+
+```python
+%matplotlib inline
+
+import matplotlib.pyplot as plt
+from IPython import display
+import cPickle
+
+import paddle.v2 as paddle
+
+paddle.init(use_gpu=False)
+```
+
+### Model Configuration
+
+```python
+uid = paddle.layer.data(
+        name='user_id',
+        type=paddle.data_type.integer_value(
+            paddle.dataset.movielens.max_user_id() + 1))
+usr_emb = paddle.layer.embedding(input=uid, size=32)
+
+usr_gender_id = paddle.layer.data(
+        name='gender_id', type=paddle.data_type.integer_value(2))
+usr_gender_emb = paddle.layer.embedding(input=usr_gender_id, size=16)
+
+usr_age_id = paddle.layer.data(
+        name='age_id',
+        type=paddle.data_type.integer_value(
+            len(paddle.dataset.movielens.age_table)))
+usr_age_emb = paddle.layer.embedding(input=usr_age_id, size=16)
+
+usr_job_id = paddle.layer.data(
+        name='job_id',
+        type=paddle.data_type.integer_value(paddle.dataset.movielens.max_job_id(
+        ) + 1))
+usr_job_emb = paddle.layer.embedding(input=usr_job_id, size=16)
+```
+
+As shown in the above code, the input is four dimension integers for each user, that is,  `user_id`,`gender_id`, `age_id` and `job_id`. In order to deal with these features conveniently, we use the language model in NLP to transform these discrete values into embedding vaules `usr_emb`, `usr_gender_emb`, `usr_age_emb` and `usr_job_emb`.
+
+```python
+usr_combined_features = paddle.layer.fc(
+        input=[usr_emb, usr_gender_emb, usr_age_emb, usr_job_emb],
+        size=200,
+        act=paddle.activation.Tanh())
+```
+
+Then, employing user features as input, directly connecting to a fully-connected layer, which is used to reduce dimension to 200.
+
+Furthermore, we do a similar transformation for each movie feature. The model configuration is:
+
+```python
+mov_id = paddle.layer.data(
+    name='movie_id',
+    type=paddle.data_type.integer_value(
+        paddle.dataset.movielens.max_movie_id() + 1))
+mov_emb = paddle.layer.embedding(input=mov_id, size=32)
+
+mov_categories = paddle.layer.data(
+    name='category_id',
+    type=paddle.data_type.sparse_binary_vector(
+        len(paddle.dataset.movielens.movie_categories())))
+
+mov_categories_hidden = paddle.layer.fc(input=mov_categories, size=32)
+
 
-We don't have to download and preprocess the data.  Instead, we can use PaddlePaddle's dataset module `paddle.v2.dataset.movielens`.
+movie_title_dict = paddle.dataset.movielens.get_movie_title_dict()
+mov_title_id = paddle.layer.data(
+    name='movie_title',
+    type=paddle.data_type.integer_value_sequence(len(movie_title_dict)))
+mov_title_emb = paddle.layer.embedding(input=mov_title_id, size=32)
+mov_title_conv = paddle.networks.sequence_conv_pool(
+    input=mov_title_emb, hidden_size=32, context_len=3)
 
+mov_combined_features = paddle.layer.fc(
+    input=[mov_emb, mov_categories_hidden, mov_title_conv],
+    size=200,
+    act=paddle.activation.Tanh())
+```
 
-## Model Specification
+Movie title, a sequence of words represented by an integer word index sequence, will be feed into a `sequence_conv_pool` layer, which will apply convolution and pooling on time dimension. Because pooling is done on time dimension, the output will be a fixed-length vector regardless the length of the input sequence.
 
+Finally, we can use cosine similarity to calculate the similarity between user characteristics and movie features.
 
+```python
+inference = paddle.layer.cos_sim(a=usr_combined_features, b=mov_combined_features, size=1, scale=5)
+cost = paddle.layer.regression_cost(
+        input=inference,
+        label=paddle.layer.data(
+        name='score', type=paddle.data_type.dense_vector(1)))
+```
 
-## Training
+## Model Training
 
+### Define Parameters
 
+First, we define the model parameters according to the previous model configuration `cost`.
 
-## Inference
+```python
+# Create parameters
+parameters = paddle.parameters.create(cost)
+```
 
+### Create Trainer
+
+Before jumping into creating a training module, algorithm setting is also necessary. Here we specified Adam optimization algorithm via `paddle.optimizer`.
 
+```python
+trainer = paddle.trainer.SGD(cost=cost, parameters=parameters,
+                             update_equation=paddle.optimizer.Adam(learning_rate=1e-4))
+```
+
+```text
+[INFO 2017-03-06 17:12:13,378 networks.py:1472] The input order is [user_id, gender_id, age_id, job_id, movie_id, category_id, movie_title, score]
+[INFO 2017-03-06 17:12:13,379 networks.py:1478] The output order is [__regression_cost_0__]
+```
+
+### Training
+
+`paddle.dataset.movielens.train` will yield records during each pass, after shuffling, a batch input is generated for training.
+
+```python
+reader=paddle.reader.batch(
+    paddle.reader.shuffle(
+        paddle.dataset.movielens.trai(), buf_size=8192),
+        batch_size=256)
+```
+
+`feeding` is devoted to specifying the correspondence between each yield record and `paddle.layer.data`. For instance, the first column of data generated by `movielens.train` corresponds to `user_id` feature.
+
+```python
+feeding = {
+    'user_id': 0,
+    'gender_id': 1,
+    'age_id': 2,
+    'job_id': 3,
+    'movie_id': 4,
+    'category_id': 5,
+    'movie_title': 6,
+    'score': 7
+}
+```
+
+Callback function `event_handler` will be called during training when a pre-defined event happens.
+
+```python
+step=0
+
+train_costs=[],[]
+test_costs=[],[]
+
+def event_handler(event):
+    global step
+    global train_costs
+    global test_costs
+    if isinstance(event, paddle.event.EndIteration):
+        need_plot = False
+        if step % 10 == 0:  # every 10 batches, record a train cost
+            train_costs[0].append(step)
+            train_costs[1].append(event.cost)
+
+        if step % 1000 == 0: # every 1000 batches, record a test cost
+            result = trainer.test(reader=paddle.batch(
+                  paddle.dataset.movielens.test(), batch_size=256))
+            test_costs[0].append(step)
+            test_costs[1].append(result.cost)
+
+        if step % 100 == 0: # every 100 batches, update cost plot
+            plt.plot(*train_costs)
+            plt.plot(*test_costs)
+            plt.legend(['Train Cost', 'Test Cost'], loc='upper left')
+            display.clear_output(wait=True)
+            display.display(plt.gcf())
+            plt.gcf().clear()
+        step += 1
+```
+
+Finally, we can invoke `trainer.train` to start training:
+
+```python
+trainer.train(
+    reader=reader,
+    event_handler=event_handler,
+    feeding=feeding,
+    num_passes=200)
+```
 
 ## Conclusion
 
@@ -141,16 +409,16 @@ This tutorial goes over traditional approaches in recommender system and a deep
 
 ## Reference
 
-1. [Peter Brusilovsky](https://en.wikipedia.org/wiki/Peter_Brusilovsky) (2007). *The Adaptive Web*. p. 325.
-2. Robin Burke , [Hybrid Web Recommender Systems](http://www.dcs.warwick.ac.uk/~acristea/courses/CS411/2010/Book%20-%20The%20Adaptive%20Web/HybridWebRecommenderSystems.pdf), pp. 377-408, The Adaptive Web, Peter Brusilovsky, Alfred Kobsa, Wolfgang Nejdl (Ed.), Lecture Notes in Computer Science, Springer-Verlag, Berlin, Germany, Lecture Notes in Computer Science, Vol. 4321, May 2007, 978-3-540-72078-2.
+1. [Peter Brusilovsky](https://en.wikipedia.org/wiki/Peter_Brusilovsky) (2007). *The Adaptive Web*. p. 325.
+2. Robin Burke , [Hybrid Web Recommender Systems](http://www.dcs.warwick.ac.uk/~acristea/courses/CS411/2010/Book%20-%20The%20Adaptive%20Web/HybridWebRecommenderSystems.pdf), pp. 377-408, The Adaptive Web, Peter Brusilovsky, Alfred Kobsa, Wolfgang Nejdl (Ed.), Lecture Notes in Computer Science, Springer-Verlag, Berlin, Germany, Lecture Notes in Computer Science, Vol. 4321, May 2007, 978-3-540-72078-2.
 3. P. Resnick, N. Iacovou, etc. “[GroupLens: An Open Architecture for Collaborative Filtering of Netnews](http://ccs.mit.edu/papers/CCSWP165.html)”, Proceedings of ACM Conference on Computer Supported Cooperative Work, CSCW 1994. pp.175-186.
-4. Sarwar, Badrul, et al. "[Item-based collaborative filtering recommendation algorithms.](http://files.grouplens.org/papers/www10_sarwar.pdf)" *Proceedings of the 10th International Conference on World Wide Web*. ACM, 2001.
+4. Sarwar, Badrul, et al. "[Item-based collaborative filtering recommendation algorithms.](http://files.grouplens.org/papers/www10_sarwar.pdf)" *Proceedings of the 10th International Conference on World Wide Web*. ACM, 2001.
 5. Kautz, Henry, Bart Selman, and Mehul Shah. "[Referral Web: Combining Social networks and collaborative filtering.](http://www.cs.cornell.edu/selman/papers/pdf/97.cacm.refweb.pdf)" Communications of the ACM 40.3 (1997): 63-65. APA
-6. Yuan, Jianbo, et al. ["Solving Cold-Start Problem in Large-scale Recommendation Engines: A Deep Learning Approach."](https://arxiv.org/pdf/1611.05480v1.pdf) *arXiv preprint arXiv:1611.05480* (2016).
+6. Yuan, Jianbo, et al. ["Solving Cold-Start Problem in Large-scale Recommendation Engines: A Deep Learning Approach."](https://arxiv.org/pdf/1611.05480v1.pdf) *arXiv preprint arXiv:1611.05480* (2016).
 7. Covington P, Adams J, Sargin E. [Deep neural networks for youtube recommendations](https://static.googleusercontent.com/media/research.google.com/zh-CN//pubs/archive/45530.pdf)[C]//Proceedings of the 10th ACM Conference on Recommender Systems. ACM, 2016: 191-198.
 
 <br/>
-<a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/4.0/"><img alt="Creative Commons" style="border-width:0" src="https://i.creativecommons.org/l/by-nc-sa/4.0/88x31.png" /></a><br /><span xmlns:dct="http://purl.org/dc/terms/" href="http://purl.org/dc/dcmitype/Text" property="dct:title" rel="dct:type">This tutorial</span> was created by <a xmlns:cc="http://creativecommons.org/ns#" href="http://book.paddlepaddle.org" property="cc:attributionName" rel="cc:attributionURL">the PaddlePaddle community</a> and published under <a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/4.0/">Common Creative 4.0 License</a>。
+This tutorial is contributed by <a xmlns:cc="http://creativecommons.org/ns#" href="http://book.paddlepaddle.org" property="cc:attributionName" rel="cc:attributionURL">PaddlePaddle</a>, and licensed under a <a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/4.0/">Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License</a>.
 
 </div>
 <!-- You can change the lines below now. -->

recommender_system/index.html

@@ -42,7 +42,7 @@
 <div id="markdown" style='display:none'>
 # 个性化推荐
 
-本教程源代码目录在[book/recommender_system](https://github.com/PaddlePaddle/book/tree/develop/recommender_system)， 初次使用请参考PaddlePaddle[安装教程](http://www.paddlepaddle.org/doc_cn/build_and_install/index.html)。
+本教程源代码目录在[book/recommender_system](https://github.com/PaddlePaddle/book/tree/develop/recommender_system)， 初次使用请参考PaddlePaddle[安装教程](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/getstarted/build_and_install/docker_install_cn.rst)。
 
 ## 背景介绍
 

recommender_system/prediction.py

-#!/bin/env python2
-# Copyright (c) 2016 PaddlePaddle Authors. All Rights Reserved
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-from py_paddle import swig_paddle, DataProviderConverter
-
-from common_utils import *
-from paddle.trainer.config_parser import parse_config
-
-try:
-    import cPickle as pickle
-except ImportError:
-    import pickle
-import sys
-
-if __name__ == '__main__':
-    model_path = sys.argv[1]
-    swig_paddle.initPaddle('--use_gpu=0')
-    conf = parse_config("trainer_config.py", "is_predict=1")
-    network = swig_paddle.GradientMachine.createFromConfigProto(
-        conf.model_config)
-    assert isinstance(network, swig_paddle.GradientMachine)
-    network.loadParameters(model_path)
-    with open('./data/meta.bin', 'rb') as f:
-        meta = pickle.load(f)
-        headers = [h[1] for h in meta_to_header(meta, 'movie')]
-        headers.extend([h[1] for h in meta_to_header(meta, 'user')])
-        cvt = DataProviderConverter(headers)
-        while True:
-            movie_id = int(raw_input("Input movie_id: "))
-            user_id = int(raw_input("Input user_id: "))
-            movie_meta = meta['movie'][movie_id]  # Query Data From Meta.
-            user_meta = meta['user'][user_id]
-            data = [movie_id - 1]
-            data.extend(movie_meta)
-            data.append(user_id - 1)
-            data.extend(user_meta)
-            print "Prediction Score is %.2f" % (
-                network.forwardTest(cvt.convert([data]))[0]['value'][0][0] + 3)

recommender_system/preprocess.sh

-#!/bin/bash
-# Copyright (c) 2016 PaddlePaddle Authors. All Rights Reserved
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-set -e
-
-UNAME_STR=`uname`
-
-if [[ ${UNAME_STR} == 'Linux' ]]; then
-    SHUF_PROG='shuf'
-else
-    SHUF_PROG='gshuf'
-fi
-
-
-cd "$(dirname "$0")"
-delimiter='::'
-dir=ml-1m
-cd data
-echo 'generate meta config file'
-python config_generator.py config.json > meta_config.json
-echo 'generate meta file'
-python meta_generator.py $dir meta.bin --config=meta_config.json
-echo 'split train/test file'
-python split.py $dir/ratings.dat --delimiter=${delimiter} --test_ratio=0.1
-echo 'shuffle train file'
-${SHUF_PROG} $dir/ratings.dat.train > ratings.dat.train
-cp $dir/ratings.dat.test .
-echo "./data/ratings.dat.train" > train.list
-echo "./data/ratings.dat.test" > test.list

recommender_system/train.py

+import paddle.v2 as paddle
+import cPickle
+import copy
+
+
+def main():
+    paddle.init(use_gpu=False)
+    movie_title_dict = paddle.dataset.movielens.get_movie_title_dict()
+    uid = paddle.layer.data(
+        name='user_id',
+        type=paddle.data_type.integer_value(
+            paddle.dataset.movielens.max_user_id() + 1))
+    usr_emb = paddle.layer.embedding(input=uid, size=32)
+
+    usr_gender_id = paddle.layer.data(
+        name='gender_id', type=paddle.data_type.integer_value(2))
+    usr_gender_emb = paddle.layer.embedding(input=usr_gender_id, size=16)
+
+    usr_age_id = paddle.layer.data(
+        name='age_id',
+        type=paddle.data_type.integer_value(
+            len(paddle.dataset.movielens.age_table)))
+    usr_age_emb = paddle.layer.embedding(input=usr_age_id, size=16)
+
+    usr_job_id = paddle.layer.data(
+        name='job_id',
+        type=paddle.data_type.integer_value(
+            paddle.dataset.movielens.max_job_id() + 1))
+
+    usr_job_emb = paddle.layer.embedding(input=usr_job_id, size=16)
+
+    usr_combined_features = paddle.layer.fc(
+        input=[usr_emb, usr_gender_emb, usr_age_emb, usr_job_emb],
+        size=200,
+        act=paddle.activation.Tanh())
+
+    mov_id = paddle.layer.data(
+        name='movie_id',
+        type=paddle.data_type.integer_value(
+            paddle.dataset.movielens.max_movie_id() + 1))
+    mov_emb = paddle.layer.embedding(input=mov_id, size=32)
+
+    mov_categories = paddle.layer.data(
+        name='category_id',
+        type=paddle.data_type.sparse_binary_vector(
+            len(paddle.dataset.movielens.movie_categories())))
+
+    mov_categories_hidden = paddle.layer.fc(input=mov_categories, size=32)
+
+    mov_title_id = paddle.layer.data(
+        name='movie_title',
+        type=paddle.data_type.integer_value_sequence(len(movie_title_dict)))
+    mov_title_emb = paddle.layer.embedding(input=mov_title_id, size=32)
+    mov_title_conv = paddle.networks.sequence_conv_pool(
+        input=mov_title_emb, hidden_size=32, context_len=3)
+
+    mov_combined_features = paddle.layer.fc(
+        input=[mov_emb, mov_categories_hidden, mov_title_conv],
+        size=200,
+        act=paddle.activation.Tanh())
+
+    inference = paddle.layer.cos_sim(
+        a=usr_combined_features, b=mov_combined_features, size=1, scale=5)
+    cost = paddle.layer.regression_cost(
+        input=inference,
+        label=paddle.layer.data(
+            name='score', type=paddle.data_type.dense_vector(1)))
+
+    parameters = paddle.parameters.create(cost)
+
+    trainer = paddle.trainer.SGD(
+        cost=cost,
+        parameters=parameters,
+        update_equation=paddle.optimizer.Adam(learning_rate=1e-4))
+    feeding = {
+        'user_id': 0,
+        'gender_id': 1,
+        'age_id': 2,
+        'job_id': 3,
+        'movie_id': 4,
+        'category_id': 5,
+        'movie_title': 6,
+        'score': 7
+    }
+
+    def event_handler(event):
+        if isinstance(event, paddle.event.EndIteration):
+            if event.batch_id % 100 == 0:
+                print "Pass %d Batch %d Cost %.2f" % (
+                    event.pass_id, event.batch_id, event.cost)
+
+    trainer.train(
+        reader=paddle.batch(
+            paddle.reader.shuffle(
+                paddle.dataset.movielens.train(), buf_size=8192),
+            batch_size=256),
+        event_handler=event_handler,
+        feeding=feeding,
+        num_passes=1)
+
+    user_id = 234
+    movie_id = 345
+
+    user = paddle.dataset.movielens.user_info()[user_id]
+    movie = paddle.dataset.movielens.movie_info()[movie_id]
+
+    feature = user.value() + movie.value()
+
+    def reader():
+        yield feature
+
+    infer_dict = copy.copy(feeding)
+    del infer_dict['score']
+
+    prediction = paddle.infer(
+        output_layer=inference,
+        parameters=parameters,
+        input=[feature],
+        feeding=infer_dict)
+    print(prediction + 5) / 2
+
+
+if __name__ == '__main__':
+    main()

recommender_system/train.sh

-#!/bin/bash
-# Copyright (c) 2016 PaddlePaddle Authors. All Rights Reserved
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-set -e
-paddle train \
-    --config=trainer_config.py \
-    --save_dir=./output \
-    --use_gpu=false \
-    --trainer_count=4\
-    --test_all_data_in_one_period=true \
-    --log_period=100 \
-    --dot_period=1 \
-    --num_passes=50  2>&1 | tee 'log.txt'

recommender_system/trainer_config.py

-# Copyright (c) 2016 PaddlePaddle Authors. All Rights Reserved
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-from paddle.trainer_config_helpers import *
-
-try:
-    import cPickle as pickle
-except ImportError:
-    import pickle
-
-is_predict = get_config_arg('is_predict', bool, False)
-
-META_FILE = 'data/meta.bin'
-
-with open(META_FILE, 'rb') as f:
-    # load meta file
-    meta = pickle.load(f)
-
-if not is_predict:
-    define_py_data_sources2(
-        'data/train.list',
-        'data/test.list',
-        module='dataprovider',
-        obj='process',
-        args={'meta': meta})
-
-settings(
-    batch_size=1600, learning_rate=1e-3, learning_method=RMSPropOptimizer())
-
-movie_meta = meta['movie']['__meta__']['raw_meta']
-user_meta = meta['user']['__meta__']['raw_meta']
-
-movie_id = data_layer('movie_id', size=movie_meta[0]['max'])
-title = data_layer('title', size=len(movie_meta[1]['dict']))
-genres = data_layer('genres', size=len(movie_meta[2]['dict']))
-user_id = data_layer('user_id', size=user_meta[0]['max'])
-gender = data_layer('gender', size=len(user_meta[1]['dict']))
-age = data_layer('age', size=len(user_meta[2]['dict']))
-occupation = data_layer('occupation', size=len(user_meta[3]['dict']))
-
-embsize = 256
-
-# construct movie feature
-movie_id_emb = embedding_layer(input=movie_id, size=embsize)
-movie_id_hidden = fc_layer(input=movie_id_emb, size=embsize)
-
-genres_emb = fc_layer(input=genres, size=embsize)
-
-title_emb = embedding_layer(input=title, size=embsize)
-title_hidden = text_conv_pool(
-    input=title_emb, context_len=5, hidden_size=embsize)
-
-movie_feature = fc_layer(
-    input=[movie_id_hidden, title_hidden, genres_emb], size=embsize)
-
-# construct user feature
-user_id_emb = embedding_layer(input=user_id, size=embsize)
-user_id_hidden = fc_layer(input=user_id_emb, size=embsize)
-
-gender_emb = embedding_layer(input=gender, size=embsize)
-gender_hidden = fc_layer(input=gender_emb, size=embsize)
-
-age_emb = embedding_layer(input=age, size=embsize)
-age_hidden = fc_layer(input=age_emb, size=embsize)
-
-occup_emb = embedding_layer(input=occupation, size=embsize)
-occup_hidden = fc_layer(input=occup_emb, size=embsize)
-
-user_feature = fc_layer(
-    input=[user_id_hidden, gender_hidden, age_hidden, occup_hidden],
-    size=embsize)
-
-similarity = cos_sim(a=movie_feature, b=user_feature, scale=2)
-
-if not is_predict:
-    lbl = data_layer('rating', size=1)
-    cost = regression_cost(input=similarity, label=lbl)
-    outputs(cost)
-
-else:
-    outputs(similarity)

tools/convert-markdown-into-ipynb-and-test.sh

+#!/bin/sh
+command -v go >/dev/null 2>&1
+if [ $? -ne 0 ]; then
+    echo >&2 "Please install go https://golang.org/doc/install#install"
+    exit 1
+fi
+
+GOPATH=/tmp/go go get -u github.com/wangkuiyi/ipynb/markdown-to-ipynb
+
+cur_path=$(dirname $(readlink -f $0))
+cd $cur_path/../
+
+#convert md to ipynb
+for file in */{README,README\.en}.md ; do
+    /tmp/go/bin/markdown-to-ipynb < $file > ${file%.*}".ipynb"
+    if [ $? -ne 0 ]; then
+        echo >&2 "markdown-to-ipynb $file error"
+        exit 1
+    fi
+done
+
+if [[ -z $TEST_EMBEDDED_PYTHON_SCRIPTS ]]; then
+    exit 0
+fi
+
+#exec ipynb's py file
+for file in */{README,README\.en}.ipynb ; do
+    pushd $PWD > /dev/null
+    cd $(dirname $file) > /dev/null
+
+    echo "begin test $file"
+    jupyter nbconvert --to python $(basename $file) --stdout | python
+
+    popd > /dev/null
+    #break
+done

understand_sentiment/README.en.md

 # Sentiment Analysis
 
-The source codes of this section can be located at [book/understand_sentiment](https://github.com/PaddlePaddle/book/tree/develop/understand_sentiment). First-time users may refer to PaddlePaddle for [Installation guide](http://www.paddlepaddle.org/doc_cn/build_and_install/index.html).
+The source codes of this section can be located at [book/understand_sentiment](https://github.com/PaddlePaddle/book/tree/develop/understand_sentiment). First-time users may refer to PaddlePaddle for [Installation guide](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/getstarted/build_and_install/docker_install_en.rst).
+
+## Background
 
-## Background Introduction
 In natural language processing, sentiment analysis refers to describing emotion status in texts. The texts may refer to a sentence, a paragraph or a document. Emotion status can be a binary classification problem (positive/negative or happy/sad), or a three-class problem (positive/neutral/negative). Sentiment analysis can be applied widely in various situations, such as online shopping (Amazon, Taobao), travel and movie websites. It can be used to grasp from the reviews how the customers feel about the product. Table 1 is an example of sentiment analysis in movie reviews:
 
 | Movie Review       | Category  |
@@ -22,10 +23,12 @@ For a piece of text, BOW model ignores its word order, grammar and syntax, and r
 In this chapter, we introduce our deep learning model which handles these issues in BOW. Our model embeds texts into a low-dimensional space and takes word order into consideration. It is an end-to-end framework, and has large performance improvement over traditional methods \[[1](#Reference)\].
 
 ## Model Overview
+
 The model we used in this chapter is the CNN (Convolutional Neural Networks) and RNN (Recurrent Neural Networks) with some specific extension.
 
 
 ### Convolutional Neural Networks for Texts (CNN)
+
 Convolutional Neural Networks are always applied in data with grid-like topology, such as 2-d images and 1-d texts. CNN can combine extracted multiple local features to produce higher-level abstract semantics. Experimentally, CNN is very efficient for image and text modeling.
 
 CNN mainly contains convolution and pooling operation, with various extensions. We briefly describe CNN here with an example \[[1](#Refernce)\]. As shown in Figure 1：
@@ -55,7 +58,8 @@ Finally, the CNN features are concatenated together to produce a fixed-length re
 
 For short texts, above CNN model can achieve high accuracy \[[1](#Reference)\]. If we want to extract more abstract representation, we may apply a deeper CNN model \[[2](#Reference),[3](#Reference)\].
 
-### Recurrent Neural Network（RNN）
+### Recurrent Neural Network (RNN)
+
 RNN is an effective model for sequential data. Theoretical, the  computational ability of RNN is Turing-complete \[[4](#Reference)\]. NLP is a classical sequential data, and RNN (especially its variant LSTM\[[5](#Reference)\]) achieves State-of-the-Art performance on various tasks in NLP, such as language modeling, syntax parsing, POS-tagging, image captioning, dialog, machine translation and so forth.
 
 <p align="center">
@@ -70,8 +74,9 @@ where $W_{xh}$ is the weight matrix from input to latent; $W_{hh}$ is the latent
 
 In NLP, words are first represented as a one-hot vector and then mapped to an embedding. The embedded feature goes through an RNN as input $x_t$ at every time step. Moreover, we can add other layers on top of RNN. e.g., a deep or stacked RNN. Also, the last latent state can be used as a feature for sentence classification.
 
-### Long-Short Term Memory
-For data of long sequence, training RNN sometimes has gradient vanishing and explosion problem \[[6](#)\]. To solve this problem Hochreiter S, Schmidhuber J. (1997) proposed the LSTM(long short term memory\[[5](#Refernce)\]).  
+### Long-Short Term Memory (LSTM)
+
+For data of long sequence, training RNN sometimes has gradient vanishing and explosion problem \[[6](#)\]. To solve this problem Hochreiter S, Schmidhuber J. (1997) proposed the LSTM(long short term memory\[[5](#Reference)\]).  
 
 Compared with simple RNN, the structrue of LSTM has included memory cell $c$, input gate $i$, forget gate $f$ and output gate $o$. These gates and memory cells largely improves the ability of handling long sequences. We can formulate LSTM-RNN as a function $F$ as：
 
@@ -99,6 +104,7 @@ $$ h_t=Recrurent(x_t,h_{t-1})$$
 where $Recrurent$ is a simple RNN, GRU or LSTM.
 
 ### Stacked Bidirectional LSTM
+
 For vanilla LSTM, $h_t$ contains input information from previous time-step $1..t-1$ context. We can also apply an RNN with reverse-direction to take successive context $t+1…n$ into consideration. Combining constructing deep RNN (deeper RNN can contain more abstract and higher level semantic), we can design structures with deep stacked bidirectional LSTM to model sequential data\[[9](#Reference)\].
 
 As shown in Figure 4 (3-layer RNN), odd/even layers are forward/reverse LSTM. Higher layers of LSTM take lower-layers LSTM as input, and the top-layer LSTM produces a fixed length vector by max-pooling (this representation considers contexts from previous and successive words for higher-level abstractions). Finally, we concatenate the output to a softmax layer for classification.
@@ -108,377 +114,247 @@ As shown in Figure 4 (3-layer RNN), odd/even layers are forward/reverse LSTM. Hi
 Figure 4. Stacked Bidirectional LSTM for NLP modeling.
 </p>
 
-## Data Preparation
-### Data introduction and Download
-We taks the [IMDB sentiment analysis dataset](http://ai.stanford.edu/%7Eamaas/data/sentiment/) as an example. IMDB dataset contains training and testing set, with 25000 movie reviews. With a 1-10 score, negative reviews are those with score<=4, while positives are those with score>=7. You may use following scripts to download the IMDB dataset and [Moses](http://www.statmt.org/moses/) toolbox:
+## Dataset
 
+We use [IMDB](http://ai.stanford.edu/%7Eamaas/data/sentiment/) dataset for sentiment analysis in this tutorial, which consists of 50,000 movie reviews split evenly into 25k train and 25k test sets. In the labeled train/test sets, a negative review has a score <= 4 out of 10, and a positive review has a score >= 7 out of 10.
 
-```bash
-./data/get_imdb.sh
-```
-If successful, you should see the directory ```data``` with following files:
+`paddle.datasets` package encapsulates multiple public datasets, including `cifar`, `imdb`, `mnist`, `moivelens`, and `wmt14`, etc. There's no need for us to manually download and preprocess IMDB.
 
-```
-aclImdb  get_imdb.sh  imdb  mosesdecoder-master
-```
+After issuing a command `python train.py`, training will start immediately. The details will be unpacked by the following sessions to see how it works.
 
-* aclImdb: original data downloaded from the website;
-* imdb: containing only training and testing data
-* mosesdecoder-master: Moses tool
 
-### Data Preprocessing
-We use the script `preprocess.py` to preprocess the data. It will call `tokenizer.perl` in the Moses toolbox to split words and punctuations, randomly shuffle training set and construct the dictionary. Notice: we only use labeled training and testing set. Executing following commands will preprocess the data:
+## Model Structure
 
-```
-data_dir="./data/imdb"
-python preprocess.py -i $data_dir
-```
+### Initialize PaddlePaddle
 
-If it runs successfully, `./data/pre-imdb` will contain:
+We must import and initialize PaddlePaddle (enable/disable GPU, set the number of trainers, etc).
 
-```
-dict.txt  labels.list  test.list  test_part_000  train.list  train_part_000
+```python
+import sys
+import paddle.v2 as paddle
+
+# PaddlePaddle init
+paddle.init(use_gpu=False, trainer_count=1)
 ```
 
-* test\_part\_000 和 train\_part\_000: all labeled training and testing set, and the training set is shuffled.
-* train.list and test.list: training and testing file-list (containing list of file names).
-* dict.txt: dictionary generated from training set.
-* labels.list: class label, 0 stands for negative while 1 for positive.
+As alluded to in section [Model Overview](#model-overview), here we provide the implementations of both Text CNN and Stacked-bidirectional LSTM models.
 
-### Data Provider for PaddlePaddle
-PaddlePaddle can read Python-style script for configuration. The following `dataprovider.py` provides a detailed example, consisting of two parts:
+### Text Convolution Neural Network (Text CNN)
 
-* hook: define text information and class Id. Texts are defined as `integer_value_sequence` while class Ids are defined as `integer_value`.
-* process: read line by line for ID and text information split by `’\t\t’`, and yield the data as a generator.
+We create a neural network `convolution_net` as the following snippet code.
 
-```python
-from paddle.trainer.PyDataProvider2 import *
-
-def hook(settings, dictionary, **kwargs):
-settings.word_dict = dictionary
-settings.input_types = {
-'word':  integer_value_sequence(len(settings.word_dict)),
-'label': integer_value(2)
-}
-settings.logger.info('dict len : %d' % (len(settings.word_dict)))
-
-@provider(init_hook=hook)
-def process(settings, file_name):
-with open(file_name, 'r') as fdata:
-for line_count, line in enumerate(fdata):
-label, comment = line.strip().split('\t\t')
-label = int(label)
-words = comment.split()
-word_slot = [
-settings.word_dict[w] for w in words if w in settings.word_dict
-]
-yield {
-'word': word_slot,
-'label': label
-}
-```
+Note: `paddle.networks.sequence_conv_pool` includes both convolution and pooling layer operations.
 
-## Model Setup
-`trainer_config.py` is an example of a setup file.
-### Data Definition
 ```python
-from os.path import join as join_path
-from paddle.trainer_config_helpers import *
-# if it is “test” mode
-is_test = get_config_arg('is_test', bool, False)
-# if it is “predict” mode
-is_predict = get_config_arg('is_predict', bool, False)
-
-# Data path
-data_dir = "./data/pre-imdb"
-# File names
-train_list = "train.list"
-test_list = "test.list"
-dict_file = "dict.txt"
-
-# Dictionary size
-dict_dim = len(open(join_path(data_dir, "dict.txt")).readlines())
-# class number
-class_dim = len(open(join_path(data_dir, 'labels.list')).readlines())
-
-if not is_predict:
-train_list = join_path(data_dir, train_list)
-test_list = join_path(data_dir, test_list)
-dict_file = join_path(data_dir, dict_file)
-train_list = train_list if not is_test else None
-# construct the dictionary
-word_dict = dict()
-with open(dict_file, 'r') as f:
-for i, line in enumerate(open(dict_file, 'r')):
-word_dict[line.split('\t')[0]] = i
-# Call the function “define_py_data_sources2” in the file dataprovider.py to extract features
-define_py_data_sources2(
-train_list,
-test_list,
-module="dataprovider",
-obj="process",  # function to generate data
-args={'dictionary': word_dict}) # extra parameters, here refers to dictionary
+def convolution_net(input_dim, class_dim=2, emb_dim=128, hid_dim=128):
+    data = paddle.layer.data("word",
+                             paddle.data_type.integer_value_sequence(input_dim))
+    emb = paddle.layer.embedding(input=data, size=emb_dim)
+    conv_3 = paddle.networks.sequence_conv_pool(
+        input=emb, context_len=3, hidden_size=hid_dim)
+    conv_4 = paddle.networks.sequence_conv_pool(
+        input=emb, context_len=4, hidden_size=hid_dim)
+    output = paddle.layer.fc(input=[conv_3, conv_4],
+                             size=class_dim,
+                             act=paddle.activation.Softmax())
+    lbl = paddle.layer.data("label", paddle.data_type.integer_value(2))
+    cost = paddle.layer.classification_cost(input=output, label=lbl)
+    return cost
 ```
 
-### Algorithm Setup
+1. Define input data and its dimension
 
-```python
-settings(
-batch_size=128,
-learning_rate=2e-3,
-learning_method=AdamOptimizer(),
-regularization=L2Regularization(8e-4),
-gradient_clipping_threshold=25)
-```
+    Parameter `input_dim` denotes the dictionary size, and `class_dim` is the number of categories. In `convolution_net`, the input to the network is defined in `paddle.layer.data`.
 
-* Batch size set as 128;
-* Set global learning rate;
-* Apply ADAM algorithm for optimization;
-* Set up L2 regularization;
-* Set up gradient clipping threshold;
+1. Define Classifier
 
-### Model Structure
-We use PaddlePaddle to implement two classification algorithms, based on above mentioned model [Text-CNN](#Text-CNN（CNN）)和[Stacked-bidirectional LSTM](#Stacked-bidirectional LSTM（Stacked Bidirectional LSTM）)。
-#### Implementation of Text CNN
-```python
-def convolution_net(input_dim,
-class_dim=2,
-emb_dim=128,
-hid_dim=128,
-is_predict=False):
-# network input: id denotes word order, dictionary size as input_dim
-data = data_layer("word", input_dim)
-# Embed one-hot id to embedding subspace
-emb = embedding_layer(input=data, size=emb_dim)
-# Convolution and max-pooling operation, convolution kernel size set as 3
-conv_3 = sequence_conv_pool(input=emb, context_len=3, hidden_size=hid_dim)
-# Convolution and max-pooling, convolution kernel size set as 4
-conv_4 = sequence_conv_pool(input=emb, context_len=4, hidden_size=hid_dim)
-# Concatenate conv_3 and conv_4 as input for softmax classification, class number as class_dim
-output = fc_layer(
-input=[conv_3, conv_4], size=class_dim, act=SoftmaxActivation())
-
-if not is_predict:
-lbl = data_layer("label", 1)    #network input: class label
-outputs(classification_cost(input=output, label=lbl))
-else:
-outputs(output)
-```
+    The above Text CNN network extracts high-level features and maps them to a vector of the same size as the categories. `paddle.activation.Softmax` function or classifier is then used for calculating the probability of the sentence belonging to each category.
+
+1. Define Loss Function
+
+    In the context of supervised learning, labels of the training set are defined in `paddle.layer.data`, too. During training, cross-entropy is used as loss function in `paddle.layer.classification_cost` and as the output of the network; During testing, the outputs are the probabilities calculated in the classifier.
 
-In our implementation, we can use just a single layer [`sequence_conv_pool`](https://github.com/PaddlePaddle/Paddle/blob/develop/python/paddle/trainer_config_helpers/networks.py) to do convolution and pooling operation, convolution kernel size set as hidden_size parameters.
+#### Stacked bidirectional LSTM
 
-#### Implementation of Stacked bidirectional LSTM
+We create a neural network `stacked_lstm_net` as below.
 
 ```python
 def stacked_lstm_net(input_dim,
-class_dim=2,
-emb_dim=128,
-hid_dim=512,
-stacked_num=3,
-is_predict=False):
-
-# layer number of LSTM “stacked_num” is an odd number to confirm the top-layer LSTM is forward
-assert stacked_num % 2 == 1
-# network attributes setup
-layer_attr = ExtraLayerAttribute(drop_rate=0.5)
-# parameter attributes setup
-fc_para_attr = ParameterAttribute(learning_rate=1e-3)
-lstm_para_attr = ParameterAttribute(initial_std=0., learning_rate=1.)
-para_attr = [fc_para_attr, lstm_para_attr]
-bias_attr = ParameterAttribute(initial_std=0., l2_rate=0.)
-# Activation functions
-relu = ReluActivation()
-linear = LinearActivation()
-
-
-# Network input: id as word order, dictionary size is set as input_dim
-data = data_layer("word", input_dim)
-# Mapping id from word to the embedding subspace
-emb = embedding_layer(input=data, size=emb_dim)
-
-fc1 = fc_layer(input=emb, size=hid_dim, act=linear, bias_attr=bias_attr)
-# LSTM-based RNN
-lstm1 = lstmemory(
-input=fc1, act=relu, bias_attr=bias_attr, layer_attr=layer_attr)
-
-# Construct stacked bidirectional LSTM with fc_layer and lstmemory with layer depth as stacked_num:
-inputs = [fc1, lstm1]
-for i in range(2, stacked_num + 1):
-fc = fc_layer(
-input=inputs,
-size=hid_dim,
-act=linear,
-param_attr=para_attr,
-bias_attr=bias_attr)
-lstm = lstmemory(
-input=fc,
-# Odd number-th layer: forward, Even number-th reverse.
-reverse=(i % 2) == 0,
-act=relu,
-bias_attr=bias_attr,
-layer_attr=layer_attr)
-inputs = [fc, lstm]
-
-# Apply max-pooling along the temporal dimension on the last fc_layer to produce a fixed length vector
-fc_last = pooling_layer(input=inputs[0], pooling_type=MaxPooling())
-# Apply max-pooling along tempoeral dim of lstmemory to obtain fixed length feature vector
-lstm_last = pooling_layer(input=inputs[1], pooling_type=MaxPooling())
-# concatenate fc_last and lstm_last as input for a softmax classification layer, with class number equals class_dim
-output = fc_layer(
-input=[fc_last, lstm_last],
-size=class_dim,
-act=SoftmaxActivation(),
-bias_attr=bias_attr,
-param_attr=para_attr)
-
-if is_predict:
-outputs(output)
-else:
-outputs(classification_cost(input=output, label=data_layer('label', 1)))
+                     class_dim=2,
+                     emb_dim=128,
+                     hid_dim=512,
+                     stacked_num=3):
+    """
+    A Wrapper for sentiment classification task.
+    This network uses bi-directional recurrent network,
+    consisting three LSTM layers. This configure is referred to
+    the paper as following url, but use fewer layrs.
+        http://www.aclweb.org/anthology/P15-1109
+    input_dim: here is word dictionary dimension.
+    class_dim: number of categories.
+    emb_dim: dimension of word embedding.
+    hid_dim: dimension of hidden layer.
+    stacked_num: number of stacked lstm-hidden layer.
+    """
+    assert stacked_num % 2 == 1
+
+    layer_attr = paddle.attr.Extra(drop_rate=0.5)
+    fc_para_attr = paddle.attr.Param(learning_rate=1e-3)
+    lstm_para_attr = paddle.attr.Param(initial_std=0., learning_rate=1.)
+    para_attr = [fc_para_attr, lstm_para_attr]
+    bias_attr = paddle.attr.Param(initial_std=0., l2_rate=0.)
+    relu = paddle.activation.Relu()
+    linear = paddle.activation.Linear()
+
+    data = paddle.layer.data("word",
+                             paddle.data_type.integer_value_sequence(input_dim))
+    emb = paddle.layer.embedding(input=data, size=emb_dim)
+
+    fc1 = paddle.layer.fc(input=emb,
+                          size=hid_dim,
+                          act=linear,
+                          bias_attr=bias_attr)
+    lstm1 = paddle.layer.lstmemory(
+        input=fc1, act=relu, bias_attr=bias_attr, layer_attr=layer_attr)
+
+    inputs = [fc1, lstm1]
+    for i in range(2, stacked_num + 1):
+        fc = paddle.layer.fc(input=inputs,
+                             size=hid_dim,
+                             act=linear,
+                             param_attr=para_attr,
+                             bias_attr=bias_attr)
+        lstm = paddle.layer.lstmemory(
+            input=fc,
+            reverse=(i % 2) == 0,
+            act=relu,
+            bias_attr=bias_attr,
+            layer_attr=layer_attr)
+        inputs = [fc, lstm]
+
+    fc_last = paddle.layer.pooling(
+        input=inputs[0], pooling_type=paddle.pooling.Max())
+    lstm_last = paddle.layer.pooling(
+        input=inputs[1], pooling_type=paddle.pooling.Max())
+    output = paddle.layer.fc(input=[fc_last, lstm_last],
+                             size=class_dim,
+                             act=paddle.activation.Softmax(),
+                             bias_attr=bias_attr,
+                             param_attr=para_attr)
+
+    lbl = paddle.layer.data("label", paddle.data_type.integer_value(2))
+    cost = paddle.layer.classification_cost(input=output, label=lbl)
+    return cost
 ```
 
-Our model defined in `trainer_config.py` uses the `stacked_lstm_net` structure as default. If you want to use `convolution_net`, you can comment related lines.
+1. Define input data and its dimension
+
+    Parameter `input_dim` denotes the dictionary size, and `class_dim` is the number of categories. In `stacked_lstm_net`, the input to the network is defined in `paddle.layer.data`.
+
+1. Define Classifier
+
+    The above stacked bidirectional LSTM network extracts high-level features and maps them to a vector of the same size as the categories. `paddle.activation.Softmax` function or classifier is then used for calculating the probability of the sentence belonging to each category.
+
+1. Define Loss Function
+
+    In the context of supervised learning, labels of the training set are defined in `paddle.layer.data`, too. During training, cross-entropy is used as loss function in `paddle.layer.classification_cost` and as the output of the network; During testing, the outputs are the probabilities calculated in the classifier.
+
+
+To reiterate, we can either invoke `convolution_net` or `stacked_lstm_net`.
 
 ```python
-stacked_lstm_net(
-dict_dim, class_dim=class_dim, stacked_num=3, is_predict=is_predict)
-# convolution_net(dict_dim, class_dim=class_dim, is_predict=is_predict)
+word_dict = paddle.dataset.imdb.word_dict()
+dict_dim = len(word_dict)
+class_dim = 2
+
+# option 1
+cost = convolution_net(dict_dim, class_dim=class_dim)
+# option 2
+# cost = stacked_lstm_net(dict_dim, class_dim=class_dim, stacked_num=3)
 ```
 
 ## Model Training
-Use `train.sh` script to run local training:
 
-```
-./train.sh
-```
+### Define Parameters
 
-train.sh is as following:
-
-```bash
-paddle train --config=trainer_config.py \
---save_dir=./model_output \
---job=train \
---use_gpu=false \
---trainer_count=4 \
---num_passes=10 \
---log_period=20 \
---dot_period=20 \
---show_parameter_stats_period=100 \
---test_all_data_in_one_period=1 \
-2>&1 | tee 'train.log'
+First, we create the model parameters according to the previous model configuration `cost`.
+
+```python
+# create parameters
+parameters = paddle.parameters.create(cost)
 ```
 
-* \--config=trainer_config.py: set up model configuration.
-* \--save\_dir=./model_output: set up output folder to save model parameters.
-* \--job=train: set job mode as training.
-* \--use\_gpu=false: Use CPU for training. If you have installed GPU-version PaddlePaddle and want to try GPU training, you may set this term as true.
-* \--trainer\_count=4: setup thread number (or GPU numer）.
-* \--num\_passes=15: Setup pass. In PaddlePaddle, a pass means a training epoch over all samples.
-* \--log\_period=20: print log every 20 batches.
-* \--show\_parameter\_stats\_period=100: Print statistics to screen every 100 batch.
-* \--test\_all_data\_in\_one\_period=1: Predict all testing data every time.
+### Create Trainer
 
-If it is running sussefully, the output log will be saved at `train.log`, model parameters will be saved at the directory `model_output/`. Output log will be as following:
+Before jumping into creating a training module, algorithm setting is also necessary.
+Here we specified `Adam` optimization algorithm via `paddle.optimizer`.
 
+```python
+# create optimizer
+adam_optimizer = paddle.optimizer.Adam(
+    learning_rate=2e-3,
+    regularization=paddle.optimizer.L2Regularization(rate=8e-4),
+    model_average=paddle.optimizer.ModelAverage(average_window=0.5))
+
+# create trainer
+trainer = paddle.trainer.SGD(cost=cost,
+                                parameters=parameters,
+                                update_equation=adam_optimizer)
 ```
-Batch=20 samples=2560 AvgCost=0.681644 CurrentCost=0.681644 Eval: classification_error_evaluator=0.36875  CurrentEval: classification_error_evaluator=0.36875
-...
-Pass=0 Batch=196 samples=25000 AvgCost=0.418964 Eval: classification_error_evaluator=0.1922
-Test samples=24999 cost=0.39297 Eval: classification_error_evaluator=0.149406
-```
-
-* Batch=xx: Already |xx| Batch trained.
-* samples=xx: xx samples have been processed during training.
-* AvgCost=xx: Average loss from 0-th batch to the current batch.
-* CurrentCost=xx: loss of the latest |log_period|-th batch;
-* Eval: classification\_error\_evaluator=xx: Average accuracy from 0-th batch to current batch;
-* CurrentEval: classification\_error\_evaluator: latest |log_period| batches of classification error;
-* Pass=0: Running over all data in the training set is called as a Pass. Pass “0” denotes the first round.
 
+### Training
 
-## Application models
-### Testing
+`paddle.dataset.imdb.train()` will yield records during each pass, after shuffling, a batch input is generated for training.
 
-Testing refers to use trained model to evaluate labeled dataset.
+```python
+train_reader = paddle.batch(
+    paddle.reader.shuffle(
+        lambda: paddle.dataset.imdb.train(word_dict), buf_size=1000),
+    batch_size=100)
 
+test_reader = paddle.batch(
+    lambda: paddle.dataset.imdb.test(word_dict), batch_size=100)
 ```
-./test.sh
-```
 
-Scripts for testing `test.sh` is as following, where the function `get_best_pass` ranks classification accuracy to obtain the best model:
-
-```bash
-function get_best_pass() {
-cat $1  | grep -Pzo 'Test .*\n.*pass-.*' | \
-sed  -r 'N;s/Test.* error=([0-9]+\.[0-9]+).*\n.*pass-([0-9]+)/\1 \2/g' | \
-sort | head -n 1
-}
-
-log=train.log
-LOG=`get_best_pass $log`
-LOG=(${LOG})
-evaluate_pass="model_output/pass-${LOG[1]}"
-
-echo 'evaluating from pass '$evaluate_pass
-
-model_list=./model.list
-touch $model_list | echo $evaluate_pass > $model_list
-net_conf=trainer_config.py
-paddle train --config=$net_conf \
---model_list=$model_list \
---job=test \
---use_gpu=false \
---trainer_count=4 \
---config_args=is_test=1 \
-2>&1 | tee 'test.log'
+`feeding` is devoted to specifying the correspondence between each yield record and `paddle.layer.data`. For instance, the first column of data generated by `paddle.dataset.imdb.train()` corresponds to `word` feature.
+
+```python
+feeding = {'word': 0, 'label': 1}
 ```
 
-Different from training, testing requires denoting `--job = test` and model path `--model_list = $model_list`. If successful, log will be saved at `test.log`. In our test, the best model is `model_output/pass-00002`, with classification error rate as 0.115645：
+Callback function `event_handler` will be invoked to track training progress when a pre-defined event happens.
 
-```
-Pass=0 samples=24999 AvgCost=0.280471 Eval: classification_error_evaluator=0.115645
+```python
+def event_handler(event):
+    if isinstance(event, paddle.event.EndIteration):
+        if event.batch_id % 100 == 0:
+            print "\nPass %d, Batch %d, Cost %f, %s" % (
+                event.pass_id, event.batch_id, event.cost, event.metrics)
+        else:
+            sys.stdout.write('.')
+            sys.stdout.flush()
+    if isinstance(event, paddle.event.EndPass):
+        result = trainer.test(reader=test_reader, reader_dict=reader_dict)
+        print "\nTest with Pass %d, %s" % (event.pass_id, result.metrics)
 ```
 
-### Prediction
-`predict.py` script provides an API. Predicting IMDB data without labels as following:
+Finally, we can invoke `trainer.train` to start training:
 
+```python
+trainer.train(
+    reader=train_reader,
+    event_handler=event_handler,
+    feeding=feedig,
+    num_passes=10)
 ```
-./predict.sh
-```
-predict.sh is as following（default model path `model_output/pass-00002` may exist or modified to others）:
-
-```bash
-model=model_output/pass-00002/
-config=trainer_config.py
-label=data/pre-imdb/labels.list
-cat ./data/aclImdb/test/pos/10007_10.txt | python predict.py \
---tconf=$config \
---model=$model \
---label=$label \
---dict=./data/pre-imdb/dict.txt \
---batch_size=1
-```
-
-* `cat ./data/aclImdb/test/pos/10007_10.txt` : Input prediction samples.
-* `predict.py` : Prediction script.
-* `--tconf=$config` : Network set up.
-* `--model=$model` : Model path set up.
-* `--label=$label` : set up the label dictionary, mapping integer IDs to string labels.
-* `--dict=data/pre-imdb/dict.txt` : set up the dictionary file.
-* `--batch_size=1` : batch size during prediction.
 
 
-Prediction result of our example:
+## Conclusion
 
-```
-Loading parameters from model_output/pass-00002/
-predicting label is pos
-```
+In this chapter, we use sentiment analysis as an example to introduce applying deep learning models on end-to-end short text classification, as well as how to use PaddlePaddle to implement the model. Meanwhile, we briefly introduce two models for text processing: CNN and RNN. In following chapters, we will see how these models can be applied in other tasks.
 
-`10007_10.txt` in folder`./data/aclImdb/test/pos`, the predicted label is also pos，so the prediction is correct.
-## Summary
-In this chapter, we use sentiment analysis as an example to introduce applying deep learning models on end-to-end short text classification, as well as how to use PaddlePaddle to implement the model. Meanwhile, we briefly introduce two models for text processing: CNN and RNN. In following chapters we will see how these models can be applied in other tasks.
 ## Reference
+
 1. Kim Y. [Convolutional neural networks for sentence classification](http://arxiv.org/pdf/1408.5882)[J]. arXiv preprint arXiv:1408.5882, 2014.
 2. Kalchbrenner N, Grefenstette E, Blunsom P. [A convolutional neural network for modelling sentences](http://arxiv.org/pdf/1404.2188.pdf?utm_medium=App.net&utm_source=PourOver)[J]. arXiv preprint arXiv:1404.2188, 2014.
 3. Yann N. Dauphin, et al. [Language Modeling with Gated Convolutional Networks](https://arxiv.org/pdf/1612.08083v1.pdf)[J] arXiv preprint arXiv:1612.08083, 2016.
@@ -490,4 +366,4 @@ In this chapter, we use sentiment analysis as an example to introduce applying d
 9. Zhou J, Xu W. [End-to-end learning of semantic role labeling using recurrent neural networks](http://www.aclweb.org/anthology/P/P15/P15-1109.pdf)[C]//Proceedings of the Annual Meeting of the Association for Computational Linguistics. 2015.
 
 <br/>
-<a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/4.0/"><img alt="知识共享许可协议" style="border-width:0" src="https://i.creativecommons.org/l/by-nc-sa/4.0/88x31.png" /></a><br /><span xmlns:dct="http://purl.org/dc/terms/" href="http://purl.org/dc/dcmitype/Text" property="dct:title" rel="dct:type">本教程</span> 由 <a xmlns:cc="http://creativecommons.org/ns#" href="http://book.paddlepaddle.org" property="cc:attributionName" rel="cc:attributionURL">PaddlePaddle</a> 创作，采用 <a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/4.0/">知识共享 署名-非商业性使用-相同方式共享 4.0 国际 许可协议</a>进行许可。
+This tutorial is contributed by <a xmlns:cc="http://creativecommons.org/ns#" href="http://book.paddlepaddle.org" property="cc:attributionName" rel="cc:attributionURL">PaddlePaddle</a>, and licensed under a <a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/4.0/">Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License</a>.

understand_sentiment/README.md

 # 情感分析
 
-本教程源代码目录在[book/understand_sentiment](https://github.com/PaddlePaddle/book/tree/develop/understand_sentiment)， 初次使用请参考PaddlePaddle[安装教程](http://www.paddlepaddle.org/doc_cn/build_and_install/index.html)。
+本教程源代码目录在[book/understand_sentiment](https://github.com/PaddlePaddle/book/tree/develop/understand_sentiment)， 初次使用请参考PaddlePaddle[安装教程](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/getstarted/build_and_install/docker_install_cn.rst)。
 
 ## 背景介绍
 在自然语言处理中，情感分析一般是指判断一段文本所表达的情绪状态。其中，一段文本可以是一个句子，一个段落或一个文档。情绪状态可以是两类，如（正面，负面），（高兴，悲伤）；也可以是三类，如（积极，消极，中性）等等。情感分析的应用场景十分广泛，如把用户在购物网站（亚马逊、天猫、淘宝等）、旅游网站、电影评论网站上发表的评论分成正面评论和负面评论；或为了分析用户对于某一产品的整体使用感受，抓取产品的用户评论并进行情感分析等等。表格1展示了对电影评论进行情感分析的例子：
@@ -108,14 +108,14 @@ aclImdb
 ```
 Paddle在`dataset/imdb.py`中提实现了imdb数据集的自动下载和读取，并提供了读取字典、训练数据、测试数据等API。
 
-```
+```python
 import sys
 import paddle.v2 as paddle
 ```
 ## 配置模型
 在该示例中，我们实现了两种文本分类算法，分别基于上文所述的[文本卷积神经网络](#文本卷积神经网络（CNN）)和[栈式双向LSTM](#栈式双向LSTM（Stacked Bidirectional LSTM）)。
 ### 文本卷积神经网络
-```
+```python
 def convolution_net(input_dim,
                     class_dim=2,
                     emb_dim=128,
@@ -136,7 +136,7 @@ def convolution_net(input_dim,
 ```
 网络的输入`input_dim`表示的是词典的大小，`class_dim`表示类别数。这里，我们使用[`sequence_conv_pool`](https://github.com/PaddlePaddle/Paddle/blob/develop/python/paddle/trainer_config_helpers/networks.py) API实现了卷积和池化操作。
 ### 栈式双向LSTM
-```
+```python
 def stacked_lstm_net(input_dim,
                      class_dim=2,
                      emb_dim=128,
@@ -205,7 +205,7 @@ def stacked_lstm_net(input_dim,
 ```
 网络的输入`stacked_num`表示的是LSTM的层数，需要是奇数，确保最高层LSTM正向。Paddle里面是通过一个fc和一个lstmemory来实现基于LSTM的循环神经网络。
 ## 训练模型
-```
+```python
 if __name__ == '__main__':
     # init
     paddle.init(use_gpu=False)
@@ -213,14 +213,14 @@ if __name__ == '__main__':
 启动paddle程序，use_gpu=False表示用CPU训练，如果系统支持GPU也可以修改成True使用GPU训练。
 ### 训练数据
 使用Paddle提供的数据集`dataset.imdb`中的API来读取训练数据。
-```
+```python
     print 'load dictionary...'
     word_dict = paddle.dataset.imdb.word_dict()
     dict_dim = len(word_dict)
     class_dim = 2
 ```
 加载数据字典，这里通过`word_dict()`API可以直接构造字典。`class_dim`是指样本类别数，该示例中样本只有正负两类。
-```
+```python
     train_reader = paddle.batch(
         paddle.reader.shuffle(
             lambda: paddle.dataset.imdb.train(word_dict), buf_size=1000),
@@ -230,12 +230,12 @@ if __name__ == '__main__':
         batch_size=100)
 ```
 这里，`dataset.imdb.train()`和`dataset.imdb.test()`分别是`dataset.imdb`中的训练数据和测试数据API。`train_reader`在训练时使用，意义是将读取的训练数据进行shuffle后，组成一个batch数据。同理，`test_reader`是在测试的时候使用，将读取的测试数据组成一个batch。
-```
+```python
     feeding={'word': 0, 'label': 1}
 ```
 `feeding`用来指定`train_reader`和`test_reader`返回的数据与模型配置中data_layer的对应关系。这里表示reader返回的第0列数据对应`word`层，第1列数据对应`label`层。
 ### 构造模型
-```
+```python
     # Please choose the way to build the network
     # by uncommenting the corresponding line.
     cost = convolution_net(dict_dim, class_dim=class_dim)
@@ -243,13 +243,13 @@ if __name__ == '__main__':
 ```
 该示例中默认使用`convolution_net`网络，如果使用`stacked_lstm_net`网络，注释相应的行即可。其中cost是网络的优化目标，同时cost包含了整个网络的拓扑信息。
 ### 网络参数
-```
+```python
     # create parameters
     parameters = paddle.parameters.create(cost)
 ```
 根据网络的拓扑构造网络参数。这里parameters是整个网络的参数集。
 ### 优化算法
-```
+```python
     # create optimizer
     adam_optimizer = paddle.optimizer.Adam(
         learning_rate=2e-3,
@@ -259,7 +259,7 @@ if __name__ == '__main__':
 Paddle中提供了一系列优化算法的API，这里使用Adam优化算法。
 ### 训练
 可以通过`paddle.trainer.SGD`构造一个sgd trainer，并调用`trainer.train`来训练模型。
-```
+```python
     # End batch and end pass event handler
     def event_handler(event):
         if isinstance(event, paddle.event.EndIteration):
@@ -274,7 +274,7 @@ Paddle中提供了一系列优化算法的API，这里使用Adam优化算法。
             print "\nTest with Pass %d, %s" % (event.pass_id, result.metrics)
 ```
 可以通过给train函数传递一个`event_handler`来获取每个batch和每个pass结束的状态。比如构造如下一个`event_handler`可以在每100个batch结束后输出cost和error；在每个pass结束后调用`trainer.test`计算一遍测试集并获得当前模型在测试集上的error。
-```
+```python
     # create trainer
     trainer = paddle.trainer.SGD(cost=cost,
                                  parameters=parameters,
@@ -287,7 +287,7 @@ Paddle中提供了一系列优化算法的API，这里使用Adam优化算法。
         num_passes=2)
 ```
 程序运行之后的输出如下。
-```
+```text
 Pass 0, Batch 0, Cost 0.693721, {'classification_error_evaluator': 0.5546875}
 ...................................................................................................
 Pass 0, Batch 100, Cost 0.294321, {'classification_error_evaluator': 0.1015625}

understand_sentiment/index.en.html

@@ -42,9 +42,10 @@
 <div id="markdown" style='display:none'>
 # Sentiment Analysis
 
-The source codes of this section can be located at [book/understand_sentiment](https://github.com/PaddlePaddle/book/tree/develop/understand_sentiment). First-time users may refer to PaddlePaddle for [Installation guide](http://www.paddlepaddle.org/doc_cn/build_and_install/index.html).
+The source codes of this section can be located at [book/understand_sentiment](https://github.com/PaddlePaddle/book/tree/develop/understand_sentiment). First-time users may refer to PaddlePaddle for [Installation guide](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/getstarted/build_and_install/docker_install_en.rst).
+
+## Background
 
-## Background Introduction
 In natural language processing, sentiment analysis refers to describing emotion status in texts. The texts may refer to a sentence, a paragraph or a document. Emotion status can be a binary classification problem (positive/negative or happy/sad), or a three-class problem (positive/neutral/negative). Sentiment analysis can be applied widely in various situations, such as online shopping (Amazon, Taobao), travel and movie websites. It can be used to grasp from the reviews how the customers feel about the product. Table 1 is an example of sentiment analysis in movie reviews:
 
 | Movie Review       | Category  |
@@ -64,10 +65,12 @@ For a piece of text, BOW model ignores its word order, grammar and syntax, and r
 In this chapter, we introduce our deep learning model which handles these issues in BOW. Our model embeds texts into a low-dimensional space and takes word order into consideration. It is an end-to-end framework, and has large performance improvement over traditional methods \[[1](#Reference)\].
 
 ## Model Overview
+
 The model we used in this chapter is the CNN (Convolutional Neural Networks) and RNN (Recurrent Neural Networks) with some specific extension.
 
 
 ### Convolutional Neural Networks for Texts (CNN)
+
 Convolutional Neural Networks are always applied in data with grid-like topology, such as 2-d images and 1-d texts. CNN can combine extracted multiple local features to produce higher-level abstract semantics. Experimentally, CNN is very efficient for image and text modeling.
 
 CNN mainly contains convolution and pooling operation, with various extensions. We briefly describe CNN here with an example \[[1](#Refernce)\]. As shown in Figure 1：
@@ -97,7 +100,8 @@ Finally, the CNN features are concatenated together to produce a fixed-length re
 
 For short texts, above CNN model can achieve high accuracy \[[1](#Reference)\]. If we want to extract more abstract representation, we may apply a deeper CNN model \[[2](#Reference),[3](#Reference)\].
 
-### Recurrent Neural Network（RNN）
+### Recurrent Neural Network (RNN)
+
 RNN is an effective model for sequential data. Theoretical, the  computational ability of RNN is Turing-complete \[[4](#Reference)\]. NLP is a classical sequential data, and RNN (especially its variant LSTM\[[5](#Reference)\]) achieves State-of-the-Art performance on various tasks in NLP, such as language modeling, syntax parsing, POS-tagging, image captioning, dialog, machine translation and so forth.
 
 <p align="center">
@@ -112,8 +116,9 @@ where $W_{xh}$ is the weight matrix from input to latent; $W_{hh}$ is the latent
 
 In NLP, words are first represented as a one-hot vector and then mapped to an embedding. The embedded feature goes through an RNN as input $x_t$ at every time step. Moreover, we can add other layers on top of RNN. e.g., a deep or stacked RNN. Also, the last latent state can be used as a feature for sentence classification.
 
-### Long-Short Term Memory
-For data of long sequence, training RNN sometimes has gradient vanishing and explosion problem \[[6](#)\]. To solve this problem Hochreiter S, Schmidhuber J. (1997) proposed the LSTM(long short term memory\[[5](#Refernce)\]).  
+### Long-Short Term Memory (LSTM)
+
+For data of long sequence, training RNN sometimes has gradient vanishing and explosion problem \[[6](#)\]. To solve this problem Hochreiter S, Schmidhuber J. (1997) proposed the LSTM(long short term memory\[[5](#Reference)\]).  
 
 Compared with simple RNN, the structrue of LSTM has included memory cell $c$, input gate $i$, forget gate $f$ and output gate $o$. These gates and memory cells largely improves the ability of handling long sequences. We can formulate LSTM-RNN as a function $F$ as：
 
@@ -141,6 +146,7 @@ $$ h_t=Recrurent(x_t,h_{t-1})$$
 where $Recrurent$ is a simple RNN, GRU or LSTM.
 
 ### Stacked Bidirectional LSTM
+
 For vanilla LSTM, $h_t$ contains input information from previous time-step $1..t-1$ context. We can also apply an RNN with reverse-direction to take successive context $t+1…n$ into consideration. Combining constructing deep RNN (deeper RNN can contain more abstract and higher level semantic), we can design structures with deep stacked bidirectional LSTM to model sequential data\[[9](#Reference)\].
 
 As shown in Figure 4 (3-layer RNN), odd/even layers are forward/reverse LSTM. Higher layers of LSTM take lower-layers LSTM as input, and the top-layer LSTM produces a fixed length vector by max-pooling (this representation considers contexts from previous and successive words for higher-level abstractions). Finally, we concatenate the output to a softmax layer for classification.
@@ -150,377 +156,247 @@ As shown in Figure 4 (3-layer RNN), odd/even layers are forward/reverse LSTM. Hi
 Figure 4. Stacked Bidirectional LSTM for NLP modeling.
 </p>
 
-## Data Preparation
-### Data introduction and Download
-We taks the [IMDB sentiment analysis dataset](http://ai.stanford.edu/%7Eamaas/data/sentiment/) as an example. IMDB dataset contains training and testing set, with 25000 movie reviews. With a 1-10 score, negative reviews are those with score<=4, while positives are those with score>=7. You may use following scripts to download the IMDB dataset and [Moses](http://www.statmt.org/moses/) toolbox:
+## Dataset
 
+We use [IMDB](http://ai.stanford.edu/%7Eamaas/data/sentiment/) dataset for sentiment analysis in this tutorial, which consists of 50,000 movie reviews split evenly into 25k train and 25k test sets. In the labeled train/test sets, a negative review has a score <= 4 out of 10, and a positive review has a score >= 7 out of 10.
 
-```bash
-./data/get_imdb.sh
-```
-If successful, you should see the directory ```data``` with following files:
+`paddle.datasets` package encapsulates multiple public datasets, including `cifar`, `imdb`, `mnist`, `moivelens`, and `wmt14`, etc. There's no need for us to manually download and preprocess IMDB.
 
-```
-aclImdb  get_imdb.sh  imdb  mosesdecoder-master
-```
+After issuing a command `python train.py`, training will start immediately. The details will be unpacked by the following sessions to see how it works.
 
-* aclImdb: original data downloaded from the website;
-* imdb: containing only training and testing data
-* mosesdecoder-master: Moses tool
 
-### Data Preprocessing
-We use the script `preprocess.py` to preprocess the data. It will call `tokenizer.perl` in the Moses toolbox to split words and punctuations, randomly shuffle training set and construct the dictionary. Notice: we only use labeled training and testing set. Executing following commands will preprocess the data:
+## Model Structure
 
-```
-data_dir="./data/imdb"
-python preprocess.py -i $data_dir
-```
+### Initialize PaddlePaddle
 
-If it runs successfully, `./data/pre-imdb` will contain:
+We must import and initialize PaddlePaddle (enable/disable GPU, set the number of trainers, etc).
 
-```
-dict.txt  labels.list  test.list  test_part_000  train.list  train_part_000
-```
+```python
+import sys
+import paddle.v2 as paddle
 
-* test\_part\_000 和 train\_part\_000: all labeled training and testing set, and the training set is shuffled.
-* train.list and test.list: training and testing file-list (containing list of file names).
-* dict.txt: dictionary generated from training set.
-* labels.list: class label, 0 stands for negative while 1 for positive.
+# PaddlePaddle init
+paddle.init(use_gpu=False, trainer_count=1)
+```
 
-### Data Provider for PaddlePaddle
-PaddlePaddle can read Python-style script for configuration. The following `dataprovider.py` provides a detailed example, consisting of two parts:
+As alluded to in section [Model Overview](#model-overview), here we provide the implementations of both Text CNN and Stacked-bidirectional LSTM models.
 
-* hook: define text information and class Id. Texts are defined as `integer_value_sequence` while class Ids are defined as `integer_value`.
-* process: read line by line for ID and text information split by `’\t\t’`, and yield the data as a generator.
+### Text Convolution Neural Network (Text CNN)
 
-```python
-from paddle.trainer.PyDataProvider2 import *
+We create a neural network `convolution_net` as the following snippet code.
 
-def hook(settings, dictionary, **kwargs):
-settings.word_dict = dictionary
-settings.input_types = {
-'word':  integer_value_sequence(len(settings.word_dict)),
-'label': integer_value(2)
-}
-settings.logger.info('dict len : %d' % (len(settings.word_dict)))
-
-@provider(init_hook=hook)
-def process(settings, file_name):
-with open(file_name, 'r') as fdata:
-for line_count, line in enumerate(fdata):
-label, comment = line.strip().split('\t\t')
-label = int(label)
-words = comment.split()
-word_slot = [
-settings.word_dict[w] for w in words if w in settings.word_dict
-]
-yield {
-'word': word_slot,
-'label': label
-}
-```
+Note: `paddle.networks.sequence_conv_pool` includes both convolution and pooling layer operations.
 
-## Model Setup
-`trainer_config.py` is an example of a setup file.
-### Data Definition
 ```python
-from os.path import join as join_path
-from paddle.trainer_config_helpers import *
-# if it is “test” mode
-is_test = get_config_arg('is_test', bool, False)
-# if it is “predict” mode
-is_predict = get_config_arg('is_predict', bool, False)
-
-# Data path
-data_dir = "./data/pre-imdb"
-# File names
-train_list = "train.list"
-test_list = "test.list"
-dict_file = "dict.txt"
-
-# Dictionary size
-dict_dim = len(open(join_path(data_dir, "dict.txt")).readlines())
-# class number
-class_dim = len(open(join_path(data_dir, 'labels.list')).readlines())
-
-if not is_predict:
-train_list = join_path(data_dir, train_list)
-test_list = join_path(data_dir, test_list)
-dict_file = join_path(data_dir, dict_file)
-train_list = train_list if not is_test else None
-# construct the dictionary
-word_dict = dict()
-with open(dict_file, 'r') as f:
-for i, line in enumerate(open(dict_file, 'r')):
-word_dict[line.split('\t')[0]] = i
-# Call the function “define_py_data_sources2” in the file dataprovider.py to extract features
-define_py_data_sources2(
-train_list,
-test_list,
-module="dataprovider",
-obj="process",  # function to generate data
-args={'dictionary': word_dict}) # extra parameters, here refers to dictionary
+def convolution_net(input_dim, class_dim=2, emb_dim=128, hid_dim=128):
+    data = paddle.layer.data("word",
+                             paddle.data_type.integer_value_sequence(input_dim))
+    emb = paddle.layer.embedding(input=data, size=emb_dim)
+    conv_3 = paddle.networks.sequence_conv_pool(
+        input=emb, context_len=3, hidden_size=hid_dim)
+    conv_4 = paddle.networks.sequence_conv_pool(
+        input=emb, context_len=4, hidden_size=hid_dim)
+    output = paddle.layer.fc(input=[conv_3, conv_4],
+                             size=class_dim,
+                             act=paddle.activation.Softmax())
+    lbl = paddle.layer.data("label", paddle.data_type.integer_value(2))
+    cost = paddle.layer.classification_cost(input=output, label=lbl)
+    return cost
 ```
 
-### Algorithm Setup
+1. Define input data and its dimension
 
-```python
-settings(
-batch_size=128,
-learning_rate=2e-3,
-learning_method=AdamOptimizer(),
-regularization=L2Regularization(8e-4),
-gradient_clipping_threshold=25)
-```
+    Parameter `input_dim` denotes the dictionary size, and `class_dim` is the number of categories. In `convolution_net`, the input to the network is defined in `paddle.layer.data`.
 
-* Batch size set as 128;
-* Set global learning rate;
-* Apply ADAM algorithm for optimization;
-* Set up L2 regularization;
-* Set up gradient clipping threshold;
+1. Define Classifier
 
-### Model Structure
-We use PaddlePaddle to implement two classification algorithms, based on above mentioned model [Text-CNN](#Text-CNN（CNN）)和[Stacked-bidirectional LSTM](#Stacked-bidirectional LSTM（Stacked Bidirectional LSTM）)。
-#### Implementation of Text CNN
-```python
-def convolution_net(input_dim,
-class_dim=2,
-emb_dim=128,
-hid_dim=128,
-is_predict=False):
-# network input: id denotes word order, dictionary size as input_dim
-data = data_layer("word", input_dim)
-# Embed one-hot id to embedding subspace
-emb = embedding_layer(input=data, size=emb_dim)
-# Convolution and max-pooling operation, convolution kernel size set as 3
-conv_3 = sequence_conv_pool(input=emb, context_len=3, hidden_size=hid_dim)
-# Convolution and max-pooling, convolution kernel size set as 4
-conv_4 = sequence_conv_pool(input=emb, context_len=4, hidden_size=hid_dim)
-# Concatenate conv_3 and conv_4 as input for softmax classification, class number as class_dim
-output = fc_layer(
-input=[conv_3, conv_4], size=class_dim, act=SoftmaxActivation())
-
-if not is_predict:
-lbl = data_layer("label", 1)    #network input: class label
-outputs(classification_cost(input=output, label=lbl))
-else:
-outputs(output)
-```
+    The above Text CNN network extracts high-level features and maps them to a vector of the same size as the categories. `paddle.activation.Softmax` function or classifier is then used for calculating the probability of the sentence belonging to each category.
+
+1. Define Loss Function
 
-In our implementation, we can use just a single layer [`sequence_conv_pool`](https://github.com/PaddlePaddle/Paddle/blob/develop/python/paddle/trainer_config_helpers/networks.py) to do convolution and pooling operation, convolution kernel size set as hidden_size parameters.
+    In the context of supervised learning, labels of the training set are defined in `paddle.layer.data`, too. During training, cross-entropy is used as loss function in `paddle.layer.classification_cost` and as the output of the network; During testing, the outputs are the probabilities calculated in the classifier.
 
-#### Implementation of Stacked bidirectional LSTM
+#### Stacked bidirectional LSTM
+
+We create a neural network `stacked_lstm_net` as below.
 
 ```python
 def stacked_lstm_net(input_dim,
-class_dim=2,
-emb_dim=128,
-hid_dim=512,
-stacked_num=3,
-is_predict=False):
-
-# layer number of LSTM “stacked_num” is an odd number to confirm the top-layer LSTM is forward
-assert stacked_num % 2 == 1
-# network attributes setup
-layer_attr = ExtraLayerAttribute(drop_rate=0.5)
-# parameter attributes setup
-fc_para_attr = ParameterAttribute(learning_rate=1e-3)
-lstm_para_attr = ParameterAttribute(initial_std=0., learning_rate=1.)
-para_attr = [fc_para_attr, lstm_para_attr]
-bias_attr = ParameterAttribute(initial_std=0., l2_rate=0.)
-# Activation functions
-relu = ReluActivation()
-linear = LinearActivation()
-
-
-# Network input: id as word order, dictionary size is set as input_dim
-data = data_layer("word", input_dim)
-# Mapping id from word to the embedding subspace
-emb = embedding_layer(input=data, size=emb_dim)
-
-fc1 = fc_layer(input=emb, size=hid_dim, act=linear, bias_attr=bias_attr)
-# LSTM-based RNN
-lstm1 = lstmemory(
-input=fc1, act=relu, bias_attr=bias_attr, layer_attr=layer_attr)
-
-# Construct stacked bidirectional LSTM with fc_layer and lstmemory with layer depth as stacked_num:
-inputs = [fc1, lstm1]
-for i in range(2, stacked_num + 1):
-fc = fc_layer(
-input=inputs,
-size=hid_dim,
-act=linear,
-param_attr=para_attr,
-bias_attr=bias_attr)
-lstm = lstmemory(
-input=fc,
-# Odd number-th layer: forward, Even number-th reverse.
-reverse=(i % 2) == 0,
-act=relu,
-bias_attr=bias_attr,
-layer_attr=layer_attr)
-inputs = [fc, lstm]
-
-# Apply max-pooling along the temporal dimension on the last fc_layer to produce a fixed length vector
-fc_last = pooling_layer(input=inputs[0], pooling_type=MaxPooling())
-# Apply max-pooling along tempoeral dim of lstmemory to obtain fixed length feature vector
-lstm_last = pooling_layer(input=inputs[1], pooling_type=MaxPooling())
-# concatenate fc_last and lstm_last as input for a softmax classification layer, with class number equals class_dim
-output = fc_layer(
-input=[fc_last, lstm_last],
-size=class_dim,
-act=SoftmaxActivation(),
-bias_attr=bias_attr,
-param_attr=para_attr)
-
-if is_predict:
-outputs(output)
-else:
-outputs(classification_cost(input=output, label=data_layer('label', 1)))
+                     class_dim=2,
+                     emb_dim=128,
+                     hid_dim=512,
+                     stacked_num=3):
+    """
+    A Wrapper for sentiment classification task.
+    This network uses bi-directional recurrent network,
+    consisting three LSTM layers. This configure is referred to
+    the paper as following url, but use fewer layrs.
+        http://www.aclweb.org/anthology/P15-1109
+    input_dim: here is word dictionary dimension.
+    class_dim: number of categories.
+    emb_dim: dimension of word embedding.
+    hid_dim: dimension of hidden layer.
+    stacked_num: number of stacked lstm-hidden layer.
+    """
+    assert stacked_num % 2 == 1
+
+    layer_attr = paddle.attr.Extra(drop_rate=0.5)
+    fc_para_attr = paddle.attr.Param(learning_rate=1e-3)
+    lstm_para_attr = paddle.attr.Param(initial_std=0., learning_rate=1.)
+    para_attr = [fc_para_attr, lstm_para_attr]
+    bias_attr = paddle.attr.Param(initial_std=0., l2_rate=0.)
+    relu = paddle.activation.Relu()
+    linear = paddle.activation.Linear()
+
+    data = paddle.layer.data("word",
+                             paddle.data_type.integer_value_sequence(input_dim))
+    emb = paddle.layer.embedding(input=data, size=emb_dim)
+
+    fc1 = paddle.layer.fc(input=emb,
+                          size=hid_dim,
+                          act=linear,
+                          bias_attr=bias_attr)
+    lstm1 = paddle.layer.lstmemory(
+        input=fc1, act=relu, bias_attr=bias_attr, layer_attr=layer_attr)
+
+    inputs = [fc1, lstm1]
+    for i in range(2, stacked_num + 1):
+        fc = paddle.layer.fc(input=inputs,
+                             size=hid_dim,
+                             act=linear,
+                             param_attr=para_attr,
+                             bias_attr=bias_attr)
+        lstm = paddle.layer.lstmemory(
+            input=fc,
+            reverse=(i % 2) == 0,
+            act=relu,
+            bias_attr=bias_attr,
+            layer_attr=layer_attr)
+        inputs = [fc, lstm]
+
+    fc_last = paddle.layer.pooling(
+        input=inputs[0], pooling_type=paddle.pooling.Max())
+    lstm_last = paddle.layer.pooling(
+        input=inputs[1], pooling_type=paddle.pooling.Max())
+    output = paddle.layer.fc(input=[fc_last, lstm_last],
+                             size=class_dim,
+                             act=paddle.activation.Softmax(),
+                             bias_attr=bias_attr,
+                             param_attr=para_attr)
+
+    lbl = paddle.layer.data("label", paddle.data_type.integer_value(2))
+    cost = paddle.layer.classification_cost(input=output, label=lbl)
+    return cost
 ```
 
-Our model defined in `trainer_config.py` uses the `stacked_lstm_net` structure as default. If you want to use `convolution_net`, you can comment related lines.
+1. Define input data and its dimension
+
+    Parameter `input_dim` denotes the dictionary size, and `class_dim` is the number of categories. In `stacked_lstm_net`, the input to the network is defined in `paddle.layer.data`.
+
+1. Define Classifier
+
+    The above stacked bidirectional LSTM network extracts high-level features and maps them to a vector of the same size as the categories. `paddle.activation.Softmax` function or classifier is then used for calculating the probability of the sentence belonging to each category.
+
+1. Define Loss Function
+
+    In the context of supervised learning, labels of the training set are defined in `paddle.layer.data`, too. During training, cross-entropy is used as loss function in `paddle.layer.classification_cost` and as the output of the network; During testing, the outputs are the probabilities calculated in the classifier.
+
+
+To reiterate, we can either invoke `convolution_net` or `stacked_lstm_net`.
 
 ```python
-stacked_lstm_net(
-dict_dim, class_dim=class_dim, stacked_num=3, is_predict=is_predict)
-# convolution_net(dict_dim, class_dim=class_dim, is_predict=is_predict)
+word_dict = paddle.dataset.imdb.word_dict()
+dict_dim = len(word_dict)
+class_dim = 2
+
+# option 1
+cost = convolution_net(dict_dim, class_dim=class_dim)
+# option 2
+# cost = stacked_lstm_net(dict_dim, class_dim=class_dim, stacked_num=3)
 ```
 
 ## Model Training
-Use `train.sh` script to run local training:
 
-```
-./train.sh
-```
+### Define Parameters
 
-train.sh is as following:
-
-```bash
-paddle train --config=trainer_config.py \
---save_dir=./model_output \
---job=train \
---use_gpu=false \
---trainer_count=4 \
---num_passes=10 \
---log_period=20 \
---dot_period=20 \
---show_parameter_stats_period=100 \
---test_all_data_in_one_period=1 \
-2>&1 | tee 'train.log'
+First, we create the model parameters according to the previous model configuration `cost`.
+
+```python
+# create parameters
+parameters = paddle.parameters.create(cost)
 ```
 
-* \--config=trainer_config.py: set up model configuration.
-* \--save\_dir=./model_output: set up output folder to save model parameters.
-* \--job=train: set job mode as training.
-* \--use\_gpu=false: Use CPU for training. If you have installed GPU-version PaddlePaddle and want to try GPU training, you may set this term as true.
-* \--trainer\_count=4: setup thread number (or GPU numer）.
-* \--num\_passes=15: Setup pass. In PaddlePaddle, a pass means a training epoch over all samples.
-* \--log\_period=20: print log every 20 batches.
-* \--show\_parameter\_stats\_period=100: Print statistics to screen every 100 batch.
-* \--test\_all_data\_in\_one\_period=1: Predict all testing data every time.
+### Create Trainer
 
-If it is running sussefully, the output log will be saved at `train.log`, model parameters will be saved at the directory `model_output/`. Output log will be as following:
+Before jumping into creating a training module, algorithm setting is also necessary.
+Here we specified `Adam` optimization algorithm via `paddle.optimizer`.
 
+```python
+# create optimizer
+adam_optimizer = paddle.optimizer.Adam(
+    learning_rate=2e-3,
+    regularization=paddle.optimizer.L2Regularization(rate=8e-4),
+    model_average=paddle.optimizer.ModelAverage(average_window=0.5))
+
+# create trainer
+trainer = paddle.trainer.SGD(cost=cost,
+                                parameters=parameters,
+                                update_equation=adam_optimizer)
 ```
-Batch=20 samples=2560 AvgCost=0.681644 CurrentCost=0.681644 Eval: classification_error_evaluator=0.36875  CurrentEval: classification_error_evaluator=0.36875
-...
-Pass=0 Batch=196 samples=25000 AvgCost=0.418964 Eval: classification_error_evaluator=0.1922
-Test samples=24999 cost=0.39297 Eval: classification_error_evaluator=0.149406
-```
-
-* Batch=xx: Already |xx| Batch trained.
-* samples=xx: xx samples have been processed during training.
-* AvgCost=xx: Average loss from 0-th batch to the current batch.
-* CurrentCost=xx: loss of the latest |log_period|-th batch;
-* Eval: classification\_error\_evaluator=xx: Average accuracy from 0-th batch to current batch;
-* CurrentEval: classification\_error\_evaluator: latest |log_period| batches of classification error;
-* Pass=0: Running over all data in the training set is called as a Pass. Pass “0” denotes the first round.
 
+### Training
 
-## Application models
-### Testing
+`paddle.dataset.imdb.train()` will yield records during each pass, after shuffling, a batch input is generated for training.
 
-Testing refers to use trained model to evaluate labeled dataset.
+```python
+train_reader = paddle.batch(
+    paddle.reader.shuffle(
+        lambda: paddle.dataset.imdb.train(word_dict), buf_size=1000),
+    batch_size=100)
 
+test_reader = paddle.batch(
+    lambda: paddle.dataset.imdb.test(word_dict), batch_size=100)
 ```
-./test.sh
-```
-
-Scripts for testing `test.sh` is as following, where the function `get_best_pass` ranks classification accuracy to obtain the best model:
 
-```bash
-function get_best_pass() {
-cat $1  | grep -Pzo 'Test .*\n.*pass-.*' | \
-sed  -r 'N;s/Test.* error=([0-9]+\.[0-9]+).*\n.*pass-([0-9]+)/\1 \2/g' | \
-sort | head -n 1
-}
+`feeding` is devoted to specifying the correspondence between each yield record and `paddle.layer.data`. For instance, the first column of data generated by `paddle.dataset.imdb.train()` corresponds to `word` feature.
 
-log=train.log
-LOG=`get_best_pass $log`
-LOG=(${LOG})
-evaluate_pass="model_output/pass-${LOG[1]}"
-
-echo 'evaluating from pass '$evaluate_pass
-
-model_list=./model.list
-touch $model_list | echo $evaluate_pass > $model_list
-net_conf=trainer_config.py
-paddle train --config=$net_conf \
---model_list=$model_list \
---job=test \
---use_gpu=false \
---trainer_count=4 \
---config_args=is_test=1 \
-2>&1 | tee 'test.log'
+```python
+feeding = {'word': 0, 'label': 1}
 ```
 
-Different from training, testing requires denoting `--job = test` and model path `--model_list = $model_list`. If successful, log will be saved at `test.log`. In our test, the best model is `model_output/pass-00002`, with classification error rate as 0.115645：
+Callback function `event_handler` will be invoked to track training progress when a pre-defined event happens.
 
-```
-Pass=0 samples=24999 AvgCost=0.280471 Eval: classification_error_evaluator=0.115645
+```python
+def event_handler(event):
+    if isinstance(event, paddle.event.EndIteration):
+        if event.batch_id % 100 == 0:
+            print "\nPass %d, Batch %d, Cost %f, %s" % (
+                event.pass_id, event.batch_id, event.cost, event.metrics)
+        else:
+            sys.stdout.write('.')
+            sys.stdout.flush()
+    if isinstance(event, paddle.event.EndPass):
+        result = trainer.test(reader=test_reader, reader_dict=reader_dict)
+        print "\nTest with Pass %d, %s" % (event.pass_id, result.metrics)
 ```
 
-### Prediction
-`predict.py` script provides an API. Predicting IMDB data without labels as following:
+Finally, we can invoke `trainer.train` to start training:
 
+```python
+trainer.train(
+    reader=train_reader,
+    event_handler=event_handler,
+    feeding=feedig,
+    num_passes=10)
 ```
-./predict.sh
-```
-predict.sh is as following（default model path `model_output/pass-00002` may exist or modified to others）:
-
-```bash
-model=model_output/pass-00002/
-config=trainer_config.py
-label=data/pre-imdb/labels.list
-cat ./data/aclImdb/test/pos/10007_10.txt | python predict.py \
---tconf=$config \
---model=$model \
---label=$label \
---dict=./data/pre-imdb/dict.txt \
---batch_size=1
-```
-
-* `cat ./data/aclImdb/test/pos/10007_10.txt` : Input prediction samples.
-* `predict.py` : Prediction script.
-* `--tconf=$config` : Network set up.
-* `--model=$model` : Model path set up.
-* `--label=$label` : set up the label dictionary, mapping integer IDs to string labels.
-* `--dict=data/pre-imdb/dict.txt` : set up the dictionary file.
-* `--batch_size=1` : batch size during prediction.
 
 
-Prediction result of our example:
+## Conclusion
 
-```
-Loading parameters from model_output/pass-00002/
-predicting label is pos
-```
+In this chapter, we use sentiment analysis as an example to introduce applying deep learning models on end-to-end short text classification, as well as how to use PaddlePaddle to implement the model. Meanwhile, we briefly introduce two models for text processing: CNN and RNN. In following chapters, we will see how these models can be applied in other tasks.
 
-`10007_10.txt` in folder`./data/aclImdb/test/pos`, the predicted label is also pos，so the prediction is correct.
-## Summary
-In this chapter, we use sentiment analysis as an example to introduce applying deep learning models on end-to-end short text classification, as well as how to use PaddlePaddle to implement the model. Meanwhile, we briefly introduce two models for text processing: CNN and RNN. In following chapters we will see how these models can be applied in other tasks.
 ## Reference
+
 1. Kim Y. [Convolutional neural networks for sentence classification](http://arxiv.org/pdf/1408.5882)[J]. arXiv preprint arXiv:1408.5882, 2014.
 2. Kalchbrenner N, Grefenstette E, Blunsom P. [A convolutional neural network for modelling sentences](http://arxiv.org/pdf/1404.2188.pdf?utm_medium=App.net&utm_source=PourOver)[J]. arXiv preprint arXiv:1404.2188, 2014.
 3. Yann N. Dauphin, et al. [Language Modeling with Gated Convolutional Networks](https://arxiv.org/pdf/1612.08083v1.pdf)[J] arXiv preprint arXiv:1612.08083, 2016.
@@ -532,7 +408,7 @@ In this chapter, we use sentiment analysis as an example to introduce applying d
 9. Zhou J, Xu W. [End-to-end learning of semantic role labeling using recurrent neural networks](http://www.aclweb.org/anthology/P/P15/P15-1109.pdf)[C]//Proceedings of the Annual Meeting of the Association for Computational Linguistics. 2015.
 
 <br/>
-<a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/4.0/"><img alt="知识共享许可协议" style="border-width:0" src="https://i.creativecommons.org/l/by-nc-sa/4.0/88x31.png" /></a><br /><span xmlns:dct="http://purl.org/dc/terms/" href="http://purl.org/dc/dcmitype/Text" property="dct:title" rel="dct:type">本教程</span> 由 <a xmlns:cc="http://creativecommons.org/ns#" href="http://book.paddlepaddle.org" property="cc:attributionName" rel="cc:attributionURL">PaddlePaddle</a> 创作，采用 <a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/4.0/">知识共享 署名-非商业性使用-相同方式共享 4.0 国际 许可协议</a>进行许可。
+This tutorial is contributed by <a xmlns:cc="http://creativecommons.org/ns#" href="http://book.paddlepaddle.org" property="cc:attributionName" rel="cc:attributionURL">PaddlePaddle</a>, and licensed under a <a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/4.0/">Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License</a>.
 
 </div>
 <!-- You can change the lines below now. -->

understand_sentiment/index.html

@@ -42,7 +42,7 @@
 <div id="markdown" style='display:none'>
 # 情感分析
 
-本教程源代码目录在[book/understand_sentiment](https://github.com/PaddlePaddle/book/tree/develop/understand_sentiment)， 初次使用请参考PaddlePaddle[安装教程](http://www.paddlepaddle.org/doc_cn/build_and_install/index.html)。
+本教程源代码目录在[book/understand_sentiment](https://github.com/PaddlePaddle/book/tree/develop/understand_sentiment)， 初次使用请参考PaddlePaddle[安装教程](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/getstarted/build_and_install/docker_install_cn.rst)。
 
 ## 背景介绍
 在自然语言处理中，情感分析一般是指判断一段文本所表达的情绪状态。其中，一段文本可以是一个句子，一个段落或一个文档。情绪状态可以是两类，如（正面，负面），（高兴，悲伤）；也可以是三类，如（积极，消极，中性）等等。情感分析的应用场景十分广泛，如把用户在购物网站（亚马逊、天猫、淘宝等）、旅游网站、电影评论网站上发表的评论分成正面评论和负面评论；或为了分析用户对于某一产品的整体使用感受，抓取产品的用户评论并进行情感分析等等。表格1展示了对电影评论进行情感分析的例子：
@@ -150,14 +150,14 @@ aclImdb
 ```
 Paddle在`dataset/imdb.py`中提实现了imdb数据集的自动下载和读取，并提供了读取字典、训练数据、测试数据等API。
 
-```
+```python
 import sys
 import paddle.v2 as paddle
 ```
 ## 配置模型
 在该示例中，我们实现了两种文本分类算法，分别基于上文所述的[文本卷积神经网络](#文本卷积神经网络（CNN）)和[栈式双向LSTM](#栈式双向LSTM（Stacked Bidirectional LSTM）)。
 ### 文本卷积神经网络
-```
+```python
 def convolution_net(input_dim,
                     class_dim=2,
                     emb_dim=128,
@@ -178,7 +178,7 @@ def convolution_net(input_dim,
 ```
 网络的输入`input_dim`表示的是词典的大小，`class_dim`表示类别数。这里，我们使用[`sequence_conv_pool`](https://github.com/PaddlePaddle/Paddle/blob/develop/python/paddle/trainer_config_helpers/networks.py) API实现了卷积和池化操作。
 ### 栈式双向LSTM
-```
+```python
 def stacked_lstm_net(input_dim,
                      class_dim=2,
                      emb_dim=128,
@@ -247,7 +247,7 @@ def stacked_lstm_net(input_dim,
 ```
 网络的输入`stacked_num`表示的是LSTM的层数，需要是奇数，确保最高层LSTM正向。Paddle里面是通过一个fc和一个lstmemory来实现基于LSTM的循环神经网络。
 ## 训练模型
-```
+```python
 if __name__ == '__main__':
     # init
     paddle.init(use_gpu=False)
@@ -255,14 +255,14 @@ if __name__ == '__main__':
 启动paddle程序，use_gpu=False表示用CPU训练，如果系统支持GPU也可以修改成True使用GPU训练。
 ### 训练数据
 使用Paddle提供的数据集`dataset.imdb`中的API来读取训练数据。
-```
+```python
     print 'load dictionary...'
     word_dict = paddle.dataset.imdb.word_dict()
     dict_dim = len(word_dict)
     class_dim = 2
 ```
 加载数据字典，这里通过`word_dict()`API可以直接构造字典。`class_dim`是指样本类别数，该示例中样本只有正负两类。
-```
+```python
     train_reader = paddle.batch(
         paddle.reader.shuffle(
             lambda: paddle.dataset.imdb.train(word_dict), buf_size=1000),
@@ -272,12 +272,12 @@ if __name__ == '__main__':
         batch_size=100)
 ```
 这里，`dataset.imdb.train()`和`dataset.imdb.test()`分别是`dataset.imdb`中的训练数据和测试数据API。`train_reader`在训练时使用，意义是将读取的训练数据进行shuffle后，组成一个batch数据。同理，`test_reader`是在测试的时候使用，将读取的测试数据组成一个batch。
-```
+```python
     feeding={'word': 0, 'label': 1}
 ```
 `feeding`用来指定`train_reader`和`test_reader`返回的数据与模型配置中data_layer的对应关系。这里表示reader返回的第0列数据对应`word`层，第1列数据对应`label`层。
 ### 构造模型
-```
+```python
     # Please choose the way to build the network
     # by uncommenting the corresponding line.
     cost = convolution_net(dict_dim, class_dim=class_dim)
@@ -285,13 +285,13 @@ if __name__ == '__main__':
 ```
 该示例中默认使用`convolution_net`网络，如果使用`stacked_lstm_net`网络，注释相应的行即可。其中cost是网络的优化目标，同时cost包含了整个网络的拓扑信息。
 ### 网络参数
-```
+```python
     # create parameters
     parameters = paddle.parameters.create(cost)
 ```
 根据网络的拓扑构造网络参数。这里parameters是整个网络的参数集。
 ### 优化算法
-```
+```python
     # create optimizer
     adam_optimizer = paddle.optimizer.Adam(
         learning_rate=2e-3,
@@ -301,7 +301,7 @@ if __name__ == '__main__':
 Paddle中提供了一系列优化算法的API，这里使用Adam优化算法。
 ### 训练
 可以通过`paddle.trainer.SGD`构造一个sgd trainer，并调用`trainer.train`来训练模型。
-```
+```python
     # End batch and end pass event handler
     def event_handler(event):
         if isinstance(event, paddle.event.EndIteration):
@@ -316,7 +316,7 @@ Paddle中提供了一系列优化算法的API，这里使用Adam优化算法。
             print "\nTest with Pass %d, %s" % (event.pass_id, result.metrics)
 ```
 可以通过给train函数传递一个`event_handler`来获取每个batch和每个pass结束的状态。比如构造如下一个`event_handler`可以在每100个batch结束后输出cost和error；在每个pass结束后调用`trainer.test`计算一遍测试集并获得当前模型在测试集上的error。
-```
+```python
     # create trainer
     trainer = paddle.trainer.SGD(cost=cost,
                                  parameters=parameters,
@@ -329,7 +329,7 @@ Paddle中提供了一系列优化算法的API，这里使用Adam优化算法。
         num_passes=2)
 ```
 程序运行之后的输出如下。
-```
+```text
 Pass 0, Batch 0, Cost 0.693721, {'classification_error_evaluator': 0.5546875}
 ...................................................................................................
 Pass 0, Batch 100, Cost 0.294321, {'classification_error_evaluator': 0.1015625}

understand_sentiment/train.py

@@ -24,9 +24,8 @@ def convolution_net(input_dim, class_dim=2, emb_dim=128, hid_dim=128):
         input=emb, context_len=3, hidden_size=hid_dim)
     conv_4 = paddle.networks.sequence_conv_pool(
         input=emb, context_len=4, hidden_size=hid_dim)
-    output = paddle.layer.fc(input=[conv_3, conv_4],
-                             size=class_dim,
-                             act=paddle.activation.Softmax())
+    output = paddle.layer.fc(
+        input=[conv_3, conv_4], size=class_dim, act=paddle.activation.Softmax())
     lbl = paddle.layer.data("label", paddle.data_type.integer_value(2))
     cost = paddle.layer.classification_cost(input=output, label=lbl)
     return cost
@@ -64,20 +63,19 @@ def stacked_lstm_net(input_dim,
                              paddle.data_type.integer_value_sequence(input_dim))
     emb = paddle.layer.embedding(input=data, size=emb_dim)
 
-    fc1 = paddle.layer.fc(input=emb,
-                          size=hid_dim,
-                          act=linear,
-                          bias_attr=bias_attr)
+    fc1 = paddle.layer.fc(
+        input=emb, size=hid_dim, act=linear, bias_attr=bias_attr)
     lstm1 = paddle.layer.lstmemory(
         input=fc1, act=relu, bias_attr=bias_attr, layer_attr=layer_attr)
 
     inputs = [fc1, lstm1]
     for i in range(2, stacked_num + 1):
-        fc = paddle.layer.fc(input=inputs,
-                             size=hid_dim,
-                             act=linear,
-                             param_attr=para_attr,
-                             bias_attr=bias_attr)
+        fc = paddle.layer.fc(
+            input=inputs,
+            size=hid_dim,
+            act=linear,
+            param_attr=para_attr,
+            bias_attr=bias_attr)
         lstm = paddle.layer.lstmemory(
             input=fc,
             reverse=(i % 2) == 0,
@@ -90,11 +88,12 @@ def stacked_lstm_net(input_dim,
         input=inputs[0], pooling_type=paddle.pooling.Max())
     lstm_last = paddle.layer.pooling(
         input=inputs[1], pooling_type=paddle.pooling.Max())
-    output = paddle.layer.fc(input=[fc_last, lstm_last],
-                             size=class_dim,
-                             act=paddle.activation.Softmax(),
-                             bias_attr=bias_attr,
-                             param_attr=para_attr)
+    output = paddle.layer.fc(
+        input=[fc_last, lstm_last],
+        size=class_dim,
+        act=paddle.activation.Softmax(),
+        bias_attr=bias_attr,
+        param_attr=para_attr)
 
     lbl = paddle.layer.data("label", paddle.data_type.integer_value(2))
     cost = paddle.layer.classification_cost(input=output, label=lbl)
@@ -148,9 +147,8 @@ if __name__ == '__main__':
             print "\nTest with Pass %d, %s" % (event.pass_id, result.metrics)
 
     # create trainer
-    trainer = paddle.trainer.SGD(cost=cost,
-                                 parameters=parameters,
-                                 update_equation=adam_optimizer)
+    trainer = paddle.trainer.SGD(
+        cost=cost, parameters=parameters, update_equation=adam_optimizer)
 
     trainer.train(
         reader=train_reader,

word2vec/README.en.md

@@ -2,7 +2,7 @@
 
 This is intended as a reference tutorial. The source code of this tutorial lives on [book/word2vec](https://github.com/PaddlePaddle/book/tree/develop/word2vec).
 
-For instructions on getting started with PaddlePaddle, see [PaddlePaddle installation guide](http://www.paddlepaddle.org/doc_cn/build_and_install/index.html).
+For instructions on getting started with PaddlePaddle, see [PaddlePaddle installation guide](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/getstarted/build_and_install/docker_install_en.rst).
 
 ## Background Introduction
 
@@ -149,19 +149,257 @@ The advantages of CBOW is that it smooths over the word embeddings of the contex
 
 As illustrated in the figure above, skip-gram model maps the word embedding of the given word onto $2n$ word embeddings (including $n$ words before and $n$ words after the given word), and then combine the classification loss of all those $2n$ words by softmax.
 
-## Data Preparation
+## Dataset
+
+We will use Peen Treebank (PTB) (Tomas Mikolov's pre-processed version) dataset. PTB is a small dataset, used in Recurrent Neural Network Language Modeling Toolkit\[[2](#reference)\]. Its statistics are as follows:
+
+<p align="center">
+<table>
+    <tr>
+        <td>training set</td>
+        <td>validation set</td>
+        <td>test set</td>
+    </tr>
+    <tr>
+        <td>ptb.train.txt</td>
+        <td>ptb.valid.txt</td>
+        <td>ptb.test.txt</td>
+    </tr>
+    <tr>
+        <td>42068 lines</td>
+        <td>3370 lines</td>
+        <td>3761 lines</td>
+    </tr>
+</table>
+</p>
+
+### Python Dataset Module
+
+We encapsulated the PTB Data Set in our Python module `paddle.dataset.imikolov`. This module can
+
+1. download the dataset to `~/.cache/paddle/dataset/imikolov`, if not yet, and
+2. [preprocesses](#preprocessing) the dataset.
+
+### Preprocessing
+
+We will be training a 5-gram model. Given five words in a window, we will predict the fifth word given the first four words.
+
+Beginning and end of a sentence have a special meaning, so we will add begin token `<s>` in the front of the sentence. And end token `<e>` in the end of the sentence. By moving the five word window in the sentence, data instances are generated.
+
+For example, the sentence "I have a dream that one day" generates five data instances:
+
+```text
+<s> I have a dream
+I have a dream that
+have a dream that one
+a dream that one day
+dream that one day <e>
+```
+
+At last, each data instance will be converted into an integer sequence according it's words' index inside the dictionary.
+
+## Training
+
+The neural network that we will be using is illustrated in the graph below:
 
-## Model Configuration
 <p align="center">
     <img src="image/ngram.en.png" width=400><br/>
     Figure 5. N-gram neural network model in model configuration
 </p>
 
+`word2vec/train.py` demonstrates training word2vec using PaddlePaddle:
+
+- Import packages.
+
+```python
+import math
+import paddle.v2 as paddle
+```
+
+- Configure parameter.
+
+```python
+embsize = 32 # word vector dimension
+hiddensize = 256 # hidden layer dimension
+N = 5 # train 5-gram
+```
+
+- Map the $n-1$ words $w_{t-n+1},...w_{t-1}$ before $w_t$ to a D-dimensional vector though matrix of dimention $|V|\times D$ (D=32 in this example).
+
+```python
+def wordemb(inlayer):
+    wordemb = paddle.layer.table_projection(
+        input=inlayer,
+        size=embsize,
+        param_attr=paddle.attr.Param(
+            name="_proj",
+            initial_std=0.001,
+            learning_rate=1,
+            l2_rate=0, ))
+    return wordemb
+```
+
+- Define name and type for input to data layer.
+
+```python
+paddle.init(use_gpu=False, trainer_count=3)
+word_dict = paddle.dataset.imikolov.build_dict()
+dict_size = len(word_dict)
+# Every layer takes integer value of range [0, dict_size)
+firstword = paddle.layer.data(
+    name="firstw", type=paddle.data_type.integer_value(dict_size))
+secondword = paddle.layer.data(
+    name="secondw", type=paddle.data_type.integer_value(dict_size))
+thirdword = paddle.layer.data(
+    name="thirdw", type=paddle.data_type.integer_value(dict_size))
+fourthword = paddle.layer.data(
+    name="fourthw", type=paddle.data_type.integer_value(dict_size))
+nextword = paddle.layer.data(
+    name="fifthw", type=paddle.data_type.integer_value(dict_size))
+
+Efirst = wordemb(firstword)
+Esecond = wordemb(secondword)
+Ethird = wordemb(thirdword)
+Efourth = wordemb(fourthword)
+```
+
+- Concatenate n-1 word embedding vectors into a single feature vector.
+
+```python
+contextemb = paddle.layer.concat(input=[Efirst, Esecond, Ethird, Efourth])
+```
+
+- Feature vector will go through a fully connected layer which outputs a hidden feature vector.
+
+```python
+hidden1 = paddle.layer.fc(input=contextemb,
+                          size=hiddensize,
+                          act=paddle.activation.Sigmoid(),
+                          layer_attr=paddle.attr.Extra(drop_rate=0.5),
+                          bias_attr=paddle.attr.Param(learning_rate=2),
+                          param_attr=paddle.attr.Param(
+                                initial_std=1. / math.sqrt(embsize * 8),
+                                learning_rate=1))
+```
+
+- Hidden feature vector will go through another fully conected layer, turn into a $|V|$ dimensional vector. At the same time softmax will be applied to get the probability of each word being generated.
+
+```python
+predictword = paddle.layer.fc(input=hidden1,
+                              size=dict_size,
+                              bias_attr=paddle.attr.Param(learning_rate=2),
+                              act=paddle.activation.Softmax())
+```
+
+- We will use cross-entropy cost function.
+
+```python
+cost = paddle.layer.classification_cost(input=predictword, label=nextword)
+```
+
+- Create parameter, optimizer and trainer.
+
+```python
+parameters = paddle.parameters.create(cost)
+adam_optimizer = paddle.optimizer.Adam(
+    learning_rate=3e-3,
+    regularization=paddle.optimizer.L2Regularization(8e-4))
+trainer = paddle.trainer.SGD(cost, parameters, adam_optimizer)
+```
+
+Next, we will begin the training process. `paddle.dataset.imikolov.train()` and `paddle.dataset.imikolov.test()` is our training set and test set. Both of the function will return a **reader**: In PaddlePaddle, reader is a python function which returns a Python iterator which output a single data instance at a time.
+
+`paddle.batch` takes reader as input, outputs a **batched reader**: In PaddlePaddle, a reader outputs a single data instance at a time but batched reader outputs a minibatch of data instances.
+
+```python
+import gzip
+
+def event_handler(event):
+    if isinstance(event, paddle.event.EndIteration):
+        if event.batch_id % 100 == 0:
+            print "Pass %d, Batch %d, Cost %f, %s" % (
+                event.pass_id, event.batch_id, event.cost, event.metrics)
+
+    if isinstance(event, paddle.event.EndPass):
+        result = trainer.test(
+                    paddle.batch(
+                        paddle.dataset.imikolov.test(word_dict, N), 32))
+        print "Pass %d, Testing metrics %s" % (event.pass_id, result.metrics)
+        with gzip.open("model_%d.tar.gz"%event.pass_id, 'w') as f:
+            parameters.to_tar(f)
+
+trainer.train(
+    paddle.batch(paddle.dataset.imikolov.train(word_dict, N), 32),
+    num_passes=100,
+    event_handler=event_handler)
+```
+
+`trainer.train` will start training, the output of `event_handler` will be similar to following:
+```text
+Pass 0, Batch 0, Cost 7.870579, {'classification_error_evaluator': 1.0}, Testing metrics {'classification_error_evaluator': 0.999591588973999}
+Pass 0, Batch 100, Cost 6.136420, {'classification_error_evaluator': 0.84375}, Testing metrics {'classification_error_evaluator': 0.8328699469566345}
+Pass 0, Batch 200, Cost 5.786797, {'classification_error_evaluator': 0.8125}, Testing metrics {'classification_error_evaluator': 0.8328542709350586}
+...
+```
+
+After 30 passes, we can get average error rate around 0.735611.
 
-## Model Training
 
 ## Model Application
 
+After the model is trained, we can load saved model parameters and uses it for other models. We can also use the parameters in applications.
+
+### Viewing Word Vector
+
+Parameters trained by PaddlePaddle can be viewed by `parameters.get()`. For example, we can check the word vector for word `apple`.
+
+```python
+embeddings = parameters.get("_proj").reshape(len(word_dict), embsize)
+
+print embeddings[word_dict['apple']]
+```
+
+```text
+[-0.38961065 -0.02392169 -0.00093231  0.36301503  0.13538605  0.16076435
+-0.0678709   0.1090285   0.42014077 -0.24119169 -0.31847557  0.20410083
+0.04910378  0.19021918 -0.0122014  -0.04099389 -0.16924137  0.1911236
+-0.10917275  0.13068172 -0.23079982  0.42699069 -0.27679482 -0.01472992
+0.2069038   0.09005053 -0.3282454   0.12717034 -0.24218646  0.25304323
+0.19072419 -0.24286366]
+```
+
+### Modifying Word Vector
+
+Word vectors (`embeddings`) that we get is a numpy array. We can modify this array and set it back to `parameters`.
+
+
+```python
+def modify_embedding(emb):
+    # Add your modification here.
+    pass
+
+modify_embedding(embeddings)
+parameters.set("_proj", embeddings)
+```
+
+### Calculating Cosine Similarity
+
+Cosine similarity is one way of quantifying the similarity between two vectors. The range of result is $[-1, 1]$. The bigger the value, the similar two vectors are:
+
+
+```python
+from scipy import spatial
+
+emb_1 = embeddings[word_dict['world']]
+emb_2 = embeddings[word_dict['would']]
+
+print spatial.distance.cosine(emb_1, emb_2)
+```
+
+```text
+0.99375076448
+```
+
 ## Conclusion
 
 This chapter introduces word embedding, the relationship between language model and word embedding, and how to train neural networks to learn word embedding.
@@ -177,4 +415,4 @@ In information retrieval, the relevance between the query and document keyword c
 5. https://en.wikipedia.org/wiki/Singular_value_decomposition
 
 <br/>
-<a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/4.0/"><img alt="知识共享许可协议" style="border-width:0" src="https://i.creativecommons.org/l/by-nc-sa/4.0/88x31.png" /></a><br /><span xmlns:dct="http://purl.org/dc/terms/" href="http://purl.org/dc/dcmitype/Text" property="dct:title" rel="dct:type">本教程</span> 由 <a xmlns:cc="http://creativecommons.org/ns#" href="http://book.paddlepaddle.org" property="cc:attributionName" rel="cc:attributionURL">PaddlePaddle</a> 创作，采用 <a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/4.0/">知识共享 署名-非商业性使用-相同方式共享 4.0 国际 许可协议</a>进行许可。
+This tutorial is contributed by <a xmlns:cc="http://creativecommons.org/ns#" href="http://book.paddlepaddle.org" property="cc:attributionName" rel="cc:attributionURL">PaddlePaddle</a>, and licensed under a <a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/4.0/">Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License</a>.

word2vec/README.md

 
 # 词向量
 
-本教程源代码目录在[book/word2vec](https://github.com/PaddlePaddle/book/tree/develop/word2vec)， 初次使用请参考PaddlePaddle[安装教程](http://www.paddlepaddle.org/doc_cn/build_and_install/index.html)。
+本教程源代码目录在[book/word2vec](https://github.com/PaddlePaddle/book/tree/develop/word2vec)， 初次使用请参考PaddlePaddle[安装教程](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/getstarted/build_and_install/docker_install_cn.rst)。
 
 ## 背景介绍
 
@@ -32,8 +32,8 @@ $$X = USV^T$$
 本章中，当词向量训练好后，我们可以用数据可视化算法t-SNE\[[4](#参考文献)\]画出词语特征在二维上的投影（如下图所示）。从图中可以看出，语义相关的词语（如a, the, these; big, huge）在投影上距离很近，语意无关的词（如say, business; decision, japan）在投影上的距离很远。
 
 <p align="center">
-	<img src = "image/2d_similarity.png" width=400><br/>
-	图1. 词向量的二维投影
+    <img src = "image/2d_similarity.png" width=400><br/>
+    图1. 词向量的二维投影
 </p>
 
 另一方面，我们知道两个向量的余弦值在$[-1,1]$的区间内：两个完全相同的向量余弦值为1, 两个相互垂直的向量之间余弦值为0，两个方向完全相反的向量余弦值为-1，即相关性和余弦值大小成正比。因此我们还可以计算两个词向量的余弦相似度:
@@ -86,8 +86,8 @@ $$\frac{1}{T}\sum_t f(w_t, w_{t-1}, ..., w_{t-n+1};\theta) + R(\theta)$$
 其中$f(w_t, w_{t-1}, ..., w_{t-n+1})$表示根据历史n-1个词得到当前词$w_t$的条件概率，$R(\theta)$表示参数正则项。
 
 <p align="center">
-   	<img src="image/nnlm.png" width=500><br/>
-   	图2. N-gram神经网络模型
+       <img src="image/nnlm.png" width=500><br/>
+       图2. N-gram神经网络模型
 </p>
 
 图2展示了N-gram神经网络模型，从下往上看，该模型分为以下几个部分：
@@ -97,7 +97,7 @@ $$\frac{1}{T}\sum_t f(w_t, w_{t-1}, ..., w_{t-n+1};\theta) + R(\theta)$$
 
  - 然后所有词语的词向量连接成一个大向量，并经过一个非线性映射得到历史词语的隐层表示：
 
-	$$g=Utanh(\theta^Tx + b_1) + Wx + b_2$$
+    $$g=Utanh(\theta^Tx + b_1) + Wx + b_2$$
 
     其中，$x$为所有词语的词向量连接成的大向量，表示文本历史特征；$\theta$、$U$、$b_1$、$b_2$和$W$分别为词向量层到隐层连接的参数。$g$表示未经归一化的所有输出单词概率，$g_i$表示未经归一化的字典中第$i$个单词的输出概率。
 
@@ -118,8 +118,8 @@ $$\frac{1}{T}\sum_t f(w_t, w_{t-1}, ..., w_{t-n+1};\theta) + R(\theta)$$
 CBOW模型通过一个词的上下文（各N个词）预测当前词。当N=2时，模型如下图所示：
 
 <p align="center">
-	<img src="image/cbow.png" width=250><br/>
-	图3. CBOW模型
+    <img src="image/cbow.png" width=250><br/>
+    图3. CBOW模型
 </p>
 
 具体来说，不考虑上下文的词语输入顺序，CBOW是用上下文词语的词向量的均值来预测当前词。即：
@@ -133,8 +133,8 @@ $$context = \frac{x_{t-1} + x_{t-2} + x_{t+1} + x_{t+2}}{4}$$
 CBOW的好处是对上下文词语的分布在词向量上进行了平滑，去掉了噪声，因此在小数据集上很有效。而Skip-gram的方法中，用一个词预测其上下文，得到了当前词上下文的很多样本，因此可用于更大的数据集。
 
 <p align="center">
-	<img src="image/skipgram.png" width=250><br/>
-	图4. Skip-gram模型
+    <img src="image/skipgram.png" width=250><br/>
+    图4. Skip-gram模型
 </p>
 
 如上图所示，Skip-gram模型的具体做法是，将一个词的词向量映射到$2n$个词的词向量（$2n$表示当前输入词的前后各$n$个词），然后分别通过softmax得到这$2n$个词的分类损失值之和。
@@ -144,25 +144,25 @@ CBOW的好处是对上下文词语的分布在词向量上进行了平滑，去
 
 ### 数据介绍
 
-本教程使用Penn Tree Bank (PTB)数据集。PTB数据集较小，训练速度快，应用于Mikolov的公开语言模型训练工具\[[2](#参考文献)\]中。其统计情况如下：
+本教程使用Penn Treebank （PTB）（经Tomas Mikolov预处理过的版本）数据集。PTB数据集较小，训练速度快，应用于Mikolov的公开语言模型训练工具\[[2](#参考文献)\]中。其统计情况如下：
 
 <p align="center">
 <table>
-	<tr>
-		<td>训练数据</td>
-		<td>验证数据</td>
-		<td>测试数据</td>
-	</tr>
-	<tr>
-		<td>ptb.train.txt</td>
-		<td>ptb.valid.txt</td>
-		<td>ptb.test.txt</td>
-	</tr>
-	<tr>
-		<td>42068句</td>
-		<td>3370句</td>
-		<td>3761句</td>
-	</tr>
+    <tr>
+        <td>训练数据</td>
+        <td>验证数据</td>
+        <td>测试数据</td>
+    </tr>
+    <tr>
+        <td>ptb.train.txt</td>
+        <td>ptb.valid.txt</td>
+        <td>ptb.test.txt</td>
+    </tr>
+    <tr>
+        <td>42068句</td>
+        <td>3370句</td>
+        <td>3761句</td>
+    </tr>
 </table>
 </p>
 
@@ -183,13 +183,14 @@ a dream that one day
 dream that one day <e>
 ```
 
+最后，每个输入会按其单词次在字典里的位置，转化成整数的索引序列，作为PaddlePaddle的输入。
 ## 编程实现
 
 本配置的模型结构如下图所示：
 
 <p align="center">
-	<img src="image/ngram.png" width=400><br/>
-	图5. 模型配置中的N-gram神经网络模型
+    <img src="image/ngram.png" width=400><br/>
+    图5. 模型配置中的N-gram神经网络模型
 </p>
 
 首先，加载所需要的包：
@@ -245,7 +246,6 @@ Efirst = wordemb(firstword)
 Esecond = wordemb(secondword)
 Ethird = wordemb(thirdword)
 Efourth = wordemb(fourthword)
-
 ```
 
 - 将这n-1个词向量经过concat_layer连接成一个大向量作为历史文本特征。
@@ -323,11 +323,12 @@ trainer.train(
     event_handler=event_handler)
 ```
 
-    ...
-    Pass 0, Batch 25000, Cost 4.251861, {'classification_error_evaluator': 0.84375}
-    Pass 0, Batch 25100, Cost 4.847692, {'classification_error_evaluator': 0.8125}
-    Pass 0, Testing metrics {'classification_error_evaluator': 0.7417652606964111}
-
+```text
+Pass 0, Batch 0, Cost 7.870579, {'classification_error_evaluator': 1.0}, Testing metrics {'classification_error_evaluator': 0.999591588973999}
+Pass 0, Batch 100, Cost 6.136420, {'classification_error_evaluator': 0.84375}, Testing metrics {'classification_error_evaluator': 0.8328699469566345}
+Pass 0, Batch 200, Cost 5.786797, {'classification_error_evaluator': 0.8125}, Testing metrics {'classification_error_evaluator': 0.8328542709350586}
+...
+```
 
 训练过程是完全自动的，event_handler里打印的日志类似如上所示：
 
@@ -340,22 +341,23 @@ trainer.train(
 
 ### 查看词向量
 
-PaddlePaddle训练出来的参数可以直接使用`parameters.get()`获取出来。例如查看单词的word的词向量，即为
+PaddlePaddle训练出来的参数可以直接使用`parameters.get()`获取出来。例如查看单词`apple`的词向量，即为
 
 
 ```python
 embeddings = parameters.get("_proj").reshape(len(word_dict), embsize)
 
-print embeddings[word_dict['word']]
+print embeddings[word_dict['apple']]
 ```
 
-    [-0.38961065 -0.02392169 -0.00093231  0.36301503  0.13538605  0.16076435
-     -0.0678709   0.1090285   0.42014077 -0.24119169 -0.31847557  0.20410083
-      0.04910378  0.19021918 -0.0122014  -0.04099389 -0.16924137  0.1911236
-     -0.10917275  0.13068172 -0.23079982  0.42699069 -0.27679482 -0.01472992
-      0.2069038   0.09005053 -0.3282454   0.12717034 -0.24218646  0.25304323
-      0.19072419 -0.24286366]
-
+```text
+[-0.38961065 -0.02392169 -0.00093231  0.36301503  0.13538605  0.16076435
+-0.0678709   0.1090285   0.42014077 -0.24119169 -0.31847557  0.20410083
+0.04910378  0.19021918 -0.0122014  -0.04099389 -0.16924137  0.1911236
+-0.10917275  0.13068172 -0.23079982  0.42699069 -0.27679482 -0.01472992
+0.2069038   0.09005053 -0.3282454   0.12717034 -0.24218646  0.25304323
+0.19072419 -0.24286366]
+```
 
 
 ### 修改词向量
@@ -387,8 +389,9 @@ emb_2 = embeddings[word_dict['would']]
 print spatial.distance.cosine(emb_1, emb_2)
 ```
 
-    0.99375076448
-
+```text
+0.99375076448
+```
 
 ## 总结
 本章中，我们介绍了词向量、语言模型和词向量的关系、以及如何通过训练神经网络模型获得词向量。在信息检索中，我们可以根据向量间的余弦夹角，来判断query和文档关键词这二者间的相关性。在句法分析和语义分析中，训练好的词向量可以用来初始化模型，以得到更好的效果。在文档分类中，有了词向量之后，可以用聚类的方法将文档中同义词进行分组。希望大家在本章后能够自行运用词向量进行相关领域的研究。

word2vec/index.en.html

@@ -44,7 +44,7 @@
 
 This is intended as a reference tutorial. The source code of this tutorial lives on [book/word2vec](https://github.com/PaddlePaddle/book/tree/develop/word2vec).
 
-For instructions on getting started with PaddlePaddle, see [PaddlePaddle installation guide](http://www.paddlepaddle.org/doc_cn/build_and_install/index.html).
+For instructions on getting started with PaddlePaddle, see [PaddlePaddle installation guide](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/getstarted/build_and_install/docker_install_en.rst).
 
 ## Background Introduction
 
@@ -191,19 +191,257 @@ The advantages of CBOW is that it smooths over the word embeddings of the contex
 
 As illustrated in the figure above, skip-gram model maps the word embedding of the given word onto $2n$ word embeddings (including $n$ words before and $n$ words after the given word), and then combine the classification loss of all those $2n$ words by softmax.
 
-## Data Preparation
+## Dataset
+
+We will use Peen Treebank (PTB) (Tomas Mikolov's pre-processed version) dataset. PTB is a small dataset, used in Recurrent Neural Network Language Modeling Toolkit\[[2](#reference)\]. Its statistics are as follows:
+
+<p align="center">
+<table>
+    <tr>
+        <td>training set</td>
+        <td>validation set</td>
+        <td>test set</td>
+    </tr>
+    <tr>
+        <td>ptb.train.txt</td>
+        <td>ptb.valid.txt</td>
+        <td>ptb.test.txt</td>
+    </tr>
+    <tr>
+        <td>42068 lines</td>
+        <td>3370 lines</td>
+        <td>3761 lines</td>
+    </tr>
+</table>
+</p>
+
+### Python Dataset Module
+
+We encapsulated the PTB Data Set in our Python module `paddle.dataset.imikolov`. This module can
+
+1. download the dataset to `~/.cache/paddle/dataset/imikolov`, if not yet, and
+2. [preprocesses](#preprocessing) the dataset.
+
+### Preprocessing
+
+We will be training a 5-gram model. Given five words in a window, we will predict the fifth word given the first four words.
+
+Beginning and end of a sentence have a special meaning, so we will add begin token `<s>` in the front of the sentence. And end token `<e>` in the end of the sentence. By moving the five word window in the sentence, data instances are generated.
+
+For example, the sentence "I have a dream that one day" generates five data instances:
+
+```text
+<s> I have a dream
+I have a dream that
+have a dream that one
+a dream that one day
+dream that one day <e>
+```
+
+At last, each data instance will be converted into an integer sequence according it's words' index inside the dictionary.
+
+## Training
+
+The neural network that we will be using is illustrated in the graph below:
 
-## Model Configuration
 <p align="center">
     <img src="image/ngram.en.png" width=400><br/>
     Figure 5. N-gram neural network model in model configuration
 </p>
 
+`word2vec/train.py` demonstrates training word2vec using PaddlePaddle:
+
+- Import packages.
+
+```python
+import math
+import paddle.v2 as paddle
+```
+
+- Configure parameter.
+
+```python
+embsize = 32 # word vector dimension
+hiddensize = 256 # hidden layer dimension
+N = 5 # train 5-gram
+```
+
+- Map the $n-1$ words $w_{t-n+1},...w_{t-1}$ before $w_t$ to a D-dimensional vector though matrix of dimention $|V|\times D$ (D=32 in this example).
+
+```python
+def wordemb(inlayer):
+    wordemb = paddle.layer.table_projection(
+        input=inlayer,
+        size=embsize,
+        param_attr=paddle.attr.Param(
+            name="_proj",
+            initial_std=0.001,
+            learning_rate=1,
+            l2_rate=0, ))
+    return wordemb
+```
+
+- Define name and type for input to data layer.
+
+```python
+paddle.init(use_gpu=False, trainer_count=3)
+word_dict = paddle.dataset.imikolov.build_dict()
+dict_size = len(word_dict)
+# Every layer takes integer value of range [0, dict_size)
+firstword = paddle.layer.data(
+    name="firstw", type=paddle.data_type.integer_value(dict_size))
+secondword = paddle.layer.data(
+    name="secondw", type=paddle.data_type.integer_value(dict_size))
+thirdword = paddle.layer.data(
+    name="thirdw", type=paddle.data_type.integer_value(dict_size))
+fourthword = paddle.layer.data(
+    name="fourthw", type=paddle.data_type.integer_value(dict_size))
+nextword = paddle.layer.data(
+    name="fifthw", type=paddle.data_type.integer_value(dict_size))
+
+Efirst = wordemb(firstword)
+Esecond = wordemb(secondword)
+Ethird = wordemb(thirdword)
+Efourth = wordemb(fourthword)
+```
+
+- Concatenate n-1 word embedding vectors into a single feature vector.
+
+```python
+contextemb = paddle.layer.concat(input=[Efirst, Esecond, Ethird, Efourth])
+```
+
+- Feature vector will go through a fully connected layer which outputs a hidden feature vector.
+
+```python
+hidden1 = paddle.layer.fc(input=contextemb,
+                          size=hiddensize,
+                          act=paddle.activation.Sigmoid(),
+                          layer_attr=paddle.attr.Extra(drop_rate=0.5),
+                          bias_attr=paddle.attr.Param(learning_rate=2),
+                          param_attr=paddle.attr.Param(
+                                initial_std=1. / math.sqrt(embsize * 8),
+                                learning_rate=1))
+```
+
+- Hidden feature vector will go through another fully conected layer, turn into a $|V|$ dimensional vector. At the same time softmax will be applied to get the probability of each word being generated.
+
+```python
+predictword = paddle.layer.fc(input=hidden1,
+                              size=dict_size,
+                              bias_attr=paddle.attr.Param(learning_rate=2),
+                              act=paddle.activation.Softmax())
+```
+
+- We will use cross-entropy cost function.
+
+```python
+cost = paddle.layer.classification_cost(input=predictword, label=nextword)
+```
+
+- Create parameter, optimizer and trainer.
+
+```python
+parameters = paddle.parameters.create(cost)
+adam_optimizer = paddle.optimizer.Adam(
+    learning_rate=3e-3,
+    regularization=paddle.optimizer.L2Regularization(8e-4))
+trainer = paddle.trainer.SGD(cost, parameters, adam_optimizer)
+```
+
+Next, we will begin the training process. `paddle.dataset.imikolov.train()` and `paddle.dataset.imikolov.test()` is our training set and test set. Both of the function will return a **reader**: In PaddlePaddle, reader is a python function which returns a Python iterator which output a single data instance at a time.
+
+`paddle.batch` takes reader as input, outputs a **batched reader**: In PaddlePaddle, a reader outputs a single data instance at a time but batched reader outputs a minibatch of data instances.
+
+```python
+import gzip
+
+def event_handler(event):
+    if isinstance(event, paddle.event.EndIteration):
+        if event.batch_id % 100 == 0:
+            print "Pass %d, Batch %d, Cost %f, %s" % (
+                event.pass_id, event.batch_id, event.cost, event.metrics)
+
+    if isinstance(event, paddle.event.EndPass):
+        result = trainer.test(
+                    paddle.batch(
+                        paddle.dataset.imikolov.test(word_dict, N), 32))
+        print "Pass %d, Testing metrics %s" % (event.pass_id, result.metrics)
+        with gzip.open("model_%d.tar.gz"%event.pass_id, 'w') as f:
+            parameters.to_tar(f)
+
+trainer.train(
+    paddle.batch(paddle.dataset.imikolov.train(word_dict, N), 32),
+    num_passes=100,
+    event_handler=event_handler)
+```
+
+`trainer.train` will start training, the output of `event_handler` will be similar to following:
+```text
+Pass 0, Batch 0, Cost 7.870579, {'classification_error_evaluator': 1.0}, Testing metrics {'classification_error_evaluator': 0.999591588973999}
+Pass 0, Batch 100, Cost 6.136420, {'classification_error_evaluator': 0.84375}, Testing metrics {'classification_error_evaluator': 0.8328699469566345}
+Pass 0, Batch 200, Cost 5.786797, {'classification_error_evaluator': 0.8125}, Testing metrics {'classification_error_evaluator': 0.8328542709350586}
+...
+```
+
+After 30 passes, we can get average error rate around 0.735611.
 
-## Model Training
 
 ## Model Application
 
+After the model is trained, we can load saved model parameters and uses it for other models. We can also use the parameters in applications.
+
+### Viewing Word Vector
+
+Parameters trained by PaddlePaddle can be viewed by `parameters.get()`. For example, we can check the word vector for word `apple`.
+
+```python
+embeddings = parameters.get("_proj").reshape(len(word_dict), embsize)
+
+print embeddings[word_dict['apple']]
+```
+
+```text
+[-0.38961065 -0.02392169 -0.00093231  0.36301503  0.13538605  0.16076435
+-0.0678709   0.1090285   0.42014077 -0.24119169 -0.31847557  0.20410083
+0.04910378  0.19021918 -0.0122014  -0.04099389 -0.16924137  0.1911236
+-0.10917275  0.13068172 -0.23079982  0.42699069 -0.27679482 -0.01472992
+0.2069038   0.09005053 -0.3282454   0.12717034 -0.24218646  0.25304323
+0.19072419 -0.24286366]
+```
+
+### Modifying Word Vector
+
+Word vectors (`embeddings`) that we get is a numpy array. We can modify this array and set it back to `parameters`.
+
+
+```python
+def modify_embedding(emb):
+    # Add your modification here.
+    pass
+
+modify_embedding(embeddings)
+parameters.set("_proj", embeddings)
+```
+
+### Calculating Cosine Similarity
+
+Cosine similarity is one way of quantifying the similarity between two vectors. The range of result is $[-1, 1]$. The bigger the value, the similar two vectors are:
+
+
+```python
+from scipy import spatial
+
+emb_1 = embeddings[word_dict['world']]
+emb_2 = embeddings[word_dict['would']]
+
+print spatial.distance.cosine(emb_1, emb_2)
+```
+
+```text
+0.99375076448
+```
+
 ## Conclusion
 
 This chapter introduces word embedding, the relationship between language model and word embedding, and how to train neural networks to learn word embedding.
@@ -219,7 +457,7 @@ In information retrieval, the relevance between the query and document keyword c
 5. https://en.wikipedia.org/wiki/Singular_value_decomposition
 
 <br/>
-<a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/4.0/"><img alt="知识共享许可协议" style="border-width:0" src="https://i.creativecommons.org/l/by-nc-sa/4.0/88x31.png" /></a><br /><span xmlns:dct="http://purl.org/dc/terms/" href="http://purl.org/dc/dcmitype/Text" property="dct:title" rel="dct:type">本教程</span> 由 <a xmlns:cc="http://creativecommons.org/ns#" href="http://book.paddlepaddle.org" property="cc:attributionName" rel="cc:attributionURL">PaddlePaddle</a> 创作，采用 <a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/4.0/">知识共享 署名-非商业性使用-相同方式共享 4.0 国际 许可协议</a>进行许可。
+This tutorial is contributed by <a xmlns:cc="http://creativecommons.org/ns#" href="http://book.paddlepaddle.org" property="cc:attributionName" rel="cc:attributionURL">PaddlePaddle</a>, and licensed under a <a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/4.0/">Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License</a>.
 
 </div>
 <!-- You can change the lines below now. -->

word2vec/index.html

@@ -43,7 +43,7 @@
 
 # 词向量
 
-本教程源代码目录在[book/word2vec](https://github.com/PaddlePaddle/book/tree/develop/word2vec)， 初次使用请参考PaddlePaddle[安装教程](http://www.paddlepaddle.org/doc_cn/build_and_install/index.html)。
+本教程源代码目录在[book/word2vec](https://github.com/PaddlePaddle/book/tree/develop/word2vec)， 初次使用请参考PaddlePaddle[安装教程](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/getstarted/build_and_install/docker_install_cn.rst)。
 
 ## 背景介绍
 
@@ -74,8 +74,8 @@ $$X = USV^T$$
 本章中，当词向量训练好后，我们可以用数据可视化算法t-SNE\[[4](#参考文献)\]画出词语特征在二维上的投影（如下图所示）。从图中可以看出，语义相关的词语（如a, the, these; big, huge）在投影上距离很近，语意无关的词（如say, business; decision, japan）在投影上的距离很远。
 
 <p align="center">
-	<img src = "image/2d_similarity.png" width=400><br/>
-	图1. 词向量的二维投影
+    <img src = "image/2d_similarity.png" width=400><br/>
+    图1. 词向量的二维投影
 </p>
 
 另一方面，我们知道两个向量的余弦值在$[-1,1]$的区间内：两个完全相同的向量余弦值为1, 两个相互垂直的向量之间余弦值为0，两个方向完全相反的向量余弦值为-1，即相关性和余弦值大小成正比。因此我们还可以计算两个词向量的余弦相似度:
@@ -128,8 +128,8 @@ $$\frac{1}{T}\sum_t f(w_t, w_{t-1}, ..., w_{t-n+1};\theta) + R(\theta)$$
 其中$f(w_t, w_{t-1}, ..., w_{t-n+1})$表示根据历史n-1个词得到当前词$w_t$的条件概率，$R(\theta)$表示参数正则项。
 
 <p align="center">
-   	<img src="image/nnlm.png" width=500><br/>
-   	图2. N-gram神经网络模型
+       <img src="image/nnlm.png" width=500><br/>
+       图2. N-gram神经网络模型
 </p>
 
 图2展示了N-gram神经网络模型，从下往上看，该模型分为以下几个部分：
@@ -139,7 +139,7 @@ $$\frac{1}{T}\sum_t f(w_t, w_{t-1}, ..., w_{t-n+1};\theta) + R(\theta)$$
 
  - 然后所有词语的词向量连接成一个大向量，并经过一个非线性映射得到历史词语的隐层表示：
 
-	$$g=Utanh(\theta^Tx + b_1) + Wx + b_2$$
+    $$g=Utanh(\theta^Tx + b_1) + Wx + b_2$$
 
     其中，$x$为所有词语的词向量连接成的大向量，表示文本历史特征；$\theta$、$U$、$b_1$、$b_2$和$W$分别为词向量层到隐层连接的参数。$g$表示未经归一化的所有输出单词概率，$g_i$表示未经归一化的字典中第$i$个单词的输出概率。
 
@@ -160,8 +160,8 @@ $$\frac{1}{T}\sum_t f(w_t, w_{t-1}, ..., w_{t-n+1};\theta) + R(\theta)$$
 CBOW模型通过一个词的上下文（各N个词）预测当前词。当N=2时，模型如下图所示：
 
 <p align="center">
-	<img src="image/cbow.png" width=250><br/>
-	图3. CBOW模型
+    <img src="image/cbow.png" width=250><br/>
+    图3. CBOW模型
 </p>
 
 具体来说，不考虑上下文的词语输入顺序，CBOW是用上下文词语的词向量的均值来预测当前词。即：
@@ -175,8 +175,8 @@ $$context = \frac{x_{t-1} + x_{t-2} + x_{t+1} + x_{t+2}}{4}$$
 CBOW的好处是对上下文词语的分布在词向量上进行了平滑，去掉了噪声，因此在小数据集上很有效。而Skip-gram的方法中，用一个词预测其上下文，得到了当前词上下文的很多样本，因此可用于更大的数据集。
 
 <p align="center">
-	<img src="image/skipgram.png" width=250><br/>
-	图4. Skip-gram模型
+    <img src="image/skipgram.png" width=250><br/>
+    图4. Skip-gram模型
 </p>
 
 如上图所示，Skip-gram模型的具体做法是，将一个词的词向量映射到$2n$个词的词向量（$2n$表示当前输入词的前后各$n$个词），然后分别通过softmax得到这$2n$个词的分类损失值之和。
@@ -186,25 +186,25 @@ CBOW的好处是对上下文词语的分布在词向量上进行了平滑，去
 
 ### 数据介绍
 
-本教程使用Penn Tree Bank (PTB)数据集。PTB数据集较小，训练速度快，应用于Mikolov的公开语言模型训练工具\[[2](#参考文献)\]中。其统计情况如下：
+本教程使用Penn Treebank （PTB）（经Tomas Mikolov预处理过的版本）数据集。PTB数据集较小，训练速度快，应用于Mikolov的公开语言模型训练工具\[[2](#参考文献)\]中。其统计情况如下：
 
 <p align="center">
 <table>
-	<tr>
-		<td>训练数据</td>
-		<td>验证数据</td>
-		<td>测试数据</td>
-	</tr>
-	<tr>
-		<td>ptb.train.txt</td>
-		<td>ptb.valid.txt</td>
-		<td>ptb.test.txt</td>
-	</tr>
-	<tr>
-		<td>42068句</td>
-		<td>3370句</td>
-		<td>3761句</td>
-	</tr>
+    <tr>
+        <td>训练数据</td>
+        <td>验证数据</td>
+        <td>测试数据</td>
+    </tr>
+    <tr>
+        <td>ptb.train.txt</td>
+        <td>ptb.valid.txt</td>
+        <td>ptb.test.txt</td>
+    </tr>
+    <tr>
+        <td>42068句</td>
+        <td>3370句</td>
+        <td>3761句</td>
+    </tr>
 </table>
 </p>
 
@@ -225,13 +225,14 @@ a dream that one day
 dream that one day <e>
 ```
 
+最后，每个输入会按其单词次在字典里的位置，转化成整数的索引序列，作为PaddlePaddle的输入。
 ## 编程实现
 
 本配置的模型结构如下图所示：
 
 <p align="center">
-	<img src="image/ngram.png" width=400><br/>
-	图5. 模型配置中的N-gram神经网络模型
+    <img src="image/ngram.png" width=400><br/>
+    图5. 模型配置中的N-gram神经网络模型
 </p>
 
 首先，加载所需要的包：
@@ -287,7 +288,6 @@ Efirst = wordemb(firstword)
 Esecond = wordemb(secondword)
 Ethird = wordemb(thirdword)
 Efourth = wordemb(fourthword)
-
 ```
 
 - 将这n-1个词向量经过concat_layer连接成一个大向量作为历史文本特征。
@@ -365,11 +365,12 @@ trainer.train(
     event_handler=event_handler)
 ```
 
-    ...
-    Pass 0, Batch 25000, Cost 4.251861, {'classification_error_evaluator': 0.84375}
-    Pass 0, Batch 25100, Cost 4.847692, {'classification_error_evaluator': 0.8125}
-    Pass 0, Testing metrics {'classification_error_evaluator': 0.7417652606964111}
-
+```text
+Pass 0, Batch 0, Cost 7.870579, {'classification_error_evaluator': 1.0}, Testing metrics {'classification_error_evaluator': 0.999591588973999}
+Pass 0, Batch 100, Cost 6.136420, {'classification_error_evaluator': 0.84375}, Testing metrics {'classification_error_evaluator': 0.8328699469566345}
+Pass 0, Batch 200, Cost 5.786797, {'classification_error_evaluator': 0.8125}, Testing metrics {'classification_error_evaluator': 0.8328542709350586}
+...
+```
 
 训练过程是完全自动的，event_handler里打印的日志类似如上所示：
 
@@ -382,22 +383,23 @@ trainer.train(
 
 ### 查看词向量
 
-PaddlePaddle训练出来的参数可以直接使用`parameters.get()`获取出来。例如查看单词的word的词向量，即为
+PaddlePaddle训练出来的参数可以直接使用`parameters.get()`获取出来。例如查看单词`apple`的词向量，即为
 
 
 ```python
 embeddings = parameters.get("_proj").reshape(len(word_dict), embsize)
 
-print embeddings[word_dict['word']]
+print embeddings[word_dict['apple']]
 ```
 
-    [-0.38961065 -0.02392169 -0.00093231  0.36301503  0.13538605  0.16076435
-     -0.0678709   0.1090285   0.42014077 -0.24119169 -0.31847557  0.20410083
-      0.04910378  0.19021918 -0.0122014  -0.04099389 -0.16924137  0.1911236
-     -0.10917275  0.13068172 -0.23079982  0.42699069 -0.27679482 -0.01472992
-      0.2069038   0.09005053 -0.3282454   0.12717034 -0.24218646  0.25304323
-      0.19072419 -0.24286366]
-
+```text
+[-0.38961065 -0.02392169 -0.00093231  0.36301503  0.13538605  0.16076435
+-0.0678709   0.1090285   0.42014077 -0.24119169 -0.31847557  0.20410083
+0.04910378  0.19021918 -0.0122014  -0.04099389 -0.16924137  0.1911236
+-0.10917275  0.13068172 -0.23079982  0.42699069 -0.27679482 -0.01472992
+0.2069038   0.09005053 -0.3282454   0.12717034 -0.24218646  0.25304323
+0.19072419 -0.24286366]
+```
 
 
 ### 修改词向量
@@ -429,8 +431,9 @@ emb_2 = embeddings[word_dict['would']]
 print spatial.distance.cosine(emb_1, emb_2)
 ```
 
-    0.99375076448
-
+```text
+0.99375076448
+```
 
 ## 总结
 本章中，我们介绍了词向量、语言模型和词向量的关系、以及如何通过训练神经网络模型获得词向量。在信息检索中，我们可以根据向量间的余弦夹角，来判断query和文档关键词这二者间的相关性。在句法分析和语义分析中，训练好的词向量可以用来初始化模型，以得到更好的效果。在文档分类中，有了词向量之后，可以用聚类的方法将文档中同义词进行分组。希望大家在本章后能够自行运用词向量进行相关领域的研究。

word2vec/train.py

@@ -40,18 +40,19 @@ def main():
     Efourth = wordemb(fourthword)
 
     contextemb = paddle.layer.concat(input=[Efirst, Esecond, Ethird, Efourth])
-    hidden1 = paddle.layer.fc(input=contextemb,
-                              size=hiddensize,
-                              act=paddle.activation.Sigmoid(),
-                              layer_attr=paddle.attr.Extra(drop_rate=0.5),
-                              bias_attr=paddle.attr.Param(learning_rate=2),
-                              param_attr=paddle.attr.Param(
-                                  initial_std=1. / math.sqrt(embsize * 8),
-                                  learning_rate=1))
-    predictword = paddle.layer.fc(input=hidden1,
-                                  size=dict_size,
-                                  bias_attr=paddle.attr.Param(learning_rate=2),
-                                  act=paddle.activation.Softmax())
+    hidden1 = paddle.layer.fc(
+        input=contextemb,
+        size=hiddensize,
+        act=paddle.activation.Sigmoid(),
+        layer_attr=paddle.attr.Extra(drop_rate=0.5),
+        bias_attr=paddle.attr.Param(learning_rate=2),
+        param_attr=paddle.attr.Param(
+            initial_std=1. / math.sqrt(embsize * 8), learning_rate=1))
+    predictword = paddle.layer.fc(
+        input=hidden1,
+        size=dict_size,
+        bias_attr=paddle.attr.Param(learning_rate=2),
+        act=paddle.activation.Softmax())
 
     def event_handler(event):
         if isinstance(event, paddle.event.EndIteration):