Skip to content

B
book

PaddlePaddle / book

通知 17
Star 4
Fork 0
B
book
未验证 提交 63b5ec3f 编写于 作者: N Nicky Chan 提交者： GitHub
浏览文件
操作

Merge pull request #561 from PaddlePaddle/high-level-api-branch 

Merge High level api branch
上级 330cf3ee 21e1717c
展开全部 显示空白变更内容
内联 并排
Showing with 7222 addition and 5885 deletion
01.fit_a_line/README.cn.md
浏览文件 @ 63b5ec3f
...@@ -47,22 +47,6 @@ $$MSE=\frac{1}{n}\sum_{i=1}^{n}{(\hat{Y_i}-Y_i)}^2$$ ...@@ -47,22 +47,6 @@ $$MSE=\frac{1}{n}\sum_{i=1}^{n}{(\hat{Y_i}-Y_i)}^2$$
## 数据集 ## 数据集
### 数据集接口的封装
首先加载需要的包
```python
import paddle.v2 as paddle
import paddle.v2.dataset.uci_housing as uci_housing
```
我们通过uci_housing模块引入了数据集合[UCI Housing Data Set](https://archive.ics.uci.edu/ml/datasets/Housing)
其中，在uci_housing模块中封装了：
1. 数据下载的过程。下载数据保存在~/.cache/paddle/dataset/uci_housing/housing.data。
2. [数据预处理](#数据预处理)的过程。
### 数据集介绍 ### 数据集介绍
这份数据集共506行，每行包含了波士顿郊区的一类房屋的相关信息及该类房屋价格的中位数。其各维属性的意义如下： 这份数据集共506行，每行包含了波士顿郊区的一类房屋的相关信息及该类房屋价格的中位数。其各维属性的意义如下：
...@@ -110,157 +94,158 @@ import paddle.v2.dataset.uci_housing as uci_housing ...@@ -110,157 +94,158 @@ import paddle.v2.dataset.uci_housing as uci_housing
`fit_a_line/trainer.py`演示了训练的整体过程。 `fit_a_line/trainer.py`演示了训练的整体过程。
### 初始化PaddlePaddle ### 配置数据提供器(Datafeeder)
首先我们引入必要的库：
```python ```python
paddle.init(use_gpu=False, trainer_count=1) import paddle
import paddle.fluid as fluid
import numpy
``` ```
### 模型配置 我们通过uci_housing模块引入了数据集合[UCI Housing Data Set](https://archive.ics.uci.edu/ml/datasets/Housing)
线性回归的模型其实就是一个采用线性激活函数（linear activation，`LinearActivation`）的全连接层（fully-connected layer，`fc_layer`）： 其中，在uci_housing模块中封装了：
```python 1. 数据下载的过程。下载数据保存在~/.cache/paddle/dataset/uci_housing/housing.data。
x = paddle.layer.data(name='x', type=paddle.data_type.dense_vector(13)) 2. [数据预处理](#数据预处理)的过程。
y_predict = paddle.layer.fc(input=x,
size=1,
act=paddle.activation.Linear())
y = paddle.layer.data(name='y', type=paddle.data_type.dense_vector(1))
cost = paddle.layer.square_error_cost(input=y_predict, label=y)
```
### 保存网络拓扑 接下来我们定义了用于训练和测试的数据提供器。提供器每次读入一个大小为`BATCH_SIZE`的数据批次。如果用户希望加一些随机性，她可以同时定义一个批次大小和一个缓存大小。这样的话，每次数据提供器会从缓存中随机读取批次大小那么多的数据。
```python ```python
# Save the inference topology to protobuf. BATCH_SIZE = 20
inference_topology = paddle.topology.Topology(layers=y_predict)
with open("inference_topology.pkl", 'wb') as f:
inference_topology.serialize_for_inference(f)
```
### 创建参数 train_reader = paddle.batch(
paddle.reader.shuffle(
paddle.dataset.uci_housing.train(), buf_size=500),
batch_size=BATCH_SIZE)
```python test_reader = paddle.batch(
parameters = paddle.parameters.create(cost) paddle.reader.shuffle(
paddle.dataset.uci_housing.test(), buf_size=500),
batch_size=BATCH_SIZE)
``` ```
### 创建Trainer ### 配置训练程序
训练程序的目的是定义一个训练模型的网络结构。对于线性回归来讲，它就是一个从输入到输出的简单的全连接层。更加复杂的结果，比如卷积神经网络，递归神经网络等会在随后的章节中介绍。训练程序必须返回`平均损失`作为第一个返回值，因为它会被后面反向传播算法所用到。
```python ```python
optimizer = paddle.optimizer.Momentum(momentum=0) def train_program():
y = fluid.layers.data(name='y', shape=[1], dtype='float32')
# feature vector of length 13
x = fluid.layers.data(name='x', shape=[13], dtype='float32')
y_predict = fluid.layers.fc(input=x, size=1, act=None)
loss = fluid.layers.square_error_cost(input=y_predict, label=y)
avg_loss = fluid.layers.mean(loss)
trainer = paddle.trainer.SGD(cost=cost, return avg_loss
parameters=parameters,
update_equation=optimizer)
``` ```
### 读取数据且打印训练的中间信息 ### 定义运算场所
我们可以定义运算是发生在CPU还是GPU
PaddlePaddle提供一个 ```python
[reader机制](https://github.com/PaddlePaddle/Paddle/tree/develop/doc/design/reader) use_cuda = False
来读取数据。 Reader返回的数据可以包括多列，我们需要一个Python dict把列 place = fluid.CUDAPlace(0) if use_cuda else fluid.CPUPlace()
序号映射到网络里的数据层。 ```
### 创建训练器
训练器会读入一个训练程序和一些必要的其他参数：
```python ```python
feeding={'x': 0, 'y': 1} trainer = fluid.Trainer(
train_func=train_program,
place=place,
optimizer_func=fluid.optimizer.SGD(learning_rate=0.001))
``` ```
此外，我们还可以提供一个 event handler，来打印训练的进度： ### 开始提供数据
PaddlePaddle提供了读取数据者发生器机制来读取训练数据。读取数据者会一次提供多列数据，因此我们需要一个Python的list来定义读取顺序。
```python ```python
# event_handler to print training and testing info feed_order=['x', 'y']
def event_handler(event):
if isinstance(event, paddle.event.EndIteration):
if event.batch_id % 100 == 0:
print "Pass %d, Batch %d, Cost %f" % (
event.pass_id, event.batch_id, event.cost)
if isinstance(event, paddle.event.EndPass):
result = trainer.test(
reader=paddle.batch(
uci_housing.test(), batch_size=2),
feeding=feeding)
print "Test %d, Cost %f" % (event.pass_id, result.cost)
``` ```
除此之外，可以定义一个事件相应器来处理类似`打印训练进程`的事件：
```python ```python
# event_handler to print training and testing info # Specify the directory path to save the parameters
from paddle.v2.plot import Ploter params_dirname = "fit_a_line.inference.model"
# Plot data
from paddle.v2.plot import Ploter
train_title = "Train cost" train_title = "Train cost"
test_title = "Test cost" test_title = "Test cost"
cost_ploter = Ploter(train_title, test_title) plot_cost = Ploter(train_title, test_title)
step = 0 step = 0
# event_handler to print training and testing info
def event_handler_plot(event): def event_handler_plot(event):
global step global step
if isinstance(event, paddle.event.EndIteration): if isinstance(event, fluid.EndStepEvent):
if step % 10 == 0: # every 10 batches, record a train cost if event.step % 10 == 0: # every 10 batches, record a test cost
cost_ploter.append(train_title, step, event.cost) test_metrics = trainer.test(
reader=test_reader, feed_order=feed_order)
if step % 100 == 0: # every 100 batches, record a test cost plot_cost.append(test_title, step, test_metrics[0])
result = trainer.test( plot_cost.plot()
reader=paddle.batch(
uci_housing.test(), batch_size=2),
feeding=feeding)
cost_ploter.append(test_title, step, result.cost)
if step % 100 == 0: # every 100 batches, update cost plot if test_metrics[0] < 10.0:
cost_ploter.plot() # If the accuracy is good enough, we can stop the training.
print('loss is less than 10.0, stop')
trainer.stop()
step += 1 # We can save the trained parameters for the inferences later
if params_dirname is not None:
trainer.save_params(params_dirname)
if isinstance(event, paddle.event.EndPass): step += 1
if event.pass_id % 10 == 0:
with open('params_pass_%d.tar' % event.pass_id, 'w') as f:
trainer.save_parameter_to_tar(f)
``` ```
### 开始训练 ### 开始训练
我们现在可以通过调用`trainer.train()`来开始训练
```python ```python
%matplotlib inline
# The training could take up to a few minutes.
trainer.train( trainer.train(
reader=paddle.batch( reader=train_reader,
paddle.reader.shuffle( num_epochs=100,
uci_housing.train(), buf_size=500),
batch_size=2),
feeding=feeding,
event_handler=event_handler_plot, event_handler=event_handler_plot,
num_passes=30) feed_order=feed_order)
``` ```
![png](./image/train_and_test.png) ![png](./image/train_and_test.png)
### 应用模型 ## 预测
提供一个`inference_program`和一个`params_dirname`来初始化预测器。`params_dirname`用来存储我们的参数。
### 设定预测程序
类似于`trainer.train`，预测器需要一个预测程序来做预测。我们可以稍加修改我们的训练程序来把预测值包含进来。
#### 1. 生成测试数据
```python ```python
test_data_creator = paddle.dataset.uci_housing.test() def inference_program():
test_data = [] x = fluid.layers.data(name='x', shape=[13], dtype='float32')
test_label = [] y_predict = fluid.layers.fc(input=x, size=1, act=None)
return y_predict
for item in test_data_creator():
test_data.append((item[0],))
test_label.append(item[1])
if len(test_data) == 5:
break
``` ```
#### 2. 推测 inference ### 预测
预测器会从`params_dirname`中读取已经训练好的模型，来对从未遇见过的数据进行预测。
```python ```python
# load parameters from tar file. inferencer = fluid.Inferencer(
# users can remove the comments and change the model name infer_func=inference_program, param_path=params_dirname, place=place)
# with open('params_pass_20.tar', 'r') as f:
# parameters = paddle.parameters.Parameters.from_tar(f)
probs = paddle.infer( batch_size = 10
output_layer=y_predict, parameters=parameters, input=test_data) tensor_x = numpy.random.uniform(0, 10, [batch_size, 13]).astype("float32")
for i in xrange(len(probs)): results = inferencer.infer({'x': tensor_x})
print "label=" + str(test_label[i][0]) + ", predict=" + str(probs[i][0]) print("infer results: ", results[0])
``` ```
## 总结 ## 总结
......
01.fit_a_line/README.md
浏览文件 @ 63b5ec3f
...@@ -39,7 +39,7 @@ $$MSE=\frac{1}{n}\sum_{i=1}^{n}{(\hat{Y_i}-Y_i)}^2$$ ...@@ -39,7 +39,7 @@ $$MSE=\frac{1}{n}\sum_{i=1}^{n}{(\hat{Y_i}-Y_i)}^2$$
That is, for a dataset of size $n$, MSE is the average value of the the prediction sqaure errors. That is, for a dataset of size $n$, MSE is the average value of the the prediction sqaure errors.
### Training ### Training Process
After setting up our model, there are several major steps to go through to train it: After setting up our model, there are several major steps to go through to train it:
1. Initialize the parameters including the weights $\vec{\omega}$ and the bias $b$. For example, we can set their mean values as $0$s, and their standard deviations as $1$s. 1. Initialize the parameters including the weights $\vec{\omega}$ and the bias $b$. For example, we can set their mean values as $0$s, and their standard deviations as $1$s.
...@@ -48,21 +48,6 @@ After setting up our model, there are several major steps to go through to train ...@@ -48,21 +48,6 @@ After setting up our model, there are several major steps to go through to train
4. Repeat steps 2~3, until the loss is below a predefined threshold or the maximum number of epochs is reached. 4. Repeat steps 2~3, until the loss is below a predefined threshold or the maximum number of epochs is reached.
## Dataset ## Dataset
### Python Dataset Modules
Our program starts with importing necessary packages:
```python
import paddle.v2 as paddle
import paddle.v2.dataset.uci_housing as uci_housing
```
We encapsulated the [UCI Housing Data Set](https://archive.ics.uci.edu/ml/datasets/Housing) in our Python module `uci_housing`. This module can
1. download the dataset to `~/.cache/paddle/dataset/uci_housing/housing.data`, if you haven't yet, and
2. [preprocess](#preprocessing) the dataset.
### An Introduction of the Dataset ### An Introduction of the Dataset
The UCI housing dataset has 506 instances. Each instance describes the attributes of a house in surburban Boston. The attributes are explained below: The UCI housing dataset has 506 instances. Each instance describes the attributes of a house in surburban Boston. The attributes are explained below:
...@@ -116,49 +101,71 @@ When training complex models, we usually have one more split: the validation set ...@@ -116,49 +101,71 @@ When training complex models, we usually have one more split: the validation set
`fit_a_line/trainer.py` demonstrates the training using [PaddlePaddle](http://paddlepaddle.org). `fit_a_line/trainer.py` demonstrates the training using [PaddlePaddle](http://paddlepaddle.org).
### Initialize PaddlePaddle ### Datafeeder Configuration
Our program starts with importing necessary packages:
```python ```python
paddle.init(use_gpu=False, trainer_count=1) import paddle
import paddle.fluid as fluid
import numpy
``` ```
### Model Configuration We encapsulated the [UCI Housing Data Set](https://archive.ics.uci.edu/ml/datasets/Housing) in our Python module `uci_housing`. This module can
1. download the dataset to `~/.cache/paddle/dataset/uci_housing/housing.data`, if you haven't yet, and
2. [preprocess](#preprocessing) the dataset.
Linear regression is essentially a fully-connected layer with linear activation: We define data feeders for test and train. The feeder reads a `BATCH_SIZE` of data each time and feed them to the training/testing process. If the user wants some randomness on the data order, she can define both a `BATCH_SIZE` and a `buf_size`. That way the datafeeder will yield the first `BATCH_SIZE` data out of a shuffle of the first `buf_size` data.
```python ```python
x = paddle.layer.data(name='x', type=paddle.data_type.dense_vector(13)) BATCH_SIZE = 20
y_predict = paddle.layer.fc(input=x,
size=1, train_reader = paddle.batch(
act=paddle.activation.Linear()) paddle.reader.shuffle(
y = paddle.layer.data(name='y', type=paddle.data_type.dense_vector(1)) paddle.dataset.uci_housing.train(), buf_size=500),
cost = paddle.layer.square_error_cost(input=y_predict, label=y) batch_size=BATCH_SIZE)
test_reader = paddle.batch(
paddle.reader.shuffle(
paddle.dataset.uci_housing.test(), buf_size=500),
batch_size=BATCH_SIZE)
``` ```
### Save Topology ### Train Program Configuration
`train_program` sets up the network structure of this current training model. For linear regression, it is simply a fully connected layer from the input to the output. More complex structures like CNN and RNN will be introduced in later chapters. The `train_program` must return an avg_loss as its first returned parameter because it is needed in backpropagation.
```python ```python
# Save the inference topology to protobuf. def train_program():
inference_topology = paddle.topology.Topology(layers=y_predict) y = fluid.layers.data(name='y', shape=[1], dtype='float32')
with open("inference_topology.pkl", 'wb') as f:
inference_topology.serialize_for_inference(f) # feature vector of length 13
x = fluid.layers.data(name='x', shape=[13], dtype='float32')
y_predict = fluid.layers.fc(input=x, size=1, act=None)
loss = fluid.layers.square_error_cost(input=y_predict, label=y)
avg_loss = fluid.layers.mean(loss)
return avg_loss
``` ```
### Create Parameters ### Specify Place
Specify your training environment, you should specify if the training is on CPU or GPU.
```python ```python
parameters = paddle.parameters.create(cost) use_cuda = False
place = fluid.CUDAPlace(0) if use_cuda else fluid.CPUPlace()
``` ```
### Create Trainer ### Create Trainer
The trainer will take the `train_program` as input.
```python ```python
optimizer = paddle.optimizer.Momentum(momentum=0) trainer = fluid.Trainer(
train_func=train_program,
trainer = paddle.trainer.SGD(cost=cost, place=place,
parameters=parameters, optimizer_func=fluid.optimizer.SGD(learning_rate=0.001))
update_equation=optimizer)
``` ```
### Feeding Data ### Feeding Data
...@@ -168,105 +175,90 @@ PaddlePaddle provides the ...@@ -168,105 +175,90 @@ PaddlePaddle provides the
for loading the training data. A reader may return multiple columns, and we need a Python dictionary to specify the mapping from column index to data layers. for loading the training data. A reader may return multiple columns, and we need a Python dictionary to specify the mapping from column index to data layers.
```python ```python
feeding={'x': 0, 'y': 1} feed_order=['x', 'y']
``` ```
Moreover, an event handler is provided to print the training progress: Moreover, an event handler is provided to print the training progress:
```python ```python
# event_handler to print training and testing info # Specify the directory path to save the parameters
def event_handler(event): params_dirname = "fit_a_line.inference.model"
if isinstance(event, paddle.event.EndIteration):
if event.batch_id % 100 == 0:
print "Pass %d, Batch %d, Cost %f" % (
event.pass_id, event.batch_id, event.cost)
if isinstance(event, paddle.event.EndPass):
result = trainer.test(
reader=paddle.batch(
uci_housing.test(), batch_size=2),
feeding=feeding)
print "Test %d, Cost %f" % (event.pass_id, result.cost)
```
```python # Plot data
# event_handler to plot training and testing info
from paddle.v2.plot import Ploter from paddle.v2.plot import Ploter
train_title = "Train cost" train_title = "Train cost"
test_title = "Test cost" test_title = "Test cost"
plot_cost = Ploter(train_title, test_title) plot_cost = Ploter(train_title, test_title)
step = 0 step = 0
# event_handler to print training and testing info
def event_handler_plot(event): def event_handler_plot(event):
global step global step
if isinstance(event, paddle.event.EndIteration): if isinstance(event, fluid.EndStepEvent):
if step % 10 == 0: # every 10 batches, record a train cost if event.step % 10 == 0: # every 10 batches, record a test cost
plot_cost.append(train_title, step, event.cost) test_metrics = trainer.test(
reader=test_reader, feed_order=feed_order)
if step % 100 == 0: # every 100 batches, record a test cost
result = trainer.test( plot_cost.append(test_title, step, test_metrics[0])
reader=paddle.batch(
uci_housing.test(), batch_size=2),
feeding=feeding)
plot_cost.append(test_title, step, result.cost)
if step % 100 == 0: # every 100 batches, update cost plot
plot_cost.plot() plot_cost.plot()
step += 1 if test_metrics[0] < 10.0:
# If the accuracy is good enough, we can stop the training.
print('loss is less than 10.0, stop')
trainer.stop()
if isinstance(event, paddle.event.EndPass): # We can save the trained parameters for the inferences later
if event.pass_id % 10 == 0: if params_dirname is not None:
with open('params_pass_%d.tar' % event.pass_id, 'w') as f: trainer.save_params(params_dirname)
trainer.save_parameter_to_tar(f)
step += 1
``` ```
### Start Training ### Start Training
We now can start training by calling `trainer.train()`.
```python ```python
%matplotlib inline
# The training could take up to a few minutes.
trainer.train( trainer.train(
reader=paddle.batch( reader=train_reader,
paddle.reader.shuffle( num_epochs=100,
uci_housing.train(), buf_size=500),
batch_size=2),
feeding=feeding,
event_handler=event_handler_plot, event_handler=event_handler_plot,
num_passes=30) feed_order=feed_order)
``` ```
![png](./image/train_and_test.png) ![png](./image/train_and_test.png)
### Apply model ## Inference
Initialize the Inferencer with the inference_program and the params_dirname, which is where we saved our params
#### 1. generate testing data ### Setup the Inference Program
Similar to the trainer.train, the Inferencer needs to take an inference_program to do inference.
Prune the train_program to only have the y_predict.
```python ```python
test_data_creator = paddle.dataset.uci_housing.test() def inference_program():
test_data = [] x = fluid.layers.data(name='x', shape=[13], dtype='float32')
test_label = [] y_predict = fluid.layers.fc(input=x, size=1, act=None)
return y_predict
for item in test_data_creator():
test_data.append((item[0],))
test_label.append(item[1])
if len(test_data) == 5:
break
``` ```
#### 2. inference ### Infer
Inferencer will load the trained model from `params_dirname` and use it to infer the unseen data.
```python ```python
# load parameters from tar file. inferencer = fluid.Inferencer(
# users can remove the comments and change the model name infer_func=inference_program, param_path=params_dirname, place=place)
# with open('params_pass_20.tar', 'r') as f:
# parameters = paddle.parameters.Parameters.from_tar(f)
probs = paddle.infer( batch_size = 10
output_layer=y_predict, parameters=parameters, input=test_data) tensor_x = numpy.random.uniform(0, 10, [batch_size, 13]).astype("float32")
for i in xrange(len(probs)): results = inferencer.infer({'x': tensor_x})
print "label=" + str(test_label[i][0]) + ", predict=" + str(probs[i][0]) print("infer results: ", results[0])
``` ```
## Summary ## Summary
......
01.fit_a_line/index.html
浏览文件 @ 63b5ec3f
...@@ -81,7 +81,7 @@ $$MSE=\frac{1}{n}\sum_{i=1}^{n}{(\hat{Y_i}-Y_i)}^2$$ ...@@ -81,7 +81,7 @@ $$MSE=\frac{1}{n}\sum_{i=1}^{n}{(\hat{Y_i}-Y_i)}^2$$
That is, for a dataset of size $n$, MSE is the average value of the the prediction sqaure errors. That is, for a dataset of size $n$, MSE is the average value of the the prediction sqaure errors.
### Training ### Training Process
After setting up our model, there are several major steps to go through to train it: After setting up our model, there are several major steps to go through to train it:
1. Initialize the parameters including the weights $\vec{\omega}$ and the bias $b$. For example, we can set their mean values as $0$s, and their standard deviations as $1$s. 1. Initialize the parameters including the weights $\vec{\omega}$ and the bias $b$. For example, we can set their mean values as $0$s, and their standard deviations as $1$s.
...@@ -90,21 +90,6 @@ After setting up our model, there are several major steps to go through to train ...@@ -90,21 +90,6 @@ After setting up our model, there are several major steps to go through to train
4. Repeat steps 2~3, until the loss is below a predefined threshold or the maximum number of epochs is reached. 4. Repeat steps 2~3, until the loss is below a predefined threshold or the maximum number of epochs is reached.
## Dataset ## Dataset
### Python Dataset Modules
Our program starts with importing necessary packages:
```python
import paddle.v2 as paddle
import paddle.v2.dataset.uci_housing as uci_housing
```
We encapsulated the [UCI Housing Data Set](https://archive.ics.uci.edu/ml/datasets/Housing) in our Python module `uci_housing`. This module can
1. download the dataset to `~/.cache/paddle/dataset/uci_housing/housing.data`, if you haven't yet, and
2. [preprocess](#preprocessing) the dataset.
### An Introduction of the Dataset ### An Introduction of the Dataset
The UCI housing dataset has 506 instances. Each instance describes the attributes of a house in surburban Boston. The attributes are explained below: The UCI housing dataset has 506 instances. Each instance describes the attributes of a house in surburban Boston. The attributes are explained below:
...@@ -158,49 +143,71 @@ When training complex models, we usually have one more split: the validation set ...@@ -158,49 +143,71 @@ When training complex models, we usually have one more split: the validation set
`fit_a_line/trainer.py` demonstrates the training using [PaddlePaddle](http://paddlepaddle.org). `fit_a_line/trainer.py` demonstrates the training using [PaddlePaddle](http://paddlepaddle.org).
### Initialize PaddlePaddle ### Datafeeder Configuration
Our program starts with importing necessary packages:
```python ```python
paddle.init(use_gpu=False, trainer_count=1) import paddle
import paddle.fluid as fluid
import numpy
``` ```
### Model Configuration We encapsulated the [UCI Housing Data Set](https://archive.ics.uci.edu/ml/datasets/Housing) in our Python module `uci_housing`. This module can
1. download the dataset to `~/.cache/paddle/dataset/uci_housing/housing.data`, if you haven't yet, and
2. [preprocess](#preprocessing) the dataset.
Linear regression is essentially a fully-connected layer with linear activation: We define data feeders for test and train. The feeder reads a `BATCH_SIZE` of data each time and feed them to the training/testing process. If the user wants some randomness on the data order, she can define both a `BATCH_SIZE` and a `buf_size`. That way the datafeeder will yield the first `BATCH_SIZE` data out of a shuffle of the first `buf_size` data.
```python ```python
x = paddle.layer.data(name='x', type=paddle.data_type.dense_vector(13)) BATCH_SIZE = 20
y_predict = paddle.layer.fc(input=x,
size=1, train_reader = paddle.batch(
act=paddle.activation.Linear()) paddle.reader.shuffle(
y = paddle.layer.data(name='y', type=paddle.data_type.dense_vector(1)) paddle.dataset.uci_housing.train(), buf_size=500),
cost = paddle.layer.square_error_cost(input=y_predict, label=y) batch_size=BATCH_SIZE)
test_reader = paddle.batch(
paddle.reader.shuffle(
paddle.dataset.uci_housing.test(), buf_size=500),
batch_size=BATCH_SIZE)
``` ```
### Save Topology ### Train Program Configuration
`train_program` sets up the network structure of this current training model. For linear regression, it is simply a fully connected layer from the input to the output. More complex structures like CNN and RNN will be introduced in later chapters. The `train_program` must return an avg_loss as its first returned parameter because it is needed in backpropagation.
```python ```python
# Save the inference topology to protobuf. def train_program():
inference_topology = paddle.topology.Topology(layers=y_predict) y = fluid.layers.data(name='y', shape=[1], dtype='float32')
with open("inference_topology.pkl", 'wb') as f:
inference_topology.serialize_for_inference(f) # feature vector of length 13
x = fluid.layers.data(name='x', shape=[13], dtype='float32')
y_predict = fluid.layers.fc(input=x, size=1, act=None)
loss = fluid.layers.square_error_cost(input=y_predict, label=y)
avg_loss = fluid.layers.mean(loss)
return avg_loss
``` ```
### Create Parameters ### Specify Place
Specify your training environment, you should specify if the training is on CPU or GPU.
```python ```python
parameters = paddle.parameters.create(cost) use_cuda = False
place = fluid.CUDAPlace(0) if use_cuda else fluid.CPUPlace()
``` ```
### Create Trainer ### Create Trainer
The trainer will take the `train_program` as input.
```python ```python
optimizer = paddle.optimizer.Momentum(momentum=0) trainer = fluid.Trainer(
train_func=train_program,
trainer = paddle.trainer.SGD(cost=cost, place=place,
parameters=parameters, optimizer=fluid.optimizer.SGD(learning_rate=0.001))
update_equation=optimizer)
``` ```
### Feeding Data ### Feeding Data
...@@ -210,105 +217,90 @@ PaddlePaddle provides the ...@@ -210,105 +217,90 @@ PaddlePaddle provides the
for loading the training data. A reader may return multiple columns, and we need a Python dictionary to specify the mapping from column index to data layers. for loading the training data. A reader may return multiple columns, and we need a Python dictionary to specify the mapping from column index to data layers.
```python ```python
feeding={'x': 0, 'y': 1} feed_order=['x', 'y']
``` ```
Moreover, an event handler is provided to print the training progress: Moreover, an event handler is provided to print the training progress:
```python ```python
# event_handler to print training and testing info # Specify the directory path to save the parameters
def event_handler(event): params_dirname = "fit_a_line.inference.model"
if isinstance(event, paddle.event.EndIteration):
if event.batch_id % 100 == 0:
print "Pass %d, Batch %d, Cost %f" % (
event.pass_id, event.batch_id, event.cost)
if isinstance(event, paddle.event.EndPass):
result = trainer.test(
reader=paddle.batch(
uci_housing.test(), batch_size=2),
feeding=feeding)
print "Test %d, Cost %f" % (event.pass_id, result.cost)
```
```python # Plot data
# event_handler to plot training and testing info
from paddle.v2.plot import Ploter from paddle.v2.plot import Ploter
train_title = "Train cost" train_title = "Train cost"
test_title = "Test cost" test_title = "Test cost"
plot_cost = Ploter(train_title, test_title) plot_cost = Ploter(train_title, test_title)
step = 0 step = 0
# event_handler to print training and testing info
def event_handler_plot(event): def event_handler_plot(event):
global step global step
if isinstance(event, paddle.event.EndIteration): if isinstance(event, fluid.EndStepEvent):
if step % 10 == 0: # every 10 batches, record a train cost if event.step % 10 == 0: # every 10 batches, record a test cost
plot_cost.append(train_title, step, event.cost) test_metrics = trainer.test(
reader=test_reader, feed_order=feed_order)
if step % 100 == 0: # every 100 batches, record a test cost
result = trainer.test( plot_cost.append(test_title, step, test_metrics[0])
reader=paddle.batch(
uci_housing.test(), batch_size=2),
feeding=feeding)
plot_cost.append(test_title, step, result.cost)
if step % 100 == 0: # every 100 batches, update cost plot
plot_cost.plot() plot_cost.plot()
step += 1 if test_metrics[0] < 10.0:
# If the accuracy is good enough, we can stop the training.
print('loss is less than 10.0, stop')
trainer.stop()
if isinstance(event, paddle.event.EndPass): # We can save the trained parameters for the inferences later
if event.pass_id % 10 == 0: if params_dirname is not None:
with open('params_pass_%d.tar' % event.pass_id, 'w') as f: trainer.save_params(params_dirname)
trainer.save_parameter_to_tar(f)
step += 1
``` ```
### Start Training ### Start Training
We now can start training by calling `trainer.train()`.
```python ```python
%matplotlib inline
# The training could take up to a few minutes.
trainer.train( trainer.train(
reader=paddle.batch( reader=train_reader,
paddle.reader.shuffle( num_epochs=100,
uci_housing.train(), buf_size=500),
batch_size=2),
feeding=feeding,
event_handler=event_handler_plot, event_handler=event_handler_plot,
num_passes=30) feed_order=feed_order)
``` ```
![png](./image/train_and_test.png) ![png](./image/train_and_test.png)
### Apply model ## Inference
Initialize the Inferencer with the inference_program and the params_dirname, which is where we saved our params
#### 1. generate testing data ### Setup the Inference Program
Similar to the trainer.train, the Inferencer needs to take an inference_program to do inference.
Prune the train_program to only have the y_predict.
```python ```python
test_data_creator = paddle.dataset.uci_housing.test() def inference_program():
test_data = [] x = fluid.layers.data(name='x', shape=[13], dtype='float32')
test_label = [] y_predict = fluid.layers.fc(input=x, size=1, act=None)
return y_predict
for item in test_data_creator():
test_data.append((item[0],))
test_label.append(item[1])
if len(test_data) == 5:
break
``` ```
#### 2. inference ### Infer
Inferencer will load the trained model from `params_dirname` and use it to infer the unseen data.
```python ```python
# load parameters from tar file. inferencer = fluid.Inferencer(
# users can remove the comments and change the model name infer_func=inference_program, param_path=params_dirname, place=place)
# with open('params_pass_20.tar', 'r') as f:
# parameters = paddle.parameters.Parameters.from_tar(f)
probs = paddle.infer( batch_size = 10
output_layer=y_predict, parameters=parameters, input=test_data) tensor_x = numpy.random.uniform(0, 10, [batch_size, 13]).astype("float32")
for i in xrange(len(probs)): results = inferencer.infer({'x': tensor_x})
print "label=" + str(test_label[i][0]) + ", predict=" + str(probs[i][0]) print("infer results: ", results[0])
``` ```
## Summary ## Summary
......
01.fit_a_line/infer.py 已删除 100644 → 0
浏览文件 @ 330cf3ee
import paddle.v2 as paddle
# Initialize PaddlePaddle.
paddle.init(use_gpu=False, trainer_count=1)
# Configure the neural network.
x = paddle.layer.data(name='x', type=paddle.data_type.dense_vector(13))
y_predict = paddle.layer.fc(input=x, size=1, act=paddle.activation.Linear())
# Infer using provided test data.
probs = paddle.infer(
output_layer=y_predict,
parameters=paddle.dataset.uci_housing.model(),
input=[item for item in paddle.dataset.uci_housing.test()()])
for i in xrange(len(probs)):
print 'Predicted price: ${:,.2f}'.format(probs[i][0] * 1000)
01.fit_a_line/train.py
浏览文件 @ 63b5ec3f
import os # Copyright (c) 2018 PaddlePaddle Authors. All Rights Reserved.
import paddle.v2 as paddle #
import paddle.v2.dataset.uci_housing as uci_housing # Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
with_gpu = os.getenv('WITH_GPU', '0') != '0' # You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
def main(): #
# init # Unless required by applicable law or agreed to in writing, software
paddle.init(use_gpu=with_gpu, trainer_count=1) # distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# network config # See the License for the specific language governing permissions and
x = paddle.layer.data(name='x', type=paddle.data_type.dense_vector(13)) # limitations under the License.
y_predict = paddle.layer.fc(input=x, size=1, act=paddle.activation.Linear())
y = paddle.layer.data(name='y', type=paddle.data_type.dense_vector(1)) import paddle
cost = paddle.layer.square_error_cost(input=y_predict, label=y) import paddle.fluid as fluid
import numpy
# Save the inference topology to protobuf.
inference_topology = paddle.topology.Topology(layers=y_predict) BATCH_SIZE = 20
with open("inference_topology.pkl", 'wb') as f:
inference_topology.serialize_for_inference(f) train_reader = paddle.batch(
paddle.reader.shuffle(paddle.dataset.uci_housing.train(), buf_size=500),
# create parameters batch_size=BATCH_SIZE)
parameters = paddle.parameters.create(cost)
test_reader = paddle.batch(
# create optimizer paddle.reader.shuffle(paddle.dataset.uci_housing.test(), buf_size=500),
optimizer = paddle.optimizer.Momentum(momentum=0) batch_size=BATCH_SIZE)
trainer = paddle.trainer.SGD(
cost=cost, parameters=parameters, update_equation=optimizer) def train_program():
y = fluid.layers.data(name='y', shape=[1], dtype='float32')
feeding = {'x': 0, 'y': 1}
# feature vector of length 13
# event_handler to print training and testing info x = fluid.layers.data(name='x', shape=[13], dtype='float32')
def event_handler(event): y_predict = fluid.layers.fc(input=x, size=1, act=None)
if isinstance(event, paddle.event.EndIteration):
if event.batch_id % 100 == 0: loss = fluid.layers.square_error_cost(input=y_predict, label=y)
print "Pass %d, Batch %d, Cost %f" % ( avg_loss = fluid.layers.mean(loss)
event.pass_id, event.batch_id, event.cost)
return avg_loss
if isinstance(event, paddle.event.EndPass):
if event.pass_id % 10 == 0:
with open('params_pass_%d.tar' % event.pass_id, 'w') as f: def optimizer_program():
trainer.save_parameter_to_tar(f) return fluid.optimizer.SGD(learning_rate=0.001)
result = trainer.test(
reader=paddle.batch(uci_housing.test(), batch_size=2),
feeding=feeding) # can use CPU or GPU
print "Test %d, Cost %f" % (event.pass_id, result.cost) use_cuda = False
place = fluid.CUDAPlace(0) if use_cuda else fluid.CPUPlace()
# training
trainer.train( trainer = fluid.Trainer(
reader=paddle.batch( train_func=train_program,
paddle.reader.shuffle(uci_housing.train(), buf_size=500), place=place,
batch_size=2), optimizer_func=optimizer_program)
feeding=feeding,
event_handler=event_handler, feed_order = ['x', 'y']
num_passes=30)
# Specify the directory path to save the parameters
# inference params_dirname = "fit_a_line.inference.model"
test_data_creator = paddle.dataset.uci_housing.test()
test_data = [] # Plot data
test_label = [] from paddle.v2.plot import Ploter
train_title = "Train cost"
for item in test_data_creator(): test_title = "Test cost"
test_data.append((item[0], )) plot_cost = Ploter(train_title, test_title)
test_label.append(item[1])
if len(test_data) == 5: step = 0
break
# load parameters from tar file. # event_handler to print training and testing info
# users can remove the comments and change the model name def event_handler_plot(event):
# with open('params_pass_20.tar', 'r') as f: global step
# parameters = paddle.parameters.Parameters.from_tar(f) if isinstance(event, fluid.EndStepEvent):
if event.step % 10 == 0: # every 10 batches, record a test cost
probs = paddle.infer( test_metrics = trainer.test(
output_layer=y_predict, parameters=parameters, input=test_data) reader=test_reader, feed_order=feed_order)
for i in xrange(len(probs)): plot_cost.append(test_title, step, test_metrics[0])
print "label=" + str(test_label[i][0]) + ", predict=" + str(probs[i][0]) plot_cost.plot()
if test_metrics[0] < 10.0:
if __name__ == '__main__': # If the accuracy is good enough, we can stop the training.
main() print('loss is less than 10.0, stop')
trainer.stop()
# We can save the trained parameters for the inferences later
if params_dirname is not None:
trainer.save_params(params_dirname)
step += 1
# The training could take up to a few minutes.
trainer.train(
reader=train_reader,
num_epochs=100,
event_handler=event_handler_plot,
feed_order=feed_order)
def inference_program():
x = fluid.layers.data(name='x', shape=[13], dtype='float32')
y_predict = fluid.layers.fc(input=x, size=1, act=None)
return y_predict
inferencer = fluid.Inferencer(
infer_func=inference_program, param_path=params_dirname, place=place)
batch_size = 10
tensor_x = numpy.random.uniform(0, 10, [batch_size, 13]).astype("float32")
results = inferencer.infer({'x': tensor_x})
print("infer results: ", results[0])
02.recognize_digits/README.cn.md
浏览文件 @ 63b5ec3f
...@@ -127,112 +127,188 @@ PaddlePaddle在API中提供了自动加载[MNIST](http://yann.lecun.com/exdb/mni ...@@ -127,112 +127,188 @@ PaddlePaddle在API中提供了自动加载[MNIST](http://yann.lecun.com/exdb/mni
|t10k-images-idx3-ubyte | 测试数据图片，10,000条数据 | |t10k-images-idx3-ubyte | 测试数据图片，10,000条数据 |
|t10k-labels-idx1-ubyte | 测试数据标签，10,000条数据 | |t10k-labels-idx1-ubyte | 测试数据标签，10,000条数据 |
## 配置说明 ## Fluid API 概述
演示将使用最新的 `Fluid API`。Fluid API是最新的 PaddlePaddle API。它在不牺牲性能的情况下简化了模型配置。
我们建议使用 Fluid API，因为它更容易学起来。
下面是快速的 Fluid API 概述。
1. `inference_program`：指定如何从数据输入中获得预测的函数。
这是指定网络流的地方。
1. `train_program`：指定如何从 `inference_program` 和`标签值`中获取 `loss` 的函数。
这是指定损失计算的地方。
1. `optimizer_func`: “指定优化器配置的函数。优化器负责减少损失并驱动培训。Paddle 支持多种不同的优化器。
首先，加载PaddlePaddle的V2 api包。 1. `Trainer`：PaddlePaddle Trainer 管理由 `train_program` 和 `optimizer` 指定的训练过程。
通过 `event_handler` 回调函数，用户可以监控培训的进展。
1. `Inferencer`：Fluid inferencer 加载 `inference_program` 和由 Trainer 训练的参数。
然后，它可以推断数据和返回预测。
在这个演示中，我们将深入了解它们。
## 配置说明
加载 PaddlePaddle 的 Fluid API 包。
```python ```python
import paddle.v2 as paddle import paddle
import paddle.fluid as fluid
``` ```
其次，定义三个不同的分类器：
### Program Functions 配置
我们需要设置“推理程序”函数。我们想用这个程序来演示三个不同的分类器，每个分类器都定义为 Python 函数。
我们需要将图像数据馈送到分类器。Paddle 为读取数据提供了一个特殊的层 `layer.data` 层。
让我们创建一个数据层来读取图像并将其连接到分类网络。
- Softmax回归：只通过一层简单的以softmax为激活函数的全连接层，就可以得到分类的结果。 - Softmax回归：只通过一层简单的以softmax为激活函数的全连接层，就可以得到分类的结果。
```python ```python
def softmax_regression(img): def softmax_regression():
predict = paddle.layer.fc(input=img, img = fluid.layers.data(name='img', shape=[1, 28, 28], dtype='float32')
size=10, predict = fluid.layers.fc(
act=paddle.activation.Softmax()) input=img, size=10, act='softmax')
return predict return predict
``` ```
- 多层感知器：下面代码实现了一个含有两个隐藏层（即全连接层）的多层感知器。其中两个隐藏层的激活函数均采用ReLU，输出层的激活函数用Softmax。 - 多层感知器：下面代码实现了一个含有两个隐藏层（即全连接层）的多层感知器。其中两个隐藏层的激活函数均采用ReLU，输出层的激活函数用Softmax。
```python ```python
def multilayer_perceptron(img): def multilayer_perceptron():
img = fluid.layers.data(name='img', shape=[1, 28, 28], dtype='float32')
# 第一个全连接层，激活函数为ReLU # 第一个全连接层，激活函数为ReLU
hidden1 = paddle.layer.fc(input=img, size=128, act=paddle.activation.Relu()) hidden = fluid.layers.fc(input=img, size=200, act='relu')
# 第二个全连接层，激活函数为ReLU # 第二个全连接层，激活函数为ReLU
hidden2 = paddle.layer.fc(input=hidden1, hidden = fluid.layers.fc(input=hidden, size=200, act='relu')
size=64,
act=paddle.activation.Relu())
# 以softmax为激活函数的全连接输出层，输出层的大小必须为数字的个数10 # 以softmax为激活函数的全连接输出层，输出层的大小必须为数字的个数10
predict = paddle.layer.fc(input=hidden2, prediction = fluid.layers.fc(input=hidden, size=10, act='softmax')
size=10, return prediction
act=paddle.activation.Softmax())
return predict
``` ```
- 卷积神经网络LeNet-5: 输入的二维图像，首先经过两次卷积层到池化层，再经过全连接层，最后使用以softmax为激活函数的全连接层作为输出层。 - 卷积神经网络LeNet-5: 输入的二维图像，首先经过两次卷积层到池化层，再经过全连接层，最后使用以softmax为激活函数的全连接层作为输出层。
```python ```python
def convolutional_neural_network(img): def convolutional_neural_network():
img = fluid.layers.data(name='img', shape=[1, 28, 28], dtype='float32')
# 第一个卷积-池化层 # 第一个卷积-池化层
conv_pool_1 = paddle.networks.simple_img_conv_pool( conv_pool_1 = fluid.nets.simple_img_conv_pool(
input=img, input=img,
filter_size=5, filter_size=5,
num_filters=20, num_filters=20,
num_channel=1,
pool_size=2, pool_size=2,
pool_stride=2, pool_stride=2,
act=paddle.activation.Relu()) act="relu")
conv_pool_1 = fluid.layers.batch_norm(conv_pool_1)
# 第二个卷积-池化层 # 第二个卷积-池化层
conv_pool_2 = paddle.networks.simple_img_conv_pool( conv_pool_2 = fluid.nets.simple_img_conv_pool(
input=conv_pool_1, input=conv_pool_1,
filter_size=5, filter_size=5,
num_filters=50, num_filters=50,
num_channel=20,
pool_size=2, pool_size=2,
pool_stride=2, pool_stride=2,
act=paddle.activation.Relu()) act="relu")
# 以softmax为激活函数的全连接输出层，输出层的大小必须为数字的个数10 # 以softmax为激活函数的全连接输出层，输出层的大小必须为数字的个数10
predict = paddle.layer.fc(input=conv_pool_2, prediction = fluid.layers.fc(input=conv_pool_2, size=10, act='softmax')
size=10, return prediction
act=paddle.activation.Softmax())
return predict
``` ```
接着，通过`layer.data`调用来获取数据，然后调用分类器（这里我们提供了三个不同的分类器）得到分类结果。训练时，对该结果计算其损失函数，分类问题常常选择交叉熵损失函数。 #### Train Program 配置
然后我们需要设置训练程序 `train_program`。它首先从分类器中进行预测。
在训练期间，它将从预测中计算 `avg_cost`。
**注意:** 训练程序应该返回一个数组，第一个返回参数必须是 `avg_cost`。训练器使用它来计算梯度。
请随意修改代码，测试 Softmax 回归 `softmax_regression`, `MLP` 和 卷积神经网络 `convolutional neural network` 分类器之间的不同结果。
```python ```python
# 该模型运行在单个CPU上 def train_program():
paddle.init(use_gpu=False, trainer_count=1) label = fluid.layers.data(name='label', shape=[1], dtype='int64')
images = paddle.layer.data( # predict = softmax_regression() # uncomment for Softmax回归
name='pixel', type=paddle.data_type.dense_vector(784)) # predict = multilayer_perceptron() # uncomment for 多层感知器
label = paddle.layer.data( predict = convolutional_neural_network() # uncomment for LeNet5卷积神经网络
name='label', type=paddle.data_type.integer_value(10)) cost = fluid.layers.cross_entropy(input=predict, label=label)
avg_cost = fluid.layers.mean(cost)
acc = fluid.layers.accuracy(input=predict, label=label)
return [avg_cost, acc]
# predict = softmax_regression(images) # Softmax回归
# predict = multilayer_perceptron(images) #多层感知器
predict = convolutional_neural_network(images) #LeNet5卷积神经网络
cost = paddle.layer.classification_cost(input=predict, label=label) # 该模型运行在单个CPU上
``` ```
然后，指定训练相关的参数。 #### Optimizer Function 配置
- 训练方法（optimizer)： 代表训练过程在更新权重时采用动量优化器 `Momentum` ，其中参数0.9代表动量优化每次保持前一次速度的0.9倍。
- 训练速度（learning_rate）： 迭代的速度，与网络的训练收敛速度有关系。
- 正则化（regularization）： 是防止网络过拟合的一种手段，此处采用L2正则化。
```python
parameters = paddle.parameters.create(cost)
optimizer = paddle.optimizer.Momentum( 在下面的 `Adam optimizer`，`learning_rate` 是训练的速度，与网络的训练收敛速度有关系。
learning_rate=0.1 / 128.0,
momentum=0.9,
regularization=paddle.optimizer.L2Regularization(rate=0.0005 * 128))
trainer = paddle.trainer.SGD(cost=cost, ```python
parameters=parameters, def optimizer_program():
update_equation=optimizer) return fluid.optimizer.Adam(learning_rate=0.001)
``` ```
### 数据集 Feeders 配置
下一步，我们开始训练过程。`paddle.dataset.movielens.train()`和`paddle.dataset.movielens.test()`分别做训练和测试数据集。这两个函数各自返回一个reader——PaddlePaddle中的reader是一个Python函数，每次调用的时候返回一个Python yield generator。 下一步，我们开始训练过程。`paddle.dataset.movielens.train()`和`paddle.dataset.movielens.test()`分别做训练和测试数据集。这两个函数各自返回一个reader——PaddlePaddle中的reader是一个Python函数，每次调用的时候返回一个Python yield generator。
下面`shuffle`是一个reader decorator，它接受一个reader A，返回另一个reader B —— reader B 每次读入`buffer_size`条训练数据到一个buffer里，然后随机打乱其顺序，并且逐条输出。 下面`shuffle`是一个reader decorator，它接受一个reader A，返回另一个reader B —— reader B 每次读入`buffer_size`条训练数据到一个buffer里，然后随机打乱其顺序，并且逐条输出。
`batch`是一个特殊的decorator，它的输入是一个reader，输出是一个batched reader —— 在PaddlePaddle里，一个reader每次yield一条训练数据，而一个batched reader每次yield一个minibatch。 `batch`是一个特殊的decorator，它的输入是一个reader，输出是一个batched reader —— 在PaddlePaddle里，一个reader每次yield一条训练数据，而一个batched reader每次yield一个minibatch。
`event_handler_plot`可以用来在训练过程中画图如下： ```python
train_reader = paddle.batch(
paddle.reader.shuffle(
paddle.dataset.mnist.train(), buf_size=500),
batch_size=64)
test_reader = paddle.batch(
paddle.dataset.mnist.test(), batch_size=64)
```
### Trainer 配置
现在，我们需要配置 `Trainer`。`Trainer` 需要接受训练程序 `train_program`, `place` 和优化器 `optimizer`。
```python
# 该模型运行在单个CPU上
use_cuda = False # set to True if training with GPU
place = fluid.CUDAPlace(0) if use_cuda else fluid.CPUPlace()
trainer = fluid.Trainer(
train_func=train_program, place=place, optimizer_func=optimizer_program)
```
#### Event Handler 配置
Fluid API 在训练期间为回调函数提供了一个钩子。用户能够通过机制监控培训进度。
我们将在这里演示两个 `event_handler` 程序。请随意修改 Jupyter 笔记本 ，看看有什么不同。
`event_handler` 用来在训练过程中输出训练结果
```python
# Save the parameter into a directory. The Inferencer can load the parameters from it to do infer
params_dirname = "recognize_digits_network.inference.model"
lists = []
def event_handler(event):
if isinstance(event, fluid.EndStepEvent):
if event.step % 100 == 0:
# event.metrics maps with train program return arguments.
# event.metrics[0] will yeild avg_cost and event.metrics[1] will yeild acc in this example.
print "Pass %d, Batch %d, Cost %f" % (
event.step, event.epoch, event.metrics[0])
if isinstance(event, fluid.EndEpochEvent):
avg_cost, acc = trainer.test(
reader=test_reader, feed_order=['img', 'label'])
print("Test with Epoch %d, avg_cost: %s, acc: %s" % (event.epoch, avg_cost, acc))
# save parameters
trainer.save_params(params_dirname)
lists.append((event.epoch, avg_cost, acc))
```
`event_handler_plot` 可以用来在训练过程中画图如下：
![png](./image/train_and_test.png) ![png](./image/train_and_test.png)
...@@ -242,68 +318,57 @@ from paddle.v2.plot import Ploter ...@@ -242,68 +318,57 @@ from paddle.v2.plot import Ploter
train_title = "Train cost" train_title = "Train cost"
test_title = "Test cost" test_title = "Test cost"
cost_ploter = Ploter(train_title, test_title) cost_ploter = Ploter(train_title, test_title)
step = 0 step = 0
lists = []
# event_handler to plot a figure # event_handler to plot a figure
def event_handler_plot(event): def event_handler_plot(event):
global step global step
if isinstance(event, paddle.event.EndIteration): if isinstance(event, fluid.EndStepEvent):
if step % 100 == 0: if step % 100 == 0:
cost_ploter.append(train_title, step, event.cost) # event.metrics maps with train program return arguments.
# event.metrics[0] will yeild avg_cost and event.metrics[1] will yeild acc in this example.
cost_ploter.append(train_title, step, event.metrics[0])
cost_ploter.plot() cost_ploter.plot()
step += 1 step += 1
if isinstance(event, paddle.event.EndPass): if isinstance(event, fluid.EndEpochEvent):
# save parameters # save parameters
with open('params_pass_%d.tar' % event.pass_id, 'w') as f: trainer.save_params(params_dirname)
trainer.save_parameter_to_tar(f)
result = trainer.test(reader=paddle.batch( avg_cost, acc = trainer.test(
paddle.dataset.mnist.test(), batch_size=128)) reader=test_reader, feed_order=['img', 'label'])
cost_ploter.append(test_title, step, result.cost) cost_ploter.append(test_title, step, avg_cost)
lists.append((event.epoch, avg_cost, acc))
``` ```
`event_handler` 用来在训练过程中输出训练结果 #### 开始训练
```python
lists = []
def event_handler(event): 既然我们设置了 `event_handler` 和 `data reader`，我们就可以开始训练模型了。
if isinstance(event, paddle.event.EndIteration):
if event.batch_id % 100 == 0: `feed_order` 用于将数据目录映射到 `train_program`
print "Pass %d, Batch %d, Cost %f, %s" % (
event.pass_id, event.batch_id, event.cost, event.metrics)
if isinstance(event, paddle.event.EndPass):
# save parameters
with open('params_pass_%d.tar' % event.pass_id, 'w') as f:
trainer.save_parameter_to_tar(f)
result = trainer.test(reader=paddle.batch(
paddle.dataset.mnist.test(), batch_size=128))
print "Test with Pass %d, Cost %f, %s\n" % (
event.pass_id, result.cost, result.metrics)
lists.append((event.pass_id, result.cost,
result.metrics['classification_error_evaluator']))
```
```python ```python
trainer.train( trainer.train(
reader=paddle.batch( num_epochs=5,
paddle.reader.shuffle( event_handler=event_handler,
paddle.dataset.mnist.train(), buf_size=8192), reader=train_reader,
batch_size=128), feed_order=['img', 'label'])
event_handler=event_handler_plot,
num_passes=5)
``` ```
训练过程是完全自动的，event_handler里打印的日志类似如下所示： 训练过程是完全自动的，event_handler里打印的日志类似如下所示：
``` ```
# Pass 0, Batch 0, Cost 2.780790, {'classification_error_evaluator': 0.9453125} Pass 0, Batch 0, Cost 0.125650
# Pass 0, Batch 100, Cost 0.635356, {'classification_error_evaluator': 0.2109375} Pass 100, Batch 0, Cost 0.161387
# Pass 0, Batch 200, Cost 0.326094, {'classification_error_evaluator': 0.1328125} Pass 200, Batch 0, Cost 0.040036
# Pass 0, Batch 300, Cost 0.361920, {'classification_error_evaluator': 0.1015625} Pass 300, Batch 0, Cost 0.023391
# Pass 0, Batch 400, Cost 0.410101, {'classification_error_evaluator': 0.125} Pass 400, Batch 0, Cost 0.005856
# Test with Pass 0, Cost 0.326659, {'classification_error_evaluator': 0.09470000118017197} Pass 500, Batch 0, Cost 0.003315
Pass 600, Batch 0, Cost 0.009977
Pass 700, Batch 0, Cost 0.020959
Pass 800, Batch 0, Cost 0.105560
Pass 900, Batch 0, Cost 0.239809
Test with Epoch 0, avg_cost: 0.053097883707459624, acc: 0.9822850318471338
``` ```
训练之后，检查模型的预测准确度。用 MNIST 训练的时候，一般 softmax回归模型的分类准确率为约为 92.34%，多层感知器为97.66%，卷积神经网络可以达到 99.20%。 训练之后，检查模型的预测准确度。用 MNIST 训练的时候，一般 softmax回归模型的分类准确率为约为 92.34%，多层感知器为97.66%，卷积神经网络可以达到 99.20%。
...@@ -311,27 +376,50 @@ trainer.train( ...@@ -311,27 +376,50 @@ trainer.train(
## 应用模型 ## 应用模型
可以使用训练好的模型对手写体数字图片进行分类，下面程序展示了如何使用paddle.infer接口进行推断。 可以使用训练好的模型对手写体数字图片进行分类，下面程序展示了如何使用 `fluid.Inferencer` 接口进行推断。
### Inference 配置
`Inference` 需要一个 `infer_func` 和 `param_path` 来设置网络和经过训练的参数。
我们可以简单地插入在此之前定义的分类器。
```python ```python
from PIL import Image inferencer = fluid.Inferencer(
import numpy as np # infer_func=softmax_regression, # uncomment for softmax regression
# infer_func=multilayer_perceptron, # uncomment for MLP
infer_func=convolutional_neural_network, # uncomment for LeNet5
param_path=params_dirname,
place=place)
```
### 生成预测输入数据
`infer_3.png` 是数字 3 的一个示例图像。把它变成一个 numpy 数组以匹配数据馈送格式。
```python
# Prepare the test image
import os import os
import numpy as np
from PIL import Image
def load_image(file): def load_image(file):
im = Image.open(file).convert('L') im = Image.open(file).convert('L')
im = im.resize((28, 28), Image.ANTIALIAS) im = im.resize((28, 28), Image.ANTIALIAS)
im = np.array(im).astype(np.float32).flatten() im = np.array(im).reshape(1, 1, 28, 28).astype(np.float32)
im = im / 255.0 * 2.0 - 1.0 im = im / 255.0 * 2.0 - 1.0
return im return im
test_data = [] cur_dir = cur_dir = os.getcwd()
cur_dir = os.getcwd() img = load_image(cur_dir + '/image/infer_3.png')
test_data.append((load_image(cur_dir + '/image/infer_3.png'),)) ```
### 预测
probs = paddle.infer( 现在我们准备做预测。
output_layer=predict, parameters=parameters, input=test_data)
lab = np.argsort(-probs) # probs and lab are the results of one batch data ```python
print "Label of image/infer_3.png is: %d" % lab[0][0] results = inferencer.infer({'img': img})
lab = np.argsort(results) # probs and lab are the results of one batch data
print "Label of image/infer_3.png is: %d" % lab[0][0][-1]
``` ```
## 总结 ## 总结
......
02.recognize_digits/README.md
浏览文件 @ 63b5ec3f
此差异已折叠。
02.recognize_digits/index.cn.html
浏览文件 @ 63b5ec3f
...@@ -169,112 +169,188 @@ PaddlePaddle在API中提供了自动加载[MNIST](http://yann.lecun.com/exdb/mni ...@@ -169,112 +169,188 @@ PaddlePaddle在API中提供了自动加载[MNIST](http://yann.lecun.com/exdb/mni
|t10k-images-idx3-ubyte | 测试数据图片，10,000条数据 | |t10k-images-idx3-ubyte | 测试数据图片，10,000条数据 |
|t10k-labels-idx1-ubyte | 测试数据标签，10,000条数据 | |t10k-labels-idx1-ubyte | 测试数据标签，10,000条数据 |
## 配置说明 ## Fluid API 概述
演示将使用最新的 `Fluid API`。Fluid API是最新的 PaddlePaddle API。它在不牺牲性能的情况下简化了模型配置。
我们建议使用 Fluid API，因为它更容易学起来。
下面是快速的 Fluid API 概述。
1. `inference_program`：指定如何从数据输入中获得预测的函数。
这是指定网络流的地方。
1. `train_program`：指定如何从 `inference_program` 和`标签值`中获取 `loss` 的函数。
这是指定损失计算的地方。
1. `optimizer_func`: “指定优化器配置的函数。优化器负责减少损失并驱动培训。Paddle 支持多种不同的优化器。
首先，加载PaddlePaddle的V2 api包。 1. `Trainer`：PaddlePaddle Trainer 管理由 `train_program` 和 `optimizer` 指定的训练过程。
通过 `event_handler` 回调函数，用户可以监控培训的进展。
1. `Inferencer`：Fluid inferencer 加载 `inference_program` 和由 Trainer 训练的参数。
然后，它可以推断数据和返回预测。
在这个演示中，我们将深入了解它们。
## 配置说明
加载 PaddlePaddle 的 Fluid API 包。
```python ```python
import paddle.v2 as paddle import paddle
import paddle.fluid as fluid
``` ```
其次，定义三个不同的分类器：
### Program Functions 配置
我们需要设置“推理程序”函数。我们想用这个程序来演示三个不同的分类器，每个分类器都定义为 Python 函数。
我们需要将图像数据馈送到分类器。Paddle 为读取数据提供了一个特殊的层 `layer.data` 层。
让我们创建一个数据层来读取图像并将其连接到分类网络。
- Softmax回归：只通过一层简单的以softmax为激活函数的全连接层，就可以得到分类的结果。 - Softmax回归：只通过一层简单的以softmax为激活函数的全连接层，就可以得到分类的结果。
```python ```python
def softmax_regression(img): def softmax_regression():
predict = paddle.layer.fc(input=img, img = fluid.layers.data(name='img', shape=[1, 28, 28], dtype='float32')
size=10, predict = fluid.layers.fc(
act=paddle.activation.Softmax()) input=img, size=10, act='softmax')
return predict return predict
``` ```
- 多层感知器：下面代码实现了一个含有两个隐藏层（即全连接层）的多层感知器。其中两个隐藏层的激活函数均采用ReLU，输出层的激活函数用Softmax。 - 多层感知器：下面代码实现了一个含有两个隐藏层（即全连接层）的多层感知器。其中两个隐藏层的激活函数均采用ReLU，输出层的激活函数用Softmax。
```python ```python
def multilayer_perceptron(img): def multilayer_perceptron():
img = fluid.layers.data(name='img', shape=[1, 28, 28], dtype='float32')
# 第一个全连接层，激活函数为ReLU # 第一个全连接层，激活函数为ReLU
hidden1 = paddle.layer.fc(input=img, size=128, act=paddle.activation.Relu()) hidden = fluid.layers.fc(input=img, size=200, act='relu')
# 第二个全连接层，激活函数为ReLU # 第二个全连接层，激活函数为ReLU
hidden2 = paddle.layer.fc(input=hidden1, hidden = fluid.layers.fc(input=hidden, size=200, act='relu')
size=64,
act=paddle.activation.Relu())
# 以softmax为激活函数的全连接输出层，输出层的大小必须为数字的个数10 # 以softmax为激活函数的全连接输出层，输出层的大小必须为数字的个数10
predict = paddle.layer.fc(input=hidden2, prediction = fluid.layers.fc(input=hidden, size=10, act='softmax')
size=10, return prediction
act=paddle.activation.Softmax())
return predict
``` ```
- 卷积神经网络LeNet-5: 输入的二维图像，首先经过两次卷积层到池化层，再经过全连接层，最后使用以softmax为激活函数的全连接层作为输出层。 - 卷积神经网络LeNet-5: 输入的二维图像，首先经过两次卷积层到池化层，再经过全连接层，最后使用以softmax为激活函数的全连接层作为输出层。
```python ```python
def convolutional_neural_network(img): def convolutional_neural_network():
img = fluid.layers.data(name='img', shape=[1, 28, 28], dtype='float32')
# 第一个卷积-池化层 # 第一个卷积-池化层
conv_pool_1 = paddle.networks.simple_img_conv_pool( conv_pool_1 = fluid.nets.simple_img_conv_pool(
input=img, input=img,
filter_size=5, filter_size=5,
num_filters=20, num_filters=20,
num_channel=1,
pool_size=2, pool_size=2,
pool_stride=2, pool_stride=2,
act=paddle.activation.Relu()) act="relu")
conv_pool_1 = fluid.layers.batch_norm(conv_pool_1)
# 第二个卷积-池化层 # 第二个卷积-池化层
conv_pool_2 = paddle.networks.simple_img_conv_pool( conv_pool_2 = fluid.nets.simple_img_conv_pool(
input=conv_pool_1, input=conv_pool_1,
filter_size=5, filter_size=5,
num_filters=50, num_filters=50,
num_channel=20,
pool_size=2, pool_size=2,
pool_stride=2, pool_stride=2,
act=paddle.activation.Relu()) act="relu")
# 以softmax为激活函数的全连接输出层，输出层的大小必须为数字的个数10 # 以softmax为激活函数的全连接输出层，输出层的大小必须为数字的个数10
predict = paddle.layer.fc(input=conv_pool_2, prediction = fluid.layers.fc(input=conv_pool_2, size=10, act='softmax')
size=10, return prediction
act=paddle.activation.Softmax())
return predict
``` ```
接着，通过`layer.data`调用来获取数据，然后调用分类器（这里我们提供了三个不同的分类器）得到分类结果。训练时，对该结果计算其损失函数，分类问题常常选择交叉熵损失函数。 #### Train Program 配置
然后我们需要设置训练程序 `train_program`。它首先从分类器中进行预测。
在训练期间，它将从预测中计算 `avg_cost`。
**注意:** 训练程序应该返回一个数组，第一个返回参数必须是 `avg_cost`。训练器使用它来计算梯度。
请随意修改代码，测试 Softmax 回归 `softmax_regression`, `MLP` 和 卷积神经网络 `convolutional neural network` 分类器之间的不同结果。
```python ```python
# 该模型运行在单个CPU上 def train_program():
paddle.init(use_gpu=False, trainer_count=1) label = fluid.layers.data(name='label', shape=[1], dtype='int64')
images = paddle.layer.data( # predict = softmax_regression() # uncomment for Softmax回归
name='pixel', type=paddle.data_type.dense_vector(784)) # predict = multilayer_perceptron() # uncomment for 多层感知器
label = paddle.layer.data( predict = convolutional_neural_network() # uncomment for LeNet5卷积神经网络
name='label', type=paddle.data_type.integer_value(10)) cost = fluid.layers.cross_entropy(input=predict, label=label)
avg_cost = fluid.layers.mean(cost)
acc = fluid.layers.accuracy(input=predict, label=label)
return [avg_cost, acc]
# predict = softmax_regression(images) # Softmax回归
# predict = multilayer_perceptron(images) #多层感知器
predict = convolutional_neural_network(images) #LeNet5卷积神经网络
cost = paddle.layer.classification_cost(input=predict, label=label) # 该模型运行在单个CPU上
``` ```
然后，指定训练相关的参数。 #### Optimizer Function 配置
- 训练方法（optimizer)： 代表训练过程在更新权重时采用动量优化器 `Momentum` ，其中参数0.9代表动量优化每次保持前一次速度的0.9倍。
- 训练速度（learning_rate）： 迭代的速度，与网络的训练收敛速度有关系。
- 正则化（regularization）： 是防止网络过拟合的一种手段，此处采用L2正则化。
```python
parameters = paddle.parameters.create(cost)
optimizer = paddle.optimizer.Momentum( 在下面的 `Adam optimizer`，`learning_rate` 是训练的速度，与网络的训练收敛速度有关系。
learning_rate=0.1 / 128.0,
momentum=0.9,
regularization=paddle.optimizer.L2Regularization(rate=0.0005 * 128))
trainer = paddle.trainer.SGD(cost=cost, ```python
parameters=parameters, def optimizer_program():
update_equation=optimizer) return fluid.optimizer.Adam(learning_rate=0.001)
``` ```
### 数据集 Feeders 配置
下一步，我们开始训练过程。`paddle.dataset.movielens.train()`和`paddle.dataset.movielens.test()`分别做训练和测试数据集。这两个函数各自返回一个reader——PaddlePaddle中的reader是一个Python函数，每次调用的时候返回一个Python yield generator。 下一步，我们开始训练过程。`paddle.dataset.movielens.train()`和`paddle.dataset.movielens.test()`分别做训练和测试数据集。这两个函数各自返回一个reader——PaddlePaddle中的reader是一个Python函数，每次调用的时候返回一个Python yield generator。
下面`shuffle`是一个reader decorator，它接受一个reader A，返回另一个reader B —— reader B 每次读入`buffer_size`条训练数据到一个buffer里，然后随机打乱其顺序，并且逐条输出。 下面`shuffle`是一个reader decorator，它接受一个reader A，返回另一个reader B —— reader B 每次读入`buffer_size`条训练数据到一个buffer里，然后随机打乱其顺序，并且逐条输出。
`batch`是一个特殊的decorator，它的输入是一个reader，输出是一个batched reader —— 在PaddlePaddle里，一个reader每次yield一条训练数据，而一个batched reader每次yield一个minibatch。 `batch`是一个特殊的decorator，它的输入是一个reader，输出是一个batched reader —— 在PaddlePaddle里，一个reader每次yield一条训练数据，而一个batched reader每次yield一个minibatch。
`event_handler_plot`可以用来在训练过程中画图如下： ```python
train_reader = paddle.batch(
paddle.reader.shuffle(
paddle.dataset.mnist.train(), buf_size=500),
batch_size=64)
test_reader = paddle.batch(
paddle.dataset.mnist.test(), batch_size=64)
```
### Trainer 配置
现在，我们需要配置 `Trainer`。`Trainer` 需要接受训练程序 `train_program`, `place` 和优化器 `optimizer`。
```python
# 该模型运行在单个CPU上
use_cuda = False # set to True if training with GPU
place = fluid.CUDAPlace(0) if use_cuda else fluid.CPUPlace()
trainer = fluid.Trainer(
train_func=train_program, place=place, optimizer_func=optimizer_program)
```
#### Event Handler 配置
Fluid API 在训练期间为回调函数提供了一个钩子。用户能够通过机制监控培训进度。
我们将在这里演示两个 `event_handler` 程序。请随意修改 Jupyter 笔记本 ，看看有什么不同。
`event_handler` 用来在训练过程中输出训练结果
```python
# Save the parameter into a directory. The Inferencer can load the parameters from it to do infer
params_dirname = "recognize_digits_network.inference.model"
lists = []
def event_handler(event):
if isinstance(event, fluid.EndStepEvent):
if event.step % 100 == 0:
# event.metrics maps with train program return arguments.
# event.metrics[0] will yeild avg_cost and event.metrics[1] will yeild acc in this example.
print "Pass %d, Batch %d, Cost %f" % (
event.step, event.epoch, event.metrics[0])
if isinstance(event, fluid.EndEpochEvent):
avg_cost, acc = trainer.test(
reader=test_reader, feed_order=['img', 'label'])
print("Test with Epoch %d, avg_cost: %s, acc: %s" % (event.epoch, avg_cost, acc))
# save parameters
trainer.save_params(params_dirname)
lists.append((event.epoch, avg_cost, acc))
```
`event_handler_plot` 可以用来在训练过程中画图如下：
![png](./image/train_and_test.png) ![png](./image/train_and_test.png)
...@@ -284,68 +360,57 @@ from paddle.v2.plot import Ploter ...@@ -284,68 +360,57 @@ from paddle.v2.plot import Ploter
train_title = "Train cost" train_title = "Train cost"
test_title = "Test cost" test_title = "Test cost"
cost_ploter = Ploter(train_title, test_title) cost_ploter = Ploter(train_title, test_title)
step = 0 step = 0
lists = []
# event_handler to plot a figure # event_handler to plot a figure
def event_handler_plot(event): def event_handler_plot(event):
global step global step
if isinstance(event, paddle.event.EndIteration): if isinstance(event, fluid.EndStepEvent):
if step % 100 == 0: if step % 100 == 0:
cost_ploter.append(train_title, step, event.cost) # event.metrics maps with train program return arguments.
# event.metrics[0] will yeild avg_cost and event.metrics[1] will yeild acc in this example.
cost_ploter.append(train_title, step, event.metrics[0])
cost_ploter.plot() cost_ploter.plot()
step += 1 step += 1
if isinstance(event, paddle.event.EndPass): if isinstance(event, fluid.EndEpochEvent):
# save parameters # save parameters
with open('params_pass_%d.tar' % event.pass_id, 'w') as f: trainer.save_params(params_dirname)
trainer.save_parameter_to_tar(f)
result = trainer.test(reader=paddle.batch( avg_cost, acc = trainer.test(
paddle.dataset.mnist.test(), batch_size=128)) reader=test_reader, feed_order=['img', 'label'])
cost_ploter.append(test_title, step, result.cost) cost_ploter.append(test_title, step, avg_cost)
lists.append((event.epoch, avg_cost, acc))
``` ```
`event_handler` 用来在训练过程中输出训练结果 #### 开始训练
```python
lists = []
def event_handler(event): 既然我们设置了 `event_handler` 和 `data reader`，我们就可以开始训练模型了。
if isinstance(event, paddle.event.EndIteration):
if event.batch_id % 100 == 0: `feed_order` 用于将数据目录映射到 `train_program`
print "Pass %d, Batch %d, Cost %f, %s" % (
event.pass_id, event.batch_id, event.cost, event.metrics)
if isinstance(event, paddle.event.EndPass):
# save parameters
with open('params_pass_%d.tar' % event.pass_id, 'w') as f:
trainer.save_parameter_to_tar(f)
result = trainer.test(reader=paddle.batch(
paddle.dataset.mnist.test(), batch_size=128))
print "Test with Pass %d, Cost %f, %s\n" % (
event.pass_id, result.cost, result.metrics)
lists.append((event.pass_id, result.cost,
result.metrics['classification_error_evaluator']))
```
```python ```python
trainer.train( trainer.train(
reader=paddle.batch( num_epochs=5,
paddle.reader.shuffle( event_handler=event_handler,
paddle.dataset.mnist.train(), buf_size=8192), reader=train_reader,
batch_size=128), feed_order=['img', 'label'])
event_handler=event_handler_plot,
num_passes=5)
``` ```
训练过程是完全自动的，event_handler里打印的日志类似如下所示： 训练过程是完全自动的，event_handler里打印的日志类似如下所示：
``` ```
# Pass 0, Batch 0, Cost 2.780790, {'classification_error_evaluator': 0.9453125} Pass 0, Batch 0, Cost 0.125650
# Pass 0, Batch 100, Cost 0.635356, {'classification_error_evaluator': 0.2109375} Pass 100, Batch 0, Cost 0.161387
# Pass 0, Batch 200, Cost 0.326094, {'classification_error_evaluator': 0.1328125} Pass 200, Batch 0, Cost 0.040036
# Pass 0, Batch 300, Cost 0.361920, {'classification_error_evaluator': 0.1015625} Pass 300, Batch 0, Cost 0.023391
# Pass 0, Batch 400, Cost 0.410101, {'classification_error_evaluator': 0.125} Pass 400, Batch 0, Cost 0.005856
# Test with Pass 0, Cost 0.326659, {'classification_error_evaluator': 0.09470000118017197} Pass 500, Batch 0, Cost 0.003315
Pass 600, Batch 0, Cost 0.009977
Pass 700, Batch 0, Cost 0.020959
Pass 800, Batch 0, Cost 0.105560
Pass 900, Batch 0, Cost 0.239809
Test with Epoch 0, avg_cost: 0.053097883707459624, acc: 0.9822850318471338
``` ```
训练之后，检查模型的预测准确度。用 MNIST 训练的时候，一般 softmax回归模型的分类准确率为约为 92.34%，多层感知器为97.66%，卷积神经网络可以达到 99.20%。 训练之后，检查模型的预测准确度。用 MNIST 训练的时候，一般 softmax回归模型的分类准确率为约为 92.34%，多层感知器为97.66%，卷积神经网络可以达到 99.20%。
...@@ -353,27 +418,50 @@ trainer.train( ...@@ -353,27 +418,50 @@ trainer.train(
## 应用模型 ## 应用模型
可以使用训练好的模型对手写体数字图片进行分类，下面程序展示了如何使用paddle.infer接口进行推断。 可以使用训练好的模型对手写体数字图片进行分类，下面程序展示了如何使用 `fluid.Inferencer` 接口进行推断。
### Inference 配置
`Inference` 需要一个 `infer_func` 和 `param_path` 来设置网络和经过训练的参数。
我们可以简单地插入在此之前定义的分类器。
```python ```python
from PIL import Image inferencer = fluid.Inferencer(
import numpy as np # infer_func=softmax_regression, # uncomment for softmax regression
# infer_func=multilayer_perceptron, # uncomment for MLP
infer_func=convolutional_neural_network, # uncomment for LeNet5
param_path=params_dirname,
place=place)
```
### 生成预测输入数据
`infer_3.png` 是数字 3 的一个示例图像。把它变成一个 numpy 数组以匹配数据馈送格式。
```python
# Prepare the test image
import os import os
import numpy as np
from PIL import Image
def load_image(file): def load_image(file):
im = Image.open(file).convert('L') im = Image.open(file).convert('L')
im = im.resize((28, 28), Image.ANTIALIAS) im = im.resize((28, 28), Image.ANTIALIAS)
im = np.array(im).astype(np.float32).flatten() im = np.array(im).reshape(1, 1, 28, 28).astype(np.float32)
im = im / 255.0 * 2.0 - 1.0 im = im / 255.0 * 2.0 - 1.0
return im return im
test_data = [] cur_dir = cur_dir = os.getcwd()
cur_dir = os.getcwd() img = load_image(cur_dir + '/image/infer_3.png')
test_data.append((load_image(cur_dir + '/image/infer_3.png'),)) ```
### 预测
probs = paddle.infer( 现在我们准备做预测。
output_layer=predict, parameters=parameters, input=test_data)
lab = np.argsort(-probs) # probs and lab are the results of one batch data ```python
print "Label of image/infer_3.png is: %d" % lab[0][0] results = inferencer.infer({'img': img})
lab = np.argsort(results) # probs and lab are the results of one batch data
print "Label of image/infer_3.png is: %d" % lab[0][0][-1]
``` ```
## 总结 ## 总结
......
02.recognize_digits/index.html
浏览文件 @ 63b5ec3f
此差异已折叠。
02.recognize_digits/train.py
浏览文件 @ 63b5ec3f
import os import os
from PIL import Image from PIL import Image
import numpy as np import numpy as np
import paddle.v2 as paddle import paddle
import paddle.fluid as fluid
with_gpu = os.getenv('WITH_GPU', '0') != '0'
def softmax_regression():
def softmax_regression(img): img = fluid.layers.data(name='img', shape=[1, 28, 28], dtype='float32')
predict = paddle.layer.fc( predict = fluid.layers.fc(input=img, size=10, act='softmax')
input=img, size=10, act=paddle.activation.Softmax())
return predict return predict
def multilayer_perceptron(img): def multilayer_perceptron():
# The first fully-connected layer img = fluid.layers.data(name='img', shape=[1, 28, 28], dtype='float32')
hidden1 = paddle.layer.fc(input=img, size=128, act=paddle.activation.Relu()) # first fully-connected layer, using ReLu as its activation function
# The second fully-connected layer and the according activation function hidden = fluid.layers.fc(input=img, size=128, act='relu')
hidden2 = paddle.layer.fc( # second fully-connected layer, using ReLu as its activation function
input=hidden1, size=64, act=paddle.activation.Relu()) hidden = fluid.layers.fc(input=hidden, size=64, act='relu')
# The thrid fully-connected layer, note that the hidden size should be 10, # The thrid fully-connected layer, note that the hidden size should be 10,
# which is the number of unique digits # which is the number of unique digits
predict = paddle.layer.fc( prediction = fluid.layers.fc(input=hidden, size=10, act='softmax')
input=hidden2, size=10, act=paddle.activation.Softmax()) return prediction
return predict
def convolutional_neural_network(img): def convolutional_neural_network():
# first conv layer img = fluid.layers.data(name='img', shape=[1, 28, 28], dtype='float32')
conv_pool_1 = paddle.networks.simple_img_conv_pool( # first conv pool
conv_pool_1 = fluid.nets.simple_img_conv_pool(
input=img, input=img,
filter_size=5, filter_size=5,
num_filters=20, num_filters=20,
num_channel=1,
pool_size=2, pool_size=2,
pool_stride=2, pool_stride=2,
act=paddle.activation.Relu()) act="relu")
# second conv layer conv_pool_1 = fluid.layers.batch_norm(conv_pool_1)
conv_pool_2 = paddle.networks.simple_img_conv_pool( # second conv pool
conv_pool_2 = fluid.nets.simple_img_conv_pool(
input=conv_pool_1, input=conv_pool_1,
filter_size=5, filter_size=5,
num_filters=50, num_filters=50,
num_channel=20,
pool_size=2, pool_size=2,
pool_stride=2, pool_stride=2,
act=paddle.activation.Relu()) act="relu")
# fully-connected layer # output layer with softmax activation function. size = 10 since there are only 10 possible digits.
predict = paddle.layer.fc( prediction = fluid.layers.fc(input=conv_pool_2, size=10, act='softmax')
input=conv_pool_2, size=10, act=paddle.activation.Softmax()) return prediction
return predict
def main():
paddle.init(use_gpu=with_gpu, trainer_count=1)
# define network topology def train_program():
images = paddle.layer.data( label = fluid.layers.data(name='label', shape=[1], dtype='int64')
name='pixel', type=paddle.data_type.dense_vector(784))
label = paddle.layer.data(
name='label', type=paddle.data_type.integer_value(10))
# Here we can build the prediction network in different ways. Please # Here we can build the prediction network in different ways. Please
# choose one by uncomment corresponding line. # predict = softmax_regression() # uncomment for Softmax
# predict = softmax_regression(images) # predict = multilayer_perceptron() # uncomment for MLP
# predict = multilayer_perceptron(images) predict = convolutional_neural_network() # uncomment for LeNet5
predict = convolutional_neural_network(images)
# Calculate the cost from the prediction and label.
cost = fluid.layers.cross_entropy(input=predict, label=label)
avg_cost = fluid.layers.mean(cost)
acc = fluid.layers.accuracy(input=predict, label=label)
return [avg_cost, acc]
def optimizer_program():
return fluid.optimizer.Adam(learning_rate=0.001)
cost = paddle.layer.classification_cost(input=predict, label=label)
parameters = paddle.parameters.create(cost) def main():
train_reader = paddle.batch(
paddle.reader.shuffle(paddle.dataset.mnist.train(), buf_size=500),
batch_size=64)
test_reader = paddle.batch(paddle.dataset.mnist.test(), batch_size=64)
optimizer = paddle.optimizer.Momentum( use_cuda = os.getenv('WITH_GPU', '0') != '0'
learning_rate=0.1 / 128.0, place = fluid.CUDAPlace(0) if use_cuda else fluid.CPUPlace()
momentum=0.9,
regularization=paddle.optimizer.L2Regularization(rate=0.0005 * 128))
trainer = paddle.trainer.SGD( trainer = fluid.Trainer(
cost=cost, parameters=parameters, update_equation=optimizer) train_func=train_program, place=place, optimizer_func=optimizer_program)
# Save the parameter into a directory. The Inferencer can load the parameters from it to do infer
params_dirname = "recognize_digits_network.inference.model"
lists = [] lists = []
def event_handler(event): def event_handler(event):
if isinstance(event, paddle.event.EndIteration): if isinstance(event, fluid.EndStepEvent):
if event.batch_id % 100 == 0: if event.step % 100 == 0:
print "Pass %d, Batch %d, Cost %f, %s" % ( # event.metrics maps with train program return arguments.
event.pass_id, event.batch_id, event.cost, event.metrics) # event.metrics[0] will yeild avg_cost and event.metrics[1] will yeild acc in this example.
if isinstance(event, paddle.event.EndPass): print "Pass %d, Batch %d, Cost %f" % (event.step, event.epoch,
# save parameters event.metrics[0])
with open('params_pass_%d.tar' % event.pass_id, 'w') as f:
trainer.save_parameter_to_tar(f)
result = trainer.test(reader=paddle.batch( if isinstance(event, fluid.EndEpochEvent):
paddle.dataset.mnist.test(), batch_size=128)) avg_cost, acc = trainer.test(
print "Test with Pass %d, Cost %f, %s\n" % ( reader=test_reader, feed_order=['img', 'label'])
event.pass_id, result.cost, result.metrics)
lists.append((event.pass_id, result.cost,
result.metrics['classification_error_evaluator']))
print("Test with Epoch %d, avg_cost: %s, acc: %s" %
(event.epoch, avg_cost, acc))
# save parameters
trainer.save_params(params_dirname)
lists.append((event.epoch, avg_cost, acc))
# Train the model now
trainer.train( trainer.train(
reader=paddle.batch( num_epochs=5,
paddle.reader.shuffle(paddle.dataset.mnist.train(), buf_size=8192),
batch_size=128),
event_handler=event_handler, event_handler=event_handler,
num_passes=5) reader=train_reader,
feed_order=['img', 'label'])
# find the best pass # find the best pass
best = sorted(lists, key=lambda list: float(list[1]))[0] best = sorted(lists, key=lambda list: float(list[1]))[0]
print 'Best pass is %s, testing Avgcost is %s' % (best[0], best[1]) print 'Best pass is %s, testing Avgcost is %s' % (best[0], best[1])
print 'The classification accuracy is %.2f%%' % (100 - float(best[2]) * 100) print 'The classification accuracy is %.2f%%' % (float(best[2]) * 100)
def load_image(file): def load_image(file):
im = Image.open(file).convert('L') im = Image.open(file).convert('L')
im = im.resize((28, 28), Image.ANTIALIAS) im = im.resize((28, 28), Image.ANTIALIAS)
im = np.array(im).astype(np.float32).flatten() im = np.array(im).reshape(1, 1, 28, 28).astype(np.float32)
im = im / 255.0 * 2.0 - 1.0 im = im / 255.0 * 2.0 - 1.0
return im return im
test_data = []
cur_dir = os.path.dirname(os.path.realpath(__file__)) cur_dir = os.path.dirname(os.path.realpath(__file__))
test_data.append((load_image(cur_dir + '/image/infer_3.png'), )) img = load_image(cur_dir + '/image/infer_3.png')
inferencer = fluid.Inferencer(
probs = paddle.infer( # infer_func=softmax_regression, # uncomment for softmax regression
output_layer=predict, parameters=parameters, input=test_data) # infer_func=multilayer_perceptron, # uncomment for MLP
lab = np.argsort(-probs) # probs and lab are the results of one batch data infer_func=convolutional_neural_network, # uncomment for LeNet5
print "Label of image/infer_3.png is: %d" % lab[0][0] param_path=params_dirname,
place=place)
results = inferencer.infer({'img': img})
lab = np.argsort(results) # probs and lab are the results of one batch data
print "Label of image/infer_3.png is: %d" % lab[0][0][-1]
if __name__ == '__main__': if __name__ == '__main__':
......
03.image_classification/README.md
浏览文件 @ 63b5ec3f
此差异已折叠。
03.image_classification/index.html
浏览文件 @ 63b5ec3f
此差异已折叠。
03.image_classification/resnet.py
浏览文件 @ 63b5ec3f
此差异已折叠。
03.image_classification/train.py
浏览文件 @ 63b5ec3f
此差异已折叠。
03.image_classification/vgg.py
浏览文件 @ 63b5ec3f
此差异已折叠。
04.word2vec/README.cn.md
浏览文件 @ 63b5ec3f
此差异已折叠。
04.word2vec/README.md
浏览文件 @ 63b5ec3f
此差异已折叠。
04.word2vec/index.html
浏览文件 @ 63b5ec3f
此差异已折叠。
04.word2vec/train.py
浏览文件 @ 63b5ec3f
此差异已折叠。
05.recommender_system/README.cn.md
浏览文件 @ 63b5ec3f
此差异已折叠。
05.recommender_system/README.md
浏览文件 @ 63b5ec3f
此差异已折叠。
05.recommender_system/index.html
浏览文件 @ 63b5ec3f
此差异已折叠。
05.recommender_system/train.py
浏览文件 @ 63b5ec3f
此差异已折叠。
06.understand_sentiment/README.md
浏览文件 @ 63b5ec3f
此差异已折叠。
06.understand_sentiment/index.html
浏览文件 @ 63b5ec3f
此差异已折叠。
06.understand_sentiment/train.py 已删除 100644 → 0
浏览文件 @ 330cf3ee
此差异已折叠。
06.understand_sentiment/train_conv.py 0 → 100644
浏览文件 @ 63b5ec3f
此差异已折叠。
06.understand_sentiment/train_dyn_rnn.py 0 → 100644
浏览文件 @ 63b5ec3f
此差异已折叠。
06.understand_sentiment/train_stacked_lstm.py 0 → 100644
浏览文件 @ 63b5ec3f
此差异已折叠。
07.label_semantic_roles/README.md
浏览文件 @ 63b5ec3f
此差异已折叠。
07.label_semantic_roles/train.py
浏览文件 @ 63b5ec3f
此差异已折叠。
08.machine_translation/README.cn.md
浏览文件 @ 63b5ec3f
此差异已折叠。
08.machine_translation/README.md
浏览文件 @ 63b5ec3f
此差异已折叠。
08.machine_translation/index.html
浏览文件 @ 63b5ec3f
此差异已折叠。
08.machine_translation/infer.py 0 → 100644
浏览文件 @ 63b5ec3f
此差异已折叠。
08.machine_translation/train.py
浏览文件 @ 63b5ec3f
此差异已折叠。
Markdown is supported
0% 或 .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑！
想要评论请 注册 或