Skip to content

P
Paddle

Crayon鑫 / Paddle
与 Fork 源项目一致

Fork自 PaddlePaddle / Paddle

通知 1
Star 1
Fork 0
P
Paddle
提交 0a404363 编写于 作者: T Travis CI
浏览文件
操作

Deploy to GitHub Pages: 8fe7bf38

上级 c15f0925
展开全部 隐藏空白更改
内联 并排
Showing with 30 addition and 9082 deletion
develop/doc/_sources/howto/deep_model/rnn/rnn_config_en.rst.txt
浏览文件 @ 0a404363
......@@ -19,7 +19,7 @@ Simple Gated Recurrent Neural Network
Recurrent neural network process a sequence at each time step sequentially. An example of the architecture of LSTM is listed below.
.. image:: ../../../tutorials/sentiment_analysis/src/bi_lstm.jpg
.. image:: src/bi_lstm.jpg
:align: center
Generally speaking, a recurrent network perform the following operations from :math:`t=1` to :math:`t=T`, or reversely from :math:`t=T` to :math:`t=1`.
......@@ -78,7 +78,7 @@ Sequence to Sequence Model with Attention
-----------------------------------------
We will use the sequence to sequence model with attention as an example to demonstrate how you can configure complex recurrent neural network models. An illustration of the sequence to sequence model with attention is shown in the following figure.
.. image:: ../../../tutorials/text_generation/encoder-decoder-attention-model.png
.. image:: src/encoder-decoder-attention-model.png
:align: center
In this model, the source sequence :math:`S = \{s_1, \dots, s_T\}` is encoded with a bidirectional gated recurrent neural networks. The hidden states of the bidirectional gated recurrent neural network :math:`H_S = \{H_1, \dots, H_T\}` is called *encoder vector* The decoder is a gated recurrent neural network. When decoding each token :math:`y_t`, the gated recurrent neural network generates a set of weights :math:`W_S^t = \{W_1^t, \dots, W_T^t\}`, which are used to compute a weighted sum of the encoder vector. The weighted sum of the encoder vector is utilized to condition the generation of the token :math:`y_t`.
......
develop/doc/_sources/tutorials/image_classification/index_en.md.txt 已删除 100644 → 0
浏览文件 @ c15f0925
Image Classification Tutorial
==============================
This tutorial will guide you through training a convolutional neural network to classify objects using the CIFAR-10 image classification dataset.
As shown in the following figure, the convolutional neural network can recognize the main object in images, and output the classification result.
<center>![Image Classification](./image_classification.png)</center>
## Data Preparation
First, download CIFAR-10 dataset. CIFAR-10 dataset can be downloaded from its official website.
<https://www.cs.toronto.edu/~kriz/cifar.html>
We have prepared a script to download and process CIFAR-10 dataset. The script will download CIFAR-10 dataset from the official dataset.
It will convert it to jpeg images and organize them into a directory with the required structure for the tutorial. Make sure that you have installed pillow and its dependents.
Consider the following commands:
1. install pillow dependents
```bash
sudo apt-get install libjpeg-dev
pip install pillow
```
2. download data and preparation
```bash
cd demo/image_classification/data/
sh download_cifar.sh
```
The CIFAR-10 dataset consists of 60000 32x32 color images in 10 classes, with 6000 images per class. There are 50000 training images and 10000 test images.
Here are the classes in the dataset, as well as 10 random images from each:
<center>![Image Classification](./cifar.png)</center>
After downloading and converting, we should find a directory (cifar-out) containing the dataset in the following format:
```
train
---airplane
---automobile
---bird
---cat
---deer
---dog
---frog
---horse
---ship
---truck
test
---airplane
---automobile
---bird
---cat
---deer
---dog
---frog
---horse
---ship
---truck
```
It has two directories:`train` and `test`. These two directories contain training data and testing data of CIFAR-10, respectively. Each of these two folders contains 10 sub-folders, ranging from `airplane` to `truck`. Each sub-folder contains images with the corresponding label. After the images are organized into this structure, we are ready to train an image classification model.
## Preprocess
After the data has been downloaded, it needs to be pre-processed into the Paddle format. We can run the following command for preprocessing.
```
cd demo/image_classification/
sh preprocess.sh
```
`preprocess.sh` calls `./demo/image_classification/preprocess.py` to preprocess image data.
```sh
export PYTHONPATH=$PYTHONPATH:../../
data_dir=./data/cifar-out
python preprocess.py -i $data_dir -s 32 -c 1
```
`./demo/image_classification/preprocess.py` has the following arguments
- `-i` or `--input` specifes the input data directory.
- `-s` or `--size` specifies the processed size of images.
- `-c` or `--color` specifes whether images are color images or gray images.
## Model Training
We need to create a model config file before training the model. An example of the config file (vgg_16_cifar.py) is listed below. **Note**, it is slightly different from the `vgg_16_cifar.py` which also applies to the prediction.
```python
from paddle.trainer_config_helpers import *
data_dir='data/cifar-out/batches/'
meta_path=data_dir+'batches.meta'
args = {'meta':meta_path, 'mean_img_size': 32,
'img_size': 32, 'num_classes': 10,
'use_jpeg': 1, 'color': "color"}
define_py_data_sources2(train_list=data_dir+"train.list",
test_list=data_dir+'test.list',
module='image_provider',
obj='processData',
args=args)
settings(
batch_size = 128,
learning_rate = 0.1 / 128.0,
learning_method = MomentumOptimizer(0.9),
regularization = L2Regularization(0.0005 * 128))
img = data_layer(name='image', size=3*32*32)
lbl = data_layer(name="label", size=10)
# small_vgg is predined in trainer_config_helpers.network
predict = small_vgg(input_image=img, num_channels=3)
outputs(classification_cost(input=predict, label=lbl))
```
The first line imports python functions for defining networks.
```python
from paddle.trainer_config_helpers import *
```
Then define an `define_py_data_sources2` which use python data provider
interface. The arguments in `args` are used in `image_provider.py` which
yeilds image data and transform them to Paddle.
- `meta`: the mean value of training set.
- `mean_img_size`: the size of mean feature map.
- `img_size`: the height and width of input image.
- `num_classes`: the number of classes.
- `use_jpeg`: the data storage type when preprocessing.
- `color`: specify color image.
`settings` specifies the training algorithm. In the following example,
it specifies learning rate as 0.1, but divided by batch size, and the weight decay
is 0.0005 and multiplied by batch size.
```python
settings(
batch_size = 128,
learning_rate = 0.1 / 128.0,
learning_method = MomentumOptimizer(0.9),
regularization = L2Regularization(0.0005 * 128)
)
```
The `small_vgg` specifies the network. We use a small version of VGG convolutional network as our network
for classification. A description of VGG network can be found here [http://www.robots.ox.ac.uk/~vgg/research/very_deep/](http://www.robots.ox.ac.uk/~vgg/research/very_deep/).
```python
# small_vgg is predined in trainer_config_helpers.network
predict = small_vgg(input_image=img, num_channels=3)
```
After writing the config, we can train the model by running the script train.sh.
```bash
config=vgg_16_cifar.py
output=./cifar_vgg_model
log=train.log
paddle train \
--config=$config \
--dot_period=10 \
--log_period=100 \
--test_all_data_in_one_period=1 \
--use_gpu=1 \
--save_dir=$output \
2>&1 | tee $log
python -m paddle.utils.plotcurve -i $log > plot.png
```
- Here we use GPU mode to train. If you have no gpu environment, just set `use_gpu=0`.
- `./demo/image_classification/vgg_16_cifar.py` is the network and data configuration file. The meaning of the other flags can be found in the documentation of the command line flags.
- The script `plotcurve.py` requires the python module of `matplotlib`, so if it fails, maybe you need to install `matplotlib`.
After training finishes, the training and testing error curves will be saved to `plot.png` using `plotcurve.py` script. An example of the plot is shown below:
<center>![Training and testing curves.](./plot.png)</center>
## Prediction
After we train the model, the model file as well as the model parameters are stored in path `./cifar_vgg_model/pass-%05d`. For example, the model of the 300-th pass is stored at `./cifar_vgg_model/pass-00299`.
To make a prediction for an image, one can run `predict.sh` as follows. The script will output the label of the classfiication.
```
sh predict.sh
```
predict.sh:
```
model=cifar_vgg_model/pass-00299/
image=data/cifar-out/test/airplane/seaplane_s_000978.png
use_gpu=1
python prediction.py $model $image $use_gpu
```
## Exercise
Train a image classification of birds using VGG model and CUB-200 dataset. The birds dataset can be downloaded here. It contains an image dataset with photos of 200 bird species (mostly North American).
<http://www.vision.caltech.edu/visipedia/CUB-200.html>
## Delve into Details
### Convolutional Neural Network
A Convolutional Neural Network is a feedforward neural network that uses convolution layers. It is very suitable for building neural networks that process and understand images. A standard convolutional neural network is shown below:
![Convolutional Neural Network](./lenet.png)
Convolutional Neural Network contains the following layers:
- Convolutional layer: It uses convolution operation to extract features from an image or a feature map.
- Pooling layer: It uses max-pooling to downsample feature maps.
- Fully Connected layer: It uses fully connected connections to transform features.
Convolutional Neural Network achieves amazing performance for image classification because it exploits two important characteristics of images: *local correlation* and *spatial invariance*. By iteratively applying convolution and max-pooing operations, convolutional neural network can well represent these two characteristics of images.
For more details of how to define layers and their connections, please refer to the documentation of layers.
develop/doc/_sources/tutorials/index_en.md.txt 已删除 100644 → 0
浏览文件 @ c15f0925
# TUTORIALS
There are several examples and demos here.
* [Quick Start](quick_start/index_en.md)
* [MovieLens Regression](rec/ml_regression_en.rst)
* [Image Classification](image_classification/index_en.md)
* [Sentiment Analysis](sentiment_analysis/index_en.md)
* [Semantic Role Labeling](semantic_role_labeling/index_en.md)
* [Text Generation](text_generation/index_en.md)
* [Image Auto-Generation](gan/index_en.md)
## Model Zoo
* [ImageNet: ResNet](imagenet_model/resnet_model_en.md)
* [Embedding: Chinese Word](embedding_model/index_en.md)
develop/doc/_sources/tutorials/rec/ml_dataset_en.md.txt 已删除 100644 → 0
浏览文件 @ c15f0925
```eval_rst
.. _demo_ml_dataset:
```
# MovieLens Dataset
The [MovieLens Dataset](http://grouplens.org/datasets/movielens/) was collected by GroupLens Research.
The data set contains some user information, movie information, and many movie ratings from \[1-5\].
The data sets have many version depending on the size of set.
We use [MovieLens 1M Dataset](http://files.grouplens.org/datasets/movielens/ml-1m.zip) as a demo dataset, which contains
1 million ratings from 6000 users on 4000 movies. Released 2/2003.
## Dataset Features
In [ml-1m Dataset](http://files.grouplens.org/datasets/movielens/ml-1m.zip), there are many features in these dataset.
The data files (which have ".dat" extension) in [ml-1m Dataset](http://files.grouplens.org/datasets/movielens/ml-1m.zip)
is basically CSV file that delimiter is "::". The description in README we quote here.
### RATINGS FILE DESCRIPTION(ratings.dat)
All ratings are contained in the file "ratings.dat" and are in the
following format:
UserID::MovieID::Rating::Timestamp
- UserIDs range between 1 and 6040
- MovieIDs range between 1 and 3952
- Ratings are made on a 5-star scale (whole-star ratings only)
- Timestamp is represented in seconds since the epoch as returned by time(2)
- Each user has at least 20 ratings
### USERS FILE DESCRIPTION(users.dat)
User information is in the file "users.dat" and is in the following
format:
UserID::Gender::Age::Occupation::Zip-code
All demographic information is provided voluntarily by the users and is
not checked for accuracy. Only users who have provided some demographic
information are included in this data set.
- Gender is denoted by a "M" for male and "F" for female
- Age is chosen from the following ranges:
* 1: "Under 18"
* 18: "18-24"
* 25: "25-34"
* 35: "35-44"
* 45: "45-49"
* 50: "50-55"
* 56: "56+"
- Occupation is chosen from the following choices:
* 0: "other" or not specified
* 1: "academic/educator"
* 2: "artist"
* 3: "clerical/admin"
* 4: "college/grad student"
* 5: "customer service"
* 6: "doctor/health care"
* 7: "executive/managerial"
* 8: "farmer"
* 9: "homemaker"
* 10: "K-12 student"
* 11: "lawyer"
* 12: "programmer"
* 13: "retired"
* 14: "sales/marketing"
* 15: "scientist"
* 16: "self-employed"
* 17: "technician/engineer"
* 18: "tradesman/craftsman"
* 19: "unemployed"
* 20: "writer"
### MOVIES FILE DESCRIPTION(movies.dat)
Movie information is in the file "movies.dat" and is in the following
format:
MovieID::Title::Genres
- Titles are identical to titles provided by the IMDB (including
year of release)
- Genres are pipe-separated and are selected from the following genres:
* Action
* Adventure
* Animation
* Children's
* Comedy
* Crime
* Documentary
* Drama
* Fantasy
* Film-Noir
* Horror
* Musical
* Mystery
* Romance
* Sci-Fi
* Thriller
* War
* Western
- Some MovieIDs do not correspond to a movie due to accidental duplicate
entries and/or test entries
- Movies are mostly entered by hand, so errors and inconsistencies may exist
develop/doc/_sources/tutorials/rec/ml_regression_en.rst.txt 已删除 100644 → 0
浏览文件 @ c15f0925
Regression MovieLens Ratting
============================
Here we demonstrate a **Cosine Similarity Regression** job in movie lens dataset.
This demo will show how paddle does (word) embedding job,
handles the similarity regression,
the character-level convolutional networks for text, and how does paddle handle
multiple types of inputs.
Note that the model structure is not fine-tuned and just a demo to show how paddle works.
YOU ARE WELCOME TO BUILD A BETTER DEMO
BY USING PADDLEPADDLE, AND LET US KNOW TO MAKE THIS DEMO BETTER.
Data Preparation
````````````````
Download and extract dataset
''''''''''''''''''''''''''''
We use :ref:`demo_ml_dataset` here.
To download and unzip the dataset, simply run the following commands.
.. code-block:: bash
cd demo/recommendation/data
./ml_data.sh
And the directory structure of :code:`demo/recommendation/data/ml-1m` is:
.. code-block:: text
+--ml-1m
+--- movies.dat # movie features
+--- ratings.dat # ratings
+--- users.dat # user features
+--- README # dataset description
Field config file
'''''''''''''''''
**Field config file** is used to specify the fields of the dataset and the file format,
i.e, specific **WHAT** type it is in each feature file.
The field config file of ml-1m shows in :code:`demo/recommendation/data/config.json`.
It specifics the field types and file names: 1) there are four types of field for user file\: id, gender, age and occupation;
2) the filename is "users.dat", and the delimiter of file is "::".
.. include:: ../../../demo/recommendation/data/config.json
:code: json
:literal:
Preprocess Data
```````````````
You need to install python 3rd party libraries.
IT IS HIGHLY RECOMMEND TO USE VIRTUALENV MAKE A CLEAN PYTHON ENVIRONMENT.
.. code-block:: bash
pip install -r requirements.txt
The general command for preprocessing the dataset is:
.. code-block:: bash
cd demo/recommendation
./preprocess.sh
And the detail steps are introduced as follows.
Extract Movie/User features to python object
'''''''''''''''''''''''''''''''''''''''''''''
There are many features in movie or user in movielens 1m dataset.
Each line of rating file just provides a Movie/User id to refer each movie or user.
We process the movie/user feature file first, and pickle the feature (**Meta**) object as a file.
Meta config file
................
**Meta config file** is used to specific **HOW** to parse each field in dataset.
It could be translated from field config file, or written by hand.
Its file format could be either json or yaml syntax file. Parser will automatically choose the file format by extension name.
To convert Field config file to meta config file, just run:
.. code-block:: bash
cd demo/recommendation/data
python config_generator.py config.json > meta_config.json
The meta config file shows below:
.. include:: ../../../demo/recommendation/data/meta_config.json
:code: json
:literal:
There are two kinds of features in meta\: movie and user.
* in movie file, whose name is movies.dat
* we just split each line by "::"
* pos 0 is id.
* pos 1 feature:
* name is title.
* it uses regex to parse this feature.
* it is a char based word embedding feature.
* it is a sequence.
* pos 2 feature:
* name is genres.
* type is one hot dense vector.
* dictionary is auto generated by parsing, each key is split by '|'
* in user file, whose name is users.dat
* we just split each line by "::"
* pos 0 is id.
* pos 1 feature:
* name is gender
* just simple char based embedding.
* pos 2 feature:
* name is age
* just whole word embedding.
* embedding id will be sort by word.
* pos 3 feature:
* name is occupation.
* just simple whole word embedding.
Meta file
'''''''''
After having meta config file, we can generate **Meta file**, a python pickle object which stores movie/user information.
The following commands could be run to generate it.
.. code-block:: bash
python meta_generator.py ml-1m meta.bin --config=meta_config.json
And the structure of the meta file :code:`meta.bin` is:
.. code-block:: text
+--+ movie
| +--+ __meta__
| | +--+ raw_meta # each feature meta config. list
| | | +
| | | | # ID Field, we use id as key
| | | +--+ {'count': 3883, 'max': 3952, 'is_key': True, 'type': 'id', 'min': 1}
| | | |
| | | | # Titile field, the dictionary list of embedding.
| | | +--+ {'dict': [ ... ], 'type': 'embedding', 'name': 'title', 'seq': 'sequence'}
| | | |
| | | | # Genres field, the genres dictionary
| | | +--+ {'dict': [ ... ], 'type': 'one_hot_dense', 'name': 'genres'}
| | |
| | +--+ feature_map [1, 2] # a list for raw_meta index for feature field.
| | # it means there are 2 features for each key.
| | # * 0 offset of feature is raw_meta[1], Title.
| | # * 1 offset of feature is raw_meta[2], Genres.
| |
| +--+ 1 # movie 1 features
| | +
| | +---+ [[...], [...]] # title ids, genres dense vector
| |
| +--+ 2
| |
| +--+ ...
|
+--- user
+--+ __meta__
| +
| +--+ raw_meta
| | +
| | +--+ id field as user
| | |
| | +--+ {'dict': ['F', 'M'], 'type': 'embedding', 'name': 'gender', 'seq': 'no_sequence'}
| | |
| | +--+ {'dict': ['1', '18', '25', '35', '45', '50', '56'], 'type': 'embedding', 'name': 'age', 'seq': 'no_sequence'}
| | |
| | +--+ {'dict': [...], 'type': 'embedding', 'name': 'occupation', 'seq': 'no_sequence'}
| |
| +--+ feature_map [1, 2, 3]
|
+--+ 1 # user 1 features
|
+--+ 2
+--+ ...
Split Training/Testing files
''''''''''''''''''''''''''''
We split :code:`ml-1m/ratings.dat` into a training and testing file. The way to split file is for each user, we split the
rating by two parts. So each user in testing file will have some rating information in training file.
Use :code:`separate.py` to separate the training and testing file.
.. code-block:: bash
python split.py ml-1m/ratings.dat --delimiter="::" --test_ratio=0.1
Then two files will be generated\: :code:`ml-1m/ratings.dat.train` and :code:`ml-1m/rating.data.test`.
Move them to workspace :code:`data`, shuffle the train file, and prepare the file list for paddle train.
.. code-block:: bash
shuf ml-1m/ratings.dat.train > ratings.dat.train
cp ml-1m/ratings.dat.test .
echo "./data/ratings.dat.train" > train.list
echo "./data/ratings.dat.test" > test.list
Neural Network Configuration
````````````````````````````
Trainer Config File
'''''''''''''''''''
The network structure shows below.
.. image:: rec_regression_network.png
:align: center
:alt: rec_regression_network
The demo's neural network config file :code:`trainer_config.py` show as below.
.. literalinclude:: ../../../demo/recommendation/trainer_config.py
:language: python
:lines: 15-
In this :code:`trainer_config.py`, we just map each feature type to
a feature vector, following shows how to map each feature to a vector shows below.
* :code:`id`\: Just simple embedding, and then add to fully connected layer.
* :code:`embedding`\:
- if is_sequence, get the embedding and do a text convolutional operation,
get the average pooling result.
- if not sequence, get the embedding and add to fully connected layer.
* :code:`one_host_dense`\:
- just two fully connected layer.
Then we combine each features of movie into one movie feature by a
:code:`fc_layer` with multiple inputs, and do the same thing to user features,
get one user feature. Then we calculate the cosine similarity of these two
features.
In these networks, we use several APIs in :ref:`api_trainer_config` . There are
* Data Layer, :ref:`api_trainer_config_helpers_layers_data_layer`
* Fully Connected Layer, :ref:`api_trainer_config_helpers_layers_fc_layer`
* Embedding Layer, :ref:`api_trainer_config_helpers_layers_embedding_layer`
* Context Projection Layer, :ref:`api_trainer_config_helpers_layers_context_projection`
* Pooling Layer, :ref:`api_trainer_config_helpers_layers_pooling_layer`
* Cosine Similarity Layer, :ref:`api_trainer_config_helpers_layers_cos_sim`
* Text Convolution Pooling Layer, :ref:`api_trainer_config_helpers_network_text_conv_pool`
* Declare Python Data Sources :ref:`api_trainer_config_helpers_data_sources`.
Data Provider
'''''''''''''
.. literalinclude:: ../../../demo/recommendation/dataprovider.py
:language: python
:lines: 15-
The data provider just read the meta.bin and rating file, yield each sample for training.
In this :code:`dataprovider.py`, we should set\:
* obj.slots\: The feature types and dimension.
* use_seq\: Whether this :code:`dataprovider.py` in sequence mode or not.
* process\: Return each sample of data to :code:`paddle`.
The data provider details document see :ref:`api_pydataprovider2`.
Train
`````
After prepare data, config network, writting data provider, now we can run paddle training.
The :code:`run.sh` is shown as follow:
.. literalinclude:: ../../../demo/recommendation/run.sh
:language: bash
:lines: 16-
It just start a paddle training process, write the log to :code:`log.txt`,
then print it on screen.
Each command line argument in :code:`run.sh`, please refer to the :ref:`cmd_line_index` page. The short description of these arguments is shown as follow.
* config\: Tell paddle which file is neural network configuration.
* save_dir\: Tell paddle save model into :code:`./output`.
* use_gpu\: Use gpu or not. Default is false.
* trainer_count\: The compute thread in one machine.
* test_all_data_in_one_period\: Test All Data during one test period. Otherwise,
will test a :code:`batch_size` data in one test period.
* log_period\: Print log after train :code:`log_period` batches.
* dot_period\: Print a :code:`.` after train :code:`dot_period` batches.
* num_passes\: Train at most :code:`num_passes`.
If training process starts successfully, the output likes follow:
.. code-block:: text
I0601 08:07:22.832059 10549 TrainerInternal.cpp:157] Batch=100 samples=160000 AvgCost=4.13494 CurrentCost=4.13494 Eval: CurrentEval:
I0601 08:07:50.672627 10549 TrainerInternal.cpp:157] Batch=200 samples=320000 AvgCost=3.80957 CurrentCost=3.48421 Eval: CurrentEval:
I0601 08:08:18.877369 10549 TrainerInternal.cpp:157] Batch=300 samples=480000 AvgCost=3.68145 CurrentCost=3.42519 Eval: CurrentEval:
I0601 08:08:46.863963 10549 TrainerInternal.cpp:157] Batch=400 samples=640000 AvgCost=3.6007 CurrentCost=3.35847 Eval: CurrentEval:
I0601 08:09:15.413025 10549 TrainerInternal.cpp:157] Batch=500 samples=800000 AvgCost=3.54811 CurrentCost=3.33773 Eval: CurrentEval:
I0601 08:09:36.058670 10549 TrainerInternal.cpp:181] Pass=0 Batch=565 samples=902826 AvgCost=3.52368 Eval:
I0601 08:09:46.215489 10549 Tester.cpp:101] Test samples=97383 cost=3.32155 Eval:
I0601 08:09:46.215966 10549 GradientMachine.cpp:132] Saving parameters to ./output/model/pass-00000
I0601 08:09:46.233397 10549 ParamUtil.cpp:99] save dir ./output/model/pass-00000
I0601 08:09:46.233438 10549 Util.cpp:209] copy trainer_config.py to ./output/model/pass-00000
I0601 08:09:46.233541 10549 ParamUtil.cpp:147] fileName trainer_config.py
The model is saved in :code:`output/` directory. You can use :code:`Ctrl-C` to stop training whenever you want.
Evaluate and Predict
````````````````````
After training several passes, you can evaluate them and get the best pass. Just run
.. code-block:: bash
./evaluate.sh
You will see messages like this:
.. code-block:: text
Best pass is 00009, error is 3.06949, which means predict get error as 0.875998002281
evaluating from pass output/pass-00009
Then, you can predict what any user will rate a movie. Just run
.. code-block:: bash
python prediction.py 'output/pass-00009/'
Predictor will read user input, and predict scores. It has a command-line user interface as follows:
.. code-block:: text
Input movie_id: 9
Input user_id: 4
Prediction Score is 2.56
Input movie_id: 8
Input user_id: 2
Prediction Score is 3.13
develop/doc/_sources/tutorials/semantic_role_labeling/index_en.md.txt 已删除 100644 → 0
浏览文件 @ c15f0925
```eval_rst
.. _semantic_role_labeling:
```
# Semantic Role labeling Tutorial #
Semantic role labeling (SRL) is a form of shallow semantic parsing whose goal is to discover the predicate-argument structure of each predicate in a given input sentence. SRL is useful as an intermediate step in a wide range of natural language processing tasks, such as information extraction. automatic document categorization and question answering. An instance is as following [1]:
[ <sub>A0</sub> He ] [ <sub>AM-MOD</sub> would ][ <sub>AM-NEG</sub> n’t ] [ <sub>V</sub> accept] [ <sub>A1</sub> anything of value ] from [<sub>A2</sub> those he was writing about ].
- V: verb
- A0: acceptor
- A1: thing accepted
- A2: accepted-from
- A3: Attribute
- AM-MOD: modal
- AM-NEG: negation
Given the verb "accept", the chunks in sentence would play certain semantic roles. Here, the label scheme is from Penn Proposition Bank.
To this date, most of the successful SRL systems are built on top of some form of parsing results where pre-defined feature templates over the syntactic structure are used. This tutorial will present an end-to-end system using deep bidirectional long short-term memory (DB-LSTM)[2] for solving the SRL task, which largely outperforms the previous state-of-the-art systems. The system regards SRL task as the sequence labelling problem.
## Data Description
The relevant paper[2] takes the data set in CoNLL-2005&2012 Shared Task for training and testing. Accordingto data license, the demo adopts the test data set of CoNLL-2005, which can be reached on website.
To download and process the original data, user just need to execute the following command:
```bash
cd data
./get_data.sh
```
Several new files appear in the `data `directory as follows.
```bash
conll05st-release：the test data set of CoNll-2005 shared task
test.wsj.words：the Wall Street Journal data sentences
test.wsj.props: the propositional arguments
feature: the extracted features from data set
```
## Training
### DB-LSTM
Please refer to the Sentiment Analysis demo to learn more about the long short-term memory unit.
Unlike Bidirectional-LSTM that used in Sentiment Analysis demo, the DB-LSTM adopts another way to stack LSTM layer. First a standard LSTM processes the sequence in forward direction. The input and output of this LSTM layer are taken by the next LSTM layer as input, processed in reversed direction. These two standard LSTM layers compose a pair of LSTM. Then we stack LSTM layers pair after pair to obtain the deep LSTM model.
The following figure shows a temporal expanded 2-layer DB-LSTM network.
<center>
![pic](./src/network_arch.png)
</center>
### Features
Two input features play an essential role in this pipeline: predicate (pred) and argument (argu). Two other features: predicate context (ctx-p) and region mark (mr) are also adopted. Because a single predicate word can not exactly describe the predicate information, especially when the same words appear more than one times in a sentence. With the predicate context, the ambiguity can be largely eliminated. Similarly, we use region mark m<sub>r</sub> = 1 to denote the argument position if it locates in the predicate context region, or m<sub>r</sub> = 0 if does not. These four simple features are all we need for our SRL system. Features of one sample with context size set to 1 is showed as following[2]:
<center>
![pic](./src/feature.jpg)
</center>
In this sample, the coresponding labelled sentence is:
[ <sub>A1</sub> A record date ] has [ <sub>AM-NEG</sub> n't ] been [ <sub>V</sub> set ] .
In the demo, we adopt the feature template as above, consists of : `argument`, `predicate`, `ctx-p (p=-1,0,1)`, `mark` and use `B/I/O` scheme to label each argument. These features and labels are stored in `feature` file, and separated by `\t`.
### Data Provider
`dataprovider.py` is the python file to wrap data. `hook()` function is to define the data slots for network. The Six features and label are all IndexSlots.
```
def hook(settings, word_dict, label_dict, **kwargs):
settings.word_dict = word_dict
settings.label_dict = label_dict
#all inputs are integral and sequential type
settings.slots = [
integer_value_sequence(len(word_dict)),
integer_value_sequence(len(predicate_dict)),
integer_value_sequence(len(word_dict)),
integer_value_sequence(len(word_dict)),
integer_value_sequence(len(word_dict)),
integer_value_sequence(len(word_dict)),
integer_value_sequence(len(word_dict)),
integer_value_sequence(2),
integer_value_sequence(len(label_dict))]
```
The corresponding data iterator is as following:
```
@provider(init_hook=hook, should_shuffle=True, calc_batch_size=get_batch_size,
can_over_batch_size=False, cache=CacheType.CACHE_PASS_IN_MEM)
def process(settings, file_name):
with open(file_name, 'r') as fdata:
for line in fdata:
sentence, predicate, ctx_n2, ctx_n1, ctx_0, ctx_p1, ctx_p2, mark, label = \
line.strip().split('\t')
words = sentence.split()
sen_len = len(words)
word_slot = [settings.word_dict.get(w, UNK_IDX) for w in words]
predicate_slot = [settings.predicate_dict.get(predicate)] * sen_len
ctx_n2_slot = [settings.word_dict.get(ctx_n2, UNK_IDX)] * sen_len
ctx_n1_slot = [settings.word_dict.get(ctx_n1, UNK_IDX)] * sen_len
ctx_0_slot = [settings.word_dict.get(ctx_0, UNK_IDX)] * sen_len
ctx_p1_slot = [settings.word_dict.get(ctx_p1, UNK_IDX)] * sen_len
ctx_p2_slot = [settings.word_dict.get(ctx_p2, UNK_IDX)] * sen_len
marks = mark.split()
mark_slot = [int(w) for w in marks]
label_list = label.split()
label_slot = [settings.label_dict.get(w) for w in label_list]
yield word_slot, predicate_slot, ctx_n2_slot, ctx_n1_slot, \
ctx_0_slot, ctx_p1_slot, ctx_p2_slot, mark_slot, label_slot
```
The `process`function yield 9 lists which are 8 features and label.
### Neural Network Config
`db_lstm.py` is the neural network config file to load the dictionaries and define the data provider module and network architecture during the training procedure.
Nine `data_layer` load instances from data provider. Eight features are transformed into embedddings respectively, and mixed by `mixed_layer` . Deep bidirectional LSTM layers extract features for the softmax layer. The objective function is cross entropy of labels.
### Run Training
The script for training is `train.sh`, user just need to execute:
```bash
./train.sh
```
The content in `train.sh`:
```
paddle train \
--config=./db_lstm.py \
--use_gpu=0 \
--log_period=5000 \
--trainer_count=1 \
--show_parameter_stats_period=5000 \
--save_dir=./output \
--num_passes=10000 \
--average_test_period=10000000 \
--init_model_path=./data \
--load_missing_parameter_strategy=rand \
--test_all_data_in_one_period=1 \
2>&1 | tee 'train.log'
```
- \--config=./db_lstm.py : network config file.
- \--use_gpu=false: use CPU to train, set true, if you install GPU version of PaddlePaddle and want to use GPU to train, until now crf_layer do not support GPU
- \--log_period=500: print log every 20 batches.
- \--trainer_count=1: set thread number (or GPU count).
- \--show_parameter_stats_period=5000: show parameter statistic every 100 batches.
- \--save_dir=./output: output path to save models.
- \--num_passes=10000: set pass number, one pass in PaddlePaddle means training all samples in dataset one time.
- \--average_test_period=10000000: do test on average parameter every average_test_period batches
- \--init_model_path=./data: parameter initialization path
- \--load_missing_parameter_strategy=rand: random initialization unexisted parameters
- \--test_all_data_in_one_period=1: test all data in one period
After training, the models will be saved in directory `output`. Our training curve is as following:
<center>
![pic](./src/curve.jpg)
</center>
### Run testing
The script for testing is `test.sh`, user just need to execute:
```bash
./test.sh
```
The main part in `tesh.sh`
```
paddle train \
--config=./db_lstm.py \
--model_list=$model_list \
--job=test \
--config_args=is_test=1 \
```
- \--config=./db_lstm.py: network config file
- \--model_list=$model_list.list: model list file
- \--job=test: indicate the test job
- \--config_args=is_test=1: flag to indicate test
- \--test_all_data_in_one_period=1: test all data in 1 period
### Run prediction
The script for prediction is `predict.sh`, user just need to execute:
```bash
./predict.sh
```
In `predict.sh`, user should offer the network config file, model path, label file, word dictionary file, feature file
```
python predict.py
-c $config_file \
-w $best_model_path \
-l $label_file \
-p $predicate_dict_file \
-d $dict_file \
-i $input_file \
-o $output_file
```
`predict.py` is the main executable python script, which includes functions: load model, load data, data prediction. The network model will output the probability distribution of labels. In the demo, we take the label with maximum probability as result. User can also implement the beam search or viterbi decoding upon the probability distribution matrix.
After prediction, the result is saved in `predict.res`.
## Reference
[1] Martha Palmer, Dan Gildea, and Paul Kingsbury. The Proposition Bank: An Annotated Corpus of Semantic Roles , Computational Linguistics, 31(1), 2005.
[2] Zhou, Jie, and Wei Xu. "End-to-end learning of semantic role labeling using recurrent neural networks." Proceedings of the Annual Meeting of the Association for Computational Linguistics. 2015.
develop/doc/_sources/tutorials/sentiment_analysis/index_en.md.txt 已删除 100644 → 0
浏览文件 @ c15f0925
# Sentiment Analysis Tutorial
Sentiment analysis has many applications. A basic task in sentiment analysis is classifying the polarity of a given text at the document, sentence or feature/aspect level. One simple example is to classify the customer reviews in a shopping website, a tourism website, and group buying websites like Amazon, TaoBao, Tmall etc.
Sentiment analysis is also used to monitor social media based on large amount of reviews or blogs. For example, the researchers analyzed several surveys on consumer confidence and political opinion, found they correlate to sentiment word frequencies in contemporaneous Twitter messages [1]. Another example is to forecast stock movements through analyzing the text content of a daily Twitter blog [2].
On the other hand, grabbing the user comments of products and analyzing their sentiment are useful to understand user preferences for companies, products, even competing products.
This tutorial will guide you through the process of training a Long Short Term Memory (LSTM) Network to classify the sentiment of sentences from [Large Movie Review Dataset](http://ai.stanford.edu/~amaas/data/sentiment/), sometimes known as the Internet Movie Database (IMDB). This dataset contains movie reviews along with their associated binary sentiment polarity labels, namely positive and negative. So randomly guessing yields 50% accuracy.
## Data Preparation
### IMDB Data Introduction
Before training models, we need to preprocess the data and build a dictionary. First, you can use following script to download IMDB dataset and [Moses](http://www.statmt.org/moses/) tool, which is a statistical machine translation system. We provide a data preprocessing script, which is capable of handling not only IMDB data, but also other user-defined data. In order to use the pre-written script, it needs to move labeled train and test samples to another path, which has been done in `get_imdb.sh`.
```
cd demo/sentiment/data
./get_imdb.sh
```
If the data is obtained successfuly, you will see the following files at ```./demo/sentiment/data```:
```
aclImdb get_imdb.sh imdb mosesdecoder-master
```
* aclImdb: raw dataset downloaded from website.
* imdb: only contains train and test data.
* mosesdecoder-master: Moses tool.
IMDB dataset contains 25,000 highly polar movie reviews for training, and 25,000 for testing. A negative review has a score ≤ 4 out of 10, and a positive review has a score ≥ 7 out of 10. After running `./get_imdb.sh`, we can find the dataset has the following structure in `aclImdb`.
```
imdbEr.txt imdb.vocab README test train
```
* train: train sets.
* test : test sets.
* imdb.vocab: dictionary.
* imdbEr.txt: expected rating for each token in imdb.vocab.
* README: data documentation.
The file in train set directory is as follows. The test set also contains them except `unsup` and `urls_unsup.txt`.
```
labeledBow.feat neg pos unsup unsupBow.feat urls_neg.txt urls_pos.txt urls_unsup.txt
```
* pos: positive samples, contains 12,500 txt files, each file is one movie review.
* neg: negative samples, contains 12,500 txt files, each file is one movie review.
* unsup: unlabeled samples, contains 50,000 txt files.
* urls_xx.txt: urls of each reviews.
* xxBow.feat: already-tokenized bag of words (BoW) features.
### IMDB Data Preparation
In this demo, we only use labled train and test set and not use imdb.vocab as dictionary. By default, dictionary is builded on train set. Train set is shuffled and test set is not. `tokenizer.perl` in Moses tool is used to tokenize the words and punctuation. Simply execute the following command to preprcess data.
```
cd demo/sentiment/
./preprocess.sh
```
preprocess.sh:
```
data_dir="./data/imdb"
python preprocess.py -i data_dir
```
* data_dir: input data directory.
* preprocess.py: preprocess script.
If running successfully, you will see `demo/sentiment/data/pre-imdb` directory as follows:
```
dict.txt labels.list test.list test_part_000 train.list train_part_000
```
* test\_part\_000 and train\_part\_000: all labeled test and train sets. Train sets have be shuffled.
* train.list and test.list: train and test file lists.
* dict.txt: dictionary generated on train sets by default.
* labels.txt: neg 0, pos 1, means label 0 is negative review, label 1 is positive review.
### User-defined Data Preparation
If you perform other sentiment classifcation task, you can prepare data as follows. We have provided the scripts to build dictionary and preprocess data. So just organize data as follows.
```
dataset
|----train
| |----class1
| | |----text_files
| |----class2
| | |----text_files
| | ...
|----test
| |----class1
| | |----text_files
| |----class2
| | |----text_files
| | ...
```
* dataset: 1st directory.
* train, test: 2nd directory.
* class1,class2,...: 3rd directory.
* text_files: samples with text file format.
All samples with text files format under the same folder are same category. Each text file contains one or more samples and each line is one sample. In order to shuffle fully, the preprocessing is a little different for data with multiple lines in one text file, which needs to set `-m True` in `preprocess.sh`. And tokenizer.perl is used by default. If you don't need it, only set `-t False` in `preprocess.sh'.
## Training
In this task, we use Recurrent Neural Network (RNN) of LSTM architecure to train sentiment analysis model. LSTM model was introduced primarily in order to overcome the problem of vanishing gradients. LSTM network resembles a standard recurrent neural network with a hidden layer, but each ordinary node in the hidden layer is replaced by a memory cell. Each memory cell contains four main elements: an input gate, a neuron with a self-recurrent connection, a forget gate and an output gate. More details can be found in the literature [4]. The biggest advantage of the LSTM architecture is that it learns to memorize information over long time intervals without the loss of short time memory. At each time step with a new coming word, historical information stored in the memory block is updated to iteratively learn the sequence representation.
<center>![LSTM](./lstm.png)</center>
<center>Figure 1. LSTM [3]</center>
Sentiment analysis is among the most typical problems in natural language understanding. It aims at predicting the attitude expressed in a sequence. Usually, only some key words, like adjectives and adverbs words, play a major role in predicting the sentiment of sequences or paragraphs. However, some review or comment contexts are very long, such as IMDB dataset. We use LSTM to perform this task for its improved design with the gate mechanism. First, it is able to summarize the representation from word level to context level with variable context length which is adapted by the gate values. Second, it can utilize the expanded context at the sentence level, while most methods are good at utilizing n-gram level knowledge. Third, it learns the paragraph representation directly rather than combining the context level information. This results in this end-to-end framework.
In this demo we provide two network, namely bidirectional-LSTM and three layers of stacked-LSTM.
#### Bidirectional-LSTM
One is a bidirectional LSTM network, connected by fully connected layer and softmax, as shown in Figure 2.
<center>![BiLSTM](./bi_lstm.jpg)</center>
<center>Figure 2. Bidirectional-LSTM </center>
#### Stacked-LSTM
Another is three-layer LSTM structure in Figure 3. The bottom of the figure is word embedding. Next, three LSTM-Hidden layers are connected and the second LSTM is reversed. Then extract the maximum hidden vectors of all time step of hidden and LSTM layer as the representation for the entire sequence. Finally, a fully connected feed forward layer with softmax activation is used to perform the classification task. This network is refered to paper [5].
<center>![StackedLSTM](./stacked_lstm.jpg)</center>
<center>Figure 3. Stacked-LSTM for sentiment analysis </center>
**Config**
Switch into `demo/sentiment` directory, `trainer_config.py` file is an example of the config, containing algorithm and newtork configure. The first line imports predefined networks from `sentiment_net.py`.
trainer_config.py:
```python
from sentiment_net import *
data_dir = "./data/pre-imdb"
# whether this config is used for test
is_test = get_config_arg('is_test', bool, False)
# whether this config is used for prediction
is_predict = get_config_arg('is_predict', bool, False)
dict_dim, class_dim = sentiment_data(data_dir, is_test, is_predict)
################## Algorithm Config #####################
settings(
batch_size=128,
learning_rate=2e-3,
learning_method=AdamOptimizer(),
average_window=0.5,
regularization=L2Regularization(8e-4),
gradient_clipping_threshold=25
)
#################### Network Config ######################
stacked_lstm_net(dict_dim, class_dim=class_dim,
stacked_num=3, is_predict=is_predict)
#bidirectional_lstm_net(dict_dim, class_dim=class_dim, is_predict=is_predict)
```
* **Data Definition**:
* get\_config\_arg(): get arguments setted by `--config_args=xx` in commandline argument.
* Define data provider, here using Python interface to load data. For details, you can refer to the document of PyDataProvider2.
* **Algorithm Configuration**:
* set batch size of 128.
* set global learning rate.
* use adam optimization.
* set average sgd window.
* set L2 regularization.
* set gradient clipping threshold.
* **Network Configuration**:
* dict_dim: dictionary dimension.
* class_dim: category number, IMDB has two label, namely positive and negative label.
* `stacked_lstm_net`: predefined network as shown in Figure 3, use this network by default.
* `bidirectional_lstm_net`: predefined network as shown in Figure 2.
**Training**
Install PaddlePaddle first if necessary. Then you can use script `train.sh` as follows to launch local training.
```
cd demo/sentiment/
./train.sh
```
train.sh:
```
config=trainer_config.py
output=./model_output
paddle train --config=$config \
--save_dir=$output \
--job=train \
--use_gpu=false \
--trainer_count=4 \
--num_passes=10 \
--log_period=20 \
--dot_period=20 \
--show_parameter_stats_period=100 \
--test_all_data_in_one_period=1 \
2>&1 | tee 'train.log'
```
* \--config=$config: set network config.
* \--save\_dir=$output: set output path to save models.
* \--job=train: set job mode to train.
* \--use\_gpu=false: use CPU to train, set true, if you install GPU version of PaddlePaddle and want to use GPU to train.
* \--trainer\_count=4: set thread number (or GPU count).
* \--num\_passes=15: set pass number, one pass in PaddlePaddle means training all samples in dataset one time.
* \--log\_period=20: print log every 20 batches.
* \--show\_parameter\_stats\_period=100: show parameter statistic every 100 batches.
* \--test\_all_data\_in\_one\_period=1: test all data every testing.
If the run succeeds, the output log is saved in path of `demo/sentiment/train.log` and model is saved in path of `demo/sentiment/model_output/`. The output log is explained as follows.
```
Batch=20 samples=2560 AvgCost=0.681644 CurrentCost=0.681644 Eval: classification_error_evaluator=0.36875 CurrentEval: classification_error_evaluator=0.36875
...
Pass=0 Batch=196 samples=25000 AvgCost=0.418964 Eval: classification_error_evaluator=0.1922
Test samples=24999 cost=0.39297 Eval: classification_error_evaluator=0.149406
```
- Batch=xx: means passing xx batches.
- samples=xx: means passing xx samples.
- AvgCost=xx: averaged cost from 0-th batch to current batch.
- CurrentCost=xx: current cost of latest log_period batches.
- Eval: classification\_error\_evaluator=xx: means classfication error from 0-th batch ro current batch.
- CurrentEval: classification\_error\_evaluator: current classfication error of the lates log_period batches.
- Pass=0: Going through all training set one time is called one pass. 0 means going through training set first time.
By default, we use the `stacked_lstm_net` network, which converges at a faster rate than `bidirectional_lstm_net` when passing same sample number. If you want to use bidirectional LSTM, just remove comment in the last line and comment `stacked_lstm_net`.
## Testing
Testing means evaluating the labeled validation set using trained model.
```
cd demo/sentiment
./test.sh
```
test.sh:
```bash
function get_best_pass() {
cat $1 | grep -Pzo 'Test .*\n.*pass-.*' | \
sed -r 'N;s/Test.* error=([0-9]+\.[0-9]+).*\n.*pass-([0-9]+)/\1 \2/g' | \
sort | head -n 1
}
log=train.log
LOG=`get_best_pass $log`
LOG=(${LOG})
evaluate_pass="model_output/pass-${LOG[1]}"
echo 'evaluating from pass '$evaluate_pass
model_list=./model.list
touch $model_list | echo $evaluate_pass > $model_list
net_conf=trainer_config.py
paddle train --config=$net_conf \
--model_list=$model_list \
--job=test \
--use_gpu=false \
--trainer_count=4 \
--config_args=is_test=1 \
2>&1 | tee 'test.log'
```
The function `get_best_pass` gets the best model by classification error rate for testing. In this example, We use test dataset of IMDB as validation by default. Unlike training, it needs to specify `--job=test` and model path, namely `--model_list=$model_list` here. If running successfully, the log is saved in path of `demo/sentiment/test.log`. For example, in our test, the best model is `model_output/pass-00002`, the classification error is 0.115645 as follows.
```
Pass=0 samples=24999 AvgCost=0.280471 Eval: classification_error_evaluator=0.115645
```
## Prediction
`predict.py` provides a predicting interface. You should install python api of PaddlePaddle before using it. One example to predict unlabeled review of IMDB is as follows. Simply running:
```
cd demo/sentiment
./predict.sh
```
predict.sh:
```
#Note the default model is pass-00002, you shold make sure the model path
#exists or change the mode path.
model=model_output/pass-00002/
config=trainer_config.py
label=data/pre-imdb/labels.list
cat ./data/aclImdb/test/pos/10007_10.txt | python predict.py \
--tconf=$config\
--model=$model \
--label=$label \
--dict=./data/pre-imdb/dict.txt \
--batch_size=1
```
* `cat ./data/aclImdb/test/pos/10007_10.txt` : the input sample.
* `predict.py` : predicting interface.
* `--tconf=$config` : set network configure.
* ` --model=$model` : set model path.
* `--label=$label` : set dictionary about corresponding relation between integer label and string label.
* `--dict=data/pre-imdb/dict.txt` : set dictionary.
* `--batch_size=1` : set batch size.
Note you should make sure the default model path `model_output/pass-00002`
exists or change the model path.
Predicting result of this example:
```
Loading parameters from model_output/pass-00002/
./data/aclImdb/test/pos/10014_7.txt: predicting label is pos
```
We sincerely appreciate your interest and welcome your contributions.
## Reference
[1] Brendan O'Connor, Ramnath Balasubramanyan, Bryan R. Routledge, and Noah A. Smith. 2010. [From Tweets to Polls: Linking Text Sentiment to Public Opinion Time Series](http://homes.cs.washington.edu/~nasmith/papers/oconnor+balasubramanyan+routledge+smith.icwsm10.pdf). In ICWSM-2010. <br>
[2] Johan Bollen, Huina Mao, Xiaojun Zeng. 2011. [Twitter mood predicts the stock market](http://arxiv.org/abs/1010.3003), Journal of Computational Science.<br>
[3] Alex Graves, Marcus Liwicki, Santiago Fernan- dez, Roman Bertolami, Horst Bunke, and Ju ̈rgen Schmidhuber. 2009. [A novel connectionist system for unconstrained handwriting recognition. IEEE Transactions on Pattern Analysis and Machine In- telligence](http://www.cs.toronto.edu/~graves/tpami_2009.pdf), 31(5):855–868.<br>
[4] Zachary C. Lipton, [A Critical Review of Recurrent Neural Networks for Sequence Learning](http://arxiv.org/abs/1506.00019v1), arXiv:1506.00019. <br>
[5] Jie Zhou and Wei Xu; [End-to-end Learning of Semantic Role Labeling Using Recurrent Neural Networks](http://www.aclweb.org/anthology/P/P15/P15-1109.pdf); ACL-IJCNLP 2015. <br>
develop/doc/_sources/tutorials/text_generation/index_en.md.txt 已删除 100644 → 0
浏览文件 @ c15f0925
# Text generation Tutorial #
Sequence to sequence has been proven to be a powerful model for language generation. It can be used for machine translation, query rewriting, image captioning, etc.
This tutorial guides you through training a sequence to sequence model for neural machine translation (NMT) network that translates French to English.
We follow the paper [Neural Machine Translation by Jointly Learning to Align and Translate](http://arxiv.org/abs/1409.0473) , which details the model architecture and training procedure for good performance on WMT-14 dataset. This tutorial reproduces this result in PaddlePaddle.
We thank @caoying for the pull request that defines the model architecture and solver configurations.
## Data Preparation ##
### Download and Extract ###
Download the WMT-14 dataset from [http://www-lium.univ-lemans.fr/~schwenk/cslm\_joint\_paper/](http://www-lium.univ-lemans.fr/~schwenk/cslm_joint_paper/), extract it, and divide Develop and Test data into separate folder.
- **Train data**: [bitexts (after selection)](http://www-lium.univ-lemans.fr/~schwenk/cslm_joint_paper/data/bitexts.tgz)
- **Develop and Test data**: [dev+test data](http://www-lium.univ-lemans.fr/~schwenk/cslm_joint_paper/data/dev+test.tgz)
To do this, simply run the following commands in linux, otherwise, you need to download, extract, divide, and rename the file suffix respectively.
```bash
cd demo/seqToseq/data
./wmt14_data.sh
```
We should find that the dataset `wmt14` has three folders as shown in the following table.
<table border="2" cellspacing="0" cellpadding="6" rules="all" frame="border">
<colgroup>
<col class="left" />
<col class="left" />
<col class="left" />
<col class="left" />
</colgroup>
<thead>
<tr>
<th scope="col" class="left">folder name</th>
<th scope="col" class="left">French-English parallel corpora file</th>
<th scope="col" class="left">number of total file</th>
<th scope="col" class="left">size</th>
</tr>
</thead>
<tbody>
<tr>
<td class="left">train_data</td>
<td class="left">ccb2_pc30.src, ccb2_pc30.trg, etc</td>
<td class="left">twelve</td>
<td class="left">3.55G</td>
</tr>
<tr>
<td class="left">test_data</td>
<td class="left">ntst1213.src, ntst1213.trg</td>
<td class="left">two</td>
<td class="left">1636k</td>
</tr>
<tr>
<td class="left">gen_data</td>
<td class="left">ntst14.src, ntst14.trg</td>
<td class="left">two</td>
<td class="left">864k</td>
</tr>
</tbody>
</table>
<br/>
- Each folder has French-English parallel corpora
- **XXX.src** are source French files; **XXX.trg** are target English files.
- The number of lines of **XXX.src** and **XXX.trg** should be the same.
- Each line is a French/English sentence.
- There is a one-to-one correspondence between the sentence at the i-th line of **XXX.src** and **XXX.trg**.
### User Defined Dataset ###
If you need to do other sequence-to-sequence tasks, such as Paraphrasing, you only need to organize the data as follows, and place them in `demo/seqToseq/data`:
dataset
train
file1.src file1.trg
file2.src file2.trg
......
test
file1.src file1.trg
file2.src file2.trg
......
gen
file1.src file1.trg
file2.src file2.trg
......
- 1st directory: dataset folder name
- 2nd directory: folder of train, test, and gen. The names of these three folders are fixed.
- 3rd file: Source-Target parallel corpora files.
- **XXX.src** are source files, **XXX.trg** are target files.
- Each line of the file must be a sequence.
- There should be a one-to-one correspondence between the i-th sequence of **XXX.src** and **XXX.trg**.
## Data Preprocess ##
### Preprocessing Workflow ###
- Concat each Source-Target parallel corpora to be one file:
- concat each **XXX.src** and **XXX.trg** to be **XXX**.
- the i-th line of **XXX** = the i-th line of **XXX.src** + '\t' + the i-th line of **XXX.trg**
- Build source and target dictionary of train data, each dictionary has DICTSIZE words:
- the most frequent (DICTSIZE-3) words
- 3 special token:
- `<s>`: the start of a sequence
- `<e>`: the end of a sequence
- `<unk>`: a word not included in dictionary
### Preprocessing Command and Result
The general command for preprocessing the dataset is:
```python
cd demo/seqToseq/
python preprocess.py -i INPUT [-d DICTSIZE] [-m]
```
- `-i INPUT`: the path of input original dataset
- `-d DICTSIZE`: the specified word count of dictionary, if not set, dictionary will contain all the words in input dataset
- `-m --mergeDict`: merge source and target dictionary, thus, two dictionaries have the same context
And you will see messages like this:
concat parallel corpora for dataset
build source dictionary for train data
build target dictionary for train data
dictionary size is XXX
Here, you can simply run the command:
```python
python preprocess.py -i data/wmt14 -d 30000
```
It will take several minutes, and store the preprocessed dataset in `demo/seqToseq/data/pre-wmt14`, the directory has following structure.
train test gen train.list test.list gen.list src.dict trg.dict
- **train, test, gen**: folder contains French-English parallel corpora of train data, test data and gen data respectively. Each line of file in folder contains two parts, the former is a French sequence, and the latter is a corresponding English sequence.
- **train.list, test.list, gen.list**: text contains a file list in train folder, test folder and gen folder respectively
- **src.dict, trg.dict**: source (French) / target (English) dictionary, each dictionary has 30000 words: the most frequent 29997 words and 3 special token
## Model Training ##
### Introduction ###
Neural machine translation (NMT) aims at building a single neural network that can be jointly tuned to maximize translation performance. Recently proposed NMT models often belong to a family of encoder–decoder models. Encoder-Decoder models encode a source sentence into a fixed-length vector from which a decoder generates a target sentence.
In this task, we use an extension to the encoder–decoder model which learns to align and translate jointly. Each time the model generates a word in a translation, it searches for a set of positions in the source sentence for the most relevant information. The decoder predicts a target word based on the context vectors associated with these source positions and all the previous generated target words. For more detailed explanation, readers can refer to paper [Neural Machine Translation by Jointly Learning to Align and Translate](http://arxiv.org/abs/1409.0473).
The most distinguishing feature of this model is that it doesn't encode an input sentence into a single fixed-length vector. Instead, it encodes the input sentence into a sequence of vectors, where one vector corresponds to an input element. A subset of these vectors is chosen adaptively while decoding the translated sentence. This frees a NMT model from having to squash all the information of a source sentence, regardless of its length, into a fixed-length vector. The improvement of this model is more apparent for longer sentences, but the improvement can be observed for sentences of any length.
<center>![](./encoder-decoder-attention-model.png)</center>
<center>Figure 1. Encoder-Decoder-Attention-Model</center>
### Training Model in PaddlePaddle ###
We need to create a model config file before training. Here is an example `demo/seqToseq/translation/train.conf`. The first three lines import python function for defining network, and define the job_mode and attention_mode.
```python
from seqToseq_net import *
is_generating = False
### Data Definiation
train_conf = seq_to_seq_data(data_dir = "./data/pre-wmt14",
is_generating = is_generating)
### Algorithm Configuration
settings(
learning_method = AdamOptimizer(),
batch_size = 50,
learning_rate = 5e-4)
### Network Architecture
gru_encoder_decoder(train_conf, is_generating)
```
1. **Data Definiation**: We define a SeqToSeq train and test data in our example. It returns train_conf as the configuration, following is its input arguments:
- data_dir: directory of train data and test data
- is\_generating: whether this config is used for generating, here is false
2. **Algorithm Configuration**: We use the SGD training algorithm (default), ADAM learning method in our example, specify batch_size as 50, and learning rate as 5e-4.
3. **Network Architecture**: We use an attention version of GRU Encoder-Decoder network in our example. It consists a bidirectional GRU as an encoder and a decoder that emulates searching through a source sentence during decoding a translation.
### Training Command and Result###
After writing the model config, we can train the model by running the command:
```bash
cd demo/seqToseq/translation
./train.sh
```
The `train.sh` is shown as follows:
```bash
paddle train \
--config='translation/train.conf' \
--save_dir='translation/model' \
--use_gpu=false \
--num_passes=16 \
--show_parameter_stats_period=100 \
--trainer_count=4 \
--log_period=10 \
--dot_period=5 \
2>&1 | tee 'translation/train.log'
```
- config: set config of neural network
- save_dir: set output path to save models
- use_gpu: whether to use GPU to train, here use CPU
- num_passes: set number of passes. One pass in paddle means training all samples in dataset one time
- show_parameter_stats_period: here show parameter statistic every 100 batches
- trainer_count: set number of CPU threads or GPU devices
- log_period: here print log every 10 batches
- dot_period: here print '.' every 5 batches
The training loss function is printed every 10 batch by default, and you will see messages like this:
I0719 19:16:45.952062 15563 TrainerInternal.cpp:160] Batch=10 samples=500 AvgCost=198.475 CurrentCost=198.475 Eval: classification_error_evaluator=0.737155 CurrentEval: classification_error_evaluator=0.737155
I0719 19:17:56.707319 15563 TrainerInternal.cpp:160] Batch=20 samples=1000 AvgCost=157.479 CurrentCost=116.483 Eval: classification_error_evaluator=0.698392 CurrentEval: classification_error_evaluator=0.659065
.....
- AvgCost: Average Cost from 0th batch to current batch
- CurrentCost: Cost in current batch
- classification\_error\_evaluator(Eval): False prediction rate for each word from 0th evaluation to current evaluation
- classification\_error\_evaluator(CurrentEval): False prediction rate for each word in current evaluation
And when the classification\_error\_evaluator is less than 0.35, the model is trained sucessfully.
## Text Generation ##
### Introduction ###
Generally speaking, the NMT model is conditioned on the encodings of the source sentence, and then to predict the next target word by given the current target word. In the training process, the current word is always knowns as the ground truth, by contrast. In the generating process, the current word is the output of the decoder in last time step, which is accessed to from a memory in PaddlePaddle.
Besides, we use Beam Search to generate sequences. Beam search uses breadth-first search to build its search tree. At each level of the tree, it generates all successors of the states at the current level, sorting them in increasing order of heuristic cost. However, it only stores a predetermined number of best states at each level (called the beam size).
### Pretrained model ###
We trained the model on a cluster with 50 nodes, each node has two 6-core CPUs. We trained 16 passes in 5 days, where each pass takes 7 hours. The model_dir has 16 sub-folder, each of which contains the whole model parameters with 202MB size. And we find pass-00012 model has the highest BLEU 27.77 (see paper [BLEU: a Method for Automatic Evaluation of Machine Translation](http://www.aclweb.org/anthology/P02-1040.pdf)). To download and extract this model, simply run the following commands in linux.
```bash
cd demo/seqToseq/data
./wmt14_model.sh
```
### Generating Model in PaddlePaddle ###
We need to create a model config file before translating French sequence. Here is an example `demo/seqToseq/translation/gen.conf`, the first three lines import python function for defining network, and define the job\_mode and attention\_mode.
```python
from seqToseq_net import *
is_generating = True
################## Data Definiation #####################
gen_conf = seq_to_seq_data(data_dir = "./data/pre-wmt14",
is_generating = is_generating,
gen_result = "./translation/gen_result")
############## Algorithm Configuration ##################
settings(
learning_method = AdamOptimizer(),
batch_size = 1,
learning_rate = 0)
################# Network configure #####################
gru_encoder_decoder(gen_conf, is_generating)
```
1. **Data Definiation**: We defines an SeqToSeq gen data in our example. It returns gen_conf as the configuration, following is its input arguments:
- data\_dir: directory of gen data
  - is\_generating: whether this config is used for generating, here is true
  - gen\_result: file to store the generation result
2. **Algorithm Configuration**: We use SGD traing algorithm in generation, and specify batch_size as 1 (each time generate one sequence), and learning rate as 0.
3. **Network Architecture**: Essentially the same as the training model.
### Generating Command and Result ###
After writing the model config, we can do text translation from French to English by running the command:
```bash
cd demo/seqToseq/translation
./gen.sh
```
The `gen.sh` is shown as follows, unlike training, there are some different arguments to specify:
```bash
paddle train \
--job=test \
--config='translation/gen.conf' \
--save_dir='data/wmt14_model' \
--use_gpu=true \
--num_passes=13 \
--test_pass=12 \
--trainer_count=1 \
2>&1 | tee 'translation/gen.log'
```
- job: set job mode to test
- save_dir: the path of saved models
- num_passes and test_pass: loading model parameters from test_pass to (num_passes - 1), here only loads `data/wmt14_model/pass-00012`
You will see messages like this:
I0706 14:48:31.178915 31441 GradientMachine.cpp:143] Loading parameters from data/wmt14_model/pass-00012
I0706 14:48:40.012039 31441 Tester.cpp:125] Batch=100 samples=100 AvgCost=0
I0706 14:48:48.898632 31441 Tester.cpp:125] Batch=200 samples=200 AvgCost=0
...
And the generating result in `demo/seqToseq/translation/gen_result` likes:
0
0 -11.1314 The <unk> <unk> about the width of the seats while large controls are at stake <e>
1 -11.1519 The <unk> <unk> on the width of the seats while large controls are at stake <e>
2 -11.5988 The <unk> <unk> about the width of the seats while large controls are at stake . <e>
1
0 -24.4149 The dispute is between the major aircraft manufacturers about the width of the tourist seats on the <unk> flights , paving the way for a <unk> confrontation during the month of the Dubai <unk> . <e>
1 -26.9524 The dispute is between the major aircraft manufacturers about the width of the tourist seats on the <unk> flights , paving the way for a <unk> confrontation during the month of Dubai &apos; s <unk> . <e>
2 -27.9574 The dispute is between the major aircraft manufacturers about the width of the tourist seats on the <unk> flights , paving the way for a <unk> confrontation during the month of Dubai &apos; s Dubai <unk> . <e>
...
- This is the beam search result, where beam size is 3
- '0' in 1st-line and '1' in 6th-line mean the sequence-id in gen data
- Other six lines list the beam search results
- The 2nd-column is the score of beam search (from large to small)
- The 3rd-colunm is the generating English sequence
- There is 2 special tokens:
- `<e>`: the end of a sequence
- `<unk>`: a word not included in dictionary
### Bleu Evalutaion ###
Human evaluations of machine translation are extensive but expensive. Paper [BLEU: a Method for Automatic Evaluation of Machine Translation](http://www.aclweb.org/anthology/P02-1040.pdf) presents a method as an automated understudy to skilled human judges which substitutes for them when there is need for quick or frequent evaluations. [Moses](http://www.statmt.org/moses/) is a statistical machine translation system, and we use [multi-bleu.perl](https://github.com/moses-smt/mosesdecoder/blob/master/scripts/generic/multi-bleu.perl) of it to do Bleu Evalution. To download this script, simply run the following command:
```bash
cd demo/seqToseq/translation
./moses_bleu.sh
```
Since the standard translation is alrealy downloaded as `data/wmt14/gen/ntst14.trg`, we can do Bleu Evalution by running the command:
```bash
cd demo/seqToseq/translation
./eval_bleu.sh FILE BEAMSIZE
```
- FILE: the generation result file
- BEAMSIZE: expand width in beam search
develop/doc/_sources/v1_api_tutorials/README.md.txt 0 → 100644
浏览文件 @ 0a404363
The tutorials in v1_api_tutorials are using v1_api currently, and will be upgraded to v2_api later.
Thus, v1_api_tutorials is a temporary directory. We decide not to maintain it and will delete it in future.
Please go to [PaddlePaddle/book](https://github.com/PaddlePaddle/book) and
[PaddlePaddle/models](https://github.com/PaddlePaddle/models) to learn PaddlePaddle.
develop/doc/searchindex.js
浏览文件 @ 0a404363
因为 它太大了无法显示 source diff 。你可以改为 查看blob。
develop/doc/tutorials/rec/ml_dataset_en.html 已删除 100644 → 0
浏览文件 @ c15f0925
<!DOCTYPE html>
<!--[if IE 8]><html class="no-js lt-ie9" lang="en" > <![endif]-->
<!--[if gt IE 8]><!--> <html class="no-js" lang="en" > <!--<![endif]-->
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>MovieLens Dataset &mdash; PaddlePaddle documentation</title>
<link rel="stylesheet" href="../../_static/css/theme.css" type="text/css" />
<link rel="index" title="Index"
href="../../genindex.html"/>
<link rel="search" title="Search" href="../../search.html"/>
<link rel="top" title="PaddlePaddle documentation" href="../../index.html"/>
<link rel="stylesheet" href="https://cdn.jsdelivr.net/perfect-scrollbar/0.6.14/css/perfect-scrollbar.min.css" type="text/css" />
<link rel="stylesheet" href="../../_static/css/override.css" type="text/css" />
<script>
var _hmt = _hmt || [];
(function() {
var hm = document.createElement("script");
hm.src = "//hm.baidu.com/hm.js?b9a314ab40d04d805655aab1deee08ba";
var s = document.getElementsByTagName("script")[0];
s.parentNode.insertBefore(hm, s);
})();
</script>
<script src="../../_static/js/modernizr.min.js"></script>
</head>
<body class="wy-body-for-nav" role="document">
<header class="site-header">
<div class="site-logo">
<a href="/"><img src="../../_static/images/PP_w.png"></a>
</div>
<div class="site-nav-links">
<div class="site-menu">
<a class="fork-on-github" href="https://github.com/PaddlePaddle/Paddle" target="_blank"><i class="fa fa-github"></i>Fork me on Github</a>
<div class="language-switcher dropdown">
<a type="button" data-toggle="dropdown">
<span>English</span>
<i class="fa fa-angle-up"></i>
<i class="fa fa-angle-down"></i>
</a>
<ul class="dropdown-menu">
<li><a href="/doc_cn">中文</a></li>
<li><a href="/doc">English</a></li>
</ul>
</div>
<ul class="site-page-links">
<li><a href="/">Home</a></li>
</ul>
</div>
<div class="doc-module">
<ul>
<li class="toctree-l1"><a class="reference internal" href="../../getstarted/index_en.html">GET STARTED</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../howto/index_en.html">HOW TO</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../api/index_en.html">API</a></li>
</ul>
<div role="search">
<form id="rtd-search-form" class="wy-form" action="../../search.html" method="get">
<input type="text" name="q" placeholder="Search docs" />
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
</form>
</div>
</div>
</div>
</header>
<div class="main-content-wrap">
<nav class="doc-menu-vertical" role="navigation">
<ul>
<li class="toctree-l1"><a class="reference internal" href="../../getstarted/index_en.html">GET STARTED</a><ul>
<li class="toctree-l2"><a class="reference internal" href="../../getstarted/build_and_install/index_en.html">Install and Build</a><ul>
<li class="toctree-l3"><a class="reference internal" href="../../getstarted/build_and_install/docker_install_en.html">PaddlePaddle in Docker Containers</a></li>
<li class="toctree-l3"><a class="reference internal" href="../../getstarted/build_and_install/build_from_source_en.html">Installing from Sources</a></li>
</ul>
</li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="../../howto/index_en.html">HOW TO</a><ul>
<li class="toctree-l2"><a class="reference internal" href="../../howto/usage/cmd_parameter/index_en.html">Set Command-line Parameters</a><ul>
<li class="toctree-l3"><a class="reference internal" href="../../howto/usage/cmd_parameter/use_case_en.html">Use Case</a></li>
<li class="toctree-l3"><a class="reference internal" href="../../howto/usage/cmd_parameter/arguments_en.html">Argument Outline</a></li>
<li class="toctree-l3"><a class="reference internal" href="../../howto/usage/cmd_parameter/detail_introduction_en.html">Detail Description</a></li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="../../howto/usage/cluster/cluster_train_en.html">Run Distributed Training</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../howto/usage/k8s/k8s_en.html">Paddle On Kubernetes</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../howto/usage/k8s/k8s_aws_en.html">Distributed PaddlePaddle Training on AWS with Kubernetes</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../howto/dev/build_en.html">Build PaddlePaddle from Source Code and Run Unit Test</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../howto/dev/new_layer_en.html">Write New Layers</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../howto/dev/contribute_to_paddle_en.html">Contribute Code</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../howto/deep_model/rnn/index_en.html">RNN Models</a><ul>
<li class="toctree-l3"><a class="reference internal" href="../../howto/deep_model/rnn/rnn_config_en.html">RNN Configuration</a></li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="../../howto/optimization/gpu_profiling_en.html">Tune GPU Performance</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="../../api/index_en.html">API</a><ul>
<li class="toctree-l2"><a class="reference internal" href="../../api/v2/model_configs.html">Model Configuration</a><ul>
<li class="toctree-l3"><a class="reference internal" href="../../api/v2/config/activation.html">Activation</a></li>
<li class="toctree-l3"><a class="reference internal" href="../../api/v2/config/layer.html">Layers</a></li>
<li class="toctree-l3"><a class="reference internal" href="../../api/v2/config/evaluators.html">Evaluators</a></li>
<li class="toctree-l3"><a class="reference internal" href="../../api/v2/config/optimizer.html">Optimizer</a></li>
<li class="toctree-l3"><a class="reference internal" href="../../api/v2/config/pooling.html">Pooling</a></li>
<li class="toctree-l3"><a class="reference internal" href="../../api/v2/config/networks.html">Networks</a></li>
<li class="toctree-l3"><a class="reference internal" href="../../api/v2/config/attr.html">Parameter Attribute</a></li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="../../api/v2/data.html">Data Reader Interface and DataSets</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../api/v2/run_logic.html">Training and Inference</a></li>
</ul>
</li>
</ul>
</nav>
<section class="doc-content-wrap">
<div role="navigation" aria-label="breadcrumbs navigation">
<ul class="wy-breadcrumbs">
<li>MovieLens Dataset</li>
</ul>
</div>
<div class="wy-nav-content" id="doc-content">
<div class="rst-content">
<div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article">
<div itemprop="articleBody">
<div class="section" id="movielens-dataset">
<span id="movielens-dataset"></span><span id="demo-ml-dataset"></span><h1>MovieLens Dataset<a class="headerlink" href="#movielens-dataset" title="Permalink to this headline"></a></h1>
<p>The <a class="reference external" href="http://grouplens.org/datasets/movielens/">MovieLens Dataset</a> was collected by GroupLens Research.
The data set contains some user information, movie information, and many movie ratings from [1-5].
The data sets have many version depending on the size of set.
We use <a class="reference external" href="http://files.grouplens.org/datasets/movielens/ml-1m.zip">MovieLens 1M Dataset</a> as a demo dataset, which contains
1 million ratings from 6000 users on 4000 movies. Released 2/2003.</p>
<div class="section" id="dataset-features">
<span id="dataset-features"></span><h2>Dataset Features<a class="headerlink" href="#dataset-features" title="Permalink to this headline"></a></h2>
<p>In <a class="reference external" href="http://files.grouplens.org/datasets/movielens/ml-1m.zip">ml-1m Dataset</a>, there are many features in these dataset.
The data files (which have &#8221;.dat&#8221; extension) in <a class="reference external" href="http://files.grouplens.org/datasets/movielens/ml-1m.zip">ml-1m Dataset</a>
is basically CSV file that delimiter is &#8221;::&#8221;. The description in README we quote here.</p>
<div class="section" id="ratings-file-description-ratings-dat">
<span id="ratings-file-description-ratings-dat"></span><h3>RATINGS FILE DESCRIPTION(ratings.dat)<a class="headerlink" href="#ratings-file-description-ratings-dat" title="Permalink to this headline"></a></h3>
<p>All ratings are contained in the file &#8220;ratings.dat&#8221; and are in the
following format:</p>
<p>UserID::MovieID::Rating::Timestamp</p>
<ul class="simple">
<li>UserIDs range between 1 and 6040</li>
<li>MovieIDs range between 1 and 3952</li>
<li>Ratings are made on a 5-star scale (whole-star ratings only)</li>
<li>Timestamp is represented in seconds since the epoch as returned by time(2)</li>
<li>Each user has at least 20 ratings</li>
</ul>
</div>
<div class="section" id="users-file-description-users-dat">
<span id="users-file-description-users-dat"></span><h3>USERS FILE DESCRIPTION(users.dat)<a class="headerlink" href="#users-file-description-users-dat" title="Permalink to this headline"></a></h3>
<p>User information is in the file &#8220;users.dat&#8221; and is in the following
format:</p>
<p>UserID::Gender::Age::Occupation::Zip-code</p>
<p>All demographic information is provided voluntarily by the users and is
not checked for accuracy. Only users who have provided some demographic
information are included in this data set.</p>
<ul class="simple">
<li>Gender is denoted by a &#8220;M&#8221; for male and &#8220;F&#8221; for female</li>
<li>Age is chosen from the following ranges:<ul>
<li>1: &#8220;Under 18&#8221;</li>
<li>18: &#8220;18-24&#8221;</li>
<li>25: &#8220;25-34&#8221;</li>
<li>35: &#8220;35-44&#8221;</li>
<li>45: &#8220;45-49&#8221;</li>
<li>50: &#8220;50-55&#8221;</li>
<li>56: &#8220;56+&#8221;</li>
</ul>
</li>
<li>Occupation is chosen from the following choices:<ul>
<li>0: &#8220;other&#8221; or not specified</li>
<li>1: &#8220;academic/educator&#8221;</li>
<li>2: &#8220;artist&#8221;</li>
<li>3: &#8220;clerical/admin&#8221;</li>
<li>4: &#8220;college/grad student&#8221;</li>
<li>5: &#8220;customer service&#8221;</li>
<li>6: &#8220;doctor/health care&#8221;</li>
<li>7: &#8220;executive/managerial&#8221;</li>
<li>8: &#8220;farmer&#8221;</li>
<li>9: &#8220;homemaker&#8221;</li>
<li>10: &#8220;K-12 student&#8221;</li>
<li>11: &#8220;lawyer&#8221;</li>
<li>12: &#8220;programmer&#8221;</li>
<li>13: &#8220;retired&#8221;</li>
<li>14: &#8220;sales/marketing&#8221;</li>
<li>15: &#8220;scientist&#8221;</li>
<li>16: &#8220;self-employed&#8221;</li>
<li>17: &#8220;technician/engineer&#8221;</li>
<li>18: &#8220;tradesman/craftsman&#8221;</li>
<li>19: &#8220;unemployed&#8221;</li>
<li>20: &#8220;writer&#8221;</li>
</ul>
</li>
</ul>
</div>
<div class="section" id="movies-file-description-movies-dat">
<span id="movies-file-description-movies-dat"></span><h3>MOVIES FILE DESCRIPTION(movies.dat)<a class="headerlink" href="#movies-file-description-movies-dat" title="Permalink to this headline"></a></h3>
<p>Movie information is in the file &#8220;movies.dat&#8221; and is in the following
format:</p>
<p>MovieID::Title::Genres</p>
<ul class="simple">
<li>Titles are identical to titles provided by the IMDB (including
year of release)</li>
<li>Genres are pipe-separated and are selected from the following genres:<ul>
<li>Action</li>
<li>Adventure</li>
<li>Animation</li>
<li>Children&#8217;s</li>
<li>Comedy</li>
<li>Crime</li>
<li>Documentary</li>
<li>Drama</li>
<li>Fantasy</li>
<li>Film-Noir</li>
<li>Horror</li>
<li>Musical</li>
<li>Mystery</li>
<li>Romance</li>
<li>Sci-Fi</li>
<li>Thriller</li>
<li>War</li>
<li>Western</li>
</ul>
</li>
<li>Some MovieIDs do not correspond to a movie due to accidental duplicate
entries and/or test entries</li>
<li>Movies are mostly entered by hand, so errors and inconsistencies may exist</li>
</ul>
</div>
</div>
</div>
</div>
</div>
<footer>
<hr/>
<div role="contentinfo">
<p>
&copy; Copyright 2016, PaddlePaddle developers.
</p>
</div>
Built with <a href="http://sphinx-doc.org/">Sphinx</a> using a <a href="https://github.com/snide/sphinx_rtd_theme">theme</a> provided by <a href="https://readthedocs.org">Read the Docs</a>.
</footer>
</div>
</div>
</section>
</div>
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT:'../../',
VERSION:'',
COLLAPSE_INDEX:false,
FILE_SUFFIX:'.html',
HAS_SOURCE: true,
SOURCELINK_SUFFIX: ".txt",
};
</script>
<script type="text/javascript" src="../../_static/jquery.js"></script>
<script type="text/javascript" src="../../_static/underscore.js"></script>
<script type="text/javascript" src="../../_static/doctools.js"></script>
<script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.0/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script>
<script type="text/javascript" src="../../_static/js/theme.js"></script>
<script src="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/js/bootstrap.min.js" integrity="sha384-Tc5IQib027qvyjSMfHjOMaLkfuWVxZxUPnCJA7l2mCWNIpG9mGCD8wGNIcPD7Txa" crossorigin="anonymous"></script>
<script src="https://cdn.jsdelivr.net/perfect-scrollbar/0.6.14/js/perfect-scrollbar.jquery.min.js"></script>
<script src="../../_static/js/paddle_doc_init.js"></script>
</body>
</html>
\ No newline at end of file
develop/doc/tutorials/index_en.html → develop/doc/v1_api_tutorials/README.html
浏览文件 @ 0a404363
......@@ -8,7 +8,7 @@
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>TUTORIALS &mdash; PaddlePaddle documentation</title>
<title>&lt;no title&gt; &mdash; PaddlePaddle documentation</title>
......@@ -168,7 +168,7 @@
<div role="navigation" aria-label="breadcrumbs navigation">
<ul class="wy-breadcrumbs">
<li>TUTORIALS</li>
<li>&lt;no title&gt;</li>
</ul>
</div>
......@@ -177,30 +177,10 @@
<div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article">
<div itemprop="articleBody">
<div class="section" id="tutorials">
<span id="tutorials"></span><h1>TUTORIALS<a class="headerlink" href="#tutorials" title="Permalink to this headline"></a></h1>
<p>There are several examples and demos here.</p>
<div class="toctree-wrapper compound">
<ul>
<li class="toctree-l1"><a class="reference internal" href="quick_start/index_en.html">Quick Start</a></li>
<li class="toctree-l1"><a class="reference internal" href="rec/ml_regression_en.html">MovieLens Regression</a></li>
<li class="toctree-l1"><a class="reference internal" href="image_classification/index_en.html">Image Classification</a></li>
<li class="toctree-l1"><a class="reference internal" href="sentiment_analysis/index_en.html">Sentiment Analysis</a></li>
<li class="toctree-l1"><a class="reference internal" href="semantic_role_labeling/index_en.html">Semantic Role Labeling</a></li>
<li class="toctree-l1"><a class="reference internal" href="text_generation/index_en.html">Text Generation</a></li>
<li class="toctree-l1"><a class="reference internal" href="gan/index_en.html">Image Auto-Generation</a></li>
</ul>
</div>
<div class="section" id="model-zoo">
<span id="model-zoo"></span><h2>Model Zoo<a class="headerlink" href="#model-zoo" title="Permalink to this headline"></a></h2>
<div class="toctree-wrapper compound">
<ul>
<li class="toctree-l1"><a class="reference internal" href="imagenet_model/resnet_model_en.html">ImageNet: ResNet</a></li>
<li class="toctree-l1"><a class="reference internal" href="embedding_model/index_en.html">Embedding: Chinese Word</a></li>
</ul>
</div>
</div>
</div>
<p>The tutorials in v1_api_tutorials are using v1_api currently, and will be upgraded to v2_api later.
Thus, v1_api_tutorials is a temporary directory. We decide not to maintain it and will delete it in future.</p>
<p>Please go to <a class="reference external" href="https://github.com/PaddlePaddle/book">PaddlePaddle/book</a> and
<a class="reference external" href="https://github.com/PaddlePaddle/models">PaddlePaddle/models</a> to learn PaddlePaddle.</p>
</div>
......
develop/doc/tutorials/embedding_model/index_en.html → develop/doc/v1_api_tutorials/embedding_model/index_en.html
浏览文件 @ 0a404363
......@@ -285,7 +285,7 @@ python extract_para.py --preModel PREMODEL --preDict PREDICT --usrModel USRMODEL
<li><code class="docutils literal"><span class="pre">--init_model_path</span></code>: path of the initialization model, here is <code class="docutils literal"><span class="pre">data/paraphrase_model</span></code></li>
<li><code class="docutils literal"><span class="pre">--load_missing_parameter_strategy</span></code>: operations when model file is missing, here use a normal distibution to initialize the other parameters except for the embedding layer</li>
</ul>
<p>For users who want to understand the dataset format, model architecture and training procedure in detail, please refer to <a class="reference internal" href="../text_generation/index_en.html"><span class="doc">Text generation Tutorial</span></a>.</p>
<p>For users who want to understand the dataset format, model architecture and training procedure in detail, please refer to <a class="reference external" href="v1_api_tutorials/text_generation/index_en.md">Text generation Tutorial</a>.</p>
</div>
</div>
<div class="section" id="optional-function">
......
develop/doc_cn/_sources/howto/deep_model/rnn/rnn_config_cn.rst.txt
浏览文件 @ 0a404363
......@@ -21,7 +21,7 @@ wmt14数据的提供文件在 `python/paddle/v2/dataset/wmt14.py <https://github
循环神经网络在每个时间步骤顺序地处理序列。下面列出了 LSTM 的架构的示例。
.. image:: ../../../tutorials/sentiment_analysis/bi_lstm.jpg
.. image:: src/bi_lstm.jpg
:align: center
一般来说，循环网络从 :math:`t=1` 到 :math:`t=T` 或者反向地从 :math:`t=T` 到 :math:`t=1` 执行以下操作。
......@@ -96,7 +96,7 @@ Sequence to Sequence Model with Attention
我们将使用 sequence to sequence model with attention
作为例子演示如何配置复杂的循环神经网络模型。该模型的说明如下图所示。
.. image:: ../../../tutorials/text_generation/encoder-decoder-attention-model.png
.. image:: src/encoder-decoder-attention-model.png
:align: center
在这个模型中，源序列 :math:`S = \{s_1, \dots, s_T\}`
......
develop/doc_cn/_sources/tutorials/image_classification/index_cn.md.txt 已删除 100644 → 0
浏览文件 @ c15f0925
图像分类教程
==========
在本教程中，我们将使用CIFAR-10数据集训练一个卷积神经网络，并使用这个神经网络来对图片进行分类。如下图所示，卷积神经网络可以辨识图片中的主体，并给出分类结果。
<center>![Image Classification](./image_classification.png)</center>
## 数据准备
首先下载CIFAR-10数据集。下面是CIFAR-10数据集的官方网址：
<https://www.cs.toronto.edu/~kriz/cifar.html>
我们准备了一个脚本，可以用于从官方网站上下载CIFAR-10数据集，转为jpeg文件并存入特定的目录。使用这个脚本前请确认已经安装了pillow及相关依赖模块。可以参照下面的命令进行安装：
1. 安装pillow
```bash
sudo apt-get install libjpeg-dev
pip install pillow
```
2. 下载数据集
```bash
cd demo/image_classification/data/
sh download_cifar.sh
```
CIFAR-10数据集包含60000张32x32的彩色图片。图片分为10类，每个类包含6000张。其中50000张图片作为训练集，10000张作为测试集。
下图展示了所有的图片类别，每个类别中随机抽取了10张图片。
<center>![Image Classification](./cifar.png)</center>
脚本运行完成后，我们应当会得到一个名为cifar-out的文件夹，其下子文件夹的结构如下
```
train
---airplane
---automobile
---bird
---cat
---deer
---dog
---frog
---horse
---ship
---truck
test
---airplane
---automobile
---bird
---cat
---deer
---dog
---frog
---horse
---ship
---truck
```
cifar-out下包含`train`和`test`两个文件夹，其中分别包含了CIFAR-10中的训练集和测试集。这两个文件夹下各自有10个子文件夹，每个子文件夹下存储相应分类的图片。将图片按照上述结构存储好之后，我们就可以着手对分类模型进行训练了。
## 预处理
数据下载之后，还需要进行预处理，将数据转换为Paddle的格式。我们可以通过如下命令进行预处理工作：
```
cd demo/image_classification/
sh preprocess.sh
```
其中`preprocess.sh` 调用 `./demo/image_classification/preprocess.py` 对图片进行预处理
```sh
export PYTHONPATH=$PYTHONPATH:../../
data_dir=./data/cifar-out
python preprocess.py -i $data_dir -s 32 -c 1
```
`./demo/image_classification/preprocess.py` 使用如下参数：
- `-i` 或 `--input` 给出输入数据所在路径；
- `-s` 或 `--size` 给出图片尺寸；
- `-c` 或 `--color` 标示图片是彩色图或灰度图
## 模型训练
在开始训练之前，我们需要先创建一个模型配置文件。下面我们给出了一个配置示例。**注意**，这里的列出的和`vgg_16_cifar.py`文件稍有差别，因为该文件可适用于预测。
```python
from paddle.trainer_config_helpers import *
data_dir='data/cifar-out/batches/'
meta_path=data_dir+'batches.meta'
args = {'meta':meta_path, 'mean_img_size': 32,
'img_size': 32, 'num_classes': 10,
'use_jpeg': 1, 'color': "color"}
define_py_data_sources2(train_list=data_dir+"train.list",
test_list=data_dir+'test.list',
module='image_provider',
obj='processData',
args=args)
settings(
batch_size = 128,
learning_rate = 0.1 / 128.0,
learning_method = MomentumOptimizer(0.9),
regularization = L2Regularization(0.0005 * 128))
img = data_layer(name='image', size=3*32*32)
lbl = data_layer(name="label", size=10)
# small_vgg is predined in trainer_config_helpers.network
predict = small_vgg(input_image=img, num_channels=3)
outputs(classification_cost(input=predict, label=lbl))
```
在第一行中我们载入用于定义网络的函数。
```python
from paddle.trainer_config_helpers import *
```
之后定义的`define_py_data_sources2`使用Python数据提供器，其中 `args`将在`image_provider.py`进行使用，该文件负责产生图片数据并传递给Paddle系统
- `meta`: 训练集平均值。
- `mean_img_size`: 平均特征图的高度及宽度。
- `img_size`：输入图片的高度及宽度。
- `num_classes`：类别个数。
- `use_jpeg`：处理过程中数据存储格式。
- `color`：标示是否为彩色图片。
`settings`用于设置训练算法。在下面的例子中，learning rate被设置为0.1除以batch size，而weight decay则为0.0005乘以batch size。
```python
settings(
batch_size = 128,
learning_rate = 0.1 / 128.0,
learning_method = MomentumOptimizer(0.9),
regularization = L2Regularization(0.0005 * 128)
)
```
`small_vgg`定义了网络结构。这里我们使用的是一个小的VGG网络。关于VGG卷积神经网络的描述可以参考：[http://www.robots.ox.ac.uk/~vgg/research/very_deep/](http://www.robots.ox.ac.uk/~vgg/research/very_deep/)。
```python
# small_vgg is predined in trainer_config_helpers.network
predict = small_vgg(input_image=img, num_channels=3)
```
配置创建完毕后，可以运行脚本train.sh来训练模型。
```bash
config=vgg_16_cifar.py
output=./cifar_vgg_model
log=train.log
paddle train \
--config=$config \
--dot_period=10 \
--log_period=100 \
--test_all_data_in_one_period=1 \
--use_gpu=1 \
--save_dir=$output \
2>&1 | tee $log
python -m paddle.utils.plotcurve -i $log > plot.png
```
- 这里我们使用的是GPU模式进行训练。如果你没有GPU环境，可以设置`use_gpu=0`。
- `./demo/image_classification/vgg_16_cifar.py`是网络和数据配置文件。各项参数的详细说明可以在命令行参数相关文档中找到。
- 脚本`plotcurve.py`依赖于python的`matplotlib`模块。因此如果这个脚本运行失败，也许是因为需要安装`matplotlib`。
在训练完成后，训练及测试误差曲线图会被`plotcurve.py`脚本保存在 `plot.png`中。下面是一个误差曲线图的示例：
<center>![Training and testing curves.](./plot.png)</center>
## 预测
在训练完成后，模型及参数会被保存在路径`./cifar_vgg_model/pass-%05d`下。例如第300个pass的模型会被保存在`./cifar_vgg_model/pass-00299`。
要对一个图片的进行分类预测，我们可以使用`predict.sh`，该脚本将输出预测分类的标签：
```
sh predict.sh
```
predict.sh:
```
model=cifar_vgg_model/pass-00299/
image=data/cifar-out/test/airplane/seaplane_s_000978.png
use_gpu=1
python prediction.py $model $image $use_gpu
```
## 练习
在CUB-200数据集上使用VGG模型训练一个鸟类图片分类模型。相关的鸟类数据集可以从如下地址下载，其中包含了200种鸟类的照片（主要来自北美洲）。
<http://www.vision.caltech.edu/visipedia/CUB-200.html>
## 细节探究
### 卷积神经网络
卷积神经网络是一种使用卷积层的前向神经网络，很适合构建用于理解图片内容的模型。一个典型的神经网络如下图所示：
![Convolutional Neural Network](./lenet.png)
一个卷积神经网络包含如下层：
- 卷积层：通过卷积操作从图片或特征图中提取特征
- 池化层：使用max-pooling对特征图下采样
- 全连接层：使输入层到隐藏层的神经元是全部连接的。
卷积神经网络在图片分类上有着惊人的性能，这是因为它发掘出了图片的两类重要信息：局部关联性质和空间不变性质。通过交替使用卷积和池化处理， 卷积神经网络能够很好的表示这两类信息。
关于如何定义网络中的层，以及如何在层之间进行连接，请参考Layer文档。
develop/doc_cn/_sources/tutorials/index_cn.md.txt 已删除 100644 → 0
浏览文件 @ c15f0925
# 完整教程
* [快速入门](quick_start/index_cn.rst)
* [个性化推荐](rec/ml_regression_cn.rst)
* [图像分类](image_classification/index_cn.md)
* [情感分析](sentiment_analysis/index_cn.md)
* [语义角色标注](semantic_role_labeling/index_cn.md)
* [机器翻译](text_generation/index_cn.md)
## 常用模型
* [ResNet模型](imagenet_model/resnet_model_cn.md)
* [词向量模型](embedding_model/index_cn.md)
develop/doc_cn/_sources/tutorials/rec/ml_dataset_cn.md.txt 已删除 100644 → 0
浏览文件 @ c15f0925
```eval_rst
.. _demo_ml_dataset:
```
# MovieLens数据集
[MovieLens 数据集](http://grouplens.org/datasets/movielens/)由GroupLens Research实验室搜集整理。
该数据集包含一些用户信息、电影信息以及电影评分\[1-5\]。根据数据量规模，该数据及有很多不同的版本。
我们用[MovieLens 百万数据集](http://files.grouplens.org/datasets/movielens/ml-1m.zip)作为示例数据
集，其中包含6,000位用户对4,000部电影的1,000,000条评价。该数据集于2003年2月发布。
## 数据集特征
在[ml-1m 数据集](http://files.grouplens.org/datasets/movielens/ml-1m.zip)中有许多的特征。在[ml-1m 数据集]
(http://files.grouplens.org/datasets/movielens/ml-1m.zip)中的这些数据文件(含有".dat"的后缀)实际上是CSV文件，
分隔符为"::"。以下我们翻译数据集网站中README文件的描述:
### 评分文件描述(ratings.dat)
所有的评分数据都包含在"ratings.dat"文件中，遵循如下的格式:
用户ID::电影ID::评分::时间戳
- 用户ID范围从1到6040
- 电影ID范围从1到3952
- 评分被调整为5星的规模(只允许整数的星级)
- 时间戳表示为从1970-01-01(UTC)来的秒数，与time(2)的返回值一致
- 每位用户至少有20条评分
### 用户文件描述(users.dat)
所有的用户信息都包含在"users.dat"文件中，遵循如下的格式:
用户ID::性别::年龄::职业::邮编
所有的人口统计学信息由用户自愿提供，没有进行正确性的检查。只有含有人
口统计学信息的用户才被包含在数据集中。
- 性别，用"M"表示男性，"F"表示女性
- 年龄从下列列表范围中选取:
* 1: "18岁以下"
* 18: "18-24岁"
* 25: "25-34岁"
* 35: "35-44岁"
* 45: "45-49岁"
* 50: "50-55岁"
* 56: "56+"
- 职业从下面所列中选择:
* 0: "其他"或不确定
* 1: "学术/教育工作者"
* 2: "艺术家"
* 3: "文书工作/管理员"
* 4: "大学生/研究生"
* 5: "客户服务"
* 6: "医生/医疗保健"
* 7: "行政工作/管理人员"
* 8: "农民"
* 9: "操持家务者"
* 10: "高中毕业生"
* 11: "律师"
* 12: "程序员"
* 13: "退休人员"
* 14: "销售/市场"
* 15: "科学家"
* 16: "自由职业者"
* 17: "技术员/工程师"
* 18: "推销员/手工艺者"
* 19: "无业人士"
* 20: "作家"
### 电影文件描述(movies.dat)
所有的电影信息都包含在"movies.dat"文件中，遵循如下的格式:
电影ID::电影名称::电影类型
- 电影名称（包括发行时间）与IMDB网站提供的一致
- 电影类型如符合多种用管道符号|分割，选自下列类型:
* 动作片
* 冒险片
* 动画片
* 儿童片
* 喜剧片
* 犯罪片
* 纪录片
* 戏剧
* 奇幻片
* 黑色电影
* 恐怖片
* 音乐剧
* 悬疑片
* 浪漫片
* 科幻片
* 惊险电影
* 战争片
* 西部片
- 由于意外的副本记录和测试记录，有些电影ID可能与实际电影不相符合
- 电影大部分是手工输入数据，因此可能会有一些错误和不一致发生
develop/doc_cn/_sources/v1_api_tutorials/README.md.txt 0 → 100644
浏览文件 @ 0a404363
The tutorials in v1_api_tutorials are using v1_api currently, and will be upgraded to v2_api later.
Thus, v1_api_tutorials is a temporary directory. We decide not to maintain it and will delete it in future.
Please go to [PaddlePaddle/book](https://github.com/PaddlePaddle/book) and
[PaddlePaddle/models](https://github.com/PaddlePaddle/models) to learn PaddlePaddle.
develop/doc_cn/searchindex.js
浏览文件 @ 0a404363
此差异已折叠。
develop/doc_cn/tutorials/index_cn.html → develop/doc_cn/v1_api_tutorials/README.html
浏览文件 @ 0a404363
......@@ -8,7 +8,7 @@
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>完整教程 &mdash; PaddlePaddle 文档</title>
<title>&lt;no title&gt; &mdash; PaddlePaddle 文档</title>
......@@ -182,7 +182,7 @@
<div role="navigation" aria-label="breadcrumbs navigation">
<ul class="wy-breadcrumbs">
<li>完整教程</li>
<li>&lt;no title&gt;</li>
</ul>
</div>
......@@ -191,28 +191,10 @@
<div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article">
<div itemprop="articleBody">
<div class="section" id="">
<span id="id1"></span><h1>完整教程<a class="headerlink" href="#" title="永久链接至标题"></a></h1>
<div class="toctree-wrapper compound">
<ul>
<li class="toctree-l1"><a class="reference internal" href="quick_start/index_cn.html">快速入门</a></li>
<li class="toctree-l1"><a class="reference internal" href="rec/ml_regression_cn.html">个性化推荐</a></li>
<li class="toctree-l1"><a class="reference internal" href="image_classification/index_cn.html">图像分类</a></li>
<li class="toctree-l1"><a class="reference internal" href="sentiment_analysis/index_cn.html">情感分析</a></li>
<li class="toctree-l1"><a class="reference internal" href="semantic_role_labeling/index_cn.html">语义角色标注</a></li>
<li class="toctree-l1"><a class="reference internal" href="text_generation/index_cn.html">机器翻译</a></li>
</ul>
</div>
<div class="section" id="">
<span id="id2"></span><h2>常用模型<a class="headerlink" href="#" title="永久链接至标题"></a></h2>
<div class="toctree-wrapper compound">
<ul>
<li class="toctree-l1"><a class="reference internal" href="imagenet_model/resnet_model_cn.html">ResNet模型</a></li>
<li class="toctree-l1"><a class="reference internal" href="embedding_model/index_cn.html">词向量模型</a></li>
</ul>
</div>
</div>
</div>
<p>The tutorials in v1_api_tutorials are using v1_api currently, and will be upgraded to v2_api later.
Thus, v1_api_tutorials is a temporary directory. We decide not to maintain it and will delete it in future.</p>
<p>Please go to <a class="reference external" href="https://github.com/PaddlePaddle/book">PaddlePaddle/book</a> and
<a class="reference external" href="https://github.com/PaddlePaddle/models">PaddlePaddle/models</a> to learn PaddlePaddle.</p>
</div>
......
develop/doc_cn/tutorials/embedding_model/index_cn.html → develop/doc_cn/v1_api_tutorials/embedding_model/index_cn.html
浏览文件 @ 0a404363
......@@ -297,7 +297,7 @@ python extract_para.py &#8211;preModel PREMODEL &#8211;preDict PREDICT &#8211;us
<li><code class="docutils literal"><span class="pre">--init_model_path</span></code>: 初始化模型的路径配置为<code class="docutils literal"><span class="pre">data/paraphrase_modeldata/paraphrase_model</span></code></li>
<li><code class="docutils literal"><span class="pre">--load_missing_parameter_strategy</span></code>：如果参数模型文件缺失，除词向量模型外的参数将使用正态分布随机初始化</li>
</ul>
<p>如果用户想要了解详细的数据集的格式、模型的结构和训练过程，请查看 <a class="reference internal" href="../text_generation/index_cn.html"><span class="doc">Text generation Tutorial</span></a>.</p>
<p>如果用户想要了解详细的数据集的格式、模型的结构和训练过程，请查看 <a class="reference external" href="v1_api_tutorials/text_generation/index_cn.md">Text generation Tutorial</a>.</p>
</div>
</div>
<div class="section" id="">
......
Markdown is supported
0% 或 .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑！
想要评论请 注册 或