Skip to content

P
Paddle

BaiXuePrincess / Paddle
与 Fork 源项目一致

Fork自 PaddlePaddle / Paddle

通知 1
Star 1
Fork 0
P
Paddle
提交 22cb62db 编写于 作者: T Travis CI
浏览文件
操作

Deploy to GitHub Pages: 703cce35

上级 a0ec673b
展开全部 隐藏空白更改
内联 并排
Showing with 9780 addition and 7596 deletion
doc/.buildinfo
浏览文件 @ 22cb62db
# Sphinx build info version 1 # Sphinx build info version 1
# This file hashes the configuration used when building these files. When it is not found, a full rebuild will be done. # This file hashes the configuration used when building these files. When it is not found, a full rebuild will be done.
config: 5bb206a2182263ffcb7c4270c50bc7c9 config: 68487318b6dbd705fd569c9b901c758d
tags: 645f666f9bcd5a90fca523b33c5a78b7 tags: 645f666f9bcd5a90fca523b33c5a78b7
doc/_sources/algorithm/rnn/rnn.txt
浏览文件 @ 22cb62db
...@@ -30,7 +30,7 @@ Then at the :code:`process` function, each :code:`yield` function will return th ...@@ -30,7 +30,7 @@ Then at the :code:`process` function, each :code:`yield` function will return th
yield src_ids, trg_ids, trg_ids_next yield src_ids, trg_ids, trg_ids_next
For more details description of how to write a data provider, please refer to :doc:`Python Data Provider <../py_data_provider_wrapper>`. The full data provider file is located at :code:`demo/seqToseq/dataprovider.py`. For more details description of how to write a data provider, please refer to `PyDataProvider2 <../../ui/data_provider/index.html>`_. The full data provider file is located at :code:`demo/seqToseq/dataprovider.py`.
=============================================== ===============================================
Configure Recurrent Neural Network Architecture Configure Recurrent Neural Network Architecture
...@@ -106,7 +106,7 @@ We will use the sequence to sequence model with attention as an example to demon ...@@ -106,7 +106,7 @@ We will use the sequence to sequence model with attention as an example to demon
In this model, the source sequence :math:`S = \{s_1, \dots, s_T\}` is encoded with a bidirectional gated recurrent neural networks. The hidden states of the bidirectional gated recurrent neural network :math:`H_S = \{H_1, \dots, H_T\}` is called *encoder vector* The decoder is a gated recurrent neural network. When decoding each token :math:`y_t`, the gated recurrent neural network generates a set of weights :math:`W_S^t = \{W_1^t, \dots, W_T^t\}`, which are used to compute a weighted sum of the encoder vector. The weighted sum of the encoder vector is utilized to condition the generation of the token :math:`y_t`. In this model, the source sequence :math:`S = \{s_1, \dots, s_T\}` is encoded with a bidirectional gated recurrent neural networks. The hidden states of the bidirectional gated recurrent neural network :math:`H_S = \{H_1, \dots, H_T\}` is called *encoder vector* The decoder is a gated recurrent neural network. When decoding each token :math:`y_t`, the gated recurrent neural network generates a set of weights :math:`W_S^t = \{W_1^t, \dots, W_T^t\}`, which are used to compute a weighted sum of the encoder vector. The weighted sum of the encoder vector is utilized to condition the generation of the token :math:`y_t`.
The encoder part of the model is listed below. It calls :code:`grumemory` to represent gated recurrent neural network. It is the recommended way of using recurrent neural network if the network architecture is simple, because it is faster than :code:`recurrent_group`. We have implemented most of the commonly used recurrent neural network architectures, you can refer to :doc:`Layers <../trainer_config_helpers/layers>` for more details. The encoder part of the model is listed below. It calls :code:`grumemory` to represent gated recurrent neural network. It is the recommended way of using recurrent neural network if the network architecture is simple, because it is faster than :code:`recurrent_group`. We have implemented most of the commonly used recurrent neural network architectures, you can refer to `Layers <../../ui/api/trainer_config_helpers/layers_index.html>`_ for more details.
We also project the encoder vector to :code:`decoder_size` dimensional space, get the first instance of the backward recurrent network, and project it to :code:`decoder_size` dimensional space: We also project the encoder vector to :code:`decoder_size` dimensional space, get the first instance of the backward recurrent network, and project it to :code:`decoder_size` dimensional space:
...@@ -246,6 +246,6 @@ The code is listed below: ...@@ -246,6 +246,6 @@ The code is listed below:
outputs(beam_gen) outputs(beam_gen)
Notice that this generation technique is only useful for decoder like generation process. If you are working on sequence tagging tasks, please refer to :doc:`Semantic Role Labeling Demo <../../../demo/semantic_role_labeling>` for more details. Notice that this generation technique is only useful for decoder like generation process. If you are working on sequence tagging tasks, please refer to `Semantic Role Labeling Demo <../../demo/semantic_role_labeling/index.html>`_ for more details.
The full configuration file is located at :code:`demo/seqToseq/seqToseq_net.py`. The full configuration file is located at :code:`demo/seqToseq/seqToseq_net.py`.
doc/_sources/build/build_from_source.txt
浏览文件 @ 22cb62db
...@@ -37,11 +37,12 @@ PaddlePaddle also support some build options, you have to install related librar ...@@ -37,11 +37,12 @@ PaddlePaddle also support some build options, you have to install related librar
```bash ```bash
# necessary # necessary
sudo apt-get update sudo apt-get update
sudo apt-get install -y g++ make cmake build-essential libatlas-base-dev python python-pip libpython-dev m4 libprotobuf-dev protobuf-compiler python-protobuf python-numpy git sudo apt-get install -y g++ make cmake build-essential libatlas-base-dev python python-pip libpython-dev m4 libprotobuf-dev protobuf-compiler python-protobuf python-numpy git
# optional # optional
sudo apt-get install libgoogle-glog-dev sudo apt-get install libgoogle-glog-dev
sudo apt-get install libgflags-dev sudo apt-get install libgflags-dev
sudo apt-get install libgtest-dev sudo apt-get install libgtest-dev
sudo pip install wheel
pushd /usr/src/gtest pushd /usr/src/gtest
cmake . cmake .
make make
...@@ -125,6 +126,8 @@ mkdir build ...@@ -125,6 +126,8 @@ mkdir build
cd build cd build
# you can add build option here, such as: # you can add build option here, such as:
cmake -DWITH_GPU=ON -DWITH_DOC=OFF -DCMAKE_INSTALL_PREFIX=<path to install> .. cmake -DWITH_GPU=ON -DWITH_DOC=OFF -DCMAKE_INSTALL_PREFIX=<path to install> ..
# please use sudo make install, if you want
# to install PaddlePaddle into the system
make -j `nproc` && make install make -j `nproc` && make install
# PaddlePaddle installation path # PaddlePaddle installation path
export PATH=<path to install>/bin:$PATH export PATH=<path to install>/bin:$PATH
......
doc/_sources/build/contribute_to_paddle.txt
浏览文件 @ 22cb62db
...@@ -25,7 +25,7 @@ repo or just head straight to the command line: ...@@ -25,7 +25,7 @@ repo or just head straight to the command line:
```shell ```shell
# Clone your fork to your local machine # Clone your fork to your local machine
git clone git@github.com:USERNAME/paddle.git git clone git@github.com:USERNAME/Paddle.git
``` ```
Then you can start to develop. Then you can start to develop.
...@@ -52,7 +52,7 @@ To do this, you'll need to add a remote at first: ...@@ -52,7 +52,7 @@ To do this, you'll need to add a remote at first:
# see the current configured remote repository # see the current configured remote repository
git remote -v git remote -v
# add upstream repository # add upstream repository
git remote add upstream https://github.com/paddle/paddle.git git remote add upstream https://github.com/baidu/Paddle.git
# verify the new upstream # verify the new upstream
git remote -v git remote -v
``` ```
......
doc/_sources/build/docker_install.txt
浏览文件 @ 22cb62db
...@@ -8,12 +8,12 @@ Docker is a tool designed to make it easier to create, deploy, and run applicati ...@@ -8,12 +8,12 @@ Docker is a tool designed to make it easier to create, deploy, and run applicati
### PaddlePaddle Docker images ### PaddlePaddle Docker images
There are six Docker images: There are six Docker images:
- paddledev/paddle:latest-cpu: PaddlePaddle CPU binary image. - paddledev/paddle:cpu-latest: PaddlePaddle CPU binary image.
- paddledev/paddle:latest-gpu: PaddlePaddle GPU binary image. - paddledev/paddle:gpu-latest: PaddlePaddle GPU binary image.
- paddledev/paddle:latest-cpu-devel: PaddlePaddle CPU binary image plus source code. - paddledev/paddle:cpu-devel-latest: PaddlePaddle CPU binary image plus source code.
- paddledev/paddle:latest-gpu-devel: PaddlePaddle GPU binary image plus source code. - paddledev/paddle:gpu-devel-latest: PaddlePaddle GPU binary image plus source code.
- paddledev/paddle:latest-cpu-demo: PaddlePaddle CPU binary image plus source code and demo - paddledev/paddle:cpu-demo-latest: PaddlePaddle CPU binary image plus source code and demo
- paddledev/paddle:latest-gpu-demo: PaddlePaddle GPU binary image plus source code and demo - paddledev/paddle:gpu-demo-latest: PaddlePaddle GPU binary image plus source code and demo
Tags with latest will be replaced by a released version. Tags with latest will be replaced by a released version.
...@@ -23,7 +23,7 @@ You have to install Docker in your machine which has linux kernel version 3.10+ ...@@ -23,7 +23,7 @@ You have to install Docker in your machine which has linux kernel version 3.10+
You can use ```docker pull ```to download images first, or just launch a container with ```docker run```: You can use ```docker pull ```to download images first, or just launch a container with ```docker run```:
```bash ```bash
docker run -it paddledev/paddle:lastest-cpu docker run -it paddledev/paddle:cpu-latest
``` ```
If you want to launch container with GPU support, you need to set some environment variables at the same time: If you want to launch container with GPU support, you need to set some environment variables at the same time:
...@@ -31,7 +31,7 @@ If you want to launch container with GPU support, you need to set some environme ...@@ -31,7 +31,7 @@ If you want to launch container with GPU support, you need to set some environme
```bash ```bash
export CUDA_SO="$(\ls /usr/lib64/libcuda* | xargs -I{} echo '-v {}:{}') $(\ls /usr/lib64/libnvidia* | xargs -I{} echo '-v {}:{}" export CUDA_SO="$(\ls /usr/lib64/libcuda* | xargs -I{} echo '-v {}:{}') $(\ls /usr/lib64/libnvidia* | xargs -I{} echo '-v {}:{}"
export DEVICES=$(\ls /dev/nvidia* | xargs -I{} echo '--device {}:{}') export DEVICES=$(\ls /dev/nvidia* | xargs -I{} echo '--device {}:{}')
docker run -it paddledev/paddle:latest-gpu docker run -it paddledev/paddle:gpu-latest
``` ```
### Notice ### Notice
......
doc/_sources/build/index.txt
浏览文件 @ 22cb62db
...@@ -5,9 +5,11 @@ Install PaddlePaddle ...@@ -5,9 +5,11 @@ Install PaddlePaddle
---------------------- ----------------------
.. toctree:: .. toctree::
:maxdepth: 1
:glob: :glob:
install_* install_*
internal/install_from_jumbo.md
Build from Source Build from Source
----------------- -----------------
...@@ -15,6 +17,7 @@ Build from Source ...@@ -15,6 +17,7 @@ Build from Source
If you want to hack and contribute PaddlePaddle source code, following guides can help you\: If you want to hack and contribute PaddlePaddle source code, following guides can help you\:
.. toctree:: .. toctree::
:maxdepth: 1
:glob: :glob:
build_from_source.md build_from_source.md
...@@ -29,6 +32,7 @@ state and your experience of installation may not be smooth. ...@@ -29,6 +32,7 @@ state and your experience of installation may not be smooth.
If you want to pack docker image, the following guide can help you\: If you want to pack docker image, the following guide can help you\:
.. toctree:: .. toctree::
:maxdepth: 1
:glob: :glob:
docker_install.md docker_install.md
......
doc/_sources/cluster/index.txt
浏览文件 @ 22cb62db
...@@ -5,3 +5,4 @@ Cluster Train ...@@ -5,3 +5,4 @@ Cluster Train
:glob: :glob:
opensource/cluster_train.md opensource/cluster_train.md
internal/index.md
doc/_sources/demo/embedding_model/index.txt
浏览文件 @ 22cb62db
...@@ -93,7 +93,7 @@ where `train.sh` is almost the same as `demo/seqToseq/translation/train.sh`, the ...@@ -93,7 +93,7 @@ where `train.sh` is almost the same as `demo/seqToseq/translation/train.sh`, the
- `--init_model_path`: path of the initialization model, here is `data/paraphrase_model` - `--init_model_path`: path of the initialization model, here is `data/paraphrase_model`
- `--load_missing_parameter_strategy`: operations when model file is missing, here use a normal distibution to initialize the other parameters except for the embedding layer - `--load_missing_parameter_strategy`: operations when model file is missing, here use a normal distibution to initialize the other parameters except for the embedding layer
For users who want to understand the dataset format, model architecture and training procedure in detail, please refer to [Text generation Tutorial](text_generation.md). For users who want to understand the dataset format, model architecture and training procedure in detail, please refer to [Text generation Tutorial](../text_generation/text_generation.md).
## Optional Function ## ## Optional Function ##
### Embedding Parameters Observation ### Embedding Parameters Observation
......
doc/_sources/demo/imagenet_model/resnet_model.txt
浏览文件 @ 22cb62db
...@@ -165,7 +165,7 @@ We provide both C++ and Python interfaces to extract features. The following exa ...@@ -165,7 +165,7 @@ We provide both C++ and Python interfaces to extract features. The following exa
### C++ Interface ### C++ Interface
First, specify image data list in `define_py_data_sources` in the config, see example `demo/model_zoo/resnet/resnet.py`. First, specify image data list in `define_py_data_sources2` in the config, see example `demo/model_zoo/resnet/resnet.py`.
``` ```
train_list = 'train.list' if not is_test else None train_list = 'train.list' if not is_test else None
......
doc/_sources/demo/quick_start/index_en.txt
浏览文件 @ 22cb62db
...@@ -59,7 +59,7 @@ To build your text classification system, your code will need to perform five st ...@@ -59,7 +59,7 @@ To build your text classification system, your code will need to perform five st
## Preprocess data into standardized format ## Preprocess data into standardized format
In this example, you are going to use [Amazon electronic product review dataset](http://jmcauley.ucsd.edu/data/amazon/) to build a bunch of deep neural network models for text classification. Each text in this dataset is a product review. This dataset has two categories: “positive” and “negative”. Positive means the reviewer likes the product, while negative means the reviewer does not like the product. In this example, you are going to use [Amazon electronic product review dataset](http://jmcauley.ucsd.edu/data/amazon/) to build a bunch of deep neural network models for text classification. Each text in this dataset is a product review. This dataset has two categories: “positive” and “negative”. Positive means the reviewer likes the product, while negative means the reviewer does not like the product.
`demo/quick_start` provides scripts for downloading data and preprocessing data as shown below. The data process takes several minutes (about 3 minutes in our machine). `demo/quick_start` in the [source code](https://github.com/baidu/Paddle) provides scripts for downloading data and preprocessing data as shown below. The data process takes several minutes (about 3 minutes in our machine).
```bash ```bash
cd demo/quick_start cd demo/quick_start
...@@ -157,9 +157,7 @@ define_py_data_sources2(train_list='data/train.list', ...@@ -157,9 +157,7 @@ define_py_data_sources2(train_list='data/train.list',
obj="process", obj="process",
args={"dictionary": word_dict}) args={"dictionary": word_dict})
``` ```
You can refer to the following link for more detailed examples and data formats: <a href = "../../ui/data_provider/pydataprovider2.html">PyDataProvider2</a>.
You can refer to the following link for more detailed examples
: <a href = "../../ui/data_provider/python_case.html">Python Use Case</a>，The detailed documentation on data format is: <a href = "../../ui/api/py_data_provider_wrapper.html"> PyDataProviderWrapper</a>。
## Network Architecture ## Network Architecture
You will describe four kinds of network architectures in this section. You will describe four kinds of network architectures in this section.
...@@ -425,7 +423,7 @@ paddle train \ ...@@ -425,7 +423,7 @@ paddle train \
mv rank-00000 result.txt mv rank-00000 result.txt
``` ```
There are several differences between training and inference network configurations. User can choose the best model base on the training log instead of model `output/pass-00003`. There are several differences between training and inference network configurations.
- You do not need labels during inference. - You do not need labels during inference.
- Outputs need to be specified to the classification probability layer (the output of softmax layer), or the id of maximum probability (`max_id` layer). An example to output the id and probability is given in the code snippet. - Outputs need to be specified to the classification probability layer (the output of softmax layer), or the id of maximum probability (`max_id` layer). An example to output the id and probability is given in the code snippet.
- batch_size = 1. - batch_size = 1.
......
doc/_sources/demo/rec/ml_regression.txt
浏览文件 @ 22cb62db
...@@ -219,9 +219,9 @@ The network structure shows below. ...@@ -219,9 +219,9 @@ The network structure shows below.
The demo's neural network config file "trainer_config.py" show as below. The demo's neural network config file "trainer_config.py" show as below.
.. include:: ../../../demo/recommendation/trainer_config.py .. literalinclude:: ../../../demo/recommendation/trainer_config.py
:code: python :language: python
:literal: :lines: 15-
In this :code:`trainer_config.py`, we just map each feature type to In this :code:`trainer_config.py`, we just map each feature type to
a feature vector, following shows how to map each feature to a vector shows below. a feature vector, following shows how to map each feature to a vector shows below.
...@@ -257,15 +257,15 @@ In these network, we use several api in `trainer_config_helpers ...@@ -257,15 +257,15 @@ In these network, we use several api in `trainer_config_helpers
* Text Convolution Pooling Layer, `text_conv_pool * Text Convolution Pooling Layer, `text_conv_pool
<../../ui/api/trainer_config_helpers/networks.html <../../ui/api/trainer_config_helpers/networks.html
#trainer_config_helpers.networks.text_conv_pool>`_ #trainer_config_helpers.networks.text_conv_pool>`_
* Declare Python Data Sources, `define_py_data_sources * Declare Python Data Sources, `define_py_data_sources2
<../../ui/api/trainer_config_helpers/data_sources.html>`_ <../../ui/api/trainer_config_helpers/data_sources.html>`_
Data Provider Data Provider
''''''''''''' '''''''''''''
.. include:: ../../../demo/recommendation/dataprovider.py .. literalinclude:: ../../../demo/recommendation/dataprovider.py
:code: python :language: python
:literal: :lines: 15-
The data provider just read the meta.bin and rating file, yield each sample for training. The data provider just read the meta.bin and rating file, yield each sample for training.
In this :code:`dataprovider.py`, we should set\: In this :code:`dataprovider.py`, we should set\:
...@@ -274,7 +274,7 @@ In this :code:`dataprovider.py`, we should set\: ...@@ -274,7 +274,7 @@ In this :code:`dataprovider.py`, we should set\:
* use_seq\: Whether this :code:`dataprovider.py` in sequence mode or not. * use_seq\: Whether this :code:`dataprovider.py` in sequence mode or not.
* process\: Return each sample of data to :code:`paddle`. * process\: Return each sample of data to :code:`paddle`.
The data provider details document see `there <../../ui/DataProvider.html>`_. The data provider details document see `there <../../ui/data_provider/pydataprovider2.html>`_.
Train Train
````` `````
...@@ -283,15 +283,15 @@ After prepare data, config network, writting data provider, now we can run paddl ...@@ -283,15 +283,15 @@ After prepare data, config network, writting data provider, now we can run paddl
The run.sh is shown as follow: The run.sh is shown as follow:
.. include:: ../../../demo/recommendation/run.sh .. literalinclude:: ../../../demo/recommendation/run.sh
:code: bash :language: bash
:literal: :lines: 16-
It just start a paddle training process, write the log to `log.txt`, It just start a paddle training process, write the log to `log.txt`,
then print it on screen. then print it on screen.
Each command line argument in :code:`run.sh`, please refer to the `command line Each command line argument in :code:`run.sh`, please refer to the `command line
arguments <TBD>`_ page. The short description of these arguments is shown as follow. arguments <../../ui/index.html#command-line-argument>`_ page. The short description of these arguments is shown as follow.
* config\: Tell paddle which file is neural network configuration. * config\: Tell paddle which file is neural network configuration.
* save_dir\: Tell paddle save model into './output' * save_dir\: Tell paddle save model into './output'
...@@ -303,8 +303,6 @@ arguments <TBD>`_ page. The short description of these arguments is shown as fol ...@@ -303,8 +303,6 @@ arguments <TBD>`_ page. The short description of these arguments is shown as fol
* dot_period\: Print a :code:`.` after train :code:`dot_period` batches. * dot_period\: Print a :code:`.` after train :code:`dot_period` batches.
* num_passes\: Train at most :code:`num_passes`. * num_passes\: Train at most :code:`num_passes`.
If training process starts successfully, the output likes follow: If training process starts successfully, the output likes follow:
.. code-block:: text .. code-block:: text
......
doc/_sources/demo/semantic_role_labeling/index.txt 0 → 100644
浏览文件 @ 22cb62db
Semantic Role Labeling Tutorial
===============================
.. toctree::
:maxdepth: 3
semantic_role_labeling.md
doc/_sources/demo/semantic_role_labeling/semantic_role_labeling.txt 0 → 100644
浏览文件 @ 22cb62db
# Semantic Role labeling Tutorial #
Semantic role labeling (SRL) is a form of shallow semantic parsing whose goal is to discover the predicate-argument structure of each predicate in a given input sentence. SRL is useful as an intermediate step in a wide range of natural language processing tasks, such as information extraction. automatic document categorization and question answering. An instance is as following [1]:
[ <sub>A0</sub> He ] [ <sub>AM-MOD</sub> would ][ <sub>AM-NEG</sub> n’t ] [ <sub>V</sub> accept] [ <sub>A1</sub> anything of value ] from [<sub>A2</sub> those he was writing about ].
- V: verb
- A0: acceptor
- A1: thing accepted
- A2: accepted-from
- A3: Attribute
- AM-MOD: modal
- AM-NEG: negation
Given the verb "accept", the chunks in sentence would play certain semantic roles. Here, the label scheme is from Penn Proposition Bank.
To this date, most of the successful SRL systems are built on top of some form of parsing results where pre-defined feature templates over the syntactic structure are used. This tutorial will present an end-to-end system using deep bidirectional long short-term memory (DB-LSTM)[2] for solving the SRL task, which largely outperforms the previous state-of-the-art systems. The system regards SRL task as the sequence labelling problem.
## Data Description
The relevant paper[2] takes the data set in CoNLL-2005&2012 Shared Task for training and testing. Accordingto data license, the demo adopts the test data set of CoNLL-2005, which can be reached on website.
To download and process the original data, user just need to execute the following command:
```bash
cd data
./get_data.sh
```
Several new files appear in the `data `directory as follows.
```bash
conll05st-release：the test data set of CoNll-2005 shared task
test.wsj.words：the Wall Street Journal data sentences
test.wsj.props: the propositional arguments
src.dict：the dictionary of words in sentences
tgt.dict：the labels dictionary
feature: the extracted features from data set
```
## Training
### DB-LSTM
Please refer to the Sentiment Analysis demo to learn more about the long short-term memory unit.
Unlike Bidirectional-LSTM that used in Sentiment Analysis demo, the DB-LSTM adopts another way to stack LSTM layer. First a standard LSTM processes the sequence in forward direction. The input and output of this LSTM layer are taken by the next LSTM layer as input, processed in reversed direction. These two standard LSTM layers compose a pair of LSTM. Then we stack LSTM layers pair after pair to obtain the deep LSTM model.
The following figure shows a temporal expanded 2-layer DB-LSTM network.
<center>
![pic](./network_arch.png)
</center>
### Features
Two input features play an essential role in this pipeline: predicate (pred) and argument (argu). Two other features: predicate context (ctx-p) and region mark (mr) are also adopted. Because a single predicate word can not exactly describe the predicate information, especially when the same words appear more than one times in a sentence. With the predicate context, the ambiguity can be largely eliminated. Similarly, we use region mark m<sub>r</sub> = 1 to denote the argument position if it locates in the predicate context region, or m<sub>r</sub> = 0 if does not. These four simple features are all we need for our SRL system. Features of one sample with context size set to 1 is showed as following[2]:
<center>
![pic](./feature.jpg)
</center>
In this sample, the coresponding labelled sentence is:
[ <sub>A1</sub> A record date ] has [ <sub>AM-NEG</sub> n't ] been [ <sub>V</sub> set ] .
In the demo, we adopt the feature template as above, consists of : `argument`, `predicate`, `ctx-p (p=-1,0,1)`, `mark` and use `B/I/O` scheme to label each argument. These features and labels are stored in `feature` file, and separated by `\t`.
### Data Provider
`dataprovider.py` is the python file to wrap data. `hook()` function is to define the data slots for network. The Six features and label are all IndexSlots.
```
def hook(settings, word_dict, label_dict, **kwargs):
settings.word_dict = word_dict
settings.label_dict = label_dict
#all inputs are integral and sequential type
settings.slots = [
integer_value_sequence(len(word_dict)),
integer_value_sequence(len(word_dict)),
integer_value_sequence(len(word_dict)),
integer_value_sequence(len(word_dict)),
integer_value_sequence(len(word_dict)),
integer_value_sequence(2),
integer_value_sequence(len(label_dict))]
```
The corresponding data iterator is as following:
```
@provider(use_seq=True, init_hook=hook)
def process(obj, file_name):
with open(file_name, 'r') as fdata:
for line in fdata:
sentence, predicate, ctx_n1, ctx_0, ctx_p1, mark, label = line.strip().split('\t')
words = sentence.split()
sen_len = len(words)
word_slot = [obj.word_dict.get(w, UNK_IDX) for w in words]
predicate_slot = [obj.word_dict.get(predicate, UNK_IDX)] * sen_len
ctx_n1_slot = [obj.word_dict.get(ctx_n1, UNK_IDX) ] * sen_len
ctx_0_slot = [obj.word_dict.get(ctx_0, UNK_IDX) ] * sen_len
ctx_p1_slot = [obj.word_dict.get(ctx_p1, UNK_IDX) ] * sen_len
marks = mark.split()
mark_slot = [int(w) for w in marks]
label_list = label.split()
label_slot = [obj.label_dict.get(w) for w in label_list]
yield word_slot, predicate_slot, ctx_n1_slot, ctx_0_slot, ctx_p1_slot, mark_slot, label_slot
```
The `process`function yield 7 lists which are six features and labels.
### Neural Network Config
`db_lstm.py` is the neural network config file to load the dictionaries and define the data provider module and network architecture during the training procedure.
Seven `data_layer` load instances from data provider. Six features are transformed into embedddings respectively, and mixed by `mixed_layer` . Deep bidirectional LSTM layers extract features for the softmax layer. The objective function is cross entropy of labels.
### Run Training
The script for training is `train.sh`, user just need to execute:
```bash
./train.sh
```
The content in `train.sh`:
```
paddle train \
--config=./db_lstm.py \
--save_dir=./output \
--trainer_count=4 \
--log_period=10 \
--num_passes=500 \
--use_gpu=false \
--show_parameter_stats_period=10 \
--test_all_data_in_one_period=1 \
2>&1 | tee 'train.log'
```
- \--config=./db_lstm.py : network config file.
- \--save_di=./output: output path to save models.
- \--trainer_count=4 : set thread number (or GPU count).
- \--log_period=10 : print log every 20 batches.
- \--num_passes=500: set pass number, one pass in PaddlePaddle means training all samples in dataset one time.
- \--use_gpu=false: use CPU to train, set true, if you install GPU version of PaddlePaddle and want to use GPU to train.
- \--show_parameter_stats_period=10: show parameter statistic every 100 batches.
- \--test_all_data_in_one_period=1: test all data in every testing.
After training, the models will be saved in directory `output`.
### Run testing
The script for testing is `test.sh`, user just need to execute:
```bash
./test.sh
```
The main part in `tesh.sh`
```
paddle train \
--config=./db_lstm.py \
--model_list=$model_list \
--job=test \
--config_args=is_test=1 \
```
- \--config=./db_lstm.py: network config file
- \--model_list=$model_list.list: model list file
- \--job=test: indicate the test job
- \--config_args=is_test=1: flag to indicate test
### Run prediction
The script for prediction is `predict.sh`, user just need to execute:
```bash
./predict.sh
```
In `predict.sh`, user should offer the network config file, model path, label file, word dictionary file, feature file
```
python predict.py
-c $config_file
-w $model_path
-l $label_file
-d $dict_file
-i $input_file
```
`predict.py` is the main executable python script, which includes functions: load model, load data, data prediction. The network model will output the probability distribution of labels. In the demo, we take the label with maximum probability as result. User can also implement the beam search or viterbi decoding upon the probability distribution matrix.
After prediction, the result is saved in `predict.res`.
## Reference
[1] Martha Palmer, Dan Gildea, and Paul Kingsbury. The Proposition Bank: An Annotated Corpus of Semantic Roles , Computational Linguistics, 31(1), 2005.
[2] Zhou, Jie, and Wei Xu. "End-to-end learning of semantic role labeling using recurrent neural networks." Proceedings of the Annual Meeting of the Association for Computational Linguistics. 2015.
doc/_sources/source/math/utils/utils.txt
浏览文件 @ 22cb62db
Utils Utils
======= =======
Bits
-------
.. doxygenfile:: paddle/math/Bits.h
Memory Handle Memory Handle
-------------- --------------
.. doxygenfile:: paddle/math/MemoryHandle.h .. doxygenfile:: paddle/math/MemoryHandle.h
......
doc/_sources/ui/api/trainer_config_helpers/layers.txt
浏览文件 @ 22cb62db
...@@ -82,12 +82,6 @@ img_cmrnorm_layer ...@@ -82,12 +82,6 @@ img_cmrnorm_layer
:members: img_cmrnorm_layer :members: img_cmrnorm_layer
:noindex: :noindex:
img_rnorm_layer
-----------------
.. automodule:: paddle.trainer_config_helpers.layers
:members: img_rnorm_layer
:noindex:
batch_norm_layer batch_norm_layer
--------------------- ---------------------
.. automodule:: paddle.trainer_config_helpers.layers .. automodule:: paddle.trainer_config_helpers.layers
...@@ -251,10 +245,10 @@ addto_layer ...@@ -251,10 +245,10 @@ addto_layer
:members: addto_layer :members: addto_layer
:noindex: :noindex:
convex_comb_layer linear_comb_layer
----------------- -----------------
.. automodule:: paddle.trainer_config_helpers.layers .. automodule:: paddle.trainer_config_helpers.layers
:members: convex_comb_layer :members: linear_comb_layer
:noindex: :noindex:
interpolation_layer interpolation_layer
...@@ -286,7 +280,13 @@ tensor_layer ...@@ -286,7 +280,13 @@ tensor_layer
.. automodule:: paddle.trainer_config_helpers.layers .. automodule:: paddle.trainer_config_helpers.layers
:members: tensor_layer :members: tensor_layer
:noindex: :noindex:
cos_sim
-------
.. automodule:: paddle.trainer_config_helpers.layers
:members: cos_sim
:noindex:
trans_layer trans_layer
------------ ------------
.. automodule:: paddle.trainer_config_helpers.layers .. automodule:: paddle.trainer_config_helpers.layers
...@@ -347,12 +347,6 @@ rank_cost ...@@ -347,12 +347,6 @@ rank_cost
:members: rank_cost :members: rank_cost
:noindex: :noindex:
cos_sim
-------
.. automodule:: paddle.trainer_config_helpers.layers
:members: cos_sim
:noindex:
crf_layer crf_layer
----------------- -----------------
.. automodule:: paddle.trainer_config_helpers.layers .. automodule:: paddle.trainer_config_helpers.layers
......
doc/_sources/ui/data_provider/index.txt
浏览文件 @ 22cb62db
PaddlePaddle DataProvider Introduction DataProvider Introduction
================================ =========================
DataProvider is a module that loads training or testing data into cpu or gpu DataProvider is a module that loads training or testing data into cpu or gpu
memory for the following triaining or testing process. memory for the following triaining or testing process.
......
doc/_sources/ui/data_provider/pydataprovider2.txt
浏览文件 @ 22cb62db
...@@ -152,7 +152,6 @@ Please refer to the following section reference for details. ...@@ -152,7 +152,6 @@ Please refer to the following section reference for details.
Reference Reference
--------- ---------
.. _@provider::
@provider @provider
+++++++++ +++++++++
...@@ -170,31 +169,28 @@ PaddlePaddle from a user defined function. Its parameters are: ...@@ -170,31 +169,28 @@ PaddlePaddle from a user defined function. Its parameters are:
usefull in sequential model, that defines batch size is counted upon sequence usefull in sequential model, that defines batch size is counted upon sequence
or token. By default, each sample or sequence counts to 1 when calculating or token. By default, each sample or sequence counts to 1 when calculating
batch size. batch size.
* cache is a data cache strategy, see `cache`_ * cache is a data cache strategy, see `cache`_.
* Init_hook function is invoked once the data provider is initialized, * Init_hook function is invoked once the data provider is initialized,
see `init_hook`_ see `init_hook`_.
.. _input_types::
input_types input_types
+++++++++++ +++++++++++
PaddlePaddle has four data types, and three sequence types. PaddlePaddle has four data types, and three sequence types.
The four data types are: The four data types are:
* dense_vector represents dense float vector. * :code:`dense_vector`: dense float vector.
* sparse_binary_vector sparse binary vector, most of the value is 0, and * :code:`sparse_binary_vector`: sparse binary vector, most of the value is 0, and
the non zero elements are fixed to 1. the non zero elements are fixed to 1.
* sparse_float_vector sparse float vector, most of the value is 0, and some * :code:`sparse_float_vector`: sparse float vector, most of the value is 0, and some
non zero elements that can be any float value. They are given by the user. non zero elements can be any float value. They are given by the user.
* integer represents an integer scalar, that is especially used for label or * :code:`integer`: an integer scalar, that is especially used for label or word index.
word index.
The three sequence types are:
The three sequence types are * :code:`SequenceType.NO_SEQUENCE` means the sample is not a sequence.
* :code:`SequenceType.SEQUENCE` means the sample is a sequence.
* SequenceType.NO_SEQUENCE means the sample is not a sequence * :code:`SequenceType.SUB_SEQUENCE` means it is a nested sequence, that each timestep of
* SequenceType.SEQUENCE means the sample is a sequence
* SequenceType.SUB_SEQUENCE means it is a nested sequence, that each timestep of
the input sequence is also a sequence. the input sequence is also a sequence.
Different input type has a defferenct input format. Their formats are shown Different input type has a defferenct input format. Their formats are shown
...@@ -214,36 +210,39 @@ in the above table. ...@@ -214,36 +210,39 @@ in the above table.
where f represents a float value, i represents an integer value. where f represents a float value, i represents an integer value.
.. _init_hook::
.. _settings::
init_hook init_hook
+++++++++ +++++++++
init_hook is a function that is invoked once the data provoder is initialized. init_hook is a function that is invoked once the data provoder is initialized.
Its parameters lists as follows: Its parameters lists as follows:
* The first parameter is a settings object, which is the same to :code:'settings' * The first parameter is a settings object, which is the same to :code:`settings`
in :code:`process` method. The object contains several attributes, including: in :code:`process` method. The object contains several attributes, including:
* settings.input_types the input types. Reference `input_types`_
* settings.logger a logging object * :code:`settings.input_types`: the input types. Reference `input_types`_.
* :code:`settings.logger`: a logging object.
* The rest parameters are the key word arguments. It is made up of PaddpePaddle * The rest parameters are the key word arguments. It is made up of PaddpePaddle
pre-defined parameters and user defined parameters. pre-defined parameters and user defined parameters.
* PaddlePaddle defines parameters including:
* is_train is a bool parameter that indicates the DataProvider is used in * PaddlePaddle-defined parameters including:
training or testing
* file_list is the list of all files. * :code:`is_train` is a bool parameter that indicates the DataProvider is used in
training or testing.
* :code:`file_list` is the list of all files.
* User-defined parameters args can be set in training configuration. * User-defined parameters args can be set in training configuration.
Note, PaddlePaddle reserves the right to add pre-defined parameter, so please Note, PaddlePaddle reserves the right to add pre-defined parameter, so please
use :code:`**kwargs` in init_hook to ensure compatibility by accepting the use :code:`**kwargs` in init_hook to ensure compatibility by accepting the
parameters which your init_hook does not use. parameters which your init_hook does not use.
.. _cache ::
cache cache
+++++ +++++
DataProvider provides two simple cache strategy. They are DataProvider provides two simple cache strategy. They are:
* CacheType.NO_CACHE means do not cache any data, then data is read at runtime by
* :code:`CacheType.NO_CACHE` means do not cache any data, then data is read at runtime by
the user implemented python module every pass. the user implemented python module every pass.
* CacheType.CACHE_PASS_IN_MEM means the first pass reads data by the user * :code:`CacheType.CACHE_PASS_IN_MEM` means the first pass reads data by the user
implemented python module, and the rest passes will directly read data from implemented python module, and the rest passes will directly read data from
memory. memory.
doc/_sources/ui/index.txt
浏览文件 @ 22cb62db
...@@ -7,7 +7,7 @@ ...@@ -7,7 +7,7 @@
## API Reference ## API Reference
* [Trainer Config Helpers](api/trainer_config_helpers/index.md) * [Model Config Interface](api/trainer_config_helpers/index.md)
## Command Line Argument ## Command Line Argument
......
doc/_sources/ui/predict/swig_py_paddle_en.txt
浏览文件 @ 22cb62db
...@@ -10,27 +10,35 @@ SWIG. The main steps of predict values in python are: ...@@ -10,27 +10,35 @@ SWIG. The main steps of predict values in python are:
* Predict * Predict
Here is a sample python script that shows the typical prediction process for the Here is a sample python script that shows the typical prediction process for the
MNIST classification problem. MNIST classification problem. A complete sample code could be found at
:code:`src_root/doc/ui/predict/predict_sample.py`.
.. literalinclude:: ./predict_sample.py .. literalinclude:: ./predict_sample.py
:language: python :language: python
:linenos: :lines: 15-18,90-100,101-104
The module that does the most of the job is py_paddle.swig_paddle, it's The module that does the most of the job is py_paddle.swig_paddle, it's
generated by SWIG and has complete documents, for more details you can use generated by SWIG and has complete documents, for more details you can use
python's :code:`help()` function. Let's walk through the above python script: python's :code:`help()` function. Let's walk through the above python script:
* At the beginning, initialize PaddlePaddle with command line arguments(line 90). * At the beginning, use :code:`swig_paddle.initPaddle()` to initialize
* Parse the configuration file that is used in training(line 93). PaddlePaddle with command line arguments, for more about command line arguments
* Create a neural network at line 95 according the parsed configuration, then see `Command Line Arguments <../cmd_argument/detail_introduction.html>`_.
load the trained parameters from model at line 97. * Parse the configuration file that is used in training with :code:`parse_config()`.
* A utility class for data transformation is created at line 98. Because data to predict with always have no label, and output of prediction work
normally is the output layer rather than the cost layer, so you should modify
the configuration file accordingly before using it in the prediction work.
* Create a neural network with
:code:`swig_paddle.GradientMachine.createFromConfigproto()`, which takes the
parsed configuration :code:`conf.model_config` as argument. Then load the
trained parameters from the model with :code:`network.loadParameters()`.
* Create a data converter object of utility class :code:`DataProviderConverter`.
- Note: As swig_paddle can only accept C++ matrices, we offer a utility - Note: As swig_paddle can only accept C++ matrices, we offer a utility
class DataProviderWraaperConverter that can accept the same input data with class DataProviderConverter that can accept the same input data with
PyDataProviderWrapper, for more information please refer to document PyDataProvider2, for more information please refer to document
of `PyDataProviderWrapper <../py_data_provider_wrapper_api.html>`_. of `PyDataProvider2 <../data_provider/pydataprovider2.html>`_.
* Do the prediction and output the result at line 100, forwardTest is another * Do the prediction with :code:`forwardTest()`, which takes the converted
utility class that directly takes the activations of the output layer. input data and outputs the activations of the output layer.
Here is a typical output: Here is a typical output:
......
doc/_static/basic.css
浏览文件 @ 22cb62db
...@@ -52,6 +52,8 @@ div.sphinxsidebar { ...@@ -52,6 +52,8 @@ div.sphinxsidebar {
width: 230px; width: 230px;
margin-left: -100%; margin-left: -100%;
font-size: 90%; font-size: 90%;
word-wrap: break-word;
overflow-wrap : break-word;
} }
div.sphinxsidebar ul { div.sphinxsidebar ul {
...@@ -83,10 +85,6 @@ div.sphinxsidebar #searchbox input[type="text"] { ...@@ -83,10 +85,6 @@ div.sphinxsidebar #searchbox input[type="text"] {
width: 170px; width: 170px;
} }
div.sphinxsidebar #searchbox input[type="submit"] {
width: 30px;
}
img { img {
border: 0; border: 0;
max-width: 100%; max-width: 100%;
...@@ -187,6 +185,13 @@ div.genindex-jumpbox { ...@@ -187,6 +185,13 @@ div.genindex-jumpbox {
/* -- general body styles --------------------------------------------------- */ /* -- general body styles --------------------------------------------------- */
div.body p, div.body dd, div.body li, div.body blockquote {
-moz-hyphens: auto;
-ms-hyphens: auto;
-webkit-hyphens: auto;
hyphens: auto;
}
a.headerlink { a.headerlink {
visibility: hidden; visibility: hidden;
} }
......
doc/_static/doctools.js
浏览文件 @ 22cb62db
...@@ -124,6 +124,7 @@ var Documentation = { ...@@ -124,6 +124,7 @@ var Documentation = {
this.fixFirefoxAnchorBug(); this.fixFirefoxAnchorBug();
this.highlightSearchWords(); this.highlightSearchWords();
this.initIndexTable(); this.initIndexTable();
}, },
/** /**
...@@ -252,6 +253,29 @@ var Documentation = { ...@@ -252,6 +253,29 @@ var Documentation = {
}); });
var url = parts.join('/'); var url = parts.join('/');
return path.substring(url.lastIndexOf('/') + 1, path.length - 1); return path.substring(url.lastIndexOf('/') + 1, path.length - 1);
},
initOnKeyListeners: function() {
$(document).keyup(function(event) {
var activeElementType = document.activeElement.tagName;
// don't navigate when in search box or textarea
if (activeElementType !== 'TEXTAREA' && activeElementType !== 'INPUT' && activeElementType !== 'SELECT') {
switch (event.keyCode) {
case 37: // left
var prevHref = $('link[rel="prev"]').prop('href');
if (prevHref) {
window.location.href = prevHref;
return false;
}
case 39: // right
var nextHref = $('link[rel="next"]').prop('href');
if (nextHref) {
window.location.href = nextHref;
return false;
}
}
}
});
} }
}; };
...@@ -260,4 +284,4 @@ _ = Documentation.gettext; ...@@ -260,4 +284,4 @@ _ = Documentation.gettext;
$(document).ready(function() { $(document).ready(function() {
Documentation.init(); Documentation.init();
}); });
\ No newline at end of file
doc/_static/searchtools.js
浏览文件 @ 22cb62db
...@@ -2,7 +2,7 @@ ...@@ -2,7 +2,7 @@
* searchtools.js_t * searchtools.js_t
* ~~~~~~~~~~~~~~~~ * ~~~~~~~~~~~~~~~~
* *
* Sphinx JavaScript utilties for the full-text search. * Sphinx JavaScript utilities for the full-text search.
* *
* :copyright: Copyright 2007-2016 by the Sphinx team, see AUTHORS. * :copyright: Copyright 2007-2016 by the Sphinx team, see AUTHORS.
* :license: BSD, see LICENSE for details. * :license: BSD, see LICENSE for details.
...@@ -623,7 +623,7 @@ var Search = { ...@@ -623,7 +623,7 @@ var Search = {
* helper function to return a node containing the * helper function to return a node containing the
* search summary for a given text. keywords is a list * search summary for a given text. keywords is a list
* of stemmed words, hlwords is the list of normal, unstemmed * of stemmed words, hlwords is the list of normal, unstemmed
* words. the first one is used to find the occurance, the * words. the first one is used to find the occurrence, the
* latter for highlighting it. * latter for highlighting it.
*/ */
makeSearchSummary : function(text, keywords, hlwords) { makeSearchSummary : function(text, keywords, hlwords) {
......
doc/_static/websupport.js
浏览文件 @ 22cb62db
...@@ -2,7 +2,7 @@ ...@@ -2,7 +2,7 @@
* websupport.js * websupport.js
* ~~~~~~~~~~~~~ * ~~~~~~~~~~~~~
* *
* sphinx.websupport utilties for all documentation. * sphinx.websupport utilities for all documentation.
* *
* :copyright: Copyright 2007-2016 by the Sphinx team, see AUTHORS. * :copyright: Copyright 2007-2016 by the Sphinx team, see AUTHORS.
* :license: BSD, see LICENSE for details. * :license: BSD, see LICENSE for details.
......
doc/algorithm/rnn/rnn.html
浏览文件 @ 22cb62db
...@@ -6,7 +6,7 @@ ...@@ -6,7 +6,7 @@
<head> <head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>Recurrent Neural Network Configuration &mdash; PaddlePaddle documentation</title> <title>Recurrent Neural Network Configuration &#8212; PaddlePaddle documentation</title>
<link rel="stylesheet" href="../../_static/classic.css" type="text/css" /> <link rel="stylesheet" href="../../_static/classic.css" type="text/css" />
<link rel="stylesheet" href="../../_static/pygments.css" type="text/css" /> <link rel="stylesheet" href="../../_static/pygments.css" type="text/css" />
...@@ -40,7 +40,7 @@ ...@@ -40,7 +40,7 @@
<li class="right" > <li class="right" >
<a href="../../source/utils/thread.html" title="Lock" <a href="../../source/utils/thread.html" title="Lock"
accesskey="P">previous</a> |</li> accesskey="P">previous</a> |</li>
<li class="nav-item nav-item-0"><a href="../../index.html">PaddlePaddle documentation</a> &raquo;</li> <li class="nav-item nav-item-0"><a href="../../index.html">PaddlePaddle documentation</a> &#187;</li>
</ul> </ul>
</div> </div>
...@@ -71,7 +71,7 @@ ...@@ -71,7 +71,7 @@
<div class="highlight-python"><div class="highlight"><pre><span></span><span class="k">yield</span> <span class="n">src_ids</span><span class="p">,</span> <span class="n">trg_ids</span><span class="p">,</span> <span class="n">trg_ids_next</span> <div class="highlight-python"><div class="highlight"><pre><span></span><span class="k">yield</span> <span class="n">src_ids</span><span class="p">,</span> <span class="n">trg_ids</span><span class="p">,</span> <span class="n">trg_ids_next</span>
</pre></div> </pre></div>
</div> </div>
<p>For more details description of how to write a data provider, please refer to <code class="xref doc docutils literal"><span class="pre">Python</span> <span class="pre">Data</span> <span class="pre">Provider</span></code>. The full data provider file is located at <code class="code docutils literal"><span class="pre">demo/seqToseq/dataprovider.py</span></code>.</p> <p>For more details description of how to write a data provider, please refer to <a class="reference external" href="../../ui/data_provider/index.html">PyDataProvider2</a>. The full data provider file is located at <code class="code docutils literal"><span class="pre">demo/seqToseq/dataprovider.py</span></code>.</p>
</div> </div>
<div class="section" id="configure-recurrent-neural-network-architecture"> <div class="section" id="configure-recurrent-neural-network-architecture">
<h2>Configure Recurrent Neural Network Architecture<a class="headerlink" href="#configure-recurrent-neural-network-architecture" title="Permalink to this headline"></a></h2> <h2>Configure Recurrent Neural Network Architecture<a class="headerlink" href="#configure-recurrent-neural-network-architecture" title="Permalink to this headline"></a></h2>
...@@ -121,7 +121,7 @@ Its <strong>output function</strong> simply takes <span class="math">\(x_t\)</sp ...@@ -121,7 +121,7 @@ Its <strong>output function</strong> simply takes <span class="math">\(x_t\)</sp
<p>We will use the sequence to sequence model with attention as an example to demonstrate how you can configure complex recurrent neural network models. An illustration of the sequence to sequence model with attention is shown in the following figure.</p> <p>We will use the sequence to sequence model with attention as an example to demonstrate how you can configure complex recurrent neural network models. An illustration of the sequence to sequence model with attention is shown in the following figure.</p>
<img alt="../../_images/encoder-decoder-attention-model.png" class="align-center" src="../../_images/encoder-decoder-attention-model.png" /> <img alt="../../_images/encoder-decoder-attention-model.png" class="align-center" src="../../_images/encoder-decoder-attention-model.png" />
<p>In this model, the source sequence <span class="math">\(S = \{s_1, \dots, s_T\}\)</span> is encoded with a bidirectional gated recurrent neural networks. The hidden states of the bidirectional gated recurrent neural network <span class="math">\(H_S = \{H_1, \dots, H_T\}\)</span> is called <em>encoder vector</em> The decoder is a gated recurrent neural network. When decoding each token <span class="math">\(y_t\)</span>, the gated recurrent neural network generates a set of weights <span class="math">\(W_S^t = \{W_1^t, \dots, W_T^t\}\)</span>, which are used to compute a weighted sum of the encoder vector. The weighted sum of the encoder vector is utilized to condition the generation of the token <span class="math">\(y_t\)</span>.</p> <p>In this model, the source sequence <span class="math">\(S = \{s_1, \dots, s_T\}\)</span> is encoded with a bidirectional gated recurrent neural networks. The hidden states of the bidirectional gated recurrent neural network <span class="math">\(H_S = \{H_1, \dots, H_T\}\)</span> is called <em>encoder vector</em> The decoder is a gated recurrent neural network. When decoding each token <span class="math">\(y_t\)</span>, the gated recurrent neural network generates a set of weights <span class="math">\(W_S^t = \{W_1^t, \dots, W_T^t\}\)</span>, which are used to compute a weighted sum of the encoder vector. The weighted sum of the encoder vector is utilized to condition the generation of the token <span class="math">\(y_t\)</span>.</p>
<p>The encoder part of the model is listed below. It calls <code class="code docutils literal"><span class="pre">grumemory</span></code> to represent gated recurrent neural network. It is the recommended way of using recurrent neural network if the network architecture is simple, because it is faster than <code class="code docutils literal"><span class="pre">recurrent_group</span></code>. We have implemented most of the commonly used recurrent neural network architectures, you can refer to <code class="xref doc docutils literal"><span class="pre">Layers</span></code> for more details.</p> <p>The encoder part of the model is listed below. It calls <code class="code docutils literal"><span class="pre">grumemory</span></code> to represent gated recurrent neural network. It is the recommended way of using recurrent neural network if the network architecture is simple, because it is faster than <code class="code docutils literal"><span class="pre">recurrent_group</span></code>. We have implemented most of the commonly used recurrent neural network architectures, you can refer to <a class="reference external" href="../../ui/api/trainer_config_helpers/layers_index.html">Layers</a> for more details.</p>
<p>We also project the encoder vector to <code class="code docutils literal"><span class="pre">decoder_size</span></code> dimensional space, get the first instance of the backward recurrent network, and project it to <code class="code docutils literal"><span class="pre">decoder_size</span></code> dimensional space:</p> <p>We also project the encoder vector to <code class="code docutils literal"><span class="pre">decoder_size</span></code> dimensional space, get the first instance of the backward recurrent network, and project it to <code class="code docutils literal"><span class="pre">decoder_size</span></code> dimensional space:</p>
<div class="highlight-python"><div class="highlight"><pre><span></span><span class="c1"># Define the data layer of the source sentence.</span> <div class="highlight-python"><div class="highlight"><pre><span></span><span class="c1"># Define the data layer of the source sentence.</span>
<span class="n">src_word_id</span> <span class="o">=</span> <span class="n">data_layer</span><span class="p">(</span><span class="n">name</span><span class="o">=</span><span class="s1">&#39;source_language_word&#39;</span><span class="p">,</span> <span class="n">size</span><span class="o">=</span><span class="n">source_dict_dim</span><span class="p">)</span> <span class="n">src_word_id</span> <span class="o">=</span> <span class="n">data_layer</span><span class="p">(</span><span class="n">name</span><span class="o">=</span><span class="s1">&#39;source_language_word&#39;</span><span class="p">,</span> <span class="n">size</span><span class="o">=</span><span class="n">source_dict_dim</span><span class="p">)</span>
...@@ -250,7 +250,7 @@ Its <strong>output function</strong> simply takes <span class="math">\(x_t\)</sp ...@@ -250,7 +250,7 @@ Its <strong>output function</strong> simply takes <span class="math">\(x_t\)</sp
<span class="n">outputs</span><span class="p">(</span><span class="n">beam_gen</span><span class="p">)</span> <span class="n">outputs</span><span class="p">(</span><span class="n">beam_gen</span><span class="p">)</span>
</pre></div> </pre></div>
</div> </div>
<p>Notice that this generation technique is only useful for decoder like generation process. If you are working on sequence tagging tasks, please refer to <code class="xref doc docutils literal"><span class="pre">Semantic</span> <span class="pre">Role</span> <span class="pre">Labeling</span> <span class="pre">Demo</span></code> for more details.</p> <p>Notice that this generation technique is only useful for decoder like generation process. If you are working on sequence tagging tasks, please refer to <a class="reference external" href="../../demo/semantic_role_labeling/index.html">Semantic Role Labeling Demo</a> for more details.</p>
<p>The full configuration file is located at <code class="code docutils literal"><span class="pre">demo/seqToseq/seqToseq_net.py</span></code>.</p> <p>The full configuration file is located at <code class="code docutils literal"><span class="pre">demo/seqToseq/seqToseq_net.py</span></code>.</p>
</div> </div>
</div> </div>
...@@ -288,14 +288,11 @@ Its <strong>output function</strong> simply takes <span class="math">\(x_t\)</sp ...@@ -288,14 +288,11 @@ Its <strong>output function</strong> simply takes <span class="math">\(x_t\)</sp
<div id="searchbox" style="display: none" role="search"> <div id="searchbox" style="display: none" role="search">
<h3>Quick search</h3> <h3>Quick search</h3>
<form class="search" action="../../search.html" method="get"> <form class="search" action="../../search.html" method="get">
<input type="text" name="q" /> <div><input type="text" name="q" /></div>
<input type="submit" value="Go" /> <div><input type="submit" value="Go" /></div>
<input type="hidden" name="check_keywords" value="yes" /> <input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" /> <input type="hidden" name="area" value="default" />
</form> </form>
<p class="searchtip" style="font-size: 90%">
Enter search terms or a module, class or function name.
</p>
</div> </div>
<script type="text/javascript">$('#searchbox').show(0);</script> <script type="text/javascript">$('#searchbox').show(0);</script>
</div> </div>
...@@ -314,12 +311,12 @@ Its <strong>output function</strong> simply takes <span class="math">\(x_t\)</sp ...@@ -314,12 +311,12 @@ Its <strong>output function</strong> simply takes <span class="math">\(x_t\)</sp
<li class="right" > <li class="right" >
<a href="../../source/utils/thread.html" title="Lock" <a href="../../source/utils/thread.html" title="Lock"
>previous</a> |</li> >previous</a> |</li>
<li class="nav-item nav-item-0"><a href="../../index.html">PaddlePaddle documentation</a> &raquo;</li> <li class="nav-item nav-item-0"><a href="../../index.html">PaddlePaddle documentation</a> &#187;</li>
</ul> </ul>
</div> </div>
<div class="footer" role="contentinfo"> <div class="footer" role="contentinfo">
&copy; Copyright 2016, PaddlePaddle developers. &#169; Copyright 2016, PaddlePaddle developers.
Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.3.5. Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.4.6.
</div> </div>
</body> </body>
</html> </html>
\ No newline at end of file
doc/build/build_from_source.html
浏览文件 @ 22cb62db
...@@ -6,7 +6,7 @@ ...@@ -6,7 +6,7 @@
<head> <head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>Build and Install &mdash; PaddlePaddle documentation</title> <title>Build and Install &#8212; PaddlePaddle documentation</title>
<link rel="stylesheet" href="../_static/classic.css" type="text/css" /> <link rel="stylesheet" href="../_static/classic.css" type="text/css" />
<link rel="stylesheet" href="../_static/pygments.css" type="text/css" /> <link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
...@@ -45,8 +45,8 @@ ...@@ -45,8 +45,8 @@
<li class="right" > <li class="right" >
<a href="index.html" title="Build And Install PaddlePaddle" <a href="index.html" title="Build And Install PaddlePaddle"
accesskey="P">previous</a> |</li> accesskey="P">previous</a> |</li>
<li class="nav-item nav-item-0"><a href="../index.html">PaddlePaddle documentation</a> &raquo;</li> <li class="nav-item nav-item-0"><a href="../index.html">PaddlePaddle documentation</a> &#187;</li>
<li class="nav-item nav-item-1"><a href="index.html" accesskey="U">Build And Install PaddlePaddle</a> &raquo;</li> <li class="nav-item nav-item-1"><a href="index.html" accesskey="U">Build And Install PaddlePaddle</a> &#187;</li>
</ul> </ul>
</div> </div>
...@@ -98,11 +98,12 @@ ...@@ -98,11 +98,12 @@
</ul> </ul>
<div class="highlight-bash"><div class="highlight"><pre><span></span><span class="c1"># necessary</span> <div class="highlight-bash"><div class="highlight"><pre><span></span><span class="c1"># necessary</span>
sudo apt-get update sudo apt-get update
sudo apt-get install -y g++ make cmake build-essential libatlas-base-dev python python-pip libpython-dev m4 libprotobuf-dev protobuf-compiler python-protobuf python-numpy git sudo apt-get install -y g++ make cmake build-essential libatlas-base-dev python python-pip libpython-dev m4 libprotobuf-dev protobuf-compiler python-protobuf python-numpy git
<span class="c1"># optional</span> <span class="c1"># optional</span>
sudo apt-get install libgoogle-glog-dev sudo apt-get install libgoogle-glog-dev
sudo apt-get install libgflags-dev sudo apt-get install libgflags-dev
sudo apt-get install libgtest-dev sudo apt-get install libgtest-dev
sudo pip install wheel
<span class="nb">pushd</span> /usr/src/gtest <span class="nb">pushd</span> /usr/src/gtest
cmake . cmake .
make make
...@@ -171,6 +172,8 @@ mkdir build ...@@ -171,6 +172,8 @@ mkdir build
<span class="nb">cd</span> build <span class="nb">cd</span> build
<span class="c1"># you can add build option here, such as: </span> <span class="c1"># you can add build option here, such as: </span>
cmake -DWITH_GPU<span class="o">=</span>ON -DWITH_DOC<span class="o">=</span>OFF -DCMAKE_INSTALL_PREFIX<span class="o">=</span>&lt;path to install&gt; .. cmake -DWITH_GPU<span class="o">=</span>ON -DWITH_DOC<span class="o">=</span>OFF -DCMAKE_INSTALL_PREFIX<span class="o">=</span>&lt;path to install&gt; ..
<span class="c1"># please use sudo make install, if you want</span>
<span class="c1"># to install PaddlePaddle into the system</span>
make -j <span class="sb">`</span>nproc<span class="sb">`</span> <span class="o">&amp;&amp;</span> make install make -j <span class="sb">`</span>nproc<span class="sb">`</span> <span class="o">&amp;&amp;</span> make install
<span class="c1"># PaddlePaddle installation path</span> <span class="c1"># PaddlePaddle installation path</span>
<span class="nb">export</span> <span class="nv">PATH</span><span class="o">=</span>&lt;path to install&gt;/bin:<span class="nv">$PATH</span> <span class="nb">export</span> <span class="nv">PATH</span><span class="o">=</span>&lt;path to install&gt;/bin:<span class="nv">$PATH</span>
...@@ -224,14 +227,11 @@ make -j <span class="sb">`</span>nproc<span class="sb">`</span> <span class="o"> ...@@ -224,14 +227,11 @@ make -j <span class="sb">`</span>nproc<span class="sb">`</span> <span class="o">
<div id="searchbox" style="display: none" role="search"> <div id="searchbox" style="display: none" role="search">
<h3>Quick search</h3> <h3>Quick search</h3>
<form class="search" action="../search.html" method="get"> <form class="search" action="../search.html" method="get">
<input type="text" name="q" /> <div><input type="text" name="q" /></div>
<input type="submit" value="Go" /> <div><input type="submit" value="Go" /></div>
<input type="hidden" name="check_keywords" value="yes" /> <input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" /> <input type="hidden" name="area" value="default" />
</form> </form>
<p class="searchtip" style="font-size: 90%">
Enter search terms or a module, class or function name.
</p>
</div> </div>
<script type="text/javascript">$('#searchbox').show(0);</script> <script type="text/javascript">$('#searchbox').show(0);</script>
</div> </div>
...@@ -253,13 +253,13 @@ make -j <span class="sb">`</span>nproc<span class="sb">`</span> <span class="o"> ...@@ -253,13 +253,13 @@ make -j <span class="sb">`</span>nproc<span class="sb">`</span> <span class="o">
<li class="right" > <li class="right" >
<a href="index.html" title="Build And Install PaddlePaddle" <a href="index.html" title="Build And Install PaddlePaddle"
>previous</a> |</li> >previous</a> |</li>
<li class="nav-item nav-item-0"><a href="../index.html">PaddlePaddle documentation</a> &raquo;</li> <li class="nav-item nav-item-0"><a href="../index.html">PaddlePaddle documentation</a> &#187;</li>
<li class="nav-item nav-item-1"><a href="index.html" >Build And Install PaddlePaddle</a> &raquo;</li> <li class="nav-item nav-item-1"><a href="index.html" >Build And Install PaddlePaddle</a> &#187;</li>
</ul> </ul>
</div> </div>
<div class="footer" role="contentinfo"> <div class="footer" role="contentinfo">
&copy; Copyright 2016, PaddlePaddle developers. &#169; Copyright 2016, PaddlePaddle developers.
Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.3.5. Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.4.6.
</div> </div>
</body> </body>
</html> </html>
\ No newline at end of file
doc/build/contribute_to_paddle.html
浏览文件 @ 22cb62db
...@@ -6,7 +6,7 @@ ...@@ -6,7 +6,7 @@
<head> <head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>Contribute to PaddlePaddle &mdash; PaddlePaddle documentation</title> <title>Contribute to PaddlePaddle &#8212; PaddlePaddle documentation</title>
<link rel="stylesheet" href="../_static/classic.css" type="text/css" /> <link rel="stylesheet" href="../_static/classic.css" type="text/css" />
<link rel="stylesheet" href="../_static/pygments.css" type="text/css" /> <link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
...@@ -45,8 +45,8 @@ ...@@ -45,8 +45,8 @@
<li class="right" > <li class="right" >
<a href="build_from_source.html" title="Build and Install" <a href="build_from_source.html" title="Build and Install"
accesskey="P">previous</a> |</li> accesskey="P">previous</a> |</li>
<li class="nav-item nav-item-0"><a href="../index.html">PaddlePaddle documentation</a> &raquo;</li> <li class="nav-item nav-item-0"><a href="../index.html">PaddlePaddle documentation</a> &#187;</li>
<li class="nav-item nav-item-1"><a href="index.html" accesskey="U">Build And Install PaddlePaddle</a> &raquo;</li> <li class="nav-item nav-item-1"><a href="index.html" accesskey="U">Build And Install PaddlePaddle</a> &#187;</li>
</ul> </ul>
</div> </div>
...@@ -81,7 +81,7 @@ It&#8217;s just that simple.</p> ...@@ -81,7 +81,7 @@ It&#8217;s just that simple.</p>
<p>Once you&#8217;ve created a fork, you can use your favorite git client to clone your <p>Once you&#8217;ve created a fork, you can use your favorite git client to clone your
repo or just head straight to the command line:</p> repo or just head straight to the command line:</p>
<div class="highlight-shell"><div class="highlight"><pre><span></span><span class="c1"># Clone your fork to your local machine</span> <div class="highlight-shell"><div class="highlight"><pre><span></span><span class="c1"># Clone your fork to your local machine</span>
git clone git@github.com:USERNAME/paddle.git git clone git@github.com:USERNAME/Paddle.git
</pre></div> </pre></div>
</div> </div>
<p>Then you can start to develop.</p> <p>Then you can start to develop.</p>
...@@ -106,7 +106,7 @@ To do this, you&#8217;ll need to add a remote at first:</p> ...@@ -106,7 +106,7 @@ To do this, you&#8217;ll need to add a remote at first:</p>
<div class="highlight-shell"><div class="highlight"><pre><span></span><span class="c1"># see the current configured remote repository</span> <div class="highlight-shell"><div class="highlight"><pre><span></span><span class="c1"># see the current configured remote repository</span>
git remote -v git remote -v
<span class="c1"># add upstream repository</span> <span class="c1"># add upstream repository</span>
git remote add upstream https://github.com/paddle/paddle.git git remote add upstream https://github.com/baidu/Paddle.git
<span class="c1"># verify the new upstream</span> <span class="c1"># verify the new upstream</span>
git remote -v git remote -v
</pre></div> </pre></div>
...@@ -171,14 +171,11 @@ and click the <strong>pull request button</strong>.</p> ...@@ -171,14 +171,11 @@ and click the <strong>pull request button</strong>.</p>
<div id="searchbox" style="display: none" role="search"> <div id="searchbox" style="display: none" role="search">
<h3>Quick search</h3> <h3>Quick search</h3>
<form class="search" action="../search.html" method="get"> <form class="search" action="../search.html" method="get">
<input type="text" name="q" /> <div><input type="text" name="q" /></div>
<input type="submit" value="Go" /> <div><input type="submit" value="Go" /></div>
<input type="hidden" name="check_keywords" value="yes" /> <input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" /> <input type="hidden" name="area" value="default" />
</form> </form>
<p class="searchtip" style="font-size: 90%">
Enter search terms or a module, class or function name.
</p>
</div> </div>
<script type="text/javascript">$('#searchbox').show(0);</script> <script type="text/javascript">$('#searchbox').show(0);</script>
</div> </div>
...@@ -200,13 +197,13 @@ and click the <strong>pull request button</strong>.</p> ...@@ -200,13 +197,13 @@ and click the <strong>pull request button</strong>.</p>
<li class="right" > <li class="right" >
<a href="build_from_source.html" title="Build and Install" <a href="build_from_source.html" title="Build and Install"
>previous</a> |</li> >previous</a> |</li>
<li class="nav-item nav-item-0"><a href="../index.html">PaddlePaddle documentation</a> &raquo;</li> <li class="nav-item nav-item-0"><a href="../index.html">PaddlePaddle documentation</a> &#187;</li>
<li class="nav-item nav-item-1"><a href="index.html" >Build And Install PaddlePaddle</a> &raquo;</li> <li class="nav-item nav-item-1"><a href="index.html" >Build And Install PaddlePaddle</a> &#187;</li>
</ul> </ul>
</div> </div>
<div class="footer" role="contentinfo"> <div class="footer" role="contentinfo">
&copy; Copyright 2016, PaddlePaddle developers. &#169; Copyright 2016, PaddlePaddle developers.
Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.3.5. Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.4.6.
</div> </div>
</body> </body>
</html> </html>
\ No newline at end of file
doc/build/docker_install.html
浏览文件 @ 22cb62db
...@@ -6,7 +6,7 @@ ...@@ -6,7 +6,7 @@
<head> <head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>Docker installation guide &mdash; PaddlePaddle documentation</title> <title>Docker installation guide &#8212; PaddlePaddle documentation</title>
<link rel="stylesheet" href="../_static/classic.css" type="text/css" /> <link rel="stylesheet" href="../_static/classic.css" type="text/css" />
<link rel="stylesheet" href="../_static/pygments.css" type="text/css" /> <link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
...@@ -45,8 +45,8 @@ ...@@ -45,8 +45,8 @@
<li class="right" > <li class="right" >
<a href="contribute_to_paddle.html" title="Contribute to PaddlePaddle" <a href="contribute_to_paddle.html" title="Contribute to PaddlePaddle"
accesskey="P">previous</a> |</li> accesskey="P">previous</a> |</li>
<li class="nav-item nav-item-0"><a href="../index.html">PaddlePaddle documentation</a> &raquo;</li> <li class="nav-item nav-item-0"><a href="../index.html">PaddlePaddle documentation</a> &#187;</li>
<li class="nav-item nav-item-1"><a href="index.html" accesskey="U">Build And Install PaddlePaddle</a> &raquo;</li> <li class="nav-item nav-item-1"><a href="index.html" accesskey="U">Build And Install PaddlePaddle</a> &#187;</li>
</ul> </ul>
</div> </div>
...@@ -65,12 +65,12 @@ ...@@ -65,12 +65,12 @@
<span id="paddlepaddle-docker-images"></span><h3>PaddlePaddle Docker images<a class="headerlink" href="#paddlepaddle-docker-images" title="Permalink to this headline"></a></h3> <span id="paddlepaddle-docker-images"></span><h3>PaddlePaddle Docker images<a class="headerlink" href="#paddlepaddle-docker-images" title="Permalink to this headline"></a></h3>
<p>There are six Docker images:</p> <p>There are six Docker images:</p>
<ul class="simple"> <ul class="simple">
<li>paddledev/paddle:latest-cpu: PaddlePaddle CPU binary image.</li> <li>paddledev/paddle:cpu-latest: PaddlePaddle CPU binary image.</li>
<li>paddledev/paddle:latest-gpu: PaddlePaddle GPU binary image.</li> <li>paddledev/paddle:gpu-latest: PaddlePaddle GPU binary image.</li>
<li>paddledev/paddle:latest-cpu-devel: PaddlePaddle CPU binary image plus source code.</li> <li>paddledev/paddle:cpu-devel-latest: PaddlePaddle CPU binary image plus source code.</li>
<li>paddledev/paddle:latest-gpu-devel: PaddlePaddle GPU binary image plus source code.</li> <li>paddledev/paddle:gpu-devel-latest: PaddlePaddle GPU binary image plus source code.</li>
<li>paddledev/paddle:latest-cpu-demo: PaddlePaddle CPU binary image plus source code and demo</li> <li>paddledev/paddle:cpu-demo-latest: PaddlePaddle CPU binary image plus source code and demo</li>
<li>paddledev/paddle:latest-gpu-demo: PaddlePaddle GPU binary image plus source code and demo</li> <li>paddledev/paddle:gpu-demo-latest: PaddlePaddle GPU binary image plus source code and demo</li>
</ul> </ul>
<p>Tags with latest will be replaced by a released version.</p> <p>Tags with latest will be replaced by a released version.</p>
</div> </div>
...@@ -78,13 +78,13 @@ ...@@ -78,13 +78,13 @@
<span id="download-and-run-docker-images"></span><h3>Download and Run Docker images<a class="headerlink" href="#download-and-run-docker-images" title="Permalink to this headline"></a></h3> <span id="download-and-run-docker-images"></span><h3>Download and Run Docker images<a class="headerlink" href="#download-and-run-docker-images" title="Permalink to this headline"></a></h3>
<p>You have to install Docker in your machine which has linux kernel version 3.10+ first. You can refer to the official guide https://docs.docker.com/engine/installation/ for further information.</p> <p>You have to install Docker in your machine which has linux kernel version 3.10+ first. You can refer to the official guide https://docs.docker.com/engine/installation/ for further information.</p>
<p>You can use <code class="docutils literal"><span class="pre">docker</span> <span class="pre">pull</span></code>to download images first, or just launch a container with <code class="docutils literal"><span class="pre">docker</span> <span class="pre">run</span></code>:</p> <p>You can use <code class="docutils literal"><span class="pre">docker</span> <span class="pre">pull</span></code>to download images first, or just launch a container with <code class="docutils literal"><span class="pre">docker</span> <span class="pre">run</span></code>:</p>
<div class="highlight-bash"><div class="highlight"><pre><span></span>docker run -it paddledev/paddle:lastest-cpu <div class="highlight-bash"><div class="highlight"><pre><span></span>docker run -it paddledev/paddle:cpu-latest
</pre></div> </pre></div>
</div> </div>
<p>If you want to launch container with GPU support, you need to set some environment variables at the same time:</p> <p>If you want to launch container with GPU support, you need to set some environment variables at the same time:</p>
<div class="highlight-bash"><div class="highlight"><pre><span></span>export CUDA_SO=&quot;$(\ls /usr/lib64/libcuda* | xargs -I{} echo &#39;-v {}:{}&#39;) $(\ls /usr/lib64/libnvidia* | xargs -I{} echo &#39;-v {}:{}&quot; <div class="highlight-bash"><div class="highlight"><pre><span></span>export CUDA_SO=&quot;$(\ls /usr/lib64/libcuda* | xargs -I{} echo &#39;-v {}:{}&#39;) $(\ls /usr/lib64/libnvidia* | xargs -I{} echo &#39;-v {}:{}&quot;
export DEVICES=$(\ls /dev/nvidia* | xargs -I{} echo &#39;--device {}:{}&#39;) export DEVICES=$(\ls /dev/nvidia* | xargs -I{} echo &#39;--device {}:{}&#39;)
docker run -it paddledev/paddle:latest-gpu docker run -it paddledev/paddle:gpu-latest
</pre></div> </pre></div>
</div> </div>
</div> </div>
...@@ -178,14 +178,11 @@ docker rm paddle_ssh_machine ...@@ -178,14 +178,11 @@ docker rm paddle_ssh_machine
<div id="searchbox" style="display: none" role="search"> <div id="searchbox" style="display: none" role="search">
<h3>Quick search</h3> <h3>Quick search</h3>
<form class="search" action="../search.html" method="get"> <form class="search" action="../search.html" method="get">
<input type="text" name="q" /> <div><input type="text" name="q" /></div>
<input type="submit" value="Go" /> <div><input type="submit" value="Go" /></div>
<input type="hidden" name="check_keywords" value="yes" /> <input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" /> <input type="hidden" name="area" value="default" />
</form> </form>
<p class="searchtip" style="font-size: 90%">
Enter search terms or a module, class or function name.
</p>
</div> </div>
<script type="text/javascript">$('#searchbox').show(0);</script> <script type="text/javascript">$('#searchbox').show(0);</script>
</div> </div>
...@@ -207,13 +204,13 @@ docker rm paddle_ssh_machine ...@@ -207,13 +204,13 @@ docker rm paddle_ssh_machine
<li class="right" > <li class="right" >
<a href="contribute_to_paddle.html" title="Contribute to PaddlePaddle" <a href="contribute_to_paddle.html" title="Contribute to PaddlePaddle"
>previous</a> |</li> >previous</a> |</li>
<li class="nav-item nav-item-0"><a href="../index.html">PaddlePaddle documentation</a> &raquo;</li> <li class="nav-item nav-item-0"><a href="../index.html">PaddlePaddle documentation</a> &#187;</li>
<li class="nav-item nav-item-1"><a href="index.html" >Build And Install PaddlePaddle</a> &raquo;</li> <li class="nav-item nav-item-1"><a href="index.html" >Build And Install PaddlePaddle</a> &#187;</li>
</ul> </ul>
</div> </div>
<div class="footer" role="contentinfo"> <div class="footer" role="contentinfo">
&copy; Copyright 2016, PaddlePaddle developers. &#169; Copyright 2016, PaddlePaddle developers.
Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.3.5. Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.4.6.
</div> </div>
</body> </body>
</html> </html>
\ No newline at end of file
doc/build/index.html
浏览文件 @ 22cb62db
...@@ -6,7 +6,7 @@ ...@@ -6,7 +6,7 @@
<head> <head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>Build And Install PaddlePaddle &mdash; PaddlePaddle documentation</title> <title>Build And Install PaddlePaddle &#8212; PaddlePaddle documentation</title>
<link rel="stylesheet" href="../_static/classic.css" type="text/css" /> <link rel="stylesheet" href="../_static/classic.css" type="text/css" />
<link rel="stylesheet" href="../_static/pygments.css" type="text/css" /> <link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
...@@ -44,7 +44,7 @@ ...@@ -44,7 +44,7 @@
<li class="right" > <li class="right" >
<a href="../demo/quick_start/index_en.html" title="Quick Start Tutorial" <a href="../demo/quick_start/index_en.html" title="Quick Start Tutorial"
accesskey="P">previous</a> |</li> accesskey="P">previous</a> |</li>
<li class="nav-item nav-item-0"><a href="../index.html">PaddlePaddle documentation</a> &raquo;</li> <li class="nav-item nav-item-0"><a href="../index.html">PaddlePaddle documentation</a> &#187;</li>
</ul> </ul>
</div> </div>
...@@ -58,8 +58,6 @@ ...@@ -58,8 +58,6 @@
<div class="section" id="install-paddlepaddle"> <div class="section" id="install-paddlepaddle">
<h2>Install PaddlePaddle<a class="headerlink" href="#install-paddlepaddle" title="Permalink to this headline"></a></h2> <h2>Install PaddlePaddle<a class="headerlink" href="#install-paddlepaddle" title="Permalink to this headline"></a></h2>
<div class="toctree-wrapper compound"> <div class="toctree-wrapper compound">
<ul class="simple">
</ul>
</div> </div>
</div> </div>
<div class="section" id="build-from-source"> <div class="section" id="build-from-source">
...@@ -67,29 +65,8 @@ ...@@ -67,29 +65,8 @@
<p>If you want to hack and contribute PaddlePaddle source code, following guides can help you:</p> <p>If you want to hack and contribute PaddlePaddle source code, following guides can help you:</p>
<div class="toctree-wrapper compound"> <div class="toctree-wrapper compound">
<ul> <ul>
<li class="toctree-l1"><a class="reference internal" href="build_from_source.html">Build and Install</a><ul> <li class="toctree-l1"><a class="reference internal" href="build_from_source.html">Build and Install</a></li>
<li class="toctree-l2"><a class="reference internal" href="build_from_source.html#requirement">Requirement</a><ul> <li class="toctree-l1"><a class="reference internal" href="contribute_to_paddle.html">Contribute to PaddlePaddle</a></li>
<li class="toctree-l3"><a class="reference internal" href="build_from_source.html#dependents">Dependents</a></li>
<li class="toctree-l3"><a class="reference internal" href="build_from_source.html#optional">Optional</a></li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="build_from_source.html#building-on-ubuntu14-04">Building on Ubuntu14.04</a><ul>
<li class="toctree-l3"><a class="reference internal" href="build_from_source.html#install-dependencies">Install Dependencies</a></li>
<li class="toctree-l3"><a class="reference internal" href="build_from_source.html#build-and-install">Build and Install</a></li>
</ul>
</li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="contribute_to_paddle.html">Contribute to PaddlePaddle</a><ul>
<li class="toctree-l2"><a class="reference internal" href="contribute_to_paddle.html#code-requirements">Code Requirements</a></li>
<li class="toctree-l2"><a class="reference internal" href="contribute_to_paddle.html#creating-a-fork">Creating a Fork</a></li>
<li class="toctree-l2"><a class="reference internal" href="contribute_to_paddle.html#clone">Clone</a></li>
<li class="toctree-l2"><a class="reference internal" href="contribute_to_paddle.html#commit">Commit</a></li>
<li class="toctree-l2"><a class="reference internal" href="contribute_to_paddle.html#keeping-fork-up-to-date">Keeping Fork Up to Date</a></li>
<li class="toctree-l2"><a class="reference internal" href="contribute_to_paddle.html#push-to-github">Push to GitHub</a></li>
<li class="toctree-l2"><a class="reference internal" href="contribute_to_paddle.html#pull-request">Pull Request</a></li>
</ul>
</li>
</ul> </ul>
</div> </div>
</div> </div>
...@@ -100,23 +77,8 @@ state and your experience of installation may not be smooth.</p> ...@@ -100,23 +77,8 @@ state and your experience of installation may not be smooth.</p>
<p>If you want to pack docker image, the following guide can help you:</p> <p>If you want to pack docker image, the following guide can help you:</p>
<div class="toctree-wrapper compound"> <div class="toctree-wrapper compound">
<ul> <ul>
<li class="toctree-l1"><a class="reference internal" href="docker_install.html">Docker installation guide</a><ul> <li class="toctree-l1"><a class="reference internal" href="docker_install.html">Docker installation guide</a></li>
<li class="toctree-l2"><a class="reference internal" href="docker_install.html#docker-installation">Docker installation</a><ul> <li class="toctree-l1"><a class="reference internal" href="ubuntu_install.html">Debian Package installation guide</a></li>
<li class="toctree-l3"><a class="reference internal" href="docker_install.html#paddlepaddle-docker-images">PaddlePaddle Docker images</a></li>
<li class="toctree-l3"><a class="reference internal" href="docker_install.html#download-and-run-docker-images">Download and Run Docker images</a></li>
<li class="toctree-l3"><a class="reference internal" href="docker_install.html#notice">Notice</a><ul>
<li class="toctree-l4"><a class="reference internal" href="docker_install.html#performance">Performance</a></li>
<li class="toctree-l4"><a class="reference internal" href="docker_install.html#remote-access">Remote access</a></li>
</ul>
</li>
</ul>
</li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="ubuntu_install.html">Debian Package installation guide</a><ul>
<li class="toctree-l2"><a class="reference internal" href="ubuntu_install.html#debian-package-installation">Debian Package installation</a></li>
</ul>
</li>
</ul> </ul>
</div> </div>
</div> </div>
...@@ -154,14 +116,11 @@ state and your experience of installation may not be smooth.</p> ...@@ -154,14 +116,11 @@ state and your experience of installation may not be smooth.</p>
<div id="searchbox" style="display: none" role="search"> <div id="searchbox" style="display: none" role="search">
<h3>Quick search</h3> <h3>Quick search</h3>
<form class="search" action="../search.html" method="get"> <form class="search" action="../search.html" method="get">
<input type="text" name="q" /> <div><input type="text" name="q" /></div>
<input type="submit" value="Go" /> <div><input type="submit" value="Go" /></div>
<input type="hidden" name="check_keywords" value="yes" /> <input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" /> <input type="hidden" name="area" value="default" />
</form> </form>
<p class="searchtip" style="font-size: 90%">
Enter search terms or a module, class or function name.
</p>
</div> </div>
<script type="text/javascript">$('#searchbox').show(0);</script> <script type="text/javascript">$('#searchbox').show(0);</script>
</div> </div>
...@@ -183,12 +142,12 @@ state and your experience of installation may not be smooth.</p> ...@@ -183,12 +142,12 @@ state and your experience of installation may not be smooth.</p>
<li class="right" > <li class="right" >
<a href="../demo/quick_start/index_en.html" title="Quick Start Tutorial" <a href="../demo/quick_start/index_en.html" title="Quick Start Tutorial"
>previous</a> |</li> >previous</a> |</li>
<li class="nav-item nav-item-0"><a href="../index.html">PaddlePaddle documentation</a> &raquo;</li> <li class="nav-item nav-item-0"><a href="../index.html">PaddlePaddle documentation</a> &#187;</li>
</ul> </ul>
</div> </div>
<div class="footer" role="contentinfo"> <div class="footer" role="contentinfo">
&copy; Copyright 2016, PaddlePaddle developers. &#169; Copyright 2016, PaddlePaddle developers.
Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.3.5. Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.4.6.
</div> </div>
</body> </body>
</html> </html>
\ No newline at end of file
doc/build/ubuntu_install.html
浏览文件 @ 22cb62db
...@@ -6,7 +6,7 @@ ...@@ -6,7 +6,7 @@
<head> <head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>Debian Package installation guide &mdash; PaddlePaddle documentation</title> <title>Debian Package installation guide &#8212; PaddlePaddle documentation</title>
<link rel="stylesheet" href="../_static/classic.css" type="text/css" /> <link rel="stylesheet" href="../_static/classic.css" type="text/css" />
<link rel="stylesheet" href="../_static/pygments.css" type="text/css" /> <link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
...@@ -45,8 +45,8 @@ ...@@ -45,8 +45,8 @@
<li class="right" > <li class="right" >
<a href="docker_install.html" title="Docker installation guide" <a href="docker_install.html" title="Docker installation guide"
accesskey="P">previous</a> |</li> accesskey="P">previous</a> |</li>
<li class="nav-item nav-item-0"><a href="../index.html">PaddlePaddle documentation</a> &raquo;</li> <li class="nav-item nav-item-0"><a href="../index.html">PaddlePaddle documentation</a> &#187;</li>
<li class="nav-item nav-item-1"><a href="index.html" accesskey="U">Build And Install PaddlePaddle</a> &raquo;</li> <li class="nav-item nav-item-1"><a href="index.html" accesskey="U">Build And Install PaddlePaddle</a> &#187;</li>
</ul> </ul>
</div> </div>
...@@ -103,14 +103,11 @@ apt-get install -f ...@@ -103,14 +103,11 @@ apt-get install -f
<div id="searchbox" style="display: none" role="search"> <div id="searchbox" style="display: none" role="search">
<h3>Quick search</h3> <h3>Quick search</h3>
<form class="search" action="../search.html" method="get"> <form class="search" action="../search.html" method="get">
<input type="text" name="q" /> <div><input type="text" name="q" /></div>
<input type="submit" value="Go" /> <div><input type="submit" value="Go" /></div>
<input type="hidden" name="check_keywords" value="yes" /> <input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" /> <input type="hidden" name="area" value="default" />
</form> </form>
<p class="searchtip" style="font-size: 90%">
Enter search terms or a module, class or function name.
</p>
</div> </div>
<script type="text/javascript">$('#searchbox').show(0);</script> <script type="text/javascript">$('#searchbox').show(0);</script>
</div> </div>
...@@ -132,13 +129,13 @@ apt-get install -f ...@@ -132,13 +129,13 @@ apt-get install -f
<li class="right" > <li class="right" >
<a href="docker_install.html" title="Docker installation guide" <a href="docker_install.html" title="Docker installation guide"
>previous</a> |</li> >previous</a> |</li>
<li class="nav-item nav-item-0"><a href="../index.html">PaddlePaddle documentation</a> &raquo;</li> <li class="nav-item nav-item-0"><a href="../index.html">PaddlePaddle documentation</a> &#187;</li>
<li class="nav-item nav-item-1"><a href="index.html" >Build And Install PaddlePaddle</a> &raquo;</li> <li class="nav-item nav-item-1"><a href="index.html" >Build And Install PaddlePaddle</a> &#187;</li>
</ul> </ul>
</div> </div>
<div class="footer" role="contentinfo"> <div class="footer" role="contentinfo">
&copy; Copyright 2016, PaddlePaddle developers. &#169; Copyright 2016, PaddlePaddle developers.
Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.3.5. Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.4.6.
</div> </div>
</body> </body>
</html> </html>
\ No newline at end of file
doc/cluster/index.html
浏览文件 @ 22cb62db
...@@ -6,7 +6,7 @@ ...@@ -6,7 +6,7 @@
<head> <head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>Cluster Train &mdash; PaddlePaddle documentation</title> <title>Cluster Train &#8212; PaddlePaddle documentation</title>
<link rel="stylesheet" href="../_static/classic.css" type="text/css" /> <link rel="stylesheet" href="../_static/classic.css" type="text/css" />
<link rel="stylesheet" href="../_static/pygments.css" type="text/css" /> <link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
...@@ -44,7 +44,7 @@ ...@@ -44,7 +44,7 @@
<li class="right" > <li class="right" >
<a href="../demo/embedding_model/index.html" title="Chinese Word Embedding Model Tutorial" <a href="../demo/embedding_model/index.html" title="Chinese Word Embedding Model Tutorial"
accesskey="P">previous</a> |</li> accesskey="P">previous</a> |</li>
<li class="nav-item nav-item-0"><a href="../index.html">PaddlePaddle documentation</a> &raquo;</li> <li class="nav-item nav-item-0"><a href="../index.html">PaddlePaddle documentation</a> &#187;</li>
</ul> </ul>
</div> </div>
...@@ -95,14 +95,11 @@ ...@@ -95,14 +95,11 @@
<div id="searchbox" style="display: none" role="search"> <div id="searchbox" style="display: none" role="search">
<h3>Quick search</h3> <h3>Quick search</h3>
<form class="search" action="../search.html" method="get"> <form class="search" action="../search.html" method="get">
<input type="text" name="q" /> <div><input type="text" name="q" /></div>
<input type="submit" value="Go" /> <div><input type="submit" value="Go" /></div>
<input type="hidden" name="check_keywords" value="yes" /> <input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" /> <input type="hidden" name="area" value="default" />
</form> </form>
<p class="searchtip" style="font-size: 90%">
Enter search terms or a module, class or function name.
</p>
</div> </div>
<script type="text/javascript">$('#searchbox').show(0);</script> <script type="text/javascript">$('#searchbox').show(0);</script>
</div> </div>
...@@ -124,12 +121,12 @@ ...@@ -124,12 +121,12 @@
<li class="right" > <li class="right" >
<a href="../demo/embedding_model/index.html" title="Chinese Word Embedding Model Tutorial" <a href="../demo/embedding_model/index.html" title="Chinese Word Embedding Model Tutorial"
>previous</a> |</li> >previous</a> |</li>
<li class="nav-item nav-item-0"><a href="../index.html">PaddlePaddle documentation</a> &raquo;</li> <li class="nav-item nav-item-0"><a href="../index.html">PaddlePaddle documentation</a> &#187;</li>
</ul> </ul>
</div> </div>
<div class="footer" role="contentinfo"> <div class="footer" role="contentinfo">
&copy; Copyright 2016, PaddlePaddle developers. &#169; Copyright 2016, PaddlePaddle developers.
Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.3.5. Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.4.6.
</div> </div>
</body> </body>
</html> </html>
\ No newline at end of file
doc/cluster/opensource/cluster_train.html
浏览文件 @ 22cb62db
...@@ -6,7 +6,7 @@ ...@@ -6,7 +6,7 @@
<head> <head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>Cluster Training &mdash; PaddlePaddle documentation</title> <title>Cluster Training &#8212; PaddlePaddle documentation</title>
<link rel="stylesheet" href="../../_static/classic.css" type="text/css" /> <link rel="stylesheet" href="../../_static/classic.css" type="text/css" />
<link rel="stylesheet" href="../../_static/pygments.css" type="text/css" /> <link rel="stylesheet" href="../../_static/pygments.css" type="text/css" />
...@@ -45,8 +45,8 @@ ...@@ -45,8 +45,8 @@
<li class="right" > <li class="right" >
<a href="../index.html" title="Cluster Train" <a href="../index.html" title="Cluster Train"
accesskey="P">previous</a> |</li> accesskey="P">previous</a> |</li>
<li class="nav-item nav-item-0"><a href="../../index.html">PaddlePaddle documentation</a> &raquo;</li> <li class="nav-item nav-item-0"><a href="../../index.html">PaddlePaddle documentation</a> &#187;</li>
<li class="nav-item nav-item-1"><a href="../index.html" accesskey="U">Cluster Train</a> &raquo;</li> <li class="nav-item nav-item-1"><a href="../index.html" accesskey="U">Cluster Train</a> &#187;</li>
</ul> </ul>
</div> </div>
...@@ -77,7 +77,7 @@ ...@@ -77,7 +77,7 @@
<p>Following steps are based on demo/recommendation demo in demo directory.</p> <p>Following steps are based on demo/recommendation demo in demo directory.</p>
<p>You just go through demo/recommendation tutorial doc until <code class="docutils literal"><span class="pre">Train</span></code> section, and at last you will get train/test data and model configuration file. Finaly, just use demo/recommendation as workspace for cluster training.</p> <p>You just go through demo/recommendation tutorial doc until <code class="docutils literal"><span class="pre">Train</span></code> section, and at last you will get train/test data and model configuration file. Finaly, just use demo/recommendation as workspace for cluster training.</p>
<p>At last your workspace should look like as follow:</p> <p>At last your workspace should look like as follow:</p>
<div class="highlight-python"><div class="highlight"><pre><span></span>. <div class="highlight-default"><div class="highlight"><pre><span></span>.
|-- common_utils.py |-- common_utils.py
|-- data |-- data
| |-- config.json | |-- config.json
...@@ -158,7 +158,7 @@ all files in data directory are refered by train.list/test.list which are refere ...@@ -158,7 +158,7 @@ all files in data directory are refered by train.list/test.list which are refere
<code class="docutils literal"><span class="pre">job_workspace</span></code> set it with already deployed workspace directory, <code class="docutils literal"><span class="pre">paddle.py</span></code> will skip dispatch stage to directly launch cluster job with all nodes. It could help to reduce heavy <code class="docutils literal"><span class="pre">job_workspace</span></code> set it with already deployed workspace directory, <code class="docutils literal"><span class="pre">paddle.py</span></code> will skip dispatch stage to directly launch cluster job with all nodes. It could help to reduce heavy
dispatch latency.</p> dispatch latency.</p>
<p><code class="docutils literal"><span class="pre">cluster_train/run.sh</span></code> provides command line sample to run <code class="docutils literal"><span class="pre">demo/recommendation</span></code> cluster job, just modify <code class="docutils literal"><span class="pre">job_dispatch_package</span></code> and <code class="docutils literal"><span class="pre">job_workspace</span></code> with your defined directory, then:</p> <p><code class="docutils literal"><span class="pre">cluster_train/run.sh</span></code> provides command line sample to run <code class="docutils literal"><span class="pre">demo/recommendation</span></code> cluster job, just modify <code class="docutils literal"><span class="pre">job_dispatch_package</span></code> and <code class="docutils literal"><span class="pre">job_workspace</span></code> with your defined directory, then:</p>
<div class="highlight-python"><div class="highlight"><pre><span></span>sh run.sh <div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">sh</span> <span class="n">run</span><span class="o">.</span><span class="n">sh</span>
</pre></div> </pre></div>
</div> </div>
<p>The cluster Job will start in several seconds.</p> <p>The cluster Job will start in several seconds.</p>
...@@ -225,14 +225,11 @@ It provides stderr and stdout of trainer process. Check error log if training cr ...@@ -225,14 +225,11 @@ It provides stderr and stdout of trainer process. Check error log if training cr
<div id="searchbox" style="display: none" role="search"> <div id="searchbox" style="display: none" role="search">
<h3>Quick search</h3> <h3>Quick search</h3>
<form class="search" action="../../search.html" method="get"> <form class="search" action="../../search.html" method="get">
<input type="text" name="q" /> <div><input type="text" name="q" /></div>
<input type="submit" value="Go" /> <div><input type="submit" value="Go" /></div>
<input type="hidden" name="check_keywords" value="yes" /> <input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" /> <input type="hidden" name="area" value="default" />
</form> </form>
<p class="searchtip" style="font-size: 90%">
Enter search terms or a module, class or function name.
</p>
</div> </div>
<script type="text/javascript">$('#searchbox').show(0);</script> <script type="text/javascript">$('#searchbox').show(0);</script>
</div> </div>
...@@ -254,13 +251,13 @@ It provides stderr and stdout of trainer process. Check error log if training cr ...@@ -254,13 +251,13 @@ It provides stderr and stdout of trainer process. Check error log if training cr
<li class="right" > <li class="right" >
<a href="../index.html" title="Cluster Train" <a href="../index.html" title="Cluster Train"
>previous</a> |</li> >previous</a> |</li>
<li class="nav-item nav-item-0"><a href="../../index.html">PaddlePaddle documentation</a> &raquo;</li> <li class="nav-item nav-item-0"><a href="../../index.html">PaddlePaddle documentation</a> &#187;</li>
<li class="nav-item nav-item-1"><a href="../index.html" >Cluster Train</a> &raquo;</li> <li class="nav-item nav-item-1"><a href="../index.html" >Cluster Train</a> &#187;</li>
</ul> </ul>
</div> </div>
<div class="footer" role="contentinfo"> <div class="footer" role="contentinfo">
&copy; Copyright 2016, PaddlePaddle developers. &#169; Copyright 2016, PaddlePaddle developers.
Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.3.5. Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.4.6.
</div> </div>
</body> </body>
</html> </html>
\ No newline at end of file
doc/demo/embedding_model/index.html
浏览文件 @ 22cb62db
...@@ -6,7 +6,7 @@ ...@@ -6,7 +6,7 @@
<head> <head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>Chinese Word Embedding Model Tutorial &mdash; PaddlePaddle documentation</title> <title>Chinese Word Embedding Model Tutorial &#8212; PaddlePaddle documentation</title>
<link rel="stylesheet" href="../../_static/classic.css" type="text/css" /> <link rel="stylesheet" href="../../_static/classic.css" type="text/css" />
<link rel="stylesheet" href="../../_static/pygments.css" type="text/css" /> <link rel="stylesheet" href="../../_static/pygments.css" type="text/css" />
...@@ -45,8 +45,8 @@ ...@@ -45,8 +45,8 @@
<li class="right" > <li class="right" >
<a href="../imagenet_model/resnet_model.html" title="Model Zoo - ImageNet" <a href="../imagenet_model/resnet_model.html" title="Model Zoo - ImageNet"
accesskey="P">previous</a> |</li> accesskey="P">previous</a> |</li>
<li class="nav-item nav-item-0"><a href="../../index.html">PaddlePaddle documentation</a> &raquo;</li> <li class="nav-item nav-item-0"><a href="../../index.html">PaddlePaddle documentation</a> &#187;</li>
<li class="nav-item nav-item-1"><a href="../index.html" accesskey="U">Examples and demos</a> &raquo;</li> <li class="nav-item nav-item-1"><a href="../index.html" accesskey="U">Examples and demos</a> &#187;</li>
</ul> </ul>
</div> </div>
...@@ -80,7 +80,7 @@ ...@@ -80,7 +80,7 @@
<div class="section" id="download-and-extract"> <div class="section" id="download-and-extract">
<span id="download-and-extract"></span><h3>Download and Extract<a class="headerlink" href="#download-and-extract" title="Permalink to this headline"></a></h3> <span id="download-and-extract"></span><h3>Download and Extract<a class="headerlink" href="#download-and-extract" title="Permalink to this headline"></a></h3>
<p>To download and extract our dictionary and pretrained model, run the following commands.</p> <p>To download and extract our dictionary and pretrained model, run the following commands.</p>
<div class="highlight-python"><div class="highlight"><pre><span></span>cd $PADDLE_ROOT/demo/model_zoo/embedding <div class="highlight-default"><div class="highlight"><pre><span></span>cd $PADDLE_ROOT/demo/model_zoo/embedding
./pre_DictAndModel.sh ./pre_DictAndModel.sh
</pre></div> </pre></div>
</div> </div>
...@@ -92,12 +92,12 @@ ...@@ -92,12 +92,12 @@
<div class="section" id="data-preparation-and-preprocess"> <div class="section" id="data-preparation-and-preprocess">
<span id="data-preparation-and-preprocess"></span><h3>Data Preparation and Preprocess<a class="headerlink" href="#data-preparation-and-preprocess" title="Permalink to this headline"></a></h3> <span id="data-preparation-and-preprocess"></span><h3>Data Preparation and Preprocess<a class="headerlink" href="#data-preparation-and-preprocess" title="Permalink to this headline"></a></h3>
<p>First, run the following commands to download and extract the in-house dataset. The dataset (using UTF-8 format) has 20 training samples, 5 testing samples and 2 generating samples.</p> <p>First, run the following commands to download and extract the in-house dataset. The dataset (using UTF-8 format) has 20 training samples, 5 testing samples and 2 generating samples.</p>
<div class="highlight-python"><div class="highlight"><pre><span></span>cd $PADDLE_ROOT/demo/seqToseq/data <div class="highlight-default"><div class="highlight"><pre><span></span>cd $PADDLE_ROOT/demo/seqToseq/data
./paraphrase_data.sh ./paraphrase_data.sh
</pre></div> </pre></div>
</div> </div>
<p>Second, preprocess data and build dictionary on train data by running the following commands, and the preprocessed dataset is stored in <code class="docutils literal"><span class="pre">$PADDLE_SOURCE_ROOT/demo/seqToseq/data/pre-paraphrase</span></code>:</p> <p>Second, preprocess data and build dictionary on train data by running the following commands, and the preprocessed dataset is stored in <code class="docutils literal"><span class="pre">$PADDLE_SOURCE_ROOT/demo/seqToseq/data/pre-paraphrase</span></code>:</p>
<div class="highlight-python"><div class="highlight"><pre><span></span>cd $PADDLE_ROOT/demo/seqToseq/ <div class="highlight-default"><div class="highlight"><pre><span></span>cd $PADDLE_ROOT/demo/seqToseq/
python preprocess.py -i data/paraphrase [--mergeDict] python preprocess.py -i data/paraphrase [--mergeDict]
</pre></div> </pre></div>
</div> </div>
...@@ -108,7 +108,7 @@ python preprocess.py -i data/paraphrase [--mergeDict] ...@@ -108,7 +108,7 @@ python preprocess.py -i data/paraphrase [--mergeDict]
<div class="section" id="user-specified-embedding-model"> <div class="section" id="user-specified-embedding-model">
<span id="user-specified-embedding-model"></span><h3>User Specified Embedding Model<a class="headerlink" href="#user-specified-embedding-model" title="Permalink to this headline"></a></h3> <span id="user-specified-embedding-model"></span><h3>User Specified Embedding Model<a class="headerlink" href="#user-specified-embedding-model" title="Permalink to this headline"></a></h3>
<p>The general command of extracting desired parameters from the pretrained embedding model based on user dictionary is:</p> <p>The general command of extracting desired parameters from the pretrained embedding model based on user dictionary is:</p>
<div class="highlight-python"><div class="highlight"><pre><span></span>cd $PADDLE_ROOT/demo/model_zoo/embedding <div class="highlight-default"><div class="highlight"><pre><span></span>cd $PADDLE_ROOT/demo/model_zoo/embedding
python extract_para.py --preModel PREMODEL --preDict PREDICT --usrModel USRMODEL--usrDict USRDICT -d DIM python extract_para.py --preModel PREMODEL --preDict PREDICT --usrModel USRMODEL--usrDict USRDICT -d DIM
</pre></div> </pre></div>
</div> </div>
...@@ -120,22 +120,22 @@ python extract_para.py --preModel PREMODEL --preDict PREDICT --usrModel USRMODEL ...@@ -120,22 +120,22 @@ python extract_para.py --preModel PREMODEL --preDict PREDICT --usrModel USRMODEL
<li><code class="docutils literal"><span class="pre">-d</span> <span class="pre">DIM</span></code>: dimension of parameter</li> <li><code class="docutils literal"><span class="pre">-d</span> <span class="pre">DIM</span></code>: dimension of parameter</li>
</ul> </ul>
<p>Here, you can simply run the command:</p> <p>Here, you can simply run the command:</p>
<div class="highlight-python"><div class="highlight"><pre><span></span>cd $PADDLE_ROOT/demo/seqToseq/data/ <div class="highlight-default"><div class="highlight"><pre><span></span>cd $PADDLE_ROOT/demo/seqToseq/data/
./paraphrase_model.sh ./paraphrase_model.sh
</pre></div> </pre></div>
</div> </div>
<p>And you will see following embedding model structure:</p> <p>And you will see following embedding model structure:</p>
<div class="highlight-python"><div class="highlight"><pre><span></span>paraphrase_model <div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">paraphrase_model</span>
|--- _source_language_embedding <span class="o">|---</span> <span class="n">_source_language_embedding</span>
|--- _target_language_embedding <span class="o">|---</span> <span class="n">_target_language_embedding</span>
</pre></div> </pre></div>
</div> </div>
</div> </div>
<div class="section" id="training-model-in-paddlepaddle"> <div class="section" id="training-model-in-paddlepaddle">
<span id="training-model-in-paddlepaddle"></span><h3>Training Model in PaddlePaddle<a class="headerlink" href="#training-model-in-paddlepaddle" title="Permalink to this headline"></a></h3> <span id="training-model-in-paddlepaddle"></span><h3>Training Model in PaddlePaddle<a class="headerlink" href="#training-model-in-paddlepaddle" title="Permalink to this headline"></a></h3>
<p>First, create a model config file, see example <code class="docutils literal"><span class="pre">demo/seqToseq/paraphrase/train.conf</span></code>:</p> <p>First, create a model config file, see example <code class="docutils literal"><span class="pre">demo/seqToseq/paraphrase/train.conf</span></code>:</p>
<div class="highlight-python"><div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">seqToseq_net</span> <span class="kn">import</span> <span class="o">*</span> <div class="highlight-default"><div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">seqToseq_net</span> <span class="k">import</span> <span class="o">*</span>
<span class="n">is_generating</span> <span class="o">=</span> <span class="bp">False</span> <span class="n">is_generating</span> <span class="o">=</span> <span class="kc">False</span>
<span class="c1">################## Data Definition #####################</span> <span class="c1">################## Data Definition #####################</span>
<span class="n">train_conf</span> <span class="o">=</span> <span class="n">seq_to_seq_data</span><span class="p">(</span><span class="n">data_dir</span> <span class="o">=</span> <span class="s2">&quot;./data/pre-paraphrase&quot;</span><span class="p">,</span> <span class="n">train_conf</span> <span class="o">=</span> <span class="n">seq_to_seq_data</span><span class="p">(</span><span class="n">data_dir</span> <span class="o">=</span> <span class="s2">&quot;./data/pre-paraphrase&quot;</span><span class="p">,</span>
...@@ -145,7 +145,7 @@ python extract_para.py --preModel PREMODEL --preDict PREDICT --usrModel USRMODEL ...@@ -145,7 +145,7 @@ python extract_para.py --preModel PREMODEL --preDict PREDICT --usrModel USRMODEL
<span class="n">settings</span><span class="p">(</span> <span class="n">settings</span><span class="p">(</span>
<span class="n">learning_method</span> <span class="o">=</span> <span class="n">AdamOptimizer</span><span class="p">(),</span> <span class="n">learning_method</span> <span class="o">=</span> <span class="n">AdamOptimizer</span><span class="p">(),</span>
<span class="n">batch_size</span> <span class="o">=</span> <span class="mi">50</span><span class="p">,</span> <span class="n">batch_size</span> <span class="o">=</span> <span class="mi">50</span><span class="p">,</span>
<span class="n">learning_rate</span> <span class="o">=</span> <span class="mf">5e-4</span><span class="p">)</span> <span class="n">learning_rate</span> <span class="o">=</span> <span class="mi">5</span><span class="n">e</span><span class="o">-</span><span class="mi">4</span><span class="p">)</span>
<span class="c1">################# Network configure #####################</span> <span class="c1">################# Network configure #####################</span>
<span class="n">gru_encoder_decoder</span><span class="p">(</span><span class="n">train_conf</span><span class="p">,</span> <span class="n">is_generating</span><span class="p">,</span> <span class="n">word_vector_dim</span> <span class="o">=</span> <span class="mi">32</span><span class="p">)</span> <span class="n">gru_encoder_decoder</span><span class="p">(</span><span class="n">train_conf</span><span class="p">,</span> <span class="n">is_generating</span><span class="p">,</span> <span class="n">word_vector_dim</span> <span class="o">=</span> <span class="mi">32</span><span class="p">)</span>
...@@ -153,7 +153,7 @@ python extract_para.py --preModel PREMODEL --preDict PREDICT --usrModel USRMODEL ...@@ -153,7 +153,7 @@ python extract_para.py --preModel PREMODEL --preDict PREDICT --usrModel USRMODEL
</div> </div>
<p>This config is almost the same as <code class="docutils literal"><span class="pre">demo/seqToseq/translation/train.conf</span></code>.</p> <p>This config is almost the same as <code class="docutils literal"><span class="pre">demo/seqToseq/translation/train.conf</span></code>.</p>
<p>Then, train the model by running the command:</p> <p>Then, train the model by running the command:</p>
<div class="highlight-python"><div class="highlight"><pre><span></span>cd $PADDLE_SOURCE_ROOT/demo/seqToseq/paraphrase <div class="highlight-default"><div class="highlight"><pre><span></span>cd $PADDLE_SOURCE_ROOT/demo/seqToseq/paraphrase
./train.sh ./train.sh
</pre></div> </pre></div>
</div> </div>
...@@ -162,7 +162,7 @@ python extract_para.py --preModel PREMODEL --preDict PREDICT --usrModel USRMODEL ...@@ -162,7 +162,7 @@ python extract_para.py --preModel PREMODEL --preDict PREDICT --usrModel USRMODEL
<li><code class="docutils literal"><span class="pre">--init_model_path</span></code>: path of the initialization model, here is <code class="docutils literal"><span class="pre">data/paraphrase_model</span></code></li> <li><code class="docutils literal"><span class="pre">--init_model_path</span></code>: path of the initialization model, here is <code class="docutils literal"><span class="pre">data/paraphrase_model</span></code></li>
<li><code class="docutils literal"><span class="pre">--load_missing_parameter_strategy</span></code>: operations when model file is missing, here use a normal distibution to initialize the other parameters except for the embedding layer</li> <li><code class="docutils literal"><span class="pre">--load_missing_parameter_strategy</span></code>: operations when model file is missing, here use a normal distibution to initialize the other parameters except for the embedding layer</li>
</ul> </ul>
<p>For users who want to understand the dataset format, model architecture and training procedure in detail, please refer to <a class="reference external" href="demo/embedding_model/text_generation.md">Text generation Tutorial</a>.</p> <p>For users who want to understand the dataset format, model architecture and training procedure in detail, please refer to <a class="reference internal" href="../text_generation/text_generation.html"><span class="doc">Text generation Tutorial</span></a>.</p>
</div> </div>
</div> </div>
<div class="section" id="optional-function"> <div class="section" id="optional-function">
...@@ -170,7 +170,7 @@ python extract_para.py --preModel PREMODEL --preDict PREDICT --usrModel USRMODEL ...@@ -170,7 +170,7 @@ python extract_para.py --preModel PREMODEL --preDict PREDICT --usrModel USRMODEL
<div class="section" id="embedding-parameters-observation"> <div class="section" id="embedding-parameters-observation">
<span id="embedding-parameters-observation"></span><h3>Embedding Parameters Observation<a class="headerlink" href="#embedding-parameters-observation" title="Permalink to this headline"></a></h3> <span id="embedding-parameters-observation"></span><h3>Embedding Parameters Observation<a class="headerlink" href="#embedding-parameters-observation" title="Permalink to this headline"></a></h3>
<p>For users who want to observe the embedding parameters, this function can convert a PaddlePaddle binary embedding model to a text model by running the command:</p> <p>For users who want to observe the embedding parameters, this function can convert a PaddlePaddle binary embedding model to a text model by running the command:</p>
<div class="highlight-python"><div class="highlight"><pre><span></span>cd $PADDLE_ROOT/demo/model_zoo/embedding <div class="highlight-default"><div class="highlight"><pre><span></span>cd $PADDLE_ROOT/demo/model_zoo/embedding
python paraconvert.py --b2t -i INPUT -o OUTPUT -d DIM python paraconvert.py --b2t -i INPUT -o OUTPUT -d DIM
</pre></div> </pre></div>
</div> </div>
...@@ -180,7 +180,7 @@ python paraconvert.py --b2t -i INPUT -o OUTPUT -d DIM ...@@ -180,7 +180,7 @@ python paraconvert.py --b2t -i INPUT -o OUTPUT -d DIM
<li><code class="docutils literal"><span class="pre">-d</span> <span class="pre">DIM</span></code>: the dimension of parameter</li> <li><code class="docutils literal"><span class="pre">-d</span> <span class="pre">DIM</span></code>: the dimension of parameter</li>
</ul> </ul>
<p>You will see parameters like this in output text model:</p> <p>You will see parameters like this in output text model:</p>
<div class="highlight-python"><div class="highlight"><pre><span></span><span class="mi">0</span><span class="p">,</span><span class="mi">4</span><span class="p">,</span><span class="mi">32156096</span> <div class="highlight-default"><div class="highlight"><pre><span></span><span class="mi">0</span><span class="p">,</span><span class="mi">4</span><span class="p">,</span><span class="mi">32156096</span>
<span class="o">-</span><span class="mf">0.7845433</span><span class="p">,</span><span class="mf">1.1937413</span><span class="p">,</span><span class="o">-</span><span class="mf">0.1704215</span><span class="p">,</span><span class="mf">0.4154715</span><span class="p">,</span><span class="mf">0.9566584</span><span class="p">,</span><span class="o">-</span><span class="mf">0.5558153</span><span class="p">,</span><span class="o">-</span><span class="mf">0.2503305</span><span class="p">,</span> <span class="o">......</span> <span class="o">-</span><span class="mf">0.7845433</span><span class="p">,</span><span class="mf">1.1937413</span><span class="p">,</span><span class="o">-</span><span class="mf">0.1704215</span><span class="p">,</span><span class="mf">0.4154715</span><span class="p">,</span><span class="mf">0.9566584</span><span class="p">,</span><span class="o">-</span><span class="mf">0.5558153</span><span class="p">,</span><span class="o">-</span><span class="mf">0.2503305</span><span class="p">,</span> <span class="o">......</span>
<span class="mf">0.0000909</span><span class="p">,</span><span class="mf">0.0009465</span><span class="p">,</span><span class="o">-</span><span class="mf">0.0008813</span><span class="p">,</span><span class="o">-</span><span class="mf">0.0008428</span><span class="p">,</span><span class="mf">0.0007879</span><span class="p">,</span><span class="mf">0.0000183</span><span class="p">,</span><span class="mf">0.0001984</span><span class="p">,</span> <span class="o">......</span> <span class="mf">0.0000909</span><span class="p">,</span><span class="mf">0.0009465</span><span class="p">,</span><span class="o">-</span><span class="mf">0.0008813</span><span class="p">,</span><span class="o">-</span><span class="mf">0.0008428</span><span class="p">,</span><span class="mf">0.0007879</span><span class="p">,</span><span class="mf">0.0000183</span><span class="p">,</span><span class="mf">0.0001984</span><span class="p">,</span> <span class="o">......</span>
<span class="o">......</span> <span class="o">......</span>
...@@ -203,7 +203,7 @@ python paraconvert.py --b2t -i INPUT -o OUTPUT -d DIM ...@@ -203,7 +203,7 @@ python paraconvert.py --b2t -i INPUT -o OUTPUT -d DIM
<div class="section" id="embedding-parameters-revision"> <div class="section" id="embedding-parameters-revision">
<span id="embedding-parameters-revision"></span><h3>Embedding Parameters Revision<a class="headerlink" href="#embedding-parameters-revision" title="Permalink to this headline"></a></h3> <span id="embedding-parameters-revision"></span><h3>Embedding Parameters Revision<a class="headerlink" href="#embedding-parameters-revision" title="Permalink to this headline"></a></h3>
<p>For users who want to revise the embedding parameters, this function can convert a revised text embedding model to a PaddlePaddle binary model by running the command:</p> <p>For users who want to revise the embedding parameters, this function can convert a revised text embedding model to a PaddlePaddle binary model by running the command:</p>
<div class="highlight-python"><div class="highlight"><pre><span></span>cd $PADDLE_ROOT/demo/model_zoo/embedding <div class="highlight-default"><div class="highlight"><pre><span></span>cd $PADDLE_ROOT/demo/model_zoo/embedding
python paraconvert.py --t2b -i INPUT -o OUTPUT python paraconvert.py --t2b -i INPUT -o OUTPUT
</pre></div> </pre></div>
</div> </div>
...@@ -212,7 +212,7 @@ python paraconvert.py --t2b -i INPUT -o OUTPUT ...@@ -212,7 +212,7 @@ python paraconvert.py --t2b -i INPUT -o OUTPUT
<li><code class="docutils literal"><span class="pre">-o</span> <span class="pre">OUTPUT</span></code>: the name of output binary embedding model</li> <li><code class="docutils literal"><span class="pre">-o</span> <span class="pre">OUTPUT</span></code>: the name of output binary embedding model</li>
</ul> </ul>
<p>Note that the format of input text model is as follows:</p> <p>Note that the format of input text model is as follows:</p>
<div class="highlight-python"><div class="highlight"><pre><span></span><span class="o">-</span><span class="mf">0.7845433</span><span class="p">,</span><span class="mf">1.1937413</span><span class="p">,</span><span class="o">-</span><span class="mf">0.1704215</span><span class="p">,</span><span class="mf">0.4154715</span><span class="p">,</span><span class="mf">0.9566584</span><span class="p">,</span><span class="o">-</span><span class="mf">0.5558153</span><span class="p">,</span><span class="o">-</span><span class="mf">0.2503305</span><span class="p">,</span> <span class="o">......</span> <div class="highlight-default"><div class="highlight"><pre><span></span><span class="o">-</span><span class="mf">0.7845433</span><span class="p">,</span><span class="mf">1.1937413</span><span class="p">,</span><span class="o">-</span><span class="mf">0.1704215</span><span class="p">,</span><span class="mf">0.4154715</span><span class="p">,</span><span class="mf">0.9566584</span><span class="p">,</span><span class="o">-</span><span class="mf">0.5558153</span><span class="p">,</span><span class="o">-</span><span class="mf">0.2503305</span><span class="p">,</span> <span class="o">......</span>
<span class="mf">0.0000909</span><span class="p">,</span><span class="mf">0.0009465</span><span class="p">,</span><span class="o">-</span><span class="mf">0.0008813</span><span class="p">,</span><span class="o">-</span><span class="mf">0.0008428</span><span class="p">,</span><span class="mf">0.0007879</span><span class="p">,</span><span class="mf">0.0000183</span><span class="p">,</span><span class="mf">0.0001984</span><span class="p">,</span> <span class="o">......</span> <span class="mf">0.0000909</span><span class="p">,</span><span class="mf">0.0009465</span><span class="p">,</span><span class="o">-</span><span class="mf">0.0008813</span><span class="p">,</span><span class="o">-</span><span class="mf">0.0008428</span><span class="p">,</span><span class="mf">0.0007879</span><span class="p">,</span><span class="mf">0.0000183</span><span class="p">,</span><span class="mf">0.0001984</span><span class="p">,</span> <span class="o">......</span>
<span class="o">......</span> <span class="o">......</span>
</pre></div> </pre></div>
...@@ -271,14 +271,11 @@ python paraconvert.py --t2b -i INPUT -o OUTPUT ...@@ -271,14 +271,11 @@ python paraconvert.py --t2b -i INPUT -o OUTPUT
<div id="searchbox" style="display: none" role="search"> <div id="searchbox" style="display: none" role="search">
<h3>Quick search</h3> <h3>Quick search</h3>
<form class="search" action="../../search.html" method="get"> <form class="search" action="../../search.html" method="get">
<input type="text" name="q" /> <div><input type="text" name="q" /></div>
<input type="submit" value="Go" /> <div><input type="submit" value="Go" /></div>
<input type="hidden" name="check_keywords" value="yes" /> <input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" /> <input type="hidden" name="area" value="default" />
</form> </form>
<p class="searchtip" style="font-size: 90%">
Enter search terms or a module, class or function name.
</p>
</div> </div>
<script type="text/javascript">$('#searchbox').show(0);</script> <script type="text/javascript">$('#searchbox').show(0);</script>
</div> </div>
...@@ -300,13 +297,13 @@ python paraconvert.py --t2b -i INPUT -o OUTPUT ...@@ -300,13 +297,13 @@ python paraconvert.py --t2b -i INPUT -o OUTPUT
<li class="right" > <li class="right" >
<a href="../imagenet_model/resnet_model.html" title="Model Zoo - ImageNet" <a href="../imagenet_model/resnet_model.html" title="Model Zoo - ImageNet"
>previous</a> |</li> >previous</a> |</li>
<li class="nav-item nav-item-0"><a href="../../index.html">PaddlePaddle documentation</a> &raquo;</li> <li class="nav-item nav-item-0"><a href="../../index.html">PaddlePaddle documentation</a> &#187;</li>
<li class="nav-item nav-item-1"><a href="../index.html" >Examples and demos</a> &raquo;</li> <li class="nav-item nav-item-1"><a href="../index.html" >Examples and demos</a> &#187;</li>
</ul> </ul>
</div> </div>
<div class="footer" role="contentinfo"> <div class="footer" role="contentinfo">
&copy; Copyright 2016, PaddlePaddle developers. &#169; Copyright 2016, PaddlePaddle developers.
Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.3.5. Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.4.6.
</div> </div>
</body> </body>
</html> </html>
\ No newline at end of file
doc/demo/image_classification/image_classification.html
浏览文件 @ 22cb62db
...@@ -6,7 +6,7 @@ ...@@ -6,7 +6,7 @@
<head> <head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>Image Classification Tutorial &mdash; PaddlePaddle documentation</title> <title>Image Classification Tutorial &#8212; PaddlePaddle documentation</title>
<link rel="stylesheet" href="../../_static/classic.css" type="text/css" /> <link rel="stylesheet" href="../../_static/classic.css" type="text/css" />
<link rel="stylesheet" href="../../_static/pygments.css" type="text/css" /> <link rel="stylesheet" href="../../_static/pygments.css" type="text/css" />
...@@ -45,9 +45,9 @@ ...@@ -45,9 +45,9 @@
<li class="right" > <li class="right" >
<a href="index.html" title="Image Classification Tutorial" <a href="index.html" title="Image Classification Tutorial"
accesskey="P">previous</a> |</li> accesskey="P">previous</a> |</li>
<li class="nav-item nav-item-0"><a href="../../index.html">PaddlePaddle documentation</a> &raquo;</li> <li class="nav-item nav-item-0"><a href="../../index.html">PaddlePaddle documentation</a> &#187;</li>
<li class="nav-item nav-item-1"><a href="../index.html" >Examples and demos</a> &raquo;</li> <li class="nav-item nav-item-1"><a href="../index.html" >Examples and demos</a> &#187;</li>
<li class="nav-item nav-item-2"><a href="index.html" accesskey="U">Image Classification Tutorial</a> &raquo;</li> <li class="nav-item nav-item-2"><a href="index.html" accesskey="U">Image Classification Tutorial</a> &#187;</li>
</ul> </ul>
</div> </div>
...@@ -86,7 +86,7 @@ sh download_cifar.sh ...@@ -86,7 +86,7 @@ sh download_cifar.sh
<p>Here are the classes in the dataset, as well as 10 random images from each: <p>Here are the classes in the dataset, as well as 10 random images from each:
<center><img alt="Image Classification" src="../../_images/cifar.png" /></center></p> <center><img alt="Image Classification" src="../../_images/cifar.png" /></center></p>
<p>After downloading and converting, we should find a directory (cifar-out) containing the dataset in the following format:</p> <p>After downloading and converting, we should find a directory (cifar-out) containing the dataset in the following format:</p>
<div class="highlight-python"><div class="highlight"><pre><span></span><span class="n">train</span> <div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">train</span>
<span class="o">---</span><span class="n">airplane</span> <span class="o">---</span><span class="n">airplane</span>
<span class="o">---</span><span class="n">automobile</span> <span class="o">---</span><span class="n">automobile</span>
<span class="o">---</span><span class="n">bird</span> <span class="o">---</span><span class="n">bird</span>
...@@ -115,8 +115,8 @@ sh download_cifar.sh ...@@ -115,8 +115,8 @@ sh download_cifar.sh
<div class="section" id="preprocess"> <div class="section" id="preprocess">
<span id="preprocess"></span><h2>Preprocess<a class="headerlink" href="#preprocess" title="Permalink to this headline"></a></h2> <span id="preprocess"></span><h2>Preprocess<a class="headerlink" href="#preprocess" title="Permalink to this headline"></a></h2>
<p>After the data has been downloaded, it needs to be pre-processed into the Paddle format. We can run the following command for preprocessing.</p> <p>After the data has been downloaded, it needs to be pre-processed into the Paddle format. We can run the following command for preprocessing.</p>
<div class="highlight-python"><div class="highlight"><pre><span></span>cd demo/image_classification/ <div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">cd</span> <span class="n">demo</span><span class="o">/</span><span class="n">image_classification</span><span class="o">/</span>
sh preprocess.sh <span class="n">sh</span> <span class="n">preprocess</span><span class="o">.</span><span class="n">sh</span>
</pre></div> </pre></div>
</div> </div>
<p><code class="docutils literal"><span class="pre">preprocess.sh</span></code> calls <code class="docutils literal"><span class="pre">./demo/image_classification/preprocess.py</span></code> to preprocess image data.</p> <p><code class="docutils literal"><span class="pre">preprocess.sh</span></code> calls <code class="docutils literal"><span class="pre">./demo/image_classification/preprocess.py</span></code> to preprocess image data.</p>
...@@ -220,11 +220,11 @@ python -m paddle.utils.plotcurve -i <span class="nv">$log</span> &gt; plot.png ...@@ -220,11 +220,11 @@ python -m paddle.utils.plotcurve -i <span class="nv">$log</span> &gt; plot.png
<span id="prediction"></span><h2>Prediction<a class="headerlink" href="#prediction" title="Permalink to this headline"></a></h2> <span id="prediction"></span><h2>Prediction<a class="headerlink" href="#prediction" title="Permalink to this headline"></a></h2>
<p>After we train the model, the model file as well as the model parameters are stored in path <code class="docutils literal"><span class="pre">./cifar_vgg_model/pass-%05d</span></code>. For example, the model of the 300-th pass is stored at <code class="docutils literal"><span class="pre">./cifar_vgg_model/pass-00299</span></code>.</p> <p>After we train the model, the model file as well as the model parameters are stored in path <code class="docutils literal"><span class="pre">./cifar_vgg_model/pass-%05d</span></code>. For example, the model of the 300-th pass is stored at <code class="docutils literal"><span class="pre">./cifar_vgg_model/pass-00299</span></code>.</p>
<p>To make a prediction for an image, one can run <code class="docutils literal"><span class="pre">predict.sh</span></code> as follows. The script will output the label of the classfiication.</p> <p>To make a prediction for an image, one can run <code class="docutils literal"><span class="pre">predict.sh</span></code> as follows. The script will output the label of the classfiication.</p>
<div class="highlight-python"><div class="highlight"><pre><span></span>sh predict.sh <div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">sh</span> <span class="n">predict</span><span class="o">.</span><span class="n">sh</span>
</pre></div> </pre></div>
</div> </div>
<p>predict.sh:</p> <p>predict.sh:</p>
<div class="highlight-python"><div class="highlight"><pre><span></span>model=cifar_vgg_model/pass-00299/ <div class="highlight-default"><div class="highlight"><pre><span></span>model=cifar_vgg_model/pass-00299/
image=data/cifar-out/test/airplane/seaplane_s_000978.png image=data/cifar-out/test/airplane/seaplane_s_000978.png
use_gpu=1 use_gpu=1
python prediction.py $model $image $use_gpu python prediction.py $model $image $use_gpu
...@@ -292,14 +292,11 @@ python prediction.py $model $image $use_gpu ...@@ -292,14 +292,11 @@ python prediction.py $model $image $use_gpu
<div id="searchbox" style="display: none" role="search"> <div id="searchbox" style="display: none" role="search">
<h3>Quick search</h3> <h3>Quick search</h3>
<form class="search" action="../../search.html" method="get"> <form class="search" action="../../search.html" method="get">
<input type="text" name="q" /> <div><input type="text" name="q" /></div>
<input type="submit" value="Go" /> <div><input type="submit" value="Go" /></div>
<input type="hidden" name="check_keywords" value="yes" /> <input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" /> <input type="hidden" name="area" value="default" />
</form> </form>
<p class="searchtip" style="font-size: 90%">
Enter search terms or a module, class or function name.
</p>
</div> </div>
<script type="text/javascript">$('#searchbox').show(0);</script> <script type="text/javascript">$('#searchbox').show(0);</script>
</div> </div>
...@@ -321,14 +318,14 @@ python prediction.py $model $image $use_gpu ...@@ -321,14 +318,14 @@ python prediction.py $model $image $use_gpu
<li class="right" > <li class="right" >
<a href="index.html" title="Image Classification Tutorial" <a href="index.html" title="Image Classification Tutorial"
>previous</a> |</li> >previous</a> |</li>
<li class="nav-item nav-item-0"><a href="../../index.html">PaddlePaddle documentation</a> &raquo;</li> <li class="nav-item nav-item-0"><a href="../../index.html">PaddlePaddle documentation</a> &#187;</li>
<li class="nav-item nav-item-1"><a href="../index.html" >Examples and demos</a> &raquo;</li> <li class="nav-item nav-item-1"><a href="../index.html" >Examples and demos</a> &#187;</li>
<li class="nav-item nav-item-2"><a href="index.html" >Image Classification Tutorial</a> &raquo;</li> <li class="nav-item nav-item-2"><a href="index.html" >Image Classification Tutorial</a> &#187;</li>
</ul> </ul>
</div> </div>
<div class="footer" role="contentinfo"> <div class="footer" role="contentinfo">
&copy; Copyright 2016, PaddlePaddle developers. &#169; Copyright 2016, PaddlePaddle developers.
Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.3.5. Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.4.6.
</div> </div>
</body> </body>
</html> </html>
\ No newline at end of file
doc/demo/image_classification/index.html
浏览文件 @ 22cb62db
...@@ -6,7 +6,7 @@ ...@@ -6,7 +6,7 @@
<head> <head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>Image Classification Tutorial &mdash; PaddlePaddle documentation</title> <title>Image Classification Tutorial &#8212; PaddlePaddle documentation</title>
<link rel="stylesheet" href="../../_static/classic.css" type="text/css" /> <link rel="stylesheet" href="../../_static/classic.css" type="text/css" />
<link rel="stylesheet" href="../../_static/pygments.css" type="text/css" /> <link rel="stylesheet" href="../../_static/pygments.css" type="text/css" />
...@@ -45,8 +45,8 @@ ...@@ -45,8 +45,8 @@
<li class="right" > <li class="right" >
<a href="../index.html" title="Examples and demos" <a href="../index.html" title="Examples and demos"
accesskey="P">previous</a> |</li> accesskey="P">previous</a> |</li>
<li class="nav-item nav-item-0"><a href="../../index.html">PaddlePaddle documentation</a> &raquo;</li> <li class="nav-item nav-item-0"><a href="../../index.html">PaddlePaddle documentation</a> &#187;</li>
<li class="nav-item nav-item-1"><a href="../index.html" accesskey="U">Examples and demos</a> &raquo;</li> <li class="nav-item nav-item-1"><a href="../index.html" accesskey="U">Examples and demos</a> &#187;</li>
</ul> </ul>
</div> </div>
...@@ -97,14 +97,11 @@ ...@@ -97,14 +97,11 @@
<div id="searchbox" style="display: none" role="search"> <div id="searchbox" style="display: none" role="search">
<h3>Quick search</h3> <h3>Quick search</h3>
<form class="search" action="../../search.html" method="get"> <form class="search" action="../../search.html" method="get">
<input type="text" name="q" /> <div><input type="text" name="q" /></div>
<input type="submit" value="Go" /> <div><input type="submit" value="Go" /></div>
<input type="hidden" name="check_keywords" value="yes" /> <input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" /> <input type="hidden" name="area" value="default" />
</form> </form>
<p class="searchtip" style="font-size: 90%">
Enter search terms or a module, class or function name.
</p>
</div> </div>
<script type="text/javascript">$('#searchbox').show(0);</script> <script type="text/javascript">$('#searchbox').show(0);</script>
</div> </div>
...@@ -126,13 +123,13 @@ ...@@ -126,13 +123,13 @@
<li class="right" > <li class="right" >
<a href="../index.html" title="Examples and demos" <a href="../index.html" title="Examples and demos"
>previous</a> |</li> >previous</a> |</li>
<li class="nav-item nav-item-0"><a href="../../index.html">PaddlePaddle documentation</a> &raquo;</li> <li class="nav-item nav-item-0"><a href="../../index.html">PaddlePaddle documentation</a> &#187;</li>
<li class="nav-item nav-item-1"><a href="../index.html" >Examples and demos</a> &raquo;</li> <li class="nav-item nav-item-1"><a href="../index.html" >Examples and demos</a> &#187;</li>
</ul> </ul>
</div> </div>
<div class="footer" role="contentinfo"> <div class="footer" role="contentinfo">
&copy; Copyright 2016, PaddlePaddle developers. &#169; Copyright 2016, PaddlePaddle developers.
Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.3.5. Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.4.6.
</div> </div>
</body> </body>
</html> </html>
\ No newline at end of file
doc/demo/index.html
浏览文件 @ 22cb62db
...@@ -6,7 +6,7 @@ ...@@ -6,7 +6,7 @@
<head> <head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>Examples and demos &mdash; PaddlePaddle documentation</title> <title>Examples and demos &#8212; PaddlePaddle documentation</title>
<link rel="stylesheet" href="../_static/classic.css" type="text/css" /> <link rel="stylesheet" href="../_static/classic.css" type="text/css" />
<link rel="stylesheet" href="../_static/pygments.css" type="text/css" /> <link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
...@@ -44,7 +44,7 @@ ...@@ -44,7 +44,7 @@
<li class="right" > <li class="right" >
<a href="../ui/predict/swig_py_paddle_en.html" title="Python Prediction API" <a href="../ui/predict/swig_py_paddle_en.html" title="Python Prediction API"
accesskey="P">previous</a> |</li> accesskey="P">previous</a> |</li>
<li class="nav-item nav-item-0"><a href="../index.html">PaddlePaddle documentation</a> &raquo;</li> <li class="nav-item nav-item-0"><a href="../index.html">PaddlePaddle documentation</a> &#187;</li>
</ul> </ul>
</div> </div>
...@@ -70,6 +70,7 @@ ...@@ -70,6 +70,7 @@
<ul> <ul>
<li class="toctree-l1"><a class="reference internal" href="sentiment_analysis/index.html">Sentiment Analysis</a></li> <li class="toctree-l1"><a class="reference internal" href="sentiment_analysis/index.html">Sentiment Analysis</a></li>
<li class="toctree-l1"><a class="reference internal" href="text_generation/index.html">Text Generation</a></li> <li class="toctree-l1"><a class="reference internal" href="text_generation/index.html">Text Generation</a></li>
<li class="toctree-l1"><a class="reference internal" href="semantic_role_labeling/index.html">Semantic Role Labeling</a></li>
</ul> </ul>
</div> </div>
</div> </div>
...@@ -126,14 +127,11 @@ ...@@ -126,14 +127,11 @@
<div id="searchbox" style="display: none" role="search"> <div id="searchbox" style="display: none" role="search">
<h3>Quick search</h3> <h3>Quick search</h3>
<form class="search" action="../search.html" method="get"> <form class="search" action="../search.html" method="get">
<input type="text" name="q" /> <div><input type="text" name="q" /></div>
<input type="submit" value="Go" /> <div><input type="submit" value="Go" /></div>
<input type="hidden" name="check_keywords" value="yes" /> <input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" /> <input type="hidden" name="area" value="default" />
</form> </form>
<p class="searchtip" style="font-size: 90%">
Enter search terms or a module, class or function name.
</p>
</div> </div>
<script type="text/javascript">$('#searchbox').show(0);</script> <script type="text/javascript">$('#searchbox').show(0);</script>
</div> </div>
...@@ -155,12 +153,12 @@ ...@@ -155,12 +153,12 @@
<li class="right" > <li class="right" >
<a href="../ui/predict/swig_py_paddle_en.html" title="Python Prediction API" <a href="../ui/predict/swig_py_paddle_en.html" title="Python Prediction API"
>previous</a> |</li> >previous</a> |</li>
<li class="nav-item nav-item-0"><a href="../index.html">PaddlePaddle documentation</a> &raquo;</li> <li class="nav-item nav-item-0"><a href="../index.html">PaddlePaddle documentation</a> &#187;</li>
</ul> </ul>
</div> </div>
<div class="footer" role="contentinfo"> <div class="footer" role="contentinfo">
&copy; Copyright 2016, PaddlePaddle developers. &#169; Copyright 2016, PaddlePaddle developers.
Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.3.5. Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.4.6.
</div> </div>
</body> </body>
</html> </html>
\ No newline at end of file
doc/demo/quick_start/index_en.html
浏览文件 @ 22cb62db
...@@ -6,7 +6,7 @@ ...@@ -6,7 +6,7 @@
<head> <head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>Quick Start Tutorial &mdash; PaddlePaddle documentation</title> <title>Quick Start Tutorial &#8212; PaddlePaddle documentation</title>
<link rel="stylesheet" href="../../_static/classic.css" type="text/css" /> <link rel="stylesheet" href="../../_static/classic.css" type="text/css" />
<link rel="stylesheet" href="../../_static/pygments.css" type="text/css" /> <link rel="stylesheet" href="../../_static/pygments.css" type="text/css" />
...@@ -44,7 +44,7 @@ ...@@ -44,7 +44,7 @@
<li class="right" > <li class="right" >
<a href="../../index.html" title="PaddlePaddle Documentation" <a href="../../index.html" title="PaddlePaddle Documentation"
accesskey="P">previous</a> |</li> accesskey="P">previous</a> |</li>
<li class="nav-item nav-item-0"><a href="../../index.html">PaddlePaddle documentation</a> &raquo;</li> <li class="nav-item nav-item-0"><a href="../../index.html">PaddlePaddle documentation</a> &#187;</li>
</ul> </ul>
</div> </div>
...@@ -72,11 +72,11 @@ ...@@ -72,11 +72,11 @@
<span id="overview"></span><h2>Overview<a class="headerlink" href="#overview" title="Permalink to this headline"></a></h2> <span id="overview"></span><h2>Overview<a class="headerlink" href="#overview" title="Permalink to this headline"></a></h2>
<p>For the first step, you will use PaddlePaddle to build a <strong>text classification</strong> system. For example, suppose you run an e-commence website, and you want to analyze the sentiment of user reviews to evaluate product quality.</p> <p>For the first step, you will use PaddlePaddle to build a <strong>text classification</strong> system. For example, suppose you run an e-commence website, and you want to analyze the sentiment of user reviews to evaluate product quality.</p>
<p>For example, given the input</p> <p>For example, given the input</p>
<div class="highlight-python"><div class="highlight"><pre><span></span>This monitor is fantastic. <div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">This</span> <span class="n">monitor</span> <span class="ow">is</span> <span class="n">fantastic</span><span class="o">.</span>
</pre></div> </pre></div>
</div> </div>
<p>Your classifier should output “positive”, since this text snippet shows that the user is satisfied with the product. Given this input:</p> <p>Your classifier should output “positive”, since this text snippet shows that the user is satisfied with the product. Given this input:</p>
<div class="highlight-python"><div class="highlight"><pre><span></span>The monitor breaks down two months after purchase. <div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">The</span> <span class="n">monitor</span> <span class="n">breaks</span> <span class="n">down</span> <span class="n">two</span> <span class="n">months</span> <span class="n">after</span> <span class="n">purchase</span><span class="o">.</span>
</pre></div> </pre></div>
</div> </div>
<p>the classifier should output “negative“.</p> <p>the classifier should output “negative“.</p>
...@@ -114,7 +114,7 @@ ...@@ -114,7 +114,7 @@
<div class="section" id="preprocess-data-into-standardized-format"> <div class="section" id="preprocess-data-into-standardized-format">
<span id="preprocess-data-into-standardized-format"></span><h2>Preprocess data into standardized format<a class="headerlink" href="#preprocess-data-into-standardized-format" title="Permalink to this headline"></a></h2> <span id="preprocess-data-into-standardized-format"></span><h2>Preprocess data into standardized format<a class="headerlink" href="#preprocess-data-into-standardized-format" title="Permalink to this headline"></a></h2>
<p>In this example, you are going to use <a class="reference external" href="http://jmcauley.ucsd.edu/data/amazon/">Amazon electronic product review dataset</a> to build a bunch of deep neural network models for text classification. Each text in this dataset is a product review. This dataset has two categories: “positive” and “negative”. Positive means the reviewer likes the product, while negative means the reviewer does not like the product.</p> <p>In this example, you are going to use <a class="reference external" href="http://jmcauley.ucsd.edu/data/amazon/">Amazon electronic product review dataset</a> to build a bunch of deep neural network models for text classification. Each text in this dataset is a product review. This dataset has two categories: “positive” and “negative”. Positive means the reviewer likes the product, while negative means the reviewer does not like the product.</p>
<p><code class="docutils literal"><span class="pre">demo/quick_start</span></code> provides scripts for downloading data and preprocessing data as shown below. The data process takes several minutes (about 3 minutes in our machine).</p> <p><code class="docutils literal"><span class="pre">demo/quick_start</span></code> in the <a class="reference external" href="https://github.com/baidu/Paddle">source code</a> provides scripts for downloading data and preprocessing data as shown below. The data process takes several minutes (about 3 minutes in our machine).</p>
<div class="highlight-bash"><div class="highlight"><pre><span></span><span class="nb">cd</span> demo/quick_start <div class="highlight-bash"><div class="highlight"><pre><span></span><span class="nb">cd</span> demo/quick_start
./data/get_data.sh ./data/get_data.sh
./preprocess.sh ./preprocess.sh
...@@ -214,8 +214,7 @@ ...@@ -214,8 +214,7 @@
<span class="n">args</span><span class="o">=</span><span class="p">{</span><span class="s2">&quot;dictionary&quot;</span><span class="p">:</span> <span class="n">word_dict</span><span class="p">})</span> <span class="n">args</span><span class="o">=</span><span class="p">{</span><span class="s2">&quot;dictionary&quot;</span><span class="p">:</span> <span class="n">word_dict</span><span class="p">})</span>
</pre></div> </pre></div>
</div> </div>
<p>You can refer to the following link for more detailed examples <p>You can refer to the following link for more detailed examples and data formats: <a href = "../../ui/data_provider/pydataprovider2.html">PyDataProvider2</a>.</p>
: <a href = "../../ui/data_provider/python_case.html">Python Use Case</a>，The detailed documentation on data format is: <a href = "../../ui/api/py_data_provider_wrapper.html"> PyDataProviderWrapper</a>。</p>
</div> </div>
</div> </div>
<div class="section" id="network-architecture"> <div class="section" id="network-architecture">
...@@ -443,7 +442,7 @@ paddle train <span class="se">\</span> ...@@ -443,7 +442,7 @@ paddle train <span class="se">\</span>
mv rank-00000 result.txt mv rank-00000 result.txt
</pre></div> </pre></div>
</div> </div>
<p>There are several differences between training and inference network configurations.</p> <p>User can choose the best model base on the training log instead of model <code class="docutils literal"><span class="pre">output/pass-00003</span></code>. There are several differences between training and inference network configurations.</p>
<ul class="simple"> <ul class="simple">
<li>You do not need labels during inference.</li> <li>You do not need labels during inference.</li>
<li>Outputs need to be specified to the classification probability layer (the output of softmax layer), or the id of maximum probability (<code class="docutils literal"><span class="pre">max_id</span></code> layer). An example to output the id and probability is given in the code snippet.</li> <li>Outputs need to be specified to the classification probability layer (the output of softmax layer), or the id of maximum probability (<code class="docutils literal"><span class="pre">max_id</span></code> layer). An example to output the id and probability is given in the code snippet.</li>
...@@ -451,8 +450,8 @@ mv rank-00000 result.txt ...@@ -451,8 +450,8 @@ mv rank-00000 result.txt
<li>You need to specify the location of <code class="docutils literal"><span class="pre">test_list</span></code> in the test data.</li> <li>You need to specify the location of <code class="docutils literal"><span class="pre">test_list</span></code> in the test data.</li>
</ul> </ul>
<p>The results in <code class="docutils literal"><span class="pre">result.txt</span></code> is as follows, each line is one sample.</p> <p>The results in <code class="docutils literal"><span class="pre">result.txt</span></code> is as follows, each line is one sample.</p>
<div class="highlight-python"><div class="highlight"><pre><span></span>predicted_label_id;probability_of_label_0 probability_of_label_1 # the first sample <div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">predicted_label_id</span><span class="p">;</span><span class="n">probability_of_label_0</span> <span class="n">probability_of_label_1</span> <span class="c1"># the first sample</span>
predicted_label_id;probability_of_label_0 probability_of_label_1 # the second sample <span class="n">predicted_label_id</span><span class="p">;</span><span class="n">probability_of_label_0</span> <span class="n">probability_of_label_1</span> <span class="c1"># the second sample</span>
</pre></div> </pre></div>
</div> </div>
<div class="highlight-python"><div class="highlight"><pre><span></span><span class="n">is_predict</span> <span class="o">=</span> <span class="n">get_config_arg</span><span class="p">(</span><span class="s1">&#39;is_predict&#39;</span><span class="p">,</span> <span class="nb">bool</span><span class="p">,</span> <span class="bp">False</span><span class="p">)</span> <div class="highlight-python"><div class="highlight"><pre><span></span><span class="n">is_predict</span> <span class="o">=</span> <span class="n">get_config_arg</span><span class="p">(</span><span class="s1">&#39;is_predict&#39;</span><span class="p">,</span> <span class="nb">bool</span><span class="p">,</span> <span class="bp">False</span><span class="p">)</span>
...@@ -520,7 +519,7 @@ predicted_label_id;probability_of_label_0 probability_of_label_1 # the second s ...@@ -520,7 +519,7 @@ predicted_label_id;probability_of_label_0 probability_of_label_1 # the second s
</div> </div>
<div class="section" id="log"> <div class="section" id="log">
<span id="log"></span><h3>Log<a class="headerlink" href="#log" title="Permalink to this headline"></a></h3> <span id="log"></span><h3>Log<a class="headerlink" href="#log" title="Permalink to this headline"></a></h3>
<div class="highlight-python"><div class="highlight"><pre><span></span>TrainerInternal.cpp:160] Batch=20 samples=2560 AvgCost=0.628761 CurrentCost=0.628761 Eval: classification_error_evaluator=0.304297 CurrentEval: classification_error_evaluator=0.304297 <div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">TrainerInternal</span><span class="o">.</span><span class="n">cpp</span><span class="p">:</span><span class="mi">160</span><span class="p">]</span> <span class="n">Batch</span><span class="o">=</span><span class="mi">20</span> <span class="n">samples</span><span class="o">=</span><span class="mi">2560</span> <span class="n">AvgCost</span><span class="o">=</span><span class="mf">0.628761</span> <span class="n">CurrentCost</span><span class="o">=</span><span class="mf">0.628761</span> <span class="n">Eval</span><span class="p">:</span> <span class="n">classification_error_evaluator</span><span class="o">=</span><span class="mf">0.304297</span> <span class="n">CurrentEval</span><span class="p">:</span> <span class="n">classification_error_evaluator</span><span class="o">=</span><span class="mf">0.304297</span>
</pre></div> </pre></div>
</div> </div>
<p>During model training, you will see the log like the examples above: <p>During model training, you will see the log like the examples above:
...@@ -607,14 +606,11 @@ predicted_label_id;probability_of_label_0 probability_of_label_1 # the second s ...@@ -607,14 +606,11 @@ predicted_label_id;probability_of_label_0 probability_of_label_1 # the second s
<div id="searchbox" style="display: none" role="search"> <div id="searchbox" style="display: none" role="search">
<h3>Quick search</h3> <h3>Quick search</h3>
<form class="search" action="../../search.html" method="get"> <form class="search" action="../../search.html" method="get">
<input type="text" name="q" /> <div><input type="text" name="q" /></div>
<input type="submit" value="Go" /> <div><input type="submit" value="Go" /></div>
<input type="hidden" name="check_keywords" value="yes" /> <input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" /> <input type="hidden" name="area" value="default" />
</form> </form>
<p class="searchtip" style="font-size: 90%">
Enter search terms or a module, class or function name.
</p>
</div> </div>
<script type="text/javascript">$('#searchbox').show(0);</script> <script type="text/javascript">$('#searchbox').show(0);</script>
</div> </div>
...@@ -636,12 +632,12 @@ predicted_label_id;probability_of_label_0 probability_of_label_1 # the second s ...@@ -636,12 +632,12 @@ predicted_label_id;probability_of_label_0 probability_of_label_1 # the second s
<li class="right" > <li class="right" >
<a href="../../index.html" title="PaddlePaddle Documentation" <a href="../../index.html" title="PaddlePaddle Documentation"
>previous</a> |</li> >previous</a> |</li>
<li class="nav-item nav-item-0"><a href="../../index.html">PaddlePaddle documentation</a> &raquo;</li> <li class="nav-item nav-item-0"><a href="../../index.html">PaddlePaddle documentation</a> &#187;</li>
</ul> </ul>
</div> </div>
<div class="footer" role="contentinfo"> <div class="footer" role="contentinfo">
&copy; Copyright 2016, PaddlePaddle developers. &#169; Copyright 2016, PaddlePaddle developers.
Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.3.5. Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.4.6.
</div> </div>
</body> </body>
</html> </html>
\ No newline at end of file
doc/demo/rec/ml_dataset.html
浏览文件 @ 22cb62db
...@@ -6,7 +6,7 @@ ...@@ -6,7 +6,7 @@
<head> <head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>MovieLens Dataset &mdash; PaddlePaddle documentation</title> <title>MovieLens Dataset &#8212; PaddlePaddle documentation</title>
<link rel="stylesheet" href="../../_static/classic.css" type="text/css" /> <link rel="stylesheet" href="../../_static/classic.css" type="text/css" />
<link rel="stylesheet" href="../../_static/pygments.css" type="text/css" /> <link rel="stylesheet" href="../../_static/pygments.css" type="text/css" />
...@@ -27,7 +27,7 @@ ...@@ -27,7 +27,7 @@
<link rel="top" title="PaddlePaddle documentation" href="../../index.html" /> <link rel="top" title="PaddlePaddle documentation" href="../../index.html" />
<link rel="up" title="Examples and demos" href="../index.html" /> <link rel="up" title="Examples and demos" href="../index.html" />
<link rel="next" title="Regression MovieLens Ratting" href="ml_regression.html" /> <link rel="next" title="Regression MovieLens Ratting" href="ml_regression.html" />
<link rel="prev" title="Text generation Tutorial" href="../text_generation/text_generation.html" /> <link rel="prev" title="Semantic Role labeling Tutorial" href="../semantic_role_labeling/semantic_role_labeling.html" />
</head> </head>
<body role="document"> <body role="document">
<div class="related" role="navigation" aria-label="related navigation"> <div class="related" role="navigation" aria-label="related navigation">
...@@ -43,10 +43,10 @@ ...@@ -43,10 +43,10 @@
<a href="ml_regression.html" title="Regression MovieLens Ratting" <a href="ml_regression.html" title="Regression MovieLens Ratting"
accesskey="N">next</a> |</li> accesskey="N">next</a> |</li>
<li class="right" > <li class="right" >
<a href="../text_generation/text_generation.html" title="Text generation Tutorial" <a href="../semantic_role_labeling/semantic_role_labeling.html" title="Semantic Role labeling Tutorial"
accesskey="P">previous</a> |</li> accesskey="P">previous</a> |</li>
<li class="nav-item nav-item-0"><a href="../../index.html">PaddlePaddle documentation</a> &raquo;</li> <li class="nav-item nav-item-0"><a href="../../index.html">PaddlePaddle documentation</a> &#187;</li>
<li class="nav-item nav-item-1"><a href="../index.html" accesskey="U">Examples and demos</a> &raquo;</li> <li class="nav-item nav-item-1"><a href="../index.html" accesskey="U">Examples and demos</a> &#187;</li>
</ul> </ul>
</div> </div>
...@@ -183,8 +183,8 @@ entries and/or test entries</li> ...@@ -183,8 +183,8 @@ entries and/or test entries</li>
</ul> </ul>
<h4>Previous topic</h4> <h4>Previous topic</h4>
<p class="topless"><a href="../text_generation/text_generation.html" <p class="topless"><a href="../semantic_role_labeling/semantic_role_labeling.html"
title="previous chapter">Text generation Tutorial</a></p> title="previous chapter">Semantic Role labeling Tutorial</a></p>
<h4>Next topic</h4> <h4>Next topic</h4>
<p class="topless"><a href="ml_regression.html" <p class="topless"><a href="ml_regression.html"
title="next chapter">Regression MovieLens Ratting</a></p> title="next chapter">Regression MovieLens Ratting</a></p>
...@@ -198,14 +198,11 @@ entries and/or test entries</li> ...@@ -198,14 +198,11 @@ entries and/or test entries</li>
<div id="searchbox" style="display: none" role="search"> <div id="searchbox" style="display: none" role="search">
<h3>Quick search</h3> <h3>Quick search</h3>
<form class="search" action="../../search.html" method="get"> <form class="search" action="../../search.html" method="get">
<input type="text" name="q" /> <div><input type="text" name="q" /></div>
<input type="submit" value="Go" /> <div><input type="submit" value="Go" /></div>
<input type="hidden" name="check_keywords" value="yes" /> <input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" /> <input type="hidden" name="area" value="default" />
</form> </form>
<p class="searchtip" style="font-size: 90%">
Enter search terms or a module, class or function name.
</p>
</div> </div>
<script type="text/javascript">$('#searchbox').show(0);</script> <script type="text/javascript">$('#searchbox').show(0);</script>
</div> </div>
...@@ -225,15 +222,15 @@ entries and/or test entries</li> ...@@ -225,15 +222,15 @@ entries and/or test entries</li>
<a href="ml_regression.html" title="Regression MovieLens Ratting" <a href="ml_regression.html" title="Regression MovieLens Ratting"
>next</a> |</li> >next</a> |</li>
<li class="right" > <li class="right" >
<a href="../text_generation/text_generation.html" title="Text generation Tutorial" <a href="../semantic_role_labeling/semantic_role_labeling.html" title="Semantic Role labeling Tutorial"
>previous</a> |</li> >previous</a> |</li>
<li class="nav-item nav-item-0"><a href="../../index.html">PaddlePaddle documentation</a> &raquo;</li> <li class="nav-item nav-item-0"><a href="../../index.html">PaddlePaddle documentation</a> &#187;</li>
<li class="nav-item nav-item-1"><a href="../index.html" >Examples and demos</a> &raquo;</li> <li class="nav-item nav-item-1"><a href="../index.html" >Examples and demos</a> &#187;</li>
</ul> </ul>
</div> </div>
<div class="footer" role="contentinfo"> <div class="footer" role="contentinfo">
&copy; Copyright 2016, PaddlePaddle developers. &#169; Copyright 2016, PaddlePaddle developers.
Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.3.5. Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.4.6.
</div> </div>
</body> </body>
</html> </html>
\ No newline at end of file
doc/demo/rec/ml_regression.html
浏览文件 @ 22cb62db
...@@ -6,7 +6,7 @@ ...@@ -6,7 +6,7 @@
<head> <head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>Regression MovieLens Ratting &mdash; PaddlePaddle documentation</title> <title>Regression MovieLens Ratting &#8212; PaddlePaddle documentation</title>
<link rel="stylesheet" href="../../_static/classic.css" type="text/css" /> <link rel="stylesheet" href="../../_static/classic.css" type="text/css" />
<link rel="stylesheet" href="../../_static/pygments.css" type="text/css" /> <link rel="stylesheet" href="../../_static/pygments.css" type="text/css" />
...@@ -45,8 +45,8 @@ ...@@ -45,8 +45,8 @@
<li class="right" > <li class="right" >
<a href="ml_dataset.html" title="MovieLens Dataset" <a href="ml_dataset.html" title="MovieLens Dataset"
accesskey="P">previous</a> |</li> accesskey="P">previous</a> |</li>
<li class="nav-item nav-item-0"><a href="../../index.html">PaddlePaddle documentation</a> &raquo;</li> <li class="nav-item nav-item-0"><a href="../../index.html">PaddlePaddle documentation</a> &#187;</li>
<li class="nav-item nav-item-1"><a href="../index.html" accesskey="U">Examples and demos</a> &raquo;</li> <li class="nav-item nav-item-1"><a href="../index.html" accesskey="U">Examples and demos</a> &#187;</li>
</ul> </ul>
</div> </div>
...@@ -91,7 +91,7 @@ i.e, specific <strong>WHAT</strong> type it is in each feature file.</p> ...@@ -91,7 +91,7 @@ i.e, specific <strong>WHAT</strong> type it is in each feature file.</p>
<p>The field config file of ml-1m shows in <code class="code docutils literal"><span class="pre">demo/recommendation/data/config.json</span></code>. <p>The field config file of ml-1m shows in <code class="code docutils literal"><span class="pre">demo/recommendation/data/config.json</span></code>.
It specifics the field types and file names: 1) there are four types of field for user file: id, gender, age and occupation; It specifics the field types and file names: 1) there are four types of field for user file: id, gender, age and occupation;
2) the filename is &#8220;users.dat&#8221;, and the delimiter of file is &#8221;::&#8221;.</p> 2) the filename is &#8220;users.dat&#8221;, and the delimiter of file is &#8221;::&#8221;.</p>
<div class="highlight-python"><div class="highlight"><pre><span></span><span class="p">{</span> <div class="highlight-default"><div class="highlight"><pre><span></span><span class="p">{</span>
<span class="s2">&quot;user&quot;</span><span class="p">:</span> <span class="p">{</span> <span class="s2">&quot;user&quot;</span><span class="p">:</span> <span class="p">{</span>
<span class="s2">&quot;file&quot;</span><span class="p">:</span> <span class="p">{</span> <span class="s2">&quot;file&quot;</span><span class="p">:</span> <span class="p">{</span>
<span class="s2">&quot;name&quot;</span><span class="p">:</span> <span class="s2">&quot;users.dat&quot;</span><span class="p">,</span> <span class="s2">&quot;name&quot;</span><span class="p">:</span> <span class="s2">&quot;users.dat&quot;</span><span class="p">,</span>
...@@ -141,7 +141,7 @@ python config_generator.py config.json &gt; meta_config.json ...@@ -141,7 +141,7 @@ python config_generator.py config.json &gt; meta_config.json
</pre></div> </pre></div>
</div> </div>
<p>The meta config file shows below:</p> <p>The meta config file shows below:</p>
<div class="highlight-python"><div class="highlight"><pre><span></span><span class="p">{</span> <div class="highlight-default"><div class="highlight"><pre><span></span><span class="p">{</span>
<span class="s2">&quot;meta&quot;</span><span class="p">:</span> <span class="p">{</span> <span class="s2">&quot;meta&quot;</span><span class="p">:</span> <span class="p">{</span>
<span class="s2">&quot;movie&quot;</span><span class="p">:</span> <span class="p">{</span> <span class="s2">&quot;movie&quot;</span><span class="p">:</span> <span class="p">{</span>
<span class="s2">&quot;fields&quot;</span><span class="p">:</span> <span class="p">[</span> <span class="s2">&quot;fields&quot;</span><span class="p">:</span> <span class="p">[</span>
...@@ -381,21 +381,7 @@ cp ml-1m/ratings.dat.test . ...@@ -381,21 +381,7 @@ cp ml-1m/ratings.dat.test .
<p>The network structure shows below.</p> <p>The network structure shows below.</p>
<img alt="rec_regression_network" class="align-center" src="../../_images/rec_regression_network.png" /> <img alt="rec_regression_network" class="align-center" src="../../_images/rec_regression_network.png" />
<p>The demo&#8217;s neural network config file &#8220;trainer_config.py&#8221; show as below.</p> <p>The demo&#8217;s neural network config file &#8220;trainer_config.py&#8221; show as below.</p>
<div class="highlight-python"><div class="highlight"><pre><span></span><span class="c1"># Copyright (c) 2016 Baidu, Inc. All Rights Reserved</span> <div class="highlight-python"><div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">paddle.trainer_config_helpers</span> <span class="kn">import</span> <span class="o">*</span>
<span class="c1">#</span>
<span class="c1"># Licensed under the Apache License, Version 2.0 (the &quot;License&quot;);</span>
<span class="c1"># you may not use this file except in compliance with the License.</span>
<span class="c1"># You may obtain a copy of the License at</span>
<span class="c1">#</span>
<span class="c1"># http://www.apache.org/licenses/LICENSE-2.0</span>
<span class="c1">#</span>
<span class="c1"># Unless required by applicable law or agreed to in writing, software</span>
<span class="c1"># distributed under the License is distributed on an &quot;AS IS&quot; BASIS,</span>
<span class="c1"># WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.</span>
<span class="c1"># See the License for the specific language governing permissions and</span>
<span class="c1"># limitations under the License.</span>
<span class="kn">from</span> <span class="nn">paddle.trainer_config_helpers</span> <span class="kn">import</span> <span class="o">*</span>
<span class="k">try</span><span class="p">:</span> <span class="k">try</span><span class="p">:</span>
<span class="kn">import</span> <span class="nn">cPickle</span> <span class="kn">as</span> <span class="nn">pickle</span> <span class="kn">import</span> <span class="nn">cPickle</span> <span class="kn">as</span> <span class="nn">pickle</span>
...@@ -519,34 +505,14 @@ features.</p> ...@@ -519,34 +505,14 @@ features.</p>
<li>Pooling Layer, <a class="reference external" href="../../ui/api/trainer_config_helpers/layers.html#pooling-layer">pooling_layer</a></li> <li>Pooling Layer, <a class="reference external" href="../../ui/api/trainer_config_helpers/layers.html#pooling-layer">pooling_layer</a></li>
<li>Cosine Similarity Layer, <a class="reference external" href="../../ui/api/trainer_config_helpers/layers.html#cos-sim">cos_sim</a></li> <li>Cosine Similarity Layer, <a class="reference external" href="../../ui/api/trainer_config_helpers/layers.html#cos-sim">cos_sim</a></li>
<li>Text Convolution Pooling Layer, <a class="reference external" href="../../ui/api/trainer_config_helpers/networks.html#trainer_config_helpers.networks.text_conv_pool">text_conv_pool</a></li> <li>Text Convolution Pooling Layer, <a class="reference external" href="../../ui/api/trainer_config_helpers/networks.html#trainer_config_helpers.networks.text_conv_pool">text_conv_pool</a></li>
<li>Declare Python Data Sources, <a class="reference external" href="../../ui/api/trainer_config_helpers/data_sources.html">define_py_data_sources</a></li> <li>Declare Python Data Sources, <a class="reference external" href="../../ui/api/trainer_config_helpers/data_sources.html">define_py_data_sources2</a></li>
</ul> </ul>
</div> </div>
<div class="section" id="data-provider"> <div class="section" id="data-provider">
<h3>Data Provider<a class="headerlink" href="#data-provider" title="Permalink to this headline"></a></h3> <h3>Data Provider<a class="headerlink" href="#data-provider" title="Permalink to this headline"></a></h3>
<div class="highlight-python"><div class="highlight"><pre><span></span><span class="c1"># Copyright (c) 2016 Baidu, Inc. All Rights Reserved</span> <div class="highlight-python"><div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">paddle.trainer.PyDataProvider2</span> <span class="kn">import</span> <span class="o">*</span>
<span class="c1">#</span>
<span class="c1"># Licensed under the Apache License, Version 2.0 (the &quot;License&quot;);</span>
<span class="c1"># you may not use this file except in compliance with the License.</span>
<span class="c1"># You may obtain a copy of the License at</span>
<span class="c1">#</span>
<span class="c1"># http://www.apache.org/licenses/LICENSE-2.0</span>
<span class="c1">#</span>
<span class="c1"># Unless required by applicable law or agreed to in writing, software</span>
<span class="c1"># distributed under the License is distributed on an &quot;AS IS&quot; BASIS,</span>
<span class="c1"># WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.</span>
<span class="c1"># See the License for the specific language governing permissions and</span>
<span class="c1"># limitations under the License.</span>
<span class="k">try</span><span class="p">:</span>
<span class="kn">import</span> <span class="nn">cPickle</span> <span class="kn">as</span> <span class="nn">pickle</span>
<span class="k">except</span> <span class="ne">ImportError</span><span class="p">:</span>
<span class="kn">import</span> <span class="nn">pickle</span>
<span class="kn">from</span> <span class="nn">paddle.trainer.PyDataProvider2</span> <span class="kn">import</span> <span class="o">*</span>
<span class="kn">import</span> <span class="nn">common_utils</span> <span class="c1"># parse</span> <span class="kn">import</span> <span class="nn">common_utils</span> <span class="c1"># parse</span>
<span class="k">def</span> <span class="nf">hook</span><span class="p">(</span><span class="n">settings</span><span class="p">,</span> <span class="n">meta</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span> <span class="k">def</span> <span class="nf">hook</span><span class="p">(</span><span class="n">settings</span><span class="p">,</span> <span class="n">meta</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span>
<span class="sd">&quot;&quot;&quot;</span> <span class="sd">&quot;&quot;&quot;</span>
<span class="sd"> Init hook is invoked before process data. It will set obj.slots and store</span> <span class="sd"> Init hook is invoked before process data. It will set obj.slots and store</span>
...@@ -573,7 +539,6 @@ features.</p> ...@@ -573,7 +539,6 @@ features.</p>
<span class="n">settings</span><span class="o">.</span><span class="n">input_types</span> <span class="o">=</span> <span class="n">headers</span> <span class="n">settings</span><span class="o">.</span><span class="n">input_types</span> <span class="o">=</span> <span class="n">headers</span>
<span class="n">settings</span><span class="o">.</span><span class="n">meta</span> <span class="o">=</span> <span class="n">meta</span> <span class="n">settings</span><span class="o">.</span><span class="n">meta</span> <span class="o">=</span> <span class="n">meta</span>
<span class="nd">@provider</span><span class="p">(</span><span class="n">init_hook</span><span class="o">=</span><span class="n">hook</span><span class="p">,</span> <span class="n">cache</span><span class="o">=</span><span class="n">CacheType</span><span class="o">.</span><span class="n">CACHE_PASS_IN_MEM</span><span class="p">)</span> <span class="nd">@provider</span><span class="p">(</span><span class="n">init_hook</span><span class="o">=</span><span class="n">hook</span><span class="p">,</span> <span class="n">cache</span><span class="o">=</span><span class="n">CacheType</span><span class="o">.</span><span class="n">CACHE_PASS_IN_MEM</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">process</span><span class="p">(</span><span class="n">settings</span><span class="p">,</span> <span class="n">filename</span><span class="p">):</span> <span class="k">def</span> <span class="nf">process</span><span class="p">(</span><span class="n">settings</span><span class="p">,</span> <span class="n">filename</span><span class="p">):</span>
<span class="k">with</span> <span class="nb">open</span><span class="p">(</span><span class="n">filename</span><span class="p">,</span> <span class="s1">&#39;r&#39;</span><span class="p">)</span> <span class="k">as</span> <span class="n">f</span><span class="p">:</span> <span class="k">with</span> <span class="nb">open</span><span class="p">(</span><span class="n">filename</span><span class="p">,</span> <span class="s1">&#39;r&#39;</span><span class="p">)</span> <span class="k">as</span> <span class="n">f</span><span class="p">:</span>
...@@ -614,42 +579,27 @@ In this <code class="code docutils literal"><span class="pre">dataprovider.py</s ...@@ -614,42 +579,27 @@ In this <code class="code docutils literal"><span class="pre">dataprovider.py</s
<li>use_seq: Whether this <code class="code docutils literal"><span class="pre">dataprovider.py</span></code> in sequence mode or not.</li> <li>use_seq: Whether this <code class="code docutils literal"><span class="pre">dataprovider.py</span></code> in sequence mode or not.</li>
<li>process: Return each sample of data to <code class="code docutils literal"><span class="pre">paddle</span></code>.</li> <li>process: Return each sample of data to <code class="code docutils literal"><span class="pre">paddle</span></code>.</li>
</ul> </ul>
<p>The data provider details document see <a class="reference external" href="../../ui/DataProvider.html">there</a>.</p> <p>The data provider details document see <a class="reference external" href="../../ui/data_provider/pydataprovider2.html">there</a>.</p>
</div> </div>
</div> </div>
<div class="section" id="train"> <div class="section" id="train">
<h2>Train<a class="headerlink" href="#train" title="Permalink to this headline"></a></h2> <h2>Train<a class="headerlink" href="#train" title="Permalink to this headline"></a></h2>
<p>After prepare data, config network, writting data provider, now we can run paddle training.</p> <p>After prepare data, config network, writting data provider, now we can run paddle training.</p>
<p>The run.sh is shown as follow:</p> <p>The run.sh is shown as follow:</p>
<div class="highlight-python"><div class="highlight"><pre><span></span>#!/bin/bash <div class="highlight-bash"><div class="highlight"><pre><span></span>paddle train <span class="se">\</span>
# Copyright (c) 2016 Baidu, Inc. All Rights Reserved --config<span class="o">=</span>trainer_config.py <span class="se">\</span>
# --save_dir<span class="o">=</span>./output <span class="se">\</span>
# Licensed under the Apache License, Version 2.0 (the &quot;License&quot;); --use_gpu<span class="o">=</span><span class="nb">false</span> <span class="se">\</span>
# you may not use this file except in compliance with the License. --trainer_count<span class="o">=</span>4<span class="se">\</span>
# You may obtain a copy of the License at --test_all_data_in_one_period<span class="o">=</span><span class="nb">true</span> <span class="se">\</span>
# --log_period<span class="o">=</span><span class="m">100</span> <span class="se">\</span>
# http://www.apache.org/licenses/LICENSE-2.0 --dot_period<span class="o">=</span><span class="m">1</span> <span class="se">\</span>
# --num_passes<span class="o">=</span><span class="m">50</span> 2&gt;<span class="p">&amp;</span><span class="m">1</span> <span class="p">|</span> tee <span class="s1">&#39;log.txt&#39;</span>
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an &quot;AS IS&quot; BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
set -e
paddle train \
--config=trainer_config.py \
--save_dir=./output \
--use_gpu=false \
--trainer_count=4\
--test_all_data_in_one_period=true \
--log_period=100 \
--dot_period=1 \
--num_passes=50 2&gt;&amp;1 | tee &#39;log.txt&#39;
</pre></div> </pre></div>
</div> </div>
<p>It just start a paddle training process, write the log to <cite>log.txt</cite>, <p>It just start a paddle training process, write the log to <cite>log.txt</cite>,
then print it on screen.</p> then print it on screen.</p>
<p>Each command line argument in <code class="code docutils literal"><span class="pre">run.sh</span></code>, please refer to the <a class="reference external" href="TBD">command line <p>Each command line argument in <code class="code docutils literal"><span class="pre">run.sh</span></code>, please refer to the <a class="reference external" href="../../ui/index.html#command-line-argument">command line
arguments</a> page. The short description of these arguments is shown as follow.</p> arguments</a> page. The short description of these arguments is shown as follow.</p>
<ul class="simple"> <ul class="simple">
<li>config: Tell paddle which file is neural network configuration.</li> <li>config: Tell paddle which file is neural network configuration.</li>
...@@ -759,14 +709,11 @@ Prediction Score is 3.13 ...@@ -759,14 +709,11 @@ Prediction Score is 3.13
<div id="searchbox" style="display: none" role="search"> <div id="searchbox" style="display: none" role="search">
<h3>Quick search</h3> <h3>Quick search</h3>
<form class="search" action="../../search.html" method="get"> <form class="search" action="../../search.html" method="get">
<input type="text" name="q" /> <div><input type="text" name="q" /></div>
<input type="submit" value="Go" /> <div><input type="submit" value="Go" /></div>
<input type="hidden" name="check_keywords" value="yes" /> <input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" /> <input type="hidden" name="area" value="default" />
</form> </form>
<p class="searchtip" style="font-size: 90%">
Enter search terms or a module, class or function name.
</p>
</div> </div>
<script type="text/javascript">$('#searchbox').show(0);</script> <script type="text/javascript">$('#searchbox').show(0);</script>
</div> </div>
...@@ -788,13 +735,13 @@ Prediction Score is 3.13 ...@@ -788,13 +735,13 @@ Prediction Score is 3.13
<li class="right" > <li class="right" >
<a href="ml_dataset.html" title="MovieLens Dataset" <a href="ml_dataset.html" title="MovieLens Dataset"
>previous</a> |</li> >previous</a> |</li>
<li class="nav-item nav-item-0"><a href="../../index.html">PaddlePaddle documentation</a> &raquo;</li> <li class="nav-item nav-item-0"><a href="../../index.html">PaddlePaddle documentation</a> &#187;</li>
<li class="nav-item nav-item-1"><a href="../index.html" >Examples and demos</a> &raquo;</li> <li class="nav-item nav-item-1"><a href="../index.html" >Examples and demos</a> &#187;</li>
</ul> </ul>
</div> </div>
<div class="footer" role="contentinfo"> <div class="footer" role="contentinfo">
&copy; Copyright 2016, PaddlePaddle developers. &#169; Copyright 2016, PaddlePaddle developers.
Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.3.5. Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.4.6.
</div> </div>
</body> </body>
</html> </html>
\ No newline at end of file
doc_cn/build_and_install/install/index.html → doc/demo/semantic_role_labeling/index.html
浏览文件 @ 22cb62db
...@@ -6,7 +6,7 @@ ...@@ -6,7 +6,7 @@
<head> <head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>安装PaddlePaddle &mdash; PaddlePaddle documentation</title> <title>Semantic Role Labeling Tutorial &#8212; PaddlePaddle documentation</title>
<link rel="stylesheet" href="../../_static/classic.css" type="text/css" /> <link rel="stylesheet" href="../../_static/classic.css" type="text/css" />
<link rel="stylesheet" href="../../_static/pygments.css" type="text/css" /> <link rel="stylesheet" href="../../_static/pygments.css" type="text/css" />
...@@ -25,9 +25,9 @@ ...@@ -25,9 +25,9 @@
<script type="text/javascript" src="../../_static/doctools.js"></script> <script type="text/javascript" src="../../_static/doctools.js"></script>
<script type="text/javascript" src="https://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script> <script type="text/javascript" src="https://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script>
<link rel="top" title="PaddlePaddle documentation" href="../../index.html" /> <link rel="top" title="PaddlePaddle documentation" href="../../index.html" />
<link rel="up" title="编译与安装" href="../index.html" /> <link rel="up" title="Examples and demos" href="../index.html" />
<link rel="next" title="安装PaddlePaddle的Docker镜像" href="docker_install.html" /> <link rel="next" title="Semantic Role labeling Tutorial" href="semantic_role_labeling.html" />
<link rel="prev" title="编译与安装" href="../index.html" /> <link rel="prev" title="Text generation Tutorial" href="../text_generation/text_generation.html" />
</head> </head>
<body role="document"> <body role="document">
<div class="related" role="navigation" aria-label="related navigation"> <div class="related" role="navigation" aria-label="related navigation">
...@@ -37,13 +37,16 @@ ...@@ -37,13 +37,16 @@
<a href="../../genindex.html" title="General Index" <a href="../../genindex.html" title="General Index"
accesskey="I">index</a></li> accesskey="I">index</a></li>
<li class="right" > <li class="right" >
<a href="docker_install.html" title="安装PaddlePaddle的Docker镜像" <a href="../../py-modindex.html" title="Python Module Index"
>modules</a> |</li>
<li class="right" >
<a href="semantic_role_labeling.html" title="Semantic Role labeling Tutorial"
accesskey="N">next</a> |</li> accesskey="N">next</a> |</li>
<li class="right" > <li class="right" >
<a href="../index.html" title="编译与安装" <a href="../text_generation/text_generation.html" title="Text generation Tutorial"
accesskey="P">previous</a> |</li> accesskey="P">previous</a> |</li>
<li class="nav-item nav-item-0"><a href="../../index.html">PaddlePaddle documentation</a> &raquo;</li> <li class="nav-item nav-item-0"><a href="../../index.html">PaddlePaddle documentation</a> &#187;</li>
<li class="nav-item nav-item-1"><a href="../index.html" accesskey="U">编译与安装</a> &raquo;</li> <li class="nav-item nav-item-1"><a href="../index.html" accesskey="U">Examples and demos</a> &#187;</li>
</ul> </ul>
</div> </div>
...@@ -52,33 +55,23 @@ ...@@ -52,33 +55,23 @@
<div class="bodywrapper"> <div class="bodywrapper">
<div class="body" role="main"> <div class="body" role="main">
<div class="section" id="paddlepaddle"> <div class="section" id="semantic-role-labeling-tutorial">
<h1>安装PaddlePaddle<a class="headerlink" href="#paddlepaddle" title="Permalink to this headline"></a></h1> <h1>Semantic Role Labeling Tutorial<a class="headerlink" href="#semantic-role-labeling-tutorial" title="Permalink to this headline"></a></h1>
<p>PaddlePaddle提供数个预编译的二进制来进行安装。他们包括Docker镜像，ubuntu的deb安装包等
。欢迎贡献更多的安装包。我们更推荐使用Docker镜像来部署PaddlePaddle环境。</p>
<p>Note: The intallation packages are still in pre-release
state and your experience of installation may not be smooth.</p>
<p>注意!目前PaddlePaddle的安装包还处在pre-release的状态，
使用起来或许会不是很顺畅。</p>
<div class="toctree-wrapper compound"> <div class="toctree-wrapper compound">
<ul> <ul>
<li class="toctree-l1"><a class="reference internal" href="docker_install.html">安装PaddlePaddle的Docker镜像</a><ul> <li class="toctree-l1"><a class="reference internal" href="semantic_role_labeling.html">Semantic Role labeling Tutorial</a><ul>
<li class="toctree-l2"><a class="reference internal" href="docker_install.html#id1">PaddlePaddle提供的Docker镜像版本</a></li> <li class="toctree-l2"><a class="reference internal" href="semantic_role_labeling.html#data-description">Data Description</a></li>
<li class="toctree-l2"><a class="reference internal" href="docker_install.html#docker">下载和运行Docker镜像</a></li> <li class="toctree-l2"><a class="reference internal" href="semantic_role_labeling.html#training">Training</a><ul>
<li class="toctree-l2"><a class="reference internal" href="docker_install.html#id3">注意事项</a><ul> <li class="toctree-l3"><a class="reference internal" href="semantic_role_labeling.html#db-lstm">DB-LSTM</a></li>
<li class="toctree-l3"><a class="reference internal" href="docker_install.html#id4">性能问题</a></li> <li class="toctree-l3"><a class="reference internal" href="semantic_role_labeling.html#features">Features</a></li>
<li class="toctree-l3"><a class="reference internal" href="docker_install.html#id5">远程访问问题和二次开发</a></li> <li class="toctree-l3"><a class="reference internal" href="semantic_role_labeling.html#data-provider">Data Provider</a></li>
</ul> <li class="toctree-l3"><a class="reference internal" href="semantic_role_labeling.html#neural-network-config">Neural Network Config</a></li>
</li> <li class="toctree-l3"><a class="reference internal" href="semantic_role_labeling.html#run-training">Run Training</a></li>
</ul> <li class="toctree-l3"><a class="reference internal" href="semantic_role_labeling.html#run-testing">Run testing</a></li>
</li> <li class="toctree-l3"><a class="reference internal" href="semantic_role_labeling.html#run-prediction">Run prediction</a></li>
<li class="toctree-l1"><a class="reference internal" href="ubuntu_install.html">使用deb包在Ubuntu上安装PaddlePaddle</a><ul>
<li class="toctree-l2"><a class="reference internal" href="ubuntu_install.html#id2">可能遇到的问题</a><ul>
<li class="toctree-l3"><a class="reference internal" href="ubuntu_install.html#libcudart-so-libcudnn-so">libcudart.so/libcudnn.so找不到</a></li>
<li class="toctree-l3"><a class="reference internal" href="ubuntu_install.html#cuda-driver">CUDA Driver找不到</a></li>
<li class="toctree-l3"><a class="reference internal" href="ubuntu_install.html#config">config文件找不到</a></li>
</ul> </ul>
</li> </li>
<li class="toctree-l2"><a class="reference internal" href="semantic_role_labeling.html#reference">Reference</a></li>
</ul> </ul>
</li> </li>
</ul> </ul>
...@@ -92,29 +85,26 @@ state and your experience of installation may not be smooth.</p> ...@@ -92,29 +85,26 @@ state and your experience of installation may not be smooth.</p>
<div class="sphinxsidebar" role="navigation" aria-label="main navigation"> <div class="sphinxsidebar" role="navigation" aria-label="main navigation">
<div class="sphinxsidebarwrapper"> <div class="sphinxsidebarwrapper">
<h4>Previous topic</h4> <h4>Previous topic</h4>
<p class="topless"><a href="../index.html" <p class="topless"><a href="../text_generation/text_generation.html"
title="previous chapter">编译与安装</a></p> title="previous chapter">Text generation Tutorial</a></p>
<h4>Next topic</h4> <h4>Next topic</h4>
<p class="topless"><a href="docker_install.html" <p class="topless"><a href="semantic_role_labeling.html"
title="next chapter">安装PaddlePaddle的Docker镜像</a></p> title="next chapter">Semantic Role labeling Tutorial</a></p>
<div role="note" aria-label="source link"> <div role="note" aria-label="source link">
<h3>This Page</h3> <h3>This Page</h3>
<ul class="this-page-menu"> <ul class="this-page-menu">
<li><a href="../../_sources/build_and_install/install/index.txt" <li><a href="../../_sources/demo/semantic_role_labeling/index.txt"
rel="nofollow">Show Source</a></li> rel="nofollow">Show Source</a></li>
</ul> </ul>
</div> </div>
<div id="searchbox" style="display: none" role="search"> <div id="searchbox" style="display: none" role="search">
<h3>Quick search</h3> <h3>Quick search</h3>
<form class="search" action="../../search.html" method="get"> <form class="search" action="../../search.html" method="get">
<input type="text" name="q" /> <div><input type="text" name="q" /></div>
<input type="submit" value="Go" /> <div><input type="submit" value="Go" /></div>
<input type="hidden" name="check_keywords" value="yes" /> <input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" /> <input type="hidden" name="area" value="default" />
</form> </form>
<p class="searchtip" style="font-size: 90%">
Enter search terms or a module, class or function name.
</p>
</div> </div>
<script type="text/javascript">$('#searchbox').show(0);</script> <script type="text/javascript">$('#searchbox').show(0);</script>
</div> </div>
...@@ -128,18 +118,21 @@ state and your experience of installation may not be smooth.</p> ...@@ -128,18 +118,21 @@ state and your experience of installation may not be smooth.</p>
<a href="../../genindex.html" title="General Index" <a href="../../genindex.html" title="General Index"
>index</a></li> >index</a></li>
<li class="right" > <li class="right" >
<a href="docker_install.html" title="安装PaddlePaddle的Docker镜像" <a href="../../py-modindex.html" title="Python Module Index"
>modules</a> |</li>
<li class="right" >
<a href="semantic_role_labeling.html" title="Semantic Role labeling Tutorial"
>next</a> |</li> >next</a> |</li>
<li class="right" > <li class="right" >
<a href="../index.html" title="编译与安装" <a href="../text_generation/text_generation.html" title="Text generation Tutorial"
>previous</a> |</li> >previous</a> |</li>
<li class="nav-item nav-item-0"><a href="../../index.html">PaddlePaddle documentation</a> &raquo;</li> <li class="nav-item nav-item-0"><a href="../../index.html">PaddlePaddle documentation</a> &#187;</li>
<li class="nav-item nav-item-1"><a href="../index.html" >编译与安装</a> &raquo;</li> <li class="nav-item nav-item-1"><a href="../index.html" >Examples and demos</a> &#187;</li>
</ul> </ul>
</div> </div>
<div class="footer" role="contentinfo"> <div class="footer" role="contentinfo">
&copy; Copyright 2016, PaddlePaddle developers. &#169; Copyright 2016, PaddlePaddle developers.
Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.3.5. Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.4.6.
</div> </div>
</body> </body>
</html> </html>
\ No newline at end of file
doc/demo/sentiment_analysis/index.html
浏览文件 @ 22cb62db
...@@ -6,7 +6,7 @@ ...@@ -6,7 +6,7 @@
<head> <head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>Sentiment Analasis Tutorial &mdash; PaddlePaddle documentation</title> <title>Sentiment Analasis Tutorial &#8212; PaddlePaddle documentation</title>
<link rel="stylesheet" href="../../_static/classic.css" type="text/css" /> <link rel="stylesheet" href="../../_static/classic.css" type="text/css" />
<link rel="stylesheet" href="../../_static/pygments.css" type="text/css" /> <link rel="stylesheet" href="../../_static/pygments.css" type="text/css" />
...@@ -45,8 +45,8 @@ ...@@ -45,8 +45,8 @@
<li class="right" > <li class="right" >
<a href="../image_classification/image_classification.html" title="Image Classification Tutorial" <a href="../image_classification/image_classification.html" title="Image Classification Tutorial"
accesskey="P">previous</a> |</li> accesskey="P">previous</a> |</li>
<li class="nav-item nav-item-0"><a href="../../index.html">PaddlePaddle documentation</a> &raquo;</li> <li class="nav-item nav-item-0"><a href="../../index.html">PaddlePaddle documentation</a> &#187;</li>
<li class="nav-item nav-item-1"><a href="../index.html" accesskey="U">Examples and demos</a> &raquo;</li> <li class="nav-item nav-item-1"><a href="../index.html" accesskey="U">Examples and demos</a> &#187;</li>
</ul> </ul>
</div> </div>
...@@ -102,14 +102,11 @@ ...@@ -102,14 +102,11 @@
<div id="searchbox" style="display: none" role="search"> <div id="searchbox" style="display: none" role="search">
<h3>Quick search</h3> <h3>Quick search</h3>
<form class="search" action="../../search.html" method="get"> <form class="search" action="../../search.html" method="get">
<input type="text" name="q" /> <div><input type="text" name="q" /></div>
<input type="submit" value="Go" /> <div><input type="submit" value="Go" /></div>
<input type="hidden" name="check_keywords" value="yes" /> <input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" /> <input type="hidden" name="area" value="default" />
</form> </form>
<p class="searchtip" style="font-size: 90%">
Enter search terms or a module, class or function name.
</p>
</div> </div>
<script type="text/javascript">$('#searchbox').show(0);</script> <script type="text/javascript">$('#searchbox').show(0);</script>
</div> </div>
...@@ -131,13 +128,13 @@ ...@@ -131,13 +128,13 @@
<li class="right" > <li class="right" >
<a href="../image_classification/image_classification.html" title="Image Classification Tutorial" <a href="../image_classification/image_classification.html" title="Image Classification Tutorial"
>previous</a> |</li> >previous</a> |</li>
<li class="nav-item nav-item-0"><a href="../../index.html">PaddlePaddle documentation</a> &raquo;</li> <li class="nav-item nav-item-0"><a href="../../index.html">PaddlePaddle documentation</a> &#187;</li>
<li class="nav-item nav-item-1"><a href="../index.html" >Examples and demos</a> &raquo;</li> <li class="nav-item nav-item-1"><a href="../index.html" >Examples and demos</a> &#187;</li>
</ul> </ul>
</div> </div>
<div class="footer" role="contentinfo"> <div class="footer" role="contentinfo">
&copy; Copyright 2016, PaddlePaddle developers. &#169; Copyright 2016, PaddlePaddle developers.
Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.3.5. Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.4.6.
</div> </div>
</body> </body>
</html> </html>
\ No newline at end of file
doc/demo/sentiment_analysis/sentiment_analysis.html
浏览文件 @ 22cb62db
...@@ -6,7 +6,7 @@ ...@@ -6,7 +6,7 @@
<head> <head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>Sentiment Analysis Tutorial &mdash; PaddlePaddle documentation</title> <title>Sentiment Analysis Tutorial &#8212; PaddlePaddle documentation</title>
<link rel="stylesheet" href="../../_static/classic.css" type="text/css" /> <link rel="stylesheet" href="../../_static/classic.css" type="text/css" />
<link rel="stylesheet" href="../../_static/pygments.css" type="text/css" /> <link rel="stylesheet" href="../../_static/pygments.css" type="text/css" />
...@@ -45,9 +45,9 @@ ...@@ -45,9 +45,9 @@
<li class="right" > <li class="right" >
<a href="index.html" title="Sentiment Analasis Tutorial" <a href="index.html" title="Sentiment Analasis Tutorial"
accesskey="P">previous</a> |</li> accesskey="P">previous</a> |</li>
<li class="nav-item nav-item-0"><a href="../../index.html">PaddlePaddle documentation</a> &raquo;</li> <li class="nav-item nav-item-0"><a href="../../index.html">PaddlePaddle documentation</a> &#187;</li>
<li class="nav-item nav-item-1"><a href="../index.html" >Examples and demos</a> &raquo;</li> <li class="nav-item nav-item-1"><a href="../index.html" >Examples and demos</a> &#187;</li>
<li class="nav-item nav-item-2"><a href="index.html" accesskey="U">Sentiment Analasis Tutorial</a> &raquo;</li> <li class="nav-item nav-item-2"><a href="index.html" accesskey="U">Sentiment Analasis Tutorial</a> &#187;</li>
</ul> </ul>
</div> </div>
...@@ -67,12 +67,12 @@ ...@@ -67,12 +67,12 @@
<div class="section" id="imdb-data-introduction"> <div class="section" id="imdb-data-introduction">
<span id="imdb-data-introduction"></span><h3>IMDB Data Introduction<a class="headerlink" href="#imdb-data-introduction" title="Permalink to this headline"></a></h3> <span id="imdb-data-introduction"></span><h3>IMDB Data Introduction<a class="headerlink" href="#imdb-data-introduction" title="Permalink to this headline"></a></h3>
<p>Before training models, we need to preprocess the data and build a dictionary. First, you can use following script to download IMDB dataset and <a class="reference external" href="http://www.statmt.org/moses/">Moses</a> tool, which is a statistical machine translation system. We provide a data preprocessing script, which is capable of handling not only IMDB data, but also other user-defined data. In order to use the pre-written script, it needs to move labeled train and test samples to another path, which has been done in <code class="docutils literal"><span class="pre">get_imdb.sh</span></code>.</p> <p>Before training models, we need to preprocess the data and build a dictionary. First, you can use following script to download IMDB dataset and <a class="reference external" href="http://www.statmt.org/moses/">Moses</a> tool, which is a statistical machine translation system. We provide a data preprocessing script, which is capable of handling not only IMDB data, but also other user-defined data. In order to use the pre-written script, it needs to move labeled train and test samples to another path, which has been done in <code class="docutils literal"><span class="pre">get_imdb.sh</span></code>.</p>
<div class="highlight-python"><div class="highlight"><pre><span></span>cd demo/sentiment/data <div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">cd</span> <span class="n">demo</span><span class="o">/</span><span class="n">sentiment</span><span class="o">/</span><span class="n">data</span>
./get_imdb.sh <span class="o">./</span><span class="n">get_imdb</span><span class="o">.</span><span class="n">sh</span>
</pre></div> </pre></div>
</div> </div>
<p>If the data is obtained successfuly, you will see the following files at <code class="docutils literal"><span class="pre">./demo/sentiment/data</span></code>:</p> <p>If the data is obtained successfuly, you will see the following files at <code class="docutils literal"><span class="pre">./demo/sentiment/data</span></code>:</p>
<div class="highlight-python"><div class="highlight"><pre><span></span>aclImdb get_imdb.sh imdb mosesdecoder-master <div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">aclImdb</span> <span class="n">get_imdb</span><span class="o">.</span><span class="n">sh</span> <span class="n">imdb</span> <span class="n">mosesdecoder</span><span class="o">-</span><span class="n">master</span>
</pre></div> </pre></div>
</div> </div>
<ul class="simple"> <ul class="simple">
...@@ -81,7 +81,7 @@ ...@@ -81,7 +81,7 @@
<li>mosesdecoder-master: Moses tool.</li> <li>mosesdecoder-master: Moses tool.</li>
</ul> </ul>
<p>IMDB dataset contains 25,000 highly polar movie reviews for training, and 25,000 for testing. A negative review has a score ≤ 4 out of 10, and a positive review has a score ≥ 7 out of 10. After running <code class="docutils literal"><span class="pre">./get_imdb.sh</span></code>, we can find the dataset has the following structure in <code class="docutils literal"><span class="pre">aclImdb</span></code>.</p> <p>IMDB dataset contains 25,000 highly polar movie reviews for training, and 25,000 for testing. A negative review has a score ≤ 4 out of 10, and a positive review has a score ≥ 7 out of 10. After running <code class="docutils literal"><span class="pre">./get_imdb.sh</span></code>, we can find the dataset has the following structure in <code class="docutils literal"><span class="pre">aclImdb</span></code>.</p>
<div class="highlight-python"><div class="highlight"><pre><span></span>imdbEr.txt imdb.vocab README test train <div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">imdbEr</span><span class="o">.</span><span class="n">txt</span> <span class="n">imdb</span><span class="o">.</span><span class="n">vocab</span> <span class="n">README</span> <span class="n">test</span> <span class="n">train</span>
</pre></div> </pre></div>
</div> </div>
<ul class="simple"> <ul class="simple">
...@@ -92,7 +92,7 @@ ...@@ -92,7 +92,7 @@
<li>README: data documentation.</li> <li>README: data documentation.</li>
</ul> </ul>
<p>Both train and test set directory contains:</p> <p>Both train and test set directory contains:</p>
<div class="highlight-python"><div class="highlight"><pre><span></span>labeledBow.feat neg pos unsup unsupBow.feat urls_neg.txt urls_pos.txt urls_unsup.txt <div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">labeledBow</span><span class="o">.</span><span class="n">feat</span> <span class="n">neg</span> <span class="n">pos</span> <span class="n">unsup</span> <span class="n">unsupBow</span><span class="o">.</span><span class="n">feat</span> <span class="n">urls_neg</span><span class="o">.</span><span class="n">txt</span> <span class="n">urls_pos</span><span class="o">.</span><span class="n">txt</span> <span class="n">urls_unsup</span><span class="o">.</span><span class="n">txt</span>
</pre></div> </pre></div>
</div> </div>
<ul class="simple"> <ul class="simple">
...@@ -106,13 +106,13 @@ ...@@ -106,13 +106,13 @@
<div class="section" id="imdb-data-preparation"> <div class="section" id="imdb-data-preparation">
<span id="imdb-data-preparation"></span><h3>IMDB Data Preparation<a class="headerlink" href="#imdb-data-preparation" title="Permalink to this headline"></a></h3> <span id="imdb-data-preparation"></span><h3>IMDB Data Preparation<a class="headerlink" href="#imdb-data-preparation" title="Permalink to this headline"></a></h3>
<p>In this demo, we only use labled train and test set and not use imdb.vocab as dictionary. By default, dictionary is builded on train set. Train set is shuffled and test set is not. <code class="docutils literal"><span class="pre">tokenizer.perl</span></code> in Moses tool is used to tokenize the words and punctuation. Simply execute the following command to preprcess data.</p> <p>In this demo, we only use labled train and test set and not use imdb.vocab as dictionary. By default, dictionary is builded on train set. Train set is shuffled and test set is not. <code class="docutils literal"><span class="pre">tokenizer.perl</span></code> in Moses tool is used to tokenize the words and punctuation. Simply execute the following command to preprcess data.</p>
<div class="highlight-python"><div class="highlight"><pre><span></span>cd demo/sentiment/ <div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">cd</span> <span class="n">demo</span><span class="o">/</span><span class="n">sentiment</span><span class="o">/</span>
./preprocess.sh <span class="o">./</span><span class="n">preprocess</span><span class="o">.</span><span class="n">sh</span>
</pre></div> </pre></div>
</div> </div>
<p>preprocess.sh:</p> <p>preprocess.sh:</p>
<div class="highlight-python"><div class="highlight"><pre><span></span>data_dir=&quot;./data/imdb&quot; <div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">data_dir</span><span class="o">=</span><span class="s2">&quot;./data/imdb&quot;</span>
python preprocess.py -i data_dir <span class="n">python</span> <span class="n">preprocess</span><span class="o">.</span><span class="n">py</span> <span class="o">-</span><span class="n">i</span> <span class="n">data_dir</span>
</pre></div> </pre></div>
</div> </div>
<ul class="simple"> <ul class="simple">
...@@ -120,7 +120,7 @@ python preprocess.py -i data_dir ...@@ -120,7 +120,7 @@ python preprocess.py -i data_dir
<li>preprocess.py: preprocess script.</li> <li>preprocess.py: preprocess script.</li>
</ul> </ul>
<p>If running successfully, you will see <code class="docutils literal"><span class="pre">demo/sentiment/data/pre-imdb</span></code> directory as follows:</p> <p>If running successfully, you will see <code class="docutils literal"><span class="pre">demo/sentiment/data/pre-imdb</span></code> directory as follows:</p>
<div class="highlight-python"><div class="highlight"><pre><span></span>dict.txt labels.list test.list test_part_000 train.list train_part_000 <div class="highlight-default"><div class="highlight"><pre><span></span><span class="nb">dict</span><span class="o">.</span><span class="n">txt</span> <span class="n">labels</span><span class="o">.</span><span class="n">list</span> <span class="n">test</span><span class="o">.</span><span class="n">list</span> <span class="n">test_part_000</span> <span class="n">train</span><span class="o">.</span><span class="n">list</span> <span class="n">train_part_000</span>
</pre></div> </pre></div>
</div> </div>
<ul class="simple"> <ul class="simple">
...@@ -133,19 +133,19 @@ python preprocess.py -i data_dir ...@@ -133,19 +133,19 @@ python preprocess.py -i data_dir
<div class="section" id="user-defined-data-preparation"> <div class="section" id="user-defined-data-preparation">
<span id="user-defined-data-preparation"></span><h3>User-defined Data Preparation<a class="headerlink" href="#user-defined-data-preparation" title="Permalink to this headline"></a></h3> <span id="user-defined-data-preparation"></span><h3>User-defined Data Preparation<a class="headerlink" href="#user-defined-data-preparation" title="Permalink to this headline"></a></h3>
<p>If you perform other sentiment classifcation task, you can prepare data as follows. We have provided the scripts to build dictionary and preprocess data. So just organize data as follows.</p> <p>If you perform other sentiment classifcation task, you can prepare data as follows. We have provided the scripts to build dictionary and preprocess data. So just organize data as follows.</p>
<div class="highlight-python"><div class="highlight"><pre><span></span>dataset <div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">dataset</span>
|----train <span class="o">|----</span><span class="n">train</span>
| |----class1 <span class="o">|</span> <span class="o">|----</span><span class="n">class1</span>
| | |----text_files <span class="o">|</span> <span class="o">|</span> <span class="o">|----</span><span class="n">text_files</span>
| |----class2 <span class="o">|</span> <span class="o">|----</span><span class="n">class2</span>
| | |----text_files <span class="o">|</span> <span class="o">|</span> <span class="o">|----</span><span class="n">text_files</span>
| | ... <span class="o">|</span> <span class="o">|</span> <span class="o">...</span>
|----test <span class="o">|----</span><span class="n">test</span>
| |----class1 <span class="o">|</span> <span class="o">|----</span><span class="n">class1</span>
| | |----text_files <span class="o">|</span> <span class="o">|</span> <span class="o">|----</span><span class="n">text_files</span>
| |----class2 <span class="o">|</span> <span class="o">|----</span><span class="n">class2</span>
| | |----text_files <span class="o">|</span> <span class="o">|</span> <span class="o">|----</span><span class="n">text_files</span>
| | ... <span class="o">|</span> <span class="o">|</span> <span class="o">...</span>
</pre></div> </pre></div>
</div> </div>
<ul class="simple"> <ul class="simple">
...@@ -227,12 +227,12 @@ python preprocess.py -i data_dir ...@@ -227,12 +227,12 @@ python preprocess.py -i data_dir
</ul> </ul>
<p><strong>Training</strong></p> <p><strong>Training</strong></p>
<p>Install PaddlePaddle first if necessary. Then you can use script <code class="docutils literal"><span class="pre">train.sh</span></code> as follows to launch local training.</p> <p>Install PaddlePaddle first if necessary. Then you can use script <code class="docutils literal"><span class="pre">train.sh</span></code> as follows to launch local training.</p>
<div class="highlight-python"><div class="highlight"><pre><span></span>cd demo/sentiment/ <div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">cd</span> <span class="n">demo</span><span class="o">/</span><span class="n">sentiment</span><span class="o">/</span>
./train.sh <span class="o">./</span><span class="n">train</span><span class="o">.</span><span class="n">sh</span>
</pre></div> </pre></div>
</div> </div>
<p>train.sh:</p> <p>train.sh:</p>
<div class="highlight-python"><div class="highlight"><pre><span></span>config=trainer_config.py <div class="highlight-default"><div class="highlight"><pre><span></span>config=trainer_config.py
output=./model_output output=./model_output
paddle train --config=$config \ paddle train --config=$config \
--save_dir=$output \ --save_dir=$output \
...@@ -259,10 +259,10 @@ paddle train --config=$config \ ...@@ -259,10 +259,10 @@ paddle train --config=$config \
<li>--test_all_data_in_one_period=1: test all data every testing.</li> <li>--test_all_data_in_one_period=1: test all data every testing.</li>
</ul> </ul>
<p>If the run succeeds, the output log is saved in path of <code class="docutils literal"><span class="pre">demo/sentiment/train.log</span></code> and model is saved in path of <code class="docutils literal"><span class="pre">demo/sentiment/model_output/</span></code>. The output log is explained as follows.</p> <p>If the run succeeds, the output log is saved in path of <code class="docutils literal"><span class="pre">demo/sentiment/train.log</span></code> and model is saved in path of <code class="docutils literal"><span class="pre">demo/sentiment/model_output/</span></code>. The output log is explained as follows.</p>
<div class="highlight-python"><div class="highlight"><pre><span></span>Batch=20 samples=2560 AvgCost=0.681644 CurrentCost=0.681644 Eval: classification_error_evaluator=0.36875 CurrentEval: classification_error_evaluator=0.36875 <div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">Batch</span><span class="o">=</span><span class="mi">20</span> <span class="n">samples</span><span class="o">=</span><span class="mi">2560</span> <span class="n">AvgCost</span><span class="o">=</span><span class="mf">0.681644</span> <span class="n">CurrentCost</span><span class="o">=</span><span class="mf">0.681644</span> <span class="n">Eval</span><span class="p">:</span> <span class="n">classification_error_evaluator</span><span class="o">=</span><span class="mf">0.36875</span> <span class="n">CurrentEval</span><span class="p">:</span> <span class="n">classification_error_evaluator</span><span class="o">=</span><span class="mf">0.36875</span>
... <span class="o">...</span>
Pass=0 Batch=196 samples=25000 AvgCost=0.418964 Eval: classification_error_evaluator=0.1922 <span class="n">Pass</span><span class="o">=</span><span class="mi">0</span> <span class="n">Batch</span><span class="o">=</span><span class="mi">196</span> <span class="n">samples</span><span class="o">=</span><span class="mi">25000</span> <span class="n">AvgCost</span><span class="o">=</span><span class="mf">0.418964</span> <span class="n">Eval</span><span class="p">:</span> <span class="n">classification_error_evaluator</span><span class="o">=</span><span class="mf">0.1922</span>
Test samples=24999 cost=0.39297 Eval: classification_error_evaluator=0.149406 <span class="n">Test</span> <span class="n">samples</span><span class="o">=</span><span class="mi">24999</span> <span class="n">cost</span><span class="o">=</span><span class="mf">0.39297</span> <span class="n">Eval</span><span class="p">:</span> <span class="n">classification_error_evaluator</span><span class="o">=</span><span class="mf">0.149406</span>
</pre></div> </pre></div>
</div> </div>
<ul class="simple"> <ul class="simple">
...@@ -280,8 +280,8 @@ Test samples=24999 cost=0.39297 Eval: classification_error_evaluator=0.149406 ...@@ -280,8 +280,8 @@ Test samples=24999 cost=0.39297 Eval: classification_error_evaluator=0.149406
<div class="section" id="testing"> <div class="section" id="testing">
<span id="testing"></span><h2>Testing<a class="headerlink" href="#testing" title="Permalink to this headline"></a></h2> <span id="testing"></span><h2>Testing<a class="headerlink" href="#testing" title="Permalink to this headline"></a></h2>
<p>Testing means evaluating the labeled validation set using trained model.</p> <p>Testing means evaluating the labeled validation set using trained model.</p>
<div class="highlight-python"><div class="highlight"><pre><span></span>cd demo/sentiment <div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">cd</span> <span class="n">demo</span><span class="o">/</span><span class="n">sentiment</span>
./test.sh <span class="o">./</span><span class="n">test</span><span class="o">.</span><span class="n">sh</span>
</pre></div> </pre></div>
</div> </div>
<p>test.sh:</p> <p>test.sh:</p>
...@@ -311,19 +311,19 @@ paddle train --config<span class="o">=</span><span class="nv">$net_conf</span> < ...@@ -311,19 +311,19 @@ paddle train --config<span class="o">=</span><span class="nv">$net_conf</span> <
</pre></div> </pre></div>
</div> </div>
<p>The function <code class="docutils literal"><span class="pre">get_best_pass</span></code> gets the best model by classification error rate for testing. In this example, We use test dataset of IMDB as validation by default. Unlike training, it needs to specify <code class="docutils literal"><span class="pre">--job=test</span></code> and model path, namely <code class="docutils literal"><span class="pre">--model_list=$model_list</span></code> here. If running successfully, the log is saved in path of <code class="docutils literal"><span class="pre">demo/sentiment/test.log</span></code>. For example, in our test, the best model is <code class="docutils literal"><span class="pre">model_output/pass-00002</span></code>, the classification error is 0.115645 as follows.</p> <p>The function <code class="docutils literal"><span class="pre">get_best_pass</span></code> gets the best model by classification error rate for testing. In this example, We use test dataset of IMDB as validation by default. Unlike training, it needs to specify <code class="docutils literal"><span class="pre">--job=test</span></code> and model path, namely <code class="docutils literal"><span class="pre">--model_list=$model_list</span></code> here. If running successfully, the log is saved in path of <code class="docutils literal"><span class="pre">demo/sentiment/test.log</span></code>. For example, in our test, the best model is <code class="docutils literal"><span class="pre">model_output/pass-00002</span></code>, the classification error is 0.115645 as follows.</p>
<div class="highlight-python"><div class="highlight"><pre><span></span>Pass=0 samples=24999 AvgCost=0.280471 Eval: classification_error_evaluator=0.115645 <div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">Pass</span><span class="o">=</span><span class="mi">0</span> <span class="n">samples</span><span class="o">=</span><span class="mi">24999</span> <span class="n">AvgCost</span><span class="o">=</span><span class="mf">0.280471</span> <span class="n">Eval</span><span class="p">:</span> <span class="n">classification_error_evaluator</span><span class="o">=</span><span class="mf">0.115645</span>
</pre></div> </pre></div>
</div> </div>
</div> </div>
<div class="section" id="prediction"> <div class="section" id="prediction">
<span id="prediction"></span><h2>Prediction<a class="headerlink" href="#prediction" title="Permalink to this headline"></a></h2> <span id="prediction"></span><h2>Prediction<a class="headerlink" href="#prediction" title="Permalink to this headline"></a></h2>
<p><code class="docutils literal"><span class="pre">predict.py</span></code> provides a predicting interface. You should install python api of PaddlePaddle before using it. One example to predict unlabeled review of IMDB is as follows. Simply running:</p> <p><code class="docutils literal"><span class="pre">predict.py</span></code> provides a predicting interface. You should install python api of PaddlePaddle before using it. One example to predict unlabeled review of IMDB is as follows. Simply running:</p>
<div class="highlight-python"><div class="highlight"><pre><span></span>cd demo/sentiment <div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">cd</span> <span class="n">demo</span><span class="o">/</span><span class="n">sentiment</span>
./predict.sh <span class="o">./</span><span class="n">predict</span><span class="o">.</span><span class="n">sh</span>
</pre></div> </pre></div>
</div> </div>
<p>predict.sh:</p> <p>predict.sh:</p>
<div class="highlight-python"><div class="highlight"><pre><span></span>#Note the default model is pass-00002, you shold make sure the model path <div class="highlight-default"><div class="highlight"><pre><span></span>#Note the default model is pass-00002, you shold make sure the model path
#exists or change the mode path. #exists or change the mode path.
model=model_output/pass-00002/ model=model_output/pass-00002/
config=trainer_config.py config=trainer_config.py
...@@ -347,8 +347,8 @@ python predict.py \ ...@@ -347,8 +347,8 @@ python predict.py \
<p>Note you should make sure the default model path <code class="docutils literal"><span class="pre">model_output/pass-00002</span></code> <p>Note you should make sure the default model path <code class="docutils literal"><span class="pre">model_output/pass-00002</span></code>
exists or change the model path.</p> exists or change the model path.</p>
<p>Predicting result of this example:</p> <p>Predicting result of this example:</p>
<div class="highlight-python"><div class="highlight"><pre><span></span>Loading parameters from model_output/pass-00002/ <div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">Loading</span> <span class="n">parameters</span> <span class="kn">from</span> <span class="nn">model_output</span><span class="o">/</span><span class="k">pass</span><span class="o">-</span><span class="mi">00002</span><span class="o">/</span>
./data/aclImdb/test/pos/10014_7.txt: predicting label is pos <span class="o">./</span><span class="n">data</span><span class="o">/</span><span class="n">aclImdb</span><span class="o">/</span><span class="n">test</span><span class="o">/</span><span class="n">pos</span><span class="o">/</span><span class="mi">10014</span><span class="n">_7</span><span class="o">.</span><span class="n">txt</span><span class="p">:</span> <span class="n">predicting</span> <span class="n">label</span> <span class="ow">is</span> <span class="n">pos</span>
</pre></div> </pre></div>
</div> </div>
<p>We sincerely appreciate your interest and welcome your contributions.</p> <p>We sincerely appreciate your interest and welcome your contributions.</p>
...@@ -406,14 +406,11 @@ exists or change the model path.</p> ...@@ -406,14 +406,11 @@ exists or change the model path.</p>
<div id="searchbox" style="display: none" role="search"> <div id="searchbox" style="display: none" role="search">
<h3>Quick search</h3> <h3>Quick search</h3>
<form class="search" action="../../search.html" method="get"> <form class="search" action="../../search.html" method="get">
<input type="text" name="q" /> <div><input type="text" name="q" /></div>
<input type="submit" value="Go" /> <div><input type="submit" value="Go" /></div>
<input type="hidden" name="check_keywords" value="yes" /> <input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" /> <input type="hidden" name="area" value="default" />
</form> </form>
<p class="searchtip" style="font-size: 90%">
Enter search terms or a module, class or function name.
</p>
</div> </div>
<script type="text/javascript">$('#searchbox').show(0);</script> <script type="text/javascript">$('#searchbox').show(0);</script>
</div> </div>
...@@ -435,14 +432,14 @@ exists or change the model path.</p> ...@@ -435,14 +432,14 @@ exists or change the model path.</p>
<li class="right" > <li class="right" >
<a href="index.html" title="Sentiment Analasis Tutorial" <a href="index.html" title="Sentiment Analasis Tutorial"
>previous</a> |</li> >previous</a> |</li>
<li class="nav-item nav-item-0"><a href="../../index.html">PaddlePaddle documentation</a> &raquo;</li> <li class="nav-item nav-item-0"><a href="../../index.html">PaddlePaddle documentation</a> &#187;</li>
<li class="nav-item nav-item-1"><a href="../index.html" >Examples and demos</a> &raquo;</li> <li class="nav-item nav-item-1"><a href="../index.html" >Examples and demos</a> &#187;</li>
<li class="nav-item nav-item-2"><a href="index.html" >Sentiment Analasis Tutorial</a> &raquo;</li> <li class="nav-item nav-item-2"><a href="index.html" >Sentiment Analasis Tutorial</a> &#187;</li>
</ul> </ul>
</div> </div>
<div class="footer" role="contentinfo"> <div class="footer" role="contentinfo">
&copy; Copyright 2016, PaddlePaddle developers. &#169; Copyright 2016, PaddlePaddle developers.
Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.3.5. Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.4.6.
</div> </div>
</body> </body>
</html> </html>
\ No newline at end of file
doc/demo/text_generation/index.html
浏览文件 @ 22cb62db
...@@ -6,7 +6,7 @@ ...@@ -6,7 +6,7 @@
<head> <head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>Text Generation Tutorial &mdash; PaddlePaddle documentation</title> <title>Text Generation Tutorial &#8212; PaddlePaddle documentation</title>
<link rel="stylesheet" href="../../_static/classic.css" type="text/css" /> <link rel="stylesheet" href="../../_static/classic.css" type="text/css" />
<link rel="stylesheet" href="../../_static/pygments.css" type="text/css" /> <link rel="stylesheet" href="../../_static/pygments.css" type="text/css" />
...@@ -45,8 +45,8 @@ ...@@ -45,8 +45,8 @@
<li class="right" > <li class="right" >
<a href="../sentiment_analysis/sentiment_analysis.html" title="Sentiment Analysis Tutorial" <a href="../sentiment_analysis/sentiment_analysis.html" title="Sentiment Analysis Tutorial"
accesskey="P">previous</a> |</li> accesskey="P">previous</a> |</li>
<li class="nav-item nav-item-0"><a href="../../index.html">PaddlePaddle documentation</a> &raquo;</li> <li class="nav-item nav-item-0"><a href="../../index.html">PaddlePaddle documentation</a> &#187;</li>
<li class="nav-item nav-item-1"><a href="../index.html" accesskey="U">Examples and demos</a> &raquo;</li> <li class="nav-item nav-item-1"><a href="../index.html" accesskey="U">Examples and demos</a> &#187;</li>
</ul> </ul>
</div> </div>
...@@ -112,14 +112,11 @@ ...@@ -112,14 +112,11 @@
<div id="searchbox" style="display: none" role="search"> <div id="searchbox" style="display: none" role="search">
<h3>Quick search</h3> <h3>Quick search</h3>
<form class="search" action="../../search.html" method="get"> <form class="search" action="../../search.html" method="get">
<input type="text" name="q" /> <div><input type="text" name="q" /></div>
<input type="submit" value="Go" /> <div><input type="submit" value="Go" /></div>
<input type="hidden" name="check_keywords" value="yes" /> <input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" /> <input type="hidden" name="area" value="default" />
</form> </form>
<p class="searchtip" style="font-size: 90%">
Enter search terms or a module, class or function name.
</p>
</div> </div>
<script type="text/javascript">$('#searchbox').show(0);</script> <script type="text/javascript">$('#searchbox').show(0);</script>
</div> </div>
...@@ -141,13 +138,13 @@ ...@@ -141,13 +138,13 @@
<li class="right" > <li class="right" >
<a href="../sentiment_analysis/sentiment_analysis.html" title="Sentiment Analysis Tutorial" <a href="../sentiment_analysis/sentiment_analysis.html" title="Sentiment Analysis Tutorial"
>previous</a> |</li> >previous</a> |</li>
<li class="nav-item nav-item-0"><a href="../../index.html">PaddlePaddle documentation</a> &raquo;</li> <li class="nav-item nav-item-0"><a href="../../index.html">PaddlePaddle documentation</a> &#187;</li>
<li class="nav-item nav-item-1"><a href="../index.html" >Examples and demos</a> &raquo;</li> <li class="nav-item nav-item-1"><a href="../index.html" >Examples and demos</a> &#187;</li>
</ul> </ul>
</div> </div>
<div class="footer" role="contentinfo"> <div class="footer" role="contentinfo">
&copy; Copyright 2016, PaddlePaddle developers. &#169; Copyright 2016, PaddlePaddle developers.
Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.3.5. Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.4.6.
</div> </div>
</body> </body>
</html> </html>
\ No newline at end of file
doc/dev/new_layer/index.html
浏览文件 @ 22cb62db
...@@ -6,7 +6,7 @@ ...@@ -6,7 +6,7 @@
<head> <head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>Writing New Layers &mdash; PaddlePaddle documentation</title> <title>Writing New Layers &#8212; PaddlePaddle documentation</title>
<link rel="stylesheet" href="../../_static/classic.css" type="text/css" /> <link rel="stylesheet" href="../../_static/classic.css" type="text/css" />
<link rel="stylesheet" href="../../_static/pygments.css" type="text/css" /> <link rel="stylesheet" href="../../_static/pygments.css" type="text/css" />
...@@ -44,7 +44,7 @@ ...@@ -44,7 +44,7 @@
<li class="right" > <li class="right" >
<a href="../../source/gserver/layers/layer.html" title="Base" <a href="../../source/gserver/layers/layer.html" title="Base"
accesskey="P">previous</a> |</li> accesskey="P">previous</a> |</li>
<li class="nav-item nav-item-0"><a href="../../index.html">PaddlePaddle documentation</a> &raquo;</li> <li class="nav-item nav-item-0"><a href="../../index.html">PaddlePaddle documentation</a> &#187;</li>
</ul> </ul>
</div> </div>
...@@ -88,14 +88,11 @@ ...@@ -88,14 +88,11 @@
<div id="searchbox" style="display: none" role="search"> <div id="searchbox" style="display: none" role="search">
<h3>Quick search</h3> <h3>Quick search</h3>
<form class="search" action="../../search.html" method="get"> <form class="search" action="../../search.html" method="get">
<input type="text" name="q" /> <div><input type="text" name="q" /></div>
<input type="submit" value="Go" /> <div><input type="submit" value="Go" /></div>
<input type="hidden" name="check_keywords" value="yes" /> <input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" /> <input type="hidden" name="area" value="default" />
</form> </form>
<p class="searchtip" style="font-size: 90%">
Enter search terms or a module, class or function name.
</p>
</div> </div>
<script type="text/javascript">$('#searchbox').show(0);</script> <script type="text/javascript">$('#searchbox').show(0);</script>
</div> </div>
...@@ -117,12 +114,12 @@ ...@@ -117,12 +114,12 @@
<li class="right" > <li class="right" >
<a href="../../source/gserver/layers/layer.html" title="Base" <a href="../../source/gserver/layers/layer.html" title="Base"
>previous</a> |</li> >previous</a> |</li>
<li class="nav-item nav-item-0"><a href="../../index.html">PaddlePaddle documentation</a> &raquo;</li> <li class="nav-item nav-item-0"><a href="../../index.html">PaddlePaddle documentation</a> &#187;</li>
</ul> </ul>
</div> </div>
<div class="footer" role="contentinfo"> <div class="footer" role="contentinfo">
&copy; Copyright 2016, PaddlePaddle developers. &#169; Copyright 2016, PaddlePaddle developers.
Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.3.5. Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.4.6.
</div> </div>
</body> </body>
</html> </html>
\ No newline at end of file
doc/dev/new_layer/new_layer.html
浏览文件 @ 22cb62db
...@@ -6,7 +6,7 @@ ...@@ -6,7 +6,7 @@
<head> <head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>Writing New Layers &mdash; PaddlePaddle documentation</title> <title>Writing New Layers &#8212; PaddlePaddle documentation</title>
<link rel="stylesheet" href="../../_static/classic.css" type="text/css" /> <link rel="stylesheet" href="../../_static/classic.css" type="text/css" />
<link rel="stylesheet" href="../../_static/pygments.css" type="text/css" /> <link rel="stylesheet" href="../../_static/pygments.css" type="text/css" />
...@@ -45,8 +45,8 @@ ...@@ -45,8 +45,8 @@
<li class="right" > <li class="right" >
<a href="index.html" title="Writing New Layers" <a href="index.html" title="Writing New Layers"
accesskey="P">previous</a> |</li> accesskey="P">previous</a> |</li>
<li class="nav-item nav-item-0"><a href="../../index.html">PaddlePaddle documentation</a> &raquo;</li> <li class="nav-item nav-item-0"><a href="../../index.html">PaddlePaddle documentation</a> &#187;</li>
<li class="nav-item nav-item-1"><a href="index.html" accesskey="U">Writing New Layers</a> &raquo;</li> <li class="nav-item nav-item-1"><a href="index.html" accesskey="U">Writing New Layers</a> &#187;</li>
</ul> </ul>
</div> </div>
...@@ -466,14 +466,11 @@ add_test<span class="o">(</span>NAME test_FCGrad ...@@ -466,14 +466,11 @@ add_test<span class="o">(</span>NAME test_FCGrad
<div id="searchbox" style="display: none" role="search"> <div id="searchbox" style="display: none" role="search">
<h3>Quick search</h3> <h3>Quick search</h3>
<form class="search" action="../../search.html" method="get"> <form class="search" action="../../search.html" method="get">
<input type="text" name="q" /> <div><input type="text" name="q" /></div>
<input type="submit" value="Go" /> <div><input type="submit" value="Go" /></div>
<input type="hidden" name="check_keywords" value="yes" /> <input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" /> <input type="hidden" name="area" value="default" />
</form> </form>
<p class="searchtip" style="font-size: 90%">
Enter search terms or a module, class or function name.
</p>
</div> </div>
<script type="text/javascript">$('#searchbox').show(0);</script> <script type="text/javascript">$('#searchbox').show(0);</script>
</div> </div>
...@@ -495,13 +492,13 @@ add_test<span class="o">(</span>NAME test_FCGrad ...@@ -495,13 +492,13 @@ add_test<span class="o">(</span>NAME test_FCGrad
<li class="right" > <li class="right" >
<a href="index.html" title="Writing New Layers" <a href="index.html" title="Writing New Layers"
>previous</a> |</li> >previous</a> |</li>
<li class="nav-item nav-item-0"><a href="../../index.html">PaddlePaddle documentation</a> &raquo;</li> <li class="nav-item nav-item-0"><a href="../../index.html">PaddlePaddle documentation</a> &#187;</li>
<li class="nav-item nav-item-1"><a href="index.html" >Writing New Layers</a> &raquo;</li> <li class="nav-item nav-item-1"><a href="index.html" >Writing New Layers</a> &#187;</li>
</ul> </ul>
</div> </div>
<div class="footer" role="contentinfo"> <div class="footer" role="contentinfo">
&copy; Copyright 2016, PaddlePaddle developers. &#169; Copyright 2016, PaddlePaddle developers.
Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.3.5. Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.4.6.
</div> </div>
</body> </body>
</html> </html>
\ No newline at end of file
doc/genindex.html
浏览文件 @ 22cb62db
此差异已折叠。
doc/index.html
浏览文件 @ 22cb62db
此差异已折叠。
doc/layer.html
浏览文件 @ 22cb62db
此差异已折叠。
doc/objects.inv
浏览文件 @ 22cb62db
无法预览此类型文件
doc/py-modindex.html
浏览文件 @ 22cb62db
此差异已折叠。
doc/search.html
浏览文件 @ 22cb62db
...@@ -6,7 +6,7 @@ ...@@ -6,7 +6,7 @@
<head> <head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>Search &mdash; PaddlePaddle documentation</title> <title>Search &#8212; PaddlePaddle documentation</title>
<link rel="stylesheet" href="_static/classic.css" type="text/css" /> <link rel="stylesheet" href="_static/classic.css" type="text/css" />
<link rel="stylesheet" href="_static/pygments.css" type="text/css" /> <link rel="stylesheet" href="_static/pygments.css" type="text/css" />
...@@ -44,7 +44,7 @@ ...@@ -44,7 +44,7 @@
<li class="right" > <li class="right" >
<a href="py-modindex.html" title="Python Module Index" <a href="py-modindex.html" title="Python Module Index"
>modules</a> |</li> >modules</a> |</li>
<li class="nav-item nav-item-0"><a href="index.html">PaddlePaddle documentation</a> &raquo;</li> <li class="nav-item nav-item-0"><a href="index.html">PaddlePaddle documentation</a> &#187;</li>
</ul> </ul>
</div> </div>
...@@ -95,12 +95,12 @@ ...@@ -95,12 +95,12 @@
<li class="right" > <li class="right" >
<a href="py-modindex.html" title="Python Module Index" <a href="py-modindex.html" title="Python Module Index"
>modules</a> |</li> >modules</a> |</li>
<li class="nav-item nav-item-0"><a href="index.html">PaddlePaddle documentation</a> &raquo;</li> <li class="nav-item nav-item-0"><a href="index.html">PaddlePaddle documentation</a> &#187;</li>
</ul> </ul>
</div> </div>
<div class="footer" role="contentinfo"> <div class="footer" role="contentinfo">
&copy; Copyright 2016, PaddlePaddle developers. &#169; Copyright 2016, PaddlePaddle developers.
Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.3.5. Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.4.6.
</div> </div>
</body> </body>
</html> </html>
\ No newline at end of file
doc/searchindex.js
浏览文件 @ 22cb62db
此差异已折叠。
doc/source/api/api.html
浏览文件 @ 22cb62db
此差异已折叠。
doc/source/cuda/cuda/cuda.html
浏览文件 @ 22cb62db
此差异已折叠。
doc/source/cuda/cuda/index.html
浏览文件 @ 22cb62db
此差异已折叠。
doc/source/cuda/matrix/index.html
浏览文件 @ 22cb62db
此差异已折叠。
doc/source/cuda/matrix/matrix.html
浏览文件 @ 22cb62db
此差异已折叠。
doc/source/cuda/rnn/index.html
浏览文件 @ 22cb62db
此差异已折叠。
doc/source/cuda/rnn/rnn.html
浏览文件 @ 22cb62db
此差异已折叠。
doc/source/cuda/utils/index.html
浏览文件 @ 22cb62db
此差异已折叠。
doc/source/cuda/utils/utils.html
浏览文件 @ 22cb62db
此差异已折叠。
doc/source/index.html
浏览文件 @ 22cb62db
此差异已折叠。
doc/source/math/matrix/index.html
浏览文件 @ 22cb62db
此差异已折叠。
doc/source/math/matrix/matrix.html
浏览文件 @ 22cb62db
此差异已折叠。
doc/source/math/utils/index.html
浏览文件 @ 22cb62db
此差异已折叠。
doc/source/math/utils/utils.html
浏览文件 @ 22cb62db
此差异已折叠。
doc/source/trainer/trainer.html
浏览文件 @ 22cb62db
此差异已折叠。
doc/source/utils/enum.html
浏览文件 @ 22cb62db
此差异已折叠。
doc/source/utils/lock.html
浏览文件 @ 22cb62db
此差异已折叠。
doc/source/utils/queue.html
浏览文件 @ 22cb62db
此差异已折叠。
doc/source/utils/thread.html
浏览文件 @ 22cb62db
此差异已折叠。
doc/ui/cmd_argument/use_case.html
浏览文件 @ 22cb62db
此差异已折叠。
doc/ui/data_provider/index.html
浏览文件 @ 22cb62db
此差异已折叠。
doc/ui/index.html
浏览文件 @ 22cb62db
此差异已折叠。
doc_cn/.buildinfo
浏览文件 @ 22cb62db
此差异已折叠。
doc_cn/_sources/algorithm/rnn/rnn.txt 已删除 100644 → 0
浏览文件 @ a0ec673b
此差异已折叠。
doc_cn/_sources/cluster/index.txt
浏览文件 @ 22cb62db
此差异已折叠。
doc_cn/_sources/demo/imagenet_model/index.txt 已删除 100644 → 0
浏览文件 @ a0ec673b
此差异已折叠。
doc_cn/_sources/demo/index.txt
浏览文件 @ 22cb62db
此差异已折叠。
doc_cn/_sources/dev/new_layer/index.txt 已删除 100644 → 0
浏览文件 @ a0ec673b
此差异已折叠。
doc_cn/_sources/index.txt
浏览文件 @ 22cb62db
此差异已折叠。
doc_cn/_sources/ui/model.txt 已删除 100644 → 0
浏览文件 @ a0ec673b
此差异已折叠。
doc_cn/_static/basic.css
浏览文件 @ 22cb62db
此差异已折叠。
doc_cn/_static/doctools.js
浏览文件 @ 22cb62db
此差异已折叠。
doc_cn/_static/searchtools.js
浏览文件 @ 22cb62db
此差异已折叠。
doc_cn/_static/websupport.js
浏览文件 @ 22cb62db
此差异已折叠。
doc_cn/algorithm/rnn/rnn.html 已删除 100644 → 0
浏览文件 @ a0ec673b
此差异已折叠。
doc_cn/build_and_install/index.html
浏览文件 @ 22cb62db
此差异已折叠。
doc_cn/cluster/index.html
浏览文件 @ 22cb62db
此差异已折叠。
doc_cn/demo/embedding_model/index.html 已删除 100644 → 0
浏览文件 @ a0ec673b
此差异已折叠。
doc_cn/demo/image_classification/index.html 已删除 100644 → 0
浏览文件 @ a0ec673b
此差异已折叠。
doc_cn/demo/imagenet_model/index.html 已删除 100644 → 0
浏览文件 @ a0ec673b
此差异已折叠。
doc_cn/demo/index.html
浏览文件 @ 22cb62db
此差异已折叠。
doc_cn/demo/quick_start/index.html
浏览文件 @ 22cb62db
此差异已折叠。
doc_cn/demo/semantic_role_labeling/index.html 已删除 100644 → 0
浏览文件 @ a0ec673b
此差异已折叠。
doc_cn/demo/sentiment_analysis/index.html 已删除 100644 → 0
浏览文件 @ a0ec673b
此差异已折叠。
doc_cn/demo/text_generation/index.html 已删除 100644 → 0
浏览文件 @ a0ec673b
此差异已折叠。
doc_cn/dev/new_layer/index.html 已删除 100644 → 0
浏览文件 @ a0ec673b
此差异已折叠。
doc_cn/genindex.html
浏览文件 @ 22cb62db
此差异已折叠。
doc_cn/index.html
浏览文件 @ 22cb62db
此差异已折叠。
doc_cn/objects.inv
浏览文件 @ 22cb62db
此差异已折叠。
doc_cn/search.html
浏览文件 @ 22cb62db
此差异已折叠。
doc_cn/searchindex.js
浏览文件 @ 22cb62db
此差异已折叠。
doc_cn/ui/cmd/dump_config.html
浏览文件 @ 22cb62db
此差异已折叠。
doc_cn/ui/cmd/index.html
浏览文件 @ 22cb62db
此差异已折叠。
doc_cn/ui/cmd/make_diagram.html
浏览文件 @ 22cb62db
此差异已折叠。
doc_cn/ui/cmd/merge_model.html
浏览文件 @ 22cb62db
此差异已折叠。
doc_cn/ui/cmd/paddle_pserver.html
浏览文件 @ 22cb62db
此差异已折叠。
doc_cn/ui/cmd/paddle_train.html
浏览文件 @ 22cb62db
此差异已折叠。
doc_cn/ui/cmd/paddle_version.html
浏览文件 @ 22cb62db
此差异已折叠。
doc_cn/ui/data_provider/index.html
浏览文件 @ 22cb62db
此差异已折叠。
doc_cn/ui/index.html
浏览文件 @ 22cb62db
此差异已折叠。
doc_cn/ui/model.html 已删除 100644 → 0
浏览文件 @ a0ec673b
此差异已折叠。
Markdown is supported
0% 或 .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑！
想要评论请 注册 或