Skip to content

P
Paddle

Crayon鑫 / Paddle
与 Fork 源项目一致

Fork自 PaddlePaddle / Paddle

通知 1
Star 1
Fork 0
P
Paddle
提交 57e43320 编写于 作者: T Travis CI
浏览文件
操作

Deploy to GitHub Pages: 5ad92e05a0bf2ef096831964f09e9d238be2211d

上级 438f7337
展开全部 隐藏空白更改
内联 并排
Showing with 661 addition and 3076 deletion
doc/_sources/build/build_from_source.txt
浏览文件 @ 57e43320
Installing from Sources
==========================
=================
* [1. Download and Setup](#download)
* [2. Requirements](#requirements)
......@@ -28,26 +28,51 @@ To compile the source code, your computer must be equipped with GCC >=4.6 or Cla
PaddlePaddle supports some build options. To enable it, first you need to install the related libraries.
<html>
<table>
<thead>
<tr>
<th scope="col" class="left">Optional</th>
<th scope="col" class="left">Description</th>
</tr>
</thead>
<tbody>
<tr><td class="left">WITH_GPU</td><td class="left">Compile with GPU mode.</td></tr>
<tr><td class="left">WITH_DOUBLE</td><td class="left">Compile with double precision floating-point, default: single precision.</td></tr>
<tr><td class="left">WITH_GLOG</td><td class="left">Compile with glog. If not found, default: an internal log implementation.</td></tr>
<tr><td class="left">WITH_GFLAGS</td><td class="left">Compile with gflags. If not found, default: an internal flag implementation.</td></tr>
<tr><td class="left">WITH_TESTING</td><td class="left">Compile with gtest for PaddlePaddle's unit testing.</td></tr>
<tr><td class="left">WITH_DOC</td><td class="left"> Compile to generate PaddlePaddle's docs, default: disabled (OFF).</td></tr>
<tr><td class="left">WITH_SWIG_PY</td><td class="left">Compile with python predict API, default: disabled (OFF).</td></tr>
<tr><td class="left">WITH_STYLE_CHECK</td><td class="left">Compile with code style check, default: enabled (ON).</td></tr>
</tbody>
<style type="text/css">
.tg {border-collapse:collapse;border-spacing:0;border-color:#ccc;}
.tg td{font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:0px;overflow:hidden;word-break:normal;border-color:#ccc;color:#333;background-color:#fff;border-top-width:1px;border-bottom-width:1px;}
.tg th{font-family:Arial, sans-serif;font-size:14px;font-weight:normal;padding:10px 5px;border-style:solid;border-width:0px;overflow:hidden;word-break:normal;border-color:#ccc;color:#333;background-color:#f0f0f0;border-top-width:1px;border-bottom-width:1px;}
.tg .tg-yw4l{vertical-align:top}
.tg .tg-9hbo{font-weight:bold;vertical-align:top}
</style>
<table class="tg">
<tr>
<th class="tg-yw4l">Optional</th>
<th class="tg-yw4l">Description</th>
</tr>
<tr>
<td class="tg-9hbo">WITH_GPU</td>
<td class="tg-yw4l">Compile with GPU mode.</td>
</tr>
<tr>
<td class="tg-9hbo">WITH_DOUBLE</td>
<td class="tg-yw4l">Compile with double precision floating-point, default: single precision.</td>
</tr>
<tr>
<td class="tg-9hbo">WITH_GLOG</td>
<td class="tg-yw4l">Compile with glog. If not found, default: an internal log implementation.</td>
</tr>
<tr>
<td class="tg-9hbo">WITH_GFLAGS</td>
<td class="tg-yw4l">Compile with gflags. If not found, default: an internal flag implementation.</td>
</tr>
<tr>
<td class="tg-9hbo">WITH_TESTING</td>
<td class="tg-yw4l">Compile with gtest for PaddlePaddle's unit testing.</td>
</tr>
<tr>
<td class="tg-9hbo">WITH_DOC</td>
<td class="tg-yw4l">Compile to generate PaddlePaddle's docs, default: disabled (OFF)</td>
</tr>
<tr>
<td class="tg-9hbo">WITH_SWIG_PY</td>
<td class="tg-yw4l">Compile with python predict API, default: disabled (OFF).</td>
</tr>
<tr>
<td class="tg-9hbo">WITH_STYLE_CHECK</td>
<td class="tg-yw4l">Compile with code style check, default: enabled (ON).</td>
</tr>
</table>
</html>
**Note:**
- The GPU version works best with Cuda Toolkit 7.5 and cuDNN v5.
......@@ -309,4 +334,4 @@ It may require sudo privileges:
sudo pip install <path to install>/opt/paddle/share/wheels/*.whl
# or just run
sudo paddle version
```
```
\ No newline at end of file
doc/_sources/build/contribute_to_paddle.txt
浏览文件 @ 57e43320
......@@ -99,7 +99,3 @@ git pull --rebase upstream HEAD
git push -f origin HEAD
```
Now your Pull Request is updated with the latest version.
## Revise your pull request
When you revise your pull request according to reviewer's comments, please use 'git commit' instead of 'git commit --amend' to commit your changes so that the reviewers can see the difference between the new pull requrest and the old pull request.
doc/_sources/build/docker_install.txt
浏览文件 @ 57e43320
Docker installation guide
==========================
====================
PaddlePaddle provides some pre-compiled binary, including Docker images, ubuntu deb packages. It is welcomed to contributed more installation package of different linux distribution (such as ubuntu, centos, debian, gentoo and so on). We recommend to use Docker images to deploy PaddlePaddle.
## Docker installation
PaddlePaddle provide the `Docker <https://www.docker.com/>`_ image. `Docker`_ is a lightweight container utilities. The performance of PaddlePaddle in `Docker`_ container is basically as same as run it in a normal linux. The `Docker`_ is a very convenient way to deliver the binary release for linux programs.
Docker is a tool designed to make it easier to create, deploy, and run applications by using containers.
.. note::
### PaddlePaddle Docker images
There are six Docker images:
The `Docker`_ image is the recommended way to run PaddlePaddle
- paddledev/paddle:cpu-latest: PaddlePaddle CPU binary image.
- paddledev/paddle:gpu-latest: PaddlePaddle GPU binary image.
- paddledev/paddle:cpu-devel-latest: PaddlePaddle CPU binary image plus source code.
- paddledev/paddle:gpu-devel-latest: PaddlePaddle GPU binary image plus source code.
- paddledev/paddle:cpu-demo-latest: PaddlePaddle CPU binary image plus source code and demo
- paddledev/paddle:gpu-demo-latest: PaddlePaddle GPU binary image plus source code and demo
PaddlePaddle Docker images
--------------------------
Tags with latest will be replaced by a released version.
There are 12 `images <https://hub.docker.com/r/paddledev/paddle/tags/>`_ for PaddlePaddle, and the name is :code:`paddle-dev/paddle`, tags are\:
+-----------------+------------------+------------------------+-----------------------+
| | normal | devel | demo |
+=================+==================+========================+=======================+
| CPU | cpu-latest | cpu-devel-latest | cpu-demo-latest |
+-----------------+------------------+------------------------+-----------------------+
| GPU | gpu-latest | gpu-devel-latest | gpu-demo-latest |
+-----------------+------------------+------------------------+-----------------------+
| CPU WITHOUT AVX | cpu-noavx-latest | cpu-devel-noavx-latest | cpu-demo-noavx-latest |
+-----------------+------------------+------------------------+-----------------------+
| GPU WITHOUT AVX | gpu-noavx-latest | gpu-devel-noavx-latest | gpu-demo-noavx-latest |
+-----------------+------------------+------------------------+-----------------------+
And the three columns are:
* normal\: The docker image only contains binary of PaddlePaddle.
* devel\: The docker image contains PaddlePaddle binary, source code and essential build environment.
* demo\: The docker image contains the dependencies to run PaddlePaddle demo.
And the four rows are:
* CPU\: CPU Version. Support CPU which has :code:`AVX` instructions.
* GPU\: GPU Version. Support GPU, and cpu has :code:`AVX` instructions.
* CPU WITHOUT AVX\: CPU Version, which support most CPU even doesn't have :code:`AVX` instructions.
* GPU WITHOUT AVX\: GPU Version, which support most CPU even doesn't have :code:`AVX` instructions.
User can choose any version depends on machine. The following script can help you to detect your CPU support :code:`AVX` or not.
.. code-block:: bash
if cat /proc/cpuinfo | grep -q avx ; then echo "Support AVX"; else echo "Not support AVX"; fi
If the output is :code:`Support AVX`, then you can choose the AVX version of PaddlePaddle, otherwise, you need select :code:`noavx` version of PaddlePaddle. For example, the CPU develop version of PaddlePaddle is :code:`paddle-dev/paddle:cpu-devel-latest`.
The PaddlePaddle images don't contain any entry command. You need to write your entry command to use this image. See :code:`Remote Access` part or just use following command to run a :code:`bash`
.. code-block:: bash
docker run -it paddledev/paddle:cpu-latest /bin/bash
Download and Run Docker images
------------------------------
### Download and Run Docker images
You have to install Docker in your machine which has linux kernel version 3.10+ first. You can refer to the official guide https://docs.docker.com/engine/installation/ for further information.
You can use :code:`docker pull ` to download images first, or just launch a container with :code:`docker run` \:
.. code-block:: bash
docker run -it paddledev/paddle:cpu-latest
You can use ```docker pull ```to download images first, or just launch a container with ```docker run```:
```bash
docker run -it paddledev/paddle:cpu-latest
```
If you want to launch container with GPU support, you need to set some environment variables at the same time:
.. code-block:: bash
export CUDA_SO="$(\ls /usr/lib64/libcuda* | xargs -I{} echo '-v {}:{}') $(\ls /usr/lib64/libnvidia* | xargs -I{} echo '-v {}:{}')"
export DEVICES=$(\ls /dev/nvidia* | xargs -I{} echo '--device {}:{}')
docker run ${CUDA_SO} ${DEVICES} -it paddledev/paddle:gpu-latest
```bash
export CUDA_SO="$(\ls /usr/lib64/libcuda* | xargs -I{} echo '-v {}:{}') $(\ls /usr/lib64/libnvidia* | xargs -I{} echo '-v {}:{}"
export DEVICES=$(\ls /dev/nvidia* | xargs -I{} echo '--device {}:{}')
docker run -it paddledev/paddle:gpu-latest
```
### Notice
Some notes for docker
---------------------
Performance
+++++++++++
#### Performance
Since Docker is based on the lightweight virtual containers, the CPU computing performance maintains well. And GPU driver and equipments are all mapped to the container, so the GPU computing performance would not be seriously affected.
......@@ -87,36 +45,47 @@ If you use high performance nic, such as RDMA(RoCE 40GbE or IB 56GbE), Ethernet(
Remote access
+++++++++++++
#### Remote access
If you want to enable ssh access background, you need to build an image by yourself. Please refer to official guide https://docs.docker.com/engine/reference/builder/ for further information.
Following is a simple Dockerfile with ssh:
```bash
FROM paddledev/paddle
.. literalinclude:: ../../doc_cn/build_and_install/install/paddle_ssh.Dockerfile
MAINTAINER PaddlePaddle dev team <paddle-dev@baidu.com>
Then you can build an image with Dockerfile and launch a container:
RUN apt-get update
RUN apt-get install -y openssh-server
RUN mkdir /var/run/sshd
RUN echo 'root:root' | chpasswd
.. code-block:: bash
RUN sed -ri 's/^PermitRootLogin\s+.*/PermitRootLogin yes/' /etc/ssh/sshd_config
RUN sed -ri 's/UsePAM yes/#UsePAM yes/g' /etc/ssh/sshd_config
# cd into Dockerfile directory
docker build . -t paddle_ssh
# run container, and map host machine port 8022 to container port 22
docker run -d -p 8022:22 --name paddle_ssh_machine paddle_ssh
EXPOSE 22
Now, you can ssh on port 8022 to access the container, username is root, password is also root:
CMD ["/usr/sbin/sshd", "-D"]
```
.. code-block:: bash
Then you can build an image with Dockerfile and launch a container:
ssh -p 8022 root@YOUR_HOST_MACHINE
```bash
# cd into Dockerfile directory
docker build . -t paddle_ssh
# run container, and map host machine port 8022 to container port 22
docker run -d -p 8022:22 --name paddle_ssh_machine paddle_ssh
```
Now, you can ssh on port 8022 to access the container, username is root, password is also root:
You can stop and delete the container as following:
```bash
ssh -p 8022 root@YOUR_HOST_MACHINE
```
.. code-block:: bash
# stop
docker stop paddle_ssh_machine
# delete
docker rm paddle_ssh_machine
You can stop and delete the container as following:
```bash
# stop
docker stop paddle_ssh_machine
# delete
docker rm paddle_ssh_machine
```
doc/_sources/build/index.txt
浏览文件 @ 57e43320
......@@ -10,20 +10,12 @@ Install PaddlePaddle
install_*
internal/install_from_jumbo.md
docker_install.rst
ubuntu_install.rst
Build from Source
-----------------
.. warning::
Please use :code:`deb` package or :code:`docker` image to install paddle. The building guide is used for hacking or contributing to PaddlePaddle.
If you want to hack and contribute PaddlePaddle source code, following guides can help you\:
.. toctree::
:maxdepth: 1
:glob:
......@@ -31,3 +23,18 @@ If you want to hack and contribute PaddlePaddle source code, following guides ca
build_from_source.md
contribute_to_paddle.md
Docker and Debian Package installation
--------------------------------------
Note: The installation packages are still in pre-release
state and your experience of installation may not be smooth.
If you want to pack docker image, the following guide can help you\:
.. toctree::
:maxdepth: 1
:glob:
docker_install.md
ubuntu_install.md
doc/_sources/build/ubuntu_install.txt
浏览文件 @ 57e43320
Debian Package installation guide
=================================
PaddlePaddle supports :code:`deb` pacakge. The installation of this :code:`deb` package is tested in ubuntu 14.04, but it should be support other debian based linux, too.
## Debian Package installation
Currently , PaddlePaddle only provides ubuntu14.04 debian packages.
There are two versions package, including CPU and GPU. The download address is:
There are four versions of debian package, :code:`cpu`, :code:`gpu`, :code:`cpu-noavx`, :code:`gpu-noavx`. And :code:`noavx` version is used to support CPU which does not contain :code:`AVX` instructions. The download url of :code:`deb` package is \: https://github.com/baidu/Paddle/releases/
https://github.com/baidu/Paddle/releases/tag/V0.8.0b0
After downloading PaddlePaddle deb packages, you can use :code:`gdebi` install.
.. code-block:: bash
gdebi paddle-*.deb
If :code:`gdebi` is not installed, you can use :code:`sudo apt-get install gdebi` to install it.
Or you can use following commands to install PaddlePaddle.
.. code-block:: bash
dpkg -i paddle-*.deb
apt-get install -f
After downloading PaddlePaddle deb packages, you can run:
```bash
dpkg -i paddle-0.8.0b-cpu.deb
apt-get install -f
```
And if you use GPU version deb package, you need to install CUDA toolkit and cuDNN, and set related environment variables(such as LD_LIBRARY_PATH) first. It is normal when `dpkg -i` get errors. `apt-get install -f` will continue install paddle, and install dependences.
**Note**
PaddlePaddle package only supports x86 CPU with AVX instructions. If not, you have to download and build from source code.
doc/_sources/demo/quick_start/index_en.txt
浏览文件 @ 57e43320
......@@ -134,7 +134,7 @@ def process(settings, file_name):
You need to add a data provider definition `define_py_data_sources2` in our network configuration. This definition specifies:
- The path of the training and testing data (`data/train.list`, `data/test.list`).
- The location of the data provider file (`dataprovider_bow`).
- The location of the data provider file (`dataprovider_pow`).
- The function to call to get data. (`process`).
- Additional arguments or data. Here it passes the path of word dictionary.
......
doc/_sources/ui/api/trainer_config_helpers/layers.txt
浏览文件 @ 57e43320
......@@ -73,12 +73,6 @@ img_pool_layer
:members: img_pool_layer
:noindex:
maxout_layer
------------
.. automodule:: paddle.trainer_config_helpers.layers
:members: maxout_layer
:noindex:
Norm Layer
==========
......@@ -136,12 +130,6 @@ gru_step_layer
Recurrent Layer Group
=====================
memory
------
.. automodule:: paddle.trainer_config_helpers.layers
:members: memory
:noindex:
recurrent_group
---------------
.. automodule:: paddle.trainer_config_helpers.layers
......@@ -383,12 +371,6 @@ ctc_layer
:members: ctc_layer
:noindex:
nce_layer
-----------
.. automodule:: paddle.trainer_config_helpers.layers
:members: nce_layer
:noindex:
hsigmoid
---------
.. automodule:: paddle.trainer_config_helpers.layers
......
doc/build/build_from_source.html
浏览文件 @ 57e43320
......@@ -31,7 +31,7 @@
<link rel="top" title="PaddlePaddle documentation" href="../index.html" />
<link rel="up" title="Build And Install PaddlePaddle" href="index.html" />
<link rel="next" title="Contribute to PaddlePaddle" href="contribute_to_paddle.html" />
<link rel="prev" title="Debian Package installation guide" href="ubuntu_install.html" />
<link rel="prev" title="Build And Install PaddlePaddle" href="index.html" />
<script>
var _hmt = _hmt || [];
(function() {
......@@ -57,7 +57,7 @@ var _hmt = _hmt || [];
<a href="contribute_to_paddle.html" title="Contribute to PaddlePaddle"
accesskey="N">next</a> |</li>
<li class="right" >
<a href="ubuntu_install.html" title="Debian Package installation guide"
<a href="index.html" title="Build And Install PaddlePaddle"
accesskey="P">previous</a> |</li>
<li class="nav-item nav-item-0"><a href="../index.html">PaddlePaddle documentation</a> &#187;</li>
<li class="nav-item nav-item-1"><a href="index.html" accesskey="U">Build And Install PaddlePaddle</a> &#187;</li>
......@@ -100,26 +100,51 @@ var _hmt = _hmt || [];
<div class="section" id="options">
<span id="options"></span><h3>Options<a class="headerlink" href="#options" title="Permalink to this headline"></a></h3>
<p>PaddlePaddle supports some build options. To enable it, first you need to install the related libraries.</p>
<p><html></p>
<table>
<thead>
<tr>
<th scope="col" class="left">Optional</th>
<th scope="col" class="left">Description</th>
</tr>
</thead>
<tbody>
<tr><td class="left">WITH_GPU</td><td class="left">Compile with GPU mode.</td></tr>
<tr><td class="left">WITH_DOUBLE</td><td class="left">Compile with double precision floating-point, default: single precision.</td></tr>
<tr><td class="left">WITH_GLOG</td><td class="left">Compile with glog. If not found, default: an internal log implementation.</td></tr>
<tr><td class="left">WITH_GFLAGS</td><td class="left">Compile with gflags. If not found, default: an internal flag implementation.</td></tr>
<tr><td class="left">WITH_TESTING</td><td class="left">Compile with gtest for PaddlePaddle's unit testing.</td></tr>
<tr><td class="left">WITH_DOC</td><td class="left"> Compile to generate PaddlePaddle's docs, default: disabled (OFF).</td></tr>
<tr><td class="left">WITH_SWIG_PY</td><td class="left">Compile with python predict API, default: disabled (OFF).</td></tr>
<tr><td class="left">WITH_STYLE_CHECK</td><td class="left">Compile with code style check, default: enabled (ON).</td></tr>
</tbody>
</table>
</html><p><strong>Note:</strong></p>
<style type="text/css">
.tg {border-collapse:collapse;border-spacing:0;border-color:#ccc;}
.tg td{font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:0px;overflow:hidden;word-break:normal;border-color:#ccc;color:#333;background-color:#fff;border-top-width:1px;border-bottom-width:1px;}
.tg th{font-family:Arial, sans-serif;font-size:14px;font-weight:normal;padding:10px 5px;border-style:solid;border-width:0px;overflow:hidden;word-break:normal;border-color:#ccc;color:#333;background-color:#f0f0f0;border-top-width:1px;border-bottom-width:1px;}
.tg .tg-yw4l{vertical-align:top}
.tg .tg-9hbo{font-weight:bold;vertical-align:top}
</style>
<table class="tg">
<tr>
<th class="tg-yw4l">Optional</th>
<th class="tg-yw4l">Description</th>
</tr>
<tr>
<td class="tg-9hbo">WITH_GPU</td>
<td class="tg-yw4l">Compile with GPU mode.</td>
</tr>
<tr>
<td class="tg-9hbo">WITH_DOUBLE</td>
<td class="tg-yw4l">Compile with double precision floating-point, default: single precision.</td>
</tr>
<tr>
<td class="tg-9hbo">WITH_GLOG</td>
<td class="tg-yw4l">Compile with glog. If not found, default: an internal log implementation.</td>
</tr>
<tr>
<td class="tg-9hbo">WITH_GFLAGS</td>
<td class="tg-yw4l">Compile with gflags. If not found, default: an internal flag implementation.</td>
</tr>
<tr>
<td class="tg-9hbo">WITH_TESTING</td>
<td class="tg-yw4l">Compile with gtest for PaddlePaddle's unit testing.</td>
</tr>
<tr>
<td class="tg-9hbo">WITH_DOC</td>
<td class="tg-yw4l">Compile to generate PaddlePaddle's docs, default: disabled (OFF)</td>
</tr>
<tr>
<td class="tg-9hbo">WITH_SWIG_PY</td>
<td class="tg-yw4l">Compile with python predict API, default: disabled (OFF).</td>
</tr>
<tr>
<td class="tg-9hbo">WITH_STYLE_CHECK</td>
<td class="tg-yw4l">Compile with code style check, default: enabled (ON).</td>
</tr>
</table><p><strong>Note:</strong></p>
<ul class="simple">
<li>The GPU version works best with Cuda Toolkit 7.5 and cuDNN v5.</li>
<li>Other versions like Cuda Toolkit 6.5, 7.0, 8.0 and cuDNN v2, v3, v4 are also supported.</li>
......@@ -400,8 +425,8 @@ sudo paddle version
</ul>
<h4>Previous topic</h4>
<p class="topless"><a href="ubuntu_install.html"
title="previous chapter">Debian Package installation guide</a></p>
<p class="topless"><a href="index.html"
title="previous chapter">Build And Install PaddlePaddle</a></p>
<h4>Next topic</h4>
<p class="topless"><a href="contribute_to_paddle.html"
title="next chapter">Contribute to PaddlePaddle</a></p>
......@@ -439,7 +464,7 @@ sudo paddle version
<a href="contribute_to_paddle.html" title="Contribute to PaddlePaddle"
>next</a> |</li>
<li class="right" >
<a href="ubuntu_install.html" title="Debian Package installation guide"
<a href="index.html" title="Build And Install PaddlePaddle"
>previous</a> |</li>
<li class="nav-item nav-item-0"><a href="../index.html">PaddlePaddle documentation</a> &#187;</li>
<li class="nav-item nav-item-1"><a href="index.html" >Build And Install PaddlePaddle</a> &#187;</li>
......
doc/build/contribute_to_paddle.html
浏览文件 @ 57e43320
......@@ -30,7 +30,7 @@
<link rel="search" title="Search" href="../search.html" />
<link rel="top" title="PaddlePaddle documentation" href="../index.html" />
<link rel="up" title="Build And Install PaddlePaddle" href="index.html" />
<link rel="next" title="User Interface" href="../ui/index.html" />
<link rel="next" title="Docker installation guide" href="docker_install.html" />
<link rel="prev" title="Installing from Sources" href="build_from_source.html" />
<script>
var _hmt = _hmt || [];
......@@ -54,7 +54,7 @@ var _hmt = _hmt || [];
<a href="../py-modindex.html" title="Python Module Index"
>modules</a> |</li>
<li class="right" >
<a href="../ui/index.html" title="User Interface"
<a href="docker_install.html" title="Docker installation guide"
accesskey="N">next</a> |</li>
<li class="right" >
<a href="build_from_source.html" title="Installing from Sources"
......@@ -165,10 +165,6 @@ git push -f origin HEAD
</div>
<p>Now your Pull Request is updated with the latest version.</p>
</div>
<div class="section" id="revise-your-pull-request">
<span id="revise-your-pull-request"></span><h2>Revise your pull request<a class="headerlink" href="#revise-your-pull-request" title="Permalink to this headline"></a></h2>
<p>When you revise your pull request according to reviewer&#8217;s comments, please use &#8216;git commit&#8217; instead of &#8216;git commit &#8211;amend&#8217; to commit your changes so that the reviewers can see the difference between the new pull requrest and the old pull request.</p>
</div>
</div>
......@@ -188,7 +184,6 @@ git push -f origin HEAD
<li><a class="reference internal" href="#push-to-github">Push to GitHub</a></li>
<li><a class="reference internal" href="#pull-request">Pull Request</a></li>
<li><a class="reference internal" href="#update-your-pull-request-with-the-lastest-version">Update your pull request with the lastest version</a></li>
<li><a class="reference internal" href="#revise-your-pull-request">Revise your pull request</a></li>
</ul>
</li>
</ul>
......@@ -197,8 +192,8 @@ git push -f origin HEAD
<p class="topless"><a href="build_from_source.html"
title="previous chapter">Installing from Sources</a></p>
<h4>Next topic</h4>
<p class="topless"><a href="../ui/index.html"
title="next chapter">User Interface</a></p>
<p class="topless"><a href="docker_install.html"
title="next chapter">Docker installation guide</a></p>
<div role="note" aria-label="source link">
<h3>This Page</h3>
<ul class="this-page-menu">
......@@ -230,7 +225,7 @@ git push -f origin HEAD
<a href="../py-modindex.html" title="Python Module Index"
>modules</a> |</li>
<li class="right" >
<a href="../ui/index.html" title="User Interface"
<a href="docker_install.html" title="Docker installation guide"
>next</a> |</li>
<li class="right" >
<a href="build_from_source.html" title="Installing from Sources"
......
doc/build/docker_install.html
浏览文件 @ 57e43320
......@@ -31,7 +31,7 @@
<link rel="top" title="PaddlePaddle documentation" href="../index.html" />
<link rel="up" title="Build And Install PaddlePaddle" href="index.html" />
<link rel="next" title="Debian Package installation guide" href="ubuntu_install.html" />
<link rel="prev" title="Build And Install PaddlePaddle" href="index.html" />
<link rel="prev" title="Contribute to PaddlePaddle" href="contribute_to_paddle.html" />
<script>
var _hmt = _hmt || [];
(function() {
......@@ -57,7 +57,7 @@ var _hmt = _hmt || [];
<a href="ubuntu_install.html" title="Debian Package installation guide"
accesskey="N">next</a> |</li>
<li class="right" >
<a href="index.html" title="Build And Install PaddlePaddle"
<a href="contribute_to_paddle.html" title="Contribute to PaddlePaddle"
accesskey="P">previous</a> |</li>
<li class="nav-item nav-item-0"><a href="../index.html">PaddlePaddle documentation</a> &#187;</li>
<li class="nav-item nav-item-1"><a href="index.html" accesskey="U">Build And Install PaddlePaddle</a> &#187;</li>
......@@ -70,115 +70,64 @@ var _hmt = _hmt || [];
<div class="body" role="main">
<div class="section" id="docker-installation-guide">
<h1>Docker installation guide<a class="headerlink" href="#docker-installation-guide" title="Permalink to this headline"></a></h1>
<p>PaddlePaddle provide the <a class="reference external" href="https://www.docker.com/">Docker</a> image. <a class="reference external" href="https://www.docker.com/">Docker</a> is a lightweight container utilities. The performance of PaddlePaddle in <a class="reference external" href="https://www.docker.com/">Docker</a> container is basically as same as run it in a normal linux. The <a class="reference external" href="https://www.docker.com/">Docker</a> is a very convenient way to deliver the binary release for linux programs.</p>
<div class="admonition note">
<p class="first admonition-title">Note</p>
<p class="last">The <a class="reference external" href="https://www.docker.com/">Docker</a> image is the recommended way to run PaddlePaddle</p>
</div>
<span id="docker-installation-guide"></span><h1>Docker installation guide<a class="headerlink" href="#docker-installation-guide" title="Permalink to this headline"></a></h1>
<p>PaddlePaddle provides some pre-compiled binary, including Docker images, ubuntu deb packages. It is welcomed to contributed more installation package of different linux distribution (such as ubuntu, centos, debian, gentoo and so on). We recommend to use Docker images to deploy PaddlePaddle.</p>
<div class="section" id="docker-installation">
<span id="docker-installation"></span><h2>Docker installation<a class="headerlink" href="#docker-installation" title="Permalink to this headline"></a></h2>
<p>Docker is a tool designed to make it easier to create, deploy, and run applications by using containers.</p>
<div class="section" id="paddlepaddle-docker-images">
<h2>PaddlePaddle Docker images<a class="headerlink" href="#paddlepaddle-docker-images" title="Permalink to this headline"></a></h2>
<p>There are 12 <a class="reference external" href="https://hub.docker.com/r/paddledev/paddle/tags/">images</a> for PaddlePaddle, and the name is <code class="code docutils literal"><span class="pre">paddle-dev/paddle</span></code>, tags are:</p>
<table border="1" class="docutils">
<colgroup>
<col width="21%" />
<col width="22%" />
<col width="29%" />
<col width="28%" />
</colgroup>
<thead valign="bottom">
<tr class="row-odd"><th class="head">&nbsp;</th>
<th class="head">normal</th>
<th class="head">devel</th>
<th class="head">demo</th>
</tr>
</thead>
<tbody valign="top">
<tr class="row-even"><td>CPU</td>
<td>cpu-latest</td>
<td>cpu-devel-latest</td>
<td>cpu-demo-latest</td>
</tr>
<tr class="row-odd"><td>GPU</td>
<td>gpu-latest</td>
<td>gpu-devel-latest</td>
<td>gpu-demo-latest</td>
</tr>
<tr class="row-even"><td>CPU WITHOUT AVX</td>
<td>cpu-noavx-latest</td>
<td>cpu-devel-noavx-latest</td>
<td>cpu-demo-noavx-latest</td>
</tr>
<tr class="row-odd"><td>GPU WITHOUT AVX</td>
<td>gpu-noavx-latest</td>
<td>gpu-devel-noavx-latest</td>
<td>gpu-demo-noavx-latest</td>
</tr>
</tbody>
</table>
<p>And the three columns are:</p>
<ul class="simple">
<li>normal: The docker image only contains binary of PaddlePaddle.</li>
<li>devel: The docker image contains PaddlePaddle binary, source code and essential build environment.</li>
<li>demo: The docker image contains the dependencies to run PaddlePaddle demo.</li>
</ul>
<p>And the four rows are:</p>
<span id="paddlepaddle-docker-images"></span><h3>PaddlePaddle Docker images<a class="headerlink" href="#paddlepaddle-docker-images" title="Permalink to this headline"></a></h3>
<p>There are six Docker images:</p>
<ul class="simple">
<li>CPU: CPU Version. Support CPU which has <code class="code docutils literal"><span class="pre">AVX</span></code> instructions.</li>
<li>GPU: GPU Version. Support GPU, and cpu has <code class="code docutils literal"><span class="pre">AVX</span></code> instructions.</li>
<li>CPU WITHOUT AVX: CPU Version, which support most CPU even doesn&#8217;t have <code class="code docutils literal"><span class="pre">AVX</span></code> instructions.</li>
<li>GPU WITHOUT AVX: GPU Version, which support most CPU even doesn&#8217;t have <code class="code docutils literal"><span class="pre">AVX</span></code> instructions.</li>
<li>paddledev/paddle:cpu-latest: PaddlePaddle CPU binary image.</li>
<li>paddledev/paddle:gpu-latest: PaddlePaddle GPU binary image.</li>
<li>paddledev/paddle:cpu-devel-latest: PaddlePaddle CPU binary image plus source code.</li>
<li>paddledev/paddle:gpu-devel-latest: PaddlePaddle GPU binary image plus source code.</li>
<li>paddledev/paddle:cpu-demo-latest: PaddlePaddle CPU binary image plus source code and demo</li>
<li>paddledev/paddle:gpu-demo-latest: PaddlePaddle GPU binary image plus source code and demo</li>
</ul>
<p>User can choose any version depends on machine. The following script can help you to detect your CPU support <code class="code docutils literal"><span class="pre">AVX</span></code> or not.</p>
<div class="highlight-bash"><div class="highlight"><pre><span></span><span class="k">if</span> cat /proc/cpuinfo <span class="p">|</span> grep -q avx <span class="p">;</span> <span class="k">then</span> <span class="nb">echo</span> <span class="s2">&quot;Support AVX&quot;</span><span class="p">;</span> <span class="k">else</span> <span class="nb">echo</span> <span class="s2">&quot;Not support AVX&quot;</span><span class="p">;</span> <span class="k">fi</span>
</pre></div>
</div>
<p>If the output is <code class="code docutils literal"><span class="pre">Support</span> <span class="pre">AVX</span></code>, then you can choose the AVX version of PaddlePaddle, otherwise, you need select <code class="code docutils literal"><span class="pre">noavx</span></code> version of PaddlePaddle. For example, the CPU develop version of PaddlePaddle is <code class="code docutils literal"><span class="pre">paddle-dev/paddle:cpu-devel-latest</span></code>.</p>
<p>The PaddlePaddle images don&#8217;t contain any entry command. You need to write your entry command to use this image. See <code class="code docutils literal"><span class="pre">Remote</span> <span class="pre">Access</span></code> part or just use following command to run a <code class="code docutils literal"><span class="pre">bash</span></code></p>
<div class="highlight-bash"><div class="highlight"><pre><span></span>docker run -it paddledev/paddle:cpu-latest /bin/bash
</pre></div>
</div>
<p>Tags with latest will be replaced by a released version.</p>
</div>
<div class="section" id="download-and-run-docker-images">
<h2>Download and Run Docker images<a class="headerlink" href="#download-and-run-docker-images" title="Permalink to this headline"></a></h2>
<p>You have to install Docker in your machine which has linux kernel version 3.10+ first. You can refer to the official guide <a class="reference external" href="https://docs.docker.com/engine/installation/">https://docs.docker.com/engine/installation/</a> for further information.</p>
<p>You can use <code class="code docutils literal"><span class="pre">docker</span> <span class="pre">pull</span> <span class="pre">`</span> <span class="pre">to</span> <span class="pre">download</span> <span class="pre">images</span> <span class="pre">first,</span> <span class="pre">or</span> <span class="pre">just</span> <span class="pre">launch</span> <span class="pre">a</span> <span class="pre">container</span> <span class="pre">with</span> <span class="pre">:code:`docker</span> <span class="pre">run</span></code> :</p>
<span id="download-and-run-docker-images"></span><h3>Download and Run Docker images<a class="headerlink" href="#download-and-run-docker-images" title="Permalink to this headline"></a></h3>
<p>You have to install Docker in your machine which has linux kernel version 3.10+ first. You can refer to the official guide https://docs.docker.com/engine/installation/ for further information.</p>
<p>You can use <code class="docutils literal"><span class="pre">docker</span> <span class="pre">pull</span></code>to download images first, or just launch a container with <code class="docutils literal"><span class="pre">docker</span> <span class="pre">run</span></code>:</p>
<div class="highlight-bash"><div class="highlight"><pre><span></span>docker run -it paddledev/paddle:cpu-latest
</pre></div>
</div>
<p>If you want to launch container with GPU support, you need to set some environment variables at the same time:</p>
<div class="highlight-bash"><div class="highlight"><pre><span></span><span class="nb">export</span> <span class="nv">CUDA_SO</span><span class="o">=</span><span class="s2">&quot;</span><span class="k">$(</span><span class="se">\l</span>s /usr/lib64/libcuda* <span class="p">|</span> xargs -I<span class="o">{}</span> <span class="nb">echo</span> <span class="s1">&#39;-v {}:{}&#39;</span><span class="k">)</span><span class="s2"> </span><span class="k">$(</span><span class="se">\l</span>s /usr/lib64/libnvidia* <span class="p">|</span> xargs -I<span class="o">{}</span> <span class="nb">echo</span> <span class="s1">&#39;-v {}:{}&#39;</span><span class="k">)</span><span class="s2">&quot;</span>
<span class="nb">export</span> <span class="nv">DEVICES</span><span class="o">=</span><span class="k">$(</span><span class="se">\l</span>s /dev/nvidia* <span class="p">|</span> xargs -I<span class="o">{}</span> <span class="nb">echo</span> <span class="s1">&#39;--device {}:{}&#39;</span><span class="k">)</span>
docker run <span class="si">${</span><span class="nv">CUDA_SO</span><span class="si">}</span> <span class="si">${</span><span class="nv">DEVICES</span><span class="si">}</span> -it paddledev/paddle:gpu-latest
<div class="highlight-bash"><div class="highlight"><pre><span></span>export CUDA_SO=&quot;$(\ls /usr/lib64/libcuda* | xargs -I{} echo &#39;-v {}:{}&#39;) $(\ls /usr/lib64/libnvidia* | xargs -I{} echo &#39;-v {}:{}&quot;
export DEVICES=$(\ls /dev/nvidia* | xargs -I{} echo &#39;--device {}:{}&#39;)
docker run -it paddledev/paddle:gpu-latest
</pre></div>
</div>
</div>
<div class="section" id="some-notes-for-docker">
<h2>Some notes for docker<a class="headerlink" href="#some-notes-for-docker" title="Permalink to this headline"></a></h2>
<div class="section" id="notice">
<span id="notice"></span><h3>Notice<a class="headerlink" href="#notice" title="Permalink to this headline"></a></h3>
<div class="section" id="performance">
<h3>Performance<a class="headerlink" href="#performance" title="Permalink to this headline"></a></h3>
<span id="performance"></span><h4>Performance<a class="headerlink" href="#performance" title="Permalink to this headline"></a></h4>
<p>Since Docker is based on the lightweight virtual containers, the CPU computing performance maintains well. And GPU driver and equipments are all mapped to the container, so the GPU computing performance would not be seriously affected.</p>
<p>If you use high performance nic, such as RDMA(RoCE 40GbE or IB 56GbE), Ethernet(10GbE), it is recommended to use config &#8220;-net = host&#8221;.</p>
</div>
<div class="section" id="remote-access">
<h3>Remote access<a class="headerlink" href="#remote-access" title="Permalink to this headline"></a></h3>
<p>If you want to enable ssh access background, you need to build an image by yourself. Please refer to official guide <a class="reference external" href="https://docs.docker.com/engine/reference/builder/">https://docs.docker.com/engine/reference/builder/</a> for further information.</p>
<span id="remote-access"></span><h4>Remote access<a class="headerlink" href="#remote-access" title="Permalink to this headline"></a></h4>
<p>If you want to enable ssh access background, you need to build an image by yourself. Please refer to official guide https://docs.docker.com/engine/reference/builder/ for further information.</p>
<p>Following is a simple Dockerfile with ssh:</p>
<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">FROM</span> <span class="n">paddledev</span><span class="o">/</span><span class="n">paddle</span><span class="p">:</span><span class="n">cpu</span><span class="o">-</span><span class="n">latest</span>
<div class="highlight-bash"><div class="highlight"><pre><span></span>FROM paddledev/paddle
<span class="n">MAINTAINER</span> <span class="n">PaddlePaddle</span> <span class="n">dev</span> <span class="n">team</span> <span class="o">&lt;</span><span class="n">paddle</span><span class="o">-</span><span class="n">dev</span><span class="nd">@baidu</span><span class="o">.</span><span class="n">com</span><span class="o">&gt;</span>
MAINTAINER PaddlePaddle dev team &lt;paddle-dev@baidu.com&gt;
<span class="n">RUN</span> <span class="n">apt</span><span class="o">-</span><span class="n">get</span> <span class="n">update</span>
<span class="n">RUN</span> <span class="n">apt</span><span class="o">-</span><span class="n">get</span> <span class="n">install</span> <span class="o">-</span><span class="n">y</span> <span class="n">openssh</span><span class="o">-</span><span class="n">server</span>
<span class="n">RUN</span> <span class="n">mkdir</span> <span class="o">/</span><span class="n">var</span><span class="o">/</span><span class="n">run</span><span class="o">/</span><span class="n">sshd</span>
<span class="n">RUN</span> <span class="n">echo</span> <span class="s1">&#39;root:root&#39;</span> <span class="o">|</span> <span class="n">chpasswd</span>
RUN apt-get update
RUN apt-get install -y openssh-server
RUN mkdir /var/run/sshd
RUN <span class="nb">echo</span> <span class="s1">&#39;root:root&#39;</span> <span class="p">|</span> chpasswd
<span class="n">RUN</span> <span class="n">sed</span> <span class="o">-</span><span class="n">ri</span> <span class="s1">&#39;s/^PermitRootLogin\s+.*/PermitRootLogin yes/&#39;</span> <span class="o">/</span><span class="n">etc</span><span class="o">/</span><span class="n">ssh</span><span class="o">/</span><span class="n">sshd_config</span>
<span class="n">RUN</span> <span class="n">sed</span> <span class="o">-</span><span class="n">ri</span> <span class="s1">&#39;s/UsePAM yes/#UsePAM yes/g&#39;</span> <span class="o">/</span><span class="n">etc</span><span class="o">/</span><span class="n">ssh</span><span class="o">/</span><span class="n">sshd_config</span>
RUN sed -ri <span class="s1">&#39;s/^PermitRootLogin\s+.*/PermitRootLogin yes/&#39;</span> /etc/ssh/sshd_config
RUN sed -ri <span class="s1">&#39;s/UsePAM yes/#UsePAM yes/g&#39;</span> /etc/ssh/sshd_config
<span class="n">EXPOSE</span> <span class="mi">22</span>
EXPOSE 22
<span class="n">CMD</span> <span class="p">[</span><span class="s2">&quot;/usr/sbin/sshd&quot;</span><span class="p">,</span> <span class="s2">&quot;-D&quot;</span><span class="p">]</span>
CMD <span class="o">[</span><span class="s2">&quot;/usr/sbin/sshd&quot;</span>, <span class="s2">&quot;-D&quot;</span><span class="o">]</span>
</pre></div>
</div>
<p>Then you can build an image with Dockerfile and launch a container:</p>
......@@ -201,6 +150,7 @@ docker rm paddle_ssh_machine
</div>
</div>
</div>
</div>
</div>
......@@ -212,20 +162,23 @@ docker rm paddle_ssh_machine
<h3><a href="../index.html">Table Of Contents</a></h3>
<ul>
<li><a class="reference internal" href="#">Docker installation guide</a><ul>
<li><a class="reference internal" href="#docker-installation">Docker installation</a><ul>
<li><a class="reference internal" href="#paddlepaddle-docker-images">PaddlePaddle Docker images</a></li>
<li><a class="reference internal" href="#download-and-run-docker-images">Download and Run Docker images</a></li>
<li><a class="reference internal" href="#some-notes-for-docker">Some notes for docker</a><ul>
<li><a class="reference internal" href="#notice">Notice</a><ul>
<li><a class="reference internal" href="#performance">Performance</a></li>
<li><a class="reference internal" href="#remote-access">Remote access</a></li>
</ul>
</li>
</ul>
</li>
</ul>
</li>
</ul>
<h4>Previous topic</h4>
<p class="topless"><a href="index.html"
title="previous chapter">Build And Install PaddlePaddle</a></p>
<p class="topless"><a href="contribute_to_paddle.html"
title="previous chapter">Contribute to PaddlePaddle</a></p>
<h4>Next topic</h4>
<p class="topless"><a href="ubuntu_install.html"
title="next chapter">Debian Package installation guide</a></p>
......@@ -263,7 +216,7 @@ docker rm paddle_ssh_machine
<a href="ubuntu_install.html" title="Debian Package installation guide"
>next</a> |</li>
<li class="right" >
<a href="index.html" title="Build And Install PaddlePaddle"
<a href="contribute_to_paddle.html" title="Contribute to PaddlePaddle"
>previous</a> |</li>
<li class="nav-item nav-item-0"><a href="../index.html">PaddlePaddle documentation</a> &#187;</li>
<li class="nav-item nav-item-1"><a href="index.html" >Build And Install PaddlePaddle</a> &#187;</li>
......
doc/build/index.html
浏览文件 @ 57e43320
......@@ -29,7 +29,7 @@
<link rel="index" title="Index" href="../genindex.html" />
<link rel="search" title="Search" href="../search.html" />
<link rel="top" title="PaddlePaddle documentation" href="../index.html" />
<link rel="next" title="Docker installation guide" href="docker_install.html" />
<link rel="next" title="Installing from Sources" href="build_from_source.html" />
<link rel="prev" title="Quick Start Tutorial" href="../demo/quick_start/index_en.html" />
<script>
var _hmt = _hmt || [];
......@@ -53,7 +53,7 @@ var _hmt = _hmt || [];
<a href="../py-modindex.html" title="Python Module Index"
>modules</a> |</li>
<li class="right" >
<a href="docker_install.html" title="Docker installation guide"
<a href="build_from_source.html" title="Installing from Sources"
accesskey="N">next</a> |</li>
<li class="right" >
<a href="../demo/quick_start/index_en.html" title="Quick Start Tutorial"
......@@ -72,18 +72,10 @@ var _hmt = _hmt || [];
<div class="section" id="install-paddlepaddle">
<h2>Install PaddlePaddle<a class="headerlink" href="#install-paddlepaddle" title="Permalink to this headline"></a></h2>
<div class="toctree-wrapper compound">
<ul>
<li class="toctree-l1"><a class="reference internal" href="docker_install.html">Docker installation guide</a></li>
<li class="toctree-l1"><a class="reference internal" href="ubuntu_install.html">Debian Package installation guide</a></li>
</ul>
</div>
</div>
<div class="section" id="build-from-source">
<h2>Build from Source<a class="headerlink" href="#build-from-source" title="Permalink to this headline"></a></h2>
<div class="admonition warning">
<p class="first admonition-title">Warning</p>
<p class="last">Please use <code class="code docutils literal"><span class="pre">deb</span></code> package or <code class="code docutils literal"><span class="pre">docker</span></code> image to install paddle. The building guide is used for hacking or contributing to PaddlePaddle.</p>
</div>
<p>If you want to hack and contribute PaddlePaddle source code, following guides can help you:</p>
<div class="toctree-wrapper compound">
<ul>
......@@ -92,6 +84,18 @@ var _hmt = _hmt || [];
</ul>
</div>
</div>
<div class="section" id="docker-and-debian-package-installation">
<h2>Docker and Debian Package installation<a class="headerlink" href="#docker-and-debian-package-installation" title="Permalink to this headline"></a></h2>
<p>Note: The installation packages are still in pre-release
state and your experience of installation may not be smooth.</p>
<p>If you want to pack docker image, the following guide can help you:</p>
<div class="toctree-wrapper compound">
<ul>
<li class="toctree-l1"><a class="reference internal" href="docker_install.html">Docker installation guide</a></li>
<li class="toctree-l1"><a class="reference internal" href="ubuntu_install.html">Debian Package installation guide</a></li>
</ul>
</div>
</div>
</div>
......@@ -105,6 +109,7 @@ var _hmt = _hmt || [];
<li><a class="reference internal" href="#">Build And Install PaddlePaddle</a><ul>
<li><a class="reference internal" href="#install-paddlepaddle">Install PaddlePaddle</a></li>
<li><a class="reference internal" href="#build-from-source">Build from Source</a></li>
<li><a class="reference internal" href="#docker-and-debian-package-installation">Docker and Debian Package installation</a></li>
</ul>
</li>
</ul>
......@@ -113,8 +118,8 @@ var _hmt = _hmt || [];
<p class="topless"><a href="../demo/quick_start/index_en.html"
title="previous chapter">Quick Start Tutorial</a></p>
<h4>Next topic</h4>
<p class="topless"><a href="docker_install.html"
title="next chapter">Docker installation guide</a></p>
<p class="topless"><a href="build_from_source.html"
title="next chapter">Installing from Sources</a></p>
<div role="note" aria-label="source link">
<h3>This Page</h3>
<ul class="this-page-menu">
......@@ -146,7 +151,7 @@ var _hmt = _hmt || [];
<a href="../py-modindex.html" title="Python Module Index"
>modules</a> |</li>
<li class="right" >
<a href="docker_install.html" title="Docker installation guide"
<a href="build_from_source.html" title="Installing from Sources"
>next</a> |</li>
<li class="right" >
<a href="../demo/quick_start/index_en.html" title="Quick Start Tutorial"
......
doc/build/ubuntu_install.html
浏览文件 @ 57e43320
......@@ -30,7 +30,7 @@
<link rel="search" title="Search" href="../search.html" />
<link rel="top" title="PaddlePaddle documentation" href="../index.html" />
<link rel="up" title="Build And Install PaddlePaddle" href="index.html" />
<link rel="next" title="Installing from Sources" href="build_from_source.html" />
<link rel="next" title="User Interface" href="../ui/index.html" />
<link rel="prev" title="Docker installation guide" href="docker_install.html" />
<script>
var _hmt = _hmt || [];
......@@ -54,7 +54,7 @@ var _hmt = _hmt || [];
<a href="../py-modindex.html" title="Python Module Index"
>modules</a> |</li>
<li class="right" >
<a href="build_from_source.html" title="Installing from Sources"
<a href="../ui/index.html" title="User Interface"
accesskey="N">next</a> |</li>
<li class="right" >
<a href="docker_install.html" title="Docker installation guide"
......@@ -70,20 +70,21 @@ var _hmt = _hmt || [];
<div class="body" role="main">
<div class="section" id="debian-package-installation-guide">
<h1>Debian Package installation guide<a class="headerlink" href="#debian-package-installation-guide" title="Permalink to this headline"></a></h1>
<p>PaddlePaddle supports <code class="code docutils literal"><span class="pre">deb</span></code> pacakge. The installation of this <code class="code docutils literal"><span class="pre">deb</span></code> package is tested in ubuntu 14.04, but it should be support other debian based linux, too.</p>
<p>There are four versions of debian package, <code class="code docutils literal"><span class="pre">cpu</span></code>, <code class="code docutils literal"><span class="pre">gpu</span></code>, <code class="code docutils literal"><span class="pre">cpu-noavx</span></code>, <code class="code docutils literal"><span class="pre">gpu-noavx</span></code>. And <code class="code docutils literal"><span class="pre">noavx</span></code> version is used to support CPU which does not contain <code class="code docutils literal"><span class="pre">AVX</span></code> instructions. The download url of <code class="code docutils literal"><span class="pre">deb</span></code> package is : <a class="reference external" href="https://github.com/baidu/Paddle/releases/">https://github.com/baidu/Paddle/releases/</a></p>
<p>After downloading PaddlePaddle deb packages, you can use <code class="code docutils literal"><span class="pre">gdebi</span></code> install.</p>
<div class="highlight-bash"><div class="highlight"><pre><span></span>gdebi paddle-*.deb
</pre></div>
</div>
<p>If <code class="code docutils literal"><span class="pre">gdebi</span></code> is not installed, you can use <code class="code docutils literal"><span class="pre">sudo</span> <span class="pre">apt-get</span> <span class="pre">install</span> <span class="pre">gdebi</span></code> to install it.</p>
<p>Or you can use following commands to install PaddlePaddle.</p>
<div class="highlight-bash"><div class="highlight"><pre><span></span>dpkg -i paddle-*.deb
<span id="debian-package-installation-guide"></span><h1>Debian Package installation guide<a class="headerlink" href="#debian-package-installation-guide" title="Permalink to this headline"></a></h1>
<div class="section" id="debian-package-installation">
<span id="debian-package-installation"></span><h2>Debian Package installation<a class="headerlink" href="#debian-package-installation" title="Permalink to this headline"></a></h2>
<p>Currently , PaddlePaddle only provides ubuntu14.04 debian packages.
There are two versions package, including CPU and GPU. The download address is:</p>
<p>https://github.com/baidu/Paddle/releases/tag/V0.8.0b0</p>
<p>After downloading PaddlePaddle deb packages, you can run:</p>
<div class="highlight-bash"><div class="highlight"><pre><span></span>dpkg -i paddle-0.8.0b-cpu.deb
apt-get install -f
</pre></div>
</div>
<p>And if you use GPU version deb package, you need to install CUDA toolkit and cuDNN, and set related environment variables(such as LD_LIBRARY_PATH) first. It is normal when <cite>dpkg -i</cite> get errors. <cite>apt-get install -f</cite> will continue install paddle, and install dependences.</p>
<p>And if you use GPU version deb package, you need to install CUDA toolkit and cuDNN, and set related environment variables(such as LD_LIBRARY_PATH) first. It is normal when <code class="docutils literal"><span class="pre">dpkg</span> <span class="pre">-i</span></code> get errors. <code class="docutils literal"><span class="pre">apt-get</span> <span class="pre">install</span> <span class="pre">-f</span></code> will continue install paddle, and install dependences.</p>
<p><strong>Note</strong></p>
<p>PaddlePaddle package only supports x86 CPU with AVX instructions. If not, you have to download and build from source code.</p>
</div>
</div>
......@@ -92,12 +93,20 @@ apt-get install -f
</div>
<div class="sphinxsidebar" role="navigation" aria-label="main navigation">
<div class="sphinxsidebarwrapper">
<h3><a href="../index.html">Table Of Contents</a></h3>
<ul>
<li><a class="reference internal" href="#">Debian Package installation guide</a><ul>
<li><a class="reference internal" href="#debian-package-installation">Debian Package installation</a></li>
</ul>
</li>
</ul>
<h4>Previous topic</h4>
<p class="topless"><a href="docker_install.html"
title="previous chapter">Docker installation guide</a></p>
<h4>Next topic</h4>
<p class="topless"><a href="build_from_source.html"
title="next chapter">Installing from Sources</a></p>
<p class="topless"><a href="../ui/index.html"
title="next chapter">User Interface</a></p>
<div role="note" aria-label="source link">
<h3>This Page</h3>
<ul class="this-page-menu">
......@@ -129,7 +138,7 @@ apt-get install -f
<a href="../py-modindex.html" title="Python Module Index"
>modules</a> |</li>
<li class="right" >
<a href="build_from_source.html" title="Installing from Sources"
<a href="../ui/index.html" title="User Interface"
>next</a> |</li>
<li class="right" >
<a href="docker_install.html" title="Docker installation guide"
......
doc/demo/quick_start/index_en.html
浏览文件 @ 57e43320
......@@ -205,7 +205,7 @@ var _hmt = _hmt || [];
<p>You need to add a data provider definition <code class="docutils literal"><span class="pre">define_py_data_sources2</span></code> in our network configuration. This definition specifies:</p>
<ul class="simple">
<li>The path of the training and testing data (<code class="docutils literal"><span class="pre">data/train.list</span></code>, <code class="docutils literal"><span class="pre">data/test.list</span></code>).</li>
<li>The location of the data provider file (<code class="docutils literal"><span class="pre">dataprovider_bow</span></code>).</li>
<li>The location of the data provider file (<code class="docutils literal"><span class="pre">dataprovider_pow</span></code>).</li>
<li>The function to call to get data. (<code class="docutils literal"><span class="pre">process</span></code>).</li>
<li>Additional arguments or data. Here it passes the path of word dictionary.</li>
</ul>
......
doc/genindex.html
浏览文件 @ 57e43320
此差异已折叠。
doc/objects.inv
浏览文件 @ 57e43320
无法预览此类型文件
doc/searchindex.js
浏览文件 @ 57e43320
此差异已折叠。
doc/source/cuda/matrix/matrix.html
浏览文件 @ 57e43320
......@@ -1926,7 +1926,7 @@ CPU element wise quaternary operator. element wise op(a, b, c, d) for 0 &lt;= i
</li>
<li><code class="first docutils literal"><span class="pre">alpha</span></code> - <p>scalar used for multiplication. </p>
</li>
<li><code class="first docutils literal"><span class="pre">beta</span></code> - <p>scalar used for multiplication. If beta is zero, C does not have to be a valid input.</p>
<li><code class="first docutils literal"><span class="pre">beta</span></code> - <p>scalar used for multiplication.</p>
</li>
</ul>
</dd>
......@@ -1961,7 +1961,7 @@ CPU element wise quaternary operator. element wise op(a, b, c, d) for 0 &lt;= i
</li>
<li><code class="first docutils literal"><span class="pre">alpha</span></code> - <p>scalar used for multiplication. </p>
</li>
<li><code class="first docutils literal"><span class="pre">beta</span></code> - <p>scalar used for multiplication. If beta is zero, C does not have to be a valid input.</p>
<li><code class="first docutils literal"><span class="pre">beta</span></code> - <p>scalar used for multiplication.</p>
</li>
</ul>
</dd>
......@@ -1996,7 +1996,7 @@ CPU element wise quaternary operator. element wise op(a, b, c, d) for 0 &lt;= i
</li>
<li><code class="first docutils literal"><span class="pre">alpha</span></code> - <p>scalar used for multiplication. </p>
</li>
<li><code class="first docutils literal"><span class="pre">beta</span></code> - <p>scalar used for multiplication. If beta is zero, C does not have to be a valid input.</p>
<li><code class="first docutils literal"><span class="pre">beta</span></code> - <p>scalar used for multiplication.</p>
</li>
</ul>
</dd>
......@@ -2066,7 +2066,7 @@ CPU element wise quaternary operator. element wise op(a, b, c, d) for 0 &lt;= i
</li>
<li><code class="first docutils literal"><span class="pre">alpha</span></code> - <p>scalar used for multiplication. </p>
</li>
<li><code class="first docutils literal"><span class="pre">beta</span></code> - <p>scalar used for multiplication. If beta is zero, C does not have to be a valid input.</p>
<li><code class="first docutils literal"><span class="pre">beta</span></code> - <p>scalar used for multiplication.</p>
</li>
</ul>
</dd>
......
doc/source/cuda/rnn/rnn.html
浏览文件 @ 57e43320
此差异已折叠。
doc/source/gserver/dataprovider/dataproviders.html
浏览文件 @ 57e43320
......@@ -184,7 +184,7 @@ var _hmt = _hmt || [];
<dl class="function">
<dt id="_CPPv2N6paddle12DataProvider6createERK10DataConfigb">
<span id="paddle::DataProvider::create__DataConfigCR.b"></span><span class="target" id="paddleclasspaddle_1_1DataProvider_1a3eb1d0e7dcb32e4e9b1271c7d745706a"></span><em class="property">static</em> <a class="reference internal" href="#_CPPv2N6paddle12DataProviderE" title="paddle::DataProvider">DataProvider</a> *<code class="descname">create</code><span class="sig-paren">(</span><em class="property">const</em> DataConfig &amp;<em>config</em>, bool <em>useGpu</em> = FLAGS_use_gpu<span class="sig-paren">)</span><a class="headerlink" href="#_CPPv2N6paddle12DataProvider6createERK10DataConfigb" title="Permalink to this definition"></a></dt>
<span id="paddle::DataProvider::create__DataConfigCR.b"></span><span class="target" id="paddleclasspaddle_1_1DataProvider_1ad782dc59f7366c19ba4375101159ba95"></span><em class="property">static</em> <a class="reference internal" href="#_CPPv2N6paddle12DataProviderE" title="paddle::DataProvider">DataProvider</a> *<code class="descname">create</code><span class="sig-paren">(</span><em class="property">const</em> DataConfig &amp;<em>config</em>, bool <em>useGpu</em><span class="sig-paren">)</span><a class="headerlink" href="#_CPPv2N6paddle12DataProvider6createERK10DataConfigb" title="Permalink to this definition"></a></dt>
<dd><p>create only used for unittest. </p>
</dd></dl>
......
doc/source/math/matrix/matrix.html
浏览文件 @ 57e43320
此差异已折叠。
doc/source/parameter/parameter/parameter.html
浏览文件 @ 57e43320
......@@ -789,11 +789,6 @@ var _hmt = _hmt || [];
<dd><p>allocate buffer for the give type </p>
</dd></dl>
<dl class="function">
<dt id="_CPPv2N6paddle9Parameter13enableBufTypeE13ParameterType">
<span id="paddle::Parameter::enableBufType__ParameterType"></span><span class="target" id="paddleclasspaddle_1_1Parameter_1abf93f4627fa8490df9737474faff4eea"></span>void <code class="descname">enableBufType</code><span class="sig-paren">(</span>ParameterType <em>type</em><span class="sig-paren">)</span><a class="headerlink" href="#_CPPv2N6paddle9Parameter13enableBufTypeE13ParameterType" title="Permalink to this definition"></a></dt>
<dd></dd></dl>
<dl class="function">
<dt id="_CPPv2N6paddle9Parameter13enableIntTypeE13ParameterType6size_t">
<span id="paddle::Parameter::enableIntType__ParameterType.s"></span><span class="target" id="paddleclasspaddle_1_1Parameter_1aece0ee015937bafc11233c4384876d3e"></span>void <code class="descname">enableIntType</code><span class="sig-paren">(</span>ParameterType <em>type</em>, size_t <em>intStoreSize</em> = 0<span class="sig-paren">)</span><a class="headerlink" href="#_CPPv2N6paddle9Parameter13enableIntTypeE13ParameterType6size_t" title="Permalink to this definition"></a></dt>
......
doc/source/utils/queue.html
浏览文件 @ 57e43320
......@@ -190,23 +190,6 @@ var _hmt = _hmt || [];
<dd><p>wait util queue is empty </p>
</dd></dl>
<dl class="function">
<dt id="_CPPv2N6paddle5Queue15waitNotEmptyForEi">
<span id="paddle::Queue::waitNotEmptyFor__i"></span><span class="target" id="paddleclasspaddle_1_1Queue_1a2e21790ab2a03898307814c188ddcac1"></span>bool <code class="descname">waitNotEmptyFor</code><span class="sig-paren">(</span>int <em>seconds</em><span class="sig-paren">)</span><a class="headerlink" href="#_CPPv2N6paddle5Queue15waitNotEmptyForEi" title="Permalink to this definition"></a></dt>
<dd><p>wait queue is not empty at most for some seconds. </p>
<p><dl class="docutils">
<dt><strong>Return</strong></dt>
<dd>true if queue is not empty. false if timeout. </dd>
<dt><strong>Parameters</strong></dt>
<dd><ul class="breatheparameterlist first last">
<li><code class="first docutils literal"><span class="pre">seconds</span></code> - <p>wait time limit. </p>
</li>
</ul>
</dd>
</dl>
</p>
</dd></dl>
</div>
</dd></dl>
......
doc/ui/api/trainer_config_helpers/layers_index.html
浏览文件 @ 57e43320
......@@ -97,7 +97,6 @@ var _hmt = _hmt || [];
</li>
<li class="toctree-l1"><a class="reference internal" href="layers.html#image-pooling-layer">Image Pooling Layer</a><ul>
<li class="toctree-l2"><a class="reference internal" href="layers.html#img-pool-layer">img_pool_layer</a></li>
<li class="toctree-l2"><a class="reference internal" href="layers.html#maxout-layer">maxout_layer</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="layers.html#norm-layer">Norm Layer</a><ul>
......@@ -115,7 +114,6 @@ var _hmt = _hmt || [];
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="layers.html#recurrent-layer-group">Recurrent Layer Group</a><ul>
<li class="toctree-l2"><a class="reference internal" href="layers.html#memory">memory</a></li>
<li class="toctree-l2"><a class="reference internal" href="layers.html#recurrent-group">recurrent_group</a></li>
<li class="toctree-l2"><a class="reference internal" href="layers.html#beam-search">beam_search</a></li>
<li class="toctree-l2"><a class="reference internal" href="layers.html#get-output-layer">get_output_layer</a></li>
......@@ -171,7 +169,6 @@ var _hmt = _hmt || [];
<li class="toctree-l2"><a class="reference internal" href="layers.html#crf-layer">crf_layer</a></li>
<li class="toctree-l2"><a class="reference internal" href="layers.html#crf-decoding-layer">crf_decoding_layer</a></li>
<li class="toctree-l2"><a class="reference internal" href="layers.html#ctc-layer">ctc_layer</a></li>
<li class="toctree-l2"><a class="reference internal" href="layers.html#nce-layer">nce_layer</a></li>
<li class="toctree-l2"><a class="reference internal" href="layers.html#hsigmoid">hsigmoid</a></li>
</ul>
</li>
......
doc/ui/api/trainer_config_helpers/networks.html
浏览文件 @ 57e43320
......@@ -202,6 +202,7 @@ False if no bias.</li>
<li><strong>bn_bias_attr</strong> &#8211; see batch_norm_layer&#8217;s document.</li>
<li><strong>bn_layer_attr</strong> &#8211; ParameterAttribute.</li>
<li><strong>pool_stride</strong> (<em>int</em>) &#8211; see img_pool_layer&#8217;s document.</li>
<li><strong>pool_start</strong> (<em>int</em>) &#8211; see img_pool_layer&#8217;s document.</li>
<li><strong>pool_padding</strong> (<em>int</em>) &#8211; see img_pool_layer&#8217;s document.</li>
<li><strong>pool_layer_attr</strong> (<a class="reference internal" href="attrs.html#paddle.trainer_config_helpers.attrs.ExtraLayerAttribute" title="paddle.trainer_config_helpers.attrs.ExtraLayerAttribute"><em>ExtraLayerAttribute</em></a>) &#8211; see img_pool_layer&#8217;s document.</li>
</ul>
......@@ -279,9 +280,10 @@ False if no bias.</li>
<li><strong>param_attr</strong> (<a class="reference internal" href="attrs.html#paddle.trainer_config_helpers.attrs.ParameterAttribute" title="paddle.trainer_config_helpers.attrs.ParameterAttribute"><em>ParameterAttribute</em></a>) &#8211; see img_conv_layer for details</li>
<li><strong>shared_bias</strong> (<em>bool</em>) &#8211; see img_conv_layer for details</li>
<li><strong>conv_layer_attr</strong> (<a class="reference internal" href="attrs.html#paddle.trainer_config_helpers.attrs.ExtraLayerAttribute" title="paddle.trainer_config_helpers.attrs.ExtraLayerAttribute"><em>ExtraLayerAttribute</em></a>) &#8211; see img_conv_layer for details</li>
<li><strong>pool_stride</strong> (<em>int</em>) &#8211; see img_pool_layer for details</li>
<li><strong>pool_padding</strong> (<em>int</em>) &#8211; see img_pool_layer for details</li>
<li><strong>pool_layer_attr</strong> (<a class="reference internal" href="attrs.html#paddle.trainer_config_helpers.attrs.ExtraLayerAttribute" title="paddle.trainer_config_helpers.attrs.ExtraLayerAttribute"><em>ExtraLayerAttribute</em></a>) &#8211; see img_pool_layer for details</li>
<li><strong>pool_stride</strong> (<em>int</em>) &#8211; see img_conv_layer for details</li>
<li><strong>pool_start</strong> (<em>int</em>) &#8211; see img_conv_layer for details</li>
<li><strong>pool_padding</strong> (<em>int</em>) &#8211; see img_conv_layer for details</li>
<li><strong>pool_layer_attr</strong> (<a class="reference internal" href="attrs.html#paddle.trainer_config_helpers.attrs.ExtraLayerAttribute" title="paddle.trainer_config_helpers.attrs.ExtraLayerAttribute"><em>ExtraLayerAttribute</em></a>) &#8211; see img_conv_layer for details</li>
</ul>
</td>
</tr>
......@@ -749,13 +751,14 @@ compute attention weight.</li>
<dl class="function">
<dt>
<code class="descclassname">paddle.trainer_config_helpers.networks.</code><code class="descname">outputs</code><span class="sig-paren">(</span><em>layers</em>, <em>*args</em><span class="sig-paren">)</span></dt>
<dd><p>Declare the outputs of network. If user have not defined the inputs of
network, this method will calculate the input order by dfs travel.</p>
<dd><p>Declare the end of network. Currently it will only calculate the
input/output order of network. It will calculate the predict network or
train network&#8217;s output automatically.</p>
<table class="docutils field-list" frame="void" rules="none">
<col class="field-name" />
<col class="field-body" />
<tbody valign="top">
<tr class="field-odd field"><th class="field-name">Parameters:</th><td class="field-body"><strong>layers</strong> (<em>list|tuple|LayerOutput</em>) &#8211; Output layers.</td>
<tr class="field-odd field"><th class="field-name">Parameters:</th><td class="field-body"><strong>layers</strong> (<em>list|tuple|LayerOutput</em>) &#8211; </td>
</tr>
<tr class="field-even field"><th class="field-name">Returns:</th><td class="field-body"></td>
</tr>
......
doc/ui/data_provider/pydataprovider2.html
浏览文件 @ 57e43320
......@@ -199,10 +199,10 @@ the <code class="code docutils literal"><span class="pre">dataprovider</span></c
<span class="c1"># Define a py data provider</span>
<span class="nd">@provider</span><span class="p">(</span><span class="n">input_types</span><span class="o">=</span><span class="p">{</span>
<span class="s1">&#39;pixel&#39;</span><span class="p">:</span> <span class="n">dense_vector</span><span class="p">(</span><span class="mi">28</span> <span class="o">*</span> <span class="mi">28</span><span class="p">),</span>
<span class="s1">&#39;label&#39;</span><span class="p">:</span> <span class="n">integer_value</span><span class="p">(</span><span class="mi">10</span><span class="p">)</span>
<span class="p">})</span>
<span class="nd">@provider</span><span class="p">(</span><span class="n">input_types</span><span class="o">=</span><span class="p">[</span>
<span class="n">dense_vector</span><span class="p">(</span><span class="mi">28</span> <span class="o">*</span> <span class="mi">28</span><span class="p">),</span>
<span class="n">integer_value</span><span class="p">(</span><span class="mi">10</span><span class="p">)</span>
<span class="p">])</span>
<span class="k">def</span> <span class="nf">process</span><span class="p">(</span><span class="n">settings</span><span class="p">,</span> <span class="n">filename</span><span class="p">):</span> <span class="c1"># settings is not used currently.</span>
<span class="n">f</span> <span class="o">=</span> <span class="nb">open</span><span class="p">(</span><span class="n">filename</span><span class="p">,</span> <span class="s1">&#39;r&#39;</span><span class="p">)</span> <span class="c1"># open one of training file</span>
......@@ -217,7 +217,7 @@ the <code class="code docutils literal"><span class="pre">dataprovider</span></c
<span class="n">pixels_float</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="nb">float</span><span class="p">(</span><span class="n">each_pixel_str</span><span class="p">))</span>
<span class="c1"># give data to paddle.</span>
<span class="k">yield</span> <span class="p">{</span><span class="s2">&quot;pixel&quot;</span><span class="p">:</span> <span class="n">pixels_float</span><span class="p">,</span> <span class="s1">&#39;label&#39;</span><span class="p">:</span> <span class="nb">int</span><span class="p">(</span><span class="n">label</span><span class="p">)}</span>
<span class="k">yield</span> <span class="p">{</span> <span class="s2">&quot;pixel&quot;</span><span class="p">:</span> <span class="n">pixels_float</span><span class="p">,</span> <span class="s1">&#39;label&#39;</span><span class="p">:</span> <span class="nb">int</span><span class="p">(</span><span class="n">label</span><span class="p">)</span> <span class="p">}</span>
<span class="n">f</span><span class="o">.</span><span class="n">close</span><span class="p">()</span> <span class="c1"># close file</span>
</pre></div>
......@@ -355,7 +355,7 @@ Please refer to the following section reference for details.</p>
<h3>&#64;provider<a class="headerlink" href="#provider" title="Permalink to this headline"></a></h3>
<dl class="function">
<dt id="paddle.trainer.PyDataProvider2.provider">
<code class="descclassname">paddle.trainer.PyDataProvider2.</code><code class="descname">provider</code><span class="sig-paren">(</span><em>input_types=None</em>, <em>should_shuffle=None</em>, <em>pool_size=-1</em>, <em>min_pool_size=-1</em>, <em>can_over_batch_size=True</em>, <em>calc_batch_size=None</em>, <em>cache=0</em>, <em>check=False</em>, <em>check_fail_continue=False</em>, <em>init_hook=None</em>, <em>**kwargs</em><span class="sig-paren">)</span><a class="headerlink" href="#paddle.trainer.PyDataProvider2.provider" title="Permalink to this definition"></a></dt>
<code class="descclassname">paddle.trainer.PyDataProvider2.</code><code class="descname">provider</code><span class="sig-paren">(</span><em>input_types=None</em>, <em>should_shuffle=None</em>, <em>pool_size=-1</em>, <em>min_pool_size=-1</em>, <em>can_over_batch_size=True</em>, <em>calc_batch_size=None</em>, <em>cache=0</em>, <em>check=False</em>, <em>check_fail_continue=False</em>, <em>use_dynamic_order=True</em>, <em>init_hook=None</em>, <em>**kwargs</em><span class="sig-paren">)</span><a class="headerlink" href="#paddle.trainer.PyDataProvider2.provider" title="Permalink to this definition"></a></dt>
<dd><p>Provider decorator. Use it to make a function into PyDataProvider2 object.
In this function, user only need to get each sample for some train/test
file.</p>
......@@ -373,13 +373,8 @@ file.</p>
<col class="field-body" />
<tbody valign="top">
<tr class="field-odd field"><th class="field-name">Parameters:</th><td class="field-body"><ul class="first last simple">
<li><strong>input_types</strong> (<em>list|tuple|dict</em>) &#8211; Specify the input types, can also be set in init_hook.
It could be a list of InputType object. For example,
input_types=[dense_vector(9), integer_value(2)]. Or user
can set a dict of InputType object, which key is
data_layer&#8217;s name. For example, input_types= {&#8216;img&#8217;: img_features, &#8216;label&#8217;: label}. when using dict of
InputType, user could yield a dict of feature values, which
key is also data_layer&#8217;s name.</li>
<li><strong>input_types</strong> (<em>list|tuple</em>) &#8211; Specify the input types, can also be set in init_hook.
It is a list of InputType object. For example, input_types= [dense_vector(9), integer_value(2)].</li>
<li><strong>should_shuffle</strong> (<em>bool</em>) &#8211; True if data should shuffle. Pass None means shuffle
when is training and not to shuffle when is testing.</li>
<li><strong>pool_size</strong> (<em>int</em>) &#8211; Max number of sample in data pool.</li>
......@@ -414,6 +409,10 @@ for debug. Default is disabled.</li>
<li><strong>check_fail_continue</strong> (<em>bool</em>) &#8211; Continue train or not when check failed. Just
drop the wrong format data when it is True. Has
no effect when check set to False.</li>
<li><strong>use_dynamic_order</strong> (<em>bool</em>) &#8211; Allow provider to yield a dictionary object, whose
key is a input data layer name, and value is the
feature value. The tuples are still allowed when
use_dynmaic_order is True.</li>
</ul>
</td>
</tr>
......
doc/ui/index.html
浏览文件 @ 57e43320
......@@ -30,7 +30,7 @@
<link rel="search" title="Search" href="../search.html" />
<link rel="top" title="PaddlePaddle documentation" href="../index.html" />
<link rel="next" title="DataProvider Introduction" href="data_provider/index.html" />
<link rel="prev" title="Contribute to PaddlePaddle" href="../build/contribute_to_paddle.html" />
<link rel="prev" title="Debian Package installation guide" href="../build/ubuntu_install.html" />
<script>
var _hmt = _hmt || [];
(function() {
......@@ -56,7 +56,7 @@ var _hmt = _hmt || [];
<a href="data_provider/index.html" title="DataProvider Introduction"
accesskey="N">next</a> |</li>
<li class="right" >
<a href="../build/contribute_to_paddle.html" title="Contribute to PaddlePaddle"
<a href="../build/ubuntu_install.html" title="Debian Package installation guide"
accesskey="P">previous</a> |</li>
<li class="nav-item nav-item-0"><a href="../index.html">PaddlePaddle documentation</a> &#187;</li>
</ul>
......@@ -124,8 +124,8 @@ var _hmt = _hmt || [];
</ul>
<h4>Previous topic</h4>
<p class="topless"><a href="../build/contribute_to_paddle.html"
title="previous chapter">Contribute to PaddlePaddle</a></p>
<p class="topless"><a href="../build/ubuntu_install.html"
title="previous chapter">Debian Package installation guide</a></p>
<h4>Next topic</h4>
<p class="topless"><a href="data_provider/index.html"
title="next chapter">DataProvider Introduction</a></p>
......@@ -163,7 +163,7 @@ var _hmt = _hmt || [];
<a href="data_provider/index.html" title="DataProvider Introduction"
>next</a> |</li>
<li class="right" >
<a href="../build/contribute_to_paddle.html" title="Contribute to PaddlePaddle"
<a href="../build/ubuntu_install.html" title="Debian Package installation guide"
>previous</a> |</li>
<li class="nav-item nav-item-0"><a href="../index.html">PaddlePaddle documentation</a> &#187;</li>
</ul>
......
doc_cn/.buildinfo
浏览文件 @ 57e43320
# Sphinx build info version 1
# This file hashes the configuration used when building these files. When it is not found, a full rebuild will be done.
config: 8f9e3b6337374f468cc7e48534c4662a
config: 70a318b9e7a63a79aedc16f559247671
tags: 645f666f9bcd5a90fca523b33c5a78b7
doc_cn/_sources/algorithm/rnn/hierarchical-layer.txt 已删除 100644 → 0
浏览文件 @ 438f7337
# 支持双层序列作为输入的Layer
## 概述
在自然语言处理任务中，序列是一种常见的数据类型。一个独立的词语，可以看作是一个非序列输入，或者，我们称之为一个0层的序列；由词语构成的句子，是一个单层序列；若干个句子构成一个段落，是一个双层的序列。
双层序列是一个嵌套的序列，它的每一个元素，又是一个单层的序列。这是一种非常灵活的数据组织方式，帮助我们构造一些复杂的输入信息。
我们可以按照如下层次定义非序列，单层序列，以及双层序列。
+ 0层序列：一个独立的元素，类型可以是PaddlePaddle支持的任意输入数据类型
+ 单层序列：排成一列的多个元素，每个元素是一个0层序列，元素之间的顺序是重要的输入信息
+ 双层序列：排成一列的多个元素，每个元素是一个单层序列，称之为双层序列的一个子序列（subseq），subseq的每个元素是一个0层序列
在 PaddlePaddle中，下面这些Layer能够接受双层序列作为输入，完成相应的计算。
## pooling_layer
pooling_layer的使用示例如下，详细见<a href = "../../../doc/ui/api/trainer_config_helpers/layers.html#pooling-layer">配置API</a>。
```python
seq_pool = pooling_layer(input=layer,
pooling_type=AvgPooling(),
agg_level=AggregateLevel.EACH_SEQUENCE)
```
- `pooling_type` 目前支持两种，分别是：MaxPooling()和AvgPooling()。
- `agg_level=AggregateLevel.TIMESTEP`时（默认值）：
- 作用：双层序列经过运算变成一个0层序列，或单层序列经过运算变成一个0层序列
- 输入：一个双层序列，或一个单层序列
- 输出：一个0层序列，即整个输入序列（单层或双层）的平均值（或最大值）
- `agg_level=AggregateLevel.EACH_SEQUENCE`时：
- 作用：一个双层序列经过运算变成一个单层序列
- 输入：必须是一个双层序列
- 输出：一个单层序列，序列的每个元素是原来双层序列每个subseq元素的平均值（或最大值）
## last_seq 和 first_seq
last_seq的使用示例如下（first_seq类似），详细见<a href = "../../../doc/ui/api/trainer_config_helpers/layers.html#last-seq">配置API</a>。
```python
last = last_seq(input=layer,
agg_level=AggregateLevel.EACH_SEQUENCE)
```
- `agg_level=AggregateLevel.TIMESTEP`时（默认值）：
- 作用：一个双层序列经过运算变成一个0层序列，或一个单层序列经过运算变成一个0层序列
- 输入：一个双层序列或一个单层序列
- 输出：一个0层序列，即整个输入序列（双层或者单层）最后一个，或第一个元素。
- `agg_level=AggregateLevel.EACH_SEQUENCE`时：
- 作用：一个双层序列经过运算变成一个单层序列
- 输入：必须是一个双层序列
- 输出：一个单层序列，其中每个元素是双层序列中每个subseq最后一个（或第一个）元素。
## expand_layer
expand_layer的使用示例如下，详细见<a href = "../../../doc/ui/api/trainer_config_helpers/layers.html#expand-layer">配置API</a>。
```python
expand = expand_layer(input=layer1,
expand_as=layer2,
expand_level=ExpandLevel.FROM_TIMESTEP)
```
- `expand_level=ExpandLevel.FROM_TIMESTEP`时（默认值）：
- 作用：一个0层序列经过运算扩展成一个单层序列，或者一个双层序列
- 输入：layer1必须是一个0层序列，是待扩展的数据；layer2可以是一个单层序列，或者是一个双层序列，提供扩展的长度信息
- 输出：一个单层序列，或一个双层序列，输出序列的类型（双层序列，或单层序列）和序列中含有元素的数目同 layer2一致。若输出是单层序列，单层序列的每个元素（0层序列），都是对layer1元素的拷贝；若输出是双层序列，双层序列每个subseq中每个元素（0层序列），都是对layer1元素的拷贝
- `expand_level=ExpandLevel.FROM_SEQUENCE`时：
- 作用：一个单层序列经过运算扩展成一个双层序列
- 输入：layer1必须是一个单层序列，是待扩展的数据；layer2必须是一个双层序列，提供扩展的长度信息
- 输出：一个双层序列，序列中含有元素的数目同layer2一致。要求单层序列含有元素的数目（0层序列），和双层序列含有subseq 的数目一致。单层序列第i个元素（0层序列），被扩展为一个单层序列，构成了输出双层序列的第i个subseq。
\ No newline at end of file
doc_cn/_sources/algorithm/rnn/rnn-tutorial.txt 已删除 100644 → 0
浏览文件 @ 438f7337
# Recurrent Group教程
## 概述
序列数据是自然语言处理任务面对的一种主要输入数据类型。
一句话是由词语构成的序列，多句话进一步构成了段落。因此，段落可以看作是一个嵌套的双层的序列，这个序列的每个元素又是一个序列。
双层序列是PaddlePaddle支持的一种非常灵活的数据组织方式，帮助我们更好地描述段落、多轮对话等更为复杂的语言数据。基于双层序列输入，我们可以设计搭建一个灵活的、层次化的RNN，分别从词语和句子级别编码输入数据，同时也能够引入更加复杂的记忆机制，更好地完成一些复杂的语言理解任务。
在PaddlePaddle中，`recurrent_group`是一种任意复杂的RNN单元，用户只需定义RNN在一个时间步内完成的计算，PaddlePaddle负责完成信息和误差在时间序列上的传播。
更进一步，`recurrent_group`同样可以扩展到双层序列的处理上。通过两个嵌套的`recurrent_group`分别定义子句级别和词语级别上需要完成的运算，最终实现一个层次化的复杂RNN。
目前，在PaddlePaddle中，能够对双向序列进行处理的有`recurrent_group`和部分Layer，具体可参考文档：<a href = "hierarchical-layer.html">支持双层序列作为输入的Layer</a>。
## 相关概念
### 基本原理
`recurrent_group` 是PaddlePaddle支持的一种任意复杂的RNN单元。使用者只需要关注于设计RNN在一个时间步之内完成的计算，PaddlePaddle负责完成信息和梯度在时间序列上的传播。
PaddlePaddle中，`recurrent_group`的一个简单调用如下：
``` python
recurrent_group(step, input, reverse)
```
- step：一个可调用的函数，定义一个时间步之内RNN单元完成的计算
- input：输入，必须是一个单层序列，或者一个双层序列
- reverse：是否以逆序处理输入序列
使用`recurrent_group`的核心是设计step函数的计算逻辑。step函数内部可以自由组合PaddlePaddle支持的各种layer，完成任意的运算逻辑。`recurrent_group` 的输入（即input）会成为step函数的输入，由于step 函数只关注于RNN一个时间步之内的计算，在这里`recurrent_group`替我们完成了原始输入数据的拆分。
### 输入
`recurrent_group`处理的输入序列主要分为以下三种类型：
- **数据输入**：一个双层序列进入`recurrent_group`会被拆解为一个单层序列，一个单层序列进入`recurrent_group`会被拆解为非序列，然后交给step函数，这一过程对用户是完全透明的。可以有以下两种：1）通过data_layer拿到的用户输入；2）其它layer的输出。
- **只读Memory输入**：`StaticInput` 定义了一个只读的Memory，由`StaticInput`指定的输入不会被`recurrent_group`拆解，`recurrent_group` 循环展开的每个时间步总是能够引用所有输入，可以是一个非序列，或者一个单层序列。
- **序列生成任务的输入**：`GeneratedInput`只用于在序列生成任务中指定输入数据。
### 输入示例
序列生成任务大多遵循encoder-decoer架构，encoder和decoder可以是能够处理序列的任意神经网络单元，而RNN是最流行的选择。
给定encoder输出和当前词，decoder每次预测产生下一个最可能的词语。在这种结构中，decoder接受两个输入：
- 要生成的目标序列：是decoder的数据输入，也是decoder循环展开的依据，`recurrent_group`会对这类输入进行拆解。
- encoder输出，可以是一个非序列，或者一个单层序列：是一个unbounded memory，decoder循环展开的每一个时间步会引用全部结果，不应该被拆解，这种类型的输入必须通过`StaticInput`指定。关于Unbounded Memory的更多讨论请参考论文 [Neural Turning Machine](https://arxiv.org/abs/1410.5401)。
在序列生成任务中，decoder RNN总是引用上一时刻预测出的词的词向量，作为当前时刻输入。`GeneratedInput`自动完成这一过程。
### 输出
`step`函数必须返回一个或多个Layer的输出，这个Layer的输出会作为整个`recurrent_group` 最终的输出结果。在输出的过程中，`recurrent_group` 会将每个时间步的输出拼接，这个过程对用户也是透明的。
### memory
memory只能在`recurrent_group`中定义和使用。memory不能独立存在，必须指向一个PaddlePaddle定义的Layer。引用memory得到这layer上一时刻输出，因此，可以将memory理解为一个时延操作。
可以显示地指定一个layer的输出用于初始化memory。不指定时，memory默认初始化为0。
## 双层RNN介绍
`recurrent_group`帮助我们完成对输入序列的拆分，对输出的合并，以及计算逻辑在序列上的循环展开。
利用这种特性，两个嵌套的`recurrent_group`能够处理双层序列，实现词语和句子两个级别的双层RNN结构。
- 单层（word-level）RNN：每个状态（state）对应一个词（word）。
- 双层（sequence-level）RNN：一个双层RNN由多个单层RNN组成，每个单层RNN（即双层RNN的每个状态）对应一个子句（subseq）。
为了描述方便，下文以NLP任务为例，将含有子句（subseq）的段落定义为一个双层序列，将含有词语的句子定义为一个单层序列，那么0层序列即为一个词语。
## 双层RNN的使用
### 训练流程的使用方法
使用 `recurrent_group`需要遵循以下约定：
- **单进单出**：输入和输出都是单层序列。
- 如果有多个输入，不同输入序列含有的词语数必须严格相等。
- 输出一个单层序列，输出序列的词语数和输入序列一致。
- memory：在step函数中定义 memory指向一个layer，通过引用memory得到这个layer上一个时刻输出，形成recurrent 连接。memory的is_seq参数必须为false。如果没有定义memory，每个时间步之内的运算是独立的。
- boot_layer：memory的初始状态，默认初始状为0，memory的is_seq参数必须为false。
- **双进双出**：输入和输出都是双层序列。
- 如果有多个输入序列，不同输入含有的子句（subseq）数必须严格相等，但子句含有的词语数可以不相等。
- 输出一个双层序列，子句（subseq）数、子句的单词数和指定的一个输入序列一致，默认为第一个输入。
- memory：在step函数中定义memory，指向一个layer，通过引用memory得到这个layer上一个时刻的输出，形成recurrent连接。定义在外层`recurrent_group` step函数中的memory，能够记录上一个subseq 的状态，可以是一个单层序列（只作为read-only memory），也可以是一个词语。如果没有定义memory，那么 subseq 之间的运算是独立的。
- boot_layer：memory 初始状态，可以是一个单层序列（只作为read-only memory）或一个向量。默认不设置，即初始状态为0。
- **双进单出**：目前还未支持，会报错"In hierachical RNN, all out links should be from sequences now"。
### 生成流程的使用方法
使用`beam_search`需要遵循以下约定：
- 单层RNN：从一个word生成下一个word。
- 双层RNN：即把单层RNN生成后的subseq给拼接成一个新的双层seq。从语义上看，也不存在一个subseq直接生成下一个subseq的情况。
\ No newline at end of file
doc_cn/_sources/build_and_install/index.txt
浏览文件 @ 57e43320
编译与安装
========================
安装
++++
PaddlePaddle提供数个预编译的二进制来进行安装，包括Docker镜像，ubuntu的deb安装包等。我们推荐使用Docker镜像来部署环境，同时欢迎贡献更多的安装包。
.. toctree::
:maxdepth: 1
:glob:
使用Jumbo安装(对内) <../build/internal/install_from_jumbo.rst>
install/docker_install.rst
install/ubuntu_install.rst
Note: The intallation packages are still in pre-release state and your experience of installation may not be smooth.
编译
++++
.. warning::
编译选项主要推荐高级用户查看，普通用户请走安装流程。
注意：目前PaddlePaddle的安装包还处在pre-release的状态，使用起来或许会不是很顺畅。
.. toctree::
:maxdepth: 1
:glob:
源码下载(对内) <../build/internal/download_paddle_source_zh_cn.rst>
使用Jumbo安装(对内) <../build/internal/install_from_jumbo.rst>
从源码编译安装(对内) <../build/internal/build_from_source_zh_cn.rst>
install/docker_install.rst
install/ubuntu_install.rst
cmake/index.rst
doc_cn/_sources/build_and_install/install/docker_install.txt
浏览文件 @ 57e43320
......@@ -14,43 +14,20 @@ PaddlePaddle提供了Docker的使用镜像。PaddlePaddle推荐使用Docker进
PaddlePaddle提供的Docker镜像版本
--------------------------------
我们提供了12个 `Docker image <https://hub.docker.com/r/paddledev/paddle/tags/>`_ ，他们的image name都是 :code:`paddle-dev/paddle` ，tag分别为
我们提供了6个Docker image\:
+-----------------+------------------+------------------------+-----------------------+
| | normal | devel | demo |
+=================+==================+========================+=======================+
| CPU | cpu-latest | cpu-devel-latest | cpu-demo-latest |
+-----------------+------------------+------------------------+-----------------------+
| GPU | gpu-latest | gpu-devel-latest | gpu-demo-latest |
+-----------------+------------------+------------------------+-----------------------+
| CPU WITHOUT AVX | cpu-noavx-latest | cpu-noavx-devel-latest | cpu-noavx-demo-latest |
+-----------------+------------------+------------------------+-----------------------+
| GPU WITHOUT AVX | gpu-noavx-latest | gpu-noavx-devel-latest | gpu-noavx-demo-latest |
+-----------------+------------------+------------------------+-----------------------+
* paddledev/paddle\:cpu-latest\: PaddlePaddle的CPU二进制
* paddledev/paddle\:gpu-latest\： PaddlePaddle的GPU二进制
* paddledev/paddle\:cpu-devel-latest\: PaddlePaddle的CPU二进制,同时包含CPU开发环境和源码
* paddledev/paddle\:gpu-devel-latest\: PaddlePaddle的GPU二进制,同时包含GPU开发环境和源码
* paddledev/paddle\:cpu-demo-latest\: PaddlePaddle的CPU二进制,同时包含CPU开发环境、源码和运行demo的必要依赖
* paddledev/paddle\:gpu-demo-latest\: PaddlePaddle的GPU二进制,同时包含GPU开发环境、源码和运行demo的必要依赖
其中，横向包括三个版本，normal，devel和demo。
* Normal: 正常的Docker image，只包括paddle的二进制
* Devel: 包括Paddle的二进制、编译环境和源代码
* Demo: 包括Paddle运行demo所需要的依赖
纵向包括四个版本，他们是。
* CPU: CPU版本。需要支持AVX指令集的CPU
* GPU: GPU版本。需要支持AVX指令集的CPU
* CPU WITHOUT AVX: CPU版本，不支持AVX指令集的CPU也可以运行
* GPU WITHOUT AVX: GPU版本，不需要AVX指令集的CPU也可以运行。
用户可以选择对应版本的docker image。使用如下脚本可以确定本机的CPU知否支持 :code:`AVX` 指令集\:
.. code-block:: bash
if cat /proc/cpuinfo | grep -q avx ; then echo "Support AVX"; else echo "Not support AVX"; fi
如果输出 :code:`Support AVX`，则可以选择上表中的AVX版本PaddlePaddle。否则需要选择非AVX的PaddlePaddle。选择普通CPU版本的devel版本的image，则可以使用 :code:`paddle-dev/paddle:cpu-devel-latest` 来引用这个image。
同时，不同的稳定版本，会将latest替换成稳定版本的版本号。
PaddlePaddle提供的镜像并不包含任何命令运行，想要运行PaddlePaddle，您需要进入镜像运行PaddlePaddle
程序或者自定义一个含有启动脚本的image。具体请参考注意事项中的 :code:`使用ssh访问PaddlePaddle镜像`
程序或者自定义一个含有启动脚本的image。具体请参考注意事项中的
`使用ssh访问PaddlePaddle镜像`
下载和运行Docker镜像
--------------------
......@@ -67,7 +44,7 @@ mac osx或者是windows机器，请参考
.. code-block:: bash
$ docker run -it paddledev/paddlepaddle:cpu-latest
$ docker run -it paddledev/paddlepaddle:latest-cpu
即可启动和进入PaddlePaddle的container。如果运行GPU版本的PaddlePaddle，则需要先将
cuda相关的Driver和设备映射进container中，脚本类似于
......@@ -76,7 +53,7 @@ cuda相关的Driver和设备映射进container中，脚本类似于
$ export CUDA_SO="$(\ls /usr/lib64/libcuda* | xargs -I{} echo '-v {}:{}') $(\ls /usr/lib64/libnvidia* | xargs -I{} echo '-v {}:{}')"
$ export DEVICES=$(\ls /dev/nvidia* | xargs -I{} echo '--device {}:{}')
$ docker run ${CUDA_SO} ${DEVICES} -it paddledev/paddlepaddle:latest-gpu
$ docker run -it paddledev/paddlepaddle:latest-gpu
进入Docker container后，运行 :code:`paddle version` 即可打印出PaddlePaddle的版本和构建
信息。安装完成的PaddlePaddle主体包括三个部分， :code:`paddle` 脚本， python的
......
doc_cn/_sources/build_and_install/install/ubuntu_install.txt
浏览文件 @ 57e43320
使用deb包在Ubuntu上安装PaddlePaddle
===================================
PaddlePaddle目前支持使用deb包安装。Paddle的 :code:`deb` 安装包在ubuntu 14.04中正确，但理论上支持其他的 debian 发行版。
PaddlePaddle目前支持ubuntu 14.04版本使用deb包安装。更多的安装包PaddlePaddle会在近期提供。
欢迎大家贡献各个发行版的安装包(例如，ubuntu，centos，debian，gentoo)。
PaddlePaddle的ubuntu安装包分为两个版本，即CPU版本，和GPU版本，他们的下载地址是\:
https://github.com/baidu/Paddle/releases/tag/V0.8.0b0
PaddlePaddle的ubuntu安装包分为四个版本，他们是 cpu、gpu、cpu-noavx、gpu-noavx 四个版本。其中 noavx 用于不支持AVX指令集的cpu。安装包的下载地址是\: https://github.com/baidu/Paddle/releases/
需要注意的是，目前PaddlePaddle的安装包只支持
`AVX <https://en.wikipedia.org/wiki/Advanced_Vector_Extensions>`_
指令集的X86 CPU。如果系统使用不支持 `AVX`_ 指令集的CPU运行PaddlePaddle，那么需要从源码
编译PaddlePaddle，请参考 `编译文档 <../cmake/index.html>`_ 。
用户需要先将PaddlePaddle安装包下载到本地，然后执行如下 :code:`gdebi` 命令即可完成安装。
.. code-block:: shell
gdebi paddle-*-cpu.deb
如果 :code:`gdebi` 没有安装,则需要使用 :code:`sudo apt-get install gdebi`, 来安装 :code:`gdebi` 。
或者使用下面一条命令安装.
用户需要先将PaddlePaddle安装包下载到本地，然后执行如下命令即可完成安装。
.. code-block:: shell
dpkg -i paddle-*-cpu.deb
dpkg -i paddle-0.8.0b-cpu.deb
apt-get install -f
在 :code:`dpkg -i` 的时候如果报一些依赖未找到的错误是正常的，
在 :code:`apt-get install -f` 里会继续安装 PaddlePaddle。
需要注意的是，如果使用GPU版本的PaddlePaddle，请安装CUDA 7.5 和CUDNN 5到本地环境中，
并设置好对应的环境变量(LD_LIBRARY_PATH等等)。
安装完成后,可以使用命令 :code:`paddle version` 查看安装后的paddle 版本。可能的输出为
.. literalinclude:: paddle_version.txt
可能遇到的问题
--------------
......
doc_cn/_sources/faq/index.txt 已删除 100644 → 0
浏览文件 @ 438f7337
####################
PaddlePaddle常见问题
####################
.. contents::
1. 如何减少PaddlePaddle的内存占用
---------------------------------
神经网络的训练本身是一个非常消耗内存和显存的工作。经常会消耗数十G的内存和数G的显存。
PaddlePaddle的内存占用主要分为如下几个方面\:
* DataProvider缓冲池内存 (只针对内存)
* 神经元激活内存 （针对内存和显存）
* 参数内存 (针对内存和显存)
* 其他内存杂项
这其中，其他内存杂项是指PaddlePaddle本身所用的一些内存，包括字符串分配，临时变量等等，
这些内存就不考虑如何缩减了。
其他的内存的减少方法依次为
减少DataProvider缓冲池内存
++++++++++++++++++++++++++
PyDataProvider使用的是异步加载，同时在内存里直接随即选取数据来做Shuffle。即
.. graphviz::
digraph {
rankdir=LR;
数据文件 -> 内存池 -> PaddlePaddle训练
}
所以，减小这个内存池即可减小内存占用，同时也可以加速开始训练前数据载入的过程。但是，这
个内存池实际上决定了shuffle的粒度。所以，如果将这个内存池减小，又要保证数据是随机的，
那么最好将数据文件在每次读取之前做一次shuffle。可能的代码为
.. literalinclude:: reduce_min_pool_size.py
这样做可以极大的减少内存占用，并且可能会加速训练过程。 详细文档参考 `这里
<../ui/data_provider/pydataprovider2.html#provider>`_ 。
神经元激活内存
++++++++++++++
神经网络在训练的时候，会对每一个激活暂存一些数据，包括激活，參差等等。
在反向传递的时候，这些数据会被用来更新参数。这些数据使用的内存主要和两个参数有关系，
一是batch size，另一个是每条序列(Sequence)长度。所以，其实也是和每个mini-batch中包含
的时间步信息成正比。
所以，做法可以有两种。他们是
* 减小batch size。 即在网络配置中 :code:`settings(batch_size=1000)` 设置成一个小一些的值。但是batch size本身是神经网络的超参数，减小batch size可能会对训练结果产生影响。
* 减小序列的长度，或者直接扔掉非常长的序列。比如，一个数据集大部分序列长度是100-200,
但是突然有一个10000长的序列，就很容易导致内存超限。特别是在LSTM等RNN中。
参数内存
++++++++
PaddlePaddle支持非常多的优化算法(Optimizer)，不同的优化算法需要使用不同大小的内存。
例如如果使用 :code:`adadelta` 算法，则需要使用参数规模大约5倍的内存。 如果参数保存下来的
文件为 :code:`100M`， 那么该优化算法至少需要 :code:`500M` 的内存。
可以考虑使用一些优化算法，例如 :code:`momentum`。
2. 如何加速PaddlePaddle的训练速度
---------------------------------
PaddlePaddle是神经网络训练平台，加速PaddlePaddle训练有如下几个方面\：
* 减少数据载入的耗时
* 加速训练速度
* 利用更多的计算资源
减少数据载入的耗时
++++++++++++++++++
使用 :code:`pydataprovider`时，可以减少缓存池的大小，同时设置内存缓存功能，即可以极大的加速数据载入流程。
:code:`DataProvider` 缓存池的减小，和之前减小通过减小缓存池来减小内存占用的原理一致。
.. literalinclude:: reduce_min_pool_size.py
同时 :code:`@provider` 接口有一个 :code:`cache` 参数来控制缓存方法，将其设置成 :code:`CacheType.CACHE_PASS_IN_MEM` 的话，会将第一个 :code:`pass` (过完所有训练数据即为一个pass)生成的数据缓存在内存里，在之后的 :code:`pass` 中，不会再从 :code:`python` 端读取数据，而是直接从内存的缓存里读取数据。这也会极大减少数据读入的耗时。
加速训练速度
++++++++++++
PaddlePaddle支持Sparse的训练，sparse训练需要训练特征是 :code:`sparse_binary_vector` 、 :code:`sparse_vector` 、或者 :code:`integer_value` 的任一一种。同时，与这个训练数据交互的Layer，需要将其Parameter设置成 sparse 更新模式，即设置 :code:`sparse_update=True`
这里使用简单的 :code:`word2vec` 训练语言模型距离，具体使用方法为\:
使用一个词前两个词和后两个词，来预测这个中间的词。这个任务的DataProvider为\:
.. literalinclude:: word2vec_dataprovider.py
这个任务的配置为\:
.. literalinclude:: word2vec_config.py
更多关于sparse训练的内容请参考 `sparse训练的文档 <TBD>`_
利用更多的计算资源
++++++++++++++++++
利用更多的计算资源可以分为一下几个方式来进行\:
* 单机CPU训练
* 使用多线程训练。设置命令行参数 :code:`trainer_count`，即可以设置参与训练的线程数量。使用方法为 :code:`paddle train --trainer_count=4`
* 单机GPU训练
* 使用显卡训练。设置命令行参数 :code:`use_gpu`。 使用方法为 :code:`paddle train --use_gpu=true`
* 使用多块显卡训练。设置命令行参数 :code:`use_gpu` 和 :code:`trainer_count`。使用 :code:`--use_gpu=True` 开启GPU训练，使用 :code:`trainer_count` 指定显卡数量。使用方法为 :code:`paddle train --use_gpu=true --trainer_count=4`
* 多机训练
* 使用多机训练的方法也比较简单，需要先在每个节点启动 :code:`paddle pserver`，在使用 :code:`paddle train --pservers=192.168.100.1,192.168.100.2` 来指定每个pserver的ip地址
* 具体的多机训练方法参考 `多机训练 <TBD>`_ 文档。
3. 遇到“非法指令”或者是“illegal instruction”
--------------------------------------------
paddle在进行计算的时候为了提升计算性能，使用了avx指令。部分老的cpu型号无法支持这样的指令。通常来说执行下grep avx /proc/cpuinfo看看是否有输出即可知道是否支持。（另：用此方法部分虚拟机可能检测到支持avx指令但是实际运行会挂掉，请当成是不支持，看下面的解决方案）
解决办法是\:
* 使用 NO_AVX的 `安装包 <../build_and_install/index.html>`_ 或者 `Docker image <../build_and_install/install/docker_install.html>`_
* 或者，使用 :code:`-DWITH_AVX=OFF` 重新编译PaddlePaddle。
4. 如何选择SGD算法的学习率
--------------------------
在采用sgd/async_sgd进行训练时，一个重要的问题是选择正确的learning_rate。如果learning_rate太大，那么训练有可能不收敛，如果learning_rate太小，那么收敛可能很慢，导致训练时间过长。
通常做法是从一个比较大的learning_rate开始试，如果不收敛，那减少学习率10倍继续试验，直到训练收敛为止。那么如何判断训练不收敛呢？可以估计出如果模型采用不变的输出最小的cost0是多少。
如果训练过程的的cost明显高于这个常数输出的cost，那么我们可以判断为训练不收敛。举一个例子，假如我们是三分类问题，采用multi-class-cross-entropy作为cost，数据中0,1,2三类的比例为 :code:`0.2, 0.5, 0.3` , 那么常数输出所能达到的最小cost是 :code:`-(0.2*log(0.2)+0.5*log(0.5)+0.3*log(0.3))=1.03` 。如果训练一个pass（或者更早）后，cost还大于这个数，那么可以认为训练不收敛，应该降低学习率。
5. 如何初始化参数
-----------------
默认情况下，PaddlePaddle使用均值0，标准差为 :math:`\frac{1}{\sqrt{d}}` 来初始化参数。其中 :math:`d` 为参数矩阵的宽度。这种初始化方式在一般情况下不会产生很差的结果。如果用户想要自定义初始化方式，PaddlePaddle目前提供两种参数初始化的方式\:
* 高斯分布。将 :code:`param_attr` 设置成 :code:`param_attr=ParamAttr(initial_mean=0.0, initial_std=1.0)`
* 均匀分布。将 :code:`param_attr` 设置成 :code:`param_attr=ParamAttr(initial_max=1.0, initial_min=-1.0)`
比如设置一个全连接层的参数初始化方式和bias初始化方式，可以使用如下代码。
.. code-block:: python
hidden = fc_layer(input=ipt, param_attr=ParamAttr(initial_max=1.0, initial_min=-1.0),
bias_attr=ParamAttr(initial_mean=1.0, initial_std=0.0))
上述代码将bias全部初始化为1.0, 同时将参数初始化为 :code:`[1.0, -1.0]` 的均匀分布。
6. 如何共享参数
---------------
PaddlePaddle的参数使用名字 :code:`name` 作为参数的ID，相同名字的参数，会共享参数。设置参数的名字，可以使用 :code:`ParamAttr(name="YOUR_PARAM_NAME")` 来设置。更方便的设置方式，是想要共享的参数使用同样的 :code:`ParamAttr` 对象。
简单的全连接网络，参数共享的配置示例为\:
.. literalinclude:: ../../python/paddle/trainer_config_helpers/tests/configs/shared_fc.py
这里 :code:`hidden_a` 和 :code:`hidden_b` 使用了同样的parameter和bias。并且softmax层的两个输入也使用了同样的参数 :code:`softmax_param`。
doc_cn/_sources/index.txt
浏览文件 @ 57e43320
......@@ -3,7 +3,6 @@ PaddlePaddle文档
使用指南
--------
* `快速入门 <demo/quick_start/index.html>`_
* `编译与安装 <build_and_install/index.html>`_
* `用户接口 <ui/index.html>`_
......@@ -17,13 +16,4 @@ PaddlePaddle文档
算法教程
--------
* `Recurrent Group教程 <algorithm/rnn/rnn-tutorial.html>`_
* `单层RNN示例 <../doc/algorithm/rnn/rnn.html>`_
* `双层RNN示例 <algorithm/rnn/hierarchical-rnn.html>`_
* `支持双层序列作为输入的Layer <algorithm/rnn/hierarchical-layer.html>`_
常见问题
--------
* `常见问题 <faq/index.html>`_
* `RNN配置 <../doc/algorithm/rnn/rnn.html>`_
doc_cn/_sources/ui/data_provider/pydataprovider2.txt
浏览文件 @ 57e43320
......@@ -141,6 +141,8 @@ DataProvider创建的时候执行。这个初始化函数具有如下参数:
是一个batch size，但是有时为了计算均衡性，可以将一条数据设置成多个batch size
* cache 是数据缓存的策略，参考 `cache`_
* init_hook 是初始化时调用的函数，参考 `init_hook`_
* use_dynamic_order 如果是true的话，可以返回一个dict，key是data_layer的名字，value是特征值。同时，也可以
返回一个list或者tuple。如果是false的话，只能够返回list或者tuple
* check 设置成true的话，会根据input_types检查数据的合法性。
* check_fail_continue 如果设置成true的话，即使在check中数据不合法，也会扔到这条数据，继续训练。 如果
check是false的话，没有作用。
......
doc_cn/algorithm/rnn/hierarchical-layer.html 已删除 100644 → 0
浏览文件 @ 438f7337
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>支持双层序列作为输入的Layer &#8212; PaddlePaddle documentation</title>
<link rel="stylesheet" href="../../_static/classic.css" type="text/css" />
<link rel="stylesheet" href="../../_static/pygments.css" type="text/css" />
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT: '../../',
VERSION: '',
COLLAPSE_INDEX: false,
FILE_SUFFIX: '.html',
HAS_SOURCE: true
};
</script>
<script type="text/javascript" src="../../_static/jquery.js"></script>
<script type="text/javascript" src="../../_static/underscore.js"></script>
<script type="text/javascript" src="../../_static/doctools.js"></script>
<script type="text/javascript" src="https://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script>
<link rel="index" title="Index" href="../../genindex.html" />
<link rel="search" title="Search" href="../../search.html" />
<link rel="top" title="PaddlePaddle documentation" href="../../index.html" />
<script>
var _hmt = _hmt || [];
(function() {
var hm = document.createElement("script");
hm.src = "//hm.baidu.com/hm.js?b9a314ab40d04d805655aab1deee08ba";
var s = document.getElementsByTagName("script")[0];
s.parentNode.insertBefore(hm, s);
})();
</script>
</head>
<body role="document">
<div class="related" role="navigation" aria-label="related navigation">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="../../genindex.html" title="General Index"
accesskey="I">index</a></li>
<li class="nav-item nav-item-0"><a href="../../index.html">PaddlePaddle documentation</a> &#187;</li>
</ul>
</div>
<div class="document">
<div class="documentwrapper">
<div class="bodywrapper">
<div class="body" role="main">
<div class="section" id="layer">
<span id="layer"></span><h1>支持双层序列作为输入的Layer<a class="headerlink" href="#layer" title="Permalink to this headline"></a></h1>
<div class="section" id="">
<span id="id1"></span><h2>概述<a class="headerlink" href="#" title="Permalink to this headline"></a></h2>
<p>在自然语言处理任务中，序列是一种常见的数据类型。一个独立的词语，可以看作是一个非序列输入，或者，我们称之为一个0层的序列；由词语构成的句子，是一个单层序列；若干个句子构成一个段落，是一个双层的序列。</p>
<p>双层序列是一个嵌套的序列，它的每一个元素，又是一个单层的序列。这是一种非常灵活的数据组织方式，帮助我们构造一些复杂的输入信息。</p>
<p>我们可以按照如下层次定义非序列，单层序列，以及双层序列。</p>
<ul class="simple">
<li>0层序列：一个独立的元素，类型可以是PaddlePaddle支持的任意输入数据类型</li>
<li>单层序列：排成一列的多个元素，每个元素是一个0层序列，元素之间的顺序是重要的输入信息</li>
<li>双层序列：排成一列的多个元素，每个元素是一个单层序列，称之为双层序列的一个子序列（subseq），subseq的每个元素是一个0层序列</li>
</ul>
<p>在 PaddlePaddle中，下面这些Layer能够接受双层序列作为输入，完成相应的计算。</p>
</div>
<div class="section" id="pooling-layer">
<span id="pooling-layer"></span><h2>pooling_layer<a class="headerlink" href="#pooling-layer" title="Permalink to this headline"></a></h2>
<p>pooling_layer的使用示例如下，详细见<a href = "../../../doc/ui/api/trainer_config_helpers/layers.html#pooling-layer">配置API</a>。</p>
<div class="highlight-python"><div class="highlight"><pre><span></span><span class="n">seq_pool</span> <span class="o">=</span> <span class="n">pooling_layer</span><span class="p">(</span><span class="nb">input</span><span class="o">=</span><span class="n">layer</span><span class="p">,</span>
<span class="n">pooling_type</span><span class="o">=</span><span class="n">AvgPooling</span><span class="p">(),</span>
<span class="n">agg_level</span><span class="o">=</span><span class="n">AggregateLevel</span><span class="o">.</span><span class="n">EACH_SEQUENCE</span><span class="p">)</span>
</pre></div>
</div>
<ul class="simple">
<li><code class="docutils literal"><span class="pre">pooling_type</span></code> 目前支持两种，分别是：MaxPooling()和AvgPooling()。</li>
<li><code class="docutils literal"><span class="pre">agg_level=AggregateLevel.TIMESTEP</span></code>时（默认值）：<ul>
<li>作用：双层序列经过运算变成一个0层序列，或单层序列经过运算变成一个0层序列</li>
<li>输入：一个双层序列，或一个单层序列</li>
<li>输出：一个0层序列，即整个输入序列（单层或双层）的平均值（或最大值）</li>
</ul>
</li>
<li><code class="docutils literal"><span class="pre">agg_level=AggregateLevel.EACH_SEQUENCE</span></code>时：<ul>
<li>作用：一个双层序列经过运算变成一个单层序列</li>
<li>输入：必须是一个双层序列</li>
<li>输出：一个单层序列，序列的每个元素是原来双层序列每个subseq元素的平均值（或最大值）</li>
</ul>
</li>
</ul>
</div>
<div class="section" id="last-seq-first-seq">
<span id="last-seq-first-seq"></span><h2>last_seq 和 first_seq<a class="headerlink" href="#last-seq-first-seq" title="Permalink to this headline"></a></h2>
<p>last_seq的使用示例如下（first_seq类似），详细见<a href = "../../../doc/ui/api/trainer_config_helpers/layers.html#last-seq">配置API</a>。</p>
<div class="highlight-python"><div class="highlight"><pre><span></span><span class="n">last</span> <span class="o">=</span> <span class="n">last_seq</span><span class="p">(</span><span class="nb">input</span><span class="o">=</span><span class="n">layer</span><span class="p">,</span>
<span class="n">agg_level</span><span class="o">=</span><span class="n">AggregateLevel</span><span class="o">.</span><span class="n">EACH_SEQUENCE</span><span class="p">)</span>
</pre></div>
</div>
<ul class="simple">
<li><code class="docutils literal"><span class="pre">agg_level=AggregateLevel.TIMESTEP</span></code>时（默认值）：<ul>
<li>作用：一个双层序列经过运算变成一个0层序列，或一个单层序列经过运算变成一个0层序列</li>
<li>输入：一个双层序列或一个单层序列</li>
<li>输出：一个0层序列，即整个输入序列（双层或者单层）最后一个，或第一个元素。</li>
</ul>
</li>
<li><code class="docutils literal"><span class="pre">agg_level=AggregateLevel.EACH_SEQUENCE</span></code>时：<ul>
<li>作用：一个双层序列经过运算变成一个单层序列</li>
<li>输入：必须是一个双层序列</li>
<li>输出：一个单层序列，其中每个元素是双层序列中每个subseq最后一个（或第一个）元素。</li>
</ul>
</li>
</ul>
</div>
<div class="section" id="expand-layer">
<span id="expand-layer"></span><h2>expand_layer<a class="headerlink" href="#expand-layer" title="Permalink to this headline"></a></h2>
<p>expand_layer的使用示例如下，详细见<a href = "../../../doc/ui/api/trainer_config_helpers/layers.html#expand-layer">配置API</a>。</p>
<div class="highlight-python"><div class="highlight"><pre><span></span><span class="n">expand</span> <span class="o">=</span> <span class="n">expand_layer</span><span class="p">(</span><span class="nb">input</span><span class="o">=</span><span class="n">layer1</span><span class="p">,</span>
<span class="n">expand_as</span><span class="o">=</span><span class="n">layer2</span><span class="p">,</span>
<span class="n">expand_level</span><span class="o">=</span><span class="n">ExpandLevel</span><span class="o">.</span><span class="n">FROM_TIMESTEP</span><span class="p">)</span>
</pre></div>
</div>
<ul class="simple">
<li><code class="docutils literal"><span class="pre">expand_level=ExpandLevel.FROM_TIMESTEP</span></code>时（默认值）：<ul>
<li>作用：一个0层序列经过运算扩展成一个单层序列，或者一个双层序列</li>
<li>输入：layer1必须是一个0层序列，是待扩展的数据；layer2可以是一个单层序列，或者是一个双层序列，提供扩展的长度信息</li>
<li>输出：一个单层序列，或一个双层序列，输出序列的类型（双层序列，或单层序列）和序列中含有元素的数目同 layer2一致。若输出是单层序列，单层序列的每个元素（0层序列），都是对layer1元素的拷贝；若输出是双层序列，双层序列每个subseq中每个元素（0层序列），都是对layer1元素的拷贝</li>
</ul>
</li>
<li><code class="docutils literal"><span class="pre">expand_level=ExpandLevel.FROM_SEQUENCE</span></code>时：<ul>
<li>作用：一个单层序列经过运算扩展成一个双层序列</li>
<li>输入：layer1必须是一个单层序列，是待扩展的数据；layer2必须是一个双层序列，提供扩展的长度信息</li>
<li>输出：一个双层序列，序列中含有元素的数目同layer2一致。要求单层序列含有元素的数目（0层序列），和双层序列含有subseq 的数目一致。单层序列第i个元素（0层序列），被扩展为一个单层序列，构成了输出双层序列的第i个subseq。</li>
</ul>
</li>
</ul>
</div>
</div>
</div>
</div>
</div>
<div class="sphinxsidebar" role="navigation" aria-label="main navigation">
<div class="sphinxsidebarwrapper">
<h3><a href="../../index.html">Table Of Contents</a></h3>
<ul>
<li><a class="reference internal" href="#">支持双层序列作为输入的Layer</a><ul>
<li><a class="reference internal" href="#">概述</a></li>
<li><a class="reference internal" href="#pooling-layer">pooling_layer</a></li>
<li><a class="reference internal" href="#last-seq-first-seq">last_seq 和 first_seq</a></li>
<li><a class="reference internal" href="#expand-layer">expand_layer</a></li>
</ul>
</li>
</ul>
<div role="note" aria-label="source link">
<h3>This Page</h3>
<ul class="this-page-menu">
<li><a href="../../_sources/algorithm/rnn/hierarchical-layer.txt"
rel="nofollow">Show Source</a></li>
</ul>
</div>
<div id="searchbox" style="display: none" role="search">
<h3>Quick search</h3>
<form class="search" action="../../search.html" method="get">
<div><input type="text" name="q" /></div>
<div><input type="submit" value="Go" /></div>
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
</form>
</div>
<script type="text/javascript">$('#searchbox').show(0);</script>
</div>
</div>
<div class="clearer"></div>
</div>
<div class="related" role="navigation" aria-label="related navigation">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="../../genindex.html" title="General Index"
>index</a></li>
<li class="nav-item nav-item-0"><a href="../../index.html">PaddlePaddle documentation</a> &#187;</li>
</ul>
</div>
<div class="footer" role="contentinfo">
&#169; Copyright 2016, PaddlePaddle developers.
Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.4.8.
</div>
</body>
</html>
\ No newline at end of file
doc_cn/algorithm/rnn/hierarchical-rnn.html 已删除 100644 → 0
浏览文件 @ 438f7337
此差异已折叠。
doc_cn/algorithm/rnn/rnn-tutorial.html 已删除 100644 → 0
浏览文件 @ 438f7337
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>Recurrent Group教程 &#8212; PaddlePaddle documentation</title>
<link rel="stylesheet" href="../../_static/classic.css" type="text/css" />
<link rel="stylesheet" href="../../_static/pygments.css" type="text/css" />
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT: '../../',
VERSION: '',
COLLAPSE_INDEX: false,
FILE_SUFFIX: '.html',
HAS_SOURCE: true
};
</script>
<script type="text/javascript" src="../../_static/jquery.js"></script>
<script type="text/javascript" src="../../_static/underscore.js"></script>
<script type="text/javascript" src="../../_static/doctools.js"></script>
<script type="text/javascript" src="https://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script>
<link rel="index" title="Index" href="../../genindex.html" />
<link rel="search" title="Search" href="../../search.html" />
<link rel="top" title="PaddlePaddle documentation" href="../../index.html" />
<script>
var _hmt = _hmt || [];
(function() {
var hm = document.createElement("script");
hm.src = "//hm.baidu.com/hm.js?b9a314ab40d04d805655aab1deee08ba";
var s = document.getElementsByTagName("script")[0];
s.parentNode.insertBefore(hm, s);
})();
</script>
</head>
<body role="document">
<div class="related" role="navigation" aria-label="related navigation">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="../../genindex.html" title="General Index"
accesskey="I">index</a></li>
<li class="nav-item nav-item-0"><a href="../../index.html">PaddlePaddle documentation</a> &#187;</li>
</ul>
</div>
<div class="document">
<div class="documentwrapper">
<div class="bodywrapper">
<div class="body" role="main">
<div class="section" id="recurrent-group">
<span id="recurrent-group"></span><h1>Recurrent Group教程<a class="headerlink" href="#recurrent-group" title="Permalink to this headline"></a></h1>
<div class="section" id="">
<span id="id1"></span><h2>概述<a class="headerlink" href="#" title="Permalink to this headline"></a></h2>
<p>序列数据是自然语言处理任务面对的一种主要输入数据类型。</p>
<p>一句话是由词语构成的序列，多句话进一步构成了段落。因此，段落可以看作是一个嵌套的双层的序列，这个序列的每个元素又是一个序列。</p>
<p>双层序列是PaddlePaddle支持的一种非常灵活的数据组织方式，帮助我们更好地描述段落、多轮对话等更为复杂的语言数据。基于双层序列输入，我们可以设计搭建一个灵活的、层次化的RNN，分别从词语和句子级别编码输入数据，同时也能够引入更加复杂的记忆机制，更好地完成一些复杂的语言理解任务。</p>
<p>在PaddlePaddle中，<code class="docutils literal"><span class="pre">recurrent_group</span></code>是一种任意复杂的RNN单元，用户只需定义RNN在一个时间步内完成的计算，PaddlePaddle负责完成信息和误差在时间序列上的传播。</p>
<p>更进一步，<code class="docutils literal"><span class="pre">recurrent_group</span></code>同样可以扩展到双层序列的处理上。通过两个嵌套的<code class="docutils literal"><span class="pre">recurrent_group</span></code>分别定义子句级别和词语级别上需要完成的运算，最终实现一个层次化的复杂RNN。</p>
<p>目前，在PaddlePaddle中，能够对双向序列进行处理的有<code class="docutils literal"><span class="pre">recurrent_group</span></code>和部分Layer，具体可参考文档：<a href = "hierarchical-layer.html">支持双层序列作为输入的Layer</a>。</p>
</div>
<div class="section" id="">
<span id="id2"></span><h2>相关概念<a class="headerlink" href="#" title="Permalink to this headline"></a></h2>
<div class="section" id="">
<span id="id3"></span><h3>基本原理<a class="headerlink" href="#" title="Permalink to this headline"></a></h3>
<p><code class="docutils literal"><span class="pre">recurrent_group</span></code> 是PaddlePaddle支持的一种任意复杂的RNN单元。使用者只需要关注于设计RNN在一个时间步之内完成的计算，PaddlePaddle负责完成信息和梯度在时间序列上的传播。</p>
<p>PaddlePaddle中，<code class="docutils literal"><span class="pre">recurrent_group</span></code>的一个简单调用如下：</p>
<div class="highlight-python"><div class="highlight"><pre><span></span><span class="n">recurrent_group</span><span class="p">(</span><span class="n">step</span><span class="p">,</span> <span class="nb">input</span><span class="p">,</span> <span class="n">reverse</span><span class="p">)</span>
</pre></div>
</div>
<ul class="simple">
<li>step：一个可调用的函数，定义一个时间步之内RNN单元完成的计算</li>
<li>input：输入，必须是一个单层序列，或者一个双层序列</li>
<li>reverse：是否以逆序处理输入序列</li>
</ul>
<p>使用<code class="docutils literal"><span class="pre">recurrent_group</span></code>的核心是设计step函数的计算逻辑。step函数内部可以自由组合PaddlePaddle支持的各种layer，完成任意的运算逻辑。<code class="docutils literal"><span class="pre">recurrent_group</span></code> 的输入（即input）会成为step函数的输入，由于step 函数只关注于RNN一个时间步之内的计算，在这里<code class="docutils literal"><span class="pre">recurrent_group</span></code>替我们完成了原始输入数据的拆分。</p>
</div>
<div class="section" id="">
<span id="id4"></span><h3>输入<a class="headerlink" href="#" title="Permalink to this headline"></a></h3>
<p><code class="docutils literal"><span class="pre">recurrent_group</span></code>处理的输入序列主要分为以下三种类型：</p>
<ul class="simple">
<li><strong>数据输入</strong>：一个双层序列进入<code class="docutils literal"><span class="pre">recurrent_group</span></code>会被拆解为一个单层序列，一个单层序列进入<code class="docutils literal"><span class="pre">recurrent_group</span></code>会被拆解为非序列，然后交给step函数，这一过程对用户是完全透明的。可以有以下两种：1）通过data_layer拿到的用户输入；2）其它layer的输出。</li>
<li><strong>只读Memory输入</strong>：<code class="docutils literal"><span class="pre">StaticInput</span></code> 定义了一个只读的Memory，由<code class="docutils literal"><span class="pre">StaticInput</span></code>指定的输入不会被<code class="docutils literal"><span class="pre">recurrent_group</span></code>拆解，<code class="docutils literal"><span class="pre">recurrent_group</span></code> 循环展开的每个时间步总是能够引用所有输入，可以是一个非序列，或者一个单层序列。</li>
<li><strong>序列生成任务的输入</strong>：<code class="docutils literal"><span class="pre">GeneratedInput</span></code>只用于在序列生成任务中指定输入数据。</li>
</ul>
</div>
<div class="section" id="">
<span id="id5"></span><h3>输入示例<a class="headerlink" href="#" title="Permalink to this headline"></a></h3>
<p>序列生成任务大多遵循encoder-decoer架构，encoder和decoder可以是能够处理序列的任意神经网络单元，而RNN是最流行的选择。</p>
<p>给定encoder输出和当前词，decoder每次预测产生下一个最可能的词语。在这种结构中，decoder接受两个输入：</p>
<ul class="simple">
<li>要生成的目标序列：是decoder的数据输入，也是decoder循环展开的依据，<code class="docutils literal"><span class="pre">recurrent_group</span></code>会对这类输入进行拆解。</li>
<li>encoder输出，可以是一个非序列，或者一个单层序列：是一个unbounded memory，decoder循环展开的每一个时间步会引用全部结果，不应该被拆解，这种类型的输入必须通过<code class="docutils literal"><span class="pre">StaticInput</span></code>指定。关于Unbounded Memory的更多讨论请参考论文 <a class="reference external" href="https://arxiv.org/abs/1410.5401">Neural Turning Machine</a>。</li>
</ul>
<p>在序列生成任务中，decoder RNN总是引用上一时刻预测出的词的词向量，作为当前时刻输入。<code class="docutils literal"><span class="pre">GeneratedInput</span></code>自动完成这一过程。</p>
</div>
<div class="section" id="">
<span id="id6"></span><h3>输出<a class="headerlink" href="#" title="Permalink to this headline"></a></h3>
<p><code class="docutils literal"><span class="pre">step</span></code>函数必须返回一个或多个Layer的输出，这个Layer的输出会作为整个<code class="docutils literal"><span class="pre">recurrent_group</span></code> 最终的输出结果。在输出的过程中，<code class="docutils literal"><span class="pre">recurrent_group</span></code> 会将每个时间步的输出拼接，这个过程对用户也是透明的。</p>
</div>
<div class="section" id="memory">
<span id="memory"></span><h3>memory<a class="headerlink" href="#memory" title="Permalink to this headline"></a></h3>
<p>memory只能在<code class="docutils literal"><span class="pre">recurrent_group</span></code>中定义和使用。memory不能独立存在，必须指向一个PaddlePaddle定义的Layer。引用memory得到这layer上一时刻输出，因此，可以将memory理解为一个时延操作。</p>
<p>可以显示地指定一个layer的输出用于初始化memory。不指定时，memory默认初始化为0。</p>
</div>
</div>
<div class="section" id="rnn">
<span id="rnn"></span><h2>双层RNN介绍<a class="headerlink" href="#rnn" title="Permalink to this headline"></a></h2>
<p><code class="docutils literal"><span class="pre">recurrent_group</span></code>帮助我们完成对输入序列的拆分，对输出的合并，以及计算逻辑在序列上的循环展开。</p>
<p>利用这种特性，两个嵌套的<code class="docutils literal"><span class="pre">recurrent_group</span></code>能够处理双层序列，实现词语和句子两个级别的双层RNN结构。</p>
<ul class="simple">
<li>单层（word-level）RNN：每个状态（state）对应一个词（word）。</li>
<li>双层（sequence-level）RNN：一个双层RNN由多个单层RNN组成，每个单层RNN（即双层RNN的每个状态）对应一个子句（subseq）。</li>
</ul>
<p>为了描述方便，下文以NLP任务为例，将含有子句（subseq）的段落定义为一个双层序列，将含有词语的句子定义为一个单层序列，那么0层序列即为一个词语。</p>
</div>
<div class="section" id="rnn">
<span id="id7"></span><h2>双层RNN的使用<a class="headerlink" href="#rnn" title="Permalink to this headline"></a></h2>
<div class="section" id="">
<span id="id8"></span><h3>训练流程的使用方法<a class="headerlink" href="#" title="Permalink to this headline"></a></h3>
<p>使用 <code class="docutils literal"><span class="pre">recurrent_group</span></code>需要遵循以下约定：</p>
<ul class="simple">
<li><strong>单进单出</strong>：输入和输出都是单层序列。<ul>
<li>如果有多个输入，不同输入序列含有的词语数必须严格相等。</li>
<li>输出一个单层序列，输出序列的词语数和输入序列一致。</li>
<li>memory：在step函数中定义 memory指向一个layer，通过引用memory得到这个layer上一个时刻输出，形成recurrent 连接。memory的is_seq参数必须为false。如果没有定义memory，每个时间步之内的运算是独立的。</li>
<li>boot_layer：memory的初始状态，默认初始状为0，memory的is_seq参数必须为false。</li>
</ul>
</li>
<li><strong>双进双出</strong>：输入和输出都是双层序列。<ul>
<li>如果有多个输入序列，不同输入含有的子句（subseq）数必须严格相等，但子句含有的词语数可以不相等。</li>
<li>输出一个双层序列，子句（subseq）数、子句的单词数和指定的一个输入序列一致，默认为第一个输入。</li>
<li>memory：在step函数中定义memory，指向一个layer，通过引用memory得到这个layer上一个时刻的输出，形成recurrent连接。定义在外层<code class="docutils literal"><span class="pre">recurrent_group</span></code> step函数中的memory，能够记录上一个subseq 的状态，可以是一个单层序列（只作为read-only memory），也可以是一个词语。如果没有定义memory，那么 subseq 之间的运算是独立的。</li>
<li>boot_layer：memory 初始状态，可以是一个单层序列（只作为read-only memory）或一个向量。默认不设置，即初始状态为0。</li>
</ul>
</li>
<li><strong>双进单出</strong>：目前还未支持，会报错&#8221;In hierachical RNN, all out links should be from sequences now&#8221;。</li>
</ul>
</div>
<div class="section" id="">
<span id="id9"></span><h3>生成流程的使用方法<a class="headerlink" href="#" title="Permalink to this headline"></a></h3>
<p>使用<code class="docutils literal"><span class="pre">beam_search</span></code>需要遵循以下约定：</p>
<ul class="simple">
<li>单层RNN：从一个word生成下一个word。</li>
<li>双层RNN：即把单层RNN生成后的subseq给拼接成一个新的双层seq。从语义上看，也不存在一个subseq直接生成下一个subseq的情况。</li>
</ul>
</div>
</div>
</div>
</div>
</div>
</div>
<div class="sphinxsidebar" role="navigation" aria-label="main navigation">
<div class="sphinxsidebarwrapper">
<h3><a href="../../index.html">Table Of Contents</a></h3>
<ul>
<li><a class="reference internal" href="#">Recurrent Group教程</a><ul>
<li><a class="reference internal" href="#">概述</a></li>
<li><a class="reference internal" href="#">相关概念</a><ul>
<li><a class="reference internal" href="#">基本原理</a></li>
<li><a class="reference internal" href="#">输入</a></li>
<li><a class="reference internal" href="#">输入示例</a></li>
<li><a class="reference internal" href="#">输出</a></li>
<li><a class="reference internal" href="#memory">memory</a></li>
</ul>
</li>
<li><a class="reference internal" href="#rnn">双层RNN介绍</a></li>
<li><a class="reference internal" href="#rnn">双层RNN的使用</a><ul>
<li><a class="reference internal" href="#">训练流程的使用方法</a></li>
<li><a class="reference internal" href="#">生成流程的使用方法</a></li>
</ul>
</li>
</ul>
</li>
</ul>
<div role="note" aria-label="source link">
<h3>This Page</h3>
<ul class="this-page-menu">
<li><a href="../../_sources/algorithm/rnn/rnn-tutorial.txt"
rel="nofollow">Show Source</a></li>
</ul>
</div>
<div id="searchbox" style="display: none" role="search">
<h3>Quick search</h3>
<form class="search" action="../../search.html" method="get">
<div><input type="text" name="q" /></div>
<div><input type="submit" value="Go" /></div>
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
</form>
</div>
<script type="text/javascript">$('#searchbox').show(0);</script>
</div>
</div>
<div class="clearer"></div>
</div>
<div class="related" role="navigation" aria-label="related navigation">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="../../genindex.html" title="General Index"
>index</a></li>
<li class="nav-item nav-item-0"><a href="../../index.html">PaddlePaddle documentation</a> &#187;</li>
</ul>
</div>
<div class="footer" role="contentinfo">
&#169; Copyright 2016, PaddlePaddle developers.
Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.4.8.
</div>
</body>
</html>
\ No newline at end of file
doc_cn/build_and_install/index.html
浏览文件 @ 57e43320
......@@ -58,28 +58,16 @@ var _hmt = _hmt || [];
<div class="section" id="id1">
<h1>编译与安装<a class="headerlink" href="#id1" title="Permalink to this headline"></a></h1>
<div class="section" id="id2">
<h2>安装<a class="headerlink" href="#id2" title="Permalink to this headline"></a></h2>
<p>PaddlePaddle提供数个预编译的二进制来进行安装，包括Docker镜像，ubuntu的deb安装包等。我们推荐使用Docker镜像来部署环境，同时欢迎贡献更多的安装包。</p>
<p>Note: The intallation packages are still in pre-release state and your experience of installation may not be smooth.</p>
<p>注意：目前PaddlePaddle的安装包还处在pre-release的状态，使用起来或许会不是很顺畅。</p>
<div class="toctree-wrapper compound">
<ul>
<li class="toctree-l1"><a class="reference internal" href="install/docker_install.html">安装PaddlePaddle的Docker镜像</a></li>
<li class="toctree-l1"><a class="reference internal" href="install/ubuntu_install.html">使用deb包在Ubuntu上安装PaddlePaddle</a></li>
</ul>
</div>
</div>
<div class="section" id="id3">
<h2>编译<a class="headerlink" href="#id3" title="Permalink to this headline"></a></h2>
<div class="admonition warning">
<p class="first admonition-title">Warning</p>
<p class="last">编译选项主要推荐高级用户查看，普通用户请走安装流程。</p>
</div>
<div class="toctree-wrapper compound">
<ul>
<li class="toctree-l1"><a class="reference internal" href="cmake/index.html">使用cmake编译PaddlePaddle</a></li>
</ul>
</div>
</div>
</div>
......@@ -88,15 +76,6 @@ var _hmt = _hmt || [];
</div>
<div class="sphinxsidebar" role="navigation" aria-label="main navigation">
<div class="sphinxsidebarwrapper">
<h3><a href="../index.html">Table Of Contents</a></h3>
<ul>
<li><a class="reference internal" href="#">编译与安装</a><ul>
<li><a class="reference internal" href="#id2">安装</a></li>
<li><a class="reference internal" href="#id3">编译</a></li>
</ul>
</li>
</ul>
<div role="note" aria-label="source link">
<h3>This Page</h3>
<ul class="this-page-menu">
......
doc_cn/build_and_install/install/docker_install.html
浏览文件 @ 57e43320
......@@ -69,64 +69,19 @@ var _hmt = _hmt || [];
</ul>
<div class="section" id="id1">
<h2>PaddlePaddle提供的Docker镜像版本<a class="headerlink" href="#id1" title="Permalink to this headline"></a></h2>
<p>我们提供了12个 <a class="reference external" href="https://hub.docker.com/r/paddledev/paddle/tags/">Docker image</a> ，他们的image name都是 <code class="code docutils literal"><span class="pre">paddle-dev/paddle</span></code> ，tag分别为</p>
<table border="1" class="docutils">
<colgroup>
<col width="21%" />
<col width="22%" />
<col width="29%" />
<col width="28%" />
</colgroup>
<thead valign="bottom">
<tr class="row-odd"><th class="head">&nbsp;</th>
<th class="head">normal</th>
<th class="head">devel</th>
<th class="head">demo</th>
</tr>
</thead>
<tbody valign="top">
<tr class="row-even"><td>CPU</td>
<td>cpu-latest</td>
<td>cpu-devel-latest</td>
<td>cpu-demo-latest</td>
</tr>
<tr class="row-odd"><td>GPU</td>
<td>gpu-latest</td>
<td>gpu-devel-latest</td>
<td>gpu-demo-latest</td>
</tr>
<tr class="row-even"><td>CPU WITHOUT AVX</td>
<td>cpu-noavx-latest</td>
<td>cpu-noavx-devel-latest</td>
<td>cpu-noavx-demo-latest</td>
</tr>
<tr class="row-odd"><td>GPU WITHOUT AVX</td>
<td>gpu-noavx-latest</td>
<td>gpu-noavx-devel-latest</td>
<td>gpu-noavx-demo-latest</td>
</tr>
</tbody>
</table>
<p>其中，横向包括三个版本，normal，devel和demo。</p>
<p>我们提供了6个Docker image:</p>
<ul class="simple">
<li>Normal: 正常的Docker image，只包括paddle的二进制</li>
<li>Devel: 包括Paddle的二进制、编译环境和源代码</li>
<li>Demo: 包括Paddle运行demo所需要的依赖</li>
<li>paddledev/paddle:cpu-latest: PaddlePaddle的CPU二进制</li>
<li>paddledev/paddle:gpu-latest： PaddlePaddle的GPU二进制</li>
<li>paddledev/paddle:cpu-devel-latest: PaddlePaddle的CPU二进制,同时包含CPU开发环境和源码</li>
<li>paddledev/paddle:gpu-devel-latest: PaddlePaddle的GPU二进制,同时包含GPU开发环境和源码</li>
<li>paddledev/paddle:cpu-demo-latest: PaddlePaddle的CPU二进制,同时包含CPU开发环境、源码和运行demo的必要依赖</li>
<li>paddledev/paddle:gpu-demo-latest: PaddlePaddle的GPU二进制,同时包含GPU开发环境、源码和运行demo的必要依赖</li>
</ul>
<p>纵向包括四个版本，他们是。</p>
<ul class="simple">
<li>CPU: CPU版本。需要支持AVX指令集的CPU</li>
<li>GPU: GPU版本。需要支持AVX指令集的CPU</li>
<li>CPU WITHOUT AVX: CPU版本，不支持AVX指令集的CPU也可以运行</li>
<li>GPU WITHOUT AVX: GPU版本，不需要AVX指令集的CPU也可以运行。</li>
</ul>
<p>用户可以选择对应版本的docker image。使用如下脚本可以确定本机的CPU知否支持 <code class="code docutils literal"><span class="pre">AVX</span></code> 指令集:</p>
<div class="highlight-bash"><div class="highlight"><pre><span></span><span class="k">if</span> cat /proc/cpuinfo <span class="p">|</span> grep -q avx <span class="p">;</span> <span class="k">then</span> <span class="nb">echo</span> <span class="s2">&quot;Support AVX&quot;</span><span class="p">;</span> <span class="k">else</span> <span class="nb">echo</span> <span class="s2">&quot;Not support AVX&quot;</span><span class="p">;</span> <span class="k">fi</span>
</pre></div>
</div>
<p>如果输出 <code class="code docutils literal"><span class="pre">Support</span> <span class="pre">AVX</span></code>，则可以选择上表中的AVX版本PaddlePaddle。否则需要选择非AVX的PaddlePaddle。选择普通CPU版本的devel版本的image，则可以使用 <code class="code docutils literal"><span class="pre">paddle-dev/paddle:cpu-devel-latest</span></code> 来引用这个image。</p>
<p>同时，不同的稳定版本，会将latest替换成稳定版本的版本号。</p>
<p>PaddlePaddle提供的镜像并不包含任何命令运行，想要运行PaddlePaddle，您需要进入镜像运行PaddlePaddle
程序或者自定义一个含有启动脚本的image。具体请参考注意事项中的 <code class="code docutils literal"><span class="pre">使用ssh访问PaddlePaddle镜像</span></code></p>
程序或者自定义一个含有启动脚本的image。具体请参考注意事项中的
<cite>使用ssh访问PaddlePaddle镜像</cite></p>
</div>
<div class="section" id="docker">
<h2>下载和运行Docker镜像<a class="headerlink" href="#docker" title="Permalink to this headline"></a></h2>
......@@ -138,14 +93,14 @@ mac osx或者是windows机器，请参考
<a class="reference external" href="https://docs.docker.com/engine/installation/windows/">windows 的安装文档</a> 。</p>
<p>您可以使用 <code class="code docutils literal"><span class="pre">docker</span> <span class="pre">pull</span></code> 命令预先下载镜像，也可以直接执行
<code class="code docutils literal"><span class="pre">docker</span> <span class="pre">run</span></code> 命令运行镜像。执行方法如下:</p>
<div class="highlight-bash"><div class="highlight"><pre><span></span>$ docker run -it paddledev/paddlepaddle:cpu-latest
<div class="highlight-bash"><div class="highlight"><pre><span></span>$ docker run -it paddledev/paddlepaddle:latest-cpu
</pre></div>
</div>
<p>即可启动和进入PaddlePaddle的container。如果运行GPU版本的PaddlePaddle，则需要先将
cuda相关的Driver和设备映射进container中，脚本类似于</p>
<div class="highlight-bash"><div class="highlight"><pre><span></span>$ <span class="nb">export</span> <span class="nv">CUDA_SO</span><span class="o">=</span><span class="s2">&quot;</span><span class="k">$(</span><span class="se">\l</span>s /usr/lib64/libcuda* <span class="p">|</span> xargs -I<span class="o">{}</span> <span class="nb">echo</span> <span class="s1">&#39;-v {}:{}&#39;</span><span class="k">)</span><span class="s2"> </span><span class="k">$(</span><span class="se">\l</span>s /usr/lib64/libnvidia* <span class="p">|</span> xargs -I<span class="o">{}</span> <span class="nb">echo</span> <span class="s1">&#39;-v {}:{}&#39;</span><span class="k">)</span><span class="s2">&quot;</span>
$ <span class="nb">export</span> <span class="nv">DEVICES</span><span class="o">=</span><span class="k">$(</span><span class="se">\l</span>s /dev/nvidia* <span class="p">|</span> xargs -I<span class="o">{}</span> <span class="nb">echo</span> <span class="s1">&#39;--device {}:{}&#39;</span><span class="k">)</span>
$ docker run <span class="si">${</span><span class="nv">CUDA_SO</span><span class="si">}</span> <span class="si">${</span><span class="nv">DEVICES</span><span class="si">}</span> -it paddledev/paddlepaddle:latest-gpu
$ docker run -it paddledev/paddlepaddle:latest-gpu
</pre></div>
</div>
<p>进入Docker container后，运行 <code class="code docutils literal"><span class="pre">paddle</span> <span class="pre">version</span></code> 即可打印出PaddlePaddle的版本和构建
......
doc_cn/build_and_install/install/ubuntu_install.html
浏览文件 @ 57e43320
......@@ -58,38 +58,25 @@ var _hmt = _hmt || [];
<div class="section" id="debubuntupaddlepaddle">
<h1>使用deb包在Ubuntu上安装PaddlePaddle<a class="headerlink" href="#debubuntupaddlepaddle" title="Permalink to this headline"></a></h1>
<p>PaddlePaddle目前支持使用deb包安装。Paddle的 <code class="code docutils literal"><span class="pre">deb</span></code> 安装包在ubuntu 14.04中正确，但理论上支持其他的 debian 发行版。</p>
<p>PaddlePaddle的ubuntu安装包分为四个版本，他们是 cpu、gpu、cpu-noavx、gpu-noavx 四个版本。其中 noavx 用于不支持AVX指令集的cpu。安装包的下载地址是: <a class="reference external" href="https://github.com/baidu/Paddle/releases/">https://github.com/baidu/Paddle/releases/</a></p>
<p>用户需要先将PaddlePaddle安装包下载到本地，然后执行如下 <code class="code docutils literal"><span class="pre">gdebi</span></code> 命令即可完成安装。</p>
<div class="highlight-shell"><div class="highlight"><pre><span></span>gdebi paddle-*-cpu.deb
</pre></div>
</div>
<p>如果 <code class="code docutils literal"><span class="pre">gdebi</span></code> 没有安装,则需要使用 <code class="code docutils literal"><span class="pre">sudo</span> <span class="pre">apt-get</span> <span class="pre">install</span> <span class="pre">gdebi</span></code>, 来安装 <code class="code docutils literal"><span class="pre">gdebi</span></code> 。</p>
<p>或者使用下面一条命令安装.</p>
<div class="highlight-shell"><div class="highlight"><pre><span></span>dpkg -i paddle-*-cpu.deb
<p>PaddlePaddle目前支持ubuntu 14.04版本使用deb包安装。更多的安装包PaddlePaddle会在近期提供。
欢迎大家贡献各个发行版的安装包(例如，ubuntu，centos，debian，gentoo)。</p>
<p>PaddlePaddle的ubuntu安装包分为两个版本，即CPU版本，和GPU版本，他们的下载地址是:
<a class="reference external" href="https://github.com/baidu/Paddle/releases/tag/V0.8.0b0">https://github.com/baidu/Paddle/releases/tag/V0.8.0b0</a></p>
<p>需要注意的是，目前PaddlePaddle的安装包只支持
<a class="reference external" href="https://en.wikipedia.org/wiki/Advanced_Vector_Extensions">AVX</a>
指令集的X86 CPU。如果系统使用不支持 <a class="reference external" href="https://en.wikipedia.org/wiki/Advanced_Vector_Extensions">AVX</a> 指令集的CPU运行PaddlePaddle，那么需要从源码
编译PaddlePaddle，请参考 <a class="reference external" href="../cmake/index.html">编译文档</a> 。</p>
<p>用户需要先将PaddlePaddle安装包下载到本地，然后执行如下命令即可完成安装。</p>
<div class="highlight-shell"><div class="highlight"><pre><span></span>dpkg -i paddle-0.8.0b-cpu.deb
apt-get install -f
</pre></div>
</div>
<p>在 <code class="code docutils literal"><span class="pre">dpkg</span> <span class="pre">-i</span></code> 的时候如果报一些依赖未找到的错误是正常的，
在 <code class="code docutils literal"><span class="pre">apt-get</span> <span class="pre">install</span> <span class="pre">-f</span></code> 里会继续安装 PaddlePaddle。</p>
<p>需要注意的是，如果使用GPU版本的PaddlePaddle，请安装CUDA 7.5 和CUDNN 5到本地环境中，
在 <code class="code docutils literal"><span class="pre">apt-get</span> <span class="pre">install</span> <span class="pre">-f</span></code> 里会继续安装 PaddlePaddle。
需要注意的是，如果使用GPU版本的PaddlePaddle，请安装CUDA 7.5 和CUDNN 5到本地环境中，
并设置好对应的环境变量(LD_LIBRARY_PATH等等)。</p>
<p>安装完成后,可以使用命令 <code class="code docutils literal"><span class="pre">paddle</span> <span class="pre">version</span></code> 查看安装后的paddle 版本。可能的输出为</p>
<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">PaddlePaddle</span> <span class="mf">0.8</span><span class="o">.</span><span class="mb">0b1</span><span class="p">,</span> <span class="n">compiled</span> <span class="k">with</span>
<span class="n">with_avx</span><span class="p">:</span> <span class="n">ON</span>
<span class="n">with_gpu</span><span class="p">:</span> <span class="n">OFF</span>
<span class="n">with_double</span><span class="p">:</span> <span class="n">OFF</span>
<span class="n">with_python</span><span class="p">:</span> <span class="n">ON</span>
<span class="n">with_rdma</span><span class="p">:</span> <span class="n">OFF</span>
<span class="n">with_glog</span><span class="p">:</span> <span class="n">ON</span>
<span class="n">with_gflags</span><span class="p">:</span> <span class="n">ON</span>
<span class="n">with_metric_learning</span><span class="p">:</span>
<span class="n">with_timer</span><span class="p">:</span> <span class="n">OFF</span>
<span class="n">with_predict_sdk</span><span class="p">:</span>
</pre></div>
</div>
<div class="section" id="id1">
<h2>可能遇到的问题<a class="headerlink" href="#id1" title="Permalink to this headline"></a></h2>
<div class="section" id="id2">
<h2>可能遇到的问题<a class="headerlink" href="#id2" title="Permalink to this headline"></a></h2>
<div class="section" id="libcudart-so-libcudnn-so">
<h3>libcudart.so/libcudnn.so找不到<a class="headerlink" href="#libcudart-so-libcudnn-so" title="Permalink to this headline"></a></h3>
<p>安装完成PaddlePaddle后，运行 <code class="code docutils literal"><span class="pre">paddle</span> <span class="pre">train</span></code> 报错:</p>
......@@ -136,7 +123,7 @@ driver添加到LD_LIBRARY_PATH中。比较可能的命令如下。</p>
<h3><a href="../../index.html">Table Of Contents</a></h3>
<ul>
<li><a class="reference internal" href="#">使用deb包在Ubuntu上安装PaddlePaddle</a><ul>
<li><a class="reference internal" href="#id1">可能遇到的问题</a><ul>
<li><a class="reference internal" href="#id2">可能遇到的问题</a><ul>
<li><a class="reference internal" href="#libcudart-so-libcudnn-so">libcudart.so/libcudnn.so找不到</a></li>
<li><a class="reference internal" href="#cuda-driver">CUDA Driver找不到</a></li>
<li><a class="reference internal" href="#config">config文件找不到</a></li>
......
doc_cn/faq/index.html 已删除 100644 → 0
浏览文件 @ 438f7337
此差异已折叠。
doc_cn/index.html
浏览文件 @ 57e43320
......@@ -78,16 +78,7 @@ var _hmt = _hmt || [];
<div class="section" id="id9">
<h2>算法教程<a class="headerlink" href="#id9" title="Permalink to this headline"></a></h2>
<ul class="simple">
<li><a class="reference external" href="algorithm/rnn/rnn-tutorial.html">Recurrent Group教程</a></li>
<li><a class="reference external" href="../doc/algorithm/rnn/rnn.html">单层RNN示例</a></li>
<li><a class="reference external" href="algorithm/rnn/hierarchical-rnn.html">双层RNN示例</a></li>
<li><a class="reference external" href="algorithm/rnn/hierarchical-layer.html">支持双层序列作为输入的Layer</a></li>
</ul>
</div>
<div class="section" id="id12">
<h2>常见问题<a class="headerlink" href="#id12" title="Permalink to this headline"></a></h2>
<ul class="simple">
<li><a class="reference external" href="faq/index.html">常见问题</a></li>
<li><a class="reference external" href="../doc/algorithm/rnn/rnn.html">RNN配置</a></li>
</ul>
</div>
</div>
......@@ -104,7 +95,6 @@ var _hmt = _hmt || [];
<li><a class="reference internal" href="#id1">使用指南</a></li>
<li><a class="reference internal" href="#id8">开发指南</a></li>
<li><a class="reference internal" href="#id9">算法教程</a></li>
<li><a class="reference internal" href="#id12">常见问题</a></li>
</ul>
</li>
</ul>
......
doc_cn/objects.inv
浏览文件 @ 57e43320
无法预览此类型文件
doc_cn/searchindex.js
浏览文件 @ 57e43320
此差异已折叠。
doc_cn/ui/data_provider/pydataprovider2.html
浏览文件 @ 57e43320
......@@ -189,10 +189,10 @@ process函数调用多次 <code class="code docutils literal"><span class="pre">
<span class="c1"># Define a py data provider</span>
<span class="nd">@provider</span><span class="p">(</span><span class="n">input_types</span><span class="o">=</span><span class="p">{</span>
<span class="s1">&#39;pixel&#39;</span><span class="p">:</span> <span class="n">dense_vector</span><span class="p">(</span><span class="mi">28</span> <span class="o">*</span> <span class="mi">28</span><span class="p">),</span>
<span class="s1">&#39;label&#39;</span><span class="p">:</span> <span class="n">integer_value</span><span class="p">(</span><span class="mi">10</span><span class="p">)</span>
<span class="p">})</span>
<span class="nd">@provider</span><span class="p">(</span><span class="n">input_types</span><span class="o">=</span><span class="p">[</span>
<span class="n">dense_vector</span><span class="p">(</span><span class="mi">28</span> <span class="o">*</span> <span class="mi">28</span><span class="p">),</span>
<span class="n">integer_value</span><span class="p">(</span><span class="mi">10</span><span class="p">)</span>
<span class="p">])</span>
<span class="k">def</span> <span class="nf">process</span><span class="p">(</span><span class="n">settings</span><span class="p">,</span> <span class="n">filename</span><span class="p">):</span> <span class="c1"># settings is not used currently.</span>
<span class="n">f</span> <span class="o">=</span> <span class="nb">open</span><span class="p">(</span><span class="n">filename</span><span class="p">,</span> <span class="s1">&#39;r&#39;</span><span class="p">)</span> <span class="c1"># open one of training file</span>
......@@ -207,7 +207,7 @@ process函数调用多次 <code class="code docutils literal"><span class="pre">
<span class="n">pixels_float</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="nb">float</span><span class="p">(</span><span class="n">each_pixel_str</span><span class="p">))</span>
<span class="c1"># give data to paddle.</span>
<span class="k">yield</span> <span class="p">{</span><span class="s2">&quot;pixel&quot;</span><span class="p">:</span> <span class="n">pixels_float</span><span class="p">,</span> <span class="s1">&#39;label&#39;</span><span class="p">:</span> <span class="nb">int</span><span class="p">(</span><span class="n">label</span><span class="p">)}</span>
<span class="k">yield</span> <span class="p">{</span> <span class="s2">&quot;pixel&quot;</span><span class="p">:</span> <span class="n">pixels_float</span><span class="p">,</span> <span class="s1">&#39;label&#39;</span><span class="p">:</span> <span class="nb">int</span><span class="p">(</span><span class="n">label</span><span class="p">)</span> <span class="p">}</span>
<span class="n">f</span><span class="o">.</span><span class="n">close</span><span class="p">()</span> <span class="c1"># close file</span>
</pre></div>
......@@ -340,6 +340,8 @@ DataProvider创建的时候执行。这个初始化函数具有如下参数:</p>
是一个batch size，但是有时为了计算均衡性，可以将一条数据设置成多个batch size</li>
<li>cache 是数据缓存的策略，参考 <a class="reference internal" href="#cache">cache</a></li>
<li>init_hook 是初始化时调用的函数，参考 <a class="reference internal" href="#init-hook">init_hook</a></li>
<li>use_dynamic_order 如果是true的话，可以返回一个dict，key是data_layer的名字，value是特征值。同时，也可以
返回一个list或者tuple。如果是false的话，只能够返回list或者tuple</li>
<li>check 设置成true的话，会根据input_types检查数据的合法性。</li>
<li>check_fail_continue 如果设置成true的话，即使在check中数据不合法，也会扔到这条数据，继续训练。 如果
check是false的话，没有作用。</li>
......
Markdown is supported
0% 或 .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑！
想要评论请 注册 或