@@ -8,200 +8,256 @@ Please be aware that you will need to change `Dockers settings
...
@@ -8,200 +8,256 @@ Please be aware that you will need to change `Dockers settings
<https://github.com/PaddlePaddle/Paddle/issues/627>`_ to make full use
<https://github.com/PaddlePaddle/Paddle/issues/627>`_ to make full use
of your hardware resource on Mac OS X and Windows.
of your hardware resource on Mac OS X and Windows.
Working With Docker
-------------------
Usage of CPU-only and GPU Images
Docker is simple as long as we understand a few basic concepts:
----------------------------------
For each version of PaddlePaddle, we release 2 types of Docker images: development
- *image*: A Docker image is a pack of software. It could contain one or more programs and all their dependencies. For example, the PaddlePaddle's Docker image includes pre-built PaddlePaddle and Python and many Python packages. We can run a Docker image directly, other than installing all these software. We can type
image and production image. Production image includes CPU-only version and a CUDA
GPU version and their no-AVX versions. We put the docker images on
`dockerhub.com <https://hub.docker.com/r/paddledev/paddle/>`_. You can find the
latest versions under "tags" tab at dockerhub.com.
1. development image :code:`paddlepaddle/paddle:<version>-dev`
This image has packed related develop tools and runtime environment. Users and
developers can use this image instead of their own local computer to accomplish
development, build, releasing, document writing etc. While different version of
paddle may depends on different version of libraries and tools, if you want to
setup a local environment, you must pay attention to the versions.
The development image contains:
- gcc/clang
- nvcc
- Python
- sphinx
- woboq
- sshd
Many developers use servers with GPUs, they can use ssh to login to the server
and run :code:`docker exec` to enter the docker container and start their work.
Also they can start a development docker image with SSHD service, so they can login to
the container and start work.
To run the CPU-only image as an interactive container:
.. code-block:: bash
docker images
to list all images in the system. We can also run
.. code-block:: bash
.. code-block:: bash
docker run -it --rm paddledev/paddle:<version> /bin/bash
docker pull paddlepaddle/paddle:0.10.0rc2
or, we can run it as a daemon container
to download a Docker image, paddlepaddle/paddle in this example,
from Dockerhub.com.
- *container*: considering a Docker image a program, a container is a
"process" that runs the image. Indeed, a container is exactly an
operating system process, but with a virtualized filesystem, network
port space, and other virtualized environment. We can type
.. code-block:: bash
.. code-block:: bash
docker run -d -p 2202:22 -p 8888:8888 paddledev/paddle:<version>
docker run paddlepaddle/paddle:0.10.0rc2
and SSH to this container using password :code:`root`:
to start a container to run a Docker image, paddlepaddle/paddle in this example.
- By default docker container have an isolated file system namespace,
we can not see the files in the host file system. By using *volume*,
mounted files in host will be visible inside docker container.
Following command will mount current dirctory into /data inside
docker container, run docker container from debian image with
command :code:`ls /data`.
.. code-block:: bash
.. code-block:: bash
ssh -p 2202 root@localhost
docker run --rm -v $(pwd):/data debian ls /data
An advantage of using SSH is that we can connect to PaddlePaddle from
Usage of CPU-only and GPU Images
more than one terminals. For example, one terminal running vi and
----------------------------------
another one running Python interpreter. Another advantage is that we
can run the PaddlePaddle container on a remote server and SSH to it
from a laptop.
For each version of PaddlePaddle, we release two types of Docker images:
development image and production image. Production image includes
CPU-only version and a CUDA GPU version and their no-AVX versions. We
put the docker images on `dockerhub.com
<https://hub.docker.com/r/paddledev/paddle/>`_. You can find the
latest versions under "tags" tab at dockerhub.com
1. Production images, this image might have multiple variants:
2. Production images, this image might have multiple variants:
docker run ${CUDA_SO} ${DEVICES} -it paddledev/paddle:<version>-gpu
2. development image :code:`paddlepaddle/paddle:<version>-dev`
3. Use production image to release you AI application
This image has packed related develop tools and runtime
Suppose that we have a simple application program in :code:`a.py`, we can test and run it using the production image:
environment. Users and developers can use this image instead of
their own local computer to accomplish development, build,
releasing, document writing etc. While different version of paddle
may depends on different version of libraries and tools, if you
want to setup a local environment, you must pay attention to the
versions. The development image contains:
```bash
- gcc/clang
docker run -it -v $PWD:/work paddle /work/a.py
- nvcc
```
- Python
- sphinx
- woboq
- sshd
But this works only if all dependencies of :code:`a.py` are in the production image. If this is not the case, we need to build a new Docker image from the production image and with more dependencies installs.
Many developers use servers with GPUs, they can use ssh to login to
the server and run :code:`docker exec` to enter the docker
container and start their work. Also they can start a development
docker image with SSHD service, so they can login to the container
and start work.
PaddlePaddle Book
Train Model Using Python API
------------------
----------------------------
The Jupyter Notebook is an open-source web application that allows
Our official docker image provides a runtime for PaddlePaddle
you to create and share documents that contain live code, equations,
programs. The typical workflow will be as follows:
visualizations and explanatory text in a single browser.
PaddlePaddle Book is an interactive Jupyter Notebook for users and developers.
Create a directory as workspace:
We already exposed port 8888 for this book. If you want to
dig deeper into deep learning, PaddlePaddle Book definitely is your best choice.
We provide a packaged book image, simply issue the command:
.. code-block:: bash
mkdir ~/workspace
Edit a PaddlePaddle python program using your favourite editor
.. code-block:: bash
.. code-block:: bash
docker run -p 8888:8888 paddlepaddle/book
emacs ~/workspace/example.py
Then, you would back and paste the address into the local browser:
Run the program using docker:
.. code-block:: text
.. code-block:: bash
http://localhost:8888/
docker run --rm -v ~/workspace:/workspace paddlepaddle/paddle:0.10.0rc2 python /workspace/example.py
That's all. Enjoy your journey!
Or if you are using GPU for training:
Development Using Docker
.. code-block:: bash
------------------------
Developers can work on PaddlePaddle using Docker. This allows
nvidia-docker run --rm -v ~/workspace:/workspace paddlepaddle/paddle:0.10.0rc2-gpu python /workspace/example.py
developers to work on different platforms -- Linux, Mac OS X, and
Windows -- in a consistent way.
1. Build the Development Docker Image
Above commands will start a docker container by running :code:`python
/workspace/example.py`. It will stop once :code:`python
/workspace/example.py` finishes.
.. code-block:: bash
Another way is to tell docker to start a :code:`/bin/bash` session and
@@ -220,169 +222,204 @@ running PaddlePaddle. This is reasonable as Docker now runs on all
...
@@ -220,169 +222,204 @@ running PaddlePaddle. This is reasonable as Docker now runs on all
major operating systems including Linux, Mac OS X, and Windows.
major operating systems including Linux, Mac OS X, and Windows.
Please be aware that you will need to change <aclass="reference external"href="https://github.com/PaddlePaddle/Paddle/issues/627">Dockers settings</a> to make full use
Please be aware that you will need to change <aclass="reference external"href="https://github.com/PaddlePaddle/Paddle/issues/627">Dockers settings</a> to make full use
of your hardware resource on Mac OS X and Windows.</p>
of your hardware resource on Mac OS X and Windows.</p>
<h2>Usage of CPU-only and GPU Images<aclass="headerlink"href="#usage-of-cpu-only-and-gpu-images"title="Permalink to this headline">¶</a></h2>
<h2>Working With Docker<aclass="headerlink"href="#working-with-docker"title="Permalink to this headline">¶</a></h2>
<p>For each version of PaddlePaddle, we release 2 types of Docker images: development
<p>Docker is simple as long as we understand a few basic concepts:</p>
image and production image. Production image includes CPU-only version and a CUDA
<ul>
GPU version and their no-AVX versions. We put the docker images on
<li><pclass="first"><em>image</em>: A Docker image is a pack of software. It could contain one or more programs and all their dependencies. For example, the PaddlePaddle’s Docker image includes pre-built PaddlePaddle and Python and many Python packages. We can run a Docker image directly, other than installing all these software. We can type</p>
<aclass="reference external"href="https://hub.docker.com/r/paddledev/paddle/">dockerhub.com</a>. You can find the
latest versions under “tags” tab at dockerhub.com.
1. development image <codeclass="code docutils literal"><spanclass="pre">paddlepaddle/paddle:<version>-dev</span></code></p>
<blockquote>
<div><p>This image has packed related develop tools and runtime environment. Users and
developers can use this image instead of their own local computer to accomplish
development, build, releasing, document writing etc. While different version of
paddle may depends on different version of libraries and tools, if you want to
setup a local environment, you must pay attention to the versions.
The development image contains:
- gcc/clang
- nvcc
- Python
- sphinx
- woboq
- sshd
Many developers use servers with GPUs, they can use ssh to login to the server
and run <codeclass="code docutils literal"><spanclass="pre">docker</span><spanclass="pre">exec</span></code> to enter the docker container and start their work.
Also they can start a development docker image with SSHD service, so they can login to
the container and start work.</p>
<p>To run the CPU-only image as an interactive container:</p>
<divclass="highlight-bash"><divclass="highlight"><pre><span></span>docker run -it --rm paddledev/paddle:<version> /bin/bash
</pre></div>
</pre></div>
</div>
</div>
<p>or, we can run it as a daemon container</p>
<p>to list all images in the system. We can also run</p>
<divclass="highlight-bash"><divclass="highlight"><pre><span></span>docker run -d -p <spanclass="m">2202</span>:22 -p <spanclass="m">8888</span>:8888 paddledev/paddle:<version>
<divclass="highlight-bash"><divclass="highlight"><pre><span></span>docker run --rm -v <spanclass="k">$(</span><spanclass="nb">pwd</span><spanclass="k">)</span>:/data debian ls /data
</pre></div>
</pre></div>
</div>
</div>
<p>An advantage of using SSH is that we can connect to PaddlePaddle from
</li>
more than one terminals. For example, one terminal running vi and
</ul>
another one running Python interpreter. Another advantage is that we
</div>
can run the PaddlePaddle container on a remote server and SSH to it
<p>Please install nvidia-docker first following this <aclass="reference external"href="https://github.com/NVIDIA/nvidia-docker#quick-start">tutorial</a>.</p>
docker run <spanclass="si">${</span><spanclass="nv">CUDA_SO</span><spanclass="si">}</span><spanclass="si">${</span><spanclass="nv">DEVICES</span><spanclass="si">}</span> -it paddledev/paddle:<version>-gpu
<p>Now you can run a GPU image:</p>
<divclass="highlight-bash"><divclass="highlight"><pre><span></span>nvidia-docker run -it --rm paddlepaddle/paddle:0.10.0rc2-gpu /bin/bash
<dt>Use production image to release you AI application</dt>
<p>This image has packed related develop tools and runtime
<dd><pclass="first">Suppose that we have a simple application program in <codeclass="code docutils literal"><spanclass="pre">a.py</span></code>, we can test and run it using the production image:</p>
environment. Users and developers can use this image instead of
releasing, document writing etc. While different version of paddle
<spanclass="pre">`</span></code></p>
may depends on different version of libraries and tools, if you
<pclass="last">But this works only if all dependencies of <codeclass="code docutils literal"><spanclass="pre">a.py</span></code> are in the production image. If this is not the case, we need to build a new Docker image from the production image and with more dependencies installs.</p>
want to setup a local environment, you must pay attention to the
</dd>
versions. The development image contains:</p>
</dl>
<ulclass="simple">
<li>gcc/clang</li>
<li>nvcc</li>
<li>Python</li>
<li>sphinx</li>
<li>woboq</li>
<li>sshd</li>
</ul>
<p>Many developers use servers with GPUs, they can use ssh to login to
the server and run <codeclass="code docutils literal"><spanclass="pre">docker</span><spanclass="pre">exec</span></code> to enter the docker
container and start their work. Also they can start a development
docker image with SSHD service, so they can login to the container
<p>Note that by default <codeclass="code docutils literal"><spanclass="pre">docker</span><spanclass="pre">build</span></code> wouldn’t import source
<p>Above commands will start a docker container by running <codeclass="code docutils literal"><spanclass="pre">python</span>
tree into the image and build it. If we want to do that, we need docker the
<spanclass="pre">/workspace/example.py</span></code>. It will stop once <codeclass="code docutils literal"><spanclass="pre">python</span>
development docker image and then run the following command:</p>
<h2>Develop PaddlePaddle or Train Model Using C++ API<aclass="headerlink"href="#develop-paddlepaddle-or-train-model-using-c-api"title="Permalink to this headline">¶</a></h2>
<p>We will be using PaddlePaddle development image since it contains all
