提交 34dc87e2 编写于 作者: Y Yu Yang 提交者: GitHub

Merge pull request #716 from wangkuiyi/docker_arg

Replace m4 template by Dockerfile ARGs
Using and Building Docker Images PaddlePaddle in Docker Containers
================================ =================================
We release PaddlePaddle in the form of `Docker <https://www.docker.com/>`_ images on `dockerhub.com <https://hub.docker.com/r/paddledev/paddle/>`_. Running as Docker containers is currently the only officially-supported way to running PaddlePaddle. Docker container is currently the only officially-supported way to
running PaddlePaddle. This is reasonable as Docker now runs on all
major operating systems including Linux, Mac OS X, and Windows.
Please be aware that you will need to change `Dockers settings
<https://github.com/PaddlePaddle/Paddle/issues/627>`_ to make full use
of your hardware resource on Mac OS X and Windows.
Run Docker images
-----------------
For each version of PaddlePaddle, we release 4 variants of Docker images: CPU-only and GPU Images
-----------------------
+-----------------+-------------+-------+ For each version of PaddlePaddle, we release 2 Docker images, a
| | CPU AVX | GPU | CPU-only one and a CUDA GPU one. We do so by configuring
+=================+=============+=======+ `dockerhub.com <https://hub.docker.com/r/paddledev/paddle/>`_
| cpu | yes | no | automatically runs the following commands:
+-----------------+-------------+-------+
| cpu-noavx | no | no |
+-----------------+-------------+-------+
| gpu | yes | yes |
+-----------------+-------------+-------+
| gpu-noavx | no | yes |
+-----------------+-------------+-------+
We run the following command on Linux to check if the CPU supports :code:`AVX`. .. code-block:: base
.. code-block:: bash docker build -t paddle:cpu-noavx -f paddle/scripts/docker/Dockerfile .
docker build -t paddle:gpu-noavx -f paddle/scripts/docker/Dockerfile.gpu .
if cat /proc/cpuinfo | grep -i avx; then echo Yes; else echo No; fi
On Mac OS X, we need to run To run the CPU-only image as an interactive container:
.. code-block:: bash .. code-block:: bash
sysctl -a | grep machdep.cpu.leaf7_features docker run -it --rm paddledev/paddle:cpu-latest /bin/bash
Once we determine the proper variant, we can cope with the Docker image tag name by appending the version number. For example, the following command runs the AVX-enabled image of the most recent version: or, we can run it as a daemon container
.. code-block:: bash .. code-block:: bash
docker run -it --rm paddledev/paddle:cpu-latest /bin/bash docker run -d -p 2202:22 paddledev/paddle:cpu-latest
To run a GPU-enabled image, you need to install CUDA and let Docker knows about it: and SSH to this container using password :code:`root`:
.. code-block:: bash .. code-block:: bash
export CUDA_SO="$(\ls /usr/lib64/libcuda* | xargs -I{} echo '-v {}:{}') $(\ls /usr/lib64/libnvidia* | xargs -I{} echo '-v {}:{}')" ssh -p 2202 root@localhost
export DEVICES=$(\ls /dev/nvidia* | xargs -I{} echo '--device {}:{}')
docker run ${CUDA_SO} ${DEVICES} -it paddledev/paddle:gpu-latest
The default entry point of all our Docker images starts the OpenSSH server. To run PaddlePaddle and to expose OpenSSH port to 2202 on the host computer:
.. code-block:: bash An advantage of using SSH is that we can connect to PaddlePaddle from
more than one terminals. For example, one terminal running vi and
another one running Python interpreter. Another advantage is that we
can run the PaddlePaddle container on a remote server and SSH to it
from a laptop.
docker run -d -p 2202:22 paddledev/paddle:cpu-latest
Then we can login to the container using username :code:`root` and password :code:`root`: Above methods work with the GPU image too -- just please don't forget
to install CUDA driver and let Docker knows about it:
.. code-block:: bash .. code-block:: bash
ssh -p 2202 root@localhost export CUDA_SO="$(\ls /usr/lib64/libcuda* | xargs -I{} echo '-v {}:{}') $(\ls /usr/lib64/libnvidia* | xargs -I{} echo '-v {}:{}')"
export DEVICES=$(\ls /dev/nvidia* | xargs -I{} echo '--device {}:{}')
docker run ${CUDA_SO} ${DEVICES} -it paddledev/paddle:gpu-latest
Build Docker images Non-AVX Images
------------------- --------------
Developers might want to build Docker images from their local commit or from a tagged version. Suppose that your local repo is at :code:`~/work/Paddle`, the following steps builds a cpu variant from your current work: Please be aware that the CPU-only and the GPU images both use the AVX
instruction set, but old computers produced before 2008 do not support
AVX. The following command checks if your Linux computer supports
AVX:
.. code-block:: bash .. code-block:: bash
cd ~/Paddle if cat /proc/cpuinfo | grep -i avx; then echo Yes; else echo No; fi
./paddle/scripts/docker/generates.sh # Use m4 to generate Dockerfiles for each variant.
docker build -t paddle:latest -f ./paddle/scripts/docker/Dockerfile.cpu
As a release engineer, you might want to build Docker images for a certain version and publish them to dockerhub.com. You can do this by switching to the right Git tag, or create a new tag, before running `docker build`. For example, the following commands build Docker images for v0.9.0: If it doesn't, we will need to build non-AVX images manually from
source code:
.. code-block:: bash .. code-block:: bash
cd ~/Paddle cd ~
git checkout tags/v0.9.0 git clone github.com/PaddlePaddle/Paddle
./paddle/scripts/docker/generates.sh # Use m4 to generate Dockerfiles for each variant. cd Paddle
docker build -t paddle:cpu-v0.9.0 -f ./paddle/scripts/docker/Dockerfile.cpu docker build --build-arg WITH_AVX=OFF -t paddle:cpu-noavx -f paddle/scripts/docker/Dockerfile .
docker build --build-arg WITH_AVX=OFF -t paddle:gpu-noavx -f paddle/scripts/docker/Dockerfile.gpu .
FROM PADDLE_BASE_IMAGE FROM ubuntu:14.04
MAINTAINER PaddlePaddle Dev Team <paddle-dev@baidu.com> MAINTAINER PaddlePaddle Authors <paddle-dev@baidu.com>
# It is good to run apt-get install with Dockerfile RUN directive,
# because if the following invocation to /root/build.sh fails, `docker
# build` wouldn't have to re-install packages after we fix
# /root/build.sh. For more about Docker build cache, please refer to
# https://docs.docker.com/engine/userguide/eng-image/dockerfile_best-practices/#/build-cache.
RUN apt-get update && \ RUN apt-get update && \
apt-get install -y cmake libprotobuf-dev protobuf-compiler git \ apt-get install -y cmake libprotobuf-dev protobuf-compiler git \
libgoogle-glog-dev libgflags-dev libatlas-dev libatlas3-base g++ m4 python-pip \ libgoogle-glog-dev libgflags-dev libatlas-dev libatlas3-base g++ m4 python-pip \
...@@ -16,13 +11,13 @@ RUN apt-get update && \ ...@@ -16,13 +11,13 @@ RUN apt-get update && \
RUN pip install BeautifulSoup docopt PyYAML pillow \ RUN pip install BeautifulSoup docopt PyYAML pillow \
'sphinx>=1.4.0' sphinx_rtd_theme breathe recommonmark 'sphinx>=1.4.0' sphinx_rtd_theme breathe recommonmark
ENV WITH_GPU=PADDLE_WITH_GPU ARG WITH_AVX
ENV WITH_AVX=PADDLE_WITH_AVX ENV WITH_AVX=${WITH_AVX:-ON}
ENV WITH_GPU=OFF
RUN mkdir /paddle RUN mkdir /paddle
COPY . /paddle/ COPY . /paddle/
COPY paddle/scripts/docker/build.sh /root/ RUN /paddle/paddle/scripts/docker/build.sh
RUN /root/build.sh
RUN echo 'export LD_LIBRARY_PATH=/usr/lib64:${LD_LIBRARY_PATH}' >> /etc/profile RUN echo 'export LD_LIBRARY_PATH=/usr/lib64:${LD_LIBRARY_PATH}' >> /etc/profile
RUN pip install /usr/local/opt/paddle/share/wheels/*.whl RUN pip install /usr/local/opt/paddle/share/wheels/*.whl
......
FROM nvidia/cuda:7.5-cudnn5-devel-ubuntu14.04
MAINTAINER PaddlePaddle Authors <paddle-dev@baidu.com>
RUN apt-get update && \
apt-get install -y cmake libprotobuf-dev protobuf-compiler git \
libgoogle-glog-dev libgflags-dev libatlas-dev libatlas3-base g++ m4 python-pip \
python-protobuf python-numpy python-dev swig openssh-server \
wget unzip python-matplotlib tar xz-utils bzip2 gzip coreutils \
sed grep graphviz libjpeg-dev zlib1g-dev doxygen && \
apt-get clean -y
RUN pip install BeautifulSoup docopt PyYAML pillow \
'sphinx>=1.4.0' sphinx_rtd_theme breathe recommonmark
ARG WITH_AVX
ENV WITH_AVX=${WITH_AVX:-ON}
ENV WITH_GPU=ON
RUN mkdir /paddle
COPY . /paddle/
RUN /paddle/paddle/scripts/docker/build.sh
RUN echo 'export LD_LIBRARY_PATH=/usr/lib64:${LD_LIBRARY_PATH}' >> /etc/profile
RUN pip install /usr/local/opt/paddle/share/wheels/*.whl
RUN paddle version # print version after build
# Configure OpenSSH server. c.f. https://docs.docker.com/engine/examples/running_ssh_service
RUN mkdir /var/run/sshd
RUN echo 'root:root' | chpasswd
RUN sed -ri 's/^PermitRootLogin\s+.*/PermitRootLogin yes/' /etc/ssh/sshd_config
RUN sed -ri 's/UsePAM yes/#UsePAM yes/g' /etc/ssh/sshd_config
EXPOSE 22
CMD ["/usr/sbin/sshd", "-D"]
#!/bin/bash
set -e
cd `dirname $0`
m4 -DPADDLE_WITH_GPU=OFF \
-DPADDLE_WITH_AVX=ON \
-DPADDLE_BASE_IMAGE=ubuntu:14.04 \
Dockerfile.m4 > Dockerfile.cpu
m4 -DPADDLE_WITH_GPU=OFF \
-DPADDLE_WITH_AVX=OFF \
-DPADDLE_BASE_IMAGE=ubuntu:14.04 \
Dockerfile.m4 > Dockerfile.cpu-noavx
m4 -DPADDLE_WITH_GPU=ON \
-DPADDLE_WITH_AVX=ON \
-DPADDLE_BASE_IMAGE=nvidia/cuda:7.5-cudnn5-devel-ubuntu14.04 \
Dockerfile.m4 > Dockerfile.gpu
m4 -DPADDLE_WITH_GPU=ON \
-DPADDLE_WITH_AVX=OFF \
-DPADDLE_BASE_IMAGE=nvidia/cuda:7.5-cudnn5-devel-ubuntu14.04 \
Dockerfile.m4 > Dockerfile.gpu-noavx
Markdown is supported
0% .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑!
想要评论请 注册