diff --git a/doc/getstarted/build_and_install/docker_install.rst b/doc/getstarted/build_and_install/docker_install.rst index 5f272aabd7c2213b89f6a6b42be34c9c492d89bd..fefa337f835bebeecc26a452612feb8b252d3eb2 100644 --- a/doc/getstarted/build_and_install/docker_install.rst +++ b/doc/getstarted/build_and_install/docker_install.rst @@ -1,81 +1,83 @@ -Using and Building Docker Images -================================ +PaddlePaddle in Docker Containers +================================= -We release PaddlePaddle in the form of `Docker `_ images on `dockerhub.com `_. Running as Docker containers is currently the only officially-supported way to running PaddlePaddle. +Docker container is currently the only officially-supported way to +running PaddlePaddle. This is reasonable as Docker now runs on all +major operating systems including Linux, Mac OS X, and Windows. +Please be aware that you will need to change `Dockers settings +`_ to make full use +of your hardware resource on Mac OS X and Windows. -Run Docker images ------------------ -For each version of PaddlePaddle, we release 4 variants of Docker images: +CPU-only and GPU Images +----------------------- -+-----------------+-------------+-------+ -| | CPU AVX | GPU | -+=================+=============+=======+ -| cpu | yes | no | -+-----------------+-------------+-------+ -| cpu-noavx | no | no | -+-----------------+-------------+-------+ -| gpu | yes | yes | -+-----------------+-------------+-------+ -| gpu-noavx | no | yes | -+-----------------+-------------+-------+ +For each version of PaddlePaddle, we release 2 Docker images, a +CPU-only one and a CUDA GPU one. We do so by configuring +`dockerhub.com `_ +automatically runs the following commands: -We run the following command on Linux to check if the CPU supports :code:`AVX`. +.. code-block:: base -.. code-block:: bash + docker build -t paddle:cpu-noavx -f paddle/scripts/docker/Dockerfile . + docker build -t paddle:gpu-noavx -f paddle/scripts/docker/Dockerfile.gpu . - if cat /proc/cpuinfo | grep -i avx; then echo Yes; else echo No; fi -On Mac OS X, we need to run +To run the CPU-only image as an interactive container: .. code-block:: bash - sysctl -a | grep machdep.cpu.leaf7_features - + docker run -it --rm paddledev/paddle:cpu-latest /bin/bash -Once we determine the proper variant, we can cope with the Docker image tag name by appending the version number. For example, the following command runs the AVX-enabled image of the most recent version: +or, we can run it as a daemon container .. code-block:: bash - docker run -it --rm paddledev/paddle:cpu-latest /bin/bash + docker run -d -p 2202:22 paddledev/paddle:cpu-latest -To run a GPU-enabled image, you need to install CUDA and let Docker knows about it: +and SSH to this container using password :code:`root`: .. code-block:: bash - export CUDA_SO="$(\ls /usr/lib64/libcuda* | xargs -I{} echo '-v {}:{}') $(\ls /usr/lib64/libnvidia* | xargs -I{} echo '-v {}:{}')" - export DEVICES=$(\ls /dev/nvidia* | xargs -I{} echo '--device {}:{}') - docker run ${CUDA_SO} ${DEVICES} -it paddledev/paddle:gpu-latest - -The default entry point of all our Docker images starts the OpenSSH server. To run PaddlePaddle and to expose OpenSSH port to 2202 on the host computer: + ssh -p 2202 root@localhost -.. code-block:: bash +An advantage of using SSH is that we can connect to PaddlePaddle from +more than one terminals. For example, one terminal running vi and +another one running Python interpreter. Another advantage is that we +can run the PaddlePaddle container on a remote server and SSH to it +from a laptop. - docker run -d -p 2202:22 paddledev/paddle:cpu-latest -Then we can login to the container using username :code:`root` and password :code:`root`: +Above methods work with the GPU image too -- just please don't forget +to install CUDA driver and let Docker knows about it: .. code-block:: bash - ssh -p 2202 root@localhost + export CUDA_SO="$(\ls /usr/lib64/libcuda* | xargs -I{} echo '-v {}:{}') $(\ls /usr/lib64/libnvidia* | xargs -I{} echo '-v {}:{}')" + export DEVICES=$(\ls /dev/nvidia* | xargs -I{} echo '--device {}:{}') + docker run ${CUDA_SO} ${DEVICES} -it paddledev/paddle:gpu-latest -Build Docker images -------------------- +Non-AVX Images +-------------- -Developers might want to build Docker images from their local commit or from a tagged version. Suppose that your local repo is at :code:`~/work/Paddle`, the following steps builds a cpu variant from your current work: +Please be aware that the CPU-only and the GPU images both use the AVX +instruction set, but old computers produced before 2008 do not support +AVX. The following command checks if your Linux computer supports +AVX: .. code-block:: bash - cd ~/Paddle - ./paddle/scripts/docker/generates.sh # Use m4 to generate Dockerfiles for each variant. - docker build -t paddle:latest -f ./paddle/scripts/docker/Dockerfile.cpu + if cat /proc/cpuinfo | grep -i avx; then echo Yes; else echo No; fi + -As a release engineer, you might want to build Docker images for a certain version and publish them to dockerhub.com. You can do this by switching to the right Git tag, or create a new tag, before running `docker build`. For example, the following commands build Docker images for v0.9.0: +If it doesn't, we will need to build non-AVX images manually from +source code: .. code-block:: bash - cd ~/Paddle - git checkout tags/v0.9.0 - ./paddle/scripts/docker/generates.sh # Use m4 to generate Dockerfiles for each variant. - docker build -t paddle:cpu-v0.9.0 -f ./paddle/scripts/docker/Dockerfile.cpu + cd ~ + git clone github.com/PaddlePaddle/Paddle + cd Paddle + docker build --build-arg WITH_AVX=OFF -t paddle:cpu-noavx -f paddle/scripts/docker/Dockerfile . + docker build --build-arg WITH_AVX=OFF -t paddle:gpu-noavx -f paddle/scripts/docker/Dockerfile.gpu . diff --git a/paddle/scripts/docker/Dockerfile.m4 b/paddle/scripts/docker/Dockerfile similarity index 65% rename from paddle/scripts/docker/Dockerfile.m4 rename to paddle/scripts/docker/Dockerfile index f2822acdde757c78769c4a4f0dba317eb2d94a4c..be077dfc000bf10ba8b3129483badef89bcd50e9 100644 --- a/paddle/scripts/docker/Dockerfile.m4 +++ b/paddle/scripts/docker/Dockerfile @@ -1,11 +1,6 @@ -FROM PADDLE_BASE_IMAGE -MAINTAINER PaddlePaddle Dev Team +FROM ubuntu:14.04 +MAINTAINER PaddlePaddle Authors -# It is good to run apt-get install with Dockerfile RUN directive, -# because if the following invocation to /root/build.sh fails, `docker -# build` wouldn't have to re-install packages after we fix -# /root/build.sh. For more about Docker build cache, please refer to -# https://docs.docker.com/engine/userguide/eng-image/dockerfile_best-practices/#/build-cache. RUN apt-get update && \ apt-get install -y cmake libprotobuf-dev protobuf-compiler git \ libgoogle-glog-dev libgflags-dev libatlas-dev libatlas3-base g++ m4 python-pip \ @@ -16,13 +11,13 @@ RUN apt-get update && \ RUN pip install BeautifulSoup docopt PyYAML pillow \ 'sphinx>=1.4.0' sphinx_rtd_theme breathe recommonmark -ENV WITH_GPU=PADDLE_WITH_GPU -ENV WITH_AVX=PADDLE_WITH_AVX +ARG WITH_AVX +ENV WITH_AVX=${WITH_AVX:-ON} +ENV WITH_GPU=OFF RUN mkdir /paddle COPY . /paddle/ -COPY paddle/scripts/docker/build.sh /root/ -RUN /root/build.sh +RUN /paddle/paddle/scripts/docker/build.sh RUN echo 'export LD_LIBRARY_PATH=/usr/lib64:${LD_LIBRARY_PATH}' >> /etc/profile RUN pip install /usr/local/opt/paddle/share/wheels/*.whl diff --git a/paddle/scripts/docker/Dockerfile.gpu b/paddle/scripts/docker/Dockerfile.gpu new file mode 100644 index 0000000000000000000000000000000000000000..292b4dc66b85cc69c8386b16593fa23fc6a05d8e --- /dev/null +++ b/paddle/scripts/docker/Dockerfile.gpu @@ -0,0 +1,32 @@ +FROM nvidia/cuda:7.5-cudnn5-devel-ubuntu14.04 +MAINTAINER PaddlePaddle Authors + +RUN apt-get update && \ + apt-get install -y cmake libprotobuf-dev protobuf-compiler git \ + libgoogle-glog-dev libgflags-dev libatlas-dev libatlas3-base g++ m4 python-pip \ + python-protobuf python-numpy python-dev swig openssh-server \ + wget unzip python-matplotlib tar xz-utils bzip2 gzip coreutils \ + sed grep graphviz libjpeg-dev zlib1g-dev doxygen && \ + apt-get clean -y +RUN pip install BeautifulSoup docopt PyYAML pillow \ + 'sphinx>=1.4.0' sphinx_rtd_theme breathe recommonmark + +ARG WITH_AVX +ENV WITH_AVX=${WITH_AVX:-ON} +ENV WITH_GPU=ON + +RUN mkdir /paddle +COPY . /paddle/ +RUN /paddle/paddle/scripts/docker/build.sh + +RUN echo 'export LD_LIBRARY_PATH=/usr/lib64:${LD_LIBRARY_PATH}' >> /etc/profile +RUN pip install /usr/local/opt/paddle/share/wheels/*.whl +RUN paddle version # print version after build + +# Configure OpenSSH server. c.f. https://docs.docker.com/engine/examples/running_ssh_service +RUN mkdir /var/run/sshd +RUN echo 'root:root' | chpasswd +RUN sed -ri 's/^PermitRootLogin\s+.*/PermitRootLogin yes/' /etc/ssh/sshd_config +RUN sed -ri 's/UsePAM yes/#UsePAM yes/g' /etc/ssh/sshd_config +EXPOSE 22 +CMD ["/usr/sbin/sshd", "-D"] diff --git a/paddle/scripts/docker/generate.sh b/paddle/scripts/docker/generate.sh deleted file mode 100755 index b808a62ec29cab6058ec76cd46fff4cbd72e36cd..0000000000000000000000000000000000000000 --- a/paddle/scripts/docker/generate.sh +++ /dev/null @@ -1,24 +0,0 @@ -#!/bin/bash - -set -e -cd `dirname $0` - -m4 -DPADDLE_WITH_GPU=OFF \ - -DPADDLE_WITH_AVX=ON \ - -DPADDLE_BASE_IMAGE=ubuntu:14.04 \ - Dockerfile.m4 > Dockerfile.cpu - -m4 -DPADDLE_WITH_GPU=OFF \ - -DPADDLE_WITH_AVX=OFF \ - -DPADDLE_BASE_IMAGE=ubuntu:14.04 \ - Dockerfile.m4 > Dockerfile.cpu-noavx - -m4 -DPADDLE_WITH_GPU=ON \ - -DPADDLE_WITH_AVX=ON \ - -DPADDLE_BASE_IMAGE=nvidia/cuda:7.5-cudnn5-devel-ubuntu14.04 \ - Dockerfile.m4 > Dockerfile.gpu - -m4 -DPADDLE_WITH_GPU=ON \ - -DPADDLE_WITH_AVX=OFF \ - -DPADDLE_BASE_IMAGE=nvidia/cuda:7.5-cudnn5-devel-ubuntu14.04 \ - Dockerfile.m4 > Dockerfile.gpu-noavx