PaddlePaddle (PArallel Distributed Deep LEarning) is a
simple, efficient and extensible deep learning framework.
Please refer to `PaddlePaddle Github <https://github.com/PaddlePaddle/Paddle>`_ for details, and `release note <../release_note_en.html>`_ for features incorporated in current version.
For beginners of PaddlePaddle, the following documentation will tutor you about installing PaddlePaddle:
- `Installation Manuals <../beginners_guide/install/index_en.html>`_ ：Installation on Ubuntu/CentOS/Windows/MacOS is supported.
The following resources are recommended for novices in deep learning:
- `Resources <../beginners_guide/basics/learning_materials.html>`_ ：Selected books and lectures about machine learning, deep learning and programming languages.
..
If you have been armed with certain level of deep learning knowledge, and it happens to be the first time to try PaddlePaddle, the following cases of model building will expedite your learning process:
- `Programming with Fluid <../beginners_guide/programming_guide/programming_guide.html>`_ ： Core concepts and basic usage of Fluid
- `Quick Start <../beginners_guide/quick_start/index.html>`_： Two easy-to-go models, linear regression model and digit recognition model, are in place to speed up your study of training neural networks
- `Deep Learning <../beginners_guide/basics/index.html>`_： This section encompasses various fields of fundamental deep learning knowledge, such as image classification, customized recommendation, machine translation, and examples implemented by Fluid are provided.
- When Ubuntu compiles, a lot of code segments are not recognized?
> This may be caused by a mismatch in the cmake version. Please use the following command in the gcc installation directory:
apt install gcc-4.8 g++-4.8
cp gcc gcc.bak
cp g++ g++.bak
rm gcc
rm g++
ln -s gcc-4.8 gcc
ln -s g++-4.8 g++
- Encountered paddlepaddle*.whl is not a supported wheel on this platform?
> The main reason for this problem is that there is no paddlepaddle installation package that matches the current system. Please check if the Python version is 2.7 series. In addition, the latest pip official source installation package defaults to the manylinux1 standard, you need to use the latest pip (>9.0.0) to install. You can update your pip by following these instructions:
> If the system supports linux_x86_64 and the installation package is manylinux1_x86_64, you need to upgrade the pip version to the latest; if the system supports manylinux1_x86_64 and the installation package (local) is linux_x86_64, you can rename this whl package to manylinux1_x86_64 and install it again.
- Is there a problem with Docker compilation?
> Please refer to [Issue12079](https://github.com/PaddlePaddle/Paddle/issues/12079) on GitHub.
- What is Docker?
> If you haven't heard of Docker, you can think of it as a virtualenv-like system, but it virtualises more than the Python runtime environment.
- Is Docker still a virtual machine?
> Someone uses a virtual machine to analogize to Docker. It should be emphasized that Docker does not virtualize any hardware. The compiler tools running in the Docker container are actually run directly on the native CPU and operating system. The performance is the same as installing the compiler on the machine.
- Why use Docker?
> Installing the tools and configurations in a Docker image standardizes the build environment. This way, if you encounter problems, others can reproduce the problem to help. In addition, for developers accustomed to using Windows and MacOS, there is no need to configure a cross-compilation environment using Docker.
- Can I choose not to use Docker?
> Of course you can. You can install development tools to the machine in the same way that you install them into Docker image. This document describes the Docker-based development process because it is easier than the other methods.
- How hard is it to learn Docker?
> It's not difficult to understand Docker. It takes about ten minutes to read this [article](https://zhuanlan.zhihu.com/p/19902938).
This can save you an hour of installing and configuring various development tools, as well as the need for new installations when switching machines. Don't forget that PaddlePaddle updates may lead to the need for new development tools. Not to mention the benefits of simplifying the recurrence of problems.
- Can I use an IDE?
> Of course, because the source code is on the machine. By default, the IDE calls a program like make to compile the source code. We only need to configure the IDE to call the Docker command to compile the source code.
Many PaddlePaddle developers use Emacs. They add two lines to their `~/.emacs` configuration file.
You can start the compilation by pressing `Ctrl-C` and` c`.
- Can I compile in parallel?
> Yes. Our Docker image runs a [Bash script](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/paddle/scripts/paddle_build.sh). This script calls `make -j$(nproc)` to start as many processes as the CPU cores to compile in parallel.
- Docker needs sudo?
> If you develop with your own computer, you will naturally have admin privileges (sudo). If you are developing from a public computer, you need to ask the administrator to install and configure Docker. In addition, the PaddlePaddle project is working hard to support other container technologies that don't require sudo, such as rkt.
- Is compiling slow on Windows/MacOS?
> Docker runs on both Windows and MacOS. However, it is actually running on a Linux virtual machine. It may be necessary to pay attention to allocate more CPU and memory to this virtual machine to ensure efficient compilation. Please refer to [issue627](https://github.com/PaddlePaddle/Paddle/issues/627) for details.
- Not enough disk?
> In the example in this article, the `--rm` parameter is used in the `docker run`command to ensure that containers after the end of the run are not retained on disk. You can use the `docker ps -a` command to see containers that are stopped but not deleted. The `docker build` command sometimes produces some intermediate results, an image with no name, and it also occupies the disk. You can refer to this [article](https://zaiste.net/removing_docker_containers/) to clean up this content.
- Can't I open `http://localhost:8888/` when using the book under DockerToolbox?
> You need to replace localhost with virtual machine ip. Generally type this in the browser: `http://192.168.99.100:8888/`
- After the pip install gpu version of PaddlePaddle runing, the SegmentFault appears as follows:
> The main reason for this problem is that your graphics card driver is lower than the corresponding CUDA version. Please ensure that your graphics card driver supports the CUDA version used.
-`Fatal Python error: PyThreadState_Get: no current thread running` error occurs when importing paddle.fluid after installing PaddlePaddle on MacOS.
- For Python2.7.x (install by brew): Please use `export LD_LIBRARY_PATH=/usr/local/Cellar/python@2/2.7.15_1/Frameworks/Python.framework/Versions/2.7 && export DYLD_LIBRARY_PATH=/usr/ Local/Cellar/python@2/2.7.15_1/Frameworks/Python.framework/Versions/2.7`
- For Python2.7.x (install by Python.org): Please use `export LD_LIBRARY_PATH=/Library/Frameworks/Python.framework/Versions/2.7 && export DYLD_LIBRARY_PATH=/Library/Frameworks/Python.framework/Versions/2.7`
- For Python3.5.x (install by Python.org): Please use `export LD_LIBRARY_PATH=/Library/Frameworks/Python.framework/Versions/3.5/ && export DYLD_LIBRARY_PATH=/Library/Frameworks/Python.framework/Versions/3.5 /`
- Use customized openblas under MACOS. See issue for details:
This instruction will show you how to compile PaddlePaddle on a 64-bit desktop or laptop and CentOS. The Centos systems we support must meet the following requirements:
* CentOS 7 / 6 (this involves whether the related tools can be installed normally)
## Determine which version to compile
***Only PaddlePaddle for CPU is supported.**
## Choose a compilation method
We provide two compilation methods under the CentOS system:
* Docker source code compilation (the CentOS 6 / 7 GPU version is not supported) (this image already contains python2.7, python3.6, python3.7 environment)
* Direct native source code compilation (does not support all versions of CentOS 6 and GPU versions of CentOS 7)
We recommend using **Docker for compilation** because we are installing both the tools and the configuration in a Docker image. This way, if you encounter problems, others can reproduce the problem to help. In addition, for developers accustomed to using Windows and MacOS, there is no need to configure a cross-compilation environment using Docker. It should be emphasized that Docker does not virtualize any hardware. The compiler tools running in the Docker container are actually running directly on the native CPU and operating system. The performance is the same as installing the compiler on the machine.
Also for those who can't install Docker for a variety of reasons, we also provide a way to **compile directly from sources**, but since the situation on host machine is more complicated, we only support specific systems.
### ***Compile with Docker***
In order to better use Docker and avoid problems, we recommend using **the highest version of Docker**. For details on **installing and using Docker**, please refer to the [official Docker documentation](https://docs.docker.com/install/).
Once you have **properly installed Docker**, you can start **compiling PaddlePaddle with Docker**:
1. First select the path where you want to store PaddlePaddle, then use the following command to clone PaddlePaddle's source code from github to a folder named Paddle in the local current directory:
3. Take advantage of the image we provided (with this command you don't have to download the image in advance):
`docker run --name paddle-test -v $PWD:/paddle --network=host -it` `hub.baidubce.com/paddlepaddle/paddle:latest-dev /bin/bash`
> `--name paddle-test` names the Docker container you created as paddle-test, `-v $PWD:/paddle` mounts the current directory to the /paddle directory in the Docker container (the PWD variable in Linux will expand to the current [Absolute path](https://baike.baidu.com/item/%E7%BB%9D%E5%AF%B9%E8%B7%AF%E5%BE%84/481185)), `-it` keeps interacting with the host, `hub.baidubce.com/paddlepaddle/paddle` creates a Docker container with an image called `hub.baidubce.com/paddlepaddle/paddle:latest-dev`, /bin/bash enters the container After starting the `/bin/bash` command.
4. After entering Docker, go to the paddle directory: `cd paddle`
5. Switch to a more stable version to compile:
`git checkout v1.1`
6. Create and enter the /paddle/build path:
`mkdir -p /paddle/build && cd /paddle/build`
7. Use the following command to install the dependencies: (For Python3: Please select the pip for the python version you wish to use, such as pip3.5, pip3.6)
For Python2: pip install protobuf==3.1.0
For Python3: pip3.5 install protobuf==3.1.0
> Install protobuf 3.1.0
`apt install patchelf`
> Installing patchelf, PatchELF is a small and useful program for modifying the dynamic linker and RPATH of ELF executables.
8. Execute cmake:
> For details on the compilation options, see the [compilation options table](../Tables.html/#Compile).
* For users who need to compile the **CPU version PaddlePaddle**:
> We currently do not support the compilation of the GPU version PaddlePaddle under CentOS.
9. Execute compilation:
`make -j$(nproc)`
> Use multicore compilation
10. After compiling successfully, go to the `/paddle/build/python/dist` directory and find the generated `.whl` package: `cd /paddle/build/python/dist`
11. Install the compiled `.whl` package on the current machine or target machine: (For Python3: Please select the pip corresponding to the python version you wish to use, such as pip3.5, pip3.6)
For Python2: pip install (whl package name)
For Python3: pip3.5 install (whl package name)
Now that you have successfully installed PaddlePaddle using Docker, you only need to run PaddlePaddle after entering the Docker container. For more Docker usage, please refer to the [official Docker documentation](https://docs.docker.com/).
> Notes: In order to reduce the size, `vim` is not installed in PaddlePaddle Docker image by default. You can edit the code in the container after executing `apt-get install -y vim` in the container.
Congratulations, you have now completed the process of compiling PaddlePaddle using Docker.
<br/><br/>
### *Local compilation*
**Please strictly follow the order of the following instructions**
1. Check that your computer and operating system meet the compilation standards we support: `uname -m && cat /etc/*release`
2. Update the source of `yum`: `yum update`, and add the necessary yum source: `yum install -y epel-release`, and install openCV in advance
3. Install the necessary tools `bzip2` and `make`: `yum install -y bzip2 `, `yum install -y make`
4. We support compiling and installing with virtualenv. First, create a virtual environment called `paddle-venv` with the following command:
* a. Install Python-dev:
For Python2: yum install python-devel
For Python3: (Please refer to the official Python installation process)
* b. Install pip:
For Python2: yum install python-pip (please have a pip version of 9.0.1 and above)
For Python3: (Please refer to the official Python installation process, and ensure that the pip3 version 9.0.1 and above, please note that in python3.6 and above, pip3 does not necessarily correspond to the python version, such as python3.7 default only Pip3.7)
* c. (Only For Python3) set Python3 related environment variables, here is python3.5 version example, please replace with the version you use (3.6, 3.7):
1. First find the path to the Python lib using ``` find `dirname $(dirname
$(which python3))` -name "libpython3.so"``` . If it is 3.6 or 3.7, change `python3` to `python3.6` or `python3.7`, then replace [python-lib-path] in the following steps with the file path found.
2. Set PYTHON_LIBRARIES: `export PYTHON_LIBRARY=[python-lib-path]`.
3. Secondly, use ```find `dirname $(dirname
$(which python3))`/include -name "python3.5m"``` to find the path to Python Include, please pay attention to the python version, then replace the following [python-include-path] to the file path found.
4. Set PYTHON_INCLUDE_DIR: `export PYTHON_INCLUDE_DIRS=[python-include-path]`
5. Set the system environment variable path: `export PATH=[python-lib-path]:$PATH `(here replace the last two levels content of [python-lib-path] with /bin/)
* d. Install the virtual environment `virtualenv` and `virtualenvwrapper` and create a virtual environment called `paddle-venv`: (please note the pip3 commands corresponding to the python version, such as pip3.6, pip3.7)
1. `pip install virtualenv` or `pip3 install virtualenv`
2. `Pip install virtualenvwrapper` or `pip3 install virtualenvwrapper`
3. Find `virtualenvwrapper.sh`: `find / -name virtualenvwrapper.sh` (please find the corresponding Python version of `virtualenvwrapper.sh`)
4. See the installation method in `virtualenvwrapper.sh`: `cat vitualenvwrapper.sh`
5. Install `virtualwrapper`
6. Create a virtual environment called `paddle-venv`: `mkvirtualenv paddle-venv`
5. Enter the virtual environment: `workon paddle-venv`
6. Before **executing the compilation**, please confirm that the related dependencies mentioned in the [compile dependency table](../Tables.html/#third_party) are installed in the virtual environment:
* Here is the installation method for `patchELF`. Other dependencies can be installed using `yum install` or `pip install`/`pip3 install` followed by the name and version:
`yum install patchelf`
> Users who can't use apt installation can refer to patchElF [github official documentation](https://gist.github.com/ruario/80fefd174b3395d34c14).
7. Put the PaddlePaddle source cloned in the Paddle folder in the current directory and go to the Paddle directory:
> If you encounter `Could NOT find PROTOBUF (missing: PROTOBUF_LIBRARY PROTOBUF_INCLUDE_DIR)`, you can re-execute the cmake command.
> Please note that the PY_VERSION parameter is replaced with the python version you need.
11. Compile with the following command:
`make -j$(nproc)`
12. After compiling successfully, go to the `/paddle/build/python/dist `directory and find the generated `.whl` package: `cd /paddle/build/python/dist`
13. Install the compiled `.whl` package on the current machine or target machine:
Congratulations, now you have completed the process of compiling PaddlePaddle natively.
<br/><br/>
### ***Verify installation***
After the installation is complete, you can use `python` to enter the Python interpreter and then use `import paddle.fluid` to verify that the installation was successful.
<br/><br/>
### ***How to uninstall***
Please use the following command to uninstall PaddlePaddle (users who use Docker to install PaddlePaddle should use the following command in the container containing PaddlePaddle. Please use the corresponding version of pip):
****CPU version of PaddlePaddle***: `pip uninstall paddlepaddle` or `pip3 uninstall paddlepaddle`
This instruction will show you how to compile PaddlePaddle on *64-bit desktops or laptops* and MacOS systems. The MacOS systems we support need to meet the following requirements:
* MacOS 10.12/10.13/10.14 (this involves whether the related tools can be installed normally)
## Determine which version to compile
***Only PaddlePaddle for CPU is supported.**
## Choose a compilation method
Under the MacOS 10.12/10.13/10.14 system we offer 2 ways to compile:
We recommend **using Docker for compilation** because we are installing both the tools and the configuration in a Docker image. This way, if you encounter problems, others can reproduce the problem to help. In addition, for developers accustomed to using Windows and MacOS, there is no need to configure a cross-compilation environment using Docker. It should be emphasized that Docker does not virtualize any hardware. The compiler tools running in the Docker container are actually running directly on the native CPU and operating system. The performance is the same as installing the compiler on the machine.
Also for those who can't install Docker for a variety of reasons, we also provide a way to **compile directly from local sources**, but since the situation on this machine is more complicated, we only support specific systems.
<br/><br/>
### ***Compile with Docker***
In order to better use Docker and avoid problems, we recommend using **the highest version of Docker**. For details on **installing and using Docker**, please refer to the [official Docker documentation](https://docs.docker.com/install/).
> Please note that running Docker on MacOS requires logging in with your dockerID, otherwise an Authenticate Failed error will occur.
Once you have **properly installed Docker**, you can start **compiling PaddlePaddle with Docker**:
1. Enter the terminal of the Mac
2. Please select the path where you want to store PaddlePaddle, and then use the following command to clone PaddlePaddle's source code from github to a folder named Paddle in the local current directory:
4. Take advantage of the image we provided (with this command you don't have to download the image in advance):
`docker run --name paddle-test -v $PWD:/paddle --network=host -it` `hub.baidubce.com/paddlepaddle/paddle:latest-dev /bin/bash`
> --name paddle-test Name the Docker container you created as paddle-test, -v $PWD:/paddle mount the current directory to the /paddle directory in the Docker container (the PWD variable in Linux will expand to the current path's [Absolute path](https://baike.baidu.com/item/绝对路径/481185)), -it keeps interacting with the host, `hub.baidubce.com/paddlepaddle/paddle:latest-dev` creates a Docker container with a mirror named `hub.baidubce.com/paddlepaddle/paddle:latest-dev`, /bin /bash starts the /bin/bash command after entering the container.
5. After entering Docker, go to the paddle directory: `cd paddle`
6. Switch to a more stable version to compile:
`git checkout v1.1`
7. Create and enter the /paddle/build path:
`mkdir -p /paddle/build && cd /paddle/build`
8. Use the following command to install the dependencies: (For Python3: Please select the pip for the python version you wish to use, such as pip3.5, pip3.6)
For Python2: pip install protobuf==3.1.0
For Python3: pip3.5 install protobuf==3.1.0
> Install protobuf 3.1.0.
`apt install patchelf`
> Installing patchelf, PatchELF is a small and useful program for modifying the dynamic linker and RPATH of ELF executables.
9. Execute cmake:
> For details on the compilation options, see the [compilation options table](../Tables.html/#Compile).
* For users who need to compile the **CPU version PaddlePaddle**:
> We currently do not support the compilation of the GPU version PaddlePaddle under CentOS.
10. Execute compilation:
`make -j$(nproc)`
> Use multicore compilation
11. After compiling successfully, go to the `/paddle/build/python/dist `directory and find the generated `.whl` package: `cd /paddle/build/python/dist`
12. Install the compiled `.whl` package on the current machine or target machine: (For Python3: Please select the pip corresponding to the python version you wish to use, such as pip3.5, pip3.6)
For Python2: pip install (whl package name)
For Python3: pip3.5 install (whl package name)
Now that you have successfully installed PaddlePaddle using Docker, you only need to run PaddlePaddle after entering the Docker container. For more Docker usage, please refer to the [official Docker documentation](https://docs.docker.com/).
> Note: In order to reduce the size, `vim` is not installed in PaddlePaddle Docker image by default. You can edit the code in the container after executing `apt-get install -y vim` in the container.
Congratulations, you have now completed the process of compiling PaddlePaddle using Docker.
<br/><br/>
### ***Native compilation***
**Please strictly follow the order of the following instructions**
1. Check that your computer and operating system meet our supported compilation standards: `uname -m` and view the system version `about this Mac`. And install openCV in advance.
2. Install python and pip:
> **Please do not use the Python initially given by MacOS**, we strongly recommend that you use [Homebrew](https://brew.sh/) to install python (for Python3 please use python [official download](https://www.python.org/downloads/mac-osx/) python3.5.x, python3.6.x, python3.7.x), pip and other dependencies, This will greatly reduce the difficulty of installing and compiling.
For python2: brew install python@2
For python3: Install using Python official website
> Please note that when you have multiple pythons installed on your mac, make sure that the python you are using is the python you wish to use.
3. (Only For Python2) Set Python-related environment variables:
- Use `find / -name libpython2.7.dylib` to find your current python `libpython2.7.dylib` path and use `export LD_LIBRARY_PATH=[libpython2.7.dylib path] && export DYLD_LIBRARY_PATH=[libpython2.7.dylib to the top two directories of the directory]`
4. (Only For Python3) Set Python-related environment variables:
- a. First use
```find `dirname $(dirname
$(which python3))` -name "libpython3.*.dylib"```
to find the path to Pythonlib (the first one it prompts is the dylib path for the python you need to use), then (below [python-lib-path] is replaced by finding the file path)
- b. Set PYTHON_LIBRARIES: `export PYTHON_LIBRARY=[python-lib-path]`
- c. Secondly use the path to find PythonInclude (usually find the above directory of [python-lib-path] as the include of the same directory, then find the path of python3.x or python2.x in the directory), then (the [python-include-path] in the following commands should be replaced by the path found here)
- d. Set PYTHON_INCLUDE_DIR: `export PYTHON_INCLUDE_DIRS=[python-include-path]`
- e. Set the system environment variable path: `export PATH=[python-bin-path]:$PATH` (here [python-bin-path] is the result of replacing the last two levels of [python-lib-path] with the path after /bin/ )
- f. Set the dynamic library link: `export LD_LIBRARY_PATH=[python-ld-path]` and `export DYLD_LIBRARY_PATH=[python-ld-path]` (here [python-ld-path] is the [python-bin-path]'s parent directory )
- g. (Optional) If you are compiling PaddlePaddle on MacOS 10.14, make sure you have the [appropriate version](http://developer.apple.com/download) of Xcode installed.
5. Before **compilation**, please confirm that the relevant dependencies mentioned in the [compilation dependency table](h../Tables.html/#third_party) are installed in your environment, otherwise we strongly recommend using `Homebrew` to install related dependencies.
> Under MacOS, if you have not modified or installed the dependencies mentioned in the "Compile Dependency Table", you only need to use `pip` to install `numpy`, `protobuf`, `wheel`, use `homebrew` to install `wget`, `swig`,then install `cmake`.
- a. Here is a special description of the installation of **CMake**:
Since we are using CMake3.4 please follow the steps below:
1. Download the CMake image from the [official CMake website](https://cmake.org/files/v3.4/cmake-3.4.3-Darwin-x86_64.dmg) and install it.
2. Enter `sudo "/Applications/CMake.app/Contents/bin/cmake-gui" –install` in the console
- b. If you do not want to use the system default blas and want to use your own installed OPENBLAS please read [FAQ](../FAQ.html/#OPENBLAS)
6. Put the PaddlePaddle source cloned in the Paddle folder in the current directory and go to the Paddle directory:
> ``-DPY_VERSION=3.5`` Please change to the Python version of the installation environment.
10. Compile with the following command:
`make -j4`
11. After compiling successfully, go to the `/paddle/build/python/dist `directory and find the generated `.whl` package: `cd /paddle/build/python/dist`
12. Install the compiled `.whl` package on the current machine or target machine:
> If you have multiple python environments and pips installed on your computer, please see the [FAQ](../Tables.html/#MACPRO).
Congratulations, now you have completed the process of compiling PaddlePaddle using this machine.
<br/><br/>
### ***Verify installation***
After the installation is complete, you can use `python` to enter the Python interpreter and then use `import paddle.fluid` to verify that the installation was successful.
<br/><br/>
### ***How to uninstall***
Please use the following command to uninstall PaddlePaddle (users who use Docker to install PaddlePaddle should use the following command in the container containing PaddlePaddle. Please use the corresponding version of pip):
****CPU version of PaddlePaddle***: `pip uninstall paddlepaddle` or `pip3 uninstall paddlepaddle`
This instruction describes how to compile PaddlePaddle on *64-bit desktops or laptops* and Ubuntu systems. The Ubuntu systems we support must meet the following requirements:
* Ubuntu 14.04/16.04/18.04 (this involves whether the related tools can be installed successfully)
## Determine which version to compile
***CPU version of PaddlePaddle**, if your system does not have an NVIDIA® GPU, you must install this version. This version is easier than the GPU version. So even if you have a GPU on your computer, we recommend that you first install the CPU version of PaddlePaddle to check if your local environment is suitable.
***GPU version of PaddlePaddle**, in order to make the PaddlePaddle program run more quickly, we usually use the GPU to accelerate the PaddlePaddle program, but the GPU version of PaddlePaddle needs to have the NVIDIA® GPU that meets the following conditions (see NVIDIA for the specific installation process and configuration). Official documentation: [For CUDA](https://docs.nvidia.com/cuda/cuda-installation-guide-linux/), For [cuDNN](https://docs.nvidia.com/deeplearning/sdk/cudnn-install/))
* *CUDA Toolkit 9.0 with cuDNN v7*
* *CUDA Toolkit 8.0 with cuDNN v7*
* *Hardware devices with GPU compute capability exceeding 1.0*
* Direct native source code compilation (does not support GPU version under ubuntu18.04)
We recommend using **Docker for compilation** because we are installing both the tools and the configuration in a Docker image. This way, if you encounter problems, others can reproduce the problem to help. In addition, for developers accustomed to using Windows and MacOS, there is no need to configure a cross-compilation environment using Docker. Someone uses a virtual machine to analogize to Docker. It should be emphasized that Docker does not virtualize any hardware. The compiler tools running in the Docker container are actually running directly on the native CPU and operating system. The performance is the same as installing the compiler on the machine.
We also provide methods that can be **compiled from local source code**, but since the situation on host machine is more complicated, we only provide support for specific systems.
<br/><br/>
## ***Compile with Docker***
In order to better use Docker and avoid problems, we recommend using **the highest version of Docker**. For details on **installing and using Docker**, please refer to [the official Docker documentation](https://docs.docker.com/install/).
> Please note that to install and use the PaddlePaddle version that supports GPU, you must first install nvidia-docker
Once you have **properly installed Docker**, you can start **compiling PaddlePaddle with Docker**:
1. First select the path where you want to store PaddlePaddle, then use the following command to clone PaddlePaddle's source code from github to a folder named Paddle in the local current directory:
3. Take advantage of the image we provided (with this command you don't have to download the image in advance):
`docker run --name paddle-test -v $PWD:/paddle --network=host -it hub.baidubce.com/paddlepaddle/paddle:latest-dev /bin/bash`
> --name paddle-test names the Docker container you created as paddle-test, -v $PWD:/paddle mounts the current directory to the /paddle directory in the Docker container (the PWD variable in Linux will expand to the current path's [absolute path](https://baike.baidu.com/item/%E7%BB%9D%E5%AF%B9%E8%B7%AF%E5%BE%84/481185)), -it keeps interacting with the host, `hub.baidubce.com/paddlepaddle/paddle:latest-dev` creates a Docker container with a mirror named `hub.baidubce.com/paddlepaddle/paddle:latest-dev`, /bin /bash Starts the /bin/bash command after entering the container.
4. After entering Docker, go to the paddle directory: `cd paddle`
5. Switch to a more stable release branch to compile: (Note that python 3.6, python 3.7 version are supported from the 1.2 branch)
`git checkout release/1.2`
6. Create and enter the /paddle/build path:
`mkdir -p /paddle/build && cd /paddle/build`
7. Use the following command to install the dependencies: (For Python3: Please select the pip for the python version you wish to use, such as pip3.5, pip3.6)
For Python2: pip install protobuf==3.1.0
For Python3: pip3.5 install protobuf==3.1.0
> Install protobuf 3.1.0.
`apt install patchelf`
> Installing patchelf, PatchELF is a small and useful program for modifying the dynamic linker and RPATH of ELF executables.
8. Execute cmake:
> For the meaning of the specific compiling options, [compilation options table](../Tables.html/#Compile) is your resort. Please note that the parameter `-DPY_VERSION` is the python version used in your current environment.
* For users who need to compile the **CPU version PaddlePaddle**:
10. After compiling successfully, go to the `/paddle/build/python/dist` directory and find the generated `.whl` package: `cd /paddle/build/python/dist`
11. Install the compiled `.whl` package on the current machine or target machine: (For Python3: Please select the pip corresponding to the python version you wish to use, such as pip3.5, pip3.6)
For Python2: pip install (whl package name)
For Python3: pip3.5 install (whl package name)
Now that you have successfully installed PaddlePaddle using Docker, you only need to run PaddlePaddle after entering the Docker container. For more Docker usage, please refer to the [official Docker documentation](https://docs.docker.com/).
> Note: In order to reduce the size, `vim` is not installed in PaddlePaddle Docker image by default. You can edit the code in the container after executing `apt-get install -y vim` in the container.
Congratulations, you have now completed the process of compiling PaddlePaddle using Docker.
<br/><br/>
### ***Local compilation***
**Please strictly follow the following instructions step by step**
1. Check that your computer and operating system meet the compilation standards we support: `uname -m && cat /etc/*release`
2. Update the source of `apt`: `apt update`, and install openCV in advance.
3. We support compiling and installing with virtualenv. First, create a virtual environment called `paddle-venv` with the following command:
* a. Install Python-dev: (Please install python3.x-dev that matches the current environment python version)
For Python2: apt install python-dev
For Python3: apt install python3.5-dev
* b. Install pip: (Please ensure that pip version is 9.0.1 and above ): (Please note that the version corresponding to python3 is modified)
* c. Install the virtual environment `virtualenv` and `virtualenvwrapper` and create a virtual environment called `paddle-venv` (please note the python version) :
1. `apt install virtualenv` or `pip install virtualenv` or `pip3 install virtualenv`
2. `apt install virtualenvwrapper` or `pip install virtualenvwrapper` or `pip3 install virtualenvwrapper`
4. (Only for Python3) Set the interpreter path for the virtual environment: `export VIRTUALENVWRAPPER_PYTHON=/usr/bin/python3.5`
5. See the installation method in `virtualenvwrapper.sh`: `cat virtualenvwrapper.sh`
6. Install `virtualwrapper` according to the installation method in `virtualenvwrapper.sh`
7. Create a virtual environment called `paddle-venv`: `mkvirtualenv paddle-venv`
4. Enter the virtual environment: `workon paddle-venv`
5. Before **executing the compilation**, please confirm that the related dependencies mentioned in [the compile dependency table](../Tables.html/#third_party) are installed in the virtual environment:
* Here is the installation method for `patchELF`. Other dependencies can be installed using `apt install` or `pip install` followed by the name and version:
`apt install patchelf`
> Users who can't use apt installation can refer to patchElF [github official documentation](https://gist.github.com/ruario/80fefd174b3395d34c14).
6. Clone the PaddlePaddle source code in the Paddle folder in the current directory and go to the Paddle directory:
7. Switch to a more stable release branch to compile, replacing the brackets and their contents with **the target branch name**:
- `git checkout [name of target branch]`
8. And please create and enter a directory called build:
`mkdir build && cd build`
9. Execute cmake:
> For details on the compilation options, see [the compilation options table](../Tables.html/#Compile).
* For users who need to compile the **CPU version of PaddlePaddle**: (For Python3: Please configure the correct python version for the PY_VERSION parameter)
For Python2: cmake .. -DWITH_FLUID_ONLY=ON -DWITH_GPU=OFF -DWITH_TESTING=OFF -DCMAKE_BUILD_TYPE=Release
For Python3: cmake .. -DPY_VERSION=3.5 -DWITH_FLUID_ONLY=ON -DWITH_GPU=OFF -DWITH_TESTING=OFF -DCMAKE_BUILD_TYPE=Release
* For users who need to compile **GPU version of PaddlePaddle**: (*only support ubuntu16.04/14.04*)
1. Please make sure that you have installed nccl2 correctly, or install nccl2 according to the following instructions (here is ubuntu 16.04, CUDA9, ncDNN7 nccl2 installation instructions), for more information on the installation information please refer to the [NVIDIA official website](https://developer.nvidia.com/nccl/nccl-download):
i. `wget http: / /developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1604/x86_64/nvidia-machine-learning-repo-ubuntu1604_1.0.0-1_amd64.deb `
ii. `dpkg -i nvidia-machine-learning-repo-ubuntu1604_1 .0.0-1_amd64.deb`
iii. `sudo apt-get install -y libnccl2=2.2.13-1+cuda9.0 libnccl-dev=2.2.13-1+cuda9.0`
2. If you have already installed `nccl2` correctly, you can start cmake: *(For Python3: Please configure the correct python version for the PY_VERSION parameter)*
For Python2: cmake .. -DWITH_FLUID_ONLY=ON -DWITH_GPU=ON -DWITH_TESTING=OFF -DCMAKE_BUILD_TYPE=Release
For Python3: cmake .. -DPY_VERSION=3.5 -DWITH_FLUID_ONLY=ON -DWITH_GPU=ON -DWITH_TESTING=OFF -DCMAKE_BUILD_TYPE=Release
> `-DPY_VERSION=3.5` Please change to the Python version of the installation environment
10. Compile with the following command:
`make -j$(nproc)`
11. After compiling successfully, go to the `/paddle/build/python/dist `directory and find the generated `.whl` package: `cd /paddle/build/python/dist`
12. Install the compiled `.whl` package on the current machine or target machine:
Congratulations, now you have completed the process of compiling PaddlePaddle natively.
<br/><br/>
### ***Verify installation***
After the installation is complete, you can use `python` or `python3` to enter the Python interpreter and then use `import paddle.fluid` to verify that the installation was successful.
<br/><br/>
### ***How to uninstall***
Please use the following command to uninstall PaddlePaddle (users who use Docker to install PaddlePaddle should use the following command in the container containing PaddlePaddle. Please use the corresponding version of pip):
-***CPU version of PaddlePaddle***: `pip uninstall paddlepaddle` or `pip3 uninstall paddlepaddle`
-***GPU version of PaddlePaddle***: `pip uninstall paddlepaddle-gpu` or `pip3 uninstall paddlepaddle-gpu`
This instruction will show you how to compile PaddlePaddle on a *64-bit desktop or laptop* and Windows 10. The Windows systems we support must meet the following requirements:
* Windows 10 Family Edition / Professional Edition / Enterprise Edition
* Visual Studio 2015 Update3
## Determine which version to compile
***Only PaddlePaddle for CPU is supported.**
## Choose a compilation method
We provide one compilation method under the Windows system:
* Direct source code compilation
Since the situation on host machine is more complicated, we only support specific systems.
Please note: The current version does not support NCCL, distributed, AVX, warpctc and MKL related functions.
### ***Local compilation***
**Please strictly follow the following instructions step by step**
1. Check that your computer and operating system meet our supported compilation standards
* Windows 10 Family Edition / Professional Edition / Enterprise Edition
* Visual Studio 2015 Update3
2. Install the necessary tools i.e. cmake, git and python :
> Cmake requires version 3.0 and above, which can be downloaded from the official website and added to the environment variable. [Download here](https://cmake.org/download/).
> Git can be downloaded on the official website and added to the environment variable. [Download here](https://gitforwindows.org/).
> Python requires version 2.7 and above, and ensure that modules such as numpy, protobuf, wheel are installed. [Download here](https://www.python.org/download/releases/2.7/).
* To Install numpy package you can use command `pip install numpy` or command `pip3 install numpy`
* To Install protobuf package you can use command `pip install protobuf` or command `pip3 install protobuf`
* To Install Wheel package you can use command `pip install wheel` or `pip3 install wheel`
3. Clone the PaddlePaddle source in the Paddle folder in the current directory and go to the Paddle directory:
4. Switch to a more stable release branch for compilation :
- `git checkout release/1.2`
5. Create a directory called build and enter it:
- `mkdir build`
- `cd build`
6. Execute cmake:
> For details on the compilation options, see [the compilation options list](../Tables.html/#Compile).
* For users who need to compile **the CPU version PaddlePaddle**:
For Python2:`cmake .. -G "Visual Studio 14 2015 Win64" -DPYTHON_INCLUDE_DIR = $ {PYTHON_INCLUDE_DIRS}
-DPYTHON_LIBRARY = $ {PYTHON_LIBRARY}
-DPYTHON_EXECUTABLE = $ {PYTHON_EXECUTABLE} -DWITH_FLUID_ONLY = ON -DWITH_GPU = OFF -DWITH_TESTING = OFF -DCMAKE_BUILD_TYPE =Release`
For Python3: `cmake .. -G "Visual Studio 14 2015 Win64" -DPY_VERSION = 3.5 -DPYTHON_INCLUDE_DIR = $ {PYTHON_INCLUDE_DIRS}
-DPYTHON_LIBRARY = $ {PYTHON_LIBRARY}
-DPYTHON_EXECUTABLE = $ {PYTHON_EXECUTABLE} -DWITH_FLUID_ONLY = ON -DWITH_GPU = OFF -DWITH_TESTING =OFF -DCMAKE_BUILD_TYPE=Release`
> If you encounter `Could NOT find PROTOBUF (missing: PROTOBUF_LIBRARY PROTOBUF_INCLUDE_DIR)`, you can re-execute the cmake command.
7. Some third-party dependencies (openblas, snappystream) currently require users to provide pre-compiled versions, or download pre-compiled files from `https://github.com/wopeizl/Paddle_deps` and place the entire `third_party` folder in the `build` directory.
8. Use Blend for Visual Studio 2015 to open `paddle.sln` file, select the platform `x64`, configure with `Release`, then begin to compile
9. Having compiled successfully, go to the `\paddle\build\python\dist`directory and find the generated `.whl` package:
`cd \paddle\build\python\dist`
10. Install the compiled `.whl` package on the current machine or target machine:
Congratulations, now you have completed the process of compiling PaddlePaddle natively.
### ***Verify installation***
After the installation is complete, you can use: `python` to enter the Python interpreter and then use `import paddle.fluid`. If there is no error prompted, the installation is successful.
### ***How to uninstall***
Please use the following command to uninstall PaddlePaddle:
****CPU version of PaddlePaddle*** : `pip uninstall paddlepaddle` or `pip3 uninstall paddlepaddle`
You can also choose to compile and install PaddlePaddle in the way of source code compilation. However, due to the diversity of the native environment, complicated problems may occur when compiling the source code, which may cause your installation to fail. In order to ensure your smooth installation, it is recommended that you prefer the normal installation method.
The manuals will guide you to install and build PaddlePaddle on your 64-bit desktop or laptop.
The versions of Python currently supported: Python 2.7-3.7
PaddlePaddle currently supports the following environments：
* *Ubuntu 14.04 /16.04 /18.04*
* *CentOS 7 / 6*
* *MacOS 10.11 / 10.12 / 10.13 / 10.14*
* *Windows7 / 8/ 10(Pro/Enterprise)*
Please make sure your environment meets the conditions above.
And the installation assumes your computer possesses 64-bit operating system, and AVX instruction set is supported by the processor, otherwise you should use the version of :code:`no_avx` in `whl package list - Dev <Tables_en.html/#ciwhls>`_ .
- If you are planning to use `pip <https://pypi.org/pypi/>`_ to install PaddlePaddle, please type the following commands directly:
:code:`pip install paddlepaddle` （latest CPU version of PaddlePaddle）
:code:`pip install paddlepaddle-gpu` （latest GPU version of PaddlePaddle）
:code:`pip install paddlepaddle==[pip version]`
where [pip version] can be looked up in `PyPi.org <https://pypi.org/search/?q=PaddlePaddle>`_
- If you are planning to use `docker <https://www.docker.com>`_ to install PaddlePaddle, please type the following commands directly:
:code:`docker run --name [Name of container] -it -v $PWD:/paddle hub.baidubce.com/paddlepaddle/paddle:[docker version] /bin/bash`
where [docker version] can be looked up in `DockerHub <https://hub.docker.com/r/paddlepaddle/paddle/tags/>`_
This note will show you how to install PaddlePaddle on a *64-bit desktop or laptop* and CentOS. The CentOS system we support needs to meet the following requirements:
Please note: Attempts on other systems may cause the installation to fail. Please ensure that your environment meets the conditions. The installation we provide by default requires your computer processor to support the AVX instruction set. Otherwise, please select the version of `no_avx` in [the latest Release installation package list](./Tables.html/#ciwhls-release).
Under CentOS you can use `cat /proc/cpuinfo | grep avx` to check if your processor supports the AVX instruction set.
* CentOS 6 / 7
## Determine which version to install
* Only PaddlePaddle for CPU is supported. If your computer does not have an NVIDIA® GPU, you can only install this version. If your computer has a GPU, it is recommended that you install the CPU version of PaddlePaddle first to check if your local environment is suitable.
* PaddlePaddle with GPU support, in order to make the PaddlePaddle program run more quickly, we accelerate the PaddlePaddle program through the GPU, but the GPU version of PaddlePaddle needs to have the NVIDIA® GPU that meets the following conditions (see the NVIDIA official for the specific installation process and configuration). Documentation: [For CUDA](https://docs.nvidia.com/cuda/cuda-installation-guide-linux/), [For cuDNN](https://docs.nvidia.com/deeplearning/sdk/cudnn-install/))
* *CUDA Toolkit 9.0 with cuDNN v7*
* *CUDA Toolkit 8.0 with cuDNN v7*
* *Hardware devices with GPU compute capability exceeding 1.0*
## Choose an installation method
We offer 4 installation methods under the CentOS system:
* Pip installation
* Docker installation (the GPU version is not supported) (the version of python in the image is 2.7)
* Source code compilation and installation (all versions of CentOS 6 and GPU version of CentOS 7 are not supported)
* Docker source compilation and installation (not supported for GPU version) (Python version 2.7, 3.5, 3.6, 3.7 in image)
**With pip installation** (the easiest way to install), we offer you a pip installation method, but it depends more on your native environment and may have some issues related to your local environment.
**Use Docker for installation** (the safest way to install), because we are installing the tools and configuration in a Docker image so that if something goes wrong, others can reproduce the problem for help. In addition, for developers accustomed to using Windows and MacOS, there is no need to configure a cross-compilation environment using Docker. It should be emphasized that Docker does not virtualize any hardware. The compiler tools running in the Docker container are actually run directly on the native CPU and operating system. The performance is the same as installing the compiler on the machine.
Compile and install from [**source**](#ct_source) and [**use Docker**](#ct_docker). This is a process of compiling the PaddlePaddle source code into a binary file and then installing the binary file. Compared with the binary form of PaddlePaddle that has been successfully tested and compiled for you, this manual compilation is more complicated, and we will answer you in detail at the end of this tutorial.
<br/><br/>
## ***Install PaddlePaddle using pip***
First, we use the following commands to check if **the environment of this machine** is suitable for installing PaddlePaddle:
`Uname -m && cat /etc/*release`
> The above command will display the operating system and processing bits of the machine. Please make sure your computer is consistent with the requirements of this tutorial.
Second, your computer needs to meet the following requirements:
* Python2.7.x (devel), Pip >= 9.0.1
> CentOS6 needs to compile Python 2.7 into a [shared library](./FAQ.html/#FAQ).
* Python3.5+.x (devel), Pip3 >= 9.0.1
> You may have installed pip on your CentOS. Please use pip -V to confirm that we recommend using pip 9.0.1 or higher to install.
Update the source of yum: `yum update` and install the extension source to install pip: `yum install -y epel-release`
Use the following command to install or upgrade Python and pip to the required version:
- For Python2: `sudo yum install python-devel python-pip`
- For Python3: (Please refer to the official Python installation, and pay attention to whether the python3 version is consistent with the python version corresponding to the pip3 command. If there are multiple python3 versions, please specify the pip version such as pip3.7, or add soft link from pip3 to the python version you use. )
> Even if you already have `Python` in your environment, you need to install the `python develop` package.
Here's how to install PaddlePaddle:
1. Use pip install to install PaddlePaddle:
* For users who need **the CPU version PaddlePaddle**: `pip install paddlepaddle` or `pip3 install paddlepaddle`
* For users who need **the GPU version PaddlePaddle**: `pip install paddlepaddle-gpu` or `pip3 install paddlepaddle-gpu`
> 1 . In order to prevent problem "nccl.h cannot be found", please first install nccl2 according to the instructions of [NVIDIA official website](https://developer.nvidia.com/nccl/nccl-download).
> 2 . If you do not specify the pypi package version number, we will by default provide you with a version of PaddlePaddle that supports Cuda 9/cuDNN v7.
* For users with `Cannot uninstall 'six'.` problems, the probable reason is the existing Python installation issues in your system. In this case, use `pip install paddlepaddle --ignore-installed six`(CPU) or `pip install paddlepaddle-gpu -- Ignore-installed six` (GPU) to resolve.
* For users with **other requirements**: `pip install paddlepaddle==[version number]` or `pip3 install paddlepaddle==[version number]`
> For `the version number`, please refer to [the latest Release installation package list](./Tables.html/#whls). If you need to obtain and install **the latest PaddlePaddle development branch**, you can download and install the latest whl installation package and c-api development package from [the latest dev installation package list](./Tables.html/#ciwhls) or our [CI system](https://paddleci.ngrok.io/project.html?projectId=Manylinux1&tab=projectOverview). To log in, click on "Log in as guest".
Now you have completed the process of installing PaddlePaddle via `pip install`.
<br/><br/>
## *Install using Docker*
In order to better use Docker and avoid problems, we recommend using **the highest version of Docker**. For details on installing and using Docker, please refer to [the official Docker documentation](https://docs.docker.com/install/).
> Please note that to install and use the PaddlePaddle version that supports GPU, you must first install [nvidia-docker](https://github.com/NVIDIA/nvidia-docker).
Once you have **properly installed Docker**, you can start **installing PaddlePaddle with Docker**.
1. Use the following command to pull the image we pre-installed for PaddlePaddle:
* For users who need a **CPU version of PaddlePaddle**, use the following command to pull the image we pre-installed for your *PaddlePaddle For CPU*:
> (Please replace [tag] with the contents of [the mirror table](./Tables.html/#dockers))
2. Use the following command to build from the already pulled image and enter the Docker container:
`Docker run --name [Name of container] -it -v $PWD:/paddle <imagename> /bin/bash`
> In the above command, --name [Name of container] sets the name of the Docker; the -it parameter indicates that the container is running interactively with the host machine; -v $PWD:/paddle specifies the current path (the PWD variable in Linux will expand to [The absolute path](https://baike.baidu.com/item/%E7%BB%9D%E5%AF%B9%E8%B7%AF%E5%BE%84/481185) of the current path ) which is mounted to the /paddle directory inside the container; `<imagename>` specifies the name of the image to use, if you need to use our image please use `hub.baidubce.com/paddlepaddle/paddle:[tag]`. Note: The meaning of the tag is the same as the second step. /bin/bash is the command to be executed in Docker.
3. (Optional: When you need to enter the Docker container a second time) re-enter the PaddlePaddle container with the following command:
`Docker start [Name of container]`
> start the container created previously
`Docker attach [Name of container]`
> Enter the started container in the last step.
Now that you have successfully installed PaddlePaddle using Docker, you only need to run PaddlePaddle after entering the Docker container. For more Docker usage, please refer to [the official Docker documentation](https://docs.docker.com/).
> Note: In order to reduce the size, `vim` is not installed in PaddlePaddle Docker image by default. You can edit the code in the container after executing `apt-get install -y vim` in the container.
<br/><br/>
## ***Verify installation***
After the installation is complete, you can use `python` or `python3` to enter the Python interpreter and then use `import paddle.fluid` to verify that the installation was successful.
<br/><br/>
## ***How to uninstall***
Please use the following command to uninstall PaddlePaddle (users who use Docker to install PaddlePaddle should use the following command in the container containing PaddlePaddle. Please use the corresponding version of pip):
****CPU version of PaddlePaddle***: `pip uninstall paddlepaddle` or `pip3 uninstall paddlepaddle`
****GPU version of PaddlePaddle***: `pip uninstall paddlepaddle-gpu` or `pip3 uninstall paddlepaddle-gpu`
This instruction will show you how to install PaddlePaddle on a *64-bit desktop or laptop* and MacOS system. The MacOS system we support must meet the following requirements.
Please note: Attempts on other systems may cause the installation to fail.
* MacOS 10.11/10.12/10.13/10.14
## Determine which version to install
* Only PaddlePaddle for CPU is supported.
## Choose an installation method
Under the MacOS system we offer 3 installation methods:
* Pip installation (not supported for GPU version) (distributed architecture is not supported under python3)
* Docker installation (the GPU version is not supported) (the version of python in the image is 2.7)
* Docker source compilation and installation (not supported for GPU version) (Python version 2.7, 3.5, 3.6, 3.7 in image)
**With pip installation** (the easiest way to install), we offer you a pip installation method, but it depends more on your local environment and may have some issues related to your local environment.
**Use Docker for installation** (the safest way to install), because we have installed the tools and configuration in a Docker image so that if something goes wrong, others can reproduce the problem for help. In addition, for developers accustomed to using Windows and MacOS, there is no need to configure a cross-compilation environment using Docker. It should be emphasized that Docker does not virtualize any hardware. The compiler tools running in the Docker container are actually running directly on the local CPU and operating system. The performance is the same as installing the compiler on the machine.
<br/><br/>
### ***Install using pip***
Due to the large difference in Python situation in MacOS, we do not provide quick installation commands. Please follow the steps below to install.
First, **check whether your computer and operating system** meet our supported compilation standards or not by `uname -m` and view the system version `About This Mac` in the Apple menu.
Second, your computer needs to meet the following requirements:
> **Please do not use the Python originally provided by MacOS**. For **Python 2**, we recommend Python2.7.15 provided by [Homebrew](https://brew.sh/) or [Python.org](https://www.python.org/ftp/python/2.7.15/python-2.7.15-macosx10.9.pkg). For Python3, please use python3.5.x, Python3.6.x or python3.7.x provided by [Python.org](https://www.python.org/downloads/mac-osx/).
For python2: brew install python@2 or use Python officially downloaded python2.7.15
For python3: Use python3.5.x, python3.6.x or python3.7.x downloaded from Python official site
* Python2.7.x, Pip >= 9.0.1
* Python3.5.x, Pip3 >= 9.0.1
* Python3.6.x, Pip3 >= 9.0.1
* Python3.7.x, Pip3 >= 9.0.1
> Note: You may have installed pip on your MacOS. Please use pip -V to confirm that its version is the recommended pip 9.0.1 or higher.
Here's how to install PaddlePaddle:
1. Use pip install to install PaddlePaddle:
* For users who need **the CPU version PaddlePaddle**: `pip install paddlepaddle` or `pip3 install paddlepaddle`
* For users with **other requirements**: `pip install paddlepaddle==[version number]` or `pip3 install paddlepaddle==[version number]`
> For `the version number`, please refer to [the latest Release installation package list](./Tables.html/#ciwhls-release). If you need to obtain and install **the latest PaddlePaddle development branch**, you can download the latest whl installation package and c-api development package from [the CI system](https://paddleci.ngrok.io/project.html?projectId=Manylinux1&tab=projectOverview) and install it. To log in, click on "Log in as guest".
Now you have completed the process of installing PaddlePaddle via `pip install`.
<br/><br/>
### ***Install using Docker***
In order to better use Docker and avoid problems, we recommend using **the highest version of Docker**. For details on **installing and using Docker**, please refer to [the official Docker documentation](https://docs.docker.com/install/).
Please note that running docker on MacOS requires logging in with your dockerID, otherwise an `Authenticate Failed` error will occur.
If Docker is **properly installed**, you can start **using Docker to install PaddlePaddle**.
1. Use the following command to pull the image we pre-installed for PaddlePaddle:
* For users who need **the CPU version of PaddlePaddle**, use the following command to pull the image we pre-installed for your *PaddlePaddle For CPU*:
> (Please replace [tag] with the contents of [the mirror table](./Tables.html/#dockers))
2. Use the following command to build from the already pulled image and enter the Docker container:
`Docker run --name [Name of container] -it -v $PWD:/paddle <imagename> /bin/bash`
> In the above command, --name [Name of container] sets the name of the Docker; the -it parameter indicates that the container is running interactively with the host machine; -v $PWD:/paddle specifies the current path (the PWD variable in Linux will expand to [The absolute path](https://baike.baidu.com/item/绝对路径/481185) ) of the current path is mounted to the /paddle directory inside the container; `<imagename>` specifies the name of the image to use, if you need to use our image please use `hub.baidubce.com/paddlepaddle/paddle:[tag]`. Note: The meaning of tag is the same as the second step; /bin/bash is the command to be executed in Docker.
3. (Optional: When you need to enter the Docker container a second time) re-enter the PaddlePaddle container with the following command:
`Docker start [Name of container]`
> start the container created previously.
`Docker attach [Name of container]`
> Enter the started container.
Now that you have successfully installed PaddlePaddle using Docker, you only need to run PaddlePaddle after entering the Docker container. For more Docker usage, please refer to [the official Docker documentation](https://docs.docker.com/).
> Note: In order to reduce the size, `vim` is not installed in PaddlePaddle Docker image by default. You can edit the code in the container after executing `apt-get install -y vim` in the container.
<br/><br/>
## ***Verify installation***
After the installation is complete, you can use `python` or `python3` to enter the python interpreter and then use `import paddle.fluid` to verify that the installation was successful.
<br/><br/>
## ***How to uninstall***
Please use the following command to uninstall PaddlePaddle (Users who use Docker to install PaddlePaddle should use the following command in the container containing PaddlePaddle. Please use the corresponding version of pip):
****CPU version of PaddlePaddle***: `pip uninstall paddlepaddle` or `pip3 uninstall paddlepaddle`
This instruction describes how to install PaddlePaddle on a *64-bit desktop or laptop* and Ubuntu system. The Ubuntu systems we support must meet the following requirements:
Please note: Attempts on other systems may cause the installation to fail. Please ensure that your environment meets the conditions. The installation we provide by default requires your computer processor to support the AVX instruction set. Otherwise, please select the version of `no_avx` in the [latest Release installation package list](./Tables.html/#ciwhls-release).
Under Ubuntu, you can use `cat /proc/cpuinfo | grep avx` to check if your processor supports the AVX instruction set.
* Ubuntu 14.04 /16.04 /18.04
## Determine which version to install
* PaddlePaddle for CPU is supported. If your computer does not have an NVIDIA® GPU, you can only install this version. If your computer has a GPU, it is also recommended that you install the CPU version of PaddlePaddle first to check if your local environment is suitable.
* PaddlePaddle for GPU is supported. In order to make the PaddlePaddle program run more quickly, we accelerate the PaddlePaddle program through the GPU, but the GPU version of the PaddlePaddle needs to have the NVIDIA® GPU that meets the following conditions (see the NVIDIA official documentation for the specific installation process and configuration: [For CUDA](https://docs.nvidia.com/cuda/cuda-installation-guide-linux/), [For cuDNN](https://docs.nvidia.com/deeplearning/sdk/cudnn-install/))
* *CUDA Toolkit 9.0 with cuDNN v7*
* *CUDA Toolkit 8.0 with cuDNN v7*
* *Hardware devices with GPU computing power exceeding 1.0*
## Choose an installation method
Under the Ubuntu system, we offer 4 installation methods:
* Pip installation
* Docker installation (the version of python in the image is 2.7)
* Source code compilation and installation
* Docker source code compilation and installation (the python version in the image is 2.7, 3.5, 3.6, 3.7)
**With pip installation** (the easiest way to install), we offer you a pip installation method, but it depends more on your native environment and may have some issues related to your local environment.
**Use Docker for installation** (the safest way to install), because we have installed the tools and configuration in a Docker image so that if something goes wrong, others can reproduce the problem for help. In addition, for developers accustomed to using Windows and MacOS, there is no need to configure a cross-compilation environment using Docker. It should be emphasized that Docker does not virtualize any hardware. The compiler tools running in the Docker container are actually running directly on the native CPU and operating system. The performance is the same as installing the compiler on the machine.
Compile and install from [**source**](#ubt_source) and [**use Docker**](#ubt_docker). This is a process of compiling the PaddlePaddle source code into a binary file and then installing the binary file. Compared to the binary PaddlePaddle that has been successfully tested and compiled for you, the manual compilation is more complicated, we will answer you in detail at the end of the description.
<br/><br/>
### ***Install using pip***
First, we use the following commands to **check if the environment of this machine** is suitable for installing PaddlePaddle:
uname -m && cat /etc/*release
>The above command will display the operating system and processing bits of the machine. Please make sure your computer is consistent with the requirements of this tutorial.
Second, your computer needs to meet any of the following requirements:
* Python2.7.x (dev), Pip >= 9.0.1
* Python3.5+.x (dev), Pip3 >= 9.0.1
>You may have installed pip on your Ubuntu. Please use pip -V or pip3 -V to confirm its version is the recommended pip 9.0.1 or higher.
Update apt source: `apt update`
Use the following command to install or upgrade Python and pip to the required version: (pip and dev installation in python3.6, python3.7 differs greatly across different Ubuntu versions, thus the steps are omitted here)
- For python2: `sudo apt install python-dev python-pip`
- For python3.5: `sudo apt install python3.5-dev and curl https://bootstrap.pypa.io/get-pip.py -o - | python3.5 && easy_install pip`
- For python3.6, python3.7: We assumed that python3.6 (3.7) and the corresponding versions of dev and pip3 are properly installed by yourself.
>Even if you already have Python 2 or Python 3 in your environment, you need to install Python-dev or Python 3.5 (3.6, 3.7) -dev.
Now let's install PaddlePaddle:
1. Use pip install to install PaddlePaddle
* For users who need **the CPU version PaddlePaddle**: `pip install paddlepaddle` or `pip3 install paddlepaddle`
* For users who need **the GPU version PaddlePaddle**: `pip install paddlepaddle-gpu` or `pip3 install paddlepaddle-gpu`
> 1.In order to prevent problem "nccl.h cannot be found", please first install nccl2 according to the following command (here is ubuntu 16.04, CUDA9, ncDNN v7 nccl2 installation instructions), for more information about the installation information, please refer to [the NVIDIA official website](https://developer.nvidia.com/nccl/nccl-download):
i. `Wget http://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1604/x86_64/nvidia-machine-learning-repo-ubuntu1604_1.0.0-1_amd64.deb`
ii. `dpkg -i nvidia-machine- Learning-repo-ubuntu1604_1.0.0-1_amd64.deb`
iii. `sudo apt-get install -y libnccl2=2.2.13-1+cuda9.0 libnccl-dev=2.2.13-1+cuda9.0`
> 2.If you do not specify the pypi package version number, we will by default provide you with a version of PaddlePaddle that supports Cuda 9/cuDNN v7.
* For users with `Cannot uninstall 'six'.` problems, the probable reason is the existing Python installation issues in your system. In this case, use `pip install paddlepaddle --ignore-installed six`(CPU) or `pip install paddlepaddle-gpu -- Ignore-installed six` (GPU) to resolve.
* For users with **other requirements**: `pip install paddlepaddle==[version number]` or `pip3 install paddlepaddle==[version number]`
> For `the version number`, please refer to [the latest Release installation package list](./Tables.html/#whls). If you need to obtain and install **the latest PaddlePaddle development branch**, you can download and install the latest whl installation package and c-api development package from [the latest dev installation package list](./Tables.html/#ciwhls) or our [CI system](https://paddleci.ngrok.io/project.html?projectId=Manylinux1&tab=projectOverview). To log in, click on "Log in as guest".
Now you have completed the process of installing PaddlePaddle using `pip install`.
<br/><br/>
### ***Install using Docker***
In order to better use Docker and avoid problems, we recommend using **the highest version of Docker**. For details on **installing and using Docker**, please refer to [the official Docker documentation](https://docs.docker.com/install/).
> Please note that to install and use the PaddlePaddle version that supports GPU, you must first install [nvidia-docker](https://github.com/NVIDIA/nvidia-docker)
If Docker is **properly installed**, you can start **installing PaddlePaddle using Docker**.
1. Use the following command to pull the image we pre-installed for PaddlePaddle:
* For users who need **a CPU version of PaddlePaddle**, use the following command to pull the image we pre-installed for your *PaddlePaddle For CPU*:
> (Please replace [tag] with the contents of [the mirror table](https://github.com/PaddlePaddle/FluidDoc/blob/develop/doc/fluid/beginners_guide/install/Tables.html/#dockers))
2. Use the following command to build from the already pulled image and enter the Docker container:
`Docker run --name [Name of container] -it -v $PWD:/paddle <imagename> /bin/bash`
> In the above command, --name [Name of container] sets the name of the Docker; the `-it` parameter indicates that the container is running interactively with the host machine; -v $PWD:/paddle specifies the current path (the PWD variable in Linux will expand to The absolute path of the current path) is mounted to the /paddle directory inside the container; `<imagename>` specifies the name of the image to use, if you need to use our image please use `hub.baidubce.com/paddlepaddle/paddle:[tag]`. Note: The meaning of tag is the same as the second step; /bin/bash is the command to be executed in Docker.
3. (Optional: When you need to enter the Docker container a second time) re-enter the PaddlePaddle container with the following command:
`Docker start [Name of container]`
> start the container created previously.
`Docker attach [Name of container]`
> Enter the started container.
Now that you have successfully installed PaddlePaddle using Docker, you only need to run PaddlePaddle after entering the Docker container. For more Docker usage, please refer to [the official Docker documentation](https://docs.docker.com/).
>Note: In order to reduce the size, `vim` is not installed in PaddlePaddle Docker image by default. You can edit the code in the container after executing `apt-get install -y vim` in the container.
<br/><br/>
## ***Verify installation***
After the installation is complete, you can use `python` or `python3` to enter the python interpreter and then use `import paddle.fluid` to verify that the installation was successful.
<br/><br/>
## ***How to uninstall***
Please use the following command to uninstall PaddlePaddle (users who use Docker to install PaddlePaddle should use the following command in the container containing PaddlePaddle. Please use the corresponding version of pip):
****CPU version of PaddlePaddle***: `pip uninstall paddlepaddle` or `pip3 uninstall paddlepaddle`
-***GPU version of PaddlePaddle***: `pip uninstall paddlepaddle-gpu` or `pip3 uninstall paddlepaddle-gpu`
This instruction will show you how to install PaddlePaddle on Windows. The following conditions must be met before you begin to install:
**a 64-bit desktop or laptop*
**Windows 7/8 , Windows 10 Professional/Enterprise Edition*
**Note** :
* The current version does not support NCCL, distributed training, AVX, warpctc and MKL related functions.
* Currently, only PaddlePaddle for CPU is supported on Windows.
## Installation Steps
### ***Install through pip***
* Check your Python versions
Python2.7.15，Python3.5.x，Python3.6.x，Python3.7.x on [Official Python](https://www.python.org/downloads/) are supported.
* Check your pip version
Version of pip or pip3 should be equal to or above 9.0.1 .
* Install PaddlePaddle
Execute `pip install paddlepaddle` or `pip3 install paddlepaddle` to download and install PaddlePaddle.
## ***Verify installation***
After completing the installation, you can use `python` or `python3` to enter the python interpreter and then use `import paddle.fluid` to verify that the installation was successful.
## ***How to uninstall***
Use the following command to uninstall PaddlePaddle : `pip uninstallpaddlepaddle `or `pip3 uninstall paddlepaddle`
@@ -5,10 +5,10 @@ In a large scale machine learning setup where the size of the training data is h
Polyak and Juditsky (1992) showed that the test performance of simple average of parameters obtained by Stochastic Gradient Descent (SGD) is as good as that of parameter values that are obtained by training the model over and over again, over the training dataset.
Hence, to accelerate the speed of Stochastic Gradient Descent, Averaged Stochastic Gradient Descent (ASGD) was proposed in Polyak and Juditsky (1992). For ASGD, the running average of parameters obtained by SGD, is used as the estimator for <imgsrc="https://raw.githubusercontent.com/PaddlePaddle/Paddle/develop/doc/fluid/images/theta_star.gif"/><br/> . The averaging is done as follows:
Hence, to accelerate the speed of Stochastic Gradient Descent, Averaged Stochastic Gradient Descent (ASGD) was proposed in Polyak and Juditsky (1992). For ASGD, the running average of parameters obtained by SGD, is used as the estimator for <imgsrc="https://raw.githubusercontent.com/PaddlePaddle/FluidDoc/develop/doc/fluid/images/theta_star.gif"/><br/> . The averaging is done as follows:
We propose averaging for any optimizer similar to how ASGD performs it, as mentioned above.
...
...
@@ -51,7 +51,7 @@ In the new design, we propose to create a new operation for averaging parameter
The ParameterAverageOptimizer op can be like any other operator with its own CPU/GPU implementation either using Eigen or separate CPU and GPU kernels. As the initial implementation, we can implement the kernel using Eigen following the abstraction pattern implemented for [Operators](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/fluid/operators/rmsprop_op.h). We also want to support the case when the Trainer/Optimizer runs on the GPU while ParameterAverageOptimizer runs on a CPU.
The idea of building an op for averaging is in sync with the refactored PaddlePaddle philosophy of using operators to represent any computation unit. The way the op will be added to the computation graph will be decided by the [layer functions](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/fluid/design/modules/python_api.md#layer-function) in Python API.
The idea of building an op for averaging is in sync with the refactored PaddlePaddle philosophy of using operators to represent any computation unit. The way the op will be added to the computation graph will be decided by the [layer functions](https://github.com/PaddlePaddle/FluidDoc/blob/develop/doc/fluid/design/modules/python_api.md#layer-function) in Python API.
### Python API implementation for ParameterAverageOptimizer
...
...
@@ -59,7 +59,7 @@ Based on Polyak and Juditsky (1992), we can generalize the averaging of updates
- Any optimizer (RMSProp , AdaGrad etc.)
- A window size. The op keeps accumulating updated parameter values over a window of N batches and takes an average. Move the averaged value to a buffer when window is full to avoid loss of precision.
Using the ParameterAverageOptimizer op, any user can add the operation to their computation graphs. However, this will require a lot of lines of code and we should design Python APIs that support averaging. As per the PaddlePaddle [Python API design](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/fluid/design/modules/python_api.md), the layer functions are responsible for creating operators, operator parameters and variables. Since ParameterAverageOptimizer will be an operator, it makes sense to create it in the layer functions.
Using the ParameterAverageOptimizer op, any user can add the operation to their computation graphs. However, this will require a lot of lines of code and we should design Python APIs that support averaging. As per the PaddlePaddle [Python API design](https://github.com/PaddlePaddle/FluidDoc/blob/develop/doc/fluid/design/modules/python_api.md), the layer functions are responsible for creating operators, operator parameters and variables. Since ParameterAverageOptimizer will be an operator, it makes sense to create it in the layer functions.
We will have a wrapper written in Python that will support the functionality and implement the actual core computation in C++ core as we have done for other [Optimizers](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/fluid/operators/rmsprop_op.cc)
#### Creation of the ParameterAverageOptimizer operator
...
...
@@ -71,4 +71,4 @@ The proposal is to add the op immediately while building the computation graph.
#### High-level API
In PaddlePaddle Python API, users will primarily rely on [layer functions](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/fluid/design/modules/python_api.md#layer-function) to create neural network layers. Hence, we also need to provide parameter average functionality in layer functions.
In PaddlePaddle Python API, users will primarily rely on [layer functions](https://github.com/PaddlePaddle/FluidDoc/blob/develop/doc/fluid/design/modules/python_api.md#layer-function) to create neural network layers. Hence, we also need to provide parameter average functionality in layer functions.
An equivalent PaddlePaddle program from the design doc of the [IfElseOp operator](../execution/if_else_op.md) is as follows:
An equivalent PaddlePaddle program from the design doc of the [IfElseOp operator](https://github.com/PaddlePaddle/FluidDoc/blob/develop/doc/fluid/design/execution/if_else_op.md) is as follows:
```python
importpaddleaspd
...
...
@@ -140,7 +140,7 @@ The difference is that variables in the C++ program contain scalar values, where
### Blocks with `for` and `RNNOp`
The following RNN model in PaddlePaddle from the [RNN design doc](../dynamic_rnn/rnn.md) :
The following RNN model in PaddlePaddle from the [RNN design doc](https://github.com/PaddlePaddle/FluidDoc/blob/develop/doc/fluid/design/dynamic_rnn/rnn_design_en.md) :
In [fluid](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/fluid/design/motivation/fluid.md), we encourage the user to use deep learning programming paradigms to describe the training process. When the user-written Python program is executed, it will first create a protobuf message
In [fluid](https://github.com/PaddlePaddle/FluidDoc/blob/develop/doc/fluid/design/motivation/fluid.md), we encourage the user to use deep learning programming paradigms to describe the training process. When the user-written Python program is executed, it will first create a protobuf message
[`ProgramDesc`](https://github.com/PaddlePaddle/Paddle/blob/a91efdde6910ce92a78e3aa7157412c4c88d9ee8/paddle/framework/framework.proto#L145) that describes the process and is conceptually like an [abstract syntax tree](https://en.wikipedia.org/wiki/Abstract_syntax_tree).
The executor runs the `ProgramDesc` like an interpreter. `ProgramDesc` contains the intrinsics (operators in this case) and variables which will be used, executor explicitly executes the stored precompiled code.
A PaddlePaddle program consists of two parts -- the first generates a `ProgramDesc` protobuf message that describes the program, and the second runs this message using a C++ class `Executor`.
A simple example PaddlePaddle program can be found in [graph.md](../others/graph.md):
A simple example PaddlePaddle program can be found in [graph.md](https://github.com/PaddlePaddle/FluidDoc/blob/develop/doc/fluid/design/others/graph.md):
```python
x=layer.data("images")
...
...
@@ -22,7 +22,7 @@ The first five lines of the following PaddlePaddle program generates, or, compil
The basic structure of a PaddlePaddle program is some nested blocks, as a C++ or Java program.
which provides one-to-one conversion between float32 and float16. These twos functions will do different conversion routines based on the current hardware. CUDA/ARM instrinsics will be used when the corresonding hardware is available. If the hardware or compiler level does not support float32 to float16 conversion, software emulation will be performed to do the conversion.
## float16 inference
In Fluid, a neural network is represented as a protobuf message called [ProgramDesc](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/fluid/design/concepts/program.md), whose Python wrapper is a [Program](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/fluid/design/modules/python_api.md#program). The basic structure of a program is some nested [blocks](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/fluid/design/modules/python_api.md#block), where each block consists of some [variable](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/fluid/design/modules/python_api.md#variable) definitions and a sequence of [operators](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/fluid/design/modules/python_api.md#operator). An [executor](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/fluid/design/concepts/executor.md) will run a given program desc by executing the sequence of operators in the entrance block of the program one by one.
In Fluid, a neural network is represented as a protobuf message called [ProgramDesc](https://github.com/PaddlePaddle/FluidDoc/blob/develop/doc/fluid/design/concepts/program.md), whose Python wrapper is a [Program](https://github.com/PaddlePaddle/FluidDoc/blob/develop/doc/fluid/design/modules/python_api.md#program). The basic structure of a program is some nested [blocks](https://github.com/PaddlePaddle/FluidDoc/blob/develop/doc/fluid/design/modules/python_api.md#block), where each block consists of some [variable](https://github.com/PaddlePaddle/FluidDoc/blob/develop/doc/fluid/design/modules/python_api.md#variable) definitions and a sequence of [operators](https://github.com/PaddlePaddle/FluidDoc/blob/develop/doc/fluid/design/modules/python_api.md#operator). An [executor](https://github.com/PaddlePaddle/FluidDoc/blob/develop/doc/fluid/design/concepts/executor.md) will run a given program desc by executing the sequence of operators in the entrance block of the program one by one.
### Operator level requirement
Each operator has many kernels for different data types, devices, and library types. The operator will select the appropriate kernel to run based on, among other things, the data type of the input variables. By default, every Fluid operator has a float data type kernel that takes float variables as input and generates float output.
...
...
@@ -105,10 +105,10 @@ This means that if we provide float input to the first operator in a program, th
The same principle applies if we want a program to run in float16 mode. We provide input variable of float16 data type to the first operator, and then one by one, each operator in the program will run the float16 kernel (provided that each operator in this program has float16 kernels registered) until we finally obtain a float16 output variable.
So the preliminary requirement for float16 inference is to add float16 kernel to operators that are needed in a specific kind of program. For example, float16 inference on an image classification neural network like Vgg or Resnet, typically requires the following operators to have float16 kernels: convolution, pooling, multiplication, addition, batch norm, dropout, relu, and softmax. Please refer to [new_op_en](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/fluid/dev/new_op_en.md) for details of how to add new kernels to an operator.
So the preliminary requirement for float16 inference is to add float16 kernel to operators that are needed in a specific kind of program. For example, float16 inference on an image classification neural network like Vgg or Resnet, typically requires the following operators to have float16 kernels: convolution, pooling, multiplication, addition, batch norm, dropout, relu, and softmax. Please refer to [new_op_en](https://github.com/PaddlePaddle/FluidDoc/blob/develop/doc/fluid/dev/new_op_en.md) for details of how to add new kernels to an operator.
### Variable level requirement
Operators including convolution and multiplication (used in fully-connected layers) takes as input not only the variables generated by the preceding operators but also [parameter](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/fluid/design/modules/python_api.md#parameter) variables, which contains the trained weights to apply to the input data. These weights are obtained in the Fluid training process and are by default of float data type.
Operators including convolution and multiplication (used in fully-connected layers) takes as input not only the variables generated by the preceding operators but also [parameter](https://github.com/PaddlePaddle/FluidDoc/blob/develop/doc/fluid/design/modules/python_api.md#parameter) variables, which contains the trained weights to apply to the input data. These weights are obtained in the Fluid training process and are by default of float data type.
When these operators are running in float16 mode, the float16 kernel requires those parameter variables to contain weights of Fluid float16 data type. Thus, we need a convenient way to convert the original float weights to float16 weights.
...
...
@@ -137,7 +137,7 @@ This problem can be solved by introducing a type-casting operator which takes an
### float16 transpiler
Put all the above requirements in mind, we designed a float16 inference transpiler that can tranpile a float32 mode inference program desc to a float16 mode one.
Given a float inference program and the corresponding variables of float32 weights in the [scope](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/fluid/design/concepts/scope.md),
Given a float inference program and the corresponding variables of float32 weights in the [scope](https://github.com/PaddlePaddle/FluidDoc/blob/develop/doc/fluid/design/concepts/scope.md),
this transpiler mainly does the following modifications:
1. Insert cast operators at the beginning of the program so that the input float data will be converted to float16 data type before feeding to subsequent operators to invoke the float16 kernel.
@@ -40,11 +40,11 @@ computation is only specified in Python code which sits outside of PaddlePaddle,
Similar to how a compiler uses an intermediate representation (IR) so that the programmer does not need to manually optimize their code for most of the cases, we can have an intermediate representation in PaddlePaddle as well. The compiler optimizes the IR as follows:
PaddlePaddle can support model parallelism by converting the IR so that the user no longer needs to manually perform the computation and operations in the Python component:
The IR for PaddlePaddle after refactoring is called a `Block`, it specifies the computation dependency graph and the variables used in the computation.
...
...
@@ -54,13 +54,13 @@ The user can not directly specify the parameter update rule for the parameter se
This could be fixed by making the parameter server also run an IR, which can be different to the trainer side
For a detailed explanation, refer to this document -
The major components are: *Python API*, *Distribute Transpiler* and *Remote Executor*.
...
...
@@ -97,9 +97,9 @@ The code above is a typical local training program, the "Training Program" is bu
`fluid.layer.fc`. The training is done by calling `Executor.run`
iteratively.
For more details, the implementation of IR is [Program](../program.md), and `ProgramDesc` is the protobuf type.
For more details, the implementation of IR is [Program](https://github.com/PaddlePaddle/FluidDoc/blob/develop/doc/fluid/design/concepts/program.md), and `ProgramDesc` is the protobuf type.
[Executor](../executor.md) simply runs the `ProgramDesc`. For local training you generally use
[Executor](https://github.com/PaddlePaddle/FluidDoc/blob/develop/doc/fluid/design/concepts/executor.md) simply runs the `ProgramDesc`. For local training you generally use
`Executor` to run the program locally. For any kind of distributed training, you can use
`RemoteExecutor` to specify desired distributed training method with some optional arguments.
...
...
@@ -152,7 +152,7 @@ for data in train_reader():
`JobDesc` object describe the distributed job resource specification to run on
@@ -171,13 +171,13 @@ In the future, a more general placement algorithm should be implemented, which m
The local training architecture will be the same as the distributed training architecture, the difference is that everything runs locally, and there is just one PaddlePaddle runtime:
1. Paddle use [SelectedRows](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/fluid/design/modules/selected_rows.md) as the storage format for the lookup table, the lookup table parameter will be split to multi-machine according to the hash of the feature ID, and data will also be split and send to the same machine to prefetch the parameter.
1. Paddle use [SelectedRows](https://github.com/PaddlePaddle/FluidDoc/blob/develop/doc/fluid/design/modules/selected_rows.md) as the storage format for the lookup table, the lookup table parameter will be split to multi-machine according to the hash of the feature ID, and data will also be split and send to the same machine to prefetch the parameter.
1. For common parameters, the trainer will get the whole parameter for training, but for the big lookup table, the trainer can not store the whole parameter. Because the input data feature is very sparse, every time we only need a few parameters for training, so we use `prefetch_op` to only prefetch the parameter needed to trainer.
1. The parameter variable W and its optimizer program are placed on the parameter server.
1. Operators are added to the program.
...
...
@@ -65,11 +65,11 @@ For embedding layers, the gradient may have many rows containing only 0 when tra
if the gradient uses a dense tensor to do parameter optimization,
it could spend unnecessary memory, slow down the calculations and waste
the bandwidth while doing distributed training.
In Fluid, we introduce [SelectedRows](../modules/selected_rows.md) to represent a list of rows containing
In Fluid, we introduce [SelectedRows](https://github.com/PaddlePaddle/FluidDoc/blob/develop/doc/fluid/design/modules/selected_rows.md) to represent a list of rows containing
non-zero gradient data. So when we do parameter optimization both locally and remotely,
we only need to send those non-zero rows to the optimizer operators:
The above diagram shows an RNN unrolled into a full network.
...
...
@@ -22,7 +22,7 @@ There are several important concepts here:
There could be local variables defined in each step-net. PaddlePaddle runtime realizes these variables in *step-scopes* which are created for each step.
@@ -49,7 +49,7 @@ or copy the memory value of the previous step to the current ex-memory variable.
### Usage in Python
For more information on Block, please refer to the [design doc](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/fluid/design/concepts/block.md).
For more information on Block, please refer to the [design doc](https://github.com/PaddlePaddle/FluidDoc/blob/develop/doc/fluid/design/concepts/block.md).
We can define an RNN's step-net using a Block:
...
...
@@ -93,7 +93,7 @@ For example, we could have a 2-level RNN, where the top level corresponds to par
The following figure illustrates feeding in text into the lower level, one sentence at a step, and the feeding in step outputs to the top level. The final top level output is about the whole text.
@@ -36,7 +36,7 @@ Please be aware that these Python classes need to maintain some construction-tim
### Program
A `ProgramDesc` describes a [DL program](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/fluid/design/concepts/program.md), which is composed of an array of `BlockDesc`s. The `BlockDesc`s in a `ProgramDesc` can have a tree-like hierarchical structure. However, the `ProgramDesc` onlys stores a flattened array of `BlockDesc`s. A `BlockDesc` refers to its parent block by its index in the array. For example, operators in the step block of an RNN operator need to be able to access variables in its ancestor blocks.
A `ProgramDesc` describes a [DL program](https://github.com/PaddlePaddle/FluidDoc/blob/develop/doc/fluid/design/concepts/program.md), which is composed of an array of `BlockDesc`s. The `BlockDesc`s in a `ProgramDesc` can have a tree-like hierarchical structure. However, the `ProgramDesc` onlys stores a flattened array of `BlockDesc`s. A `BlockDesc` refers to its parent block by its index in the array. For example, operators in the step block of an RNN operator need to be able to access variables in its ancestor blocks.
Whenever we create a block, we need to set its parent block to the current block, hence the Python class `Program` needs to maintain a data member `current_block`.
...
...
@@ -70,7 +70,7 @@ class Program(objects):
### Block
A [Block](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/fluid/design/concepts/block.md) includes
A [Block](https://github.com/PaddlePaddle/FluidDoc/blob/develop/doc/fluid/design/concepts/block.md) includes
1. a map from variable names to an instance of the Python `Variable` class, and
Using the low level ops, `L2_regularization_op` and `L1_regularization_op`, any user can add regularization to their computation graphs. However, this will require a lot of lines of code and we should design Python APIs that support regularization. An example of such an API can be seen in [Keras](https://keras.io/regularizers/). As per the PaddlePaddle [Python API design](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/fluid/design/modules/python_api.md), the layer functions are responsible for creating operators, operator parameters and variables. Since regularization is a property of parameters, it makes sense to create these in the layer functions.
Using the low level ops, `L2_regularization_op` and `L1_regularization_op`, any user can add regularization to their computation graphs. However, this will require a lot of lines of code and we should design Python APIs that support regularization. An example of such an API can be seen in [Keras](https://keras.io/regularizers/). As per the PaddlePaddle [Python API design](https://github.com/PaddlePaddle/FluidDoc/blob/develop/doc/fluid/design/modules/python_api.md), the layer functions are responsible for creating operators, operator parameters and variables. Since regularization is a property of parameters, it makes sense to create these in the layer functions.
#### Creation of Regularization ops
There are two possibilities for creating the regularization ops:
...
...
@@ -59,8 +59,8 @@ The proposal is to add these ops in a lazy manner just before the backward pass.
#### Storage of Regularization attributes
Since we want to create the regularization ops in a lazy manner, the regularization attributes (type of regularization and weight of regularization penalty) can be stored as attributes of the [`Parameter`](https://github.com/PaddlePaddle/Paddle/blob/develop/python/paddle/v2/framework/framework.py#L421) class. This is because regularization is a property of the parameters and storing regularization properties with Parameters also allows for shared parameters.
Since we want to create the regularization ops in a lazy manner, the regularization attributes (type of regularization and weight of regularization penalty) can be stored as attributes of the `Parameter` class. This is because regularization is a property of the parameters and storing regularization properties with Parameters also allows for shared parameters.
#### High-level API
In PaddlePaddle Python API, users will primarily rely on [layer functions](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/fluid/design/modules/python_api.md#layer-function) to create neural network layers. Hence, we also need to provide regularization functionality in layer functions. The design of these APIs can be postponed for later right now. A good reference for these APIs can be found in [Keras](https://keras.io/regularizers/) and also by looking at Tensorflow in [`tf.contrib.layers`](https://www.tensorflow.org/api_guides/python/contrib.layers).
In PaddlePaddle Python API, users will primarily rely on [layer functions](https://github.com/PaddlePaddle/FluidDoc/blob/develop/doc/fluid/design/modules/python_api.md#layer-function) to create neural network layers. Hence, we also need to provide regularization functionality in layer functions. The design of these APIs can be postponed for later right now. A good reference for these APIs can be found in [Keras](https://keras.io/regularizers/) and also by looking at Tensorflow in [`tf.contrib.layers`](https://www.tensorflow.org/api_guides/python/contrib.layers).
@@ -119,7 +119,7 @@ An actual Fluid example is described [here](https://github.com/PaddlePaddle/Pad
From the example, the Fluid programs look very similar to their PyTorch equivalent programs, except that Fluid's loop structure, wrapped with Python's `with` statement, could run much faster than just a Python loop.
We have more examples of the [`if-then-else`](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/fluid/design/execution/if_else_op.md) structure of Fluid.
We have more examples of the [`if-then-else`](https://github.com/PaddlePaddle/FluidDoc/tree/develop/doc/fluid/design/execution/if_else_op.md) structure of Fluid.
## Turing Completeness
...
...
@@ -131,7 +131,7 @@ There are two ways to execute a Fluid program. When a program is executed, it c
There is a C++ class [`Executor`](https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/fluid/framework/executor.h), which runs a `ProgramDesc`, similar to how an interpreter runs a Python program.
Fluid is moving towards the direction of a compiler, which is explain in [fluid_compiler.md](fluid_compiler.md).
Fluid is moving towards the direction of a compiler, which is explain in [fluid_compiler.md](../fluid_compiler.html).
# PaddlePaddle Fluid: Towards a Compiled Programming Language
As described in [fluid.md](fluid.md), when a Fluid application program
As described in [fluid.md](https://github.com/PaddlePaddle/FluidDoc/blob/develop/doc/fluid/design/motivation/fluid.md), when a Fluid application program
runs, it generates a `ProgramDesc` protobuf message as an intermediate
representation of itself. The C++ class `Executor` can run this
protobuf message as an interpreter. This article describes the Fluid
...
...
@@ -23,7 +23,7 @@ func paddlepaddle() {
}
```
This program consists of a [block](../concepts/block.md) of three operators --
This program consists of a [block](https://github.com/PaddlePaddle/FluidDoc/blob/develop/doc/fluid/design/concepts/block.md) of three operators --
`read`, `assign`, and `mult`. Its `ProgramDesc` message looks like
the following
...
...
@@ -107,4 +107,4 @@ where `cuda_context` could be a global variable of type
## Multi-Block Code Generation
Most Fluid application programs may have more than one blocks. To
execute them, we need to trace [scopes](../concepts/scope.md).
execute them, we need to trace [scopes](https://github.com/PaddlePaddle/FluidDoc/blob/develop/doc/fluid/design/concepts/scope.md).
@@ -11,7 +11,7 @@ The goals of refactoring include:
1. PaddlePaddle represents the computation, training and inference of Deep Learning models, by computation graphs.
1. Please refer to [computation graphs](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/fluid/design/others/graph.md) for a concrete example.
1. Please refer to [computation graphs](https://github.com/PaddlePaddle/FluidDoc/blob/develop/doc/fluid/design/others/graph.md) for a concrete example.
1. Users write Python programs to describe the graphs and run them (locally or remotely).
...
...
@@ -83,9 +83,9 @@ The word *graph* is interchangeable with *block* in this document. A graph cons
1. Add optimization operators to the computation graph.
1. Optionally, split the graph for distributed training.
1. The invocation of `train` or [`infer`](https://github.com/PaddlePaddle/Paddle/blob/develop/python/paddle/v2/inference.py#L108) methods in the Python program does the following:
1. The invocation of `train` or [`infer`](https://github.com/PaddlePaddle/Paddle/blob/release/1.2/python/paddle/v2/inference.py#L108) methods in the Python program does the following:
1. Create a new Scope instance in the [scope hierarchy](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/fluid/design/concepts/scope.md) for each run of a block,
1. Create a new Scope instance in the [scope hierarchy](../../concepts/scope.html) for each run of a block,
1. realize local variables defined in the BlockDesc message in the new scope,
1. a scope is similar to the stack frame in programming languages,
...
...
@@ -125,12 +125,12 @@ Compile Time -> IR -> Runtime
In PaddlePaddle's [Design](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/fluid/design/execution/switch.md), one Operator may have multiple kernels. Users may have some personal preference to choose a certain type of kernel for an operator, such as `force_cpu` to choose a CPU kernel, `use_cudnn` to choose a CUDNN kernel, we need to provide a way for users to do this.
In PaddlePaddle's [Design](https://github.com/PaddlePaddle/FluidDoc/blob/develop/doc/fluid/design/execution/switch.md), one Operator may have multiple kernels. Users may have some personal preference to choose a certain type of kernel for an operator, such as `force_cpu` to choose a CPU kernel, `use_cudnn` to choose a CUDNN kernel, we need to provide a way for users to do this.
In the current design, we use KernelType to describe one kernel.
...
...
@@ -14,7 +14,7 @@ struct KernelType {
```
`place_``data_type_` and `layout_` can be got from the input tensors of the operator, `GetActualKernelType(inputs)` use inputs to infer the proper kernel key that fit the incoming data, but users can not directly configure it.
The [design](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/fluid/design/execution/switch.md) also provides a virtual method `GetExpectedKernelType` that user can overload and use to choose the KernelType they want to use.
The [design](https://github.com/PaddlePaddle/FluidDoc/blob/develop/doc/fluid/design/execution/switch.md) also provides a virtual method `GetExpectedKernelType` that user can overload and use to choose the KernelType they want to use.
So we should send the information user defined in proto to `GetExpectedKernelType` for choosing a kernel.
@@ -11,7 +11,7 @@ In the old version of PaddlePaddle, the C++ class `RecurrentGradientMachine` imp
There are a lot of heuristic tricks in the sequence generation tasks, so the flexibility of sequence decoder is very important to users.
During the refactoring of PaddlePaddle, some new concepts are proposed such as: [LoDTensor](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/fluid/design/concepts/lod_tensor.md) and [TensorArray](https://github.com/PaddlePaddle/Paddle/blob/develop/doc/fluid/design/concepts/tensor_array.md) that can better support the sequence usage, and they can also help make the implementation of beam search based sequence decoder **more transparent and modular** .
During the refactoring of PaddlePaddle, some new concepts are proposed such as: [LoDTensor](https://github.com/PaddlePaddle/FluidDoc/blob/develop/doc/fluid/design/concepts/lod_tensor.md) and [TensorArray](https://github.com/PaddlePaddle/FluidDoc/blob/develop/doc/fluid/design/concepts/tensor_array.md) that can better support the sequence usage, and they can also help make the implementation of beam search based sequence decoder **more transparent and modular** .
For example, the RNN states, candidates IDs and probabilities of beam search can be represented all as `LoDTensors`;
the selected candidate's IDs in each time step can be stored in a `TensorArray`, and `Packed` to the sentences translated.
