# How to run PaddleServing in Docker ([简体中文](RUN_IN_DOCKER_CN.md)|English) ## Requirements Docker (GPU version requires nvidia-docker to be installed on the GPU machine) This document takes Python2 as an example to show how to run Paddle Serving in docker. You can also use Python3 to run related commands by replacing `python` with `python3`. ## CPU ### Get docker image You can get images in two ways: 1. Pull image directly ```bash docker pull hub.baidubce.com/paddlepaddle/serving:0.2.0 ``` 2. Building image based on dockerfile Create a new folder and copy [Dockerfile](../tools/Dockerfile) to this folder, and run the following command: ```bash docker build -t hub.baidubce.com/paddlepaddle/serving:0.2.0 . ``` ### Create container ```bash docker run -p 9292:9292 --name test -dit hub.baidubce.com/paddlepaddle/serving:0.2.0 docker exec -it test bash ``` The `-p` option is to map the `9292` port of the container to the `9292` port of the host. ### Install PaddleServing In order to make the image smaller, the PaddleServing package is not installed in the image. You can run the following command to install it: ```bash pip install paddle-serving-server ``` You may need to use a domestic mirror source (in China, you can use the Tsinghua mirror source of the following example) to speed up the download: ```shell pip install paddle-serving-server -i https://pypi.tuna.tsinghua.edu.cn/simple ``` ### Test example Before running the GPU version of the Server side code, you need to set the `CUDA_VISIBLE_DEVICES` environment variable to specify which GPUs the prediction service uses. The following example specifies two GPUs with indexes 0 and 1: ```bash export CUDA_VISIBLE_DEVICES=0,1 ``` Get the trained Boston house price prediction model by the following command: ```bash wget --no-check-certificate https://paddle-serving.bj.bcebos.com/uci_housing.tar.gz tar -xzf uci_housing.tar.gz ``` - Test HTTP service Running on the Server side (inside the container): ```bash python -m paddle_serving_server.serve --model uci_housing_model --thread 10 --port 9292 --name uci &>std.log 2>err.log & ``` Running on the Client side (inside or outside the container): ```bash curl -H "Content-Type:application/json" -X POST -d '{"x": [0.0137, -0.1136, 0.2553, -0.0692, 0.0582, -0.0727, -0.1583, -0.0584, 0.6283, 0.4919, 0.1856, 0.0795, -0.0332], "fetch":["price"]}' http://127.0.0.1:9292/uci/prediction ``` - Test RPC service Running on the Server side (inside the container): ```bash python -m paddle_serving_server.serve --model uci_housing_model --thread 10 --port 9292 &>std.log 2>err.log & ``` Running following Python code on the Client side (inside or outside the container, The `paddle-serving-client` package needs to be installed): ```bash from paddle_serving_client import Client client = Client() client.load_client_config("uci_housing_client/serving_client_conf.prototxt") client.connect(["127.0.0.1:9292"]) data = [0.0137, -0.1136, 0.2553, -0.0692, 0.0582, -0.0727, -0.1583, -0.0584, 0.6283, 0.4919, 0.1856, 0.0795, -0.0332] fetch_map = client.predict(feed={"x": data}, fetch=["price"]) print(fetch_map) ``` ## GPU The GPU version is basically the same as the CPU version, with only some differences in interface naming (GPU version requires nvidia-docker to be installed on the GPU machine). ### Get docker image You can also get images in two ways: 1. Pull image directly ```bash nvidia-docker pull hub.baidubce.com/paddlepaddle/serving:0.2.0-gpu ``` 2. Building image based on dockerfile Create a new folder and copy [Dockerfile.gpu](../tools/Dockerfile.gpu) to this folder, and run the following command: ```bash nvidia-docker build -t hub.baidubce.com/paddlepaddle/serving:0.2.0-gpu . ``` ### Create container ```bash nvidia-docker run -p 9292:9292 --name test -dit hub.baidubce.com/paddlepaddle/serving:0.2.0-gpu nvidia-docker exec -it test bash ``` The `-p` option is to map the `9292` port of the container to the `9292` port of the host. ### Install PaddleServing In order to make the image smaller, the PaddleServing package is not installed in the image. You can run the following command to install it: ```bash pip install paddle-serving-server-gpu ``` You may need to use a domestic mirror source (in China, you can use the Tsinghua mirror source of the following example) to speed up the download: ```shell pip install paddle-serving-server-gpu -i https://pypi.tuna.tsinghua.edu.cn/simple ``` ### Test example When running the GPU Server, you need to set the GPUs used by the prediction service through the `--gpu_ids` option, and the CPU is used by default. An error will be reported when the value of `--gpu_ids` exceeds the environment variable `CUDA_VISIBLE_DEVICES`. The following example specifies to use a GPU with index 0: ```shell export CUDA_VISIBLE_DEVICES=0,1 python -m paddle_serving_server_gpu.serve --model uci_housing_model --port 9292 --gpu_ids 0 ``` Get the trained Boston house price prediction model by the following command: ```bash wget --no-check-certificate https://paddle-serving.bj.bcebos.com/uci_housing.tar.gz tar -xzf uci_housing.tar.gz ``` - Test HTTP service Running on the Server side (inside the container): ```bash python -m paddle_serving_server_gpu.serve --model uci_housing_model --thread 10 --port 9292 --name uci --gpu_ids 0 ``` Running on the Client side (inside or outside the container): ```bash curl -H "Content-Type:application/json" -X POST -d '{"x": [0.0137, -0.1136, 0.2553, -0.0692, 0.0582, -0.0727, -0.1583, -0.0584, 0.6283, 0.4919, 0.1856, 0.0795, -0.0332], "fetch":["price"]}' http://127.0.0.1:9292/uci/prediction ``` - Test RPC service Running on the Server side (inside the container): ```bash python -m paddle_serving_server_gpu.serve --model uci_housing_model --thread 10 --port 9292 --gpu_ids 0 ``` Running following Python code on the Client side (inside or outside the container, The `paddle-serving-client` package needs to be installed): ```bash from paddle_serving_client import Client client = Client() client.load_client_config("uci_housing_client/serving_client_conf.prototxt") client.connect(["127.0.0.1:9292"]) data = [0.0137, -0.1136, 0.2553, -0.0692, 0.0582, -0.0727, -0.1583, -0.0584, 0.6283, 0.4919, 0.1856, 0.0795, -0.0332] fetch_map = client.predict(feed={"x": data}, fetch=["price"]) print(fetch_map) ``` ## Attention The images provided by this document are all runtime images, which do not support compilation. If you want to compile from source, refer to [COMPILE](COMPILE.md).