diff --git a/doc/RUN_IN_DOCKER.md b/doc/RUN_IN_DOCKER.md index 2f8dc9aec1dafc440329da1438fea4b938677ca0..caffcf3b8ea5fec820c02f1ed5e02b45c4417d93 100644 --- a/doc/RUN_IN_DOCKER.md +++ b/doc/RUN_IN_DOCKER.md @@ -32,63 +32,9 @@ The `-p` option is to map the `9292` port of the container to the `9292` port of ### Install PaddleServing -In order to make the image smaller, the PaddleServing package is not installed in the image. You can run the following command to install it: - -```bash -pip install paddle-serving-server -``` - -You may need to use a domestic mirror source (in China, you can use the Tsinghua mirror source of the following example) to speed up the download: - -```shell -pip install paddle-serving-server -i https://pypi.tuna.tsinghua.edu.cn/simple -``` - -### Test example - -Get the trained Boston house price prediction model by the following command: - -```bash -wget --no-check-certificate https://paddle-serving.bj.bcebos.com/uci_housing.tar.gz -tar -xzf uci_housing.tar.gz -``` - -- Test HTTP service - - Running on the Server side (inside the container): - - ```bash - python -m paddle_serving_server.serve --model uci_housing_model --thread 10 --port 9292 --name uci >std.log 2>err.log & - ``` - - Running on the Client side (inside or outside the container): - - ```bash - curl -H "Content-Type:application/json" -X POST -d '{"feed":{"x": [0.0137, -0.1136, 0.2553, -0.0692, 0.0582, -0.0727, -0.1583, -0.0584, 0.6283, 0.4919, 0.1856, 0.0795, -0.0332]}, "fetch":["price"]}' http://127.0.0.1:9292/uci/prediction - ``` - -- Test RPC service - - Running on the Server side (inside the container): - - ```bash - python -m paddle_serving_server.serve --model uci_housing_model --thread 10 --port 9292 >std.log 2>err.log & - ``` - - Running following Python code on the Client side (inside or outside the container, The `paddle-serving-client` package needs to be installed): - - ```bash - from paddle_serving_client import Client - - client = Client() - client.load_client_config("uci_housing_client/serving_client_conf.prototxt") - client.connect(["127.0.0.1:9292"]) - data = [0.0137, -0.1136, 0.2553, -0.0692, 0.0582, -0.0727, - -0.1583, -0.0584, 0.6283, 0.4919, 0.1856, 0.0795, -0.0332] - fetch_map = client.predict(feed={"x": data}, fetch=["price"]) - print(fetch_map) - ``` +The mirror comes with `paddle_serving_server`, `paddle_serving_client`, and `paddle_serving_app` corresponding to the mirror tag version. If users don’t need to change the version, they can use it directly, which is suitable for environments without extranet services. +If you need to change the version, please refer to the instructions on the homepage to download the pip package of the corresponding version. ## GPU @@ -100,7 +46,7 @@ The GPU version is basically the same as the CPU version, with only some differe Refer to [this document](DOCKER_IMAGES.md) for a docker image, the following is an example of an `cuda9.0-cudnn7` image: ```shell -nvidia-docker pull hub.baidubce.com/paddlepaddle/serving:latest-cuda9.0-cudnn7 +docker pull hub.baidubce.com/paddlepaddle/serving:latest-cuda9.0-cudnn7 ``` ### Create container @@ -110,77 +56,21 @@ nvidia-docker run -p 9292:9292 --name test -dit hub.baidubce.com/paddlepaddle/se nvidia-docker exec -it test bash ``` -The `-p` option is to map the `9292` port of the container to the `9292` port of the host. - -### Install PaddleServing - -In order to make the image smaller, the PaddleServing package is not installed in the image. You can run the following command to install it: +or ```bash -pip install paddle-serving-server-gpu -``` - -You may need to use a domestic mirror source (in China, you can use the Tsinghua mirror source of the following example) to speed up the download: - -```shell -pip install paddle-serving-server-gpu -i https://pypi.tuna.tsinghua.edu.cn/simple -``` - -### Test example - -When running the GPU Server, you need to set the GPUs used by the prediction service through the `--gpu_ids` option, and the CPU is used by default. An error will be reported when the value of `--gpu_ids` exceeds the environment variable `CUDA_VISIBLE_DEVICES`. The following example specifies to use a GPU with index 0: -```shell -export CUDA_VISIBLE_DEVICES=0,1 -python -m paddle_serving_server_gpu.serve --model uci_housing_model --port 9292 --gpu_ids 0 -``` - - -Get the trained Boston house price prediction model by the following command: - -```bash -wget --no-check-certificate https://paddle-serving.bj.bcebos.com/uci_housing.tar.gz -tar -xzf uci_housing.tar.gz +docker run --gpus all -p 9292:9292 --name test -dit hub.baidubce.com/paddlepaddle/serving:latest-cuda9.0-cudnn7 +docker exec -it test bash ``` -- Test HTTP service - - Running on the Server side (inside the container): - - ```bash - python -m paddle_serving_server_gpu.serve --model uci_housing_model --thread 10 --port 9292 --name uci --gpu_ids 0 - ``` - - Running on the Client side (inside or outside the container): - - ```bash - curl -H "Content-Type:application/json" -X POST -d '{"feed":{"x": [0.0137, -0.1136, 0.2553, -0.0692, 0.0582, -0.0727, -0.1583, -0.0584, 0.6283, 0.4919, 0.1856, 0.0795, -0.0332]}, "fetch":["price"]}' http://127.0.0.1:9292/uci/prediction - ``` - -- Test RPC service - - Running on the Server side (inside the container): - - ```bash - python -m paddle_serving_server_gpu.serve --model uci_housing_model --thread 10 --port 9292 --gpu_ids 0 - ``` - - Running following Python code on the Client side (inside or outside the container, The `paddle-serving-client` package needs to be installed): - - ```bash - from paddle_serving_client import Client - - client = Client() - client.load_client_config("uci_housing_client/serving_client_conf.prototxt") - client.connect(["127.0.0.1:9292"]) - data = [0.0137, -0.1136, 0.2553, -0.0692, 0.0582, -0.0727, - -0.1583, -0.0584, 0.6283, 0.4919, 0.1856, 0.0795, -0.0332] - fetch_map = client.predict(feed={"x": data}, fetch=["price"]) - print(fetch_map) - ``` +The `-p` option is to map the `9292` port of the container to the `9292` port of the host. +### Install PaddleServing +The mirror comes with `paddle_serving_server_gpu`, `paddle_serving_client`, and `paddle_serving_app` corresponding to the mirror tag version. If users don’t need to change the version, they can use it directly, which is suitable for environments without extranet services. +If you need to change the version, please refer to the instructions on the homepage to download the pip package of the corresponding version. -## Attention +## Precautious Runtime images cannot be used for compilation. If you want to compile from source, refer to [COMPILE](COMPILE.md). diff --git a/doc/RUN_IN_DOCKER_CN.md b/doc/RUN_IN_DOCKER_CN.md index bc5f03f42c9c963f91b09335c95e4d9bb881e58d..6fe4147bfb4e68fcb014e32c2a5bf0c3c4a927e7 100644 --- a/doc/RUN_IN_DOCKER_CN.md +++ b/doc/RUN_IN_DOCKER_CN.md @@ -20,7 +20,6 @@ Docker(GPU版本需要在GPU机器上安装nvidia-docker) docker pull hub.baidubce.com/paddlepaddle/serving:latest ``` - ### 创建容器并进入 ```bash @@ -32,74 +31,11 @@ docker exec -it test bash ### 安装PaddleServing -为了减小镜像的体积,镜像中没有安装Serving包,要执行下面命令进行安装。 - -```bash -pip install paddle-serving-server -``` - -您可能需要使用国内镜像源(例如清华源)来加速下载。 - -```shell -pip install paddle-serving-server -i https://pypi.tuna.tsinghua.edu.cn/simple -``` - -### 测试example - -通过下面命令获取训练好的Boston房价预估模型: - -```bash -wget --no-check-certificate https://paddle-serving.bj.bcebos.com/uci_housing.tar.gz -tar -xzf uci_housing.tar.gz -``` - -- 测试HTTP服务 - - 在Server端(容器内)运行: +镜像里自带对应镜像tag版本的`paddle_serving_server`,`paddle_serving_client`,`paddle_serving_app`,如果用户不需要更改版本,可以直接使用,适用于没有外网服务的环境。 - ```bash - python -m paddle_serving_server.serve --model uci_housing_model --thread 10 --port 9292 --name uci >std.log 2>err.log & - ``` +如果需要更换版本,请参照首页的指导,下载对应版本的pip包。 - 在Client端(容器内或容器外)运行: - - ```bash - curl -H "Content-Type:application/json" -X POST -d '{"feed":{"x": [0.0137, -0.1136, 0.2553, -0.0692, 0.0582, -0.0727, -0.1583, -0.0584, 0.6283, 0.4919, 0.1856, 0.0795, -0.0332]}, "fetch":["price"]}' http://127.0.0.1:9292/uci/prediction - ``` - -- 测试RPC服务 - - 在Server端(容器内)运行: - - ```bash - python -m paddle_serving_server.serve --model uci_housing_model --thread 10 --port 9292 >std.log 2>err.log & - ``` - - 在Client端(容器内或容器外,需要安装`paddle-serving-client`包)运行下面Python代码: - - ```python - from paddle_serving_client import Client - - client = Client() - client.load_client_config("uci_housing_client/serving_client_conf.prototxt") - client.connect(["127.0.0.1:9292"]) - data = [0.0137, -0.1136, 0.2553, -0.0692, 0.0582, -0.0727, - -0.1583, -0.0584, 0.6283, 0.4919, 0.1856, 0.0795, -0.0332] - fetch_map = client.predict(feed={"x": data}, fetch=["price"]) - print(fetch_map) - ``` - -## GPU版本 - -GPU版本与CPU版本基本一致,只有部分接口命名的差别(GPU版本需要在GPU机器上安装nvidia-docker)。 - -### 获取镜像 - -参考[该文档](DOCKER_IMAGES_CN.md)获取镜像,这里以 `cuda9.0-cudnn7` 的镜像为例: - -```shell -nvidia-docker pull hub.baidubce.com/paddlepaddle/serving:latest-cuda9.0-cudnn7 -``` +## GPU 版本 ### 创建容器并进入 @@ -107,74 +43,19 @@ nvidia-docker pull hub.baidubce.com/paddlepaddle/serving:latest-cuda9.0-cudnn7 nvidia-docker run -p 9292:9292 --name test -dit hub.baidubce.com/paddlepaddle/serving:latest-cuda9.0-cudnn7 nvidia-docker exec -it test bash ``` - -`-p`选项是为了将容器的`9292`端口映射到宿主机的`9292`端口。 - -### 安装PaddleServing - -为了减小镜像的体积,镜像中没有安装Serving包,要执行下面命令进行安装。 - -```bash -pip install paddle-serving-server-gpu -``` - -您可能需要使用国内镜像源(例如清华源)来加速下载。 - -```shell -pip install paddle-serving-server-gpu -i https://pypi.tuna.tsinghua.edu.cn/simple -``` - -### 测试example - -在运行GPU版Server时需要通过`--gpu_ids`选项设置预测服务使用的GPU,缺省状态默认使用CPU。当设置的`--gpu_ids`超出环境变量`CUDA_VISIBLE_DEVICES`时会报错。下面的示例为指定使用索引为0的GPU: -```shell -export CUDA_VISIBLE_DEVICES=0,1 -python -m paddle_serving_server_gpu.serve --model uci_housing_model --port 9292 --gpu_ids 0 -``` - - -通过下面命令获取训练好的Boston房价预估模型: - +或者 ```bash -wget --no-check-certificate https://paddle-serving.bj.bcebos.com/uci_housing.tar.gz -tar -xzf uci_housing.tar.gz +docker run --gpus all -p 9292:9292 --name test -dit hub.baidubce.com/paddlepaddle/serving:latest-cuda9.0-cudnn7 +docker exec -it test bash ``` -- 测试HTTP服务 - - 在Server端(容器内)运行: - - ```bash - python -m paddle_serving_server_gpu.serve --model uci_housing_model --thread 10 --port 9292 --name uci --gpu_ids 0 - ``` - - 在Client端(容器内或容器外)运行: - - ```bash - curl -H "Content-Type:application/json" -X POST -d '{"feed":{"x": [0.0137, -0.1136, 0.2553, -0.0692, 0.0582, -0.0727, -0.1583, -0.0584, 0.6283, 0.4919, 0.1856, 0.0795, -0.0332]}, "fetch":["price"]}' http://127.0.0.1:9292/uci/prediction - ``` - -- 测试RPC服务 - - 在Server端(容器内)运行: +`-p`选项是为了将容器的`9292`端口映射到宿主机的`9292`端口。 - ```bash - python -m paddle_serving_server_gpu.serve --model uci_housing_model --thread 10 --port 9292 --gpu_ids 0 - ``` +### 安装PaddleServing - 在Client端(容器内或容器外,需要安装`paddle-serving-client`包)运行下面Python代码: +镜像里自带对应镜像tag版本的`paddle_serving_server_gpu`,`paddle_serving_client`,`paddle_serving_app`,如果用户不需要更改版本,可以直接使用,适用于没有外网服务的环境。 - ```bash - from paddle_serving_client import Client - - client = Client() - client.load_client_config("uci_housing_client/serving_client_conf.prototxt") - client.connect(["127.0.0.1:9292"]) - data = [0.0137, -0.1136, 0.2553, -0.0692, 0.0582, -0.0727, - -0.1583, -0.0584, 0.6283, 0.4919, 0.1856, 0.0795, -0.0332] - fetch_map = client.predict(feed={"x": data}, fetch=["price"]) - print(fetch_map) - ``` +如果需要更换版本,请参照首页的指导,下载对应版本的pip包。 ## 注意事项 diff --git a/doc/WINDOWS_TUTORIAL.md b/doc/WINDOWS_TUTORIAL.md index 1fc194e728890f6541f484d50fbc1a0ee324ac29..af44d08af1137bd038646d43a182dc25d0505b1c 100644 --- a/doc/WINDOWS_TUTORIAL.md +++ b/doc/WINDOWS_TUTORIAL.md @@ -8,7 +8,7 @@ This document guides users how to build Paddle Serving service on the Windows pl ### Running Paddle Serving on Native Windows System -**Configure Python environment variables to PATH**: First, you need to add the directory where the Python executable program is located to the PATH. Usually in **System Properties/My Computer Properties**-**Advanced**-**Environment Variables**, click Path and add the path at the beginning. For example, `C:\Users\$USER\AppData\Local\Programs\Python\Python36`, and finally click **OK** continuously. If you enter python on Powershell, you can enter the python interactive interface, indicating that the environment variable configuration is successful. +**Configure Python environment variables to PATH**: **We only support Python 3.5+ on Native Windows System.**. First, you need to add the directory where the Python executable program is located to the PATH. Usually in **System Properties/My Computer Properties**-**Advanced**-**Environment Variables**, click Path and add the path at the beginning. For example, `C:\Users\$USER\AppData\Local\Programs\Python\Python36`, and finally click **OK** continuously. If you enter python on Powershell, you can enter the python interactive interface, indicating that the environment variable configuration is successful. **Install wget**: Because all the downloads in the tutorial and the built-in model download function in `paddle_serving_app` all use the wget tool, download the binary package at the [link](http://gnuwin32.sourceforge.net/packages/wget.htm), unzip and copy it to `C:\Windows\System32`, if there is a security prompt, you need to pass it. diff --git a/doc/WINDOWS_TUTORIAL_CN.md b/doc/WINDOWS_TUTORIAL_CN.md index 6f43e7dd1910761f8332d01fea05171156985317..3fba94943222dcf111bce3d1f171a9df39b4fc9d 100644 --- a/doc/WINDOWS_TUTORIAL_CN.md +++ b/doc/WINDOWS_TUTORIAL_CN.md @@ -8,7 +8,7 @@ ### 原生Windows系统运行Paddle Serving -**配置Python环境变量到PATH**:首先需要将Python的可执行程序所在目录加入到PATH当中。通常在**系统属性/我的电脑属性**-**高级**-**环境变量** ,点选Path,并在开头加上路径。例如`C:\Users\$USER\AppData\Local\Programs\Python\Python36`,最后连续点击**确定** 。在Powershell上如果输入python可以进入python交互界面,说明环境变量配置成功。 +**配置Python环境变量到PATH**:**目前原生Windows仅支持Python 3.5或更高版本**。首先需要将Python的可执行程序所在目录加入到PATH当中。通常在**系统属性/我的电脑属性**-**高级**-**环境变量** ,点选Path,并在开头加上路径。例如`C:\Users\$USER\AppData\Local\Programs\Python\Python36`,最后连续点击**确定** 。在Powershell上如果输入python可以进入python交互界面,说明环境变量配置成功。 **安装wget工具**:由于教程当中所有的下载,以及`paddle_serving_app`当中内嵌的模型下载功能,都是用到wget工具,在链接[下载wget](http://gnuwin32.sourceforge.net/packages/wget.htm),解压后复制到`C:\Windows\System32`下,如有安全提示需要通过。 diff --git a/java/README.md b/java/README.md index aac68283ae326923637804b879d93770374571ca..8e9b780e527dccd417c01bb3275db0fefce99062 100644 --- a/java/README.md +++ b/java/README.md @@ -1,6 +1,24 @@ -## Java Demo +## Tutorial of Java Client for Paddle Serving + +(English|[简体中文](./README_CN.md)) + +### Development Environment + +In order to facilitate users to use java for development, we provide the compiled Serving project to be placed in the java mirror. The way to get the mirror and enter the development environment is + +``` +docker pull hub.baidubce.com/paddlepaddle/serving:0.4.0-java +docker run --rm -dit --name java_serving hub.baidubce.com/paddlepaddle/serving:0.4.0-java +docker exec -it java_serving bash +cd Serving/java +``` + +The Serving folder is at the develop branch when the docker image is generated. You need to git pull to the latest version or git checkout to the desired branch. + +### Install client dependencies + +Due to the large number of dependent libraries, the image has been compiled once at the time of generation, and the user can perform the following operations -### Install package ``` mvn compile mvn install @@ -9,18 +27,49 @@ mvn compile mvn install ``` -### Start Server +### Start the server -take the fit_a_line demo as example +Take the fit_a_line model as an example, the server starts ``` - python -m paddle_serving_server.serve --model uci_housing_model --thread 10 --port 9393 --use_multilang #CPU -python -m paddle_serving_server_gpu.serve --model uci_housing_model --thread 10 --port 9393 --use_multilang #GPU +cd ../../python/examples/fit_a_line +sh get_data.sh +python -m paddle_serving_server.serve --model uci_housing_model --thread 10 --port 9393 --use_multilang & ``` -### Client Predict +Client prediction + ``` +cd ../../../java/examples/target java -cp paddle-serving-sdk-java-examples-0.0.1-jar-with-dependencies.jar PaddleServingClientExample fit_a_line ``` -The Java example also contains the prediction client of Bert, Model_enaemble, asyn_predict, batch_predict, Cube_local, Cube_quant, and Yolov4 models. +Take yolov4 as an example, the server starts + +``` +python -m paddle_serving_app.package --get_model yolov4 +tar -xzvf yolov4.tar.gz +python -m paddle_serving_server_gpu.serve --model yolov4_model --port 9393 --gpu_ids 0 --use_multilang & #It needs to be executed in GPU Docker, otherwise the execution method of CPU must be used. +``` + +Client prediction + +``` +# in /Serving/java/examples/target +java -cp paddle-serving-sdk-java-examples-0.0.1-jar-with-dependencies.jar PaddleServingClientExample yolov4 ../../../python/examples/yolov4/000000570688.jpg +# The case of yolov4 needs to specify a picture as input +``` + +### Customization guidance + +The above example is running in CPU mode. If GPU mode is required, there are two options. + +The first is that GPU Serving and Java Client are in the same image. After starting the corresponding image, the user needs to move /Serving/java in the java image to the corresponding image. + +The second is to deploy GPU Serving and Java Client separately. If they are on the same host, you can learn the IP address of the corresponding container through ifconfig, and then when you connect to client.connect in `examples/src/main/java/PaddleServingClientExample.java` Make changes to the endpoint, and then compile it again. Or select `--net=host` to bind the network device of docker and host when docker starts, so that it can run directly without customizing java code. + +**It should be noted that in the example, all models need to use `--use_multilang` to start GRPC multi-programming language support, and the port number is 9393. If you need another port, you need to modify it in the java file** + +**Currently Serving has launched the Pipeline mode (see [Pipeline Serving](../doc/PIPELINE_SERVING.md) for details). The next version (0.4.1) of the Pipeline Serving Client for Java will be released. ** + + diff --git a/java/README_CN.md b/java/README_CN.md index a068e8ecf47842fa57e41808b66f0a4017148d50..05f3c6039172955213271213da366a8a831c5605 100644 --- a/java/README_CN.md +++ b/java/README_CN.md @@ -1,6 +1,24 @@ -## Java 示例 +## 用于Paddle Serving的Java客户端 + +([English](./README.md)|简体中文) + +### 开发环境 + +为了方便用户使用java进行开发,我们提供了编译好的Serving工程放置在java镜像当中,获取镜像并进入开发环境的方式是 + +``` +docker pull hub.baidubce.com/paddlepaddle/serving:0.4.0-java +docker run --rm -dit --name java_serving hub.baidubce.com/paddlepaddle/serving:0.4.0-java +docker exec -it java_serving bash +cd Serving/java +``` + +Serving文件夹是镜像生成时的develop分支工程目录,需要git pull 到最新版本,或者git checkout 到想要的分支。 ### 安装客户端依赖 + +由于依赖库数量庞大,因此镜像已经在生成时编译过一次,用户执行以下操作即可 + ``` mvn compile mvn install @@ -11,16 +29,47 @@ mvn install ### 启动服务端 -以fit_a_line模型为例 +以fit_a_line模型为例,服务端启动 ``` - python -m paddle_serving_server.serve --model uci_housing_model --thread 10 --port 9393 --use_multilang #CPU -python -m paddle_serving_server_gpu.serve --model uci_housing_model --thread 10 --port 9393 --use_multilang #GPU +cd ../../python/examples/fit_a_line +sh get_data.sh +python -m paddle_serving_server.serve --model uci_housing_model --thread 10 --port 9393 --use_multilang & ``` -### 客户端预测 +客户端预测 + ``` +cd ../../../java/examples/target java -cp paddle-serving-sdk-java-examples-0.0.1-jar-with-dependencies.jar PaddleServingClientExample fit_a_line ``` -java示例中还包含了bert、model_enaemble、asyn_predict、batch_predict、cube_local、cube_quant、yolov4模型的预测客户端。 +以yolov4为例子,服务端启动 + +``` +python -m paddle_serving_app.package --get_model yolov4 +tar -xzvf yolov4.tar.gz +python -m paddle_serving_server_gpu.serve --model yolov4_model --port 9393 --gpu_ids 0 --use_multilang & #需要在GPU Docker当中执行,否则要使用CPU的执行方式。 +``` + +客户端预测 + +``` +# in /Serving/java/examples/target +java -cp paddle-serving-sdk-java-examples-0.0.1-jar-with-dependencies.jar PaddleServingClientExample yolov4 ../../../python/examples/yolov4/000000570688.jpg +# yolov4的案例需要指定一个图片作为输入 +``` + +### 二次开发指导 + +上述示例是在CPU模式下运行,如果需要GPU模式,可以有两种选择。 + +第一种是GPU Serving和Java Client在同一个镜像,需要用户在启动对应的镜像后,把java镜像当中的/Serving/java移动到对应的镜像中。 + +第二种是GPU Serving和Java Client分开部署,如果在同一台宿主机,可以通过ifconfig了解对应容器的IP地址,然后在`examples/src/main/java/PaddleServingClientExample.java`当中对client.connect时的endpoint做修改,然后再编译一次。 或者在docker启动时选择 `--net=host`来绑定docker和宿主机的网络设备,这样不需要定制java代码可以直接运行。 + +**需要注意的是,在示例中,所有模型都需要使用`--use_multilang`来启动GRPC多编程语言支持,以及端口号都是9393,如果需要别的端口,需要在java文件里修改** + +**目前Serving已推出Pipeline模式(详见[Pipeline Serving](../doc/PIPELINE_SERVING_CN.md)),下个版本(0.4.1)面向Java的Pipeline Serving Client将会发布,敬请期待。** + +