diff --git a/README.md b/README.md index ad66e52a593b0c09a7abe0a75055886761f2ceec..602566a2babbe78d5cfe9595431a9d49a65ec303 100755 --- a/README.md +++ b/README.md @@ -11,7 +11,7 @@ Build Status Docs - Release + Release Python License Forks @@ -116,13 +116,13 @@ AND GENERATION](https://arxiv.org/pdf/2112.12731.pdf)

模型库

-Paddle Serving与Paddle模型套件紧密配合，实现大量服务化部署，包括图像分类、物体检测、语言文本识别、中文词性、情感分析、内容推荐等多种类型示例，以及Paddle全链条项目，共计47个模型。 +Paddle Serving与Paddle模型套件紧密配合，实现大量服务化部署，包括图像分类、物体检测、语言文本识别、中文词性、情感分析、内容推荐等多种类型示例，以及Paddle全链条项目，共计46个模型。

-| PaddleOCR | PaddleDetection | PaddleClas | PaddleSeg | PaddleRec | Paddle NLP | Paddle Video | -| :----: | :----: | :----: | :----: | :----: | :----: | :----: | -| 8 | 12 | 14 | 2 | 3 | 7 | 1 | +| 图像分类与识别 | NLP | 推荐系统 | 人脸识别 | 目标检测 | 文字识别 | 图像分割 | 关键点检测 | 视频理解 | +| :----: | :----: | :----: | :----: | :----: | :----: | :----: | :----: | :----: | +| 14 | 6 | 3 | 1 | 10 | 8 | 2 | 1 | 1 |

diff --git a/README_CN.md b/README_CN.md index 0e9d0da59ff28d507a022cfda22b173d4c9f797f..0bc5bf5e91d9a7ef82610bcf6efa5d1d7c8d3cae 100755 --- a/README_CN.md +++ b/README_CN.md @@ -11,7 +11,7 @@ Build Status Docs - Release + Release Python License Forks @@ -125,9 +125,9 @@ Paddle Serving works closely with the Paddle model suite, and implements a large

-| PaddleOCR | PaddleDetection | PaddleClas | PaddleSeg | PaddleRec | Paddle NLP | Paddle Video | -| :----: | :----: | :----: | :----: | :----: | :----: | :----: | -| 8 | 12 | 14 | 2 | 3 | 6 | 1| +| Image Classification & Recognition | NLP | Recommend | Face Recognition | Object Detection | OCR | Image segmentation | Keypoint Detection | Video| +| :----: | :----: | :----: | :----: | :----: | :----: | :----: | :----: | :----: | +| 14 | 6 | 3 | 1 | 10 | 8 | 2 | 1 | 1 |

diff --git a/doc/Install_CN.md b/doc/Install_CN.md index 166c50f0fce4bc1a72ee7c569da34cb08b9550c8..a9b1c7462a946228071f107b0d1f993131ad14e4 100644 --- a/doc/Install_CN.md +++ b/doc/Install_CN.md @@ -2,15 +2,39 @@ (简体中文|[English](./Install_EN.md)) +- [1.使用开发镜像](#1) + - [Serving 开发镜像](#1.1) + - [Paddle 开发镜像](#1.2) +- [2.安装 Wheel 包](#2) + - [在线安装](#2.1) + - [离线安装](#2.2) +- [3.环境检查](#3) + + **强烈建议**您在**Docker内构建**Paddle Serving，更多镜像请查看[Docker镜像列表](Docker_Images_CN.md)。 **提示-1**：本项目仅支持**Python3.6/3.7/3.8/3.9**，接下来所有的与Python/Pip相关的操作都需要选择正确的Python版本。 **提示-2**：以下示例中GPU环境均为cuda11.2-cudnn8，如果您使用Python Pipeline来部署，并需要Nvidia TensorRT来优化预测性能，请参考[支持的镜像环境和说明](#4支持的镜像环境和说明)来选择其他版本。 + + +## 1.使用开发镜像 -## 1.启动开发镜像 **同时支持使用Serving镜像和Paddle镜像，1.1和1.2章节中的操作2选1即可。** 在Paddle docker镜像上部署Serving服务需要安装额外依赖库，因此，我们直接使用Serving开发镜像。 + +| 环境 | Serving开发镜像Tag | 操作系统 | Paddle开发镜像Tag | 操作系统 | +| :--------------------------: | :-------------------------------: | :-------------: | :-------------------: | :----------------: | +| CPU | 0.9.0-devel | Ubuntu 16.04 | 2.3.0 | Ubuntu 18.04. | +| CUDA10.1 + CUDNN7 | 0.9.0-cuda10.1-cudnn7-devel | Ubuntu 16.04 | 无 | 无 | +| CUDA10.2 + CUDNN8 | 0.9.0-cuda10.2-cudnn8-devel | Ubuntu 16.04 | 无 | Ubuntu 18.04 | +| CUDA11.2 + CUDNN8 | 0.9.0-cuda11.2-cudnn8-devel | Ubuntu 16.04 | 2.3.0-gpu-cuda11.2-cudnn8 | Ubuntu 18.04 | + +对于**Windows 10 用户**，请参考文档[Windows平台使用Paddle Serving指导](Windows_Tutorial_CN.md)。 + + + + ### 1.1 Serving开发镜像（CPU/GPU 2选1） **CPU：** ``` @@ -28,31 +52,37 @@ nvidia-docker run -p 9292:9292 --name test -dit registry.baidubce.com/paddlepadd nvidia-docker exec -it test bash git clone https://github.com/PaddlePaddle/Serving ``` + + + ### 1.2 Paddle开发镜像（CPU/GPU 2选1） **CPU：** ``` -# 启动 CPU Docker +### 启动 CPU Docker nvidia-docker pull registry.baidubce.com/paddlepaddle/paddle:2.3.0 docker run -p 9292:9292 --name test -dit registry.baidubce.com/paddlepaddle/paddle:2.3.0 bash docker exec -it test bash git clone https://github.com/PaddlePaddle/Serving -# Paddle开发镜像需要执行以下脚本增加Serving所需依赖项 +### Paddle开发镜像需要执行以下脚本增加Serving所需依赖项 bash Serving/tools/paddle_env_install.sh ``` **GPU：** ``` -# 启动 GPU Docker +### 启动 GPU Docker nvidia-docker pull registry.baidubce.com/paddlepaddle/paddle:2.3.0-gpu-cuda11.2-cudnn8 nvidia-docker run -p 9292:9292 --name test -dit registry.baidubce.com/paddlepaddle/paddle:2.3.0-gpu-cuda11.2-cudnn8 bash nvidia-docker exec -it test bash git clone https://github.com/PaddlePaddle/Serving -# Paddle开发镜像需要执行以下脚本增加Serving所需依赖项 +### Paddle开发镜像需要执行以下脚本增加Serving所需依赖项 bash Serving/tools/paddle_env_install.sh ``` -## 2.安装Paddle Serving稳定wheel包 + + + +## 2.安装 Wheel 包 安装所需的pip依赖 ``` @@ -66,6 +96,9 @@ pip3 install -r python/requirements.txt - post102 = CUDA10.2 + cuDNN7 + TensorRT6 (与Paddle 镜像一致) - post1028 = CUDA10.2 + cuDNN8 + TensorRT7 + + +### 2.1 在线安装 ```shell pip3 install paddle-serving-client==0.9.0 -i https://pypi.tuna.tsinghua.edu.cn/simple @@ -82,11 +115,10 @@ pip3 install paddle-serving-server-gpu==0.9.0.post112 -i https://pypi.tuna.tsing 如果需要使用develop分支编译的安装包，请从[下载Wheel包](./Latest_Packages_CN.md)中获取下载地址进行下载，使用`pip install`命令进行安装。如果您想自行编译，请参照[Paddle Serving编译文档](./Compile_CN.md)。 -paddle-serving-server和paddle-serving-server-gpu安装包支持Centos 6/7, Ubuntu 16/18和Windows 10。 +`paddle-serving-server` 和 `paddle-serving-server-gpu` 安装包支持Centos 6/7, Ubuntu 16/18和Windows 10。 -paddle-serving-client和paddle-serving-app安装包支持Linux和Windows，其中paddle-serving-client仅支持python3.6/3.7/3.8/3.9。 +`paddle-serving-client` 和 `paddle-serving-app` 安装包支持 Linux 和 Windows，其中 `paddle-serving-client` 仅支持 python3.6/3.7/3.8/3.9。 -## 3.安装Paddle相关Python库 **当您使用`paddle_serving_client.convert`命令或者`Python Pipeline框架`时才需要安装。** ``` @@ -119,17 +151,70 @@ pip3 install https://paddle-inference-lib.bj.bcebos.com/2.3.0/python/Linux/GPU/x ``` -## 4.支持的镜像环境和说明 -| 环境 | Serving开发镜像Tag | 操作系统 | Paddle开发镜像Tag | 操作系统 | -| :--------------------------: | :-------------------------------: | :-------------: | :-------------------: | :----------------: | -| CPU | 0.9.0-devel | Ubuntu 16.04 | 2.3.0 | Ubuntu 18.04. | -| CUDA10.1 + CUDNN7 | 0.9.0-cuda10.1-cudnn7-devel | Ubuntu 16.04 | 无 | 无 | -| CUDA10.2 + CUDNN8 | 0.9.0-cuda10.2-cudnn8-devel | Ubuntu 16.04 | 无 | Ubuntu 18.04 | -| CUDA11.2 + CUDNN8 | 0.9.0-cuda11.2-cudnn8-devel | Ubuntu 16.04 | 2.3.0-gpu-cuda11.2-cudnn8 | Ubuntu 18.04 | + -对于**Windows 10 用户**，请参考文档[Windows平台使用Paddle Serving指导](Windows_Tutorial_CN.md)。 +### 2.2 离线安装 + +**1.安装离线 Wheel 包** + +Serving 和 Paddle Wheel包的离线依赖包下载在 `py3x_offline_whls` 目录下 `serving_dependent_wheels/` 和 `paddle_dependent_wheels/`。 + +通过运行 `install.py` 脚本可本地安装 Serving 和 Paddle Wheel 包。`install.py` 脚本的参数列表如下： +``` +python3 install.py + --python_version : Python version for installing wheels, one of [py36, py37, py38, py39], py37 default. + --device : Type of devices, one of [cpu, gpu], cpu default. + --cuda_version : CUDA version for GPU, one of [101, 102, 112, empty], empty default. + --serving_version : Verson of Serving, one of [0.8.3, no_install], 0.8.3 default. + --paddle_version Verson of Paddle, one of [2.2.2, no_install], 2.2.2 default. +``` + +**2.在环境变量中指定 `SERVING_BIN` 路径** + +完成第1步安装后，若仅使用 python pipeline 模式可忽略此步骤。 + +如使用 C++ Serving 使用命令行方式启动服务，示例如下。则需要在命令行窗口或服务启动程序中导出环境变量 `SERVING_BIN`，使用本地的 serving 二进制程序运行服务。 + +C++ Serving 命令行启动服务示例： +``` +python3 -m paddle_serving_server.serve --model serving_model --thread 10 --port 9292 --gpu_ids 0,1,2 +``` + +由于所有版本的二进制程序包有 20 GB，非常大。因此提供多个版本的下载链接，通过手动 `wget` 下载指定版本到 `serving_bin` 目录下，解压后导出到环境变量中。 + +- cpu-avx-mkl: https://paddle-serving.bj.bcebos.com/test-dev/bin/serving-cpu-avx-mkl-0.8.3.tar.gz +- cpu-avx-openblas: https://paddle-serving.bj.bcebos.com/test-dev/bin/serving-cpu-avx-openblas-0.8.3.tar.gz +- cpu-noavx-openblas: https://paddle-serving.bj.bcebos.com/test-dev/bin/serving-cpu-noavx-openblas-0.8.3.tar.gz +- cuda10.1-cudnn7-TensorRT6: https://paddle-serving.bj.bcebos.com/test-dev/bin/serving-gpu-101-0.8.3.tar.gz +- cuda10.2-cudnn7-TensorRT6: https://paddle-serving.bj.bcebos.com/test-dev/bin/serving-gpu-102-0.8.3.tar.gz +- cuda10.2-cudnn8-TensorRT7: https://paddle-serving.bj.bcebos.com/test-dev/bin/serving-gpu-1028-0.8.3.tar.gz +- cuda11.2-cudnn8-TensorRT8: https://paddle-serving.bj.bcebos.com/test-dev/bin/serving-gpu-112-0.8.3.tar.gz + +以 GPU CUDA 10.2 为例，在命令行或启动程序中设置环境变量如下： +``` +export SERVING_BIN = $PWD/serving_bin/serving-gpu-102-0.8.3/serving +``` + +**3.运行 `install.py` 安装 Wheel 包** + +1.同时安装 Serving 和 Paddle 的 py38 版本 GPU wheel 包： +``` +python3 install.py --cuda_version="102" --python_version="py38" --device="GPU" --serving_version="0.8.3" --paddle_version="2.2.2" +``` + +2.仅安装 Serving 的 py39 版本 CPU wheel 包，设置 `--paddle_version="no_install"` 表示不安装 Paddle 预测库，设置 `--device="cpu"` 表示 cpu 版本 +``` +python3 install.py --cuda_version="" --python_version="py39" --device="cpu" --serving_version="0.8.3" --paddle_version="no_install" +``` + +3.仅安装 Paddle 的 py36 版本`cuda=11.2` 的 GPU wheel 包， +``` +python3 install.py --cuda_version="112" --python_version="py36" --device="GPU" --serving_version="no_install" --paddle_version="2.2.2" +``` + + -## 5.安装完成后的环境检查 +## 3.环境检查 当以上步骤均完成后可使用命令行运行环境检查功能，自动运行Paddle Serving相关示例，进行环境相关配置校验。 ``` python3 -m paddle_serving_server.serve check diff --git a/doc/Install_EN.md b/doc/Install_EN.md index 44ee29ab7e8a34371db4868fddbdb981cb896ca6..2c68a9554ec33c3c7c6c1d6afd2106eae521477c 100644 --- a/doc/Install_EN.md +++ b/doc/Install_EN.md @@ -2,16 +2,39 @@ ([简体中文](./Install_CN.md)|English) +- [1.Use devel docker](#1) + - [Serving devel images](#1.1) + - [Paddle devel images](#1.2) +- [2.Install Wheel Packages](#2) + - [Online Install](#2.1) + - [Offline Install](#2.2) +- [3.Installation Check](#3) + **Strongly recommend** you build **Paddle Serving** in Docker. For more images, please refer to [Docker Image List](Docker_Images_CN.md). **Tip-1**: This project only supports **Python3.6/3.7/3.8/3.9**, all subsequent operations related to Python/Pip need to select the correct Python version. **Tip-2**: The GPU environments in the following examples are all cuda11.2-cudnn8. If you use Python Pipeline to deploy and need Nvidia TensorRT to optimize prediction performance, please refer to [Supported Mirroring Environment and Instructions](#4.-Supported-Docker-Images-and-Instruction) to choose other versions. -## 1. Start the Docker Container + + +## 1.Use devel docker **Both Serving Dev Image and Paddle Dev Image are supported at the same time. You can choose 1 from the operation 2 in chapters 1.1 and 1.2.**Deploying the Serving service on the Paddle docker image requires the installation of additional dependency libraries. Therefore, we directly use the Serving development image. -### 1.1 Serving Dev Images (CPU/GPU 2 choose 1) +| Environment | Serving Development Image Tag | Operating System | Paddle Development Image Tag | Operating System | +| :--------------------------: | :-------------------------------: | :-------------: | :-------------------: | :----------------: | +| CPU | 0.9.0-devel | Ubuntu 16.04 | 2.3.0 | Ubuntu 18.04. | +| CUDA10.1 + CUDNN7 | 0.9.0-cuda10.1-cudnn7-devel | Ubuntu 16.04 | 无 | 无 | +| CUDA10.2 + CUDNN8 | 0.9.0-cuda10.2-cudnn8-devel | Ubuntu 16.04 | 无 | Ubuntu 18.04 | +| CUDA11.2 + CUDNN8 | 0.9.0-cuda11.2-cudnn8-devel | Ubuntu 16.04 | 2.3.0-gpu-cuda11.2-cudnn8 | Ubuntu 18.04 | + +For **Windows 10 users**, please refer to the document [Paddle Serving Guide for Windows Platform](Windows_Tutorial_CN.md). + + + + +### 1.1 Serving Devel Images (CPU/GPU 2 choose 1) + **CPU:** ``` # Start CPU Docker Container @@ -28,7 +51,10 @@ nvidia-docker run -p 9292:9292 --name test -dit docker pull registry.baidubce.co nvidia-docker exec -it test bash git clone https://github.com/PaddlePaddle/Serving ``` -### 1.2 Paddle Dev Images (choose any codeblock of CPU/GPU) + + + +### 1.2 Paddle Devel Images (choose any codeblock of CPU/GPU) **CPU:** ``` # Start CPU Docker Container @@ -52,7 +78,9 @@ git clone https://github.com/PaddlePaddle/Serving bash Serving/tools/paddle_env_install.sh ``` -## 2. Install Paddle Serving stable wheel packages + + +## 2. Install wheel packages Install the required pip dependencies ``` @@ -63,8 +91,12 @@ pip3 install -r python/requirements.txt Install the service whl package. There are three types of client, app and server. The server is divided into CPU and GPU. Choose one installation according to the environment. - post112 = CUDA11.2 + cuDNN8 + TensorRT8（Recommanded） - post101 = CUDA10.1 + cuDNN7 + TensorRT6 -- post102 = CUDA10.2 + cuDNN8 + TensorRT7 +- post102 = CUDA10.2 + cuDNN7 + TensorRT6 (与Paddle 镜像一致 +- post1028 = CUDA10.2 + cuDNN8 + TensorRT7 + + +### 2.1 Online Install ```shell pip3 install paddle-serving-client==0.9.0 -i https://pypi.tuna.tsinghua.edu.cn/simple @@ -87,7 +119,6 @@ The paddle-serving-server and paddle-serving-server-gpu installation packages su The paddle-serving-client and paddle-serving-app installation packages support Linux and Windows, and paddle-serving-client only supports python3.6/3.7/3.8/3.9. -## 3. Install Paddle related Python libraries **You only need to install it when you use the `paddle_serving_client.convert` command or the `Python Pipeline framework`. ** ``` # CPU environment please execute @@ -117,20 +148,63 @@ pip3 install https://paddle-inference-lib.bj.bcebos.com/2.3.0/python/Linux/GPU/x pip3 install https://paddle-inference-lib.bj.bcebos.com/2.3.0/python/Linux/GPU/x86-64_gcc8.2_avx_mkl_cuda10.2_cudnn8.1.1_trt7.2.3.4/paddlepaddle_gpu-2.3.0-cp38-cp38-linux_x86_64.whl pip3 install https://paddle-inference-lib.bj.bcebos.com/2.3.0/python/Linux/GPU/x86-64_gcc8.2_avx_mkl_cuda10.2_cudnn8.1.1_trt7.2.3.4/paddlepaddle_gpu-2.3.0-cp39-cp39-linux_x86_64.whl ``` + -## 4. Supported Docker Images and Instruction +### 2.2 Offline Install +**1.Install offline wheel packages** -| Environment | Serving Development Image Tag | Operating System | Paddle Development Image Tag | Operating System | -| :--------------------------: | :-------------------------------: | :-------------: | :-------------------: | :----------------: | -| CPU | 0.9.0-devel | Ubuntu 16.04 | 2.3.0 | Ubuntu 18.04. | -| CUDA10.1 + CUDNN7 | 0.9.0-cuda10.1-cudnn7-devel | Ubuntu 16.04 | 无 | 无 | -| CUDA10.2 + CUDNN8 | 0.9.0-cuda10.2-cudnn8-devel | Ubuntu 16.04 | 无 | Ubuntu 18.04 | -| CUDA11.2 + CUDNN8 | 0.9.0-cuda11.2-cudnn8-devel | Ubuntu 16.04 | 2.3.0-gpu-cuda11.2-cudnn8 | Ubuntu 18.04 | +The independent dependencies of the Serving and Paddle Wheel packages are downloaded in `serving_dependent_wheels/` and `paddle_dependent_wheels/` under the `py3x_offline_whls` directory. -For **Windows 10 users**, please refer to the document [Paddle Serving Guide for Windows Platform](Windows_Tutorial_CN.md). +The Serving and Paddle Wheel packages can be installed locally by running the `install.py` script. The parameter list for the `install.py` script is as follows: +``` +python3 install.py + --python_version : Python version for installing wheels, one of [py36, py37, py38, py39], py37 default. + --device : Type of devices, one of [cpu, gpu], cpu default. + --cuda_version : CUDA version for GPU, one of [101, 102, 112, empty], empty default. + --serving_version : Verson of Serving, one of [0.8.3, no_install], 0.8.3 default. + --paddle_version Verson of Paddle, one of [2.2.2, no_install], 2.2.2 default. +``` + +**2.Specify the `SERVING_BIN` path in the environment variable** +After completing step 1 of the installation, you can ignore this step if you only use the python pipeline mode. + +If you use C++ Serving to start the service using the command line, the example is as follows. Then you need to export the environment variable `SERVING_BIN` in the command line window or service launcher, and use the local serving binary to run the service. +C++ Serving command line start service example: +``` +python3 -m paddle_serving_server.serve --model serving_model --thread 10 --port 9292 --gpu_ids 0,1,2 +``` +Since the binary package for all versions has 20 GB, it is very large. Therefore, multiple versions of download links are provided. Manually `wget` downloads the specified version to the `serving_bin` directory, decompresses it and exports it to the environment variable. + +- cpu-avx-mkl: https://paddle-serving.bj.bcebos.com/test-dev/bin/serving-cpu-avx-mkl-0.8.3.tar.gz +- cpu-avx-openblas: https://paddle-serving.bj.bcebos.com/test-dev/bin/serving-cpu-avx-openblas-0.8.3.tar.gz +- cpu-noavx-openblas: https://paddle-serving.bj.bcebos.com/test-dev/bin/serving-cpu-noavx-openblas-0.8.3.tar.gz +- cuda10.1-cudnn7-TensorRT6: https://paddle-serving.bj.bcebos.com/test-dev/bin/serving-gpu-101-0.8.3.tar.gz +- cuda10.2-cudnn7-TensorRT6: https://paddle-serving.bj.bcebos.com/test-dev/bin/serving-gpu-102-0.8.3.tar.gz +- cuda10.2-cudnn8-TensorRT7: https://paddle-serving.bj.bcebos.com/test-dev/bin/serving-gpu-1028-0.8.3.tar.gz +- cuda11.2-cudnn8-TensorRT8: https://paddle-serving.bj.bcebos.com/test-dev/bin/serving-gpu-112-0.8.3.tar.gz + +Taking GPU CUDA 10.2 as an example, set the environment variables on the command line or in the launcher as follows: + +**3.Run `install.py` to install wheel packages** + +1. Install the py38 version GPU wheel package of Serving and Paddle at the same time: +``` +python3 install.py --cuda_version="102" --python_version="py38" --device="GPU" --serving_version="0.8.3" --paddle_version="2.2.2" +``` + +2.Only install the py39 version of the Serving CPU wheel package, set `--paddle_version="no_install"` to not install the Paddle prediction library, set `--device="cpu"` to indicate the cpu version +``` +python3 install.py --cuda_version="" --python_version="py39" --device="cpu" --serving_version="0.8.3" --paddle_version="no_install" +``` +3. Install only the GPU wheel package of Paddle's py36 version `cuda=11.2` +``` +python3 install.py --cuda_version="112" --python_version="py36" --device="GPU" --serving_version="no_install" --paddle_version="2.2.2" +``` + + -## 5.Installation Check +## 3.Installation Check When the above steps are completed, you can use the command line to run the environment check function to automatically run the Paddle Serving related examples to verify the environment-related configuration. ``` python3 -m paddle_serving_server.serve check