Skip to content

S
Serving

PaddlePaddle / Serving
大约 1 年 前同步成功

通知 186
Star 833
Fork 253
S
Serving
未验证 提交 dfd7f014 编写于 作者: T Thomas Young 提交者： GitHub
浏览文件
操作

Merge pull request #9 from PaddlePaddle/develop 

Develop
上级 b085c6e4 e8cfd69a
展开全部 显示空白变更内容
内联 并排
Showing with 309 addition and 163 deletion
README.md
浏览文件 @ dfd7f014
...@@ -13,7 +13,7 @@ ...@@ -13,7 +13,7 @@
<a href="https://travis-ci.com/PaddlePaddle/Serving"> <a href="https://travis-ci.com/PaddlePaddle/Serving">
<img alt="Build Status" src="https://img.shields.io/travis/com/PaddlePaddle/Serving/develop"> <img alt="Build Status" src="https://img.shields.io/travis/com/PaddlePaddle/Serving/develop">
</a> </a>
<img alt="Release" src="https://img.shields.io/badge/Release-0.0.3-yellowgreen"> <img alt="Release" src="https://img.shields.io/badge/Release-0.6.2-yellowgreen">
<img alt="Issues" src="https://img.shields.io/github/issues/PaddlePaddle/Serving"> <img alt="Issues" src="https://img.shields.io/github/issues/PaddlePaddle/Serving">
<img alt="License" src="https://img.shields.io/github/license/PaddlePaddle/Serving"> <img alt="License" src="https://img.shields.io/github/license/PaddlePaddle/Serving">
<img alt="Slack" src="https://img.shields.io/badge/Join-Slack-green"> <img alt="Slack" src="https://img.shields.io/badge/Join-Slack-green">
...@@ -86,15 +86,15 @@ We **highly recommend** you to **run Paddle Serving in Docker**, please visit [R ...@@ -86,15 +86,15 @@ We **highly recommend** you to **run Paddle Serving in Docker**, please visit [R
``` ```
# Run CPU Docker # Run CPU Docker
docker pull registry.baidubce.com/paddlepaddle/serving:0.6.0-devel docker pull registry.baidubce.com/paddlepaddle/serving:0.6.2-devel
docker run -p 9292:9292 --name test -dit registry.baidubce.com/paddlepaddle/serving:0.6.0-devel bash docker run -p 9292:9292 --name test -dit registry.baidubce.com/paddlepaddle/serving:0.6.2-devel bash
docker exec -it test bash docker exec -it test bash
git clone https://github.com/PaddlePaddle/Serving git clone https://github.com/PaddlePaddle/Serving
``` ```
``` ```
# Run GPU Docker # Run GPU Docker
nvidia-docker pull registry.baidubce.com/paddlepaddle/serving:0.6.0-cuda10.2-cudnn8-devel nvidia-docker pull registry.baidubce.com/paddlepaddle/serving:0.6.2-cuda10.2-cudnn8-devel
nvidia-docker run -p 9292:9292 --name test -dit registry.baidubce.com/paddlepaddle/serving:0.6.0-cuda10.2-cudnn8-devel bash nvidia-docker run -p 9292:9292 --name test -dit registry.baidubce.com/paddlepaddle/serving:0.6.2-cuda10.2-cudnn8-devel bash
nvidia-docker exec -it test bash nvidia-docker exec -it test bash
git clone https://github.com/PaddlePaddle/Serving git clone https://github.com/PaddlePaddle/Serving
``` ```
...@@ -105,13 +105,13 @@ pip3 install -r python/requirements.txt ...@@ -105,13 +105,13 @@ pip3 install -r python/requirements.txt
``` ```
```shell ```shell
pip3 install paddle-serving-client==0.6.0 pip3 install paddle-serving-client==0.6.2
pip3 install paddle-serving-server==0.6.0 # CPU pip3 install paddle-serving-server==0.6.2 # CPU
pip3 install paddle-serving-app==0.6.0 pip3 install paddle-serving-app==0.6.2
pip3 install paddle-serving-server-gpu==0.6.0.post102 #GPU with CUDA10.2 + TensorRT7 pip3 install paddle-serving-server-gpu==0.6.2.post102 #GPU with CUDA10.2 + TensorRT7
# DO NOT RUN ALL COMMANDS! check your GPU env and select the right one # DO NOT RUN ALL COMMANDS! check your GPU env and select the right one
pip3 install paddle-serving-server-gpu==0.6.0.post101 # GPU with CUDA10.1 + TensorRT6 pip3 install paddle-serving-server-gpu==0.6.2.post101 # GPU with CUDA10.1 + TensorRT6
pip3 install paddle-serving-server-gpu==0.6.0.post11 # GPU with CUDA10.1 + TensorRT7 pip3 install paddle-serving-server-gpu==0.6.2.post11 # GPU with CUDA10.1 + TensorRT7
``` ```
You may need to use a domestic mirror source (in China, you can use the Tsinghua mirror source, add `-i https://pypi.tuna.tsinghua.edu.cn/simple` to pip command) to speed up the download. You may need to use a domestic mirror source (in China, you can use the Tsinghua mirror source, add `-i https://pypi.tuna.tsinghua.edu.cn/simple` to pip command) to speed up the download.
...@@ -259,7 +259,7 @@ output ...@@ -259,7 +259,7 @@ output
### Developers ### Developers
- [How to deploy Paddle Serving on K8S?(Chinese)](doc/PADDLE_SERVING_ON_KUBERNETES.md) - [How to deploy Paddle Serving on K8S?(Chinese)](doc/PADDLE_SERVING_ON_KUBERNETES.md)
- [How to route Paddle Serving to secure endpoint?(Chinese)](doc/SERVIING_AUTH_DOCKER.md) - [How to route Paddle Serving to secure endpoint?(Chinese)](doc/SERVING_AUTH_DOCKER.md)
- [How to develop a new Web Service?](doc/NEW_WEB_SERVICE.md) - [How to develop a new Web Service?](doc/NEW_WEB_SERVICE.md)
- [Compile from source code](doc/COMPILE.md) - [Compile from source code](doc/COMPILE.md)
- [Develop Pipeline Serving](doc/PIPELINE_SERVING.md) - [Develop Pipeline Serving](doc/PIPELINE_SERVING.md)
......
README_CN.md
浏览文件 @ dfd7f014
...@@ -87,15 +87,15 @@ Paddle Serving开发者为您提供了简单易用的[AIStudio教程-Paddle Serv ...@@ -87,15 +87,15 @@ Paddle Serving开发者为您提供了简单易用的[AIStudio教程-Paddle Serv
``` ```
# 启动 CPU Docker # 启动 CPU Docker
docker pull registry.baidubce.com/paddlepaddle/serving:0.6.0-devel docker pull registry.baidubce.com/paddlepaddle/serving:0.6.2-devel
docker run -p 9292:9292 --name test -dit registry.baidubce.com/paddlepaddle/serving:0.6.0-devel bash docker run -p 9292:9292 --name test -dit registry.baidubce.com/paddlepaddle/serving:0.6.2-devel bash
docker exec -it test bash docker exec -it test bash
git clone https://github.com/PaddlePaddle/Serving git clone https://github.com/PaddlePaddle/Serving
``` ```
``` ```
# 启动 GPU Docker # 启动 GPU Docker
nvidia-docker pull registry.baidubce.com/paddlepaddle/serving:0.6.0-cuda10.2-cudnn8-devel nvidia-docker pull registry.baidubce.com/paddlepaddle/serving:0.6.2-cuda10.2-cudnn8-devel
nvidia-docker run -p 9292:9292 --name test -dit registry.baidubce.com/paddlepaddle/serving:0.6.0-cuda10.2-cudnn8-devel bash nvidia-docker run -p 9292:9292 --name test -dit registry.baidubce.com/paddlepaddle/serving:0.6.2-cuda10.2-cudnn8-devel bash
nvidia-docker exec -it test bash nvidia-docker exec -it test bash
git clone https://github.com/PaddlePaddle/Serving git clone https://github.com/PaddlePaddle/Serving
``` ```
...@@ -107,13 +107,13 @@ pip3 install -r python/requirements.txt ...@@ -107,13 +107,13 @@ pip3 install -r python/requirements.txt
``` ```
```shell ```shell
pip3 install paddle-serving-client==0.6.0 pip3 install paddle-serving-client==0.6.2
pip3 install paddle-serving-server==0.6.0 # CPU pip3 install paddle-serving-server==0.6.2 # CPU
pip3 install paddle-serving-app==0.6.0 pip3 install paddle-serving-app==0.6.2
pip3 install paddle-serving-server-gpu==0.6.0.post102 #GPU with CUDA10.2 + TensorRT7 pip3 install paddle-serving-server-gpu==0.6.2.post102 #GPU with CUDA10.2 + TensorRT7
# 其他GPU环境需要确认环境再选择执行哪一条 # 其他GPU环境需要确认环境再选择执行哪一条
pip3 install paddle-serving-server-gpu==0.6.0.post101 # GPU with CUDA10.1 + TensorRT6 pip3 install paddle-serving-server-gpu==0.6.2.post101 # GPU with CUDA10.1 + TensorRT6
pip3 install paddle-serving-server-gpu==0.6.0.post11 # GPU with CUDA10.1 + TensorRT7 pip3 install paddle-serving-server-gpu==0.6.2.post11 # GPU with CUDA10.1 + TensorRT7
``` ```
您可能需要使用国内镜像源（例如清华源, 在pip命令中添加`-i https://pypi.tuna.tsinghua.edu.cn/simple`）来加速下载。 您可能需要使用国内镜像源（例如清华源, 在pip命令中添加`-i https://pypi.tuna.tsinghua.edu.cn/simple`）来加速下载。
...@@ -124,7 +124,7 @@ paddle-serving-server和paddle-serving-server-gpu安装包支持Centos 6/7, Ubun ...@@ -124,7 +124,7 @@ paddle-serving-server和paddle-serving-server-gpu安装包支持Centos 6/7, Ubun
paddle-serving-client和paddle-serving-app安装包支持Linux和Windows，其中paddle-serving-client仅支持python3.6/3.7/3.8。 paddle-serving-client和paddle-serving-app安装包支持Linux和Windows，其中paddle-serving-client仅支持python3.6/3.7/3.8。
**最新的0.6.0的版本，已经不支持Cuda 9.0和Cuda 10.0，Python已不支持2.7和3.5。** **最新的0.6.2的版本，已经不支持Cuda 9.0和Cuda 10.0，Python已不支持2.7和3.5。**
推荐安装2.1.0及以上版本的paddle 推荐安装2.1.0及以上版本的paddle
...@@ -262,7 +262,7 @@ python3 pipeline_rpc_client.py ...@@ -262,7 +262,7 @@ python3 pipeline_rpc_client.py
- [如何编译PaddleServing?](doc/COMPILE_CN.md) - [如何编译PaddleServing?](doc/COMPILE_CN.md)
- [如何开发Pipeline?](doc/PIPELINE_SERVING_CN.md) - [如何开发Pipeline?](doc/PIPELINE_SERVING_CN.md)
- [如何在K8S集群上部署Paddle Serving?](doc/PADDLE_SERVING_ON_KUBERNETES.md) - [如何在K8S集群上部署Paddle Serving?](doc/PADDLE_SERVING_ON_KUBERNETES.md)
- [如何在Paddle Serving上部署安全网关?](doc/SERVIING_AUTH_DOCKER.md) - [如何在Paddle Serving上部署安全网关?](doc/SERVING_AUTH_DOCKER.md)
- [如何开发Pipeline?](doc/PIPELINE_SERVING_CN.md) - [如何开发Pipeline?](doc/PIPELINE_SERVING_CN.md)
- [如何使用uWSGI部署Web Service](doc/UWSGI_DEPLOY_CN.md) - [如何使用uWSGI部署Web Service](doc/UWSGI_DEPLOY_CN.md)
- [如何实现模型文件热加载](doc/HOT_LOADING_IN_SERVING_CN.md) - [如何实现模型文件热加载](doc/HOT_LOADING_IN_SERVING_CN.md)
......
doc/DOCKER_IMAGES.md
浏览文件 @ dfd7f014
...@@ -29,10 +29,12 @@ You can get images in two ways: ...@@ -29,10 +29,12 @@ You can get images in two ways:
Runtime images cannot be used for compilation. Runtime images cannot be used for compilation.
If you want to customize your Serving based on source code, use the version with the suffix - devel. If you want to customize your Serving based on source code, use the version with the suffix - devel.
**cuda10.1-cudnn7-gcc54 image is not ready, you should run from dockerfile if you need it.**
| Description | OS | TAG | Dockerfile | | Description | OS | TAG | Dockerfile |
| :----------------------------------------------------------: | :-----: | :--------------------------: | :----------------------------------------------------------: | | :----------------------------------------------------------: | :-----: | :--------------------------: | :----------------------------------------------------------: |
| CPU development | Ubuntu16 | latest-devel | [Dockerfile.devel](../tools/Dockerfile.devel) | | CPU development | Ubuntu16 | latest-devel | [Dockerfile.devel](../tools/Dockerfile.devel) |
| GPU (cuda10.1-cudnn7-tensorRT6-gcc54) development | Ubuntu16 | latest-cuda10.1-cudnn7-gcc54-devel | [Dockerfile.cuda10.1-cudnn7-gcc54.devel](../tools/Dockerfile.cuda10.1-cudnn7-gcc54.devel) | | GPU (cuda10.1-cudnn7-tensorRT6-gcc54) development | Ubuntu16 | latest-cuda10.1-cudnn7-gcc54-devel(not ready) | [Dockerfile.cuda10.1-cudnn7-gcc54.devel](../tools/Dockerfile.cuda10.1-cudnn7-gcc54.devel) |
| GPU (cuda10.1-cudnn7-tensorRT6) development | Ubuntu16 | latest-cuda10.1-cudnn7-devel | [Dockerfile.cuda10.1-cudnn7.devel](../tools/Dockerfile.cuda10.1-cudnn7.devel) | | GPU (cuda10.1-cudnn7-tensorRT6) development | Ubuntu16 | latest-cuda10.1-cudnn7-devel | [Dockerfile.cuda10.1-cudnn7.devel](../tools/Dockerfile.cuda10.1-cudnn7.devel) |
| GPU (cuda10.2-cudnn8-tensorRT7) development | Ubuntu16 | latest-cuda10.2-cudnn8-devel | [Dockerfile.cuda10.2-cudnn8.devel](../tools/Dockerfile.cuda10.2-cudnn8.devel) | | GPU (cuda10.2-cudnn8-tensorRT7) development | Ubuntu16 | latest-cuda10.2-cudnn8-devel | [Dockerfile.cuda10.2-cudnn8.devel](../tools/Dockerfile.cuda10.2-cudnn8.devel) |
| GPU (cuda11-cudnn8-tensorRT7) development | Ubuntu18 | latest-cuda11-cudnn8-devel | [Dockerfile.cuda11-cudnn8.devel](../tools/Dockerfile.cuda11-cudnn8.devel) | | GPU (cuda11-cudnn8-tensorRT7) development | Ubuntu18 | latest-cuda11-cudnn8-devel | [Dockerfile.cuda11-cudnn8.devel](../tools/Dockerfile.cuda11-cudnn8.devel) |
...@@ -62,18 +64,33 @@ Develop Images: ...@@ -62,18 +64,33 @@ Develop Images:
| Env | Version | Docker images tag | OS | Gcc Version | | Env | Version | Docker images tag | OS | Gcc Version |
|----------|---------|------------------------------|-----------|-------------| |----------|---------|------------------------------|-----------|-------------|
| CPU | >=0.5.0 | 0.6.0-devel | Ubuntu 16 | 8.2.0 | | CPU | >=0.5.0 | 0.6.2-devel | Ubuntu 16 | 8.2.0 |
| | <=0.4.0 | 0.4.0-devel | CentOS 7 | 4.8.5 | | | <=0.4.0 | 0.4.0-devel | CentOS 7 | 4.8.5 |
| Cuda10.1 | >=0.5.0 | 0.6.0-cuda10.1-cudnn7-devel | Ubuntu 16 | 8.2.0 | | Cuda10.1 | >=0.5.0 | 0.6.2-cuda10.1-cudnn7-devel | Ubuntu 16 | 8.2.0 |
| | 0.6.0 | 0.6.0-cuda10.1-cudnn7-gcc54-devel | Ubuntu 16 | 5.4.0 | | | 0.6.2 | 0.6.2-cuda10.1-cudnn7-gcc54-devel(not ready) | Ubuntu 16 | 5.4.0 |
| | <=0.4.0 | 0.6.0-cuda10.1-cudnn7-devel | CentOS 7 | 4.8.5 | | | <=0.4.0 | 0.6.2-cuda10.1-cudnn7-devel | CentOS 7 | 4.8.5 |
| Cuda10.2 | >=0.5.0 | 0.6.0-cuda10.2-cudnn8-devel | Ubuntu 16 | 8.2.0 | | Cuda10.2 | >=0.5.0 | 0.6.2-cuda10.2-cudnn8-devel | Ubuntu 16 | 8.2.0 |
| | <=0.4.0 | Nan | Nan | Nan | | | <=0.4.0 | Nan | Nan | Nan |
| Cuda11.0 | >=0.5.0 | 0.6.0-cuda11.0-cudnn8-devel | Ubuntu 18 | 8.2.0 | | Cuda11.0 | >=0.5.0 | 0.6.2-cuda11.0-cudnn8-devel | Ubuntu 18 | 8.2.0 |
| | <=0.4.0 | Nan | Nan | Nan | | | <=0.4.0 | Nan | Nan | Nan |
Running Images: Running Images:
Running Images is lighter than Develop Images, and Running Images are too many due to multiple combinations of python, device environment. If you want to know about it, plese check the document [Paddle Serving on Kubernetes.](PADDLE_SERVING_ON_KUBERNETES.md). Running Images is lighter than Develop Images, and Running Images are made up with serving whl and bin, but without develop tools like cmake because of lower image size. If you want to know about it, plese check the document [Paddle Serving on Kubernetes.](PADDLE_SERVING_ON_KUBERNETES.md).
| ENV | Python Version | Tag |
|------------------------------------------|----------------|-----------------------------|
| cpu | 3.6 | 0.6.2-py36-runtime |
| cpu | 3.7 | 0.6.2-py37-runtime |
| cpu | 3.8 | 0.6.2-py38-runtime |
| cuda-10.1 + cudnn-7.6.5 + tensorrt-6.0.1 | 3.6 | 0.6.2-cuda10.1-py36-runtime |
| cuda-10.1 + cudnn-7.6.5 + tensorrt-6.0.1 | 3.7 | 0.6.2-cuda10.1-py37-runtime |
| cuda-10.1 + cudnn-7.6.5 + tensorrt-6.0.1 | 3.8 | 0.6.2-cuda10.1-py38-runtime |
| cuda-10.2 + cudnn-8.2.0 + tensorrt-7.1.3 | 3.6 | 0.6.2-cuda10.2-py36-runtime |
| cuda-10.2 + cudnn-8.2.0 + tensorrt-7.1.3 | 3.7 | 0.6.2-cuda10.2-py37-runtime |
| cuda-10.2 + cudnn-8.2.0 + tensorrt-7.1.3 | 3.8 | 0.6.2-cuda10.2-py38-runtime |
| cuda-11 + cudnn-8.0.5 + tensorrt-7.1.3 | 3.6 | 0.6.2-cuda11-py36-runtime |
| cuda-11 + cudnn-8.0.5 + tensorrt-7.1.3 | 3.7 | 0.6.2-cuda11-py37-runtime |
| cuda-11 + cudnn-8.0.5 + tensorrt-7.1.3 | 3.8 | 0.6.2-cuda11-py38-runtime |
**Tips:** If you want to use CPU server and GPU server (version>=0.5.0) at the same time, you should check the gcc version, only Cuda10.1/10.2/11 can run with CPU server owing to the same gcc version(8.2). **Tips:** If you want to use CPU server and GPU server (version>=0.5.0) at the same time, you should check the gcc version, only Cuda10.1/10.2/11 can run with CPU server owing to the same gcc version(8.2).
doc/DOCKER_IMAGES_CN.md
浏览文件 @ dfd7f014
...@@ -31,11 +31,12 @@ ...@@ -31,11 +31,12 @@
若需要基于源代码二次开发编译，请使用后缀为-devel的版本。 若需要基于源代码二次开发编译，请使用后缀为-devel的版本。
**在TAG列，latest也可以替换成对应的版本号，例如0.5.0/0.4.1等，但需要注意的是，部分开发环境随着某个版本迭代才增加，因此并非所有环境都有对应的版本号可以使用。** **在TAG列，latest也可以替换成对应的版本号，例如0.5.0/0.4.1等，但需要注意的是，部分开发环境随着某个版本迭代才增加，因此并非所有环境都有对应的版本号可以使用。**
**cuda10.1-cudnn7-gcc54环境尚未同步到镜像仓库，如果您需要相关镜像请运行相关dockerfile**
| 镜像选择 | 操作系统 | TAG | Dockerfile | | 镜像选择 | 操作系统 | TAG | Dockerfile |
| :----------------------------------------------------------: | :-----: | :--------------------------: | :----------------------------------------------------------: | | :----------------------------------------------------------: | :-----: | :--------------------------: | :----------------------------------------------------------: |
| CPU development | Ubuntu16 | latest-devel | [Dockerfile.devel](../tools/Dockerfile.devel) | | CPU development | Ubuntu16 | latest-devel | [Dockerfile.devel](../tools/Dockerfile.devel) |
| GPU (cuda10.1-cudnn7-tensorRT6-gcc54) development | Ubuntu16 | latest-cuda10.1-cudnn7-gcc54-devel | [Dockerfile.cuda10.1-cudnn7-gcc54.devel](../tools/Dockerfile.cuda10.1-cudnn7-gcc54.devel) | | GPU (cuda10.1-cudnn7-tensorRT6-gcc54) development | Ubuntu16 | latest-cuda10.1-cudnn7-gcc54-devel (not ready) | [Dockerfile.cuda10.1-cudnn7-gcc54.devel](../tools/Dockerfile.cuda10.1-cudnn7-gcc54.devel) |
| GPU (cuda10.1-cudnn7-tensorRT6) development | Ubuntu16 | latest-cuda10.1-cudnn7-devel | [Dockerfile.cuda10.1-cudnn7.devel](../tools/Dockerfile.cuda10.1-cudnn7.devel) | | GPU (cuda10.1-cudnn7-tensorRT6) development | Ubuntu16 | latest-cuda10.1-cudnn7-devel | [Dockerfile.cuda10.1-cudnn7.devel](../tools/Dockerfile.cuda10.1-cudnn7.devel) |
| GPU (cuda10.2-cudnn8-tensorRT7) development | Ubuntu16 | latest-cuda10.2-cudnn8-devel | [Dockerfile.cuda10.2-cudnn8.devel](../tools/Dockerfile.cuda10.2-cudnn8.devel) | | GPU (cuda10.2-cudnn8-tensorRT7) development | Ubuntu16 | latest-cuda10.2-cudnn8-devel | [Dockerfile.cuda10.2-cudnn8.devel](../tools/Dockerfile.cuda10.2-cudnn8.devel) |
| GPU (cuda11-cudnn8-tensorRT7) development | Ubuntu18 | latest-cuda11-cudnn8-devel | [Dockerfile.cuda11-cudnn8.devel](../tools/Dockerfile.cuda11-cudnn8.devel) | | GPU (cuda11-cudnn8-tensorRT7) development | Ubuntu18 | latest-cuda11-cudnn8-devel | [Dockerfile.cuda11-cudnn8.devel](../tools/Dockerfile.cuda11-cudnn8.devel) |
...@@ -68,18 +69,32 @@ registry.baidubce.com/paddlepaddle/serving:xpu-x86 # for x86 xpu user ...@@ -68,18 +69,32 @@ registry.baidubce.com/paddlepaddle/serving:xpu-x86 # for x86 xpu user
| Env | Version | Docker images tag | OS | Gcc Version | | Env | Version | Docker images tag | OS | Gcc Version |
|----------|---------|------------------------------|-----------|-------------| |----------|---------|------------------------------|-----------|-------------|
| CPU | >=0.5.0 | 0.6.0-devel | Ubuntu 16 | 8.2.0 | | CPU | >=0.5.0 | 0.6.2-devel | Ubuntu 16 | 8.2.0 |
| | <=0.4.0 | 0.4.0-devel | CentOS 7 | 4.8.5 | | | <=0.4.0 | 0.4.0-devel | CentOS 7 | 4.8.5 |
| Cuda10.1 | >=0.5.0 | 0.6.0-cuda10.1-cudnn7-devel | Ubuntu 16 | 8.2.0 | | Cuda10.1 | >=0.5.0 | 0.6.2-cuda10.1-cudnn7-devel | Ubuntu 16 | 8.2.0 |
| | 0.6.0 | 0.6.0-cuda10.1-cudnn7-gcc54-devel | Ubuntu 16 | 5.4.0 | | | <=0.4.0 | 0.6.2-cuda10.1-cudnn7-devel | CentOS 7 | 4.8.5 |
| | <=0.4.0 | 0.6.0-cuda10.1-cudnn7-devel | CentOS 7 | 4.8.5 | | Cuda10.2 | >=0.5.0 | 0.6.2-cuda10.2-cudnn8-devel | Ubuntu 16 | 8.2.0 |
| Cuda10.2 | >=0.5.0 | 0.6.0-cuda10.2-cudnn8-devel | Ubuntu 16 | 8.2.0 |
| | <=0.4.0 | Nan | Nan | Nan | | | <=0.4.0 | Nan | Nan | Nan |
| Cuda11.0 | >=0.5.0 | 0.6.0-cuda11.0-cudnn8-devel | Ubuntu 18 | 8.2.0 | | Cuda11.0 | >=0.5.0 | 0.6.2-cuda11.0-cudnn8-devel | Ubuntu 18 | 8.2.0 |
| | <=0.4.0 | Nan | Nan | Nan | | | <=0.4.0 | Nan | Nan | Nan |
运行镜像: 运行镜像:
运行镜像比开发镜像更加轻量化, 且由于python，运行环境的多种组合，进而导致运行镜像种类过多。 如果您想了解有关信息，请检查文档[在Kubernetes上使用Paddle Serving](PADDLE_SERVING_ON_KUBERNETES.md)。 运行镜像比开发镜像更加轻量化, 运行镜像提供了serving的whl和bin，但为了运行期更小的镜像体积，没有提供诸如cmake这样但开发工具。 如果您想了解有关信息，请检查文档[在Kubernetes上使用Paddle Serving](PADDLE_SERVING_ON_KUBERNETES.md)。
| ENV | Python Version | Tag |
|------------------------------------------|----------------|-----------------------------|
| cpu | 3.6 | 0.6.2-py36-runtime |
| cpu | 3.7 | 0.6.2-py37-runtime |
| cpu | 3.8 | 0.6.2-py38-runtime |
| cuda-10.1 + cudnn-7.6.5 + tensorrt-6.0.1 | 3.6 | 0.6.2-cuda10.1-py36-runtime |
| cuda-10.1 + cudnn-7.6.5 + tensorrt-6.0.1 | 3.7 | 0.6.2-cuda10.1-py37-runtime |
| cuda-10.1 + cudnn-7.6.5 + tensorrt-6.0.1 | 3.8 | 0.6.2-cuda10.1-py38-runtime |
| cuda-10.2 + cudnn-8.2.0 + tensorrt-7.1.3 | 3.6 | 0.6.2-cuda10.2-py36-runtime |
| cuda-10.2 + cudnn-8.2.0 + tensorrt-7.1.3 | 3.7 | 0.6.2-cuda10.2-py37-runtime |
| cuda-10.2 + cudnn-8.2.0 + tensorrt-7.1.3 | 3.8 | 0.6.2-cuda10.2-py38-runtime |
| cuda-11 + cudnn-8.0.5 + tensorrt-7.1.3 | 3.6 | 0.6.2-cuda11-py36-runtime |
| cuda-11 + cudnn-8.0.5 + tensorrt-7.1.3 | 3.7 | 0.6.2-cuda11-py37-runtime |
| cuda-11 + cudnn-8.0.5 + tensorrt-7.1.3 | 3.8 | 0.6.2-cuda11-py38-runtime |
**注意事项：** 如果您在0.5.0及以上版本需要在一个容器当中同时运行CPU server和GPU server，需要选择Cuda10.1/10.2/11的镜像，因为他们和CPU环境有着相同版本的gcc。 **注意事项：** 如果您在0.5.0及以上版本需要在一个容器当中同时运行CPU server和GPU server，需要选择Cuda10.1/10.2/11的镜像，因为他们和CPU环境有着相同版本的gcc。
doc/RUN_IN_DOCKER.md
浏览文件 @ dfd7f014
...@@ -32,9 +32,7 @@ The `-p` option is to map the `9292` port of the container to the `9292` port of ...@@ -32,9 +32,7 @@ The `-p` option is to map the `9292` port of the container to the `9292` port of
### Install PaddleServing ### Install PaddleServing
The mirror comes with `paddle_serving_server`, `paddle_serving_client`, and `paddle_serving_app` corresponding to the mirror tag version. If users don’t need to change the version, they can use it directly, which is suitable for environments without extranet services. Please refer to the instructions on the homepage to download the pip package of the corresponding version.
If you need to change the version, please refer to the instructions on the homepage to download the pip package of the corresponding version.
## GPU ## GPU
......
doc/RUN_IN_DOCKER_CN.md
浏览文件 @ dfd7f014
...@@ -59,9 +59,7 @@ docker exec -it test bash ...@@ -59,9 +59,7 @@ docker exec -it test bash
### 安装PaddleServing ### 安装PaddleServing
镜像里自带对应镜像tag版本的`paddle_serving_server_gpu`，`paddle_serving_client`，`paddle_serving_app`，如果用户不需要更改版本，可以直接使用，适用于没有外网服务的环境。 请参照首页的指导，下载对应版本的pip包。[最新安装包合集](LATEST_PACKAGES.md)
如果需要更换版本，请参照首页的指导，下载对应版本的pip包。[最新安装包合集](LATEST_PACKAGES.md)
## 注意事项 ## 注意事项
......
python/examples/pipeline/PaddleClas/DarkNet53/benchmark.py
浏览文件 @ dfd7f014
...@@ -83,7 +83,7 @@ def multithread_http(thread, batch_size): ...@@ -83,7 +83,7 @@ def multithread_http(thread, batch_size):
print("Total cost: {}s".format(total_cost)) print("Total cost: {}s".format(total_cost))
print("Each thread cost: {}s. ".format(avg_cost)) print("Each thread cost: {}s. ".format(avg_cost))
print("Total count: {}. ".format(total_number)) print("Total count: {}. ".format(total_number))
print("AVG QPS: {} samples/s".format(batch_size * total_number / print("AVG_QPS: {} samples/s".format(batch_size * total_number /
total_cost)) total_cost))
show_latency(result[1]) show_latency(result[1])
......
python/examples/pipeline/PaddleClas/DarkNet53/benchmark_cfg.yaml 0 → 100644
浏览文件 @ dfd7f014
cuda_version: "10.1"
cudnn_version: "7.6"
trt_version: "6.0"
python_version: "3.7"
gcc_version: "8.2"
paddle_version: "2.0.1"
cpu: "Intel(R) Xeon(R) Gold 5117 CPU @ 2.00GHz X12"
gpu: "T4"
xpu: "None"
api: ""
owner: "cuicheng01"
model_name: "imagenet"
model_type: "static"
model_source: "PaddleClas"
model_url: ""
batch_size: 1
num_of_samples: 1000
input_shape: "3,224,224"
runtime_device: "cpu"
ir_optim: true
enable_memory_optim: true
enable_tensorrt: false
precision: "fp32"
enable_mkldnn: false
cpu_math_library_num_threads: ""
python/examples/pipeline/ocr/web_service.py
浏览文件 @ dfd7f014
...@@ -175,6 +175,6 @@ class OcrService(WebService): ...@@ -175,6 +175,6 @@ class OcrService(WebService):
return rec_op return rec_op
uci_service = OcrService(name="ocr") ocr_service = OcrService(name="ocr")
uci_service.prepare_pipeline_config("config.yml") ocr_service.prepare_pipeline_config("config.yml")
uci_service.run_service() ocr_service.run_service()
python/paddle_serving_server/server.py
浏览文件 @ dfd7f014
...@@ -35,6 +35,7 @@ import numpy as np ...@@ -35,6 +35,7 @@ import numpy as np
import grpc import grpc
import sys import sys
import collections import collections
import subprocess
from multiprocessing import Pool, Process from multiprocessing import Pool, Process
from concurrent import futures from concurrent import futures
...@@ -330,12 +331,21 @@ class Server(object): ...@@ -330,12 +331,21 @@ class Server(object):
def use_mkl(self, flag): def use_mkl(self, flag):
self.mkl_flag = flag self.mkl_flag = flag
def check_avx(self):
p = subprocess.Popen(['cat /proc/cpuinfo | grep avx 2>/dev/null'], stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True)
out, err = p.communicate()
if err == b'' and len(out) > 0:
return True
else:
return False
def get_device_version(self): def get_device_version(self):
avx_flag = False avx_flag = False
mkl_flag = self.mkl_flag avx_support = self.check_avx()
r = os.system("cat /proc/cpuinfo | grep avx > /dev/null 2>&1") if avx_support:
if r == 0:
avx_flag = True avx_flag = True
self.use_mkl(True)
mkl_flag = self.mkl_flag
if avx_flag: if avx_flag:
if mkl_flag: if mkl_flag:
device_version = "cpu-avx-mkl" device_version = "cpu-avx-mkl"
...@@ -665,7 +675,7 @@ class MultiLangServer(object): ...@@ -665,7 +675,7 @@ class MultiLangServer(object):
use_encryption_model=False, use_encryption_model=False,
cube_conf=None): cube_conf=None):
if not self._port_is_available(port): if not self._port_is_available(port):
raise SystemExit("Prot {} is already used".format(port)) raise SystemExit("Port {} is already used".format(port))
default_port = 12000 default_port = 12000
self.port_list_ = [] self.port_list_ = []
for i in range(1000): for i in range(1000):
......
tools/scripts/ipipe_py3.sh
浏览文件 @ dfd7f014
此差异已折叠。
Markdown is supported
0% 或 .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑！
想要评论请 注册 或