From e952c0ba0206817cd20da28954197bd03ca87d3b Mon Sep 17 00:00:00 2001 From: HydrogenSulfate <490868991@qq.com> Date: Fri, 10 Jun 2022 15:29:10 +0800 Subject: [PATCH] polish serving docs --- deploy/hubserving/readme.md | 267 +++++++++------ deploy/hubserving/readme_en.md | 317 ++++++++++-------- .../paddle_hub_serving_deploy_en.md | 9 +- .../paddle_hub_serving_deploy.md | 9 +- 4 files changed, 345 insertions(+), 257 deletions(-) diff --git a/deploy/hubserving/readme.md b/deploy/hubserving/readme.md index 6b2b2dd4..8506c9e4 100644 --- a/deploy/hubserving/readme.md +++ b/deploy/hubserving/readme.md @@ -1,83 +1,117 @@ -[English](readme_en.md) | 简体中文 +简体中文 | [English](readme_en.md) -# 基于PaddleHub Serving的服务部署 +# 基于 PaddleHub Serving 的服务部署 -hubserving服务部署配置服务包`clas`下包含3个必选文件，目录如下： -``` -hubserving/clas/ - └─ __init__.py 空文件，必选 - └─ config.json 配置文件，可选，使用配置启动服务时作为参数传入 - └─ module.py 主模块，必选，包含服务的完整逻辑 - └─ params.py 参数文件，必选，包含模型路径、前后处理参数等参数 +PaddleClas 支持通过 PaddleHub 快速进行服务化部署。目前支持图像分类的部署，图像识别的部署敬请期待。 + + +## 目录 +- [1. 简介](#1-简介) +- [2. 准备环境](#2-准备环境) +- [3. 下载推理模型](#3-下载推理模型) +- [4. 安装服务模块](#4-安装服务模块) +- [5. 启动服务](#5-启动服务) + - [5.1 命令行启动](#51-命令行启动) + - [5.2 配置文件启动](#52-配置文件启动) +- [6. 发送预测请求](#6-发送预测请求) +- [7. 自定义修改服务模块](#7-自定义修改服务模块) + + + +## 1. 简介 + +hubserving 服务部署配置服务包 `clas` 下包含 3 个必选文件，目录如下： + +```shell +deploy/hubserving/clas/ +├── __init__.py # 空文件，必选 +├── config.json # 配置文件，可选，使用配置启动服务时作为参数传入 +├── module.py # 主模块，必选，包含服务的完整逻辑 +└── params.py # 参数文件，必选，包含模型路径、前后处理参数等参数 ``` -## 快速启动服务 -### 1. 准备环境 + + +## 2. 准备环境 ```shell -# 安装paddlehub,请安装2.0版本 -pip3 install paddlehub==2.1.0 --upgrade -i https://pypi.tuna.tsinghua.edu.cn/simple +# 安装 paddlehub，建议安装 2.1.0 版本 +python3.7 -m pip install paddlehub==2.1.0 --upgrade -i https://pypi.tuna.tsinghua.edu.cn/simple ``` -### 2. 下载推理模型 + + +## 3. 下载推理模型 + 安装服务模块前，需要准备推理模型并放到正确路径，默认模型路径为： -``` -分类推理模型结构文件：PaddleClas/inference/inference.pdmodel -分类推理模型权重文件：PaddleClas/inference/inference.pdiparams -``` + +* 分类推理模型结构文件：`PaddleClas/inference/inference.pdmodel` +* 分类推理模型权重文件：`PaddleClas/inference/inference.pdiparams` **注意**： -* 模型文件路径可在`PaddleClas/deploy/hubserving/clas/params.py`中查看和修改： +* 模型文件路径可在 `PaddleClas/deploy/hubserving/clas/params.py` 中查看和修改： + ```python "inference_model_dir": "../inference/" ``` - 需要注意，模型文件（包括.pdmodel与.pdiparams）名称必须为`inference`。 -* 我们也提供了大量基于ImageNet-1k数据集的预训练模型，模型列表及下载地址详见[模型库概览](../../docs/zh_CN/models/models_intro.md)，也可以使用自己训练转换好的模型。 +* 模型文件（包括 `.pdmodel` 与 `.pdiparams`）的名称必须为 `inference`。 +* 我们提供了大量基于 ImageNet-1k 数据集的预训练模型，模型列表及下载地址详见[模型库概览](../../docs/zh_CN/algorithm_introduction/ImageNet_models.md)，也可以使用自己训练转换好的模型。 -### 3. 安装服务模块 -针对Linux环境和Windows环境，安装命令如下。 -* 在Linux环境下，安装示例如下： -```shell -cd PaddleClas/deploy -# 安装服务模块： -hub install hubserving/clas/ -``` + +## 4. 安装服务模块 + +* 在 Linux 环境下，安装示例如下： + ```shell + cd PaddleClas/deploy + # 安装服务模块： + hub install hubserving/clas/ + ``` + +* 在 Windows 环境下(文件夹的分隔符为`\`)，安装示例如下： + + ```shell + cd PaddleClas\deploy + # 安装服务模块： + hub install hubserving\clas\ + ``` + -* 在Windows环境下(文件夹的分隔符为`\`)，安装示例如下： + +## 5. 启动服务 + + + +### 5.1 命令行启动 + +该方式仅支持使用 CPU 预测。启动命令： ```shell -cd PaddleClas\deploy -# 安装服务模块： -hub install hubserving\clas\ +hub serving start \ +--modules clas_system +--port 8866 ``` +这样就完成了一个服务化 API 的部署，使用默认端口号 8866。 -### 4. 启动服务 -#### 方式1. 命令行命令启动（仅支持CPU） -**启动命令：** -```shell -$ hub serving start --modules Module1==Version1 \ - --port XXXX \ - --use_multiprocess \ - --workers \ -``` +**参数说明**： +| 参数 | 用途 | +| ------------------ | ----------------------------------------------------------------------------------------------------------------------------- | +| --modules/-m | [**必选**] PaddleHub Serving 预安装模型，以多个 Module==Version 键值对的形式列出
*`当不指定 Version 时，默认选择最新版本`* | +| --port/-p | [**可选**] 服务端口，默认为 8866 | +| --use_multiprocess | [**可选**] 是否启用并发方式，默认为单进程方式，推荐多核 CPU 机器使用此方式
*`Windows 操作系统只支持单进程方式`* | +| --workers | [**可选**] 在并发方式下指定的并发任务数，默认为 `2*cpu_count-1`，其中 `cpu_count` 为 CPU 核数 | +更多部署细节详见 [PaddleHub Serving模型一键服务部署](https://paddlehub.readthedocs.io/zh_CN/release-v2.1/tutorial/serving.html) -**参数：** -|参数|用途| -|-|-| -|--modules/-m| [**必选**] PaddleHub Serving预安装模型，以多个Module==Version键值对的形式列出
*`当不指定Version时，默认选择最新版本`*| -|--port/-p| [**可选**] 服务端口，默认为8866| -|--use_multiprocess| [**可选**] 是否启用并发方式，默认为单进程方式，推荐多核CPU机器使用此方式
*`Windows操作系统只支持单进程方式`*| -|--workers| [**可选**] 在并发方式下指定的并发任务数，默认为`2*cpu_count-1`，其中`cpu_count`为CPU核数| + +### 5.2 配置文件启动 -如按默认参数启动服务： ```hub serving start -m clas_system``` +该方式仅支持使用 CPU 或 GPU 预测。启动命令： -这样就完成了一个服务化API的部署，使用默认端口号8866。 +```shell +hub serving start -c config.json +``` -#### 方式2. 配置文件启动（支持CPU、GPU） -**启动命令：** -```hub serving start -c config.json``` +其中，`config.json` 格式如下： -其中，`config.json`格式如下： ```json { "modules_info": { @@ -97,92 +131,109 @@ $ hub serving start --modules Module1==Version1 \ } ``` -- `init_args`中的可配参数与`module.py`中的`_initialize`函数接口一致。其中， - - 当`use_gpu`为`true`时，表示使用GPU启动服务。 - - 当`enable_mkldnn`为`true`时，表示使用MKL-DNN加速。 -- `predict_args`中的可配参数与`module.py`中的`predict`函数接口一致。 +**参数说明**： +* `init_args` 中的可配参数与 `module.py` 中的 `_initialize` 函数接口一致。其中， + - 当 `use_gpu` 为 `true` 时，表示使用 GPU 启动服务。 + - 当 `enable_mkldnn` 为 `true` 时，表示使用 MKL-DNN 加速。 +* `predict_args` 中的可配参数与 `module.py` 中的 `predict` 函数接口一致。 -**注意:** -- 使用配置文件启动服务时，其他参数会被忽略。 -- 如果使用GPU预测(即，`use_gpu`置为`true`)，则需要在启动服务之前，设置CUDA_VISIBLE_DEVICES环境变量，如：```export CUDA_VISIBLE_DEVICES=0```，否则不用设置。 -- **`use_gpu`不可与`use_multiprocess`同时为`true`**。 -- **`use_gpu`与`enable_mkldnn`同时为`true`时，将忽略`enable_mkldnn`，而使用GPU**。 +**注意**： +* 使用配置文件启动服务时，将使用配置文件中的参数设置，其他命令行参数将被忽略； +* 如果使用 GPU 预测(即，`use_gpu` 置为 `true`)，则需要在启动服务之前，设置 `CUDA_VISIBLE_DEVICES` 环境变量来指定所使用的 GPU 卡号，如：`export CUDA_VISIBLE_DEVICES=0`； +* **`use_gpu` 不可与 `use_multiprocess` 同时为 `true`**； +* **`use_gpu` 与 `enable_mkldnn` 同时为 `true` 时，将忽略 `enable_mkldnn`，而使用 GPU**。 + +如使用 GPU 3 号卡启动服务： -如，使用GPU 3号卡启动串联服务： ```shell cd PaddleClas/deploy export CUDA_VISIBLE_DEVICES=3 hub serving start -c hubserving/clas/config.json -``` +``` -## 发送预测请求 -配置好服务端，可使用以下命令发送预测请求，获取预测结果: + +## 6. 发送预测请求 + +配置好服务端后，可使用以下命令发送预测请求，获取预测结果： ```shell cd PaddleClas/deploy -python hubserving/test_hubserving.py server_url image_path -``` - -需要给脚本传递2个必须参数： -- **server_url**：服务地址，格式为 -`http://[ip_address]:[port]/predict/[module_name]` -- **image_path**：测试图像路径，可以是单张图片路径，也可以是图像集合目录路径。 -- **batch_size**：[**可选**] 以`batch_size`大小为单位进行预测，默认为`1`。 -- **resize_short**：[**可选**] 预处理时，按短边调整大小，默认为`256`。 -- **crop_size**：[**可选**] 预处理时，居中裁剪的大小，默认为`224`。 -- **normalize**：[**可选**] 预处理时，是否进行`normalize`，默认为`True`。 -- **to_chw**：[**可选**] 预处理时，是否调整为`CHW`顺序，默认为`True`。 +python3.7 hubserving/test_hubserving.py \ +--server_url http://127.0.0.1:8866/predict/clas_system \ +--image_file ./hubserving/ILSVRC2012_val_00006666.JPEG \ +--batch_size 8 +``` +**预测输出** +```log +The result(s): class_ids: [57, 67, 68, 58, 65], label_names: ['garter snake, grass snake', 'diamondback, diamondback rattlesnake, Crotalus adamanteus', 'sidewinder, horned rattlesnake, Crotalus cerastes', 'water snake', 'sea snake'], scores: [0.21915, 0.15631, 0.14794, 0.13177, 0.12285] +The average time of prediction cost: 2.970 s/image +The average time cost: 3.014 s/image +The average top-1 score: 0.110 +``` -**注意**：如果使用`Transformer`系列模型，如`DeiT_***_384`, `ViT_***_384`等，请注意模型的输入数据尺寸，需要指定`--resize_short=384 --crop_size=384`。 +**脚本参数说明**： +* **server_url**：服务地址，格式为`http://[ip_address]:[port]/predict/[module_name]`。 +* **image_path**：测试图像路径，可以是单张图片路径，也可以是图像集合目录路径。 +* **batch_size**：[**可选**] 以 `batch_size` 大小为单位进行预测，默认为 `1`。 +* **resize_short**：[**可选**] 预处理时，按短边调整大小，默认为 `256`。 +* **crop_size**：[**可选**] 预处理时，居中裁剪的大小，默认为 `224`。 +* **normalize**：[**可选**] 预处理时，是否进行 `normalize`，默认为 `True`。 +* **to_chw**：[**可选**] 预处理时，是否调整为 `CHW` 顺序，默认为 `True`。 +**注意**：如果使用 `Transformer` 系列模型，如 `DeiT_***_384`, `ViT_***_384` 等，请注意模型的输入数据尺寸，需要指定`--resize_short=384 --crop_size=384`。 -访问示例： +**返回结果格式说明**： +返回结果为列表（list），包含 top-k 个分类结果，以及对应的得分，还有此图片预测耗时，具体如下： ```shell -python hubserving/test_hubserving.py --server_url http://127.0.0.1:8866/predict/clas_system --image_file ./hubserving/ILSVRC2012_val_00006666.JPEG --batch_size 8 -``` - -### 返回结果格式说明 -返回结果为列表（list），包含top-k个分类结果，以及对应的得分，还有此图片预测耗时，具体如下： -``` list: 返回结果 -└─ list: 第一张图片结果 - └─ list: 前k个分类结果，依score递减排序 - └─ list: 前k个分类结果对应的score，依score递减排序 - └─ float: 该图分类耗时，单位秒 +└──list: 第一张图片结果 + ├── list: 前 k 个分类结果，依 score 递减排序 + ├── list: 前 k 个分类结果对应的 score，依 score 递减排序 + └── float: 该图分类耗时，单位秒 ``` -**说明：** 如果需要增加、删除、修改返回字段，可对相应模块进行修改，完整流程参考下一节自定义修改服务模块。 -## 自定义修改服务模块 -如果需要修改服务逻辑，你一般需要操作以下步骤： -- 1、 停止服务 -```hub serving stop --port/-p XXXX``` + +## 7. 自定义修改服务模块 -- 2、 到相应的`module.py`和`params.py`等文件中根据实际需求修改代码。`module.py`修改后需要重新安装（`hub install hubserving/clas/`）并部署。在进行部署前，可通过`python hubserving/clas/module.py`测试已安装服务模块。 +如果需要修改服务逻辑，需要进行以下操作： -- 3、 卸载旧服务包 -```hub uninstall clas_system``` +1. 停止服务 + ```shell + hub serving stop --port/-p XXXX + ``` -- 4、 安装修改后的新服务包 -```hub install hubserving/clas/``` +2. 到相应的 `module.py` 和 `params.py` 等文件中根据实际需求修改代码。`module.py` 修改后需要重新安装（`hub install hubserving/clas/`）并部署。在进行部署前，可先通过 `python3.7 hubserving/clas/module.py` 命令来快速测试准备部署的代码。 -- 5、重新启动服务 -```hub serving start -m clas_system``` +3. 卸载旧服务包 + ```shell + hub uninstall clas_system + ``` + +4. 安装修改后的新服务包 + ```shell + hub install hubserving/clas/ + ``` + +5. 重新启动服务 + ```shell + hub serving start -m clas_system + ``` **注意**： -常用参数可在[params.py](./clas/params.py)中修改： +常用参数可在 `PaddleClas/deploy/hubserving/clas/params.py` 中修改： * 更换模型，需要修改模型文件路径参数: ```python "inference_model_dir": ``` - * 更改后处理时返回的`top-k`结果数量： + * 更改后处理时返回的 `top-k` 结果数量： ```python 'topk': ``` - * 更改后处理时的lable与class id对应映射文件： + * 更改后处理时的 lable 与 class id 对应映射文件： ```python 'class_id_map_file': ``` -为了避免不必要的延时以及能够以batch_size进行预测，数据预处理逻辑（包括resize、crop等操作）在客户端完成，因此需要在[test_hubserving.py](./test_hubserving.py#L35-L52)中修改。 +为了避免不必要的延时以及能够以 batch_size 进行预测，数据预处理逻辑（包括 `resize`、`crop` 等操作）均在客户端完成，因此需要在 [PaddleClas/deploy/hubserving/test_hubserving.py#L41-L47](./test_hubserving.py#L41-L47) 以及 [PaddleClas/deploy/hubserving/test_hubserving.py#L51-L76](./test_hubserving.py#L51-L76) 中修改数据预处理逻辑相关代码。 diff --git a/deploy/hubserving/readme_en.md b/deploy/hubserving/readme_en.md index bb0ddbd2..6dce5cc5 100644 --- a/deploy/hubserving/readme_en.md +++ b/deploy/hubserving/readme_en.md @@ -1,83 +1,116 @@ English | [简体中文](readme.md) -# Service deployment based on PaddleHub Serving +# Service deployment based on PaddleHub Serving + +PaddleClas supports rapid service deployment through PaddleHub. Currently, the deployment of image classification is supported. Please look forward to the deployment of image recognition. + +## Catalogue +- [1 Introduction](#1-introduction) +- [2. Prepare the environment](#2-prepare-the-environment) +- [3. Download the inference model](#3-download-the-inference-model) +- [4. Install the service module](#4-install-the-service-module) +- [5. Start service](#5-start-service) + - [5.1 Start with command line parameters](#51-start-with-command-line-parameters) + - [5.2 Start with configuration file](#52-start-with-configuration-file) +- [6. Send prediction requests](#6-send-prediction-requests) +- [7. User defined service module modification](#7-user-defined-service-module-modification) -HubServing service pack contains 3 files, the directory is as follows: -``` -hubserving/clas/ - └─ __init__.py Empty file, required - └─ config.json Configuration file, optional, passed in as a parameter when using configuration to start the service - └─ module.py Main module file, required, contains the complete logic of the service - └─ params.py Parameter file, required, including parameters such as model path, pre- and post-processing parameters -``` -## Quick start service -### 1. Prepare the environment + +## 1 Introduction + +The hubserving service deployment configuration service package `clas` contains 3 required files, the directories are as follows: + ```shell -# Install version 2.0 of PaddleHub -pip3 install paddlehub==2.1.0 --upgrade -i https://pypi.tuna.tsinghua.edu.cn/simple +deploy/hubserving/clas/ +├── __init__.py # Empty file, required +├── config.json # Configuration file, optional, passed in as a parameter when starting the service with configuration +├── module.py # The main module, required, contains the complete logic of the service +└── params.py # Parameter file, required, including model path, pre- and post-processing parameters and other parameters ``` -### 2. Download inference model -Before installing the service module, you need to prepare the inference model and put it in the correct path. The default model path is: -``` -Model structure file: PaddleClas/inference/inference.pdmodel -Model parameters file: PaddleClas/inference/inference.pdiparams + +## 2. Prepare the environment +```shell +# Install paddlehub, version 2.1.0 is recommended +python3.7 -m pip install paddlehub==2.1.0 --upgrade -i https://pypi.tuna.tsinghua.edu.cn/simple ``` -* The model file path can be viewed and modified in `PaddleClas/deploy/hubserving/clas/params.py`. - It should be noted that the prefix of model structure file and model parameters file must be `inference`. + +## 3. Download the inference model -* More models provided by PaddleClas can be obtained from the [model library](../../docs/en/models/models_intro_en.md). You can also use models trained by yourself. +Before installing the service module, you need to prepare the inference model and put it in the correct path. The default model path is: -### 3. Install Service Module +* Classification inference model structure file: `PaddleClas/inference/inference.pdmodel` +* Classification inference model weight file: `PaddleClas/inference/inference.pdiparams` -* On Linux platform, the examples are as follows. -```shell -cd PaddleClas/deploy -hub install hubserving/clas/ -``` +**Notice**: +* Model file paths can be viewed and modified in `PaddleClas/deploy/hubserving/clas/params.py`: + + ```python + "inference_model_dir": "../inference/" + ``` +* Model files (including `.pdmodel` and `.pdiparams`) must be named `inference`. +* We provide a large number of pre-trained models based on the ImageNet-1k dataset. For the model list and download address, see [Model Library Overview](../../docs/en/algorithm_introduction/ImageNet_models_en.md), or you can use your own trained and converted models. + + + +## 4. Install the service module + +* In the Linux environment, the installation example is as follows: + ```shell + cd PaddleClas/deploy + # Install the service module: + hub install hubserving/clas/ + ``` + +* In the Windows environment (the folder separator is `\`), the installation example is as follows: + + ```shell + cd PaddleClas\deploy + # Install the service module: + hub install hubserving\clas\ + ``` + + + +## 5. Start service + + + +### 5.1 Start with command line parameters + +This method only supports prediction using CPU. Start command: -* On Windows platform, the examples are as follows. ```shell -cd PaddleClas\deploy -hub install hubserving\clas\ +hub serving start \ +--modules clas_system +--port 8866 ``` +This completes the deployment of a serviced API, using the default port number 8866. -### 4. Start service -#### Way 1. Start with command line parameters (CPU only) +**Parameter Description**: +| parameters | uses | +| ------------------ | ------------------- | +| --modules/-m | [**required**] PaddleHub Serving pre-installed model, listed in the form of multiple Module==Version key-value pairs
*`When no Version is specified, the latest is selected by default version`* | +| --port/-p | [**OPTIONAL**] Service port, default is 8866 | +| --use_multiprocess | [**Optional**] Whether to enable the concurrent mode, the default is single-process mode, it is recommended to use this mode for multi-core CPU machines
*`Windows operating system only supports single-process mode`* | +| --workers | [**Optional**] The number of concurrent tasks specified in concurrent mode, the default is `2*cpu_count-1`, where `cpu_count` is the number of CPU cores | +For more deployment details, see [PaddleHub Serving Model One-Click Service Deployment](https://paddlehub.readthedocs.io/zh_CN/release-v2.1/tutorial/serving.html) -**start command：** -```shell -$ hub serving start --modules Module1==Version1 \ - --port XXXX \ - --use_multiprocess \ - --workers \ -``` -**parameters：** - -|parameters|usage| -|-|-| -|--modules/-m|PaddleHub Serving pre-installed model, listed in the form of multiple Module==Version key-value pairs
*`When Version is not specified, the latest version is selected by default`*| -|--port/-p|Service port, default is 8866| -|--use_multiprocess|Enable concurrent mode, the default is single-process mode, this mode is recommended for multi-core CPU machines
*`Windows operating system only supports single-process mode`*| -|--workers|The number of concurrent tasks specified in concurrent mode, the default is `2*cpu_count-1`, where `cpu_count` is the number of CPU cores| - -For example, start the 2-stage series service: -```shell -hub serving start -m clas_system -``` + +### 5.2 Start with configuration file -This completes the deployment of a service API, using the default port number 8866. +This method only supports prediction using CPU or GPU. Start command: -#### Way 2. Start with configuration file（CPU、GPU） -**start command：** ```shell -hub serving start --config/-c config.json -``` -Wherein, the format of `config.json` is as follows: +hub serving start -c config.json +``` + +Among them, the format of `config.json` is as follows: + ```json { "modules_info": { @@ -96,104 +129,110 @@ Wherein, the format of `config.json` is as follows: "workers": 2 } ``` -- The configurable parameters in `init_args` are consistent with the `_initialize` function interface in `module.py`. Among them, - - when `use_gpu` is `true`, it means that the GPU is used to start the service. - - when `enable_mkldnn` is `true`, it means that use MKL-DNN to accelerate. -- The configurable parameters in `predict_args` are consistent with the `predict` function interface in `module.py`. - -**Note:** -- When using the configuration file to start the service, other parameters will be ignored. -- If you use GPU prediction (that is, `use_gpu` is set to `true`), you need to set the environment variable CUDA_VISIBLE_DEVICES before starting the service, such as: ```export CUDA_VISIBLE_DEVICES=0```, otherwise you do not need to set it. -- **`use_gpu` and `use_multiprocess` cannot be `true` at the same time.** -- **When both `use_gpu` and `enable_mkldnn` are set to `true` at the same time, GPU is used to run and `enable_mkldnn` will be ignored.** - -For example, use GPU card No. 3 to start the 2-stage series service: + +**Parameter Description**: +* The configurable parameters in `init_args` are consistent with the `_initialize` function interface in `module.py`. in, + - When `use_gpu` is `true`, it means to use GPU to start the service. + - When `enable_mkldnn` is `true`, it means to use MKL-DNN acceleration. +* The configurable parameters in `predict_args` are consistent with the `predict` function interface in `module.py`. + +**Notice**: +* When using the configuration file to start the service, the parameter settings in the configuration file will be used, and other command line parameters will be ignored; +* If you use GPU prediction (ie, `use_gpu` is set to `true`), you need to set the `CUDA_VISIBLE_DEVICES` environment variable to specify the GPU card number used before starting the service, such as: `export CUDA_VISIBLE_DEVICES=0`; +* **`use_gpu` cannot be `true`** at the same time as `use_multiprocess`; +* ** When both `use_gpu` and `enable_mkldnn` are `true`, `enable_mkldnn` will be ignored and GPU** will be used. + +If you use GPU No. 3 card to start the service: + ```shell cd PaddleClas/deploy export CUDA_VISIBLE_DEVICES=3 hub serving start -c hubserving/clas/config.json -``` - -## Send prediction requests -After the service starts, you can use the following command to send a prediction request to obtain the prediction result: -```shell -cd PaddleClas/deploy -python hubserving/test_hubserving.py server_url image_path ``` -Two required parameters need to be passed to the script: -- **server_url**: service address，format of which is -`http://[ip_address]:[port]/predict/[module_name]` -- **image_path**: Test image path, can be a single image path or an image directory path -- **batch_size**: [**Optional**] batch_size. Default by `1`. -- **resize_short**: [**Optional**] In preprocessing, resize according to short size. Default by `256`。 -- **crop_size**: [**Optional**] In preprocessing, centor crop size. Default by `224`。 -- **normalize**: [**Optional**] In preprocessing, whether to do `normalize`. Default by `True`。 -- **to_chw**: [**Optional**] In preprocessing, whether to transpose to `CHW`. Default by `True`。 + +## 6. Send prediction requests -**Notice**: -If you want to use `Transformer series models`, such as `DeiT_***_384`, `ViT_***_384`, etc., please pay attention to the input size of model, and need to set `--resize_short=384`, `--crop_size=384`. +After configuring the server, you can use the following command to send a prediction request to get the prediction result: -**Eg.** ```shell -python hubserving/test_hubserving.py --server_url http://127.0.0.1:8866/predict/clas_system --image_file ./hubserving/ILSVRC2012_val_00006666.JPEG --batch_size 8 -``` - -### Returned result format -The returned result is a list, including the `top_k`'s classification results, corresponding scores and the time cost of prediction, details as follows. - -``` -list: The returned results -└─ list: The result of first picture - └─ list: The top-k classification results, sorted in descending order of score - └─ list: The scores corresponding to the top-k classification results, sorted in descending order of score - └─ float: The time cost of predicting the picture, unit second +cd PaddleClas/deploy +python3.7 hubserving/test_hubserving.py \ +--server_url http://127.0.0.1:8866/predict/clas_system \ +--image_file ./hubserving/ILSVRC2012_val_00006666.JPEG \ +--batch_size 8 +``` +**Predicted output** +```log +The result(s): class_ids: [57, 67, 68, 58, 65], label_names: ['garter snake, grass snake', 'diamondback, diamondback rattlesnake, Crotalus adamanteus', 'sidewinder, horned rattlesnake, Crotalus cerastes' , 'water snake', 'sea snake'], scores: [0.21915, 0.15631, 0.14794, 0.13177, 0.12285] +The average time of prediction cost: 2.970 s/image +The average time cost: 3.014 s/image +The average top-1 score: 0.110 +``` + +**Script parameter description**: +* **server_url**: Service address, the format is `http://[ip_address]:[port]/predict/[module_name]`. +* **image_path**: The test image path, which can be a single image path or an image collection directory path. +* **batch_size**: [**OPTIONAL**] Make predictions in `batch_size` size, default is `1`. +* **resize_short**: [**optional**] When preprocessing, resize by short edge, default is `256`. +* **crop_size**: [**Optional**] The size of the center crop during preprocessing, the default is `224`. +* **normalize**: [**Optional**] Whether to perform `normalize` during preprocessing, the default is `True`. +* **to_chw**: [**Optional**] Whether to adjust to `CHW` order when preprocessing, the default is `True`. + +**Note**: If you use `Transformer` series models, such as `DeiT_***_384`, `ViT_***_384`, etc., please pay attention to the input data size of the model, you need to specify `--resize_short=384 -- crop_size=384`. + +**Return result format description**: +The returned result is a list (list), including the top-k classification results, the corresponding scores, and the time-consuming prediction of this image, as follows: +```shell +list: return result +└──list: first image result + ├── list: the top k classification results, sorted in descending order of score + ├── list: the scores corresponding to the first k classification results, sorted in descending order of score + └── float: The image classification time, in seconds ``` -**Note：** If you need to add, delete or modify the returned fields, you can modify the corresponding module. For the details, refer to the user-defined modification service module in the next section. -## User defined service module modification -If you need to modify the service logic, the following steps are generally required: -1. Stop service -```shell -hub serving stop --port/-p XXXX -``` + +## 7. User defined service module modification -2. Modify the code in the corresponding files, like `module.py` and `params.py`, according to the actual needs. You need re-install(hub install hubserving/clas/) and re-deploy after modifing `module.py`. -After modifying and installing and before deploying, you can use `python hubserving/clas/module.py` to test the installed service module. +If you need to modify the service logic, you need to do the following: -For example, if you need to replace the model used by the deployed service, you need to modify model path parameters `cfg.model_file` and `cfg.params_file` in `params.py`. Of course, other related parameters may need to be modified at the same time. Please modify and debug according to the actual situation. - -3. Uninstall old service module -```shell -hub uninstall clas_system -``` +1. Stop the service + ```shell + hub serving stop --port/-p XXXX + ``` -4. Install modified service module -```shell -hub install hubserving/clas/ -``` +2. Go to the corresponding `module.py` and `params.py` and other files to modify the code according to actual needs. `module.py` needs to be reinstalled after modification (`hub install hubserving/clas/`) and deployed. Before deploying, you can use the `python3.7 hubserving/clas/module.py` command to quickly test the code ready for deployment. -5. Restart service -```shell -hub serving start -m clas_system -``` +3. Uninstall the old service pack + ```shell + hub uninstall clas_system + ``` -**Note**: +4. Install the new modified service pack + ```shell + hub install hubserving/clas/ + ``` -Common parameters can be modified in params.py: -* Directory of model files(include model structure file and model parameters file): - ```python - "inference_model_dir": - ``` -* The number of Top-k results returned during post-processing: - ```python - 'topk': - ``` -* Mapping file corresponding to label and class ID during post-processing: - ```python - 'class_id_map_file': - ``` +5. Restart the service + ```shell + hub serving start -m clas_system + ``` -In order to avoid unnecessary delay and be able to predict in batch, the preprocessing (include resize, crop and other) is completed in the client, so modify [test_hubserving.py](./test_hubserving.py#L35-L52) if necessary. +**Notice**: +Common parameters can be modified in `PaddleClas/deploy/hubserving/clas/params.py`: + * To replace the model, you need to modify the model file path parameters: + ```python + "inference_model_dir": + ``` + * Change the number of `top-k` results returned when postprocessing: + ```python + 'topk': + ``` + * The mapping file corresponding to the lable and class id when changing the post-processing: + ```python + 'class_id_map_file': + ``` + +In order to avoid unnecessary delay and be able to predict with batch_size, data preprocessing logic (including `resize`, `crop` and other operations) is completed on the client side, so it needs to modify data preprocessing logic related code in [PaddleClas/deploy/hubserving/test_hubserving.py# L41-L47](./test_hubserving.py#L41-L47) and [PaddleClas/deploy/hubserving/test_hubserving.py#L51-L76](./test_hubserving.py#L51-L76). diff --git a/docs/en/inference_deployment/paddle_hub_serving_deploy_en.md b/docs/en/inference_deployment/paddle_hub_serving_deploy_en.md index 0aa9ca20..4dddc94b 100644 --- a/docs/en/inference_deployment/paddle_hub_serving_deploy_en.md +++ b/docs/en/inference_deployment/paddle_hub_serving_deploy_en.md @@ -1,10 +1,9 @@ -English|[Chinese](../../zh_CN/inference_deployment/paddle_hub_serving_deploy.md) +English | [简体中文](../../zh_CN/inference_deployment/paddle_hub_serving_deploy.md) + # Service deployment based on PaddleHub Serving PaddleClas supports rapid service deployment through PaddleHub. Currently, the deployment of image classification is supported. Please look forward to the deployment of image recognition. ---- - ## Catalogue - [1. Introduction](#1) - [2. Prepare the environment](#2) @@ -34,7 +33,7 @@ deploy/hubserving/clas/ ## 2. Prepare the environment ```shell -# Install paddlehub, please install version 2.0 +# Install paddlehub, version 2.1.0 is recommended python3.7 -m pip install paddlehub==2.1.0 --upgrade -i https://pypi.tuna.tsinghua.edu.cn/simple ``` @@ -236,4 +235,4 @@ Common parameters can be modified in `PaddleClas/deploy/hubserving/clas/params.p 'class_id_map_file': ``` -In order to avoid unnecessary delay and be able to predict with batch_size, data preprocessing logic (including `resize`, `crop` and other operations) is completed on the client side, so it needs to be in [PaddleClas/deploy/hubserving/test_hubserving.py# Modify the code related to data preprocessing logic in L35-L52](../../../deploy/hubserving/test_hubserving.py). +In order to avoid unnecessary delay and be able to predict with batch_size, data preprocessing logic (including `resize`, `crop` and other operations) is completed on the client side, so it needs to be in [PaddleClas/deploy/hubserving/test_hubserving.py# L41-L47](../../../deploy/hubserving/test_hubserving.py#L41-L47) and [PaddleClas/deploy/hubserving/test_hubserving.py#L51-L76](../../../deploy/hubserving/test_hubserving.py#L51-L76) Modify the data preprocessing logic related code. diff --git a/docs/zh_CN/inference_deployment/paddle_hub_serving_deploy.md b/docs/zh_CN/inference_deployment/paddle_hub_serving_deploy.md index 96987d47..37d688b3 100644 --- a/docs/zh_CN/inference_deployment/paddle_hub_serving_deploy.md +++ b/docs/zh_CN/inference_deployment/paddle_hub_serving_deploy.md @@ -1,10 +1,9 @@ -简体中文|[English](../../en/inference_deployment/paddle_hub_serving_deploy_en.md) +简体中文 | [English](../../en/inference_deployment/paddle_hub_serving_deploy_en.md) + # 基于 PaddleHub Serving 的服务部署 PaddleClas 支持通过 PaddleHub 快速进行服务化部署。目前支持图像分类的部署，图像识别的部署敬请期待。 ---- - ## 目录 - [1. 简介](#1) @@ -35,7 +34,7 @@ deploy/hubserving/clas/ ## 2. 准备环境 ```shell -# 安装 paddlehub,请安装 2.0 版本 +# 安装 paddlehub，建议安装 2.1.0 版本 python3.7 -m pip install paddlehub==2.1.0 --upgrade -i https://pypi.tuna.tsinghua.edu.cn/simple ``` @@ -237,4 +236,4 @@ list: 返回结果 'class_id_map_file': ``` -为了避免不必要的延时以及能够以 batch_size 进行预测，数据预处理逻辑（包括 `resize`、`crop` 等操作）均在客户端完成，因此需要在 [PaddleClas/deploy/hubserving/test_hubserving.py#L35-L52](../../../deploy/hubserving/test_hubserving.py) 中修改数据预处理逻辑相关代码。 +为了避免不必要的延时以及能够以 batch_size 进行预测，数据预处理逻辑（包括 `resize`、`crop` 等操作）均在客户端完成，因此需要在 [PaddleClas/deploy/hubserving/test_hubserving.py#L41-L47](../../../deploy/hubserving/test_hubserving.py#L41-L47) 以及 [PaddleClas/deploy/hubserving/test_hubserving.py#L51-L76](../../../deploy/hubserving/test_hubserving.py#L51-L76) 中修改数据预处理逻辑相关代码。 -- GitLab