diff --git a/README_ch.md b/README_ch.md index 426edbd31ba53d60e5b7aa66bac48f060ebfb40c..2f057d772bc68247c1a28b57c81fb57a8fd6f6c6 100644 --- a/README_ch.md +++ b/README_ch.md @@ -78,7 +78,7 @@ PP-ShiTu图像识别快速体验:[点击这里](./docs/zh_CN/quick_start/quick - 推理部署 - [基于python预测引擎推理](docs/zh_CN/inference_deployment/python_deploy.md#1) - [基于C++预测引擎推理](docs/zh_CN/inference_deployment/cpp_deploy.md) - - [服务化部署](docs/zh_CN/inference_deployment/paddle_serving_deploy.md) + - [服务化部署](docs/zh_CN/inference_deployment/classification_serving_deploy.md) - [端侧部署](docs/zh_CN/inference_deployment/paddle_lite_deploy.md) - [Paddle2ONNX模型转化与预测](deploy/paddle2onnx/readme.md) - [模型压缩](deploy/slim/README.md) @@ -93,7 +93,7 @@ PP-ShiTu图像识别快速体验:[点击这里](./docs/zh_CN/quick_start/quick - 推理部署 - [基于python预测引擎推理](docs/zh_CN/inference_deployment/python_deploy.md#2) - [基于C++预测引擎推理](deploy/cpp_shitu/readme.md) - - [服务化部署](docs/zh_CN/inference_deployment/paddle_serving_deploy.md) + - [服务化部署](docs/zh_CN/inference_deployment/recognition_serving_deploy.md) - [端侧部署](deploy/lite_shitu/README.md) - PP系列骨干网络模型 - [PP-HGNet](docs/zh_CN/models/PP-HGNet.md) diff --git a/deploy/paddleserving/readme.md b/deploy/paddleserving/readme.md index 5d00914ec442ade3fcfcfb3f294f1caaa91f388d..f4efbae6c509c5ff50b2e5fbdc75ef8851d974c4 100644 --- a/deploy/paddleserving/readme.md +++ b/deploy/paddleserving/readme.md @@ -112,7 +112,7 @@ python3.7 -m pip install paddle-serving-server-gpu==0.7.0.post112 # GPU with CUD └── serving_client_conf.stream.prototxt ``` -- Serving 为了兼容不同模型的部署,提供了输入输出重命名的功能。让不同的模型在推理部署时,只需要修改配置文件的 `alias_name` 即可,无需修改代码即可完成推理部署。因此在转换完毕后需要分别修改 `ResNet50_vd_serving` 和 `ResNet50_vd_client` 下文件 `serving_server_conf.prototxt` 中的 alias 名字,将 `fetch_var` 中的 `alias_name` 改为 `prediction`,修改后的 serving_server_conf.prototxt 如下所示: +- Serving 为了兼容不同模型的部署,提供了输入输出重命名的功能。让不同的模型在推理部署时,只需要修改配置文件的 `alias_name` 即可,无需修改代码即可完成推理部署。因此在转换完毕后需要分别修改 `ResNet50_vd_serving` 下的文件 `serving_server_conf.prototxt` 和 `ResNet50_vd_client` 下的文件 `serving_client_conf.prototxt`,将 `fetch_var` 中 `alias_name:` 后的字段改为 `prediction`,修改后的 `serving_server_conf.prototxt` 和 `serving_client_conf.prototxt` 如下所示: ```log feed_var { name: "inputs" @@ -159,7 +159,7 @@ test_cpp_serving_client.py # rpc方式发送C++ serving预测请求的脚本 # 发送服务请求 python3.7 pipeline_http_client.py ``` - 成功运行后,模型预测的结果会打印在 cmd 窗口中,结果如下: + 成功运行后,模型预测的结果会打印在客户端中,如下所示: ```log {'err_no': 0, 'err_msg': '', 'key': ['label', 'prob'], 'value': ["['daisy']", '[0.9341402053833008]'], 'tensors': []} ``` @@ -180,11 +180,11 @@ test_cpp_serving_client.py # rpc方式发送C++ serving预测请求的脚本 # 进入工作目录 cd PaddleClas/deploy/paddleserving # 一键编译安装Serving server、设置 SERVING_BIN - bash ./build_server.sh python3.7 + source ./build_server.sh python3.7 ``` **注:**[build_server.sh](./build_server.sh#L55-L62)所设定的路径可能需要根据实际机器上的环境如CUDA、python版本等作一定修改,然后再编译。 -- 修改客户端文件 `ResNet50_client/serving_client_conf.prototxt` ,将 `feed_type:` 后的字段改为20,将第一个 `shape:` 后的字段改为1并删掉其余的 `shape` 字段。 +- 修改客户端文件 `ResNet50_vd_client/serving_client_conf.prototxt` ,将 `feed_type:` 后的字段改为20,将第一个 `shape:` 后的字段改为1并删掉其余的 `shape` 字段。 ```log feed_var { name: "inputs" @@ -195,16 +195,16 @@ test_cpp_serving_client.py # rpc方式发送C++ serving预测请求的脚本 } ``` - 修改 [`test_cpp_serving_client`](./test_cpp_serving_client.py) 的部分代码 - 1. 修改 [`feed={"inputs": image}`](./test_cpp_serving_client.py#L28) 部分代码,将 `load_client_config` 后的路径改为 `ResNet50_client/serving_client_conf.prototxt` 。 - 2. 修改 [`feed={"inputs": image}`](./test_cpp_serving_client.py#L45) 部分代码,将 `inputs` 改为与 `ResNet50_client/serving_client_conf.prototxt` 中 `feed_var` 字段下面的 `name` 一致。由于部分模型client文件中的 `name` 为 `x` 而不是 `inputs` ,因此使用这些模型进行C++ Serving部署时需要注意这一点。 + 1. 修改 [`load_client_config`](./test_cpp_serving_client.py#L28) 处的代码,将 `load_client_config` 后的路径改为 `ResNet50_vd_client/serving_client_conf.prototxt` 。 + 2. 修改 [`feed={"inputs": image}`](./test_cpp_serving_client.py#L45) 处的代码,将 `inputs` 改为与 `ResNet50_vd_client/serving_client_conf.prototxt` 中 `feed_var` 字段下面的 `name` 一致。由于部分模型client文件中的 `name` 为 `x` 而不是 `inputs` ,因此使用这些模型进行C++ Serving部署时需要注意这一点。 - 启动服务: ```shell # 启动服务, 服务在后台运行,运行日志保存在 nohup.txt # CPU部署 - sh run_cpp_serving.sh + bash run_cpp_serving.sh # GPU部署并指定0号卡 - sh run_cpp_serving.sh 0 + bash run_cpp_serving.sh 0 ``` - 发送请求: @@ -212,7 +212,7 @@ test_cpp_serving_client.py # rpc方式发送C++ serving预测请求的脚本 # 发送服务请求 python3.7 test_cpp_serving_client.py ``` - 成功运行后,模型预测的结果会打印在 cmd 窗口中,结果如下: + 成功运行后,模型预测的结果会打印在客户端中,如下所示: ```log prediction: daisy, probability: 0.9341399073600769 ``` diff --git a/deploy/paddleserving/recognition/readme.md b/deploy/paddleserving/recognition/readme.md index 40cd8b851a96258f25a75270881663be28c464ad..0ed87a8d3062eb2b1e8a26f547104b9233f4b0df 100644 --- a/deploy/paddleserving/recognition/readme.md +++ b/deploy/paddleserving/recognition/readme.md @@ -119,6 +119,26 @@ python3.7 -m pip install paddle-serving-server-gpu==0.7.0.post112 # GPU with CUD ``` 上述命令的参数含义与[#3.1 模型转换](#3.1)相同 + 识别推理模型转换完成后,会在当前文件夹多出 `general_PPLCNet_x2_5_lite_v1.0_serving/` 和 `general_PPLCNet_x2_5_lite_v1.0_client/` 的文件夹。分别修改 `general_PPLCNet_x2_5_lite_v1.0_serving/` 和 `general_PPLCNet_x2_5_lite_v1.0_client/` 目录下的 `serving_server_conf.prototxt` 中的 `alias` 名字: 将 `fetch_var` 中的 `alias_name` 改为 `features`。 修改后的 `serving_server_conf.prototxt` 内容如下 + + ```log + feed_var { + name: "x" + alias_name: "x" + is_lod_tensor: false + feed_type: 1 + shape: 3 + shape: 224 + shape: 224 + } + fetch_var { + name: "save_infer_model/scale_0.tmp_1" + alias_name: "features" + is_lod_tensor: false + fetch_type: 1 + shape: 512 + } + ``` 通用检测 inference 模型转换完成后,会在当前文件夹多出 `picodet_PPLCNet_x2_5_mainbody_lite_v1.0_serving/` 和 `picodet_PPLCNet_x2_5_mainbody_lite_v1.0_client/` 的文件夹,具备如下结构: ```shell ├── picodet_PPLCNet_x2_5_mainbody_lite_v1.0_serving/ @@ -182,7 +202,7 @@ python3.7 -m pip install paddle-serving-server-gpu==0.7.0.post112 # GPU with CUD ```shell python3.7 pipeline_http_client.py ``` - 成功运行后,模型预测的结果会打印在 cmd 窗口中,结果如下: + 成功运行后,模型预测的结果会打印在客户端中,如下所示: ```log {'err_no': 0, 'err_msg': '', 'key': ['result'], 'value': ["[{'bbox': [345, 95, 524, 576], 'rec_docs': '红牛-强化型', 'rec_scores': 0.79903316}]"], 'tensors': []} ``` @@ -219,9 +239,9 @@ python3.7 -m pip install paddle-serving-server-gpu==0.7.0.post112 # GPU with CUD # 端口号默认为9400;运行日志默认保存在 log_PPShiTu.txt 中 # CPU部署 - sh run_cpp_serving.sh + bash run_cpp_serving.sh # GPU部署,并指定第0号卡 - sh run_cpp_serving.sh 0 + bash run_cpp_serving.sh 0 ``` - 发送请求: @@ -229,7 +249,7 @@ python3.7 -m pip install paddle-serving-server-gpu==0.7.0.post112 # GPU with CUD # 发送服务请求 python3.7 test_cpp_serving_client.py ``` - 成功运行后,模型预测的结果会打印在客户端的终端窗口中,结果如下所示: + 成功运行后,模型预测的结果会打印在客户端中,如下所示: ```log WARNING: Logging before InitGoogleLogging() is written to STDERR I0614 03:01:36.273097 6084 naming_service_thread.cpp:202] brpc::policy::ListNamingService("127.0.0.1:9400"): added 1 diff --git a/deploy/paddleserving/recognition/run_cpp_serving.sh b/deploy/paddleserving/recognition/run_cpp_serving.sh index 75caf23f88a470c934c7f4e0c539a79faafb3cc1..e1deb1148b1705031c0e92522e7eaf7cf4679a45 100644 --- a/deploy/paddleserving/recognition/run_cpp_serving.sh +++ b/deploy/paddleserving/recognition/run_cpp_serving.sh @@ -5,10 +5,10 @@ if [[ -n "${gpu_id}" ]]; then nohup python3.7 -m paddle_serving_server.serve \ --model ../../models/picodet_PPLCNet_x2_5_mainbody_lite_v1.0_serving ../../models/general_PPLCNet_x2_5_lite_v1.0_serving \ --op GeneralPicodetOp GeneralFeatureExtractOp \ - --port 9400 --gpu_id="${gpu_id}" >>log_PPShiTu.txt 1&>2 & + --port 9400 --gpu_id="${gpu_id}" > log_PPShiTu.txt 2>&1 & else nohup python3.7 -m paddle_serving_server.serve \ --model ../../models/picodet_PPLCNet_x2_5_mainbody_lite_v1.0_serving ../../models/general_PPLCNet_x2_5_lite_v1.0_serving \ --op GeneralPicodetOp GeneralFeatureExtractOp \ - --port 9400 >>log_PPShiTu.txt 1&>2 & + --port 9400 > log_PPShiTu.txt 2>&1 & fi diff --git a/docs/zh_CN/inference_deployment/classification_serving_deploy.md b/docs/zh_CN/inference_deployment/classification_serving_deploy.md new file mode 120000 index 0000000000000000000000000000000000000000..da7321d1bc547b2e4a6e910c7cce037c4452ef16 --- /dev/null +++ b/docs/zh_CN/inference_deployment/classification_serving_deploy.md @@ -0,0 +1 @@ +../../../deploy/paddleserving/readme.md \ No newline at end of file diff --git a/docs/zh_CN/inference_deployment/inference/cpp_deploy.md b/docs/zh_CN/inference_deployment/cpp_deploy.md similarity index 100% rename from docs/zh_CN/inference_deployment/inference/cpp_deploy.md rename to docs/zh_CN/inference_deployment/cpp_deploy.md diff --git a/docs/zh_CN/inference_deployment/inference/cpp_deploy_on_windows.md b/docs/zh_CN/inference_deployment/cpp_deploy_on_windows.md similarity index 100% rename from docs/zh_CN/inference_deployment/inference/cpp_deploy_on_windows.md rename to docs/zh_CN/inference_deployment/cpp_deploy_on_windows.md diff --git a/docs/zh_CN/inference_deployment/deployment/paddle_classification_serving_deploy.md b/docs/zh_CN/inference_deployment/deployment/paddle_classification_serving_deploy.md deleted file mode 120000 index c908bb1f27d0a6a46894285800297618f9661417..0000000000000000000000000000000000000000 --- a/docs/zh_CN/inference_deployment/deployment/paddle_classification_serving_deploy.md +++ /dev/null @@ -1 +0,0 @@ -deploy/paddleserving/readme.md \ No newline at end of file diff --git a/docs/zh_CN/inference_deployment/deployment/paddle_recognition_serving_deploy.md b/docs/zh_CN/inference_deployment/deployment/paddle_recognition_serving_deploy.md deleted file mode 120000 index 57f80720019ba8f90d94046315a1fd70ccb9b13a..0000000000000000000000000000000000000000 --- a/docs/zh_CN/inference_deployment/deployment/paddle_recognition_serving_deploy.md +++ /dev/null @@ -1 +0,0 @@ -deploy/paddleserving/recognition/readme.md \ No newline at end of file diff --git a/docs/zh_CN/inference_deployment/deployment/paddle_hub_serving_deploy.md b/docs/zh_CN/inference_deployment/paddle_hub_serving_deploy.md similarity index 100% rename from docs/zh_CN/inference_deployment/deployment/paddle_hub_serving_deploy.md rename to docs/zh_CN/inference_deployment/paddle_hub_serving_deploy.md diff --git a/docs/zh_CN/inference_deployment/deployment/paddle_lite_deploy.md b/docs/zh_CN/inference_deployment/paddle_lite_deploy.md similarity index 100% rename from docs/zh_CN/inference_deployment/deployment/paddle_lite_deploy.md rename to docs/zh_CN/inference_deployment/paddle_lite_deploy.md diff --git a/docs/zh_CN/inference_deployment/inference/python_deploy.md b/docs/zh_CN/inference_deployment/python_deploy.md similarity index 96% rename from docs/zh_CN/inference_deployment/inference/python_deploy.md rename to docs/zh_CN/inference_deployment/python_deploy.md index 9d4f254fdde8400b369dc54a4437dcc5f6929126..22b871344b782098ef9ded562cc7f2ce4277f790 100644 --- a/docs/zh_CN/inference_deployment/inference/python_deploy.md +++ b/docs/zh_CN/inference_deployment/python_deploy.md @@ -8,9 +8,9 @@ - [1. 图像分类模型推理](#1) - [2. PP-ShiTu模型推理](#2) - - [2.1 主体检测模型推理](#2.1) - - [2.2 特征提取模型推理](#2.2) - - [2.3 PP-ShiTu PipeLine推理](#2.3) + - [2.1 主体检测模型推理](#2.1) + - [2.2 特征提取模型推理](#2.2) + - [2.3 PP-ShiTu PipeLine推理](#2.3) ## 1. 图像分类推理 diff --git a/docs/zh_CN/inference_deployment/recognition_serving_deploy.md b/docs/zh_CN/inference_deployment/recognition_serving_deploy.md new file mode 120000 index 0000000000000000000000000000000000000000..3692c7f63b64b444cb92cde8a087a37e13d0706a --- /dev/null +++ b/docs/zh_CN/inference_deployment/recognition_serving_deploy.md @@ -0,0 +1 @@ +../../../deploy/paddleserving/recognition/readme.md \ No newline at end of file diff --git a/docs/zh_CN/inference_deployment/inference/whl_deploy.md b/docs/zh_CN/inference_deployment/whl_deploy.md similarity index 100% rename from docs/zh_CN/inference_deployment/inference/whl_deploy.md rename to docs/zh_CN/inference_deployment/whl_deploy.md