Skip to content

P
PaddleOCR

weixin_41840029 / PaddleOCR
与 Fork 源项目一致

Fork自 PaddlePaddle / PaddleOCR

通知 1
Star 1
Fork 0
P
PaddleOCR
未验证 提交 426e3e44 编写于 作者: M MissPenguin 提交者： GitHub
浏览文件
操作

Merge pull request #2771 from tink2123/pdserving 

update pdserving
上级 bcf59095 4135eb0c
显示空白变更内容
内联 并排
Showing with 235 addition and 106 deletion
deploy/pdserving/README.md
浏览文件 @ 426e3e44
...@@ -30,6 +30,8 @@ The introduction and tutorial of Paddle Serving service deployment framework ref ...@@ -30,6 +30,8 @@ The introduction and tutorial of Paddle Serving service deployment framework ref
PaddleOCR operating environment and Paddle Serving operating environment are needed. PaddleOCR operating environment and Paddle Serving operating environment are needed.
1. Please prepare PaddleOCR operating environment reference [link](../../doc/doc_ch/installation.md). 1. Please prepare PaddleOCR operating environment reference [link](../../doc/doc_ch/installation.md).
Download the corresponding paddle whl package according to the environment, it is recommended to install version 2.0.1.
2. The steps of PaddleServing operating environment prepare are as follows: 2. The steps of PaddleServing operating environment prepare are as follows:
...@@ -45,23 +47,17 @@ PaddleOCR operating environment and Paddle Serving operating environment are nee ...@@ -45,23 +47,17 @@ PaddleOCR operating environment and Paddle Serving operating environment are nee
``` ```
3. Install the client to send requests to the service 3. Install the client to send requests to the service
``` In [download link](https://github.com/PaddlePaddle/Serving/blob/develop/doc/LATEST_PACKAGES.md) find the client installation package corresponding to the python version.
pip3 install paddle-serving-client==0.5.0 # for CPU The python3.7 version is recommended here:
pip3 install paddle-serving-client-gpu==0.5.0 # for GPU ```
wget https://paddle-serving.bj.bcebos.com/whl/paddle_serving_client-0.0.0-cp37-none-any.whl
pip3 install paddle_serving_client-0.0.0-cp37-none-any.whl
``` ```
4. Install serving-app 4. Install serving-app
``` ```
pip3 install paddle-serving-app==0.3.0 pip3 install paddle-serving-app==0.3.1
# fix local_predict to support load dynamic model
# find the install directoory of paddle_serving_app
vim /usr/local/lib/python3.7/site-packages/paddle_serving_app/local_predict.py
# replace line 85 of local_predict.py config = AnalysisConfig(model_path) with:
if os.path.exists(os.path.join(model_path, "__params__")):
config = AnalysisConfig(os.path.join(model_path, "__model__"), os.path.join(model_path, "__params__"))
else:
config = AnalysisConfig(model_path)
``` ```
**note:** If you want to install the latest version of PaddleServing, refer to [link](https://github.com/PaddlePaddle/Serving/blob/develop/doc/LATEST_PACKAGES.md). **note:** If you want to install the latest version of PaddleServing, refer to [link](https://github.com/PaddlePaddle/Serving/blob/develop/doc/LATEST_PACKAGES.md).
...@@ -74,38 +70,38 @@ When using PaddleServing for service deployment, you need to convert the saved i ...@@ -74,38 +70,38 @@ When using PaddleServing for service deployment, you need to convert the saved i
Firstly, download the [inference model](https://github.com/PaddlePaddle/PaddleOCR#pp-ocr-20-series-model-listupdate-on-dec-15) of PPOCR Firstly, download the [inference model](https://github.com/PaddlePaddle/PaddleOCR#pp-ocr-20-series-model-listupdate-on-dec-15) of PPOCR
``` ```
# Download and unzip the OCR text detection model # Download and unzip the OCR text detection model
wget https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_server_v2.0_det_infer.tar && tar xf ch_ppocr_server_v2.0_det_infer.tar wget https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_mobile_v2.0_det_infer.tar && tar xf ch_ppocr_mobile_v2.0_det_infer.tar
# Download and unzip the OCR text recognition model # Download and unzip the OCR text recognition model
wget https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_server_v2.0_rec_infer.tar && tar xf ch_ppocr_server_v2.0_rec_infer.tar wget https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_mobile_v2.0_rec_infer.tar && tar xf ch_ppocr_mobile_v2.0_rec_infer.tar
``` ```
Then, you can use installed paddle_serving_client tool to convert inference model to server model. Then, you can use installed paddle_serving_client tool to convert inference model to mobile model.
``` ```
# Detection model conversion # Detection model conversion
python3 -m paddle_serving_client.convert --dirname ./ch_ppocr_server_v2.0_det_infer/ \ python3 -m paddle_serving_client.convert --dirname ./ch_ppocr_mobile_v2.0_det_infer/ \
--model_filename inference.pdmodel \ --model_filename inference.pdmodel \
--params_filename inference.pdiparams \ --params_filename inference.pdiparams \
--serving_server ./ppocr_det_server_2.0_serving/ \ --serving_server ./ppocr_det_mobile_2.0_serving/ \
--serving_client ./ppocr_det_server_2.0_client/ --serving_client ./ppocr_det_mobile_2.0_client/
# Recognition model conversion # Recognition model conversion
python3 -m paddle_serving_client.convert --dirname ./ch_ppocr_server_v2.0_rec_infer/ \ python3 -m paddle_serving_client.convert --dirname ./ch_ppocr_mobile_v2.0_rec_infer/ \
--model_filename inference.pdmodel \ --model_filename inference.pdmodel \
--params_filename inference.pdiparams \ --params_filename inference.pdiparams \
--serving_server ./ppocr_rec_server_2.0_serving/ \ --serving_server ./ppocr_rec_mobile_2.0_serving/ \
--serving_client ./ppocr_rec_server_2.0_client/ --serving_client ./ppocr_rec_mobile_2.0_client/
``` ```
After the detection model is converted, there will be additional folders of `ppocr_det_server_2.0_serving` and `ppocr_det_server_2.0_client` in the current folder, with the following format: After the detection model is converted, there will be additional folders of `ppocr_det_mobile_2.0_serving` and `ppocr_det_mobile_2.0_client` in the current folder, with the following format:
``` ```
|- ppocr_det_server_2.0_serving/ |- ppocr_det_mobile_2.0_serving/
|- __model__ |- __model__
|- __params__ |- __params__
|- serving_server_conf.prototxt |- serving_server_conf.prototxt
|- serving_server_conf.stream.prototxt |- serving_server_conf.stream.prototxt
|- ppocr_det_server_2.0_client |- ppocr_det_mobile_2.0_client
|- serving_client_conf.prototxt |- serving_client_conf.prototxt
|- serving_client_conf.stream.prototxt |- serving_client_conf.stream.prototxt
...@@ -147,6 +143,61 @@ The recognition model is the same. ...@@ -147,6 +143,61 @@ The recognition model is the same.
After successfully running, the predicted result of the model will be printed in the cmd window. An example of the result is: After successfully running, the predicted result of the model will be printed in the cmd window. An example of the result is:
![](./imgs/results.png) ![](./imgs/results.png)
Adjust the number of concurrency in config.yml to get the largest QPS. Generally, the number of concurrent detection and recognition is 2:1
```
det:
concurrency: 8
...
rec:
concurrency: 4
...
```
Multiple service requests can be sent at the same time if necessary.
The predicted performance data will be automatically written into the `PipelineServingLogs/pipeline.tracer` file.
Tested on 200 real pictures, and limited the detection long side to 960. The average QPS on T4 GPU can reach around 23:
```
2021-05-13 03:42:36,895 ==================== TRACER ======================
2021-05-13 03:42:36,975 Op(rec):
2021-05-13 03:42:36,976 in[14.472382882882883 ms]
2021-05-13 03:42:36,976 prep[9.556855855855856 ms]
2021-05-13 03:42:36,976 midp[59.921905405405404 ms]
2021-05-13 03:42:36,976 postp[15.345945945945946 ms]
2021-05-13 03:42:36,976 out[1.9921216216216215 ms]
2021-05-13 03:42:36,976 idle[0.16254943864471572]
2021-05-13 03:42:36,976 Op(det):
2021-05-13 03:42:36,976 in[315.4468035714286 ms]
2021-05-13 03:42:36,976 prep[69.5980625 ms]
2021-05-13 03:42:36,976 midp[18.989535714285715 ms]
2021-05-13 03:42:36,976 postp[18.857803571428573 ms]
2021-05-13 03:42:36,977 out[3.1337544642857145 ms]
2021-05-13 03:42:36,977 idle[0.7477961159203756]
2021-05-13 03:42:36,977 DAGExecutor:
2021-05-13 03:42:36,977 Query count[224]
2021-05-13 03:42:36,977 QPS[22.4 q/s]
2021-05-13 03:42:36,977 Succ[0.9910714285714286]
2021-05-13 03:42:36,977 Error req[169, 170]
2021-05-13 03:42:36,977 Latency:
2021-05-13 03:42:36,977 ave[535.1678348214285 ms]
2021-05-13 03:42:36,977 .50[172.651 ms]
2021-05-13 03:42:36,977 .60[187.904 ms]
2021-05-13 03:42:36,977 .70[245.675 ms]
2021-05-13 03:42:36,977 .80[526.684 ms]
2021-05-13 03:42:36,977 .90[854.596 ms]
2021-05-13 03:42:36,977 .95[1722.728 ms]
2021-05-13 03:42:36,977 .99[3990.292 ms]
2021-05-13 03:42:36,978 Channel (server worker num[10]):
2021-05-13 03:42:36,978 chl0(In: ['@DAGExecutor'], Out: ['det']) size[0/0]
2021-05-13 03:42:36,979 chl1(In: ['det'], Out: ['rec']) size[6/0]
2021-05-13 03:42:36,979 chl2(In: ['rec'], Out: ['@DAGExecutor']) size[0/0]
```
<a name="faq"></a> <a name="faq"></a>
## FAQ ## FAQ
**Q1**: No result return after sending the request. **Q1**: No result return after sending the request.
......
deploy/pdserving/README_CN.md
浏览文件 @ 426e3e44
...@@ -29,7 +29,8 @@ PaddleOCR提供2种服务部署方式： ...@@ -29,7 +29,8 @@ PaddleOCR提供2种服务部署方式：
需要准备PaddleOCR的运行环境和Paddle Serving的运行环境。 需要准备PaddleOCR的运行环境和Paddle Serving的运行环境。
- 准备PaddleOCR的运行环境参考[链接](../../doc/doc_ch/installation.md) - 准备PaddleOCR的运行环境[链接](../../doc/doc_ch/installation.md)
根据环境下载对应的paddle whl包，推荐安装2.0.1版本
- 准备PaddleServing的运行环境，步骤如下 - 准备PaddleServing的运行环境，步骤如下
...@@ -45,25 +46,16 @@ PaddleOCR提供2种服务部署方式： ...@@ -45,25 +46,16 @@ PaddleOCR提供2种服务部署方式：
``` ```
2. 安装client，用于向服务发送请求 2. 安装client，用于向服务发送请求
``` 在[下载链接](https://github.com/PaddlePaddle/Serving/blob/develop/doc/LATEST_PACKAGES.md)中找到对应python版本的client安装包，这里推荐python3.7版本：
pip3 install paddle-serving-client==0.5.0 # for CPU
pip3 install paddle-serving-client-gpu==0.5.0 # for GPU ```
wget https://paddle-serving.bj.bcebos.com/whl/paddle_serving_client-0.0.0-cp37-none-any.whl
pip3 install paddle_serving_client-0.0.0-cp37-none-any.whl
``` ```
3. 安装serving-app 3. 安装serving-app
``` ```
pip3 install paddle-serving-app==0.3.0 pip3 install paddle-serving-app==0.3.1
```
**note:** 安装0.3.0版本的serving-app后，为了能加载动态图模型，需要修改serving_app的源码，具体为：
```
# 找到paddle_serving_app的安装目录，找到并编辑local_predict.py文件
vim /usr/local/lib/python3.7/site-packages/paddle_serving_app/local_predict.py
# 将local_predict.py 的第85行 config = AnalysisConfig(model_path) 替换为：
if os.path.exists(os.path.join(model_path, "__params__")):
config = AnalysisConfig(os.path.join(model_path, "__model__"), os.path.join(model_path, "__params__"))
else:
config = AnalysisConfig(model_path)
``` ```
**Note:** 如果要安装最新版本的PaddleServing参考[链接](https://github.com/PaddlePaddle/Serving/blob/develop/doc/LATEST_PACKAGES.md)。 **Note:** 如果要安装最新版本的PaddleServing参考[链接](https://github.com/PaddlePaddle/Serving/blob/develop/doc/LATEST_PACKAGES.md)。
...@@ -76,38 +68,38 @@ PaddleOCR提供2种服务部署方式： ...@@ -76,38 +68,38 @@ PaddleOCR提供2种服务部署方式：
首先，下载PPOCR的[inference模型](https://github.com/PaddlePaddle/PaddleOCR#pp-ocr-20-series-model-listupdate-on-dec-15) 首先，下载PPOCR的[inference模型](https://github.com/PaddlePaddle/PaddleOCR#pp-ocr-20-series-model-listupdate-on-dec-15)
``` ```
# 下载并解压 OCR 文本检测模型 # 下载并解压 OCR 文本检测模型
wget https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_server_v2.0_det_infer.tar && tar xf ch_ppocr_server_v2.0_det_infer.tar wget https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_mobile_v2.0_det_infer.tar && tar xf ch_ppocr_mobile_v2.0_det_infer.tar
# 下载并解压 OCR 文本识别模型 # 下载并解压 OCR 文本识别模型
wget https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_server_v2.0_rec_infer.tar && tar xf ch_ppocr_server_v2.0_rec_infer.tar wget https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_mobile_v2.0_rec_infer.tar && tar xf ch_ppocr_mobile_v2.0_rec_infer.tar
``` ```
接下来，用安装的paddle_serving_client把下载的inference模型转换成易于server部署的模型格式。 接下来，用安装的paddle_serving_client把下载的inference模型转换成易于server部署的模型格式。
``` ```
# 转换检测模型 # 转换检测模型
python3 -m paddle_serving_client.convert --dirname ./ch_ppocr_server_v2.0_det_infer/ \ python3 -m paddle_serving_client.convert --dirname ./ch_ppocr_mobile_v2.0_det_infer/ \
--model_filename inference.pdmodel \ --model_filename inference.pdmodel \
--params_filename inference.pdiparams \ --params_filename inference.pdiparams \
--serving_server ./ppocr_det_server_2.0_serving/ \ --serving_server ./ppocr_det_mobile_2.0_serving/ \
--serving_client ./ppocr_det_server_2.0_client/ --serving_client ./ppocr_det_mobile_2.0_client/
# 转换识别模型 # 转换识别模型
python3 -m paddle_serving_client.convert --dirname ./ch_ppocr_server_v2.0_rec_infer/ \ python3 -m paddle_serving_client.convert --dirname ./ch_ppocr_mobile_v2.0_rec_infer/ \
--model_filename inference.pdmodel \ --model_filename inference.pdmodel \
--params_filename inference.pdiparams \ --params_filename inference.pdiparams \
--serving_server ./ppocr_rec_server_2.0_serving/ \ --serving_server ./ppocr_rec_mobile_2.0_serving/ \
--serving_client ./ppocr_rec_server_2.0_client/ --serving_client ./ppocr_rec_mobile_2.0_client/
``` ```
检测模型转换完成后，会在当前文件夹多出`ppocr_det_server_2.0_serving` 和`ppocr_det_server_2.0_client`的文件夹，具备如下格式： 检测模型转换完成后，会在当前文件夹多出`ppocr_det_mobile_2.0_serving` 和`ppocr_det_mobile_2.0_client`的文件夹，具备如下格式：
``` ```
|- ppocr_det_server_2.0_serving/ |- ppocr_det_mobile_2.0_serving/
|- __model__ |- __model__
|- __params__ |- __params__
|- serving_server_conf.prototxt |- serving_server_conf.prototxt
|- serving_server_conf.stream.prototxt |- serving_server_conf.stream.prototxt
|- ppocr_det_server_2.0_client |- ppocr_det_mobile_2.0_client
|- serving_client_conf.prototxt |- serving_client_conf.prototxt
|- serving_client_conf.stream.prototxt |- serving_client_conf.stream.prototxt
...@@ -148,6 +140,60 @@ python3 -m paddle_serving_client.convert --dirname ./ch_ppocr_server_v2.0_rec_in ...@@ -148,6 +140,60 @@ python3 -m paddle_serving_client.convert --dirname ./ch_ppocr_server_v2.0_rec_in
成功运行后，模型预测的结果会打印在cmd窗口中，结果示例为： 成功运行后，模型预测的结果会打印在cmd窗口中，结果示例为：
![](./imgs/results.png) ![](./imgs/results.png)
调整 config.yml 中的并发个数获得最大的QPS, 一般检测和识别的并发数为2：1
```
det:
#并发数，is_thread_op=True时，为线程并发；否则为进程并发
concurrency: 8
...
rec:
#并发数，is_thread_op=True时，为线程并发；否则为进程并发
concurrency: 4
...
```
有需要的话可以同时发送多个服务请求
预测性能数据会被自动写入 `PipelineServingLogs/pipeline.tracer` 文件中。
在200张真实图片上测试，把检测长边限制为960。T4 GPU 上 QPS 均值可达到23左右：
```
2021-05-13 03:42:36,895 ==================== TRACER ======================
2021-05-13 03:42:36,975 Op(rec):
2021-05-13 03:42:36,976 in[14.472382882882883 ms]
2021-05-13 03:42:36,976 prep[9.556855855855856 ms]
2021-05-13 03:42:36,976 midp[59.921905405405404 ms]
2021-05-13 03:42:36,976 postp[15.345945945945946 ms]
2021-05-13 03:42:36,976 out[1.9921216216216215 ms]
2021-05-13 03:42:36,976 idle[0.16254943864471572]
2021-05-13 03:42:36,976 Op(det):
2021-05-13 03:42:36,976 in[315.4468035714286 ms]
2021-05-13 03:42:36,976 prep[69.5980625 ms]
2021-05-13 03:42:36,976 midp[18.989535714285715 ms]
2021-05-13 03:42:36,976 postp[18.857803571428573 ms]
2021-05-13 03:42:36,977 out[3.1337544642857145 ms]
2021-05-13 03:42:36,977 idle[0.7477961159203756]
2021-05-13 03:42:36,977 DAGExecutor:
2021-05-13 03:42:36,977 Query count[224]
2021-05-13 03:42:36,977 QPS[22.4 q/s]
2021-05-13 03:42:36,977 Succ[0.9910714285714286]
2021-05-13 03:42:36,977 Error req[169, 170]
2021-05-13 03:42:36,977 Latency:
2021-05-13 03:42:36,977 ave[535.1678348214285 ms]
2021-05-13 03:42:36,977 .50[172.651 ms]
2021-05-13 03:42:36,977 .60[187.904 ms]
2021-05-13 03:42:36,977 .70[245.675 ms]
2021-05-13 03:42:36,977 .80[526.684 ms]
2021-05-13 03:42:36,977 .90[854.596 ms]
2021-05-13 03:42:36,977 .95[1722.728 ms]
2021-05-13 03:42:36,977 .99[3990.292 ms]
2021-05-13 03:42:36,978 Channel (server worker num[10]):
2021-05-13 03:42:36,978 chl0(In: ['@DAGExecutor'], Out: ['det']) size[0/0]
2021-05-13 03:42:36,979 chl1(In: ['det'], Out: ['rec']) size[6/0]
2021-05-13 03:42:36,979 chl2(In: ['rec'], Out: ['@DAGExecutor']) size[0/0]
```
<a name="FAQ"></a> <a name="FAQ"></a>
## FAQ ## FAQ
......
deploy/pdserving/config.yml
浏览文件 @ 426e3e44
#rpc端口, rpc_port和http_port不允许同时为空。当rpc_port为空且http_port不为空时，会自动将rpc_port设置为http_port+1 #rpc端口, rpc_port和http_port不允许同时为空。当rpc_port为空且http_port不为空时，会自动将rpc_port设置为http_port+1
rpc_port: 18090 rpc_port: 18091
#http端口, rpc_port和http_port不允许同时为空。当rpc_port可用且http_port为空时，不自动生成http_port #http端口, rpc_port和http_port不允许同时为空。当rpc_port可用且http_port为空时，不自动生成http_port
http_port: 9999 http_port: 9998
#worker_num, 最大并发数。当build_dag_each_worker=True时, 框架会创建worker_num个进程，每个进程内构建grpcSever和DAG #worker_num, 最大并发数。当build_dag_each_worker=True时, 框架会创建worker_num个进程，每个进程内构建grpcSever和DAG
##当build_dag_each_worker=False时，框架会设置主线程grpc线程池的max_workers=worker_num ##当build_dag_each_worker=False时，框架会设置主线程grpc线程池的max_workers=worker_num
worker_num: 20 worker_num: 10
#build_dag_each_worker, False，框架在进程内创建一条DAG；True，框架会每个进程内创建多个独立的DAG #build_dag_each_worker, False，框架在进程内创建一条DAG；True，框架会每个进程内创建多个独立的DAG
build_dag_each_worker: false build_dag_each_worker: False
dag: dag:
#op资源类型, True, 为线程模型；False，为进程模型 #op资源类型, True, 为线程模型；False，为进程模型
is_thread_op: False is_thread_op: False
#重试次数 #重试次数
retry: 1 retry: 10
#使用性能分析, True，生成Timeline性能数据，对性能有一定影响；False为不使用 #使用性能分析, True，生成Timeline性能数据，对性能有一定影响；False为不使用
use_profile: False use_profile: True
tracer: tracer:
interval_s: 10 interval_s: 10
op: op:
det: det:
#并发数，is_thread_op=True时，为线程并发；否则为进程并发 #并发数，is_thread_op=True时，为线程并发；否则为进程并发
concurrency: 4 concurrency: 8
#当op配置没有server_endpoints时，从local_service_conf读取本地服务配置 #当op配置没有server_endpoints时，从local_service_conf读取本地服务配置
local_service_conf: local_service_conf:
...@@ -34,18 +34,18 @@ op: ...@@ -34,18 +34,18 @@ op:
client_type: local_predictor client_type: local_predictor
#det模型路径 #det模型路径
model_config: /paddle/serving/models/det_serving_server/ #ocr_det_model model_config: ./ppocr_det_mobile_2.0_serving
#Fetch结果列表，以client_config中fetch_var的alias_name为准 #Fetch结果列表，以client_config中fetch_var的alias_name为准
fetch_list: ["save_infer_model/scale_0.tmp_1"] fetch_list: ["save_infer_model/scale_0.tmp_1"]
#计算硬件ID，当devices为""或不写时为CPU预测；当devices为"0", "0,1,2"时为GPU预测，表示使用的GPU卡 #计算硬件ID，当devices为""或不写时为CPU预测；当devices为"0", "0,1,2"时为GPU预测，表示使用的GPU卡
devices: "2" devices: "0"
ir_optim: True ir_optim: True
rec: rec:
#并发数，is_thread_op=True时，为线程并发；否则为进程并发 #并发数，is_thread_op=True时，为线程并发；否则为进程并发
concurrency: 1 concurrency: 4
#超时时间, 单位ms #超时时间, 单位ms
timeout: -1 timeout: -1
...@@ -60,12 +60,12 @@ op: ...@@ -60,12 +60,12 @@ op:
client_type: local_predictor client_type: local_predictor
#rec模型路径 #rec模型路径
model_config: /paddle/serving/models/rec_serving_server/ #ocr_rec_model model_config: ./ppocr_rec_mobile_2.0_serving
#Fetch结果列表，以client_config中fetch_var的alias_name为准 #Fetch结果列表，以client_config中fetch_var的alias_name为准
fetch_list: ["save_infer_model/scale_0.tmp_1"] #["ctc_greedy_decoder_0.tmp_0", "softmax_0.tmp_0"] fetch_list: ["save_infer_model/scale_0.tmp_1"]
#计算硬件ID，当devices为""或不写时为CPU预测；当devices为"0", "0,1,2"时为GPU预测，表示使用的GPU卡 #计算硬件ID，当devices为""或不写时为CPU预测；当devices为"0", "0,1,2"时为GPU预测，表示使用的GPU卡
devices: "2" devices: "0"
ir_optim: True ir_optim: True
deploy/pdserving/ocr_reader.py
浏览文件 @ 426e3e44
...@@ -21,7 +21,6 @@ import sys ...@@ -21,7 +21,6 @@ import sys
import argparse import argparse
import string import string
from copy import deepcopy from copy import deepcopy
import paddle
class DetResizeForTest(object): class DetResizeForTest(object):
...@@ -34,12 +33,12 @@ class DetResizeForTest(object): ...@@ -34,12 +33,12 @@ class DetResizeForTest(object):
elif 'limit_side_len' in kwargs: elif 'limit_side_len' in kwargs:
self.limit_side_len = kwargs['limit_side_len'] self.limit_side_len = kwargs['limit_side_len']
self.limit_type = kwargs.get('limit_type', 'min') self.limit_type = kwargs.get('limit_type', 'min')
elif 'resize_long' in kwargs: elif 'resize_short' in kwargs:
self.resize_type = 2
self.resize_long = kwargs.get('resize_long', 960)
else:
self.limit_side_len = 736 self.limit_side_len = 736
self.limit_type = 'min' self.limit_type = 'min'
else:
self.resize_type = 2
self.resize_long = kwargs.get('resize_long', 960)
def __call__(self, data): def __call__(self, data):
img = deepcopy(data) img = deepcopy(data)
...@@ -227,8 +226,6 @@ class CTCLabelDecode(BaseRecLabelDecode): ...@@ -227,8 +226,6 @@ class CTCLabelDecode(BaseRecLabelDecode):
super(CTCLabelDecode, self).__init__(config) super(CTCLabelDecode, self).__init__(config)
def __call__(self, preds, label=None, *args, **kwargs): def __call__(self, preds, label=None, *args, **kwargs):
if isinstance(preds, paddle.Tensor):
preds = preds.numpy()
preds_idx = preds.argmax(axis=2) preds_idx = preds.argmax(axis=2)
preds_prob = preds.max(axis=2) preds_prob = preds.max(axis=2)
text = self.decode(preds_idx, preds_prob, is_remove_duplicate=True) text = self.decode(preds_idx, preds_prob, is_remove_duplicate=True)
......
deploy/pdserving/pipeline_http_client.py
浏览文件 @ 426e3e44
...@@ -23,8 +23,8 @@ def cv2_to_base64(image): ...@@ -23,8 +23,8 @@ def cv2_to_base64(image):
return base64.b64encode(image).decode('utf8') return base64.b64encode(image).decode('utf8')
url = "http://127.0.0.1:9999/ocr/prediction" url = "http://127.0.0.1:9998/ocr/prediction"
test_img_dir = "../doc/imgs/" test_img_dir = "../../doc/imgs/"
for idx, img_file in enumerate(os.listdir(test_img_dir)): for idx, img_file in enumerate(os.listdir(test_img_dir)):
with open(os.path.join(test_img_dir, img_file), 'rb') as file: with open(os.path.join(test_img_dir, img_file), 'rb') as file:
image_data1 = file.read() image_data1 = file.read()
...@@ -36,5 +36,5 @@ for idx, img_file in enumerate(os.listdir(test_img_dir)): ...@@ -36,5 +36,5 @@ for idx, img_file in enumerate(os.listdir(test_img_dir)):
r = requests.post(url=url, data=json.dumps(data)) r = requests.post(url=url, data=json.dumps(data))
print(r.json()) print(r.json())
test_img_dir = "../doc/imgs/" test_img_dir = "../../doc/imgs/"
print("==> total number of test imgs: ", len(os.listdir(test_img_dir))) print("==> total number of test imgs: ", len(os.listdir(test_img_dir)))
deploy/pdserving/pipeline_rpc_client.py
浏览文件 @ 426e3e44
...@@ -23,7 +23,7 @@ import base64 ...@@ -23,7 +23,7 @@ import base64
import os import os
client = PipelineClient() client = PipelineClient()
client.connect(['127.0.0.1:18090']) client.connect(['127.0.0.1:18091'])
def cv2_to_base64(image): def cv2_to_base64(image):
...@@ -39,4 +39,3 @@ for img_file in os.listdir(test_img_dir): ...@@ -39,4 +39,3 @@ for img_file in os.listdir(test_img_dir):
for i in range(1): for i in range(1):
ret = client.predict(feed_dict={"image": image}, fetch=["res"]) ret = client.predict(feed_dict={"image": image}, fetch=["res"])
print(ret) print(ret)
#print(ret)
deploy/pdserving/web_service.py
浏览文件 @ 426e3e44
...@@ -48,28 +48,24 @@ class DetOp(Op): ...@@ -48,28 +48,24 @@ class DetOp(Op):
def preprocess(self, input_dicts, data_id, log_id): def preprocess(self, input_dicts, data_id, log_id):
(_, input_dict), = input_dicts.items() (_, input_dict), = input_dicts.items()
data = base64.b64decode(input_dict["image"].encode('utf8')) data = base64.b64decode(input_dict["image"].encode('utf8'))
self.raw_im = data
data = np.fromstring(data, np.uint8) data = np.fromstring(data, np.uint8)
# Note: class variables(self.var) can only be used in process op mode # Note: class variables(self.var) can only be used in process op mode
im = cv2.imdecode(data, cv2.IMREAD_COLOR) im = cv2.imdecode(data, cv2.IMREAD_COLOR)
self.im = im
self.ori_h, self.ori_w, _ = im.shape self.ori_h, self.ori_w, _ = im.shape
det_img = self.det_preprocess(im)
det_img = self.det_preprocess(self.im)
_, self.new_h, self.new_w = det_img.shape _, self.new_h, self.new_w = det_img.shape
print("det image shape", det_img.shape)
return {"x": det_img[np.newaxis, :].copy()}, False, None, "" return {"x": det_img[np.newaxis, :].copy()}, False, None, ""
def postprocess(self, input_dicts, fetch_dict, log_id): def postprocess(self, input_dicts, fetch_dict, log_id):
print("input_dicts: ", input_dicts)
det_out = fetch_dict["save_infer_model/scale_0.tmp_1"] det_out = fetch_dict["save_infer_model/scale_0.tmp_1"]
ratio_list = [ ratio_list = [
float(self.new_h) / self.ori_h, float(self.new_w) / self.ori_w float(self.new_h) / self.ori_h, float(self.new_w) / self.ori_w
] ]
dt_boxes_list = self.post_func(det_out, [ratio_list]) dt_boxes_list = self.post_func(det_out, [ratio_list])
dt_boxes = self.filter_func(dt_boxes_list[0], [self.ori_h, self.ori_w]) dt_boxes = self.filter_func(dt_boxes_list[0], [self.ori_h, self.ori_w])
out_dict = {"dt_boxes": dt_boxes, "image": self.im} out_dict = {"dt_boxes": dt_boxes, "image": self.raw_im}
print("out dict", out_dict["dt_boxes"])
return out_dict, None, "" return out_dict, None, ""
...@@ -83,14 +79,43 @@ class RecOp(Op): ...@@ -83,14 +79,43 @@ class RecOp(Op):
def preprocess(self, input_dicts, data_id, log_id): def preprocess(self, input_dicts, data_id, log_id):
(_, input_dict), = input_dicts.items() (_, input_dict), = input_dicts.items()
im = input_dict["image"] raw_im = input_dict["image"]
data = np.frombuffer(raw_im, np.uint8)
im = cv2.imdecode(data, cv2.IMREAD_COLOR)
dt_boxes = input_dict["dt_boxes"] dt_boxes = input_dict["dt_boxes"]
dt_boxes = self.sorted_boxes(dt_boxes) dt_boxes = self.sorted_boxes(dt_boxes)
feed_list = [] feed_list = []
img_list = [] img_list = []
max_wh_ratio = 0 max_wh_ratio = 0
for i, dtbox in enumerate(dt_boxes): ## Many mini-batchs, the type of feed_data is list.
boximg = self.get_rotate_crop_image(im, dt_boxes[i]) max_batch_size = 6 # len(dt_boxes)
# If max_batch_size is 0, skipping predict stage
if max_batch_size == 0:
return {}, True, None, ""
boxes_size = len(dt_boxes)
batch_size = boxes_size // max_batch_size
rem = boxes_size % max_batch_size
for bt_idx in range(0, batch_size + 1):
imgs = None
boxes_num_in_one_batch = 0
if bt_idx == batch_size:
if rem == 0:
continue
else:
boxes_num_in_one_batch = rem
elif bt_idx < batch_size:
boxes_num_in_one_batch = max_batch_size
else:
_LOGGER.error("batch_size error, bt_idx={}, batch_size={}".
format(bt_idx, batch_size))
break
start = bt_idx * max_batch_size
end = start + boxes_num_in_one_batch
img_list = []
for box_idx in range(start, end):
boximg = self.get_rotate_crop_image(im, dt_boxes[box_idx])
img_list.append(boximg) img_list.append(boximg)
h, w = boximg.shape[0:2] h, w = boximg.shape[0:2]
wh_ratio = w * 1.0 / h wh_ratio = w * 1.0 / h
...@@ -98,20 +123,31 @@ class RecOp(Op): ...@@ -98,20 +123,31 @@ class RecOp(Op):
_, w, h = self.ocr_reader.resize_norm_img(img_list[0], _, w, h = self.ocr_reader.resize_norm_img(img_list[0],
max_wh_ratio).shape max_wh_ratio).shape
imgs = np.zeros((len(img_list), 3, w, h)).astype('float32') imgs = np.zeros((boxes_num_in_one_batch, 3, w, h)).astype('float32')
for id, img in enumerate(img_list): for id, img in enumerate(img_list):
norm_img = self.ocr_reader.resize_norm_img(img, max_wh_ratio) norm_img = self.ocr_reader.resize_norm_img(img, max_wh_ratio)
imgs[id] = norm_img imgs[id] = norm_img
print("rec image shape", imgs.shape)
feed = {"x": imgs.copy()} feed = {"x": imgs.copy()}
return feed, False, None, "" feed_list.append(feed)
def postprocess(self, input_dicts, fetch_dict, log_id): return feed_list, False, None, ""
rec_res = self.ocr_reader.postprocess(fetch_dict, with_score=True)
res_lst = [] def postprocess(self, input_dicts, fetch_data, log_id):
for res in rec_res: res_list = []
res_lst.append(res[0]) if isinstance(fetch_data, dict):
res = {"res": str(res_lst)} if len(fetch_data) > 0:
rec_batch_res = self.ocr_reader.postprocess(
fetch_data, with_score=True)
for res in rec_batch_res:
res_list.append(res[0])
elif isinstance(fetch_data, list):
for one_batch in fetch_data:
one_batch_res = self.ocr_reader.postprocess(
one_batch, with_score=True)
for res in one_batch_res:
res_list.append(res[0])
res = {"res": str(res_list)}
return res, None, "" return res, None, ""
......
Markdown is supported
0% 或 .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑！
想要评论请 注册 或