Skip to content

S
Serving

PaddlePaddle / Serving
大约 2 年 前同步成功

通知 187
Star 833
Fork 253
S
Serving
提交 12ab1380 编写于 作者: T TeslaZhao
浏览文件
操作

House cleaning

上级 d2d95908
展开全部 隐藏空白更改
内联 并排
Showing with 804 addition and 551 deletion
python/examples/bert/README.md
浏览文件 @ 12ab1380
...@@ -45,22 +45,22 @@ this script will download Chinese Dictionary File vocab.txt and Chinese Sample D ...@@ -45,22 +45,22 @@ this script will download Chinese Dictionary File vocab.txt and Chinese Sample D
### Inference Service(Support BRPC-Client、GRPC-Client、Http-Client) ### Inference Service(Support BRPC-Client、GRPC-Client、Http-Client)
start cpu inference service,Run start cpu inference service,Run
``` ```
python -m paddle_serving_server.serve --model bert_seq128_model/ --port 9292 #cpu inference service python3 -m paddle_serving_server.serve --model bert_seq128_model/ --port 9292 #cpu inference service
``` ```
Or,start gpu inference service,Run Or,start gpu inference service,Run
``` ```
python -m paddle_serving_server.serve --model bert_seq128_model/ --port 9292 --gpu_ids 0 #launch gpu inference service at GPU 0 python3 -m paddle_serving_server.serve --model bert_seq128_model/ --port 9292 --gpu_ids 0 #launch gpu inference service at GPU 0
``` ```
### BRPC-Client Inference ### BRPC-Client Inference
before prediction we should install paddle_serving_app. This module provides data preprocessing for BERT model. before prediction we should install paddle_serving_app. This module provides data preprocessing for BERT model.
``` ```
pip install paddle_serving_app pip3 install paddle_serving_app
``` ```
Run Run
``` ```
head data-c.txt | python bert_client.py --model bert_seq128_client/serving_client_conf.prototxt head data-c.txt | python3 bert_client.py --model bert_seq128_client/serving_client_conf.prototxt
``` ```
the client reads data from data-c.txt and send prediction request, the prediction is given by word vector. (Due to massive data in the word vector, we do not print it). the client reads data from data-c.txt and send prediction request, the prediction is given by word vector. (Due to massive data in the word vector, we do not print it).
...@@ -68,7 +68,7 @@ the client reads data from data-c.txt and send prediction request, the predictio ...@@ -68,7 +68,7 @@ the client reads data from data-c.txt and send prediction request, the predictio
#### GRPC-Client/HTTP-Client #### GRPC-Client/HTTP-Client
Run Run
``` ```
head data-c.txt | python bert_httpclient.py --model bert_seq128_client/serving_client_conf.prototxt head data-c.txt | python3 bert_httpclient.py --model bert_seq128_client/serving_client_conf.prototxt
``` ```
......
python/examples/bert/README_CN.md
浏览文件 @ 12ab1380
...@@ -43,12 +43,12 @@ sh get_data.sh ...@@ -43,12 +43,12 @@ sh get_data.sh
### 启动预测服务（支持BRPC-Client、GRPC-Client、HTTP-Client三种方式访问） ### 启动预测服务（支持BRPC-Client、GRPC-Client、HTTP-Client三种方式访问）
启动cpu预测服务，执行 启动cpu预测服务，执行
``` ```
python -m paddle_serving_server.serve --model bert_seq128_model/ --port 9292 #启动cpu预测服务 python3 -m paddle_serving_server.serve --model bert_seq128_model/ --port 9292 #启动cpu预测服务
``` ```
或者，启动gpu预测服务，执行 或者，启动gpu预测服务，执行
``` ```
python -m paddle_serving_server.serve --model bert_seq128_model/ --port 9292 --gpu_ids 0 #在gpu 0上启动gpu预测服务 python3 -m paddle_serving_server.serve --model bert_seq128_model/ --port 9292 --gpu_ids 0 #在gpu 0上启动gpu预测服务
``` ```
...@@ -56,13 +56,13 @@ python -m paddle_serving_server.serve --model bert_seq128_model/ --port 9292 --g ...@@ -56,13 +56,13 @@ python -m paddle_serving_server.serve --model bert_seq128_model/ --port 9292 --g
执行预测前需要安装paddle_serving_app，模块中提供了BERT模型的数据预处理方法。 执行预测前需要安装paddle_serving_app，模块中提供了BERT模型的数据预处理方法。
``` ```
pip install paddle_serving_app pip3 install paddle_serving_app
``` ```
#### BRPC-Client #### BRPC-Client
执行 执行
``` ```
head data-c.txt | python bert_client.py --model bert_seq128_client/serving_client_conf.prototxt head data-c.txt | python3 bert_client.py --model bert_seq128_client/serving_client_conf.prototxt
``` ```
启动client读取data-c.txt中的数据进行预测，预测结果为文本的向量表示（由于数据较多，脚本中没有将输出进行打印），server端的地址在脚本中修改。 启动client读取data-c.txt中的数据进行预测，预测结果为文本的向量表示（由于数据较多，脚本中没有将输出进行打印），server端的地址在脚本中修改。
...@@ -70,7 +70,7 @@ head data-c.txt | python bert_client.py --model bert_seq128_client/serving_clien ...@@ -70,7 +70,7 @@ head data-c.txt | python bert_client.py --model bert_seq128_client/serving_clien
#### GRPC-Client/HTTP-Client #### GRPC-Client/HTTP-Client
执行 执行
``` ```
head data-c.txt | python bert_httpclient.py --model bert_seq128_client/serving_client_conf.prototxt head data-c.txt | python3 bert_httpclient.py --model bert_seq128_client/serving_client_conf.prototxt
``` ```
......
python/examples/blazeface/README.md
浏览文件 @ 12ab1380
...@@ -2,7 +2,7 @@ ...@@ -2,7 +2,7 @@
## Get Model ## Get Model
``` ```
python -m paddle_serving_app.package --get_model blazeface python3 -m paddle_serving_app.package --get_model blazeface
tar -xf blazeface.tar.gz tar -xf blazeface.tar.gz
``` ```
...@@ -11,13 +11,13 @@ tar -xf blazeface.tar.gz ...@@ -11,13 +11,13 @@ tar -xf blazeface.tar.gz
### Start Service ### Start Service
``` ```
python -m paddle_serving_server.serve --model serving_server --port 9494 python3 -m paddle_serving_server.serve --model serving_server --port 9494
``` ```
### Client Prediction ### Client Prediction
``` ```
python test_client.py serving_client/serving_client_conf.prototxt test.jpg python3 test_client.py serving_client/serving_client_conf.prototxt test.jpg
``` ```
the result is in `output` folder, including a json file and image file with bounding boxes. the result is in `output` folder, including a json file and image file with bounding boxes.
python/examples/cascade_rcnn/README.md
浏览文件 @ 12ab1380
...@@ -10,12 +10,12 @@ If you want to have more detection models, please refer to [Paddle Detection Mod ...@@ -10,12 +10,12 @@ If you want to have more detection models, please refer to [Paddle Detection Mod
### Start the service ### Start the service
``` ```
python -m paddle_serving_server.serve --model serving_server --port 9292 --gpu_id 0 python3 -m paddle_serving_server.serve --model serving_server --port 9292 --gpu_id 0
``` ```
### Perform prediction ### Perform prediction
``` ```
python test_client.py python3 test_client.py
``` ```
Image with bounding boxes and json result would be saved in `output` folder. Image with bounding boxes and json result would be saved in `output` folder.
python/examples/cascade_rcnn/README_CN.md
浏览文件 @ 12ab1380
...@@ -10,12 +10,12 @@ sh get_data.sh ...@@ -10,12 +10,12 @@ sh get_data.sh
### 启动服务 ### 启动服务
``` ```
python -m paddle_serving_server.serve --model serving_server --port 9292 --gpu_id 0 python3 -m paddle_serving_server.serve --model serving_server --port 9292 --gpu_id 0
``` ```
### 执行预测 ### 执行预测
``` ```
python test_client.py python3 test_client.py
``` ```
客户端已经为图片做好了后处理，在`output`文件夹下存放各个框的json格式信息还有后处理结果图片。 客户端已经为图片做好了后处理，在`output`文件夹下存放各个框的json格式信息还有后处理结果图片。
python/examples/criteo_ctr/README.md
浏览文件 @ 12ab1380
...@@ -19,13 +19,13 @@ the directories like `ctr_serving_model` and `ctr_client_conf` will appear. ...@@ -19,13 +19,13 @@ the directories like `ctr_serving_model` and `ctr_client_conf` will appear.
### Start RPC Inference Service ### Start RPC Inference Service
``` ```
python -m paddle_serving_server.serve --model ctr_serving_model/ --port 9292 #CPU RPC Service python3 -m paddle_serving_server.serve --model ctr_serving_model/ --port 9292 #CPU RPC Service
python -m paddle_serving_server.serve --model ctr_serving_model/ --port 9292 --gpu_ids 0 #RPC Service on GPU 0 python3 -m paddle_serving_server.serve --model ctr_serving_model/ --port 9292 --gpu_ids 0 #RPC Service on GPU 0
``` ```
### RPC Infer ### RPC Infer
``` ```
python test_client.py ctr_client_conf/serving_client_conf.prototxt raw_data/part-0 python3 test_client.py ctr_client_conf/serving_client_conf.prototxt raw_data/part-0
``` ```
the latency will display in the end. the latency will display in the end.
python/examples/criteo_ctr/README_CN.md
浏览文件 @ 12ab1380
...@@ -19,13 +19,13 @@ mv models/ctr_serving_model . ...@@ -19,13 +19,13 @@ mv models/ctr_serving_model .
### 启动RPC预测服务 ### 启动RPC预测服务
``` ```
python -m paddle_serving_server.serve --model ctr_serving_model/ --port 9292 #启动CPU预测服务 python3 -m paddle_serving_server.serve --model ctr_serving_model/ --port 9292 #启动CPU预测服务
python -m paddle_serving_server.serve --model ctr_serving_model/ --port 9292 --gpu_ids 0 #在GPU 0上启动预测服务 python3 -m paddle_serving_server.serve --model ctr_serving_model/ --port 9292 --gpu_ids 0 #在GPU 0上启动预测服务
``` ```
### 执行预测 ### 执行预测
``` ```
python test_client.py ctr_client_conf/serving_client_conf.prototxt raw_data/part-0 python3 test_client.py ctr_client_conf/serving_client_conf.prototxt raw_data/part-0
``` ```
预测完毕会输出预测过程的耗时。 预测完毕会输出预测过程的耗时。
python/examples/criteo_ctr_with_cube/README.md
浏览文件 @ 12ab1380
...@@ -32,13 +32,13 @@ Here, the sparse parameter is loaded by cube sparse parameter indexing service C ...@@ -32,13 +32,13 @@ Here, the sparse parameter is loaded by cube sparse parameter indexing service C
### Start RPC Predictor, the number of serving thread is 4（configurable in test_server.py） ### Start RPC Predictor, the number of serving thread is 4（configurable in test_server.py）
``` ```
python test_server.py ctr_serving_model_kv python3 test_server.py ctr_serving_model_kv
``` ```
### Run Prediction ### Run Prediction
``` ```
python test_client.py ctr_client_conf/serving_client_conf.prototxt ./raw_data python3 test_client.py ctr_client_conf/serving_client_conf.prototxt ./raw_data
``` ```
### Benchmark ### Benchmark
......
python/examples/criteo_ctr_with_cube/README_CN.md
浏览文件 @ 12ab1380
...@@ -30,13 +30,13 @@ sh cube_prepare.sh & ...@@ -30,13 +30,13 @@ sh cube_prepare.sh &
### 启动RPC预测服务，服务端线程数为4（可在test_server.py配置） ### 启动RPC预测服务，服务端线程数为4（可在test_server.py配置）
``` ```
python test_server.py ctr_serving_model_kv python3 test_server.py ctr_serving_model_kv
``` ```
### 执行预测 ### 执行预测
``` ```
python test_client.py ctr_client_conf/serving_client_conf.prototxt ./raw_data python3 test_client.py ctr_client_conf/serving_client_conf.prototxt ./raw_data
``` ```
### Benchmark ### Benchmark
......
python/examples/deeplabv3/README.md
浏览文件 @ 12ab1380
...@@ -3,7 +3,7 @@ ...@@ -3,7 +3,7 @@
## Get Model ## Get Model
``` ```
python -m paddle_serving_app.package --get_model deeplabv3 python3 -m paddle_serving_app.package --get_model deeplabv3
tar -xzvf deeplabv3.tar.gz tar -xzvf deeplabv3.tar.gz
``` ```
...@@ -12,11 +12,11 @@ tar -xzvf deeplabv3.tar.gz ...@@ -12,11 +12,11 @@ tar -xzvf deeplabv3.tar.gz
### Start Service ### Start Service
``` ```
python -m paddle_serving_server.serve --model deeplabv3_server --gpu_ids 0 --port 9494 python3 -m paddle_serving_server.serve --model deeplabv3_server --gpu_ids 0 --port 9494
``` ```
### Client Prediction ### Client Prediction
``` ```
python deeplabv3_client.py python3 deeplabv3_client.py
``` ```
python/examples/deeplabv3/README_CN.md
浏览文件 @ 12ab1380
...@@ -3,7 +3,7 @@ ...@@ -3,7 +3,7 @@
## 获取模型 ## 获取模型
``` ```
python -m paddle_serving_app.package --get_model deeplabv3 python3 -m paddle_serving_app.package --get_model deeplabv3
tar -xzvf deeplabv3.tar.gz tar -xzvf deeplabv3.tar.gz
``` ```
...@@ -12,10 +12,10 @@ tar -xzvf deeplabv3.tar.gz ...@@ -12,10 +12,10 @@ tar -xzvf deeplabv3.tar.gz
### 启动服务端 ### 启动服务端
``` ```
python -m paddle_serving_server.serve --model deeplabv3_server --gpu_ids 0 --port 9494 python3 -m paddle_serving_server.serve --model deeplabv3_server --gpu_ids 0 --port 9494
``` ```
### 客户端预测 ### 客户端预测
``` ```
python deeplabv3_client.py python3 deeplabv3_client.py
python/examples/detection/faster_rcnn_hrnetv2p_w18_1x/README.md
浏览文件 @ 12ab1380
...@@ -10,7 +10,7 @@ wget --no-check-certificate https://paddle-serving.bj.bcebos.com/pddet_demo/2.0/ ...@@ -10,7 +10,7 @@ wget --no-check-certificate https://paddle-serving.bj.bcebos.com/pddet_demo/2.0/
### Start the service ### Start the service
``` ```
tar xf faster_rcnn_hrnetv2p_w18_1x.tar tar xf faster_rcnn_hrnetv2p_w18_1x.tar
python -m paddle_serving_server.serve --model serving_server --port 9494 --gpu_ids 0 python3 -m paddle_serving_server.serve --model serving_server --port 9494 --gpu_ids 0
``` ```
This model support TensorRT, if you want a faster inference, please use `--use_trt`. But you need to do some extra work. This model support TensorRT, if you want a faster inference, please use `--use_trt`. But you need to do some extra work.
...@@ -19,5 +19,5 @@ Please reference to https://github.com/PaddlePaddle/Paddle-Inference-Demo/blob/m ...@@ -19,5 +19,5 @@ Please reference to https://github.com/PaddlePaddle/Paddle-Inference-Demo/blob/m
### Prediction ### Prediction
``` ```
python test_client.py 000000570688.jpg python3 test_client.py 000000570688.jpg
``` ```
python/examples/detection/faster_rcnn_hrnetv2p_w18_1x/README_CN.md
浏览文件 @ 12ab1380
...@@ -11,12 +11,12 @@ wget --no-check-certificate https://paddle-serving.bj.bcebos.com/pddet_demo/2.0/ ...@@ -11,12 +11,12 @@ wget --no-check-certificate https://paddle-serving.bj.bcebos.com/pddet_demo/2.0/
### 启动服务 ### 启动服务
``` ```
tar xf faster_rcnn_hrnetv2p_w18_1x.tar tar xf faster_rcnn_hrnetv2p_w18_1x.tar
python -m paddle_serving_server.serve --model serving_server --port 9494 --gpu_ids 0 python3 -m paddle_serving_server.serve --model serving_server --port 9494 --gpu_ids 0
``` ```
该模型支持TensorRT，如果想要更快的预测速度，可以开启`--use_trt`选项,但此时需要额外设置子图的TRT变长最大最小最优shape. 该模型支持TensorRT，如果想要更快的预测速度，可以开启`--use_trt`选项,但此时需要额外设置子图的TRT变长最大最小最优shape.
请参考https://github.com/PaddlePaddle/Paddle-Inference-Demo/blob/master/c%2B%2B/paddle-trt/trt_dynamic_shape_test.cc#L40 请参考https://github.com/PaddlePaddle/Paddle-Inference-Demo/blob/master/c%2B%2B/paddle-trt/trt_dynamic_shape_test.cc#L40
### 执行预测 ### 执行预测
``` ```
python test_client.py 000000570688.jpg python3 test_client.py 000000570688.jpg
``` ```
python/examples/detection/faster_rcnn_r50_fpn_1x_coco/README.md
浏览文件 @ 12ab1380
...@@ -10,7 +10,7 @@ wget --no-check-certificate https://paddle-serving.bj.bcebos.com/pddet_demo/2.0/ ...@@ -10,7 +10,7 @@ wget --no-check-certificate https://paddle-serving.bj.bcebos.com/pddet_demo/2.0/
### Start the service ### Start the service
``` ```
tar xf faster_rcnn_r50_fpn_1x_coco.tar tar xf faster_rcnn_r50_fpn_1x_coco.tar
python -m paddle_serving_server.serve --model serving_server --port 9494 --gpu_ids 0 python3 -m paddle_serving_server.serve --model serving_server --port 9494 --gpu_ids 0
``` ```
This model support TensorRT, if you want a faster inference, please use `--use_trt`. But you need to do some extra work. This model support TensorRT, if you want a faster inference, please use `--use_trt`. But you need to do some extra work.
...@@ -19,7 +19,7 @@ Please reference to https://github.com/PaddlePaddle/Paddle-Inference-Demo/blob/m ...@@ -19,7 +19,7 @@ Please reference to https://github.com/PaddlePaddle/Paddle-Inference-Demo/blob/m
### Perform prediction ### Perform prediction
``` ```
python test_client.py 000000570688.jpg python3 test_client.py 000000570688.jpg
``` ```
## 3. Result analysis ## 3. Result analysis
......
python/examples/detection/faster_rcnn_r50_fpn_1x_coco/README_CN.md
浏览文件 @ 12ab1380
...@@ -11,14 +11,14 @@ wget --no-check-certificate https://paddle-serving.bj.bcebos.com/pddet_demo/2.0/ ...@@ -11,14 +11,14 @@ wget --no-check-certificate https://paddle-serving.bj.bcebos.com/pddet_demo/2.0/
### 启动服务 ### 启动服务
``` ```
tar xf faster_rcnn_r50_fpn_1x_coco.tar tar xf faster_rcnn_r50_fpn_1x_coco.tar
python -m paddle_serving_server.serve --model serving_server --port 9494 --gpu_ids 0 python3 -m paddle_serving_server.serve --model serving_server --port 9494 --gpu_ids 0
``` ```
该模型支持TensorRT，如果想要更快的预测速度，可以开启`--use_trt`选项,但此时需要额外设置子图的TRT变长最大最小最优shape. 该模型支持TensorRT，如果想要更快的预测速度，可以开启`--use_trt`选项,但此时需要额外设置子图的TRT变长最大最小最优shape.
请参考https://github.com/PaddlePaddle/Paddle-Inference-Demo/blob/master/c%2B%2B/paddle-trt/trt_dynamic_shape_test.cc#L40 请参考https://github.com/PaddlePaddle/Paddle-Inference-Demo/blob/master/c%2B%2B/paddle-trt/trt_dynamic_shape_test.cc#L40
### 执行预测 ### 执行预测
``` ```
python test_client.py 000000570688.jpg python3 test_client.py 000000570688.jpg
``` ```
## 3. 结果分析 ## 3. 结果分析
......
python/examples/detection/fcos_dcn_r50_fpn_1x_coco/README.md
浏览文件 @ 12ab1380
...@@ -10,11 +10,11 @@ wget --no-check-certificate https://paddle-serving.bj.bcebos.com/pddet_demo/2.0/ ...@@ -10,11 +10,11 @@ wget --no-check-certificate https://paddle-serving.bj.bcebos.com/pddet_demo/2.0/
### Start the service ### Start the service
``` ```
tar xf fcos_dcn_r50_fpn_1x_coco.tar tar xf fcos_dcn_r50_fpn_1x_coco.tar
python -m paddle_serving_server.serve --model serving_server --port 9494 --gpu_ids 0 python3 -m paddle_serving_server.serve --model serving_server --port 9494 --gpu_ids 0
``` ```
This model support TensorRT, if you want a faster inference, please use `--use_trt`. This model support TensorRT, if you want a faster inference, please use `--use_trt`.
### Perform prediction ### Perform prediction
``` ```
python test_client.py 000000570688.jpg python3 test_client.py 000000570688.jpg
``` ```
python/examples/detection/fcos_dcn_r50_fpn_1x_coco/README_CN.md
浏览文件 @ 12ab1380
...@@ -11,12 +11,12 @@ wget --no-check-certificate https://paddle-serving.bj.bcebos.com/pddet_demo/2.0/ ...@@ -11,12 +11,12 @@ wget --no-check-certificate https://paddle-serving.bj.bcebos.com/pddet_demo/2.0/
### 启动服务 ### 启动服务
``` ```
tar xf fcos_dcn_r50_fpn_1x_coco.tar tar xf fcos_dcn_r50_fpn_1x_coco.tar
python -m paddle_serving_server.serve --model serving_server --port 9494 --gpu_ids 0 python3 -m paddle_serving_server.serve --model serving_server --port 9494 --gpu_ids 0
``` ```
该模型支持TensorRT，如果想要更快的预测速度，可以开启`--use_trt`选项。 该模型支持TensorRT，如果想要更快的预测速度，可以开启`--use_trt`选项。
### 执行预测 ### 执行预测
``` ```
python test_client.py 000000570688.jpg python3 test_client.py 000000570688.jpg
``` ```
python/examples/detection/ppyolo_r50vd_dcn_1x_coco/README.md
浏览文件 @ 12ab1380
...@@ -10,13 +10,12 @@ wget --no-check-certificate https://paddle-serving.bj.bcebos.com/pddet_demo/2.0/ ...@@ -10,13 +10,12 @@ wget --no-check-certificate https://paddle-serving.bj.bcebos.com/pddet_demo/2.0/
### Start the service ### Start the service
``` ```
tar xf ppyolo_r50vd_dcn_1x_coco.tar tar xf ppyolo_r50vd_dcn_1x_coco.tar
python -m paddle_serving_server.serve --model serving_server --port 9494 --gpu_ids 0 python3 -m paddle_serving_server.serve --model serving_server --port 9494 --gpu_ids 0
``` ```
This model support TensorRT, if you want a faster inference, please use `--use_trt`. This model support TensorRT, if you want a faster inference, please use `--use_trt`.
### Perform prediction ### Perform prediction
``` ```
python test_client.py 000000570688.jpg python3 test_client.py 000000570688.jpg
``` ```
python/examples/detection/ppyolo_r50vd_dcn_1x_coco/README_CN.md
浏览文件 @ 12ab1380
...@@ -11,13 +11,12 @@ wget --no-check-certificate https://paddle-serving.bj.bcebos.com/pddet_demo/2.0/ ...@@ -11,13 +11,12 @@ wget --no-check-certificate https://paddle-serving.bj.bcebos.com/pddet_demo/2.0/
### 启动服务 ### 启动服务
``` ```
tar xf ppyolo_r50vd_dcn_1x_coco.tar tar xf ppyolo_r50vd_dcn_1x_coco.tar
python -m paddle_serving_server.serve --model serving_server --port 9494 --gpu_ids 0 python3 -m paddle_serving_server.serve --model serving_server --port 9494 --gpu_ids 0
``` ```
该模型支持TensorRT，如果想要更快的预测速度，可以开启`--use_trt`选项。 该模型支持TensorRT，如果想要更快的预测速度，可以开启`--use_trt`选项。
### 执行预测 ### 执行预测
``` ```
python test_client.py 000000570688.jpg python3 test_client.py 000000570688.jpg
``` ```
python/examples/detection/ssd_vgg16_300_240e_voc/README.md
浏览文件 @ 12ab1380
...@@ -10,11 +10,11 @@ wget --no-check-certificate https://paddle-serving.bj.bcebos.com/pddet_demo/2.0/ ...@@ -10,11 +10,11 @@ wget --no-check-certificate https://paddle-serving.bj.bcebos.com/pddet_demo/2.0/
### Start the service ### Start the service
``` ```
tar xf ssd_vgg16_300_240e_voc.tar tar xf ssd_vgg16_300_240e_voc.tar
python -m paddle_serving_server.serve --model serving_server --port 9494 --gpu_ids 0 python3 -m paddle_serving_server.serve --model serving_server --port 9494 --gpu_ids 0
``` ```
This model support TensorRT, if you want a faster inference, please use `--use_trt`. This model support TensorRT, if you want a faster inference, please use `--use_trt`.
### Perform prediction ### Perform prediction
``` ```
python test_client.py 000000570688.jpg python3 test_client.py 000000570688.jpg
``` ```
python/examples/detection/ssd_vgg16_300_240e_voc/README_CN.md
浏览文件 @ 12ab1380
...@@ -11,12 +11,12 @@ wget --no-check-certificate https://paddle-serving.bj.bcebos.com/pddet_demo/2.0/ ...@@ -11,12 +11,12 @@ wget --no-check-certificate https://paddle-serving.bj.bcebos.com/pddet_demo/2.0/
### 启动服务 ### 启动服务
``` ```
tar xf ssd_vgg16_300_240e_voc.tar tar xf ssd_vgg16_300_240e_voc.tar
python -m paddle_serving_server.serve --model serving_server --port 9494 --gpu_ids 0 python3 -m paddle_serving_server.serve --model serving_server --port 9494 --gpu_ids 0
``` ```
该模型支持TensorRT，如果想要更快的预测速度，可以开启`--use_trt`选项。 该模型支持TensorRT，如果想要更快的预测速度，可以开启`--use_trt`选项。
### 执行预测 ### 执行预测
``` ```
python test_client.py 000000570688.jpg python3 test_client.py 000000570688.jpg
``` ```
python/examples/detection/ttfnet_darknet53_1x_coco/README.md
浏览文件 @ 12ab1380
...@@ -10,12 +10,11 @@ wget --no-check-certificate https://paddle-serving.bj.bcebos.com/pddet_demo/2.0/ ...@@ -10,12 +10,11 @@ wget --no-check-certificate https://paddle-serving.bj.bcebos.com/pddet_demo/2.0/
### Start the service ### Start the service
``` ```
tar xf ttfnet_darknet53_1x_coco.tar tar xf ttfnet_darknet53_1x_coco.tar
python -m paddle_serving_server.serve --model serving_server --port 9494 --gpu_ids 0 python3 -m paddle_serving_server.serve --model serving_server --port 9494 --gpu_ids 0
``` ```
This model support TensorRT, if you want a faster inference, please use `--use_trt`. This model support TensorRT, if you want a faster inference, please use `--use_trt`.
### Perform prediction ### Perform prediction
``` ```
python test_client.py 000000570688.jpg python3 test_client.py 000000570688.jpg
``` ```
python/examples/detection/ttfnet_darknet53_1x_coco/README_CN.md
浏览文件 @ 12ab1380
...@@ -11,13 +11,12 @@ wget --no-check-certificate https://paddle-serving.bj.bcebos.com/pddet_demo/2.0/ ...@@ -11,13 +11,12 @@ wget --no-check-certificate https://paddle-serving.bj.bcebos.com/pddet_demo/2.0/
### 启动服务 ### 启动服务
``` ```
tar xf ttfnet_darknet53_1x_coco.tar tar xf ttfnet_darknet53_1x_coco.tar
python -m paddle_serving_server.serve --model serving_server --port 9494 --gpu_ids 0 python3 -m paddle_serving_server.serve --model serving_server --port 9494 --gpu_ids 0
``` ```
该模型支持TensorRT，如果想要更快的预测速度，可以开启`--use_trt`选项。 该模型支持TensorRT，如果想要更快的预测速度，可以开启`--use_trt`选项。
### 执行预测 ### 执行预测
``` ```
python test_client.py 000000570688.jpg python3 test_client.py 000000570688.jpg
``` ```
python/examples/detection/yolov3_darknet53_270e_coco/README.md
浏览文件 @ 12ab1380
...@@ -10,13 +10,12 @@ wget --no-check-certificate https://paddle-serving.bj.bcebos.com/pddet_demo/2.0/ ...@@ -10,13 +10,12 @@ wget --no-check-certificate https://paddle-serving.bj.bcebos.com/pddet_demo/2.0/
### Start the service ### Start the service
``` ```
tar xf yolov3_darknet53_270e_coco.tar tar xf yolov3_darknet53_270e_coco.tar
python -m paddle_serving_server.serve --model serving_server --port 9494 --gpu_ids 0 python3 -m paddle_serving_server.serve --model serving_server --port 9494 --gpu_ids 0
``` ```
This model support TensorRT, if you want a faster inference, please use `--use_trt`. This model support TensorRT, if you want a faster inference, please use `--use_trt`.
### Perform prediction ### Perform prediction
``` ```
python test_client.py 000000570688.jpg python3 test_client.py 000000570688.jpg
``` ```
python/examples/detection/yolov3_darknet53_270e_coco/README_CN.md
浏览文件 @ 12ab1380
...@@ -11,13 +11,12 @@ wget --no-check-certificate https://paddle-serving.bj.bcebos.com/pddet_demo/2.0/ ...@@ -11,13 +11,12 @@ wget --no-check-certificate https://paddle-serving.bj.bcebos.com/pddet_demo/2.0/
### 启动服务 ### 启动服务
``` ```
tar xf yolov3_darknet53_270e_coco.tar tar xf yolov3_darknet53_270e_coco.tar
python -m paddle_serving_server.serve --model serving_server --port 9494 --gpu_ids 0 python3 -m paddle_serving_server.serve --model serving_server --port 9494 --gpu_ids 0
``` ```
该模型支持TensorRT，如果想要更快的预测速度，可以开启`--use_trt`选项。 该模型支持TensorRT，如果想要更快的预测速度，可以开启`--use_trt`选项。
### 执行预测 ### 执行预测
``` ```
python test_client.py 000000570688.jpg python3 test_client.py 000000570688.jpg
``` ```
python/examples/encryption/README.md
浏览文件 @ 12ab1380
...@@ -12,9 +12,9 @@ sh get_data.sh ...@@ -12,9 +12,9 @@ sh get_data.sh
## Encrypt Model ## Encrypt Model
The `paddlepaddle` package is used in this example, you may need to download the corresponding package(`pip install paddlepaddle`). The `paddlepaddle` package is used in this example, you may need to download the corresponding package(`pip3 install paddlepaddle`).
[python encrypt.py](./encrypt.py) [python3 encrypt.py](./encrypt.py)
[//file]:#encrypt.py [//file]:#encrypt.py
``` python ``` python
...@@ -35,14 +35,14 @@ client-side configuration file are stored in the `encrypt_client` directory. ...@@ -35,14 +35,14 @@ client-side configuration file are stored in the `encrypt_client` directory.
## Start Encryption Service ## Start Encryption Service
CPU Service CPU Service
``` ```
python -m paddle_serving_server.serve --model encrypt_server/ --port 9393 --use_encryption_model python3 -m paddle_serving_server.serve --model encrypt_server/ --port 9393 --use_encryption_model
``` ```
GPU Service GPU Service
``` ```
python -m paddle_serving_server.serve --model encrypt_server/ --port 9393 --use_encryption_model --gpu_ids 0 python3 -m paddle_serving_server.serve --model encrypt_server/ --port 9393 --use_encryption_model --gpu_ids 0
``` ```
## Prediction ## Prediction
``` ```
python test_client.py encrypt_client/serving_client_conf.prototxt python3 test_client.py encrypt_client/serving_client_conf.prototxt
``` ```
python/examples/encryption/README_CN.md
浏览文件 @ 12ab1380
...@@ -11,9 +11,9 @@ sh get_data.sh ...@@ -11,9 +11,9 @@ sh get_data.sh
``` ```
## 模型加密 ## 模型加密
本示例中使用了`paddlepaddle`包中的模块，需要进行下载（`pip install paddlepaddle`）。 本示例中使用了`paddlepaddle`包中的模块，需要进行下载（`pip3 install paddlepaddle`）。
运行[python encrypt.py](./encrypt.py)进行模型加密 运行[python3 encrypt.py](./encrypt.py)进行模型加密
[//file]:#encrypt.py [//file]:#encrypt.py
``` python ``` python
...@@ -36,14 +36,14 @@ def serving_encryption(): ...@@ -36,14 +36,14 @@ def serving_encryption():
## 启动加密预测服务 ## 启动加密预测服务
CPU预测服务 CPU预测服务
``` ```
python -m paddle_serving_server.serve --model encrypt_server/ --port 9393 --use_encryption_model python3 -m paddle_serving_server.serve --model encrypt_server/ --port 9393 --use_encryption_model
``` ```
GPU预测服务 GPU预测服务
``` ```
python -m paddle_serving_server.serve --model encrypt_server/ --port 9393 --use_encryption_model --gpu_ids 0 python3 -m paddle_serving_server.serve --model encrypt_server/ --port 9393 --use_encryption_model --gpu_ids 0
``` ```
## 预测 ## 预测
``` ```
python test_client.py encrypt_client/serving_client_conf.prototxt python3 test_client.py encrypt_client/serving_client_conf.prototxt
``` ```
python/examples/fit_a_line/README.md
浏览文件 @ 12ab1380
...@@ -15,22 +15,22 @@ sh get_data.sh ...@@ -15,22 +15,22 @@ sh get_data.sh
### Start server ### Start server
```shell ```shell
python -m paddle_serving_server.serve --model uci_housing_model --thread 10 --port 9393 python3 -m paddle_serving_server.serve --model uci_housing_model --thread 10 --port 9393
``` ```
## Client prediction ## Client prediction
### RPC Client ### RPC Client
The `paddlepaddle` package is used in `test_client.py`, and you may need to download the corresponding package(`pip install paddlepaddle`). The `paddlepaddle` package is used in `test_client.py`, and you may need to download the corresponding package(`pip3 install paddlepaddle`).
``` shell ``` shell
python test_client.py uci_housing_client/serving_client_conf.prototxt python3 test_client.py uci_housing_client/serving_client_conf.prototxt
``` ```
### Http Client ### Http Client
``` shell ``` shell
python test_httpclient.py uci_housing_client/serving_client_conf.prototxt python3 test_httpclient.py uci_housing_client/serving_client_conf.prototxt
``` ```
......
python/examples/fit_a_line/README_CN.md
浏览文件 @ 12ab1380
...@@ -12,23 +12,23 @@ sh get_data.sh ...@@ -12,23 +12,23 @@ sh get_data.sh
## 开启服务端（支持BRPC-Client/GRPC Client/Http-Client） ## 开启服务端（支持BRPC-Client/GRPC Client/Http-Client）
```shell ```shell
python -m paddle_serving_server.serve --model uci_housing_model --thread 10 --port 9393 python3 -m paddle_serving_server.serve --model uci_housing_model --thread 10 --port 9393
``` ```
## 客户端预测 ## 客户端预测
### BRPC-Client ### BRPC-Client
`test_client.py`中使用了`paddlepaddle`包，需要进行下载（`pip install paddlepaddle`）。 `test_client.py`中使用了`paddlepaddle`包，需要进行下载（`pip3 install paddlepaddle`）。
``` shell ``` shell
python test_client.py uci_housing_client/serving_client_conf.prototxt python3 test_client.py uci_housing_client/serving_client_conf.prototxt
``` ```
### GRPC-Client/Http-Client ### GRPC-Client/Http-Client
``` shell ``` shell
python test_httpclient.py uci_housing_client/serving_client_conf.prototxt python3 test_httpclient.py uci_housing_client/serving_client_conf.prototxt
``` ```
......
python/examples/imagenet/README.md
浏览文件 @ 12ab1380
...@@ -12,7 +12,7 @@ sh get_model.sh ...@@ -12,7 +12,7 @@ sh get_model.sh
### Install preprocess module ### Install preprocess module
``` ```
pip install paddle_serving_app pip3 install paddle_serving_app
``` ```
...@@ -20,22 +20,22 @@ pip install paddle_serving_app ...@@ -20,22 +20,22 @@ pip install paddle_serving_app
launch server side launch server side
``` ```
python -m paddle_serving_server.serve --model ResNet50_vd_model --port 9696 #cpu inference service python3 -m paddle_serving_server.serve --model ResNet50_vd_model --port 9696 #cpu inference service
``` ```
``` ```
python -m paddle_serving_server.serve --model ResNet50_vd_model --port 9696 --gpu_ids 0 #gpu inference service python3 -m paddle_serving_server.serve --model ResNet50_vd_model --port 9696 --gpu_ids 0 #gpu inference service
``` ```
### BRPC-Client ### BRPC-Client
client send inference request client send inference request
``` ```
python resnet50_rpc_client.py ResNet50_vd_client_config/serving_client_conf.prototxt python3 resnet50_rpc_client.py ResNet50_vd_client_config/serving_client_conf.prototxt
``` ```
*the port of server side in this example is 9696 *the port of server side in this example is 9696
### GRPC-Client/Http-Client ### GRPC-Client/Http-Client
client send inference request client send inference request
``` ```
python resnet50_http_client.py ResNet50_vd_client_config/serving_client_conf.prototxt python3 resnet50_http_client.py ResNet50_vd_client_config/serving_client_conf.prototxt
``` ```
python/examples/imagenet/README_CN.md
浏览文件 @ 12ab1380
...@@ -12,24 +12,24 @@ sh get_model.sh ...@@ -12,24 +12,24 @@ sh get_model.sh
### 安装数据预处理模块 ### 安装数据预处理模块
``` ```
pip install paddle_serving_app pip3 install paddle_serving_app
``` ```
### 启动服务端（支持BRPC-Client、GRPC-Client、Http-Client） ### 启动服务端（支持BRPC-Client、GRPC-Client、Http-Client）
启动server端 启动server端
``` ```
python -m paddle_serving_server.serve --model ResNet50_vd_model --port 9696 #cpu预测服务 python3 -m paddle_serving_server.serve --model ResNet50_vd_model --port 9696 #cpu预测服务
``` ```
``` ```
python -m paddle_serving_server.serve --model ResNet50_vd_model --port 9696 --gpu_ids 0 #gpu预测服务 python3 -m paddle_serving_server.serve --model ResNet50_vd_model --port 9696 --gpu_ids 0 #gpu预测服务
``` ```
### BRPC-Client预测 ### BRPC-Client预测
client端进行预测 client端进行预测
``` ```
python resnet50_rpc_client.py ResNet50_vd_client_config/serving_client_conf.prototxt python3 resnet50_rpc_client.py ResNet50_vd_client_config/serving_client_conf.prototxt
``` ```
*server端示例中服务端口为9696端口 *server端示例中服务端口为9696端口
...@@ -37,5 +37,5 @@ python resnet50_rpc_client.py ResNet50_vd_client_config/serving_client_conf.prot ...@@ -37,5 +37,5 @@ python resnet50_rpc_client.py ResNet50_vd_client_config/serving_client_conf.prot
### GRPC-Client/Http-Client预测 ### GRPC-Client/Http-Client预测
client端进行预测 client端进行预测
``` ```
python resnet50_http_client.py ResNet50_vd_client_config/serving_client_conf.prototxt python3 resnet50_http_client.py ResNet50_vd_client_config/serving_client_conf.prototxt
``` ```
python/examples/imdb/README.md
浏览文件 @ 12ab1380
...@@ -12,11 +12,11 @@ the package downloaded contains cnn, lstm and bow model config along with their ...@@ -12,11 +12,11 @@ the package downloaded contains cnn, lstm and bow model config along with their
### Start inference service(Support BRPC-Client/GRPC-Client/Http-Client) ### Start inference service(Support BRPC-Client/GRPC-Client/Http-Client)
``` ```
python -m paddle_serving_server.serve --model imdb_cnn_model/ --port 9292 python3 -m paddle_serving_server.serve --model imdb_cnn_model/ --port 9292
``` ```
### BRPC-Client Infer ### BRPC-Client Infer
``` ```
head test_data/part-0 | python test_client.py imdb_cnn_client_conf/serving_client_conf.prototxt imdb.vocab head test_data/part-0 | python3 test_client.py imdb_cnn_client_conf/serving_client_conf.prototxt imdb.vocab
``` ```
it will get predict results of the first 10 test cases. it will get predict results of the first 10 test cases.
...@@ -24,5 +24,5 @@ it will get predict results of the first 10 test cases. ...@@ -24,5 +24,5 @@ it will get predict results of the first 10 test cases.
### GRPC-Client/Http-Client Infer ### GRPC-Client/Http-Client Infer
``` ```
head test_data/part-0 | python test_http_client.py imdb_cnn_client_conf/serving_client_conf.prototxt imdb.vocab head test_data/part-0 | python3 test_http_client.py imdb_cnn_client_conf/serving_client_conf.prototxt imdb.vocab
``` ```
python/examples/imdb/README_CN.md
浏览文件 @ 12ab1380
...@@ -12,15 +12,15 @@ sh get_data.sh ...@@ -12,15 +12,15 @@ sh get_data.sh
### 启动预测服务(支持BRPC-Client/GRPC-Client/Http-Client) ### 启动预测服务(支持BRPC-Client/GRPC-Client/Http-Client)
``` ```
python -m paddle_serving_server.serve --model imdb_cnn_model/ --port 9292 python3 -m paddle_serving_server.serve --model imdb_cnn_model/ --port 9292
``` ```
### BRPC-Client预测 ### BRPC-Client预测
``` ```
head test_data/part-0 | python test_client.py imdb_cnn_client_conf/serving_client_conf.prototxt imdb.vocab head test_data/part-0 | python3 test_client.py imdb_cnn_client_conf/serving_client_conf.prototxt imdb.vocab
``` ```
预测test_data/part-0的前十个样例。 预测test_data/part-0的前十个样例。
### BRPC-Client预测 ### BRPC-Client预测
``` ```
head test_data/part-0 | python test_http_client.py imdb_cnn_client_conf/serving_client_conf.prototxt imdb.vocab head test_data/part-0 | python3 test_http_client.py imdb_cnn_client_conf/serving_client_conf.prototxt imdb.vocab
``` ```
python/examples/lac/README.md
浏览文件 @ 12ab1380
...@@ -4,23 +4,23 @@ ...@@ -4,23 +4,23 @@
### Get Model ### Get Model
``` ```
python -m paddle_serving_app.package --get_model lac python3 -m paddle_serving_app.package --get_model lac
tar -xzvf lac.tar.gz tar -xzvf lac.tar.gz
``` ```
#### Start inference service(Support BRPC-Client/GRPC-Client/Http-Client) #### Start inference service(Support BRPC-Client/GRPC-Client/Http-Client)
``` ```
python -m paddle_serving_server.serve --model lac_model/ --port 9292 python3 -m paddle_serving_server.serve --model lac_model/ --port 9292
``` ```
### BRPC Infer ### BRPC Infer
``` ```
echo "我爱北京天安门" | python lac_client.py lac_client/serving_client_conf.prototxt echo "我爱北京天安门" | python3 lac_client.py lac_client/serving_client_conf.prototxt
``` ```
It will get the segmentation result. It will get the segmentation result.
### GRPC/Http Infer ### GRPC/Http Infer
``` ```
echo "我爱北京天安门" | python lac_http_client.py lac_client/serving_client_conf.prototxt echo "我爱北京天安门" | python3 lac_http_client.py lac_client/serving_client_conf.prototxt
``` ```
python/examples/lac/README_CN.md
浏览文件 @ 12ab1380
...@@ -4,23 +4,23 @@ ...@@ -4,23 +4,23 @@
### 获取模型 ### 获取模型
``` ```
python -m paddle_serving_app.package --get_model lac python3 -m paddle_serving_app.package --get_model lac
tar -xzvf lac.tar.gz tar -xzvf lac.tar.gz
``` ```
#### 开启预测服务(支持BRPC-Client/GRPC-Client/Http-Client) #### 开启预测服务(支持BRPC-Client/GRPC-Client/Http-Client)
``` ```
python -m paddle_serving_server.serve --model lac_model/ --port 9292 python3 -m paddle_serving_server.serve --model lac_model/ --port 9292
``` ```
### 执行BRPC预测 ### 执行BRPC预测
``` ```
echo "我爱北京天安门" | python lac_client.py lac_client/serving_client_conf.prototxt echo "我爱北京天安门" | python3 lac_client.py lac_client/serving_client_conf.prototxt
``` ```
我们就能得到分词结果 我们就能得到分词结果
### 执行GRPC/Http预测 ### 执行GRPC/Http预测
``` ```
echo "我爱北京天安门" | python lac_http_client.py lac_client/serving_client_conf.prototxt echo "我爱北京天安门" | python3 lac_http_client.py lac_client/serving_client_conf.prototxt
``` ```
python/examples/low_precision/resnet50/README.md
浏览文件 @ 12ab1380
...@@ -11,15 +11,15 @@ Firstly, download the [Resnet50 int8 model](https://paddle-inference-dist.bj.bce ...@@ -11,15 +11,15 @@ Firstly, download the [Resnet50 int8 model](https://paddle-inference-dist.bj.bce
wget https://paddle-inference-dist.bj.bcebos.com/inference_demo/python/resnet50/ResNet50_quant.tar.gz wget https://paddle-inference-dist.bj.bcebos.com/inference_demo/python/resnet50/ResNet50_quant.tar.gz
tar zxvf ResNet50_quant.tar.gz tar zxvf ResNet50_quant.tar.gz
python -m paddle_serving_client.convert --dirname ResNet50_quant python3 -m paddle_serving_client.convert --dirname ResNet50_quant
``` ```
Start RPC service, specify the GPU id and precision mode Start RPC service, specify the GPU id and precision mode
``` ```
python -m paddle_serving_server.serve --model serving_server --port 9393 --gpu_ids 0 --use_trt --precision int8 python3 -m paddle_serving_server.serve --model serving_server --port 9393 --gpu_ids 0 --use_trt --precision int8
``` ```
Request the serving service with Client Request the serving service with Client
``` ```
python resnet50_client.py python3 resnet50_client.py
``` ```
## Reference ## Reference
......
python/examples/low_precision/resnet50/README_CN.md
浏览文件 @ 12ab1380
...@@ -10,15 +10,15 @@ ...@@ -10,15 +10,15 @@
wget https://paddle-inference-dist.bj.bcebos.com/inference_demo/python/resnet50/ResNet50_quant.tar.gz wget https://paddle-inference-dist.bj.bcebos.com/inference_demo/python/resnet50/ResNet50_quant.tar.gz
tar zxvf ResNet50_quant.tar.gz tar zxvf ResNet50_quant.tar.gz
python -m paddle_serving_client.convert --dirname ResNet50_quant python3 -m paddle_serving_client.convert --dirname ResNet50_quant
``` ```
启动rpc服务, 设定所选GPU id、部署模型精度 启动rpc服务, 设定所选GPU id、部署模型精度
``` ```
python -m paddle_serving_server.serve --model serving_server --port 9393 --gpu_ids 0 --use_trt --precision int8 python3 -m paddle_serving_server.serve --model serving_server --port 9393 --gpu_ids 0 --use_trt --precision int8
``` ```
使用client进行请求 使用client进行请求
``` ```
python resnet50_client.py python3 resnet50_client.py
``` ```
## 参考文档 ## 参考文档
......
python/examples/mobilenet/README.md
浏览文件 @ 12ab1380
...@@ -3,7 +3,7 @@ ...@@ -3,7 +3,7 @@
## Get Model ## Get Model
``` ```
python -m paddle_serving_app.package --get_model mobilenet_v2_imagenet python3 -m paddle_serving_app.package --get_model mobilenet_v2_imagenet
tar -xzvf mobilenet_v2_imagenet.tar.gz tar -xzvf mobilenet_v2_imagenet.tar.gz
``` ```
...@@ -12,11 +12,11 @@ tar -xzvf mobilenet_v2_imagenet.tar.gz ...@@ -12,11 +12,11 @@ tar -xzvf mobilenet_v2_imagenet.tar.gz
### Start Service ### Start Service
``` ```
python -m paddle_serving_server.serve --model mobilenet_v2_imagenet_model --gpu_ids 0 --port 9393 python3 -m paddle_serving_server.serve --model mobilenet_v2_imagenet_model --gpu_ids 0 --port 9393
``` ```
### Client Prediction ### Client Prediction
``` ```
python mobilenet_tutorial.py python3 mobilenet_tutorial.py
``` ```
python/examples/mobilenet/README_CN.md
浏览文件 @ 12ab1380
...@@ -3,7 +3,7 @@ ...@@ -3,7 +3,7 @@
## 获取模型 ## 获取模型
``` ```
python -m paddle_serving_app.package --get_model mobilenet_v2_imagenet python3 -m paddle_serving_app.package --get_model mobilenet_v2_imagenet
tar -xzvf mobilenet_v2_imagenet.tar.gz tar -xzvf mobilenet_v2_imagenet.tar.gz
``` ```
...@@ -12,11 +12,11 @@ tar -xzvf mobilenet_v2_imagenet.tar.gz ...@@ -12,11 +12,11 @@ tar -xzvf mobilenet_v2_imagenet.tar.gz
### 启动服务端 ### 启动服务端
``` ```
python -m paddle_serving_server.serve --model mobilenet_v2_imagenet_model --gpu_ids 0 --port 9393 python3 -m paddle_serving_server.serve --model mobilenet_v2_imagenet_model --gpu_ids 0 --port 9393
``` ```
### 客户端预测 ### 客户端预测
``` ```
python mobilenet_tutorial.py python3 mobilenet_tutorial.py
``` ```
python/examples/ocr/README.md
浏览文件 @ 12ab1380
...@@ -4,9 +4,9 @@ ...@@ -4,9 +4,9 @@
## Get Model ## Get Model
``` ```
python -m paddle_serving_app.package --get_model ocr_rec python3 -m paddle_serving_app.package --get_model ocr_rec
tar -xzvf ocr_rec.tar.gz tar -xzvf ocr_rec.tar.gz
python -m paddle_serving_app.package --get_model ocr_det python3 -m paddle_serving_app.package --get_model ocr_det
tar -xzvf ocr_det.tar.gz tar -xzvf ocr_det.tar.gz
``` ```
...@@ -23,16 +23,16 @@ tar xf test_imgs.tar ...@@ -23,16 +23,16 @@ tar xf test_imgs.tar
``` ```
#choose one of cpu/gpu commands as following #choose one of cpu/gpu commands as following
#for cpu user #for cpu user
python -m paddle_serving_server.serve --model ocr_det_model --port 9293 python3 -m paddle_serving_server.serve --model ocr_det_model --port 9293
python ocr_web_server.py cpu python3 ocr_web_server.py cpu
#for gpu user #for gpu user
python -m paddle_serving_server.serve --model ocr_det_model --port 9293 --gpu_ids 0 python3 -m paddle_serving_server.serve --model ocr_det_model --port 9293 --gpu_ids 0
python ocr_web_server.py gpu python3 ocr_web_server.py gpu
``` ```
### Client Prediction ### Client Prediction
``` ```
python ocr_web_client.py python3 ocr_web_client.py
``` ```
If you want a faster web service, please try Web LocalPredictor Service If you want a faster web service, please try Web LocalPredictor Service
...@@ -40,14 +40,14 @@ If you want a faster web service, please try Web LocalPredictor Service ...@@ -40,14 +40,14 @@ If you want a faster web service, please try Web LocalPredictor Service
``` ```
#choose one of cpu/gpu commands as following #choose one of cpu/gpu commands as following
#for cpu user #for cpu user
python ocr_debugger_server.py cpu python3 ocr_debugger_server.py cpu
#for gpu user #for gpu user
python ocr_debugger_server.py gpu python3 ocr_debugger_server.py gpu
``` ```
## Web LocalPredictor Client Prediction ## Web LocalPredictor Client Prediction
``` ```
python ocr_web_client.py python3 ocr_web_client.py
``` ```
## Benchmark ## Benchmark
...@@ -69,34 +69,34 @@ if you are going to detect images not recognize it or directly recognize the wor ...@@ -69,34 +69,34 @@ if you are going to detect images not recognize it or directly recognize the wor
### Det Server ### Det Server
``` ```
python det_web_server.py cpu #for cpu user python3 det_web_server.py cpu #for cpu user
python det_web_server.py gpu #for gpu user python3 det_web_server.py gpu #for gpu user
#or #or
python det_debugger_server.py cpu #for cpu user python3 det_debugger_server.py cpu #for cpu user
python det_debugger_server.py gpu #for gpu user python3 det_debugger_server.py gpu #for gpu user
``` ```
### Det Client ### Det Client
``` ```
# also use ocr_web_client.py # also use ocr_web_client.py
python ocr_web_client.py python3 ocr_web_client.py
``` ```
### Rec Server ### Rec Server
``` ```
python rec_web_server.py cpu #for cpu user python3 rec_web_server.py cpu #for cpu user
python rec_web_server.py gpu #for gpu user python3 rec_web_server.py gpu #for gpu user
#or #or
python rec_debugger_server.py cpu #for cpu user python3 rec_debugger_server.py cpu #for cpu user
python rec_debugger_server.py gpu #for gpu user python3 rec_debugger_server.py gpu #for gpu user
``` ```
### Rec Client ### Rec Client
``` ```
python rec_web_client.py python3 rec_web_client.py
``` ```
## C++ OCR Service ## C++ OCR Service
...@@ -109,9 +109,9 @@ Select a startup mode according to CPU / GPU device ...@@ -109,9 +109,9 @@ Select a startup mode according to CPU / GPU device
After the -- model parameter, the folder path of multiple model files is passed in to start the prediction service of multiple model concatenation. After the -- model parameter, the folder path of multiple model files is passed in to start the prediction service of multiple model concatenation.
``` ```
#for cpu user #for cpu user
python -m paddle_serving_server.serve --model ocr_det_model ocr_rec_model --port 9293 python3 -m paddle_serving_server.serve --model ocr_det_model ocr_rec_model --port 9293
#for gpu user #for gpu user
python -m paddle_serving_server.serve --model ocr_det_model ocr_rec_model --port 9293 --gpu_ids 0 python3 -m paddle_serving_server.serve --model ocr_det_model ocr_rec_model --port 9293 --gpu_ids 0
``` ```
### Client Prediction ### Client Prediction
...@@ -123,5 +123,5 @@ for this case, `feed_type` should be 3(which means the data type is string),`sha ...@@ -123,5 +123,5 @@ for this case, `feed_type` should be 3(which means the data type is string),`sha
By passing in multiple client folder paths, the client can be started for multi model prediction. By passing in multiple client folder paths, the client can be started for multi model prediction.
``` ```
python ocr_cpp_client.py ocr_det_client ocr_rec_client python3 ocr_cpp_client.py ocr_det_client ocr_rec_client
``` ```
python/examples/ocr/README_CN.md
浏览文件 @ 12ab1380
...@@ -4,9 +4,9 @@ ...@@ -4,9 +4,9 @@
## 获取模型 ## 获取模型
``` ```
python -m paddle_serving_app.package --get_model ocr_rec python3 -m paddle_serving_app.package --get_model ocr_rec
tar -xzvf ocr_rec.tar.gz tar -xzvf ocr_rec.tar.gz
python -m paddle_serving_app.package --get_model ocr_det python3 -m paddle_serving_app.package --get_model ocr_det
tar -xzvf ocr_det.tar.gz tar -xzvf ocr_det.tar.gz
``` ```
## 获取数据集（可选） ## 获取数据集（可选）
...@@ -22,16 +22,16 @@ tar xf test_imgs.tar ...@@ -22,16 +22,16 @@ tar xf test_imgs.tar
``` ```
#根据CPU/GPU设备选择一种启动方式 #根据CPU/GPU设备选择一种启动方式
#for cpu user #for cpu user
python -m paddle_serving_server.serve --model ocr_det_model --port 9293 python3 -m paddle_serving_server.serve --model ocr_det_model --port 9293
python ocr_web_server.py cpu python3 ocr_web_server.py cpu
#for gpu user #for gpu user
python -m paddle_serving_server.serve --model ocr_det_model --port 9293 --gpu_ids 0 python3 -m paddle_serving_server.serve --model ocr_det_model --port 9293 --gpu_ids 0
python ocr_web_server.py gpu python3 ocr_web_server.py gpu
``` ```
### 启动客户端 ### 启动客户端
``` ```
python ocr_web_client.py python3 ocr_web_client.py
``` ```
如果用户需要更快的执行速度，请尝试LocalPredictor版Web服务 如果用户需要更快的执行速度，请尝试LocalPredictor版Web服务
...@@ -39,14 +39,14 @@ python ocr_web_client.py ...@@ -39,14 +39,14 @@ python ocr_web_client.py
``` ```
#根据CPU/GPU设备选择一种启动方式 #根据CPU/GPU设备选择一种启动方式
#for cpu user #for cpu user
python ocr_debugger_server.py cpu python3 ocr_debugger_server.py cpu
#for gpu user #for gpu user
python ocr_debugger_server.py gpu python3 ocr_debugger_server.py gpu
``` ```
## 启动客户端 ## 启动客户端
``` ```
python ocr_web_client.py python3 ocr_web_client.py
``` ```
## 性能指标 ## 性能指标
...@@ -69,34 +69,34 @@ GPU: Nvidia Tesla V100单卡 ...@@ -69,34 +69,34 @@ GPU: Nvidia Tesla V100单卡
### 启动检测服务 ### 启动检测服务
``` ```
python det_web_server.py cpu #for cpu user python3 det_web_server.py cpu #for cpu user
python det_web_server.py gpu #for gpu user python3 det_web_server.py gpu #for gpu user
#or #or
python det_debugger_server.py cpu #for cpu user python3 det_debugger_server.py cpu #for cpu user
python det_debugger_server.py gpu #for gpu user python3 det_debugger_server.py gpu #for gpu user
``` ```
### 检测服务客户端 ### 检测服务客户端
``` ```
# also use ocr_web_client.py # also use ocr_web_client.py
python ocr_web_client.py python3 ocr_web_client.py
``` ```
### 启动识别服务 ### 启动识别服务
``` ```
python rec_web_server.py cpu #for cpu user python3 rec_web_server.py cpu #for cpu user
python rec_web_server.py gpu #for gpu user python3 rec_web_server.py gpu #for gpu user
#or #or
python rec_debugger_server.py cpu #for cpu user python3 rec_debugger_server.py cpu #for cpu user
python rec_debugger_server.py gpu #for gpu user python3 rec_debugger_server.py gpu #for gpu user
``` ```
### 识别服务客户端 ### 识别服务客户端
``` ```
python rec_web_client.py python3 rec_web_client.py
``` ```
## C++ OCR Service服务 ## C++ OCR Service服务
...@@ -108,9 +108,9 @@ python rec_web_client.py ...@@ -108,9 +108,9 @@ python rec_web_client.py
通过--model后，指定多个模型文件的文件夹路径来启动多模型串联的预测服务。 通过--model后，指定多个模型文件的文件夹路径来启动多模型串联的预测服务。
``` ```
#for cpu user #for cpu user
python -m paddle_serving_server.serve --model ocr_det_model ocr_rec_model --port 9293 python3 -m paddle_serving_server.serve --model ocr_det_model ocr_rec_model --port 9293
#for gpu user #for gpu user
python -m paddle_serving_server.serve --model ocr_det_model ocr_rec_model --port 9293 --gpu_ids 0 python3 -m paddle_serving_server.serve --model ocr_det_model ocr_rec_model --port 9293 --gpu_ids 0
``` ```
### 启动客户端 ### 启动客户端
...@@ -122,5 +122,5 @@ python -m paddle_serving_server.serve --model ocr_det_model ocr_rec_model --port ...@@ -122,5 +122,5 @@ python -m paddle_serving_server.serve --model ocr_det_model ocr_rec_model --port
通过在客户端启动后加入多个client模型的client配置文件夹路径，启动client进行预测。 通过在客户端启动后加入多个client模型的client配置文件夹路径，启动client进行预测。
``` ```
python ocr_cpp_client.py ocr_det_client ocr_rec_client python3 ocr_cpp_client.py ocr_det_client ocr_rec_client
``` ```
python/examples/pipeline/PaddleClas/DarkNet53/README.md
浏览文件 @ 12ab1380
...@@ -10,10 +10,10 @@ sh get_model.sh ...@@ -10,10 +10,10 @@ sh get_model.sh
## Start server ## Start server
``` ```
python resnet50_web_service.py &>log.txt & python3 resnet50_web_service.py &>log.txt &
``` ```
## RPC test ## RPC test
``` ```
python pipeline_rpc_client.py python3 pipeline_rpc_client.py
``` ```
python/examples/pipeline/PaddleClas/DarkNet53/README_CN.md
浏览文件 @ 12ab1380
...@@ -10,11 +10,10 @@ sh get_model.sh ...@@ -10,11 +10,10 @@ sh get_model.sh
## 启动服务 ## 启动服务
``` ```
python resnet50_web_service.py &>log.txt & python3 resnet50_web_service.py &>log.txt &
``` ```
## 测试 ## 测试
``` ```
python pipeline_rpc_client.py python3 pipeline_rpc_client.py
``` ```
python/examples/pipeline/PaddleClas/DarkNet53/benchmark.py
浏览文件 @ 12ab1380
# Copyright (c) 2021 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import sys import sys
import os import os
import base64 import base64
...@@ -5,13 +19,13 @@ import yaml ...@@ -5,13 +19,13 @@ import yaml
import requests import requests
import time import time
import json import json
try:
from paddle_serving_server_gpu.pipeline import PipelineClient from paddle_serving_server.pipeline import PipelineClient
except ImportError:
from paddle_serving_server.pipeline import PipelineClient
import numpy as np import numpy as np
from paddle_serving_client.utils import MultiThreadRunner from paddle_serving_client.utils import MultiThreadRunner
from paddle_serving_client.utils import benchmark_args, show_latency from paddle_serving_client.utils import benchmark_args, show_latency
def parse_benchmark(filein, fileout): def parse_benchmark(filein, fileout):
with open(filein, "r") as fin: with open(filein, "r") as fin:
res = yaml.load(fin) res = yaml.load(fin)
...@@ -24,6 +38,7 @@ def parse_benchmark(filein, fileout): ...@@ -24,6 +38,7 @@ def parse_benchmark(filein, fileout):
with open(fileout, "w") as fout: with open(fileout, "w") as fout:
yaml.dump(res, fout, default_flow_style=False) yaml.dump(res, fout, default_flow_style=False)
def gen_yml(device, gpu_id): def gen_yml(device, gpu_id):
fin = open("config.yml", "r") fin = open("config.yml", "r")
config = yaml.load(fin) config = yaml.load(fin)
...@@ -34,15 +49,17 @@ def gen_yml(device, gpu_id): ...@@ -34,15 +49,17 @@ def gen_yml(device, gpu_id):
config["op"]["imagenet"]["local_service_conf"]["devices"] = gpu_id config["op"]["imagenet"]["local_service_conf"]["devices"] = gpu_id
else: else:
config["op"]["imagenet"]["local_service_conf"]["device_type"] = 0 config["op"]["imagenet"]["local_service_conf"]["device_type"] = 0
with open("config2.yml", "w") as fout: with open("config2.yml", "w") as fout:
yaml.dump(config, fout, default_flow_style=False) yaml.dump(config, fout, default_flow_style=False)
def cv2_to_base64(image): def cv2_to_base64(image):
return base64.b64encode(image).decode('utf8') return base64.b64encode(image).decode('utf8')
def run_http(idx, batch_size): def run_http(idx, batch_size):
print("start thread ({})".format(idx)) print("start thread ({})".format(idx))
url = "http://127.0.0.1:18080/imagenet/prediction" url = "http://127.0.0.1:18080/imagenet/prediction"
start = time.time() start = time.time()
with open(os.path.join(".", "daisy.jpg"), 'rb') as file: with open(os.path.join(".", "daisy.jpg"), 'rb') as file:
...@@ -68,6 +85,7 @@ def run_http(idx, batch_size): ...@@ -68,6 +85,7 @@ def run_http(idx, batch_size):
end = time.time() end = time.time()
return [[end - start], latency_list, [total_num]] return [[end - start], latency_list, [total_num]]
def multithread_http(thread, batch_size): def multithread_http(thread, batch_size):
multi_thread_runner = MultiThreadRunner() multi_thread_runner = MultiThreadRunner()
start = time.time() start = time.time()
...@@ -87,6 +105,7 @@ def multithread_http(thread, batch_size): ...@@ -87,6 +105,7 @@ def multithread_http(thread, batch_size):
total_cost)) total_cost))
show_latency(result[1]) show_latency(result[1])
def run_rpc(thread, batch_size): def run_rpc(thread, batch_size):
client = PipelineClient() client = PipelineClient()
client.connect(['127.0.0.1:18080']) client.connect(['127.0.0.1:18080'])
...@@ -107,11 +126,12 @@ def run_rpc(thread, batch_size): ...@@ -107,11 +126,12 @@ def run_rpc(thread, batch_size):
def multithread_rpc(thraed, batch_size): def multithread_rpc(thraed, batch_size):
multi_thread_runner = MultiThreadRunner() multi_thread_runner = MultiThreadRunner()
result = multi_thread_runner.run(run_rpc , thread, batch_size) result = multi_thread_runner.run(run_rpc, thread, batch_size)
if __name__ == "__main__": if __name__ == "__main__":
if sys.argv[1] == "yaml": if sys.argv[1] == "yaml":
mode = sys.argv[2] # brpc/ local predictor mode = sys.argv[2] # brpc/ local predictor
thread = int(sys.argv[3]) thread = int(sys.argv[3])
device = sys.argv[4] device = sys.argv[4]
if device == "gpu": if device == "gpu":
...@@ -120,7 +140,7 @@ if __name__ == "__main__": ...@@ -120,7 +140,7 @@ if __name__ == "__main__":
gpu_id = None gpu_id = None
gen_yml(device, gpu_id) gen_yml(device, gpu_id)
elif sys.argv[1] == "run": elif sys.argv[1] == "run":
mode = sys.argv[2] # http/ rpc mode = sys.argv[2] # http/ rpc
thread = int(sys.argv[3]) thread = int(sys.argv[3])
batch_size = int(sys.argv[4]) batch_size = int(sys.argv[4])
if mode == "http": if mode == "http":
...@@ -131,4 +151,3 @@ if __name__ == "__main__": ...@@ -131,4 +151,3 @@ if __name__ == "__main__":
filein = sys.argv[2] filein = sys.argv[2]
fileout = sys.argv[3] fileout = sys.argv[3]
parse_benchmark(filein, fileout) parse_benchmark(filein, fileout)
python/examples/pipeline/PaddleClas/DarkNet53/pipeline_rpc_client.py
浏览文件 @ 12ab1380
...@@ -11,10 +11,8 @@ ...@@ -11,10 +11,8 @@
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and # See the License for the specific language governing permissions and
# limitations under the License. # limitations under the License.
try:
from paddle_serving_server_gpu.pipeline import PipelineClient from paddle_serving_server.pipeline import PipelineClient
except ImportError:
from paddle_serving_server.pipeline import PipelineClient
import numpy as np import numpy as np
import requests import requests
import json import json
......
python/examples/pipeline/PaddleClas/HRNet_W18_C/README.md
浏览文件 @ 12ab1380
...@@ -10,10 +10,10 @@ sh get_model.sh ...@@ -10,10 +10,10 @@ sh get_model.sh
## Start server ## Start server
``` ```
python resnet50_web_service.py &>log.txt & python3 resnet50_web_service.py &>log.txt &
``` ```
## RPC test ## RPC test
``` ```
python pipeline_rpc_client.py python3 pipeline_rpc_client.py
``` ```
python/examples/pipeline/PaddleClas/HRNet_W18_C/README_CN.md
浏览文件 @ 12ab1380
...@@ -10,11 +10,10 @@ sh get_model.sh ...@@ -10,11 +10,10 @@ sh get_model.sh
## 启动服务 ## 启动服务
``` ```
python resnet50_web_service.py &>log.txt & python3 resnet50_web_service.py &>log.txt &
``` ```
## 测试 ## 测试
``` ```
python pipeline_rpc_client.py python3 pipeline_rpc_client.py
``` ```
python/examples/pipeline/PaddleClas/HRNet_W18_C/benchmark.py
浏览文件 @ 12ab1380
# Copyright (c) 2021 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import sys import sys
import os import os
import base64 import base64
...@@ -5,13 +19,13 @@ import yaml ...@@ -5,13 +19,13 @@ import yaml
import requests import requests
import time import time
import json import json
try:
from paddle_serving_server_gpu.pipeline import PipelineClient from paddle_serving_server.pipeline import PipelineClient
except ImportError:
from paddle_serving_server.pipeline import PipelineClient
import numpy as np import numpy as np
from paddle_serving_client.utils import MultiThreadRunner from paddle_serving_client.utils import MultiThreadRunner
from paddle_serving_client.utils import benchmark_args, show_latency from paddle_serving_client.utils import benchmark_args, show_latency
def parse_benchmark(filein, fileout): def parse_benchmark(filein, fileout):
with open(filein, "r") as fin: with open(filein, "r") as fin:
res = yaml.load(fin) res = yaml.load(fin)
...@@ -24,6 +38,7 @@ def parse_benchmark(filein, fileout): ...@@ -24,6 +38,7 @@ def parse_benchmark(filein, fileout):
with open(fileout, "w") as fout: with open(fileout, "w") as fout:
yaml.dump(res, fout, default_flow_style=False) yaml.dump(res, fout, default_flow_style=False)
def gen_yml(device, gpu_id): def gen_yml(device, gpu_id):
fin = open("config.yml", "r") fin = open("config.yml", "r")
config = yaml.load(fin) config = yaml.load(fin)
...@@ -34,15 +49,17 @@ def gen_yml(device, gpu_id): ...@@ -34,15 +49,17 @@ def gen_yml(device, gpu_id):
config["op"]["imagenet"]["local_service_conf"]["devices"] = gpu_id config["op"]["imagenet"]["local_service_conf"]["devices"] = gpu_id
else: else:
config["op"]["imagenet"]["local_service_conf"]["device_type"] = 0 config["op"]["imagenet"]["local_service_conf"]["device_type"] = 0
with open("config2.yml", "w") as fout: with open("config2.yml", "w") as fout:
yaml.dump(config, fout, default_flow_style=False) yaml.dump(config, fout, default_flow_style=False)
def cv2_to_base64(image): def cv2_to_base64(image):
return base64.b64encode(image).decode('utf8') return base64.b64encode(image).decode('utf8')
def run_http(idx, batch_size): def run_http(idx, batch_size):
print("start thread ({})".format(idx)) print("start thread ({})".format(idx))
url = "http://127.0.0.1:18080/imagenet/prediction" url = "http://127.0.0.1:18080/imagenet/prediction"
start = time.time() start = time.time()
with open(os.path.join(".", "daisy.jpg"), 'rb') as file: with open(os.path.join(".", "daisy.jpg"), 'rb') as file:
...@@ -68,6 +85,7 @@ def run_http(idx, batch_size): ...@@ -68,6 +85,7 @@ def run_http(idx, batch_size):
end = time.time() end = time.time()
return [[end - start], latency_list, [total_num]] return [[end - start], latency_list, [total_num]]
def multithread_http(thread, batch_size): def multithread_http(thread, batch_size):
multi_thread_runner = MultiThreadRunner() multi_thread_runner = MultiThreadRunner()
start = time.time() start = time.time()
...@@ -87,6 +105,7 @@ def multithread_http(thread, batch_size): ...@@ -87,6 +105,7 @@ def multithread_http(thread, batch_size):
total_cost)) total_cost))
show_latency(result[1]) show_latency(result[1])
def run_rpc(thread, batch_size): def run_rpc(thread, batch_size):
client = PipelineClient() client = PipelineClient()
client.connect(['127.0.0.1:18080']) client.connect(['127.0.0.1:18080'])
...@@ -107,11 +126,12 @@ def run_rpc(thread, batch_size): ...@@ -107,11 +126,12 @@ def run_rpc(thread, batch_size):
def multithread_rpc(thraed, batch_size): def multithread_rpc(thraed, batch_size):
multi_thread_runner = MultiThreadRunner() multi_thread_runner = MultiThreadRunner()
result = multi_thread_runner.run(run_rpc , thread, batch_size) result = multi_thread_runner.run(run_rpc, thread, batch_size)
if __name__ == "__main__": if __name__ == "__main__":
if sys.argv[1] == "yaml": if sys.argv[1] == "yaml":
mode = sys.argv[2] # brpc/ local predictor mode = sys.argv[2] # brpc/ local predictor
thread = int(sys.argv[3]) thread = int(sys.argv[3])
device = sys.argv[4] device = sys.argv[4]
if device == "gpu": if device == "gpu":
...@@ -120,7 +140,7 @@ if __name__ == "__main__": ...@@ -120,7 +140,7 @@ if __name__ == "__main__":
gpu_id = None gpu_id = None
gen_yml(device, gpu_id) gen_yml(device, gpu_id)
elif sys.argv[1] == "run": elif sys.argv[1] == "run":
mode = sys.argv[2] # http/ rpc mode = sys.argv[2] # http/ rpc
thread = int(sys.argv[3]) thread = int(sys.argv[3])
batch_size = int(sys.argv[4]) batch_size = int(sys.argv[4])
if mode == "http": if mode == "http":
...@@ -131,4 +151,3 @@ if __name__ == "__main__": ...@@ -131,4 +151,3 @@ if __name__ == "__main__":
filein = sys.argv[2] filein = sys.argv[2]
fileout = sys.argv[3] fileout = sys.argv[3]
parse_benchmark(filein, fileout) parse_benchmark(filein, fileout)
python/examples/pipeline/PaddleClas/HRNet_W18_C/pipeline_rpc_client.py
浏览文件 @ 12ab1380
...@@ -11,10 +11,8 @@ ...@@ -11,10 +11,8 @@
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and # See the License for the specific language governing permissions and
# limitations under the License. # limitations under the License.
try:
from paddle_serving_server_gpu.pipeline import PipelineClient from paddle_serving_server.pipeline import PipelineClient
except ImportError:
from paddle_serving_server.pipeline import PipelineClient
import numpy as np import numpy as np
import requests import requests
import json import json
......
python/examples/pipeline/PaddleClas/HRNet_W18_C/resnet50_web_service.py
浏览文件 @ 12ab1380
...@@ -13,10 +13,8 @@ ...@@ -13,10 +13,8 @@
# limitations under the License. # limitations under the License.
import sys import sys
from paddle_serving_app.reader import Sequential, URL2Image, Resize, CenterCrop, RGB2BGR, Transpose, Div, Normalize, Base64ToImage from paddle_serving_app.reader import Sequential, URL2Image, Resize, CenterCrop, RGB2BGR, Transpose, Div, Normalize, Base64ToImage
try:
from paddle_serving_server_gpu.web_service import WebService, Op from paddle_serving_server.web_service import WebService, Op
except ImportError:
from paddle_serving_server.web_service import WebService, Op
import logging import logging
import numpy as np import numpy as np
import base64, cv2 import base64, cv2
......
python/examples/pipeline/PaddleClas/MobileNetV1/README.md
浏览文件 @ 12ab1380
...@@ -10,10 +10,10 @@ sh get_model.sh ...@@ -10,10 +10,10 @@ sh get_model.sh
## Start server ## Start server
``` ```
python resnet50_web_service.py &>log.txt & python3 resnet50_web_service.py &>log.txt &
``` ```
## RPC test ## RPC test
``` ```
python pipeline_rpc_client.py python3 pipeline_rpc_client.py
``` ```
python/examples/pipeline/PaddleClas/MobileNetV1/README_CN.md
浏览文件 @ 12ab1380
...@@ -10,11 +10,10 @@ sh get_model.sh ...@@ -10,11 +10,10 @@ sh get_model.sh
## 启动服务 ## 启动服务
``` ```
python resnet50_web_service.py &>log.txt & python3 resnet50_web_service.py &>log.txt &
``` ```
## 测试 ## 测试
``` ```
python pipeline_rpc_client.py python3 pipeline_rpc_client.py
``` ```
python/examples/pipeline/PaddleClas/MobileNetV1/benchmark.py
浏览文件 @ 12ab1380
# Copyright (c) 2021 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import sys import sys
import os import os
import base64 import base64
...@@ -5,13 +19,13 @@ import yaml ...@@ -5,13 +19,13 @@ import yaml
import requests import requests
import time import time
import json import json
try:
from paddle_serving_server_gpu.pipeline import PipelineClient from paddle_serving_server.pipeline import PipelineClient
except ImportError:
from paddle_serving_server.pipeline import PipelineClient
import numpy as np import numpy as np
from paddle_serving_client.utils import MultiThreadRunner from paddle_serving_client.utils import MultiThreadRunner
from paddle_serving_client.utils import benchmark_args, show_latency from paddle_serving_client.utils import benchmark_args, show_latency
def parse_benchmark(filein, fileout): def parse_benchmark(filein, fileout):
with open(filein, "r") as fin: with open(filein, "r") as fin:
res = yaml.load(fin) res = yaml.load(fin)
...@@ -24,6 +38,7 @@ def parse_benchmark(filein, fileout): ...@@ -24,6 +38,7 @@ def parse_benchmark(filein, fileout):
with open(fileout, "w") as fout: with open(fileout, "w") as fout:
yaml.dump(res, fout, default_flow_style=False) yaml.dump(res, fout, default_flow_style=False)
def gen_yml(device, gpu_id): def gen_yml(device, gpu_id):
fin = open("config.yml", "r") fin = open("config.yml", "r")
config = yaml.load(fin) config = yaml.load(fin)
...@@ -34,15 +49,17 @@ def gen_yml(device, gpu_id): ...@@ -34,15 +49,17 @@ def gen_yml(device, gpu_id):
config["op"]["imagenet"]["local_service_conf"]["devices"] = gpu_id config["op"]["imagenet"]["local_service_conf"]["devices"] = gpu_id
else: else:
config["op"]["imagenet"]["local_service_conf"]["device_type"] = 0 config["op"]["imagenet"]["local_service_conf"]["device_type"] = 0
with open("config2.yml", "w") as fout: with open("config2.yml", "w") as fout:
yaml.dump(config, fout, default_flow_style=False) yaml.dump(config, fout, default_flow_style=False)
def cv2_to_base64(image): def cv2_to_base64(image):
return base64.b64encode(image).decode('utf8') return base64.b64encode(image).decode('utf8')
def run_http(idx, batch_size): def run_http(idx, batch_size):
print("start thread ({})".format(idx)) print("start thread ({})".format(idx))
url = "http://127.0.0.1:18080/imagenet/prediction" url = "http://127.0.0.1:18080/imagenet/prediction"
start = time.time() start = time.time()
with open(os.path.join(".", "daisy.jpg"), 'rb') as file: with open(os.path.join(".", "daisy.jpg"), 'rb') as file:
...@@ -68,6 +85,7 @@ def run_http(idx, batch_size): ...@@ -68,6 +85,7 @@ def run_http(idx, batch_size):
end = time.time() end = time.time()
return [[end - start], latency_list, [total_num]] return [[end - start], latency_list, [total_num]]
def multithread_http(thread, batch_size): def multithread_http(thread, batch_size):
multi_thread_runner = MultiThreadRunner() multi_thread_runner = MultiThreadRunner()
start = time.time() start = time.time()
...@@ -87,6 +105,7 @@ def multithread_http(thread, batch_size): ...@@ -87,6 +105,7 @@ def multithread_http(thread, batch_size):
total_cost)) total_cost))
show_latency(result[1]) show_latency(result[1])
def run_rpc(thread, batch_size): def run_rpc(thread, batch_size):
client = PipelineClient() client = PipelineClient()
client.connect(['127.0.0.1:18080']) client.connect(['127.0.0.1:18080'])
...@@ -107,11 +126,12 @@ def run_rpc(thread, batch_size): ...@@ -107,11 +126,12 @@ def run_rpc(thread, batch_size):
def multithread_rpc(thraed, batch_size): def multithread_rpc(thraed, batch_size):
multi_thread_runner = MultiThreadRunner() multi_thread_runner = MultiThreadRunner()
result = multi_thread_runner.run(run_rpc , thread, batch_size) result = multi_thread_runner.run(run_rpc, thread, batch_size)
if __name__ == "__main__": if __name__ == "__main__":
if sys.argv[1] == "yaml": if sys.argv[1] == "yaml":
mode = sys.argv[2] # brpc/ local predictor mode = sys.argv[2] # brpc/ local predictor
thread = int(sys.argv[3]) thread = int(sys.argv[3])
device = sys.argv[4] device = sys.argv[4]
if device == "gpu": if device == "gpu":
...@@ -120,7 +140,7 @@ if __name__ == "__main__": ...@@ -120,7 +140,7 @@ if __name__ == "__main__":
gpu_id = None gpu_id = None
gen_yml(device, gpu_id) gen_yml(device, gpu_id)
elif sys.argv[1] == "run": elif sys.argv[1] == "run":
mode = sys.argv[2] # http/ rpc mode = sys.argv[2] # http/ rpc
thread = int(sys.argv[3]) thread = int(sys.argv[3])
batch_size = int(sys.argv[4]) batch_size = int(sys.argv[4])
if mode == "http": if mode == "http":
...@@ -131,4 +151,3 @@ if __name__ == "__main__": ...@@ -131,4 +151,3 @@ if __name__ == "__main__":
filein = sys.argv[2] filein = sys.argv[2]
fileout = sys.argv[3] fileout = sys.argv[3]
parse_benchmark(filein, fileout) parse_benchmark(filein, fileout)
python/examples/pipeline/PaddleClas/MobileNetV1/pipeline_rpc_client.py
浏览文件 @ 12ab1380
...@@ -11,10 +11,8 @@ ...@@ -11,10 +11,8 @@
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and # See the License for the specific language governing permissions and
# limitations under the License. # limitations under the License.
try:
from paddle_serving_server_gpu.pipeline import PipelineClient from paddle_serving_server.pipeline import PipelineClient
except ImportError:
from paddle_serving_server.pipeline import PipelineClient
import numpy as np import numpy as np
import requests import requests
import json import json
......
python/examples/pipeline/PaddleClas/MobileNetV1/resnet50_web_service.py
浏览文件 @ 12ab1380
...@@ -13,10 +13,8 @@ ...@@ -13,10 +13,8 @@
# limitations under the License. # limitations under the License.
import sys import sys
from paddle_serving_app.reader import Sequential, URL2Image, Resize, CenterCrop, RGB2BGR, Transpose, Div, Normalize, Base64ToImage from paddle_serving_app.reader import Sequential, URL2Image, Resize, CenterCrop, RGB2BGR, Transpose, Div, Normalize, Base64ToImage
try:
from paddle_serving_server_gpu.web_service import WebService, Op from paddle_serving_server.web_service import WebService, Op
except ImportError:
from paddle_serving_server.web_service import WebService, Op
import logging import logging
import numpy as np import numpy as np
import base64, cv2 import base64, cv2
......
python/examples/pipeline/PaddleClas/MobileNetV2/README.md
浏览文件 @ 12ab1380
...@@ -10,10 +10,10 @@ sh get_model.sh ...@@ -10,10 +10,10 @@ sh get_model.sh
## Start server ## Start server
``` ```
python resnet50_web_service.py &>log.txt & python3 resnet50_web_service.py &>log.txt &
``` ```
## RPC test ## RPC test
``` ```
python pipeline_rpc_client.py python3 pipeline_rpc_client.py
``` ```
python/examples/pipeline/PaddleClas/MobileNetV2/README_CN.md
浏览文件 @ 12ab1380
...@@ -10,11 +10,10 @@ sh get_model.sh ...@@ -10,11 +10,10 @@ sh get_model.sh
## 启动服务 ## 启动服务
``` ```
python resnet50_web_service.py &>log.txt & python3 resnet50_web_service.py &>log.txt &
``` ```
## 测试 ## 测试
``` ```
python pipeline_rpc_client.py python3 pipeline_rpc_client.py
``` ```
python/examples/pipeline/PaddleClas/MobileNetV2/benchmark.py
浏览文件 @ 12ab1380
# Copyright (c) 2021 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import sys import sys
import os import os
import base64 import base64
...@@ -5,13 +19,13 @@ import yaml ...@@ -5,13 +19,13 @@ import yaml
import requests import requests
import time import time
import json import json
try:
from paddle_serving_server_gpu.pipeline import PipelineClient from paddle_serving_server.pipeline import PipelineClient
except ImportError:
from paddle_serving_server.pipeline import PipelineClient
import numpy as np import numpy as np
from paddle_serving_client.utils import MultiThreadRunner from paddle_serving_client.utils import MultiThreadRunner
from paddle_serving_client.utils import benchmark_args, show_latency from paddle_serving_client.utils import benchmark_args, show_latency
def parse_benchmark(filein, fileout): def parse_benchmark(filein, fileout):
with open(filein, "r") as fin: with open(filein, "r") as fin:
res = yaml.load(fin) res = yaml.load(fin)
...@@ -24,6 +38,7 @@ def parse_benchmark(filein, fileout): ...@@ -24,6 +38,7 @@ def parse_benchmark(filein, fileout):
with open(fileout, "w") as fout: with open(fileout, "w") as fout:
yaml.dump(res, fout, default_flow_style=False) yaml.dump(res, fout, default_flow_style=False)
def gen_yml(device, gpu_id): def gen_yml(device, gpu_id):
fin = open("config.yml", "r") fin = open("config.yml", "r")
config = yaml.load(fin) config = yaml.load(fin)
...@@ -34,15 +49,17 @@ def gen_yml(device, gpu_id): ...@@ -34,15 +49,17 @@ def gen_yml(device, gpu_id):
config["op"]["imagenet"]["local_service_conf"]["devices"] = gpu_id config["op"]["imagenet"]["local_service_conf"]["devices"] = gpu_id
else: else:
config["op"]["imagenet"]["local_service_conf"]["device_type"] = 0 config["op"]["imagenet"]["local_service_conf"]["device_type"] = 0
with open("config2.yml", "w") as fout: with open("config2.yml", "w") as fout:
yaml.dump(config, fout, default_flow_style=False) yaml.dump(config, fout, default_flow_style=False)
def cv2_to_base64(image): def cv2_to_base64(image):
return base64.b64encode(image).decode('utf8') return base64.b64encode(image).decode('utf8')
def run_http(idx, batch_size): def run_http(idx, batch_size):
print("start thread ({})".format(idx)) print("start thread ({})".format(idx))
url = "http://127.0.0.1:18080/imagenet/prediction" url = "http://127.0.0.1:18080/imagenet/prediction"
start = time.time() start = time.time()
with open(os.path.join(".", "daisy.jpg"), 'rb') as file: with open(os.path.join(".", "daisy.jpg"), 'rb') as file:
...@@ -68,6 +85,7 @@ def run_http(idx, batch_size): ...@@ -68,6 +85,7 @@ def run_http(idx, batch_size):
end = time.time() end = time.time()
return [[end - start], latency_list, [total_num]] return [[end - start], latency_list, [total_num]]
def multithread_http(thread, batch_size): def multithread_http(thread, batch_size):
multi_thread_runner = MultiThreadRunner() multi_thread_runner = MultiThreadRunner()
start = time.time() start = time.time()
...@@ -87,6 +105,7 @@ def multithread_http(thread, batch_size): ...@@ -87,6 +105,7 @@ def multithread_http(thread, batch_size):
total_cost)) total_cost))
show_latency(result[1]) show_latency(result[1])
def run_rpc(thread, batch_size): def run_rpc(thread, batch_size):
client = PipelineClient() client = PipelineClient()
client.connect(['127.0.0.1:18080']) client.connect(['127.0.0.1:18080'])
...@@ -107,11 +126,12 @@ def run_rpc(thread, batch_size): ...@@ -107,11 +126,12 @@ def run_rpc(thread, batch_size):
def multithread_rpc(thraed, batch_size): def multithread_rpc(thraed, batch_size):
multi_thread_runner = MultiThreadRunner() multi_thread_runner = MultiThreadRunner()
result = multi_thread_runner.run(run_rpc , thread, batch_size) result = multi_thread_runner.run(run_rpc, thread, batch_size)
if __name__ == "__main__": if __name__ == "__main__":
if sys.argv[1] == "yaml": if sys.argv[1] == "yaml":
mode = sys.argv[2] # brpc/ local predictor mode = sys.argv[2] # brpc/ local predictor
thread = int(sys.argv[3]) thread = int(sys.argv[3])
device = sys.argv[4] device = sys.argv[4]
if device == "gpu": if device == "gpu":
...@@ -120,7 +140,7 @@ if __name__ == "__main__": ...@@ -120,7 +140,7 @@ if __name__ == "__main__":
gpu_id = None gpu_id = None
gen_yml(device, gpu_id) gen_yml(device, gpu_id)
elif sys.argv[1] == "run": elif sys.argv[1] == "run":
mode = sys.argv[2] # http/ rpc mode = sys.argv[2] # http/ rpc
thread = int(sys.argv[3]) thread = int(sys.argv[3])
batch_size = int(sys.argv[4]) batch_size = int(sys.argv[4])
if mode == "http": if mode == "http":
...@@ -131,4 +151,3 @@ if __name__ == "__main__": ...@@ -131,4 +151,3 @@ if __name__ == "__main__":
filein = sys.argv[2] filein = sys.argv[2]
fileout = sys.argv[3] fileout = sys.argv[3]
parse_benchmark(filein, fileout) parse_benchmark(filein, fileout)
python/examples/pipeline/PaddleClas/MobileNetV2/pipeline_rpc_client.py
浏览文件 @ 12ab1380
...@@ -11,10 +11,8 @@ ...@@ -11,10 +11,8 @@
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and # See the License for the specific language governing permissions and
# limitations under the License. # limitations under the License.
try:
from paddle_serving_server_gpu.pipeline import PipelineClient from paddle_serving_server.pipeline import PipelineClient
except ImportError:
from paddle_serving_server.pipeline import PipelineClient
import numpy as np import numpy as np
import requests import requests
import json import json
......
python/examples/pipeline/PaddleClas/MobileNetV2/resnet50_web_service.py
浏览文件 @ 12ab1380
...@@ -13,10 +13,8 @@ ...@@ -13,10 +13,8 @@
# limitations under the License. # limitations under the License.
import sys import sys
from paddle_serving_app.reader import Sequential, URL2Image, Resize, CenterCrop, RGB2BGR, Transpose, Div, Normalize, Base64ToImage from paddle_serving_app.reader import Sequential, URL2Image, Resize, CenterCrop, RGB2BGR, Transpose, Div, Normalize, Base64ToImage
try:
from paddle_serving_server_gpu.web_service import WebService, Op from paddle_serving_server.web_service import WebService, Op
except ImportError:
from paddle_serving_server.web_service import WebService, Op
import logging import logging
import numpy as np import numpy as np
import base64, cv2 import base64, cv2
......
python/examples/pipeline/PaddleClas/MobileNetV3_large_x1_0/README.md
浏览文件 @ 12ab1380
...@@ -10,10 +10,10 @@ sh get_model.sh ...@@ -10,10 +10,10 @@ sh get_model.sh
## Start server ## Start server
``` ```
python resnet50_web_service.py &>log.txt & python3 resnet50_web_service.py &>log.txt &
``` ```
## RPC test ## RPC test
``` ```
python pipeline_rpc_client.py python3 pipeline_rpc_client.py
``` ```
python/examples/pipeline/PaddleClas/MobileNetV3_large_x1_0/README_CN.md
浏览文件 @ 12ab1380
...@@ -10,11 +10,10 @@ sh get_model.sh ...@@ -10,11 +10,10 @@ sh get_model.sh
## 启动服务 ## 启动服务
``` ```
python resnet50_web_service.py &>log.txt & python3 resnet50_web_service.py &>log.txt &
``` ```
## 测试 ## 测试
``` ```
python pipeline_rpc_client.py python3 pipeline_rpc_client.py
``` ```
python/examples/pipeline/PaddleClas/MobileNetV3_large_x1_0/benchmark.py
浏览文件 @ 12ab1380
# Copyright (c) 2021 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import sys import sys
import os import os
import base64 import base64
...@@ -5,13 +19,13 @@ import yaml ...@@ -5,13 +19,13 @@ import yaml
import requests import requests
import time import time
import json import json
try:
from paddle_serving_server_gpu.pipeline import PipelineClient from paddle_serving_server.pipeline import PipelineClient
except ImportError:
from paddle_serving_server.pipeline import PipelineClient
import numpy as np import numpy as np
from paddle_serving_client.utils import MultiThreadRunner from paddle_serving_client.utils import MultiThreadRunner
from paddle_serving_client.utils import benchmark_args, show_latency from paddle_serving_client.utils import benchmark_args, show_latency
def parse_benchmark(filein, fileout): def parse_benchmark(filein, fileout):
with open(filein, "r") as fin: with open(filein, "r") as fin:
res = yaml.load(fin) res = yaml.load(fin)
...@@ -24,6 +38,7 @@ def parse_benchmark(filein, fileout): ...@@ -24,6 +38,7 @@ def parse_benchmark(filein, fileout):
with open(fileout, "w") as fout: with open(fileout, "w") as fout:
yaml.dump(res, fout, default_flow_style=False) yaml.dump(res, fout, default_flow_style=False)
def gen_yml(device, gpu_id): def gen_yml(device, gpu_id):
fin = open("config.yml", "r") fin = open("config.yml", "r")
config = yaml.load(fin) config = yaml.load(fin)
...@@ -34,15 +49,17 @@ def gen_yml(device, gpu_id): ...@@ -34,15 +49,17 @@ def gen_yml(device, gpu_id):
config["op"]["imagenet"]["local_service_conf"]["devices"] = gpu_id config["op"]["imagenet"]["local_service_conf"]["devices"] = gpu_id
else: else:
config["op"]["imagenet"]["local_service_conf"]["device_type"] = 0 config["op"]["imagenet"]["local_service_conf"]["device_type"] = 0
with open("config2.yml", "w") as fout: with open("config2.yml", "w") as fout:
yaml.dump(config, fout, default_flow_style=False) yaml.dump(config, fout, default_flow_style=False)
def cv2_to_base64(image): def cv2_to_base64(image):
return base64.b64encode(image).decode('utf8') return base64.b64encode(image).decode('utf8')
def run_http(idx, batch_size): def run_http(idx, batch_size):
print("start thread ({})".format(idx)) print("start thread ({})".format(idx))
url = "http://127.0.0.1:18080/imagenet/prediction" url = "http://127.0.0.1:18080/imagenet/prediction"
start = time.time() start = time.time()
with open(os.path.join(".", "daisy.jpg"), 'rb') as file: with open(os.path.join(".", "daisy.jpg"), 'rb') as file:
...@@ -68,6 +85,7 @@ def run_http(idx, batch_size): ...@@ -68,6 +85,7 @@ def run_http(idx, batch_size):
end = time.time() end = time.time()
return [[end - start], latency_list, [total_num]] return [[end - start], latency_list, [total_num]]
def multithread_http(thread, batch_size): def multithread_http(thread, batch_size):
multi_thread_runner = MultiThreadRunner() multi_thread_runner = MultiThreadRunner()
start = time.time() start = time.time()
...@@ -87,6 +105,7 @@ def multithread_http(thread, batch_size): ...@@ -87,6 +105,7 @@ def multithread_http(thread, batch_size):
total_cost)) total_cost))
show_latency(result[1]) show_latency(result[1])
def run_rpc(thread, batch_size): def run_rpc(thread, batch_size):
client = PipelineClient() client = PipelineClient()
client.connect(['127.0.0.1:18080']) client.connect(['127.0.0.1:18080'])
...@@ -107,11 +126,12 @@ def run_rpc(thread, batch_size): ...@@ -107,11 +126,12 @@ def run_rpc(thread, batch_size):
def multithread_rpc(thraed, batch_size): def multithread_rpc(thraed, batch_size):
multi_thread_runner = MultiThreadRunner() multi_thread_runner = MultiThreadRunner()
result = multi_thread_runner.run(run_rpc , thread, batch_size) result = multi_thread_runner.run(run_rpc, thread, batch_size)
if __name__ == "__main__": if __name__ == "__main__":
if sys.argv[1] == "yaml": if sys.argv[1] == "yaml":
mode = sys.argv[2] # brpc/ local predictor mode = sys.argv[2] # brpc/ local predictor
thread = int(sys.argv[3]) thread = int(sys.argv[3])
device = sys.argv[4] device = sys.argv[4]
if device == "gpu": if device == "gpu":
...@@ -120,7 +140,7 @@ if __name__ == "__main__": ...@@ -120,7 +140,7 @@ if __name__ == "__main__":
gpu_id = None gpu_id = None
gen_yml(device, gpu_id) gen_yml(device, gpu_id)
elif sys.argv[1] == "run": elif sys.argv[1] == "run":
mode = sys.argv[2] # http/ rpc mode = sys.argv[2] # http/ rpc
thread = int(sys.argv[3]) thread = int(sys.argv[3])
batch_size = int(sys.argv[4]) batch_size = int(sys.argv[4])
if mode == "http": if mode == "http":
...@@ -131,4 +151,3 @@ if __name__ == "__main__": ...@@ -131,4 +151,3 @@ if __name__ == "__main__":
filein = sys.argv[2] filein = sys.argv[2]
fileout = sys.argv[3] fileout = sys.argv[3]
parse_benchmark(filein, fileout) parse_benchmark(filein, fileout)
python/examples/pipeline/PaddleClas/MobileNetV3_large_x1_0/pipeline_rpc_client.py
浏览文件 @ 12ab1380
...@@ -11,10 +11,8 @@ ...@@ -11,10 +11,8 @@
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and # See the License for the specific language governing permissions and
# limitations under the License. # limitations under the License.
try:
from paddle_serving_server_gpu.pipeline import PipelineClient from paddle_serving_server.pipeline import PipelineClient
except ImportError:
from paddle_serving_server.pipeline import PipelineClient
import numpy as np import numpy as np
import requests import requests
import json import json
......
python/examples/pipeline/PaddleClas/MobileNetV3_large_x1_0/resnet50_web_service.py
浏览文件 @ 12ab1380
...@@ -13,10 +13,8 @@ ...@@ -13,10 +13,8 @@
# limitations under the License. # limitations under the License.
import sys import sys
from paddle_serving_app.reader import Sequential, URL2Image, Resize, CenterCrop, RGB2BGR, Transpose, Div, Normalize, Base64ToImage from paddle_serving_app.reader import Sequential, URL2Image, Resize, CenterCrop, RGB2BGR, Transpose, Div, Normalize, Base64ToImage
try:
from paddle_serving_server_gpu.web_service import WebService, Op from paddle_serving_server.web_service import WebService, Op
except ImportError:
from paddle_serving_server.web_service import WebService, Op
import logging import logging
import numpy as np import numpy as np
import base64, cv2 import base64, cv2
......
python/examples/pipeline/PaddleClas/ResNeXt101_vd_64x4d/README.md
浏览文件 @ 12ab1380
...@@ -10,10 +10,10 @@ sh get_model.sh ...@@ -10,10 +10,10 @@ sh get_model.sh
## Start server ## Start server
``` ```
python resnet50_web_service.py &>log.txt & python3 resnet50_web_service.py &>log.txt &
``` ```
## RPC test ## RPC test
``` ```
python pipeline_rpc_client.py python3 pipeline_rpc_client.py
``` ```
python/examples/pipeline/PaddleClas/ResNeXt101_vd_64x4d/README_CN.md
浏览文件 @ 12ab1380
...@@ -10,11 +10,10 @@ sh get_model.sh ...@@ -10,11 +10,10 @@ sh get_model.sh
## 启动服务 ## 启动服务
``` ```
python resnet50_web_service.py &>log.txt & python3 resnet50_web_service.py &>log.txt &
``` ```
## 测试 ## 测试
``` ```
python pipeline_rpc_client.py python3 pipeline_rpc_client.py
``` ```
python/examples/pipeline/PaddleClas/ResNeXt101_vd_64x4d/benchmark.py
浏览文件 @ 12ab1380
# Copyright (c) 2021 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import sys import sys
import os import os
import base64 import base64
...@@ -5,13 +19,13 @@ import yaml ...@@ -5,13 +19,13 @@ import yaml
import requests import requests
import time import time
import json import json
try:
from paddle_serving_server_gpu.pipeline import PipelineClient from paddle_serving_server.pipeline import PipelineClient
except ImportError:
from paddle_serving_server.pipeline import PipelineClient
import numpy as np import numpy as np
from paddle_serving_client.utils import MultiThreadRunner from paddle_serving_client.utils import MultiThreadRunner
from paddle_serving_client.utils import benchmark_args, show_latency from paddle_serving_client.utils import benchmark_args, show_latency
def parse_benchmark(filein, fileout): def parse_benchmark(filein, fileout):
with open(filein, "r") as fin: with open(filein, "r") as fin:
res = yaml.load(fin) res = yaml.load(fin)
...@@ -24,6 +38,7 @@ def parse_benchmark(filein, fileout): ...@@ -24,6 +38,7 @@ def parse_benchmark(filein, fileout):
with open(fileout, "w") as fout: with open(fileout, "w") as fout:
yaml.dump(res, fout, default_flow_style=False) yaml.dump(res, fout, default_flow_style=False)
def gen_yml(device, gpu_id): def gen_yml(device, gpu_id):
fin = open("config.yml", "r") fin = open("config.yml", "r")
config = yaml.load(fin) config = yaml.load(fin)
...@@ -34,15 +49,17 @@ def gen_yml(device, gpu_id): ...@@ -34,15 +49,17 @@ def gen_yml(device, gpu_id):
config["op"]["imagenet"]["local_service_conf"]["devices"] = gpu_id config["op"]["imagenet"]["local_service_conf"]["devices"] = gpu_id
else: else:
config["op"]["imagenet"]["local_service_conf"]["device_type"] = 0 config["op"]["imagenet"]["local_service_conf"]["device_type"] = 0
with open("config2.yml", "w") as fout: with open("config2.yml", "w") as fout:
yaml.dump(config, fout, default_flow_style=False) yaml.dump(config, fout, default_flow_style=False)
def cv2_to_base64(image): def cv2_to_base64(image):
return base64.b64encode(image).decode('utf8') return base64.b64encode(image).decode('utf8')
def run_http(idx, batch_size): def run_http(idx, batch_size):
print("start thread ({})".format(idx)) print("start thread ({})".format(idx))
url = "http://127.0.0.1:18080/imagenet/prediction" url = "http://127.0.0.1:18080/imagenet/prediction"
start = time.time() start = time.time()
with open(os.path.join(".", "daisy.jpg"), 'rb') as file: with open(os.path.join(".", "daisy.jpg"), 'rb') as file:
...@@ -68,6 +85,7 @@ def run_http(idx, batch_size): ...@@ -68,6 +85,7 @@ def run_http(idx, batch_size):
end = time.time() end = time.time()
return [[end - start], latency_list, [total_num]] return [[end - start], latency_list, [total_num]]
def multithread_http(thread, batch_size): def multithread_http(thread, batch_size):
multi_thread_runner = MultiThreadRunner() multi_thread_runner = MultiThreadRunner()
start = time.time() start = time.time()
...@@ -87,6 +105,7 @@ def multithread_http(thread, batch_size): ...@@ -87,6 +105,7 @@ def multithread_http(thread, batch_size):
total_cost)) total_cost))
show_latency(result[1]) show_latency(result[1])
def run_rpc(thread, batch_size): def run_rpc(thread, batch_size):
client = PipelineClient() client = PipelineClient()
client.connect(['127.0.0.1:18080']) client.connect(['127.0.0.1:18080'])
...@@ -107,11 +126,12 @@ def run_rpc(thread, batch_size): ...@@ -107,11 +126,12 @@ def run_rpc(thread, batch_size):
def multithread_rpc(thraed, batch_size): def multithread_rpc(thraed, batch_size):
multi_thread_runner = MultiThreadRunner() multi_thread_runner = MultiThreadRunner()
result = multi_thread_runner.run(run_rpc , thread, batch_size) result = multi_thread_runner.run(run_rpc, thread, batch_size)
if __name__ == "__main__": if __name__ == "__main__":
if sys.argv[1] == "yaml": if sys.argv[1] == "yaml":
mode = sys.argv[2] # brpc/ local predictor mode = sys.argv[2] # brpc/ local predictor
thread = int(sys.argv[3]) thread = int(sys.argv[3])
device = sys.argv[4] device = sys.argv[4]
if device == "gpu": if device == "gpu":
...@@ -120,7 +140,7 @@ if __name__ == "__main__": ...@@ -120,7 +140,7 @@ if __name__ == "__main__":
gpu_id = None gpu_id = None
gen_yml(device, gpu_id) gen_yml(device, gpu_id)
elif sys.argv[1] == "run": elif sys.argv[1] == "run":
mode = sys.argv[2] # http/ rpc mode = sys.argv[2] # http/ rpc
thread = int(sys.argv[3]) thread = int(sys.argv[3])
batch_size = int(sys.argv[4]) batch_size = int(sys.argv[4])
if mode == "http": if mode == "http":
...@@ -131,4 +151,3 @@ if __name__ == "__main__": ...@@ -131,4 +151,3 @@ if __name__ == "__main__":
filein = sys.argv[2] filein = sys.argv[2]
fileout = sys.argv[3] fileout = sys.argv[3]
parse_benchmark(filein, fileout) parse_benchmark(filein, fileout)
python/examples/pipeline/PaddleClas/ResNeXt101_vd_64x4d/pipeline_rpc_client.py
浏览文件 @ 12ab1380
...@@ -11,10 +11,8 @@ ...@@ -11,10 +11,8 @@
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and # See the License for the specific language governing permissions and
# limitations under the License. # limitations under the License.
try:
from paddle_serving_server_gpu.pipeline import PipelineClient from paddle_serving_server.pipeline import PipelineClient
except ImportError:
from paddle_serving_server.pipeline import PipelineClient
import numpy as np import numpy as np
import requests import requests
import json import json
......
python/examples/pipeline/PaddleClas/ResNeXt101_vd_64x4d/resnet50_web_service.py
浏览文件 @ 12ab1380
...@@ -13,10 +13,8 @@ ...@@ -13,10 +13,8 @@
# limitations under the License. # limitations under the License.
import sys import sys
from paddle_serving_app.reader import Sequential, URL2Image, Resize, CenterCrop, RGB2BGR, Transpose, Div, Normalize, Base64ToImage from paddle_serving_app.reader import Sequential, URL2Image, Resize, CenterCrop, RGB2BGR, Transpose, Div, Normalize, Base64ToImage
try:
from paddle_serving_server_gpu.web_service import WebService, Op from paddle_serving_server.web_service import WebService, Op
except ImportError:
from paddle_serving_server.web_service import WebService, Op
import logging import logging
import numpy as np import numpy as np
import base64, cv2 import base64, cv2
......
python/examples/pipeline/PaddleClas/ResNet50_vd/README.md
浏览文件 @ 12ab1380
...@@ -10,10 +10,10 @@ sh get_model.sh ...@@ -10,10 +10,10 @@ sh get_model.sh
## Start server ## Start server
``` ```
python resnet50_web_service.py &>log.txt & python3 resnet50_web_service.py &>log.txt &
``` ```
## RPC test ## RPC test
``` ```
python pipeline_rpc_client.py python3 pipeline_rpc_client.py
``` ```
python/examples/pipeline/PaddleClas/ResNet50_vd/README_CN.md
浏览文件 @ 12ab1380
...@@ -10,11 +10,10 @@ sh get_model.sh ...@@ -10,11 +10,10 @@ sh get_model.sh
## 启动服务 ## 启动服务
``` ```
python resnet50_web_service.py &>log.txt & python3 resnet50_web_service.py &>log.txt &
``` ```
## 测试 ## 测试
``` ```
python pipeline_rpc_client.py python3 pipeline_rpc_client.py
``` ```
python/examples/pipeline/PaddleClas/ResNet50_vd/benchmark.py
浏览文件 @ 12ab1380
# Copyright (c) 2021 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import sys import sys
import os import os
import base64 import base64
...@@ -5,13 +19,13 @@ import yaml ...@@ -5,13 +19,13 @@ import yaml
import requests import requests
import time import time
import json import json
try:
from paddle_serving_server_gpu.pipeline import PipelineClient from paddle_serving_server.pipeline import PipelineClient
except ImportError:
from paddle_serving_server.pipeline import PipelineClient
import numpy as np import numpy as np
from paddle_serving_client.utils import MultiThreadRunner from paddle_serving_client.utils import MultiThreadRunner
from paddle_serving_client.utils import benchmark_args, show_latency from paddle_serving_client.utils import benchmark_args, show_latency
def parse_benchmark(filein, fileout): def parse_benchmark(filein, fileout):
with open(filein, "r") as fin: with open(filein, "r") as fin:
res = yaml.load(fin) res = yaml.load(fin)
...@@ -24,6 +38,7 @@ def parse_benchmark(filein, fileout): ...@@ -24,6 +38,7 @@ def parse_benchmark(filein, fileout):
with open(fileout, "w") as fout: with open(fileout, "w") as fout:
yaml.dump(res, fout, default_flow_style=False) yaml.dump(res, fout, default_flow_style=False)
def gen_yml(device, gpu_id): def gen_yml(device, gpu_id):
fin = open("config.yml", "r") fin = open("config.yml", "r")
config = yaml.load(fin) config = yaml.load(fin)
...@@ -34,15 +49,17 @@ def gen_yml(device, gpu_id): ...@@ -34,15 +49,17 @@ def gen_yml(device, gpu_id):
config["op"]["imagenet"]["local_service_conf"]["devices"] = gpu_id config["op"]["imagenet"]["local_service_conf"]["devices"] = gpu_id
else: else:
config["op"]["imagenet"]["local_service_conf"]["device_type"] = 0 config["op"]["imagenet"]["local_service_conf"]["device_type"] = 0
with open("config2.yml", "w") as fout: with open("config2.yml", "w") as fout:
yaml.dump(config, fout, default_flow_style=False) yaml.dump(config, fout, default_flow_style=False)
def cv2_to_base64(image): def cv2_to_base64(image):
return base64.b64encode(image).decode('utf8') return base64.b64encode(image).decode('utf8')
def run_http(idx, batch_size): def run_http(idx, batch_size):
print("start thread ({})".format(idx)) print("start thread ({})".format(idx))
url = "http://127.0.0.1:18080/imagenet/prediction" url = "http://127.0.0.1:18080/imagenet/prediction"
start = time.time() start = time.time()
with open(os.path.join(".", "daisy.jpg"), 'rb') as file: with open(os.path.join(".", "daisy.jpg"), 'rb') as file:
...@@ -68,6 +85,7 @@ def run_http(idx, batch_size): ...@@ -68,6 +85,7 @@ def run_http(idx, batch_size):
end = time.time() end = time.time()
return [[end - start], latency_list, [total_num]] return [[end - start], latency_list, [total_num]]
def multithread_http(thread, batch_size): def multithread_http(thread, batch_size):
multi_thread_runner = MultiThreadRunner() multi_thread_runner = MultiThreadRunner()
start = time.time() start = time.time()
...@@ -87,6 +105,7 @@ def multithread_http(thread, batch_size): ...@@ -87,6 +105,7 @@ def multithread_http(thread, batch_size):
total_cost)) total_cost))
show_latency(result[1]) show_latency(result[1])
def run_rpc(thread, batch_size): def run_rpc(thread, batch_size):
client = PipelineClient() client = PipelineClient()
client.connect(['127.0.0.1:18080']) client.connect(['127.0.0.1:18080'])
...@@ -107,11 +126,12 @@ def run_rpc(thread, batch_size): ...@@ -107,11 +126,12 @@ def run_rpc(thread, batch_size):
def multithread_rpc(thraed, batch_size): def multithread_rpc(thraed, batch_size):
multi_thread_runner = MultiThreadRunner() multi_thread_runner = MultiThreadRunner()
result = multi_thread_runner.run(run_rpc , thread, batch_size) result = multi_thread_runner.run(run_rpc, thread, batch_size)
if __name__ == "__main__": if __name__ == "__main__":
if sys.argv[1] == "yaml": if sys.argv[1] == "yaml":
mode = sys.argv[2] # brpc/ local predictor mode = sys.argv[2] # brpc/ local predictor
thread = int(sys.argv[3]) thread = int(sys.argv[3])
device = sys.argv[4] device = sys.argv[4]
if device == "gpu": if device == "gpu":
...@@ -120,7 +140,7 @@ if __name__ == "__main__": ...@@ -120,7 +140,7 @@ if __name__ == "__main__":
gpu_id = None gpu_id = None
gen_yml(device, gpu_id) gen_yml(device, gpu_id)
elif sys.argv[1] == "run": elif sys.argv[1] == "run":
mode = sys.argv[2] # http/ rpc mode = sys.argv[2] # http/ rpc
thread = int(sys.argv[3]) thread = int(sys.argv[3])
batch_size = int(sys.argv[4]) batch_size = int(sys.argv[4])
if mode == "http": if mode == "http":
...@@ -131,4 +151,3 @@ if __name__ == "__main__": ...@@ -131,4 +151,3 @@ if __name__ == "__main__":
filein = sys.argv[2] filein = sys.argv[2]
fileout = sys.argv[3] fileout = sys.argv[3]
parse_benchmark(filein, fileout) parse_benchmark(filein, fileout)
python/examples/pipeline/PaddleClas/ResNet50_vd/pipeline_rpc_client.py
浏览文件 @ 12ab1380
...@@ -11,10 +11,8 @@ ...@@ -11,10 +11,8 @@
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and # See the License for the specific language governing permissions and
# limitations under the License. # limitations under the License.
try:
from paddle_serving_server_gpu.pipeline import PipelineClient from paddle_serving_server.pipeline import PipelineClient
except ImportError:
from paddle_serving_server.pipeline import PipelineClient
import numpy as np import numpy as np
import requests import requests
import json import json
......
python/examples/pipeline/PaddleClas/ResNet50_vd/resnet50_web_service.py
浏览文件 @ 12ab1380
...@@ -13,10 +13,8 @@ ...@@ -13,10 +13,8 @@
# limitations under the License. # limitations under the License.
import sys import sys
from paddle_serving_app.reader import Sequential, URL2Image, Resize, CenterCrop, RGB2BGR, Transpose, Div, Normalize, Base64ToImage from paddle_serving_app.reader import Sequential, URL2Image, Resize, CenterCrop, RGB2BGR, Transpose, Div, Normalize, Base64ToImage
try:
from paddle_serving_server_gpu.web_service import WebService, Op from paddle_serving_server.web_service import WebService, Op
except ImportError:
from paddle_serving_server.web_service import WebService, Op
import logging import logging
import numpy as np import numpy as np
import base64, cv2 import base64, cv2
......
python/examples/pipeline/PaddleClas/ResNet50_vd_FPGM/README.md
浏览文件 @ 12ab1380
...@@ -10,10 +10,10 @@ sh get_model.sh ...@@ -10,10 +10,10 @@ sh get_model.sh
## Start server ## Start server
``` ```
python resnet50_web_service.py &>log.txt & python3 resnet50_web_service.py &>log.txt &
``` ```
## RPC test ## RPC test
``` ```
python pipeline_rpc_client.py python3 pipeline_rpc_client.py
``` ```
python/examples/pipeline/PaddleClas/ResNet50_vd_FPGM/README_CN.md
浏览文件 @ 12ab1380
...@@ -10,11 +10,10 @@ sh get_model.sh ...@@ -10,11 +10,10 @@ sh get_model.sh
## 启动服务 ## 启动服务
``` ```
python resnet50_web_service.py &>log.txt & python3 resnet50_web_service.py &>log.txt &
``` ```
## 测试 ## 测试
``` ```
python pipeline_rpc_client.py python3 pipeline_rpc_client.py
``` ```
python/examples/pipeline/PaddleClas/ResNet50_vd_FPGM/benchmark.py
浏览文件 @ 12ab1380
# Copyright (c) 2021 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import sys import sys
import os import os
import base64 import base64
...@@ -5,13 +19,13 @@ import yaml ...@@ -5,13 +19,13 @@ import yaml
import requests import requests
import time import time
import json import json
try:
from paddle_serving_server_gpu.pipeline import PipelineClient from paddle_serving_server.pipeline import PipelineClient
except ImportError:
from paddle_serving_server.pipeline import PipelineClient
import numpy as np import numpy as np
from paddle_serving_client.utils import MultiThreadRunner from paddle_serving_client.utils import MultiThreadRunner
from paddle_serving_client.utils import benchmark_args, show_latency from paddle_serving_client.utils import benchmark_args, show_latency
def parse_benchmark(filein, fileout): def parse_benchmark(filein, fileout):
with open(filein, "r") as fin: with open(filein, "r") as fin:
res = yaml.load(fin) res = yaml.load(fin)
...@@ -24,6 +38,7 @@ def parse_benchmark(filein, fileout): ...@@ -24,6 +38,7 @@ def parse_benchmark(filein, fileout):
with open(fileout, "w") as fout: with open(fileout, "w") as fout:
yaml.dump(res, fout, default_flow_style=False) yaml.dump(res, fout, default_flow_style=False)
def gen_yml(device, gpu_id): def gen_yml(device, gpu_id):
fin = open("config.yml", "r") fin = open("config.yml", "r")
config = yaml.load(fin) config = yaml.load(fin)
...@@ -34,15 +49,17 @@ def gen_yml(device, gpu_id): ...@@ -34,15 +49,17 @@ def gen_yml(device, gpu_id):
config["op"]["imagenet"]["local_service_conf"]["devices"] = gpu_id config["op"]["imagenet"]["local_service_conf"]["devices"] = gpu_id
else: else:
config["op"]["imagenet"]["local_service_conf"]["device_type"] = 0 config["op"]["imagenet"]["local_service_conf"]["device_type"] = 0
with open("config2.yml", "w") as fout: with open("config2.yml", "w") as fout:
yaml.dump(config, fout, default_flow_style=False) yaml.dump(config, fout, default_flow_style=False)
def cv2_to_base64(image): def cv2_to_base64(image):
return base64.b64encode(image).decode('utf8') return base64.b64encode(image).decode('utf8')
def run_http(idx, batch_size): def run_http(idx, batch_size):
print("start thread ({})".format(idx)) print("start thread ({})".format(idx))
url = "http://127.0.0.1:18080/imagenet/prediction" url = "http://127.0.0.1:18080/imagenet/prediction"
start = time.time() start = time.time()
with open(os.path.join(".", "daisy.jpg"), 'rb') as file: with open(os.path.join(".", "daisy.jpg"), 'rb') as file:
...@@ -68,6 +85,7 @@ def run_http(idx, batch_size): ...@@ -68,6 +85,7 @@ def run_http(idx, batch_size):
end = time.time() end = time.time()
return [[end - start], latency_list, [total_num]] return [[end - start], latency_list, [total_num]]
def multithread_http(thread, batch_size): def multithread_http(thread, batch_size):
multi_thread_runner = MultiThreadRunner() multi_thread_runner = MultiThreadRunner()
start = time.time() start = time.time()
...@@ -87,6 +105,7 @@ def multithread_http(thread, batch_size): ...@@ -87,6 +105,7 @@ def multithread_http(thread, batch_size):
total_cost)) total_cost))
show_latency(result[1]) show_latency(result[1])
def run_rpc(thread, batch_size): def run_rpc(thread, batch_size):
client = PipelineClient() client = PipelineClient()
client.connect(['127.0.0.1:18080']) client.connect(['127.0.0.1:18080'])
...@@ -107,11 +126,12 @@ def run_rpc(thread, batch_size): ...@@ -107,11 +126,12 @@ def run_rpc(thread, batch_size):
def multithread_rpc(thraed, batch_size): def multithread_rpc(thraed, batch_size):
multi_thread_runner = MultiThreadRunner() multi_thread_runner = MultiThreadRunner()
result = multi_thread_runner.run(run_rpc , thread, batch_size) result = multi_thread_runner.run(run_rpc, thread, batch_size)
if __name__ == "__main__": if __name__ == "__main__":
if sys.argv[1] == "yaml": if sys.argv[1] == "yaml":
mode = sys.argv[2] # brpc/ local predictor mode = sys.argv[2] # brpc/ local predictor
thread = int(sys.argv[3]) thread = int(sys.argv[3])
device = sys.argv[4] device = sys.argv[4]
if device == "gpu": if device == "gpu":
...@@ -120,7 +140,7 @@ if __name__ == "__main__": ...@@ -120,7 +140,7 @@ if __name__ == "__main__":
gpu_id = None gpu_id = None
gen_yml(device, gpu_id) gen_yml(device, gpu_id)
elif sys.argv[1] == "run": elif sys.argv[1] == "run":
mode = sys.argv[2] # http/ rpc mode = sys.argv[2] # http/ rpc
thread = int(sys.argv[3]) thread = int(sys.argv[3])
batch_size = int(sys.argv[4]) batch_size = int(sys.argv[4])
if mode == "http": if mode == "http":
...@@ -131,4 +151,3 @@ if __name__ == "__main__": ...@@ -131,4 +151,3 @@ if __name__ == "__main__":
filein = sys.argv[2] filein = sys.argv[2]
fileout = sys.argv[3] fileout = sys.argv[3]
parse_benchmark(filein, fileout) parse_benchmark(filein, fileout)
python/examples/pipeline/PaddleClas/ResNet50_vd_FPGM/pipeline_rpc_client.py
浏览文件 @ 12ab1380
...@@ -11,10 +11,8 @@ ...@@ -11,10 +11,8 @@
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and # See the License for the specific language governing permissions and
# limitations under the License. # limitations under the License.
try:
from paddle_serving_server_gpu.pipeline import PipelineClient from paddle_serving_server.pipeline import PipelineClient
except ImportError:
from paddle_serving_server.pipeline import PipelineClient
import numpy as np import numpy as np
import requests import requests
import json import json
......
python/examples/pipeline/PaddleClas/ResNet50_vd_FPGM/resnet50_web_service.py
浏览文件 @ 12ab1380
...@@ -13,10 +13,8 @@ ...@@ -13,10 +13,8 @@
# limitations under the License. # limitations under the License.
import sys import sys
from paddle_serving_app.reader import Sequential, URL2Image, Resize, CenterCrop, RGB2BGR, Transpose, Div, Normalize, Base64ToImage from paddle_serving_app.reader import Sequential, URL2Image, Resize, CenterCrop, RGB2BGR, Transpose, Div, Normalize, Base64ToImage
try:
from paddle_serving_server_gpu.web_service import WebService, Op from paddle_serving_server.web_service import WebService, Op
except ImportError:
from paddle_serving_server.web_service import WebService, Op
import logging import logging
import numpy as np import numpy as np
import base64, cv2 import base64, cv2
......
python/examples/pipeline/PaddleClas/ResNet50_vd_KL/README.md
浏览文件 @ 12ab1380
...@@ -10,10 +10,10 @@ sh get_model.sh ...@@ -10,10 +10,10 @@ sh get_model.sh
## Start server ## Start server
``` ```
python resnet50_web_service.py &>log.txt & python3 resnet50_web_service.py &>log.txt &
``` ```
## RPC test ## RPC test
``` ```
python pipeline_rpc_client.py python3 pipeline_rpc_client.py
``` ```
python/examples/pipeline/PaddleClas/ResNet50_vd_KL/README_CN.md
浏览文件 @ 12ab1380
...@@ -10,11 +10,10 @@ sh get_model.sh ...@@ -10,11 +10,10 @@ sh get_model.sh
## 启动服务 ## 启动服务
``` ```
python resnet50_web_service.py &>log.txt & python3 resnet50_web_service.py &>log.txt &
``` ```
## 测试 ## 测试
``` ```
python pipeline_rpc_client.py python3 pipeline_rpc_client.py
``` ```
python/examples/pipeline/PaddleClas/ResNet50_vd_KL/benchmark.py
浏览文件 @ 12ab1380
# Copyright (c) 2021 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import sys import sys
import os import os
import base64 import base64
...@@ -5,13 +19,13 @@ import yaml ...@@ -5,13 +19,13 @@ import yaml
import requests import requests
import time import time
import json import json
try:
from paddle_serving_server_gpu.pipeline import PipelineClient from paddle_serving_server.pipeline import PipelineClient
except ImportError:
from paddle_serving_server.pipeline import PipelineClient
import numpy as np import numpy as np
from paddle_serving_client.utils import MultiThreadRunner from paddle_serving_client.utils import MultiThreadRunner
from paddle_serving_client.utils import benchmark_args, show_latency from paddle_serving_client.utils import benchmark_args, show_latency
def parse_benchmark(filein, fileout): def parse_benchmark(filein, fileout):
with open(filein, "r") as fin: with open(filein, "r") as fin:
res = yaml.load(fin) res = yaml.load(fin)
...@@ -24,6 +38,7 @@ def parse_benchmark(filein, fileout): ...@@ -24,6 +38,7 @@ def parse_benchmark(filein, fileout):
with open(fileout, "w") as fout: with open(fileout, "w") as fout:
yaml.dump(res, fout, default_flow_style=False) yaml.dump(res, fout, default_flow_style=False)
def gen_yml(device, gpu_id): def gen_yml(device, gpu_id):
fin = open("config.yml", "r") fin = open("config.yml", "r")
config = yaml.load(fin) config = yaml.load(fin)
...@@ -34,15 +49,17 @@ def gen_yml(device, gpu_id): ...@@ -34,15 +49,17 @@ def gen_yml(device, gpu_id):
config["op"]["imagenet"]["local_service_conf"]["devices"] = gpu_id config["op"]["imagenet"]["local_service_conf"]["devices"] = gpu_id
else: else:
config["op"]["imagenet"]["local_service_conf"]["device_type"] = 0 config["op"]["imagenet"]["local_service_conf"]["device_type"] = 0
with open("config2.yml", "w") as fout: with open("config2.yml", "w") as fout:
yaml.dump(config, fout, default_flow_style=False) yaml.dump(config, fout, default_flow_style=False)
def cv2_to_base64(image): def cv2_to_base64(image):
return base64.b64encode(image).decode('utf8') return base64.b64encode(image).decode('utf8')
def run_http(idx, batch_size): def run_http(idx, batch_size):
print("start thread ({})".format(idx)) print("start thread ({})".format(idx))
url = "http://127.0.0.1:18080/imagenet/prediction" url = "http://127.0.0.1:18080/imagenet/prediction"
start = time.time() start = time.time()
with open(os.path.join(".", "daisy.jpg"), 'rb') as file: with open(os.path.join(".", "daisy.jpg"), 'rb') as file:
...@@ -68,6 +85,7 @@ def run_http(idx, batch_size): ...@@ -68,6 +85,7 @@ def run_http(idx, batch_size):
end = time.time() end = time.time()
return [[end - start], latency_list, [total_num]] return [[end - start], latency_list, [total_num]]
def multithread_http(thread, batch_size): def multithread_http(thread, batch_size):
multi_thread_runner = MultiThreadRunner() multi_thread_runner = MultiThreadRunner()
start = time.time() start = time.time()
...@@ -87,6 +105,7 @@ def multithread_http(thread, batch_size): ...@@ -87,6 +105,7 @@ def multithread_http(thread, batch_size):
total_cost)) total_cost))
show_latency(result[1]) show_latency(result[1])
def run_rpc(thread, batch_size): def run_rpc(thread, batch_size):
client = PipelineClient() client = PipelineClient()
client.connect(['127.0.0.1:18080']) client.connect(['127.0.0.1:18080'])
...@@ -107,11 +126,12 @@ def run_rpc(thread, batch_size): ...@@ -107,11 +126,12 @@ def run_rpc(thread, batch_size):
def multithread_rpc(thraed, batch_size): def multithread_rpc(thraed, batch_size):
multi_thread_runner = MultiThreadRunner() multi_thread_runner = MultiThreadRunner()
result = multi_thread_runner.run(run_rpc , thread, batch_size) result = multi_thread_runner.run(run_rpc, thread, batch_size)
if __name__ == "__main__": if __name__ == "__main__":
if sys.argv[1] == "yaml": if sys.argv[1] == "yaml":
mode = sys.argv[2] # brpc/ local predictor mode = sys.argv[2] # brpc/ local predictor
thread = int(sys.argv[3]) thread = int(sys.argv[3])
device = sys.argv[4] device = sys.argv[4]
if device == "gpu": if device == "gpu":
...@@ -120,7 +140,7 @@ if __name__ == "__main__": ...@@ -120,7 +140,7 @@ if __name__ == "__main__":
gpu_id = None gpu_id = None
gen_yml(device, gpu_id) gen_yml(device, gpu_id)
elif sys.argv[1] == "run": elif sys.argv[1] == "run":
mode = sys.argv[2] # http/ rpc mode = sys.argv[2] # http/ rpc
thread = int(sys.argv[3]) thread = int(sys.argv[3])
batch_size = int(sys.argv[4]) batch_size = int(sys.argv[4])
if mode == "http": if mode == "http":
...@@ -131,4 +151,3 @@ if __name__ == "__main__": ...@@ -131,4 +151,3 @@ if __name__ == "__main__":
filein = sys.argv[2] filein = sys.argv[2]
fileout = sys.argv[3] fileout = sys.argv[3]
parse_benchmark(filein, fileout) parse_benchmark(filein, fileout)
python/examples/pipeline/PaddleClas/ResNet50_vd_KL/pipeline_rpc_client.py
浏览文件 @ 12ab1380
...@@ -11,10 +11,8 @@ ...@@ -11,10 +11,8 @@
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and # See the License for the specific language governing permissions and
# limitations under the License. # limitations under the License.
try:
from paddle_serving_server_gpu.pipeline import PipelineClient from paddle_serving_server.pipeline import PipelineClient
except ImportError:
from paddle_serving_server.pipeline import PipelineClient
import numpy as np import numpy as np
import requests import requests
import json import json
......
python/examples/pipeline/PaddleClas/ResNet50_vd_KL/resnet50_web_service.py
浏览文件 @ 12ab1380
...@@ -13,10 +13,8 @@ ...@@ -13,10 +13,8 @@
# limitations under the License. # limitations under the License.
import sys import sys
from paddle_serving_app.reader import Sequential, URL2Image, Resize, CenterCrop, RGB2BGR, Transpose, Div, Normalize, Base64ToImage from paddle_serving_app.reader import Sequential, URL2Image, Resize, CenterCrop, RGB2BGR, Transpose, Div, Normalize, Base64ToImage
try:
from paddle_serving_server_gpu.web_service import WebService, Op from paddle_serving_server.web_service import WebService, Op
except ImportError:
from paddle_serving_server.web_service import WebService, Op
import logging import logging
import numpy as np import numpy as np
import base64, cv2 import base64, cv2
......
python/examples/pipeline/PaddleClas/ResNet50_vd_PACT/README.md
浏览文件 @ 12ab1380
...@@ -10,10 +10,10 @@ sh get_model.sh ...@@ -10,10 +10,10 @@ sh get_model.sh
## Start server ## Start server
``` ```
python resnet50_web_service.py &>log.txt & python3 resnet50_web_service.py &>log.txt &
``` ```
## RPC test ## RPC test
``` ```
python pipeline_rpc_client.py python3 pipeline_rpc_client.py
``` ```
python/examples/pipeline/PaddleClas/ResNet50_vd_PACT/README_CN.md
浏览文件 @ 12ab1380
...@@ -10,11 +10,10 @@ sh get_model.sh ...@@ -10,11 +10,10 @@ sh get_model.sh
## 启动服务 ## 启动服务
``` ```
python resnet50_web_service.py &>log.txt & python3 resnet50_web_service.py &>log.txt &
``` ```
## 测试 ## 测试
``` ```
python pipeline_rpc_client.py python3 pipeline_rpc_client.py
``` ```
python/examples/pipeline/PaddleClas/ResNet50_vd_PACT/benchmark.py
浏览文件 @ 12ab1380
# Copyright (c) 2021 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import sys import sys
import os import os
import base64 import base64
...@@ -5,13 +19,13 @@ import yaml ...@@ -5,13 +19,13 @@ import yaml
import requests import requests
import time import time
import json import json
try:
from paddle_serving_server_gpu.pipeline import PipelineClient from paddle_serving_server.pipeline import PipelineClient
except ImportError:
from paddle_serving_server.pipeline import PipelineClient
import numpy as np import numpy as np
from paddle_serving_client.utils import MultiThreadRunner from paddle_serving_client.utils import MultiThreadRunner
from paddle_serving_client.utils import benchmark_args, show_latency from paddle_serving_client.utils import benchmark_args, show_latency
def parse_benchmark(filein, fileout): def parse_benchmark(filein, fileout):
with open(filein, "r") as fin: with open(filein, "r") as fin:
res = yaml.load(fin) res = yaml.load(fin)
...@@ -24,6 +38,7 @@ def parse_benchmark(filein, fileout): ...@@ -24,6 +38,7 @@ def parse_benchmark(filein, fileout):
with open(fileout, "w") as fout: with open(fileout, "w") as fout:
yaml.dump(res, fout, default_flow_style=False) yaml.dump(res, fout, default_flow_style=False)
def gen_yml(device, gpu_id): def gen_yml(device, gpu_id):
fin = open("config.yml", "r") fin = open("config.yml", "r")
config = yaml.load(fin) config = yaml.load(fin)
...@@ -34,15 +49,17 @@ def gen_yml(device, gpu_id): ...@@ -34,15 +49,17 @@ def gen_yml(device, gpu_id):
config["op"]["imagenet"]["local_service_conf"]["devices"] = gpu_id config["op"]["imagenet"]["local_service_conf"]["devices"] = gpu_id
else: else:
config["op"]["imagenet"]["local_service_conf"]["device_type"] = 0 config["op"]["imagenet"]["local_service_conf"]["device_type"] = 0
with open("config2.yml", "w") as fout: with open("config2.yml", "w") as fout:
yaml.dump(config, fout, default_flow_style=False) yaml.dump(config, fout, default_flow_style=False)
def cv2_to_base64(image): def cv2_to_base64(image):
return base64.b64encode(image).decode('utf8') return base64.b64encode(image).decode('utf8')
def run_http(idx, batch_size): def run_http(idx, batch_size):
print("start thread ({})".format(idx)) print("start thread ({})".format(idx))
url = "http://127.0.0.1:18080/imagenet/prediction" url = "http://127.0.0.1:18080/imagenet/prediction"
start = time.time() start = time.time()
with open(os.path.join(".", "daisy.jpg"), 'rb') as file: with open(os.path.join(".", "daisy.jpg"), 'rb') as file:
...@@ -68,6 +85,7 @@ def run_http(idx, batch_size): ...@@ -68,6 +85,7 @@ def run_http(idx, batch_size):
end = time.time() end = time.time()
return [[end - start], latency_list, [total_num]] return [[end - start], latency_list, [total_num]]
def multithread_http(thread, batch_size): def multithread_http(thread, batch_size):
multi_thread_runner = MultiThreadRunner() multi_thread_runner = MultiThreadRunner()
start = time.time() start = time.time()
...@@ -87,6 +105,7 @@ def multithread_http(thread, batch_size): ...@@ -87,6 +105,7 @@ def multithread_http(thread, batch_size):
total_cost)) total_cost))
show_latency(result[1]) show_latency(result[1])
def run_rpc(thread, batch_size): def run_rpc(thread, batch_size):
client = PipelineClient() client = PipelineClient()
client.connect(['127.0.0.1:18080']) client.connect(['127.0.0.1:18080'])
...@@ -107,11 +126,12 @@ def run_rpc(thread, batch_size): ...@@ -107,11 +126,12 @@ def run_rpc(thread, batch_size):
def multithread_rpc(thraed, batch_size): def multithread_rpc(thraed, batch_size):
multi_thread_runner = MultiThreadRunner() multi_thread_runner = MultiThreadRunner()
result = multi_thread_runner.run(run_rpc , thread, batch_size) result = multi_thread_runner.run(run_rpc, thread, batch_size)
if __name__ == "__main__": if __name__ == "__main__":
if sys.argv[1] == "yaml": if sys.argv[1] == "yaml":
mode = sys.argv[2] # brpc/ local predictor mode = sys.argv[2] # brpc/ local predictor
thread = int(sys.argv[3]) thread = int(sys.argv[3])
device = sys.argv[4] device = sys.argv[4]
if device == "gpu": if device == "gpu":
...@@ -120,7 +140,7 @@ if __name__ == "__main__": ...@@ -120,7 +140,7 @@ if __name__ == "__main__":
gpu_id = None gpu_id = None
gen_yml(device, gpu_id) gen_yml(device, gpu_id)
elif sys.argv[1] == "run": elif sys.argv[1] == "run":
mode = sys.argv[2] # http/ rpc mode = sys.argv[2] # http/ rpc
thread = int(sys.argv[3]) thread = int(sys.argv[3])
batch_size = int(sys.argv[4]) batch_size = int(sys.argv[4])
if mode == "http": if mode == "http":
...@@ -131,4 +151,3 @@ if __name__ == "__main__": ...@@ -131,4 +151,3 @@ if __name__ == "__main__":
filein = sys.argv[2] filein = sys.argv[2]
fileout = sys.argv[3] fileout = sys.argv[3]
parse_benchmark(filein, fileout) parse_benchmark(filein, fileout)
python/examples/pipeline/PaddleClas/ResNet50_vd_PACT/pipeline_rpc_client.py
浏览文件 @ 12ab1380
...@@ -11,10 +11,8 @@ ...@@ -11,10 +11,8 @@
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and # See the License for the specific language governing permissions and
# limitations under the License. # limitations under the License.
try:
from paddle_serving_server_gpu.pipeline import PipelineClient from paddle_serving_server.pipeline import PipelineClient
except ImportError:
from paddle_serving_server.pipeline import PipelineClient
import numpy as np import numpy as np
import requests import requests
import json import json
......
python/examples/pipeline/PaddleClas/ResNet50_vd_PACT/resnet50_web_service.py
浏览文件 @ 12ab1380
...@@ -13,10 +13,8 @@ ...@@ -13,10 +13,8 @@
# limitations under the License. # limitations under the License.
import sys import sys
from paddle_serving_app.reader import Sequential, URL2Image, Resize, CenterCrop, RGB2BGR, Transpose, Div, Normalize, Base64ToImage from paddle_serving_app.reader import Sequential, URL2Image, Resize, CenterCrop, RGB2BGR, Transpose, Div, Normalize, Base64ToImage
try:
from paddle_serving_server_gpu.web_service import WebService, Op from paddle_serving_server.web_service import WebService, Op
except ImportError:
from paddle_serving_server.web_service import WebService, Op
import logging import logging
import numpy as np import numpy as np
import base64, cv2 import base64, cv2
......
python/examples/pipeline/PaddleClas/ResNet_V2_50/README.md
浏览文件 @ 12ab1380
...@@ -4,17 +4,17 @@ This document will takes Imagenet service as an example to introduce how to use ...@@ -4,17 +4,17 @@ This document will takes Imagenet service as an example to introduce how to use
## Get model ## Get model
``` ```
python -m paddle_serving_app.package --get_model resnet_v2_50_imagenet python3 -m paddle_serving_app.package --get_model resnet_v2_50_imagenet
tar -xzvf resnet_v2_50_imagenet.tar.gz tar -xzvf resnet_v2_50_imagenet.tar.gz
``` ```
## Start server ## Start server
``` ```
python resnet50_web_service.py &>log.txt & python3 resnet50_web_service.py &>log.txt &
``` ```
## RPC test ## RPC test
``` ```
python pipeline_rpc_client.py python3 pipeline_rpc_client.py
``` ```
python/examples/pipeline/PaddleClas/ResNet_V2_50/README_CN.md
浏览文件 @ 12ab1380
...@@ -4,18 +4,17 @@ ...@@ -4,18 +4,17 @@
## 获取模型 ## 获取模型
``` ```
python -m paddle_serving_app.package --get_model resnet_v2_50_imagenet python3 -m paddle_serving_app.package --get_model resnet_v2_50_imagenet
tar -xzvf resnet_v2_50_imagenet.tar.gz tar -xzvf resnet_v2_50_imagenet.tar.gz
``` ```
## 启动服务 ## 启动服务
``` ```
python resnet50_web_service.py &>log.txt & python3 resnet50_web_service.py &>log.txt &
``` ```
## 测试 ## 测试
``` ```
python pipeline_rpc_client.py python3 pipeline_rpc_client.py
``` ```
python/examples/pipeline/PaddleClas/ResNet_V2_50/benchmark.py
浏览文件 @ 12ab1380
# Copyright (c) 2021 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import sys import sys
import os import os
import base64 import base64
...@@ -5,13 +19,13 @@ import yaml ...@@ -5,13 +19,13 @@ import yaml
import requests import requests
import time import time
import json import json
try:
from paddle_serving_server_gpu.pipeline import PipelineClient from paddle_serving_server.pipeline import PipelineClient
except ImportError:
from paddle_serving_server.pipeline import PipelineClient
import numpy as np import numpy as np
from paddle_serving_client.utils import MultiThreadRunner from paddle_serving_client.utils import MultiThreadRunner
from paddle_serving_client.utils import benchmark_args, show_latency from paddle_serving_client.utils import benchmark_args, show_latency
def parse_benchmark(filein, fileout): def parse_benchmark(filein, fileout):
with open(filein, "r") as fin: with open(filein, "r") as fin:
res = yaml.load(fin) res = yaml.load(fin)
...@@ -24,6 +38,7 @@ def parse_benchmark(filein, fileout): ...@@ -24,6 +38,7 @@ def parse_benchmark(filein, fileout):
with open(fileout, "w") as fout: with open(fileout, "w") as fout:
yaml.dump(res, fout, default_flow_style=False) yaml.dump(res, fout, default_flow_style=False)
def gen_yml(device, gpu_id): def gen_yml(device, gpu_id):
fin = open("config.yml", "r") fin = open("config.yml", "r")
config = yaml.load(fin) config = yaml.load(fin)
...@@ -34,15 +49,17 @@ def gen_yml(device, gpu_id): ...@@ -34,15 +49,17 @@ def gen_yml(device, gpu_id):
config["op"]["imagenet"]["local_service_conf"]["devices"] = gpu_id config["op"]["imagenet"]["local_service_conf"]["devices"] = gpu_id
else: else:
config["op"]["imagenet"]["local_service_conf"]["device_type"] = 0 config["op"]["imagenet"]["local_service_conf"]["device_type"] = 0
with open("config2.yml", "w") as fout: with open("config2.yml", "w") as fout:
yaml.dump(config, fout, default_flow_style=False) yaml.dump(config, fout, default_flow_style=False)
def cv2_to_base64(image): def cv2_to_base64(image):
return base64.b64encode(image).decode('utf8') return base64.b64encode(image).decode('utf8')
def run_http(idx, batch_size): def run_http(idx, batch_size):
print("start thread ({})".format(idx)) print("start thread ({})".format(idx))
url = "http://127.0.0.1:18000/imagenet/prediction" url = "http://127.0.0.1:18000/imagenet/prediction"
start = time.time() start = time.time()
with open(os.path.join(".", "daisy.jpg"), 'rb') as file: with open(os.path.join(".", "daisy.jpg"), 'rb') as file:
...@@ -68,6 +85,7 @@ def run_http(idx, batch_size): ...@@ -68,6 +85,7 @@ def run_http(idx, batch_size):
end = time.time() end = time.time()
return [[end - start], latency_list, [total_num]] return [[end - start], latency_list, [total_num]]
def multithread_http(thread, batch_size): def multithread_http(thread, batch_size):
multi_thread_runner = MultiThreadRunner() multi_thread_runner = MultiThreadRunner()
start = time.time() start = time.time()
...@@ -87,6 +105,7 @@ def multithread_http(thread, batch_size): ...@@ -87,6 +105,7 @@ def multithread_http(thread, batch_size):
total_cost)) total_cost))
show_latency(result[1]) show_latency(result[1])
def run_rpc(thread, batch_size): def run_rpc(thread, batch_size):
client = PipelineClient() client = PipelineClient()
client.connect(['127.0.0.1:18080']) client.connect(['127.0.0.1:18080'])
...@@ -107,11 +126,12 @@ def run_rpc(thread, batch_size): ...@@ -107,11 +126,12 @@ def run_rpc(thread, batch_size):
def multithread_rpc(thraed, batch_size): def multithread_rpc(thraed, batch_size):
multi_thread_runner = MultiThreadRunner() multi_thread_runner = MultiThreadRunner()
result = multi_thread_runner.run(run_rpc , thread, batch_size) result = multi_thread_runner.run(run_rpc, thread, batch_size)
if __name__ == "__main__": if __name__ == "__main__":
if sys.argv[1] == "yaml": if sys.argv[1] == "yaml":
mode = sys.argv[2] # brpc/ local predictor mode = sys.argv[2] # brpc/ local predictor
thread = int(sys.argv[3]) thread = int(sys.argv[3])
device = sys.argv[4] device = sys.argv[4]
if device == "gpu": if device == "gpu":
...@@ -120,7 +140,7 @@ if __name__ == "__main__": ...@@ -120,7 +140,7 @@ if __name__ == "__main__":
gpu_id = None gpu_id = None
gen_yml(device, gpu_id) gen_yml(device, gpu_id)
elif sys.argv[1] == "run": elif sys.argv[1] == "run":
mode = sys.argv[2] # http/ rpc mode = sys.argv[2] # http/ rpc
thread = int(sys.argv[3]) thread = int(sys.argv[3])
batch_size = int(sys.argv[4]) batch_size = int(sys.argv[4])
if mode == "http": if mode == "http":
...@@ -131,4 +151,3 @@ if __name__ == "__main__": ...@@ -131,4 +151,3 @@ if __name__ == "__main__":
filein = sys.argv[2] filein = sys.argv[2]
fileout = sys.argv[3] fileout = sys.argv[3]
parse_benchmark(filein, fileout) parse_benchmark(filein, fileout)
python/examples/pipeline/PaddleClas/ResNet_V2_50/pipeline_rpc_client.py
浏览文件 @ 12ab1380
...@@ -11,10 +11,8 @@ ...@@ -11,10 +11,8 @@
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and # See the License for the specific language governing permissions and
# limitations under the License. # limitations under the License.
try:
from paddle_serving_server_gpu.pipeline import PipelineClient from paddle_serving_server.pipeline import PipelineClient
except ImportError:
from paddle_serving_server.pipeline import PipelineClient
import numpy as np import numpy as np
import requests import requests
import json import json
......
python/examples/pipeline/PaddleClas/ShuffleNetV2_x1_0/README.md
浏览文件 @ 12ab1380
...@@ -10,10 +10,10 @@ sh get_model.sh ...@@ -10,10 +10,10 @@ sh get_model.sh
## Start server ## Start server
``` ```
python resnet50_web_service.py &>log.txt & python3 resnet50_web_service.py &>log.txt &
``` ```
## RPC test ## RPC test
``` ```
python pipeline_rpc_client.py python3 pipeline_rpc_client.py
``` ```
python/examples/pipeline/PaddleClas/ShuffleNetV2_x1_0/README_CN.md
浏览文件 @ 12ab1380
...@@ -10,11 +10,10 @@ sh get_model.sh ...@@ -10,11 +10,10 @@ sh get_model.sh
## 启动服务 ## 启动服务
``` ```
python resnet50_web_service.py &>log.txt & python3 resnet50_web_service.py &>log.txt &
``` ```
## 测试 ## 测试
``` ```
python pipeline_rpc_client.py python3 pipeline_rpc_client.py
``` ```
python/examples/pipeline/PaddleClas/ShuffleNetV2_x1_0/benchmark.py
浏览文件 @ 12ab1380
# Copyright (c) 2021 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import sys import sys
import os import os
import base64 import base64
...@@ -5,13 +19,13 @@ import yaml ...@@ -5,13 +19,13 @@ import yaml
import requests import requests
import time import time
import json import json
try:
from paddle_serving_server_gpu.pipeline import PipelineClient from paddle_serving_server.pipeline import PipelineClient
except ImportError:
from paddle_serving_server.pipeline import PipelineClient
import numpy as np import numpy as np
from paddle_serving_client.utils import MultiThreadRunner from paddle_serving_client.utils import MultiThreadRunner
from paddle_serving_client.utils import benchmark_args, show_latency from paddle_serving_client.utils import benchmark_args, show_latency
def parse_benchmark(filein, fileout): def parse_benchmark(filein, fileout):
with open(filein, "r") as fin: with open(filein, "r") as fin:
res = yaml.load(fin) res = yaml.load(fin)
...@@ -24,6 +38,7 @@ def parse_benchmark(filein, fileout): ...@@ -24,6 +38,7 @@ def parse_benchmark(filein, fileout):
with open(fileout, "w") as fout: with open(fileout, "w") as fout:
yaml.dump(res, fout, default_flow_style=False) yaml.dump(res, fout, default_flow_style=False)
def gen_yml(device, gpu_id): def gen_yml(device, gpu_id):
fin = open("config.yml", "r") fin = open("config.yml", "r")
config = yaml.load(fin) config = yaml.load(fin)
...@@ -34,15 +49,17 @@ def gen_yml(device, gpu_id): ...@@ -34,15 +49,17 @@ def gen_yml(device, gpu_id):
config["op"]["imagenet"]["local_service_conf"]["devices"] = gpu_id config["op"]["imagenet"]["local_service_conf"]["devices"] = gpu_id
else: else:
config["op"]["imagenet"]["local_service_conf"]["device_type"] = 0 config["op"]["imagenet"]["local_service_conf"]["device_type"] = 0
with open("config2.yml", "w") as fout: with open("config2.yml", "w") as fout:
yaml.dump(config, fout, default_flow_style=False) yaml.dump(config, fout, default_flow_style=False)
def cv2_to_base64(image): def cv2_to_base64(image):
return base64.b64encode(image).decode('utf8') return base64.b64encode(image).decode('utf8')
def run_http(idx, batch_size): def run_http(idx, batch_size):
print("start thread ({})".format(idx)) print("start thread ({})".format(idx))
url = "http://127.0.0.1:18080/imagenet/prediction" url = "http://127.0.0.1:18080/imagenet/prediction"
start = time.time() start = time.time()
with open(os.path.join(".", "daisy.jpg"), 'rb') as file: with open(os.path.join(".", "daisy.jpg"), 'rb') as file:
...@@ -68,6 +85,7 @@ def run_http(idx, batch_size): ...@@ -68,6 +85,7 @@ def run_http(idx, batch_size):
end = time.time() end = time.time()
return [[end - start], latency_list, [total_num]] return [[end - start], latency_list, [total_num]]
def multithread_http(thread, batch_size): def multithread_http(thread, batch_size):
multi_thread_runner = MultiThreadRunner() multi_thread_runner = MultiThreadRunner()
start = time.time() start = time.time()
...@@ -87,6 +105,7 @@ def multithread_http(thread, batch_size): ...@@ -87,6 +105,7 @@ def multithread_http(thread, batch_size):
total_cost)) total_cost))
show_latency(result[1]) show_latency(result[1])
def run_rpc(thread, batch_size): def run_rpc(thread, batch_size):
client = PipelineClient() client = PipelineClient()
client.connect(['127.0.0.1:18080']) client.connect(['127.0.0.1:18080'])
...@@ -107,11 +126,12 @@ def run_rpc(thread, batch_size): ...@@ -107,11 +126,12 @@ def run_rpc(thread, batch_size):
def multithread_rpc(thraed, batch_size): def multithread_rpc(thraed, batch_size):
multi_thread_runner = MultiThreadRunner() multi_thread_runner = MultiThreadRunner()
result = multi_thread_runner.run(run_rpc , thread, batch_size) result = multi_thread_runner.run(run_rpc, thread, batch_size)
if __name__ == "__main__": if __name__ == "__main__":
if sys.argv[1] == "yaml": if sys.argv[1] == "yaml":
mode = sys.argv[2] # brpc/ local predictor mode = sys.argv[2] # brpc/ local predictor
thread = int(sys.argv[3]) thread = int(sys.argv[3])
device = sys.argv[4] device = sys.argv[4]
if device == "gpu": if device == "gpu":
...@@ -120,7 +140,7 @@ if __name__ == "__main__": ...@@ -120,7 +140,7 @@ if __name__ == "__main__":
gpu_id = None gpu_id = None
gen_yml(device, gpu_id) gen_yml(device, gpu_id)
elif sys.argv[1] == "run": elif sys.argv[1] == "run":
mode = sys.argv[2] # http/ rpc mode = sys.argv[2] # http/ rpc
thread = int(sys.argv[3]) thread = int(sys.argv[3])
batch_size = int(sys.argv[4]) batch_size = int(sys.argv[4])
if mode == "http": if mode == "http":
...@@ -131,4 +151,3 @@ if __name__ == "__main__": ...@@ -131,4 +151,3 @@ if __name__ == "__main__":
filein = sys.argv[2] filein = sys.argv[2]
fileout = sys.argv[3] fileout = sys.argv[3]
parse_benchmark(filein, fileout) parse_benchmark(filein, fileout)
python/examples/pipeline/PaddleClas/ShuffleNetV2_x1_0/pipeline_rpc_client.py
浏览文件 @ 12ab1380
...@@ -11,10 +11,7 @@ ...@@ -11,10 +11,7 @@
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and # See the License for the specific language governing permissions and
# limitations under the License. # limitations under the License.
try: from paddle_serving_server.pipeline import PipelineClient
from paddle_serving_server_gpu.pipeline import PipelineClient
except ImportError:
from paddle_serving_server.pipeline import PipelineClient
import numpy as np import numpy as np
import requests import requests
import json import json
......
python/examples/pipeline/PaddleClas/ShuffleNetV2_x1_0/resnet50_web_service.py
浏览文件 @ 12ab1380
...@@ -13,10 +13,8 @@ ...@@ -13,10 +13,8 @@
# limitations under the License. # limitations under the License.
import sys import sys
from paddle_serving_app.reader import Sequential, URL2Image, Resize, CenterCrop, RGB2BGR, Transpose, Div, Normalize, Base64ToImage from paddle_serving_app.reader import Sequential, URL2Image, Resize, CenterCrop, RGB2BGR, Transpose, Div, Normalize, Base64ToImage
try:
from paddle_serving_server_gpu.web_service import WebService, Op from paddle_serving_server.web_service import WebService, Op
except ImportError:
from paddle_serving_server.web_service import WebService, Op
import logging import logging
import numpy as np import numpy as np
import base64, cv2 import base64, cv2
......
python/examples/pipeline/PaddleDetection/faster_rcnn/README.md
浏览文件 @ 12ab1380
...@@ -8,11 +8,11 @@ wget --no-check-certificate https://paddle-serving.bj.bcebos.com/pddet_demo/2.0/ ...@@ -8,11 +8,11 @@ wget --no-check-certificate https://paddle-serving.bj.bcebos.com/pddet_demo/2.0/
### Start the service ### Start the service
``` ```
tar xf faster_rcnn_r50_fpn_1x_coco.tar tar xf faster_rcnn_r50_fpn_1x_coco.tar
python web_service.py python3 web_service.py
``` ```
### Perform prediction ### Perform prediction
``` ```
python pipeline_http_client.py python3 pipeline_http_client.py
``` ```
python/examples/senta/README.md
浏览文件 @ 12ab1380
此差异已折叠。
python/examples/senta/README_CN.md
浏览文件 @ 12ab1380
此差异已折叠。
python/examples/util/README.md
浏览文件 @ 12ab1380
此差异已折叠。
python/examples/util/README_CN.md
浏览文件 @ 12ab1380
此差异已折叠。
python/examples/xpu/vgg19/README.md
浏览文件 @ 12ab1380
此差异已折叠。
python/examples/yolov4/README.md
浏览文件 @ 12ab1380
此差异已折叠。
python/examples/yolov4/README_CN.md
浏览文件 @ 12ab1380
此差异已折叠。
Markdown is supported
0% 或 .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑！
想要评论请 注册 或