diff --git a/README.md b/README.md index 08e8d85a7baf1896c70ebc1999f900bd6747a895..1818ddd61cc5423c4a590815930d007303f18e81 100644 --- a/README.md +++ b/README.md @@ -151,7 +151,6 @@ Here, `client.predict` function has two arguments. `feed` is a `python dict` wit - **Distributed Key-Value indexing** supported which is especially useful for large scale sparse features as model inputs. - **Highly concurrent and efficient communication** between clients and servers supported. - **Multiple programming languages** supported on client side, such as Golang, C++ and python. -- **Extensible framework design** which can support model serving beyond Paddle.

Document

diff --git a/README_CN.md b/README_CN.md index 8b091bdc9906007a4683b50184d08cd960483730..29cf095248f4c125b3dba7146e67efe8b7abae6c 100644 --- a/README_CN.md +++ b/README_CN.md @@ -156,7 +156,6 @@ print(fetch_map) - 支持 **分布式键值对索引** 助力于大规模稀疏特征作为模型输入. - 支持客户端和服务端之间 **高并发和高效通信**. - 支持 **多种编程语言** 开发客户端,例如Golang,C++和Python. -- **可伸缩框架设计** 可支持不限于Paddle的模型服务.

文档

diff --git a/doc/COMPILE.md b/doc/COMPILE.md index 3649b0a9355e25dad2544b7898f639fc44b9a0b0..f4a6639bdb38fac97662084f7d927d24b6179717 100644 --- a/doc/COMPILE.md +++ b/doc/COMPILE.md @@ -20,7 +20,7 @@ This document will take Python2 as an example to show how to compile Paddle Serv - Set `DPYTHON_INCLUDE_DIR` to `$PYTHONROOT/include/python3.6m/` - Set `DPYTHON_LIBRARIES` to `$PYTHONROOT/lib64/libpython3.6.so` -- Set `DPYTHON_EXECUTABLE` to `$PYTHONROOT/bin/python3` +- Set `DPYTHON_EXECUTABLE` to `$PYTHONROOT/bin/python3.6` ## Get Code diff --git a/doc/COMPILE_CN.md b/doc/COMPILE_CN.md index 2b8a42d10368d924e04a67aa9720c14c3b4d95dd..d8fd277131d7d169c1a47689e15556e5d10a0fdb 100644 --- a/doc/COMPILE_CN.md +++ b/doc/COMPILE_CN.md @@ -20,7 +20,7 @@ - 将`DPYTHON_INCLUDE_DIR`设置为`$PYTHONROOT/include/python3.6m/` - 将`DPYTHON_LIBRARIES`设置为`$PYTHONROOT/lib64/libpython3.6.so` -- 将`DPYTHON_EXECUTABLE`设置为`$PYTHONROOT/bin/python3` +- 将`DPYTHON_EXECUTABLE`设置为`$PYTHONROOT/bin/python3.6` ## 获取代码 diff --git a/doc/UWSGI_DEPLOY.md b/doc/UWSGI_DEPLOY.md index cb3fb506bf6fd4461240ebe43234fa3bed3d4784..92b69fc1f3da6c791c1009d41bbb3a3ec6f30594 100644 --- a/doc/UWSGI_DEPLOY.md +++ b/doc/UWSGI_DEPLOY.md @@ -29,7 +29,7 @@ from paddle_serving_server.web_service import WebService uci_service = WebService(name = "uci") uci_service.load_model_config("./uci_housing_model") uci_service.prepare_server(workdir="./workdir", port=int(9500), device="cpu") -uci_service.run_server() +uci_service.run_rpc_service() #Get flask application app_instance = uci_service.get_app_instance() ``` diff --git a/doc/UWSGI_DEPLOY_CN.md b/doc/UWSGI_DEPLOY_CN.md index 5bb87e26bbae729f8c21b4681413a4c9f5c4e7c8..966155162f5ff90e88f9b743a3047b5d86440a46 100644 --- a/doc/UWSGI_DEPLOY_CN.md +++ b/doc/UWSGI_DEPLOY_CN.md @@ -29,7 +29,7 @@ from paddle_serving_server.web_service import WebService uci_service = WebService(name = "uci") uci_service.load_model_config("./uci_housing_model") uci_service.prepare_server(workdir="./workdir", port=int(9500), device="cpu") -uci_service.run_server() +uci_service.run_rpc_service() #获取flask服务 app_instance = uci_service.get_app_instance() ``` diff --git a/python/examples/senta/README.md b/python/examples/senta/README.md index 88aac352110850a71ae0f9a28c1a98293f8e0ab9..9aeb6d1191719e067e2cb99d408a6d091c25ede3 100644 --- a/python/examples/senta/README.md +++ b/python/examples/senta/README.md @@ -1,21 +1,20 @@ -# Chinese sentence sentiment classification +# Chinese Sentence Sentiment Classification ([简体中文](./README_CN.md)|English) -## Get model files and sample data -``` -sh get_data.sh -``` -## Install preprocess module +## Get Model ``` -pip install paddle_serving_app +python -m paddle_serving_app.package --get_model senta_bilstm +python -m paddle_serving_app.package --get_model lac ``` -## Start http service +## Start HTTP Service ``` -python senta_web_service.py senta_bilstm_model/ workdir 9292 +python -m paddle_serving_server.serve --model lac_model --port 9300 +python senta_web_service.py ``` -In the Chinese sentiment classification task, the Chinese word segmentation needs to be done through [LAC task] (../lac). Set model path by ```lac_model_path``` and dictionary path by ```lac_dict_path```. -In this demo, the LAC task is placed in the preprocessing part of the HTTP prediction service of the sentiment classification task. The LAC prediction service is deployed on the CPU, and the sentiment classification task is deployed on the GPU, which can be changed according to the actual situation. +In the Chinese sentiment classification task, the Chinese word segmentation needs to be done through [LAC task] (../lac). +In this demo, the LAC task is placed in the preprocessing part of the HTTP prediction service of the sentiment classification task. + ## Client prediction ``` curl -H "Content-Type:application/json" -X POST -d '{"feed":[{"words": "天气不错"}], "fetch":["class_probs"]}' http://127.0.0.1:9292/senta/prediction diff --git a/python/examples/senta/README_CN.md b/python/examples/senta/README_CN.md index f5011334db768c5f0869c296769ead7cb38613d8..f958af221d843748836bea325f87ba603411d39c 100644 --- a/python/examples/senta/README_CN.md +++ b/python/examples/senta/README_CN.md @@ -1,20 +1,19 @@ # 中文语句情感分类 (简体中文|[English](./README.md)) -## 获取模型文件和样例数据 -``` -sh get_data.sh -``` -## 安装数据预处理模块 + +## 获取模型文件 ``` -pip install paddle_serving_app +python -m paddle_serving_app.package --get_model senta_bilstm +python -m paddle_serving_app.package --get_model lac ``` ## 启动HTTP服务 ``` -python senta_web_service.py senta_bilstm_model/ workdir 9292 +python -m paddle_serving_server.serve --model lac_model --port 9300 +python senta_web_service.py ``` -中文情感分类任务中需要先通过[LAC任务](../lac)进行中文分词,在脚本中通过```lac_model_path```参数配置LAC任务的模型文件路径,```lac_dict_path```参数配置LAC任务词典路径。 -示例中将LAC任务放在情感分类任务的HTTP预测服务的预处理部分,LAC预测服务部署在CPU上,情感分类任务部署在GPU上,可以根据实际情况进行更改。 +中文情感分类任务中需要先通过[LAC任务](../lac)进行中文分词。 +示例中将LAC任务放在情感分类任务的HTTP预测服务的预处理部分。 ## 客户端预测 ``` diff --git a/python/examples/senta/senta_web_service.py b/python/examples/senta/senta_web_service.py index 0621ece74173596a1820f1b09258ecf5bb727f29..25c880ef8877aed0f3f9d394d1780855130f365b 100644 --- a/python/examples/senta/senta_web_service.py +++ b/python/examples/senta/senta_web_service.py @@ -1,3 +1,4 @@ +#encoding=utf-8 # Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,56 +13,28 @@ # See the License for the specific language governing permissions and # limitations under the License. -from paddle_serving_server_gpu.web_service import WebService +from paddle_serving_server.web_service import WebService from paddle_serving_client import Client from paddle_serving_app.reader import LACReader, SentaReader import os import sys -from multiprocessing import Process +#senta_web_service.py +from paddle_serving_server.web_service import WebService +from paddle_serving_client import Client +from paddle_serving_app.reader import LACReader, SentaReader -class SentaService(WebService): - def set_config( - self, - lac_model_path, - lac_dict_path, - senta_dict_path, ): - self.lac_model_path = lac_model_path - self.lac_client_config_path = lac_model_path + "/serving_server_conf.prototxt" - self.lac_dict_path = lac_dict_path - self.senta_dict_path = senta_dict_path - - def start_lac_service(self): - if not os.path.exists('./lac_serving'): - os.mkdir("./lac_serving") - os.chdir('./lac_serving') - self.lac_port = self.port + 100 - r = os.popen( - "python -m paddle_serving_server.serve --model {} --port {} &". - format("../" + self.lac_model_path, self.lac_port)) - os.chdir('..') - - def init_lac_service(self): - ps = Process(target=self.start_lac_service()) - ps.start() - self.init_lac_client() - - def lac_predict(self, feed_data): - lac_result = self.lac_client.predict( - feed={"words": feed_data}, fetch=["crf_decode"]) - return lac_result - - def init_lac_client(self): - self.lac_client = Client() - self.lac_client.load_client_config(self.lac_client_config_path) - self.lac_client.connect(["127.0.0.1:{}".format(self.lac_port)]) - def init_lac_reader(self): +class SentaService(WebService): + #初始化lac模型预测服务 + def init_lac_client(self, lac_port, lac_client_config): self.lac_reader = LACReader() - - def init_senta_reader(self): self.senta_reader = SentaReader() + self.lac_client = Client() + self.lac_client.load_client_config(lac_client_config) + self.lac_client.connect(["127.0.0.1:{}".format(lac_port)]) + #定义senta模型预测服务的预处理,调用顺序:lac reader->lac模型预测->预测结果后处理->senta reader def preprocess(self, feed=[], fetch=[]): feed_data = [{ "words": self.lac_reader.process(x["words"]) @@ -80,15 +53,9 @@ class SentaService(WebService): senta_service = SentaService(name="senta") -senta_service.set_config( - lac_model_path="./lac_model", - lac_dict_path="./lac_dict", - senta_dict_path="./vocab.txt") -senta_service.load_model_config(sys.argv[1]) -senta_service.prepare_server( - workdir=sys.argv[2], port=int(sys.argv[3]), device="cpu") -senta_service.init_lac_reader() -senta_service.init_senta_reader() -senta_service.init_lac_service() +senta_service.load_model_config("senta_bilstm_model") +senta_service.prepare_server(workdir="workdir") +senta_service.init_lac_client( + lac_port=9300, lac_client_config="lac_model/serving_server_conf.prototxt") senta_service.run_rpc_service() senta_service.run_web_service() diff --git a/python/paddle_serving_app/README.md b/python/paddle_serving_app/README.md index 07fff931e250bad59ef2bedfe1e054f4682f6c9f..6757407939c150ca14a22427a488f41a24feb7ac 100644 --- a/python/paddle_serving_app/README.md +++ b/python/paddle_serving_app/README.md @@ -21,15 +21,15 @@ python -m paddle_serving_app.package --list_model python -m paddle_serving_app.package --get_model senta_bilstm ``` -11 pre-trained models are built into paddle_serving_app, covering 6 kinds of prediction tasks. +10 pre-trained models are built into paddle_serving_app, covering 6 kinds of prediction tasks. The model files can be directly used for deployment, and the `--tutorial` argument can be added to obtain the deployment method. | Prediction task | Model name | | ------------ | ------------------------------------------------ | | SentimentAnalysis | 'senta_bilstm', 'senta_bow', 'senta_cnn' | -| SemanticRepresentation | 'ernie_base' | +| SemanticRepresentation | 'ernie' | | ChineseWordSegmentation | 'lac' | -| ObjectDetection | 'faster_rcnn', 'yolov3' | +| ObjectDetection | 'faster_rcnn' | | ImageSegmentation | 'unet', 'deeplabv3' | | ImageClassification | 'resnet_v2_50_imagenet', 'mobilenet_v2_imagenet' | @@ -76,7 +76,7 @@ Preprocessing for Chinese word segmentation task. [example](../examples/senta/senta_web_service.py) -- The image preprocessing method is more flexible than the above method, and can be combined by the following multiple classes,[example](../examples/imagenet/image_rpc_client.py) +- The image preprocessing method is more flexible than the above method, and can be combined by the following multiple classes,[example](../examples/imagenet/resnet50_rpc_client.py) - class Sequentia diff --git a/python/paddle_serving_app/README_CN.md b/python/paddle_serving_app/README_CN.md index f6fda8beaf75264d8ae5d2cbb939fdf226c342ab..d29c3fd9fff3ba2ab34ec67b6fd15ad10e3cfd07 100644 --- a/python/paddle_serving_app/README_CN.md +++ b/python/paddle_serving_app/README_CN.md @@ -20,14 +20,14 @@ python -m paddle_serving_app.package --list_model python -m paddle_serving_app.package --get_model senta_bilstm ``` -paddle_serving_app中内置了11中预训练模型,涵盖了6种预测任务。获取到的模型文件可以直接用于部署,添加`--tutorial`参数可以获取对应的部署方式。 +paddle_serving_app中内置了10种预训练模型,涵盖了6种预测任务。获取到的模型文件可以直接用于部署,添加`--tutorial`参数可以获取对应的部署方式。 | 预测服务类型 | 模型名称 | | ------------ | ------------------------------------------------ | | 中文情感分析 | 'senta_bilstm', 'senta_bow', 'senta_cnn' | -| 语义理解 | 'ernie_base' | +| 语义理解 | 'ernie' | | 中文分词 | 'lac' | -| 图像检测 | 'faster_rcnn', 'yolov3' | +| 图像检测 | 'faster_rcnn' | | 图像分割 | 'unet', 'deeplabv3' | | 图像分类 | 'resnet_v2_50_imagenet', 'mobilenet_v2_imagenet' | @@ -71,7 +71,7 @@ paddle_serving_app针对CV和NLP领域的模型任务,提供了多种常见的 [参考示例](../examples/senta/senta_web_service.py) -- 图像的预处理方法相比于上述的方法更加灵活多变,可以通过以下的多个类进行组合,[参考示例](../examples/imagenet/image_rpc_client.py) +- 图像的预处理方法相比于上述的方法更加灵活多变,可以通过以下的多个类进行组合,[参考示例](../examples/imagenet/resnet50_rpc_client.py) - class Sequentia diff --git a/python/paddle_serving_app/models/model_list.py b/python/paddle_serving_app/models/model_list.py index fa04835219b49d08497acc8ed2f29899cf0007cc..cf0ca3bf5765d65065e541462eb919ccc5c4b978 100644 --- a/python/paddle_serving_app/models/model_list.py +++ b/python/paddle_serving_app/models/model_list.py @@ -22,9 +22,9 @@ class ServingModels(object): self.model_dict = OrderedDict() self.model_dict[ "SentimentAnalysis"] = ["senta_bilstm", "senta_bow", "senta_cnn"] - self.model_dict["SemanticRepresentation"] = ["ernie_base"] + self.model_dict["SemanticRepresentation"] = ["ernie"] self.model_dict["ChineseWordSegmentation"] = ["lac"] - self.model_dict["ObjectDetection"] = ["faster_rcnn", "yolov3"] + self.model_dict["ObjectDetection"] = ["faster_rcnn"] self.model_dict["ImageSegmentation"] = [ "unet", "deeplabv3", "deeplabv3+cityscapes" ] diff --git a/python/paddle_serving_client/__init__.py b/python/paddle_serving_client/__init__.py index f2922f577b21d8acc3f8ec629f2473b5339ee725..f201eefc449b3aea11db6ae209d79fb6acb05173 100644 --- a/python/paddle_serving_client/__init__.py +++ b/python/paddle_serving_client/__init__.py @@ -189,7 +189,7 @@ class Client(object): # create predictor here if endpoints is None: if self.predictor_sdk_ is None: - raise SystemExit( + raise ValueError( "You must set the endpoints parameter or use add_variant function to create a variant." ) else: @@ -215,7 +215,7 @@ class Client(object): return if isinstance(feed[key], list) and len(feed[key]) != self.feed_tensor_len[key]: - raise SystemExit("The shape of feed tensor {} not match.".format( + raise ValueError("The shape of feed tensor {} not match.".format( key)) if type(feed[key]).__module__ == np.__name__ and np.size(feed[ key]) != self.feed_tensor_len[key]: @@ -316,7 +316,7 @@ class Client(object): int_feed_names, int_shape, fetch_names, result_batch_handle, self.pid) else: - raise SystemExit( + raise ValueError( "Please make sure the inputs are all in list type or all in numpy.array type" ) diff --git a/python/paddle_serving_server_gpu/web_service.py b/python/paddle_serving_server_gpu/web_service.py index 2328453268f6cefa9c5bddb818677cc3962ea7ea..76721de8a005dfb23fbe2427671446889aa72af1 100644 --- a/python/paddle_serving_server_gpu/web_service.py +++ b/python/paddle_serving_server_gpu/web_service.py @@ -131,7 +131,7 @@ class WebService(object): for key in fetch_map: fetch_map[key] = fetch_map[key].tolist() result = self.postprocess( - feed=feed, fetch=fetch, fetch_map=fetch_map) + feed=request.json["feed"], fetch=fetch, fetch_map=fetch_map) result = {"result": result} except ValueError: result = {"result": "Request Value Error"}