diff --git a/README.md b/README.md index 4e64ebeb80b838241ab2d038532c164a0048b828..08e8d85a7baf1896c70ebc1999f900bd6747a895 100644 --- a/README.md +++ b/README.md @@ -26,14 +26,6 @@ We consider deploying deep learning inference service online to be a user-facing

-

Some Key Features

- -- Integrate with Paddle training pipeline seamlessly, most paddle models can be deployed **with one line command**. -- **Industrial serving features** supported, such as models management, online loading, online A/B testing etc. -- **Distributed Key-Value indexing** supported which is especially useful for large scale sparse features as model inputs. -- **Highly concurrent and efficient communication** between clients and servers supported. -- **Multiple programming languages** supported on client side, such as Golang, C++ and python. -- **Extensible framework design** which can support model serving beyond Paddle.

Installation

@@ -63,8 +55,40 @@ If you need install modules compiled with develop branch, please download packag Client package support Centos 7 and Ubuntu 18, or you can use HTTP service without install client. + +

Pre-built services with Paddle Serving

+ +

Chinese Word Segmentation

+ +``` shell +> python -m paddle_serving_app.package -get_model lac +> tar -xzf lac.tar.gz +> python lac_web_service.py 9292 & +> curl -H "Content-Type:application/json" -X POST -d '{"feed":[{"words": "我爱北京天安门"}], "fetch":["word_seg"]}' http://127.0.0.1:9393/lac/prediction +{"result":[{"word_seg":"我|爱|北京|天安门"}]} +``` + +

Image Classification

+ +

+
+ +
+

+ +``` shell +> python -m paddle_serving_app.package -get_model resnet_v2_50_imagenet +> tar -xzf resnet_v2_50_imagenet.tar.gz +> python resnet50_imagenet_classify.py resnet50_serving_model & +> curl -H "Content-Type:application/json" -X POST -d '{"feed":[{"image": "https://paddle-serving.bj.bcebos.com/imagenet-example/daisy.jpg"}], "fetch": ["score"]}' http://127.0.0.1:9292/image/prediction +{"result":{"label":["daisy"],"prob":[0.9341403245925903]}} +``` + +

Quick Start Example

+This quick start example is only for users who already have a model to deploy and we prepare a ready-to-deploy model here. If you want to know how to use paddle serving from offline training to online serving, please reference to [Train_To_Service](https://github.com/PaddlePaddle/Serving/blob/develop/doc/TRAIN_TO_SERVICE.md) + ### Boston House Price Prediction model ``` shell wget --no-check-certificate https://paddle-serving.bj.bcebos.com/uci_housing.tar.gz @@ -120,138 +144,14 @@ print(fetch_map) ``` Here, `client.predict` function has two arguments. `feed` is a `python dict` with model input variable alias name and values. `fetch` assigns the prediction variables to be returned from servers. In the example, the name of `"x"` and `"price"` are assigned when the servable model is saved during training. -

Pre-built services with Paddle Serving

- -

Chinese Word Segmentation

- -- **Description**: -``` shell -Chinese word segmentation HTTP service that can be deployed with one line command. -``` - -- **Download Servable Package**: -``` shell -wget --no-check-certificate https://paddle-serving.bj.bcebos.com/lac/lac_model_jieba_web.tar.gz -``` -- **Host web service**: -``` shell -tar -xzf lac_model_jieba_web.tar.gz -python lac_web_service.py jieba_server_model/ lac_workdir 9292 -``` -- **Request sample**: -``` shell -curl -H "Content-Type:application/json" -X POST -d '{"feed":[{"words": "我爱北京天安门"}], "fetch":["word_seg"]}' http://127.0.0.1:9292/lac/prediction -``` -- **Request result**: -``` shell -{"word_seg":"我|爱|北京|天安门"} -``` - -

Image Classification

- -- **Description**: -``` shell -Image classification trained with Imagenet dataset. A label and corresponding probability will be returned. -Note: This demo needs paddle-serving-server-gpu. -``` - -- **Download Servable Package**: -``` shell -wget --no-check-certificate https://paddle-serving.bj.bcebos.com/imagenet-example/imagenet_demo.tar.gz -``` -- **Host web service**: -``` shell -tar -xzf imagenet_demo.tar.gz -python image_classification_service_demo.py resnet50_serving_model -``` -- **Request sample**: - -

-
- -
-

- -``` shell -curl -H "Content-Type:application/json" -X POST -d '{"feed":[{"url": "https://paddle-serving.bj.bcebos.com/imagenet-example/daisy.jpg"}], "fetch": ["score"]}' http://127.0.0.1:9292/image/prediction -``` -- **Request result**: -``` shell -{"label":"daisy","prob":0.9341403245925903} -``` - -

More Demos

- -| Key | Value | -| :----------------- | :----------------------------------------------------------- | -| Model Name | Bert-Base-Baike | -| URL | [https://paddle-serving.bj.bcebos.com/bert_example/bert_seq128.tar.gz](https://paddle-serving.bj.bcebos.com/bert_example%2Fbert_seq128.tar.gz) | -| Client/Server Code | https://github.com/PaddlePaddle/Serving/tree/develop/python/examples/bert | -| Description | Get semantic representation from a Chinese Sentence | - - - -| Key | Value | -| :----------------- | :----------------------------------------------------------- | -| Model Name | Resnet50-Imagenet | -| URL | [https://paddle-serving.bj.bcebos.com/imagenet-example/ResNet50_vd.tar.gz](https://paddle-serving.bj.bcebos.com/imagenet-example%2FResNet50_vd.tar.gz) | -| Client/Server Code | https://github.com/PaddlePaddle/Serving/tree/develop/python/examples/imagenet | -| Description | Get image semantic representation from an image | - - - -| Key | Value | -| :----------------- | :----------------------------------------------------------- | -| Model Name | Resnet101-Imagenet | -| URL | https://paddle-serving.bj.bcebos.com/imagenet-example/ResNet101_vd.tar.gz | -| Client/Server Code | https://github.com/PaddlePaddle/Serving/tree/develop/python/examples/imagenet | -| Description | Get image semantic representation from an image | - - - -| Key | Value | -| :----------------- | :----------------------------------------------------------- | -| Model Name | CNN-IMDB | -| URL | https://paddle-serving.bj.bcebos.com/imdb-demo/imdb_model.tar.gz | -| Client/Server Code | https://github.com/PaddlePaddle/Serving/tree/develop/python/examples/imdb | -| Description | Get category probability from an English Sentence | - - - -| Key | Value | -| :----------------- | :----------------------------------------------------------- | -| Model Name | LSTM-IMDB | -| URL | https://paddle-serving.bj.bcebos.com/imdb-demo/imdb_model.tar.gz | -| Client/Server Code | https://github.com/PaddlePaddle/Serving/tree/develop/python/examples/imdb | -| Description | Get category probability from an English Sentence | - - - -| Key | Value | -| :----------------- | :----------------------------------------------------------- | -| Model Name | BOW-IMDB | -| URL | https://paddle-serving.bj.bcebos.com/imdb-demo/imdb_model.tar.gz | -| Client/Server Code | https://github.com/PaddlePaddle/Serving/tree/develop/python/examples/imdb | -| Description | Get category probability from an English Sentence | - - - -| Key | Value | -| :----------------- | :----------------------------------------------------------- | -| Model Name | Jieba-LAC | -| URL | https://paddle-serving.bj.bcebos.com/lac/lac_model.tar.gz | -| Client/Server Code | https://github.com/PaddlePaddle/Serving/tree/develop/python/examples/lac | -| Description | Get word segmentation from a Chinese Sentence | - - - -| Key | Value | -| :----------------- | :----------------------------------------------------------- | -| Model Name | DNN-CTR | -| URL | https://paddle-serving.bj.bcebos.com/criteo_ctr_example/criteo_ctr_demo_model.tar.gz | -| Client/Server Code | https://github.com/PaddlePaddle/Serving/tree/develop/python/examples/criteo_ctr | -| Description | Get click probability from a feature vector of item | +

Some Key Features of Paddle Serving

+- Integrate with Paddle training pipeline seamlessly, most paddle models can be deployed **with one line command**. +- **Industrial serving features** supported, such as models management, online loading, online A/B testing etc. +- **Distributed Key-Value indexing** supported which is especially useful for large scale sparse features as model inputs. +- **Highly concurrent and efficient communication** between clients and servers supported. +- **Multiple programming languages** supported on client side, such as Golang, C++ and python. +- **Extensible framework design** which can support model serving beyond Paddle.

Document

diff --git a/README_CN.md b/README_CN.md index 298b8156e34d6c8834babcb68ae8f7a466b156af..8b091bdc9906007a4683b50184d08cd960483730 100644 --- a/README_CN.md +++ b/README_CN.md @@ -27,14 +27,7 @@ Paddle Serving 旨在帮助深度学习开发者轻易部署在线预测服务

-

核心功能

-- 与Paddle训练紧密连接，绝大部分Paddle模型可以 **一键部署**. -- 支持 **工业级的服务能力** 例如模型管理，在线加载，在线A/B测试等. -- 支持 **分布式键值对索引** 助力于大规模稀疏特征作为模型输入. -- 支持客户端和服务端之间 **高并发和高效通信**. -- 支持 **多种编程语言** 开发客户端，例如Golang，C++和Python. -- **可伸缩框架设计** 可支持不限于Paddle的模型服务.

安装

@@ -64,7 +57,38 @@ pip install paddle-serving-server-gpu # GPU 客户端安装包支持Centos 7和Ubuntu 18，或者您可以使用HTTP服务，这种情况下不需要安装客户端。 -

快速启动示例

+

Paddle Serving预装的服务

+ +

中文分词

+ +``` shell +> python -m paddle_serving_app.package -get_model lac +> tar -xzf lac.tar.gz +> python lac_web_service.py 9292 & +> curl -H "Content-Type:application/json" -X POST -d '{"feed":[{"words": "我爱北京天安门"}], "fetch":["word_seg"]}' http://127.0.0.1:9393/lac/prediction +{"result":[{"word_seg":"我|爱|北京|天安门"}]} +``` + +

图像分类

+ +

+
+ +
+

+ +``` shell +> python -m paddle_serving_app.package -get_model resnet_v2_50_imagenet +> tar -xzf resnet_v2_50_imagenet.tar.gz +> python resnet50_imagenet_classify.py resnet50_serving_model & +> curl -H "Content-Type:application/json" -X POST -d '{"feed":[{"image": "https://paddle-serving.bj.bcebos.com/imagenet-example/daisy.jpg"}], "fetch": ["score"]}' http://127.0.0.1:9292/image/prediction +{"result":{"label":["daisy"],"prob":[0.9341403245925903]}} +``` + + +

快速开始示例

+ +这个快速开始示例主要是为了给那些已经有一个要部署的模型的用户准备的，而且我们也提供了一个可以用来部署的模型。如果您想知道如何从离线训练到在线服务走完全流程，请参考[从训练到部署](https://github.com/PaddlePaddle/Serving/blob/develop/doc/TRAIN_TO_SERVICE_CN.md)

波士顿房价预测

@@ -125,139 +149,14 @@ print(fetch_map) ``` 在这里，`client.predict`函数具有两个参数。 `feed`是带有模型输入变量别名和值的`python dict`。 `fetch`被要从服务器返回的预测变量赋值。 在该示例中，在训练过程中保存可服务模型时，被赋值的tensor名为`"x"`和`"price"`。 -

Paddle Serving预装的服务

- -

中文分词模型

- -- **介绍**: -``` shell -本示例为中文分词HTTP服务一键部署 -``` - -- **下载服务包**: -``` shell -wget --no-check-certificate https://paddle-serving.bj.bcebos.com/lac/lac_model_jieba_web.tar.gz -``` -- **启动web服务**: -``` shell -tar -xzf lac_model_jieba_web.tar.gz -python lac_web_service.py jieba_server_model/ lac_workdir 9292 -``` -- **客户端请求示例**: -``` shell -curl -H "Content-Type:application/json" -X POST -d '{"feed":[{"words": "我爱北京天安门"}], "fetch":["word_seg"]}' http://127.0.0.1:9292/lac/prediction -``` -- **返回结果示例**: -``` shell -{"word_seg":"我|爱|北京|天安门"} -``` - -

图像分类模型

- -- **介绍**: -``` shell -图像分类模型由Imagenet数据集训练而成，该服务会返回一个标签及其概率 -注意：本示例需要安装paddle-serving-server-gpu -``` - -- **下载服务包**: -``` shell -wget --no-check-certificate https://paddle-serving.bj.bcebos.com/imagenet-example/imagenet_demo.tar.gz -``` -- **启动web服务**: -``` shell -tar -xzf imagenet_demo.tar.gz -python image_classification_service_demo.py resnet50_serving_model -``` -- **客户端请求示例**: - -

-
- -
-

- -``` shell -curl -H "Content-Type:application/json" -X POST -d '{"feed":[{"url": "https://paddle-serving.bj.bcebos.com/imagenet-example/daisy.jpg"}], "fetch": ["score"]}' http://127.0.0.1:9292/image/prediction -``` -- **返回结果示例**: -``` shell -{"label":"daisy","prob":0.9341403245925903} -``` - -

更多示例

- -| Key | Value | -| :----------------- | :----------------------------------------------------------- | -| 模型名 | Bert-Base-Baike | -| 下载链接 | [https://paddle-serving.bj.bcebos.com/bert_example/bert_seq128.tar.gz](https://paddle-serving.bj.bcebos.com/bert_example%2Fbert_seq128.tar.gz) | -| 客户端/服务端代码 | https://github.com/PaddlePaddle/Serving/tree/develop/python/examples/bert | -| 介绍 | 获得一个中文语句的语义表示 | - - - -| Key | Value | -| :----------------- | :----------------------------------------------------------- | -| 模型名 | Resnet50-Imagenet | -| 下载链接 | [https://paddle-serving.bj.bcebos.com/imagenet-example/ResNet50_vd.tar.gz](https://paddle-serving.bj.bcebos.com/imagenet-example%2FResNet50_vd.tar.gz) | -| 客户端/服务端代码 | https://github.com/PaddlePaddle/Serving/tree/develop/python/examples/imagenet | -| 介绍 | 获得一张图片的图像语义表示 | - - - -| Key | Value | -| :----------------- | :----------------------------------------------------------- | -| 模型名 | Resnet101-Imagenet | -| 下载链接 | https://paddle-serving.bj.bcebos.com/imagenet-example/ResNet101_vd.tar.gz | -| 客户端/服务端代码 | https://github.com/PaddlePaddle/Serving/tree/develop/python/examples/imagenet | -| 介绍 | 获得一张图片的图像语义表示 | - - - -| Key | Value | -| :----------------- | :----------------------------------------------------------- | -| 模型名 | CNN-IMDB | -| 下载链接 | https://paddle-serving.bj.bcebos.com/imdb-demo/imdb_model.tar.gz | -| 客户端/服务端代码 | https://github.com/PaddlePaddle/Serving/tree/develop/python/examples/imdb | -| 介绍 | 从一个中文语句获得类别及其概率 | - - - -| Key | Value | -| :----------------- | :----------------------------------------------------------- | -| 模型名 | LSTM-IMDB | -| 下载链接 | https://paddle-serving.bj.bcebos.com/imdb-demo/imdb_model.tar.gz | -| 客户端/服务端代码 | https://github.com/PaddlePaddle/Serving/tree/develop/python/examples/imdb | -| 介绍 | 从一个英文语句获得类别及其概率 | - - - -| Key | Value | -| :----------------- | :----------------------------------------------------------- | -| 模型名 | BOW-IMDB | -| 下载链接 | https://paddle-serving.bj.bcebos.com/imdb-demo/imdb_model.tar.gz | -| 客户端/服务端代码 | https://github.com/PaddlePaddle/Serving/tree/develop/python/examples/imdb | -| 介绍 | 从一个英文语句获得类别及其概率 | - - - -| Key | Value | -| :----------------- | :----------------------------------------------------------- | -| 模型名 | Jieba-LAC | -| 下载链接 | https://paddle-serving.bj.bcebos.com/lac/lac_model.tar.gz | -| 客户端/服务端代码 | https://github.com/PaddlePaddle/Serving/tree/develop/python/examples/lac | -| 介绍 | 获取中文语句的分词 | - - - -| Key | Value | -| :----------------- | :----------------------------------------------------------- | -| 模型名 | DNN-CTR | -| 下载链接 | https://paddle-serving.bj.bcebos.com/criteo_ctr_example/criteo_ctr_demo_model.tar.gz | -| 客户端/服务端代码 | https://github.com/PaddlePaddle/Serving/tree/develop/python/examples/criteo_ctr | -| 介绍 | 从项目的特征向量中获得点击概率 | - +

Paddle Serving的核心功能

+- 与Paddle训练紧密连接，绝大部分Paddle模型可以 **一键部署**. +- 支持 **工业级的服务能力** 例如模型管理，在线加载，在线A/B测试等. +- 支持 **分布式键值对索引** 助力于大规模稀疏特征作为模型输入. +- 支持客户端和服务端之间 **高并发和高效通信**. +- 支持 **多种编程语言** 开发客户端，例如Golang，C++和Python. +- **可伸缩框架设计** 可支持不限于Paddle的模型服务.

文档