Skip to content

S
Serving

PaddlePaddle / Serving
接近 2 年 前同步成功

通知 186
Star 833
Fork 253
S
Serving
未验证 提交 cbffc0d5 编写于 作者: J Jiawei Wang 提交者： GitHub
浏览文件
操作

Merge branch 'develop' into develop

上级 a3633646 1a3fcf00
展开全部 隐藏空白更改
内联 并排
Showing with 2309 addition and 596 deletion
doc/deprecated/DESIGN.md → doc/C++DESIGN.md
浏览文件 @ cbffc0d5
# Paddle Serving Design
([简体中文](./DESIGN_CN.md)|English)
([简体中文](./C++DESIGN_CN.md)|English)
## 1. Background
......@@ -45,11 +45,11 @@ Models that can be predicted using the Paddle Inference Library, models saved du
### 3.4 Server Inferface
![Server Interface](../server_interface.png)
![Server Interface](server_interface.png)
### 3.5 Client Interface
<img src='../client_inferface.png' width = "600" height = "200">
<img src='client_inferface.png' width = "600" height = "200">
### 3.6 Client io used during Training
......@@ -66,7 +66,7 @@ def save_model(server_model_folder,
## 4. Paddle Serving Underlying Framework
![Paddle-Serging Overall Architecture](../framework.png)
![Paddle-Serging Overall Architecture](framework.png)
**Model Management Framework**: Connects model files of multiple machine learning platforms and provides a unified inference interface
**Business Scheduling Framework**: Abstracts the calculation logic of various different inference models, provides a general DAG scheduling framework, and connects different operators through DAG diagrams to complete a prediction service together. This abstract model allows users to conveniently implement their own calculation logic, and at the same time facilitates operator sharing. (Users build their own forecasting services. A large part of their work is to build DAGs and provide operators.)
......@@ -102,31 +102,31 @@ class FluidFamilyCore {
With reference to the abstract idea of model calculation of the TensorFlow framework, the business logic is abstracted into a DAG diagram, driven by configuration, generating a workflow, and skipping C ++ code compilation. Each specific step of the service corresponds to a specific OP. The OP can configure the upstream OP that it depends on. Unified message passing between OPs is achieved by the thread-level bus and channel mechanisms. For example, the service process of a simple prediction service can be abstracted into 3 steps including reading request data-> calling the prediction interface-> writing back the prediction result, and correspondingly implemented to 3 OP: ReaderOp-> ClassifyOp-> WriteOp
![Infer Service](../predict-service.png)
![Infer Service](predict-service.png)
Regarding the dependencies between OPs, and the establishment of workflows through OPs, you can refer to [从零开始写一个预测服务](CREATING.md) (simplified Chinese Version)
Server instance perspective
![Server instance perspective](../server-side.png)
![Server instance perspective](server-side.png)
#### 4.2.2 Paddle Serving Multi-Service Mechanism
![Paddle Serving multi-service](../multi-service.png)
![Paddle Serving multi-service](multi-service.png)
Paddle Serving instances can load multiple models at the same time, and each model uses a Service (and its configured workflow) to undertake services. You can refer to [service configuration file in Demo example](../../tools/cpp_examples/demo-serving/conf/service.prototxt) to learn how to configure multiple services for the serving instance
Paddle Serving instances can load multiple models at the same time, and each model uses a Service (and its configured workflow) to undertake services. You can refer to [service configuration file in Demo example](../tools/cpp_examples/demo-serving/conf/service.prototxt) to learn how to configure multiple services for the serving instance
#### 4.2.3 Hierarchical relationship of business scheduling
From the client's perspective, a Paddle Serving service can be divided into three levels: Service, Endpoint, and Variant from top to bottom.
![Call hierarchy relationship](../multi-variants.png)
![Call hierarchy relationship](multi-variants.png)
One Service corresponds to one inference model, and there is one endpoint under the model. Different versions of the model are implemented through multiple variant concepts under endpoint:
The same model prediction service can configure multiple variants, and each variant has its own downstream IP list. The client code can configure relative weights for each variant to achieve the relationship of adjusting the traffic ratio (refer to the description of variant_weight_list in [Client Configuration](../CLIENT_CONFIGURE.md) section 3.2).
The same model prediction service can configure multiple variants, and each variant has its own downstream IP list. The client code can configure relative weights for each variant to achieve the relationship of adjusting the traffic ratio (refer to the description of variant_weight_list in [Client Configuration](CLIENT_CONFIGURE.md) section 3.2).
![Client-side proxy function](../client-side-proxy.png)
![Client-side proxy function](client-side-proxy.png)
## 5. User Interface
......@@ -141,7 +141,7 @@ No matter how the communication protocol changes, the framework only needs to en
### 5.1 Data Compression Method
Baidu-rpc has built-in data compression methods such as snappy, gzip, zlib, which can be configured in the configuration file (refer to [Client Configuration](../CLIENT_CONFIGURE.md) Section 3.1 for an introduction to compress_type)
Baidu-rpc has built-in data compression methods such as snappy, gzip, zlib, which can be configured in the configuration file (refer to [Client Configuration](CLIENT_CONFIGURE.md) Section 3.1 for an introduction to compress_type)
### 5.2 C ++ SDK API Interface
......
doc/deprecated/DESIGN_CN.md → doc/C++DESIGN_CN.md
浏览文件 @ cbffc0d5
# Paddle Serving设计方案
(简体中文|[English](./DESIGN.md))
(简体中文|[English](./C++DESIGN.md))
注意本页内容有已经过期，请查看：[设计文档](https://github.com/PaddlePaddle/Serving/blob/develop/doc/DESIGN_DOC_CN.md)
注意本页内容有已经过期，请查看：[设计文档](DESIGN_DOC_CN.md)
## 1. 项目背景
......@@ -47,11 +47,11 @@ PaddlePaddle是百度开源的机器学习框架，广泛支持各种深度学
### 3.4 Server Inferface
![Server Interface](../server_interface.png)
![Server Interface](server_interface.png)
### 3.5 Client Interface
<img src='../client_inferface.png' width = "600" height = "200">
<img src='client_inferface.png' width = "600" height = "200">
### 3.6 训练过程中使用的Client io
......@@ -68,7 +68,7 @@ def save_model(server_model_folder,
## 4. Paddle Serving底层框架
![Paddle-Serging总体框图](../framework.png)
![Paddle-Serging总体框图](framework.png)
**模型管理框架**：对接多种机器学习平台的模型文件，向上提供统一的inference接口
**业务调度框架**：对各种不同预测模型的计算逻辑进行抽象，提供通用的DAG调度框架，通过DAG图串联不同的算子，共同完成一次预测服务。该抽象模型使用户可以方便的实现自己的计算逻辑，同时便于算子共用。（用户搭建自己的预测服务，很大一部分工作是搭建DAG和提供算子的实现）
......@@ -104,31 +104,31 @@ class FluidFamilyCore {
参考TF框架的模型计算的抽象思想，将业务逻辑抽象成DAG图，由配置驱动，生成workflow，跳过C++代码编译。业务的每个具体步骤，对应一个具体的OP，OP可配置自己依赖的上游OP。OP之间消息传递统一由线程级Bus和channel机制实现。例如，一个简单的预测服务的服务过程，可以抽象成读请求数据->调用预测接口->写回预测结果等3个步骤，相应的实现到3个OP: ReaderOp->ClassifyOp->WriteOp
![预测服务Service](../predict-service.png)
![预测服务Service](predict-service.png)
关于OP之间的依赖关系，以及通过OP组建workflow，可以参考[从零开始写一个预测服务](https://github.com/PaddlePaddle/Serving/blob/develop/doc/deprecated/CREATING.md)的相关章节
关于OP之间的依赖关系，以及通过OP组建workflow，可以参考[从零开始写一个预测服务](CREATING.md)的相关章节
服务端实例透视图
![服务端实例透视图](../server-side.png)
![服务端实例透视图](server-side.png)
#### 4.2.2 Paddle Serving的多服务机制
![Paddle Serving的多服务机制](../multi-service.png)
![Paddle Serving的多服务机制](multi-service.png)
Paddle Serving实例可以同时加载多个模型，每个模型用一个Service（以及其所配置的workflow）承接服务。可以参考[Demo例子中的service配置文件](../..//tools/cpp_examples/demo-serving/conf/service.prototxt)了解如何为serving实例配置多个service
Paddle Serving实例可以同时加载多个模型，每个模型用一个Service（以及其所配置的workflow）承接服务。可以参考[Demo例子中的service配置文件](../tools/cpp_examples/demo-serving/conf/service.prototxt)了解如何为serving实例配置多个service
#### 4.2.3 业务调度层级关系
从客户端看，一个Paddle Serving service从顶向下可分为Service, Endpoint, Variant等3个层级
![调用层级关系](../multi-variants.png)
![调用层级关系](multi-variants.png)
一个Service对应一个预测模型，模型下有1个endpoint。模型的不同版本，通过endpoint下多个variant概念实现：
同一个模型预测服务，可以配置多个variant，每个variant有自己的下游IP列表。客户端代码可以对各个variant配置相对权重，以达到调节流量比例的关系（参考[客户端配置](../CLIENT_CONFIGURE.md)第3.2节中关于variant_weight_list的说明）。
同一个模型预测服务，可以配置多个variant，每个variant有自己的下游IP列表。客户端代码可以对各个variant配置相对权重，以达到调节流量比例的关系（参考[客户端配置](CLIENT_CONFIGURE.md)第3.2节中关于variant_weight_list的说明）。
![Client端proxy功能](../client-side-proxy.png)
![Client端proxy功能](client-side-proxy.png)
## 5. 用户接口
......@@ -143,7 +143,7 @@ Paddle Serving实例可以同时加载多个模型，每个模型用一个Servic
### 5.1 数据压缩方法
Baidu-rpc内置了snappy, gzip, zlib等数据压缩方法，可在配置文件中配置（参考[客户端配置](../CLIENT_CONFIGURE.md)第3.1节关于compress_type的介绍）
Baidu-rpc内置了snappy, gzip, zlib等数据压缩方法，可在配置文件中配置（参考[客户端配置](CLIENT_CONFIGURE.md)第3.1节关于compress_type的介绍）
### 5.2 C++ SDK API接口
......
doc/deprecated/CREATING.md → doc/CREATING.md
浏览文件 @ cbffc0d5
......@@ -4,10 +4,10 @@
图像分类是根据图像的语义信息将不同类别图像区分开来，是计算机视觉中重要的基本问题，也是图像检测、图像分割、物体跟踪、行为分析等其他高层视觉任务的基础。图像分类在很多领域有广泛应用，包括安防领域的人脸识别和智能视频分析等，交通领域的交通场景识别，互联网领域基于内容的图像检索和相册自动归类，医学领域的图像识别等。
Paddle Serving已经提供了一个基于ResNet的模型预测服务，按照INSTALL.md中所述步骤，编译Paddle Serving，然后按GETTING_STARTED.md所述步骤启动client端和server端即可看到预测服务运行效果。
本文接下来以图像分类任务为例，介绍从零搭建一个模型预测服务的步骤。
**本文件仅供参考，部分接口内容已经变动**
## 2. Serving端
......@@ -75,9 +75,9 @@ service ImageClassifyService {
#### 2.2.2 示例配置
关于Serving端的配置的详细信息，可以参考[Serving端配置](../SERVING_CONFIGURE.md)
关于Serving端的配置的详细信息，可以参考[Serving端配置](SERVING_CONFIGURE.md)
以下配置文件将ReaderOP, ClassifyOP和WriteJsonOP串联成一个workflow (关于OP/workflow等概念，可参考[设计文档](DESIGN.md))
以下配置文件将ReaderOP, ClassifyOP和WriteJsonOP串联成一个workflow (关于OP/workflow等概念，可参考[设计文档](C++DESIGN_CN.md))
- 配置文件示例：
......@@ -392,4 +392,4 @@ predictors {
}
}
```
关于客户端的详细配置选项，可参考[CLIENT CONFIGURATION](../CLIENT_CONFIGURE.md)
关于客户端的详细配置选项，可参考[CLIENT CONFIGURATION](CLIENT_CONFIGURE.md)
doc/DESIGN_DOC.md
浏览文件 @ cbffc0d5
......@@ -60,7 +60,7 @@ Docker is an open source application container engine that allows developers to
Paddle Serving provides 4 development language client SDKs, including Python, C++, Java, and Golang. Golang SDK is under construction, We hope that interested open source developers can help submit PR.
+ Python, Refer to the client example under python/examples or 4.2 web service example.
+ C++, Refer to《[从零开始写一个预测服务](deprecated/CREATING.md)》
+ C++, Refer to《[从零开始写一个预测服务](CREATING.md)》
+ Java, Refer to《[Paddle Serving Client Java SDK](JAVA_SDK.md)》
+ Golang, Refer to《[How to use Go Client of Paddle Serving](deprecated/IMDB_GO_CLIENT.md)》
......
doc/DESIGN_DOC_CN.md
浏览文件 @ cbffc0d5
......@@ -61,7 +61,7 @@ Docker 是一个开源的应用容器引擎，让开发者可以打包他们的
Paddle Serving提供了4种开发语言SDK，包括Python、C++、Java、Golang。Golang SDK在建设中，有兴趣的开源开发者可以提交PR。
+ Python，参考python/examples下client示例 或 4.2 web服务示例
+ C++，参考《[从零开始写一个预测服务](deprecated/CREATING.md)》
+ C++，参考《[从零开始写一个预测服务](CREATING.md)》
+ Java，参考《[Paddle Serving Client Java SDK](JAVA_SDK_CN.md)》
+ Golang，参考《[如何在Paddle Serving使用Go Client](deprecated/IMDB_GO_CLIENT_CN.md)》
......
doc/PIPELINE_SERVING.md
浏览文件 @ cbffc0d5
此差异已折叠。
doc/PIPELINE_SERVING_CN.md
浏览文件 @ cbffc0d5
此差异已折叠。
doc/SERVING_AUTH_DOCKER.md 0 → 100644
浏览文件 @ cbffc0d5
# 在Paddle Serving使用安全网关
## 简介
在之前的服务部署示例中，我们都从开发的角度切入，然而，在现实的生产环境中，仅仅提供一个能够预测的远端服务接口还远远不够。我们仍然要考虑以下不足。
- 这个服务还不能以网关的形式提供，访问路径难以管理。
- 这个服务接口不够安全，需要做相应的鉴权。
- 这个服务接口不能够控制流量，无法合理利用资源。
本文档的作用，就以 Uci 房价预测服务为例，来介绍如何强化预测服务API接口安全。API网关作为流量入口，对接口进行统一管理。但API网关可以提供流量加密和鉴权等安全功能。
## Docker部署
可以使用docker-compose来部署安全网关。这个示例的步骤就是 [部署本地Serving容器] - [部署本地安全网关] - [通过安全网关访问Serving]
**注明：** docker-compose与docker不一样，它依赖于docker，一次可以部署多个docker容器，可以类比于本地版的kubenetes，docker-compose的教程请参考[docker-compose安装](https://docs.docker.com/compose/install/)
```shell
docker-compose -f tools/auth/auth-serving-docker.yaml up -d
```
可以通过 `docker ps` 来查看启动的容器。
```shell
3035cf445029 pantsel/konga:next "/app/start.sh" About an hour ago Up About an hour 0.0.0.0:8005->1337/tcp anquan_konga_1
7ce3abee550c registry.baidubce.com/serving_gateway/kong:paddle "/docker-entrypoint.…" About an hour ago Up About an hour (healthy) 0.0.0.0:8000->8000/tcp, 127.0.0.1:8001->8001/tcp, 0.0.0.0:8443->8443/tcp, 127.0.0.1:8444->8444/tcp anquan_kong_1
25810fd79a27 postgres:9.6 "docker-entrypoint.s…" About an hour ago Up About an hour (healthy) 5432/tcp anquan_db_1
ee59a3dd4806 registry.baidubce.com/serving_dev/serving-runtime:cpu-py36 "bash -c ' wget --no…" About an hour ago Up About an hour 0.0.0.0:9393->9393/tcp anquan_serving_1
665fd8a34e15 redis:latest "docker-entrypoint.s…" About an hour ago Up About an hour 0.0.0.0:6379->6379/tcp anquan_redis_1
```
其中我们之前serving容器 以 9393端口暴露，KONG网关的端口是8443， KONG的Web控制台的端口是8001。接下来我们在浏览器访问 `https://$IP_ADDR:8001`, 其中 IP_ADDR就是宿主机的IP。
<img src="kong-dashboard.png">
可以看到在注册结束后，登陆，看到了 DASHBOARD，我们先看SERVICES，可以看到`serving_service`，这意味着我们端口在9393的Serving服务已经在KONG当中被注册。
<img src="kong-services.png">
<img src="kong-routes.png">
然后在ROUTES中，我们可以看到 serving 被链接到了 `/serving-uci`。
最后我们点击 CONSUMERS - default_user - Credentials - API KEYS ，我们可以看到 `Api Keys` 下看到很多key
<img src="kong-api_keys.png">
接下来可以通过curl访问
```shell
curl -H "Content-Type:application/json" -H "X-INSTANCE-ID:kong_ins" -H "apikey:hP6v25BQVS5CcS1nqKpxdrFkUxze9JWD" -X POST -d '{"feed":[{"x": [0.0137, -0.1136, 0.2553, -0.0692, 0.0582, -0.0727, -0.1583, -0.0584, 0.6283, 0.4919, 0.1856, 0.0795, -0.0332]}], "fetch":["price"]}' https://127.0.0.1:8443/serving-uci/uci/prediction -k
```
与之前的Serving HTTP服务相比，有以下区别。
- 使用https加密访问，而不是http
- 使用serving_uci的路径映射到网关
- 在header处增加了 `X-INSTANCE-ID`和`apikey`
## K8S部署
同样，我们也提供了K8S集群部署Serving安全网关的方式。
### Step 1：启动Serving服务
我们仍然以 [Uci房价预测](../python/examples/fit_a_line)服务作为例子，这里省略了镜像制作的过程，详情可以参考 [在Kubernetes集群上部署Paddle Serving](./PADDLE_SERVING_ON_KUBERNETES.md)。
在这里我们直接执行
```
kubectl apply -f tools/auth/serving-demo-k8s.yaml
```
可以看到
### Step 2: 安装 KONG (一个集群只需要执行一次就可以)
接下来我们执行KONG Ingress的安装
```
kubectl apply -f tools/auth/kong-install.yaml
```
输出是
```
namespace/kong created
customresourcedefinition.apiextensions.k8s.io/kongclusterplugins.configuration.konghq.com created
customresourcedefinition.apiextensions.k8s.io/kongconsumers.configuration.konghq.com created
customresourcedefinition.apiextensions.k8s.io/kongingresses.configuration.konghq.com created
customresourcedefinition.apiextensions.k8s.io/kongplugins.configuration.konghq.com created
customresourcedefinition.apiextensions.k8s.io/tcpingresses.configuration.konghq.com created
serviceaccount/kong-serviceaccount created
clusterrole.rbac.authorization.k8s.io/kong-ingress-clusterrole created
clusterrolebinding.rbac.authorization.k8s.io/kong-ingress-clusterrole-nisa-binding created
service/kong-proxy created
service/kong-validation-webhook created
deployment.apps/ingress-kong created
```
我们可以输入
```
kubectl get service --all-namespaces
```
会显示
```
NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
default uci ClusterIP 172.16.87.89 <none> 9393/TCP 7d7h
kong kong-proxy NodePort 172.16.23.91 <none> 80:8175/TCP,443:8521/TCP 102m
kong kong-validation-webhook ClusterIP 172.16.114.93 <none> 443/TCP 102m
```
### Step 3: 创建Ingress资源
接下来需要做Serving服务和KONG的链接
```
kubectl apply -f tools/auth/kong-ingress-k8s.yaml
```
我们也给出yaml文件内容
```
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: demo
annotations:
konghq.com/strip-path: "true"
kubernetes.io/ingress.class: kong
spec:
rules:
- http:
paths:
- path: /foo
backend:
serviceName: {{SERVING_SERVICE_NAME}}
servicePort: {{SERVICE_PORT}}
```
其中serviceName就是uci，servicePort就是9393，如果是别的服务就需要改这两个字段，最终会映射到`/foo`下。
在这一步之后，我们就可以
```
curl -H "Content-Type:application/json" -X POST -d '{"feed":[{"x": [0.0137, -0.1136, 0.2553, -0.0692, 0.0582, -0.0727, -0.1583, -0.0584, 0.6283, 0.4919, 0.1856, 0.0795, -0.0332]}], "fetch":["price"]}' http://$IP:$PORT/foo/uci/prediction
```
### Step 4: 增加安全网关限制
之前的接口没有鉴权功能，无法验证用户身份合法性，现在我们添加一个key-auth插件
执行
```
kubectl apply -f key-auth-k8s.yaml
```
其中,yaml文内容为
```
apiVersion: configuration.konghq.com/v1
kind: KongPlugin
metadata:
name: key-auth
plugin: key-auth
```
现在，需要创建secret，key值为用户指定，需要在请求时携带Header中apikey字段
执行
```
kubectl create secret generic default-apikey \
--from-literal=kongCredType=key-auth \
--from-literal=key=ZGVmYXVsdC1hcGlrZXkK
```
在这里，我们的key是随意制定了一串 `ZGVmYXVsdC1hcGlrZXkK`，实际情况也可以
创建一个用户（consumer）标识访问者身份，并未该用户绑定apikey。
执行
```
kubectl apply -f kong-consumer-k8s.yaml
```
其中,yaml文内容为
```
apiVersion: configuration.konghq.com/v1
kind: KongConsumer
metadata:
name: default
annotations:
kubernetes.io/ingress.class: kong
username: default
credentials:
- default-apikey
```
如果我们这时还想再像上一步一样的做curl访问，会发现已经无法访问，此时已经具备了安全能力，我们需要对应的key。
### Step 5: 通过API Key访问服务
执行
```
curl -H "Content-Type:application/json" -H "apikey:ZGVmYXVsdC1hcGlrZXkK" -X POST -d '{"feed":[{"x": [0.0137, -0.1136, 0.2553, -0.0692, 0.0582, -0.0727, -0.1583, -0.0584, 0.6283, 0.4919, 0.1856, 0.0795, -0.0332]}], "fetch":["price"]}' https://$IP:$PORT/foo/uci/prediction -k
```
我们可以看到 apikey 已经加入到了curl请求的header当中。
python/examples/pipeline/bert/README.md 0 → 100644
浏览文件 @ cbffc0d5
# Imagenet Pipeline WebService
This document will takes Imagenet service as an example to introduce how to use Pipeline WebService.
## Get model
```
sh get_model.sh
```
## Start server
```
python3 web_service.py &>log.txt &
```
## RPC test
```
python3 pipeline_rpc_client.py
```
python/examples/pipeline/bert/README_CN.md 0 → 100644
浏览文件 @ cbffc0d5
# Imagenet Pipeline WebService
这里以 Imagenet 服务为例来介绍 Pipeline WebService 的使用。
## 获取模型
```
sh get_model.sh
```
## 启动服务
```
python3 web_service.py &>log.txt &
```
## 测试
```
python3 pipeline_rpc_client.py
```
python/examples/pipeline/imagenet/README.md
浏览文件 @ cbffc0d5
......@@ -10,10 +10,10 @@ sh get_model.sh
## Start server
```
python resnet50_web_service.py &>log.txt &
python3 resnet50_web_service.py &>log.txt &
```
## RPC test
```
python pipeline_rpc_client.py
python3 pipeline_rpc_client.py
```
python/examples/pipeline/imagenet/README_CN.md
浏览文件 @ cbffc0d5
......@@ -10,11 +10,10 @@ sh get_model.sh
## 启动服务
```
python resnet50_web_service.py &>log.txt &
python3 resnet50_web_service.py &>log.txt &
```
## 测试
```
python pipeline_rpc_client.py
python3 pipeline_rpc_client.py
```
python/examples/pipeline/imagenet/config.yml
浏览文件 @ cbffc0d5
......@@ -31,3 +31,11 @@ op:
#Fetch结果列表，以client_config中fetch_var的alias_name为准
fetch_list: ["score"]
#precsion, 预测精度，降低预测精度可提升推理速度
#GPU 支持: "fp32"(default), "fp16", "int8"；
#CPU 支持: "fp32"(default), "fp16", "bf16"(mkldnn); 不支持: "int8"
precision: "fp16"
#ir_optim开关
ir_optim: False
python/examples/pipeline/imdb_model_ensemble/README.md
浏览文件 @ cbffc0d5
......@@ -8,12 +8,12 @@ sh get_data.sh
## Start servers
```
python -m paddle_serving_server.serve --model imdb_cnn_model --port 9292 &> cnn.log &
python -m paddle_serving_server.serve --model imdb_bow_model --port 9393 &> bow.log &
python test_pipeline_server.py &>pipeline.log &
python3 -m paddle_serving_server.serve --model imdb_cnn_model --port 9292 &> cnn.log &
python3 -m paddle_serving_server.serve --model imdb_bow_model --port 9393 &> bow.log &
python3 test_pipeline_server.py &>pipeline.log &
```
## Start clients
```
python test_pipeline_client.py
python3 test_pipeline_client.py
```
python/examples/pipeline/imdb_model_ensemble/README_CN.md
浏览文件 @ cbffc0d5
......@@ -8,12 +8,12 @@ sh get_data.sh
## 启动服务
```
python -m paddle_serving_server.serve --model imdb_cnn_model --port 9292 &> cnn.log &
python -m paddle_serving_server.serve --model imdb_bow_model --port 9393 &> bow.log &
python test_pipeline_server.py &>pipeline.log &
python3 -m paddle_serving_server.serve --model imdb_cnn_model --port 9292 &> cnn.log &
python3 -m paddle_serving_server.serve --model imdb_bow_model --port 9393 &> bow.log &
python3 test_pipeline_server.py &>pipeline.log &
```
## 启动客户端
```
python test_pipeline_client.py
python3 test_pipeline_client.py
```
python/examples/pipeline/ocr/README.md
浏览文件 @ cbffc0d5
......@@ -4,11 +4,13 @@
This document will take OCR as an example to show how to use Pipeline WebService to start multi-model tandem services.
This OCR example only supports Process OP.
## Get Model
```
python -m paddle_serving_app.package --get_model ocr_rec
python3 -m paddle_serving_app.package --get_model ocr_rec
tar -xzvf ocr_rec.tar.gz
python -m paddle_serving_app.package --get_model ocr_det
python3 -m paddle_serving_app.package --get_model ocr_det
tar -xzvf ocr_det.tar.gz
```
......@@ -18,14 +20,16 @@ wget --no-check-certificate https://paddle-serving.bj.bcebos.com/ocr/test_imgs.t
tar xf test_imgs.tar
```
## Start Service
## Run services
### 1.Start a single server and client.
```
python web_service.py &>log.txt &
python3 web_service.py &>log.txt &
```
## Test
Test
```
python pipeline_http_client.py
python3 pipeline_http_client.py
```
<!--
......@@ -35,11 +39,22 @@ python pipeline_http_client.py
### RPC
```
python pipeline_rpc_client.py
python3 pipeline_rpc_client.py
```
### HTTP
```
python pipeline_http_client.py
python3 pipeline_http_client.py
```
-->
### 2.Run benchmark
```
python3 web_service.py &>log.txt &
```
Test
```
sh benchmark.sh
```
python/examples/pipeline/ocr/README_CN.md
浏览文件 @ cbffc0d5
......@@ -3,12 +3,13 @@
([English](./README.md)|简体中文)
本文档将以 OCR 为例，介绍如何使用 Pipeline WebService 启动多模型串联的服务。
本示例仅支持进程OP模式。
## 获取模型
```
python -m paddle_serving_app.package --get_model ocr_rec
python3 -m paddle_serving_app.package --get_model ocr_rec
tar -xzvf ocr_rec.tar.gz
python -m paddle_serving_app.package --get_model ocr_det
python3 -m paddle_serving_app.package --get_model ocr_det
tar -xzvf ocr_det.tar.gz
```
......@@ -19,13 +20,15 @@ tar xf test_imgs.tar
```
## 启动 WebService
### 1.启动单server、单client
```
python web_service.py &>log.txt &
python3 web_service.py &>log.txt &
```
## 测试
```
python pipeline_http_client.py
python3 pipeline_http_client.py
```
<!--
......@@ -36,12 +39,22 @@ python pipeline_http_client.py
### RPC
```
python pipeline_rpc_client.py
python3 pipeline_rpc_client.py
```
### HTTP
```
python pipeline_http_client.py
python3 pipeline_http_client.py
```
-->
### 2.启动 benchmark
```
python3 web_service.py &>log.txt &
```
Test
```
sh benchmark.sh
```
python/examples/pipeline/simple_web_service/config.yml
浏览文件 @ cbffc0d5
......@@ -30,4 +30,12 @@ op:
client_type: local_predictor
#Fetch结果列表，以client_config中fetch_var的alias_name为准
fetch_list: ["price"]
fetch_list: ["price"]
#precsion, 预测精度，降低预测精度可提升预测速度
#GPU 支持: "fp32"(default), "fp16", "int8"；
#CPU 支持: "fp32"(default), "fp16", "bf16"(mkldnn); 不支持: "int8"
precision: "FP16"
#ir_optim开关
ir_optim: False
python/examples/resnet_v2_50/benchmark.py 0 → 100644
浏览文件 @ cbffc0d5
# -*- coding: utf-8 -*-
#
# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# pylint: disable=doc-string-missing
from __future__ import unicode_literals, absolute_import
import os
import sys
import time
import json
import requests
import numpy as np
from paddle_serving_client import Client
from paddle_serving_client.utils import MultiThreadRunner
from paddle_serving_client.utils import benchmark_args, show_latency
from paddle_serving_app.reader import Sequential, File2Image, Resize, CenterCrop
from paddle_serving_app.reader import RGB2BGR, Transpose, Div, Normalize
args = benchmark_args()
def single_func(idx, resource):
total_number = 0
profile_flags = False
latency_flags = False
if os.getenv("FLAGS_profile_client"):
profile_flags = True
if os.getenv("FLAGS_serving_latency"):
latency_flags = True
latency_list = []
if args.request == "rpc":
client = Client()
client.load_client_config(args.model)
client.connect([resource["endpoint"][idx % len(resource["endpoint"])]])
start = time.time()
for i in range(turns):
if args.batch_size >= 1:
l_start = time.time()
seq = Sequential([
File2Image(), Resize(256), CenterCrop(224), RGB2BGR(),
Transpose((2, 0, 1)), Div(255), Normalize(
[0.485, 0.456, 0.406], [0.229, 0.224, 0.225], True)
])
image_file = "daisy.jpg"
img = seq(image_file)
feed_data = np.array(img)
feed_data = np.expand_dims(feed_data, 0).repeat(
args.batch_size, axis=0)
result = client.predict(
feed={"image": feed_data},
fetch=["save_infer_model/scale_0.tmp_0"],
batch=True)
l_end = time.time()
if latency_flags:
latency_list.append(l_end * 1000 - l_start * 1000)
total_number = total_number + 1
else:
print("unsupport batch size {}".format(args.batch_size))
else:
raise ValueError("not implemented {} request".format(args.request))
end = time.time()
if latency_flags:
return [[end - start], latency_list, [total_number]]
else:
return [[end - start]]
if __name__ == '__main__':
multi_thread_runner = MultiThreadRunner()
endpoint_list = ["127.0.0.1:9393"]
turns = 1
start = time.time()
result = multi_thread_runner.run(
single_func, args.thread, {"endpoint": endpoint_list,
"turns": turns})
end = time.time()
total_cost = end - start
total_number = 0
avg_cost = 0
for i in range(args.thread):
avg_cost += result[0][i]
total_number += result[2][i]
avg_cost = avg_cost / args.thread
print("total cost-include init: {}s".format(total_cost))
print("each thread cost: {}s. ".format(avg_cost))
print("qps: {}samples/s".format(args.batch_size * total_number / (
avg_cost * args.thread)))
print("qps(request): {}samples/s".format(total_number / (avg_cost *
args.thread)))
print("total count: {} ".format(total_number))
if os.getenv("FLAGS_serving_latency"):
show_latency(result[1])
python/examples/resnet_v2_50/benchmark.sh 0 → 100644
浏览文件 @ cbffc0d5
rm profile_log*
rm -rf resnet_log*
export CUDA_VISIBLE_DEVICES=0,1,2,3
export FLAGS_profile_server=1
export FLAGS_profile_client=1
export FLAGS_serving_latency=1
gpu_id=3
#save cpu and gpu utilization log
if [ -d utilization ];then
rm -rf utilization
else
mkdir utilization
fi
#start server
python3.6 -m paddle_serving_server.serve --model $1 --port 9393 --thread 10 --gpu_ids $gpu_id --use_trt --ir_optim > elog 2>&1 &
sleep 15
#warm up
python3.6 benchmark.py --thread 1 --batch_size 1 --model $2/serving_client_conf.prototxt --request rpc > profile 2>&1
echo -e "import psutil\nimport time\nwhile True:\n\tcpu_res = psutil.cpu_percent()\n\twith open('cpu.txt', 'a+') as f:\n\t\tf.write(f'{cpu_res}\\\n')\n\ttime.sleep(0.1)" > cpu.py
for thread_num in 1 2 4 8 16
do
for batch_size in 1 4 8 16 32
do
job_bt=`date '+%Y%m%d%H%M%S'`
nvidia-smi --id=$gpu_id --query-compute-apps=used_memory --format=csv -lms 100 > gpu_memory_use.log 2>&1 &
nvidia-smi --id=$gpu_id --query-gpu=utilization.gpu --format=csv -lms 100 > gpu_utilization.log 2>&1 &
rm -rf cpu.txt
python3.6 cpu.py &
gpu_memory_pid=$!
python3.6 benchmark.py --thread $thread_num --batch_size $batch_size --model $2/serving_client_conf.prototxt --request rpc > profile 2>&1
kill `ps -ef|grep used_memory|awk '{print $2}'` > /dev/null
kill `ps -ef|grep utilization.gpu|awk '{print $2}'` > /dev/null
kill `ps -ef|grep cpu.py|awk '{print $2}'` > /dev/null
echo "model_name:" $1
echo "thread_num:" $thread_num
echo "batch_size:" $batch_size
echo "=================Done===================="
echo "model_name:$1" >> profile_log_$1
echo "batch_size:$batch_size" >> profile_log_$1
job_et=`date '+%Y%m%d%H%M%S'`
awk 'BEGIN {max = 0} {if(NR>1){if ($1 > max) max=$1}} END {print "CPU_UTILIZATION:", max}' cpu.txt >> profile_log_$1
#awk 'BEGIN {max = 0} {if(NR>1){if ($1 > max) max=$1}} END {print "MAX_GPU_MEMORY:", max}' gpu_memory_use.log >> profile_log_$1
#awk 'BEGIN {max = 0} {if(NR>1){if ($1 > max) max=$1}} END {print "GPU_UTILIZATION:", max}' gpu_utilization.log >> profile_log_$1
grep -av '^0 %' gpu_utilization.log > gpu_utilization.log.tmp
awk 'BEGIN {max = 0} {if(NR>1){if ($1 > max) max=$1}} END {print "MAX_GPU_MEMORY:", max}' gpu_memory_use.log >> profile_log_$1
awk -F" " '{sum+=$1} END {print "GPU_UTILIZATION:", sum/NR, sum, NR }' gpu_utilization.log.tmp >> profile_log_$1
rm -rf gpu_memory_use.log gpu_utilization.log gpu_utilization.log.tmp
python3.6 ../util/show_profile.py profile $thread_num >> profile_log_$1
tail -n 10 profile >> profile_log_$1
echo "" >> profile_log_$1
done
done
#Divided log
awk 'BEGIN{RS="\n\n"}{i++}{print > "resnet_log_"i}' profile_log_$1
mkdir resnet_log && mv resnet_log_* resnet_log
ps -ef|grep 'serving'|grep -v grep|cut -c 9-15 | xargs kill -9
python/examples/resnet_v2_50/run_benchmark.sh 0 → 100644
浏览文件 @ cbffc0d5
if [ ! -x "ResNet50.tar.gz"]; then
wget https://paddle-inference-dist.bj.bcebos.com/AI-Rank/models/Paddle/ResNet50.tar.gz
fi
tar -xzvf ResNet50.tar.gz
python3.6 -m paddle_serving_client.convert --dirname ./ResNet50 --model_filename model --params_filename params
bash benchmark.sh serving_server serving_client
python/paddle_serving_app/local_predict.py
浏览文件 @ cbffc0d5
......@@ -119,8 +119,11 @@ class LocalPredictor(object):
self.fetch_names_to_type_[var.alias_name] = var.fetch_type
precision_type = paddle_infer.PrecisionType.Float32
if precision.lower() in precision_map:
if precision is not None and precision.lower() in precision_map:
precision_type = precision_map[precision.lower()]
else:
logger.warning("precision error!!! Please check precision:{}".
format(precision))
if use_profile:
config.enable_profile()
if mem_optim:
......@@ -156,8 +159,11 @@ class LocalPredictor(object):
if not use_gpu and not use_lite:
if precision_type == paddle_infer.PrecisionType.Int8:
config.enable_quantizer()
if precision.lower() == "bf16":
logger.warning(
"PRECISION INT8 is not supported in CPU right now! Please use fp16 or bf16."
)
#config.enable_quantizer()
if precision is not None and precision.lower() == "bf16":
config.enable_mkldnn_bfloat16()
self.predictor = paddle_infer.create_predictor(config)
......
python/pipeline/local_service_handler.py
浏览文件 @ cbffc0d5
......@@ -44,7 +44,8 @@ class LocalServiceHandler(object):
mem_optim=True,
ir_optim=False,
available_port_generator=None,
use_profile=False):
use_profile=False,
precision="fp32"):
"""
Initialization of localservicehandler
......@@ -62,6 +63,7 @@ class LocalServiceHandler(object):
ir_optim: use calculation chart optimization, False default.
available_port_generator: generate available ports
use_profile: use profiling, False default.
precision: inference precesion, e.g. "fp32", "fp16", "int8"
Returns:
None
......@@ -137,16 +139,17 @@ class LocalServiceHandler(object):
self._server_pros = []
self._use_profile = use_profile
self._fetch_names = fetch_names
self._precision = precision
_LOGGER.info(
"Models({}) will be launched by device {}. use_gpu:{}, "
"use_trt:{}, use_lite:{}, use_xpu:{}, device_type:{}, devices:{}, "
"mem_optim:{}, ir_optim:{}, use_profile:{}, thread_num:{}, "
"client_type:{}, fetch_names:{}".format(
"client_type:{}, fetch_names:{} precision:{}".format(
model_config, self._device_name, self._use_gpu, self._use_trt,
self._use_lite, self._use_xpu, device_type, self._devices,
self._mem_optim, self._ir_optim, self._use_profile,
self._thread_num, self._client_type, self._fetch_names))
self._use_lite, self._use_xpu, device_type, self._devices, self.
_mem_optim, self._ir_optim, self._use_profile, self._thread_num,
self._client_type, self._fetch_names, self._precision))
def get_fetch_list(self):
return self._fetch_names
......@@ -197,14 +200,15 @@ class LocalServiceHandler(object):
ir_optim=self._ir_optim,
use_trt=self._use_trt,
use_lite=self._use_lite,
use_xpu=self._use_xpu)
use_xpu=self._use_xpu,
precision=self._precision)
return self._local_predictor_client
def get_client_config(self):
return os.path.join(self._model_config, "serving_server_conf.prototxt")
def _prepare_one_server(self, workdir, port, gpuid, thread_num, mem_optim,
ir_optim):
ir_optim, precision):
"""
According to self._device_name, generating one Cpu/Gpu/Arm Server, and
setting the model config amd startup params.
......@@ -216,6 +220,7 @@ class LocalServiceHandler(object):
thread_num: thread num
mem_optim: use memory/graphics memory optimization
ir_optim: use calculation chart optimization
precision: inference precison, e.g."fp32", "fp16", "int8"
Returns:
server: CpuServer/GpuServer
......@@ -256,6 +261,7 @@ class LocalServiceHandler(object):
server.set_num_threads(thread_num)
server.set_memory_optimize(mem_optim)
server.set_ir_optimize(ir_optim)
server.set_precision(precision)
server.load_model_config(self._model_config)
server.prepare_server(
......@@ -292,7 +298,8 @@ class LocalServiceHandler(object):
device_id,
thread_num=self._thread_num,
mem_optim=self._mem_optim,
ir_optim=self._ir_optim))
ir_optim=self._ir_optim,
precision=self._precision))
def start_server(self):
"""
......
python/pipeline/logger.py
浏览文件 @ cbffc0d5
......@@ -42,22 +42,28 @@ logger_config = {
},
"handlers": {
"f_pipeline.log": {
"class": "logging.FileHandler",
"class": "logging.handlers.RotatingFileHandler",
"level": "INFO",
"formatter": "normal_fmt",
"filename": os.path.join(log_dir, "pipeline.log"),
"maxBytes": 512000000,
"backupCount": 20,
},
"f_pipeline.log.wf": {
"class": "logging.FileHandler",
"class": "logging.handlers.RotatingFileHandler",
"level": "WARNING",
"formatter": "normal_fmt",
"filename": os.path.join(log_dir, "pipeline.log.wf"),
"maxBytes": 512000000,
"backupCount": 10,
},
"f_tracer.log": {
"class": "logging.FileHandler",
"class": "logging.handlers.RotatingFileHandler",
"level": "INFO",
"formatter": "tracer_fmt",
"filename": os.path.join(log_dir, "pipeline.tracer"),
"maxBytes": 512000000,
"backupCount": 5,
},
},
"loggers": {
......
python/pipeline/operator.py
浏览文件 @ cbffc0d5
......@@ -138,6 +138,7 @@ class Op(object):
self.devices = ""
self.mem_optim = False
self.ir_optim = False
self.precision = "fp32"
if self._server_endpoints is None:
server_endpoints = conf.get("server_endpoints", [])
if len(server_endpoints) != 0:
......@@ -159,6 +160,7 @@ class Op(object):
self.mem_optim = local_service_conf.get("mem_optim")
self.ir_optim = local_service_conf.get("ir_optim")
self._fetch_names = local_service_conf.get("fetch_list")
self.precision = local_service_conf.get("precision")
if self.model_config is None:
self.with_serving = False
else:
......@@ -173,7 +175,8 @@ class Op(object):
device_type=self.device_type,
devices=self.devices,
mem_optim=self.mem_optim,
ir_optim=self.ir_optim)
ir_optim=self.ir_optim,
precision=self.precision)
service_handler.prepare_server() # get fetch_list
serivce_ports = service_handler.get_port_list()
self._server_endpoints = [
......@@ -195,7 +198,8 @@ class Op(object):
devices=self.devices,
fetch_names=self._fetch_names,
mem_optim=self.mem_optim,
ir_optim=self.ir_optim)
ir_optim=self.ir_optim,
precision=self.precision)
if self._client_config is None:
self._client_config = service_handler.get_client_config(
)
......@@ -560,7 +564,7 @@ class Op(object):
self._get_output_channels(), False, trace_buffer,
self.model_config, self.workdir, self.thread_num,
self.device_type, self.devices, self.mem_optim,
self.ir_optim))
self.ir_optim, self.precision))
p.daemon = True
p.start()
process.append(p)
......@@ -594,7 +598,7 @@ class Op(object):
self._get_output_channels(), True, trace_buffer,
self.model_config, self.workdir, self.thread_num,
self.device_type, self.devices, self.mem_optim,
self.ir_optim))
self.ir_optim, self.precision))
# When a process exits, it attempts to terminate
# all of its daemonic child processes.
t.daemon = True
......@@ -1064,7 +1068,7 @@ class Op(object):
def _run(self, concurrency_idx, input_channel, output_channels,
is_thread_op, trace_buffer, model_config, workdir, thread_num,
device_type, devices, mem_optim, ir_optim):
device_type, devices, mem_optim, ir_optim, precision):
"""
_run() is the entry function of OP process / thread model.When client
type is local_predictor in process mode, the CUDA environment needs to
......@@ -1085,7 +1089,8 @@ class Op(object):
device_type: support multiple devices
devices: gpu id list[gpu], "" default[cpu]
mem_optim: use memory/graphics memory optimization, True default.
ir_optim: use calculation chart optimization, False default.
ir_optim: use calculation chart optimization, False default.
precision: inference precision, e.g. "fp32", "fp16", "int8"
Returns:
None
......@@ -1104,7 +1109,8 @@ class Op(object):
device_type=device_type,
devices=devices,
mem_optim=mem_optim,
ir_optim=ir_optim)
ir_optim=ir_optim,
precision=precision)
_LOGGER.info("Init cuda env in process {}".format(
concurrency_idx))
......
tools/auth/auth-serving-docker.yaml 0 → 100644
浏览文件 @ cbffc0d5
version: '3'
volumes:
kong_data: {}
services:
db:
image: postgres:9.6
environment:
- POSTGRES_DB=kong
- POSTGRES_USER=kong
- POSTGRES_PASSWORD=kong
volumes:
- kong_data:/var/lib/postgresql/data
healthcheck:
test: ["CMD-SHELL", "pg_isready -U postgres"]
interval: 10s
timeout: 5s
retries: 5
restart: always
kong-migrations:
image: registry.baidubce.com/serving_gateway/kong:paddle
command: kong migrations bootstrap
depends_on:
- db
environment:
- KONG_DATABASE=postgres
- KONG_PG_DATABASE=kong
- KONG_PG_HOST=db
- KONG_PG_USER=kong
- KONG_PG_PASSWORD=kong
restart: on-failure
kong:
image: registry.baidubce.com/serving_gateway/kong:paddle
depends_on:
- db
- redis
environment:
- KONG_DATABASE=postgres
- KONG_PG_HOST=db
- KONG_PG_DATABASE=kong
- KONG_PG_USER=kong
- KONG_PG_PASSWORD=kong
# - KONGKA_REDIS_HOST=redis
# - KONGKA_REDIS_PORT=6379
# - KONGKA_REDIS_DATABASE=0
ports:
- 8000:8000/tcp
- 127.0.0.1:8001:8001/tcp
- 8443:8443/tcp
- 127.0.0.1:8444:8444/tcp
healthcheck:
test: ["CMD", "kong", "health"]
interval: 10s
timeout: 10s
retries: 10
restart: always
kong-prepare:
image: registry.baidubce.com/serving_gateway/kong:paddle
entrypoint: ["bash", "/autoconfigure-admin-api.sh"]
depends_on:
- kong
restart: on-failure
konga-prepare:
image: pantsel/konga:next
command: -c prepare -a postgres -u postgresql://kong:kong@db/konga
depends_on:
- db
restart: on-failure
healthcheck:
test: "exit 0"
konga:
image: pantsel/konga:next
environment:
- DB_ADAPTER=postgres
- DB_HOST=db
- DB_USER=kong
- DB_PASSWORD=kong
- DB_DATABASE=konga
- NODE_ENV=production
depends_on:
- db
- konga-prepare
ports:
- 8005:1337/tcp
restart: always
redis:
image: redis:latest
ports:
- 6379:6379
restart: always
serving:
image: registry.baidubce.com/serving_dev/serving-runtime:cpu-py36
ports:
- 9393:9393
command: bash -c "
wget --no-check-certificate https://paddle-serving.bj.bcebos.com/uci_housing.tar.gz
&& tar -xzf uci_housing.tar.gz
&& python3.6 -m paddle_serving_server.serve --model uci_housing_model --thread 10 --port 9393 --name uci
"
restart: always
tools/auth/key-auth-k8s.yaml 0 → 100644
浏览文件 @ cbffc0d5
apiVersion: configuration.konghq.com/v1
kind: KongPlugin
metadata:
name: key-auth
plugin: key-auth
tools/auth/kong-consumer-k8s.yaml 0 → 100644
浏览文件 @ cbffc0d5
apiVersion: configuration.konghq.com/v1
kind: KongConsumer
metadata:
name: default
annotations:
kubernetes.io/ingress.class: kong
username: default
credentials:
- default-apikey
tools/auth/kong-ingress-k8s.yaml 0 → 100644
浏览文件 @ cbffc0d5
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: demo
annotations:
konghq.com/strip-path: "true"
kubernetes.io/ingress.class: kong
spec:
rules:
- http:
paths:
- path: /foo
backend:
serviceName: uci
servicePort: 9393
tools/auth/kong-ingress.yaml 0 → 100644
浏览文件 @ cbffc0d5
apiVersion: v1
kind: Namespace
metadata:
name: kong
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
name: kongclusterplugins.configuration.konghq.com
spec:
additionalPrinterColumns:
- JSONPath: .plugin
description: Name of the plugin
name: Plugin-Type
type: string
- JSONPath: .metadata.creationTimestamp
description: Age
name: Age
type: date
- JSONPath: .disabled
description: Indicates if the plugin is disabled
name: Disabled
priority: 1
type: boolean
- JSONPath: .config
description: Configuration of the plugin
name: Config
priority: 1
type: string
group: configuration.konghq.com
names:
kind: KongClusterPlugin
plural: kongclusterplugins
shortNames:
- kcp
scope: Cluster
subresources:
status: {}
validation:
openAPIV3Schema:
properties:
config:
type: object
configFrom:
properties:
secretKeyRef:
properties:
key:
type: string
name:
type: string
namespace:
type: string
required:
- name
- namespace
- key
type: object
type: object
disabled:
type: boolean
plugin:
type: string
protocols:
items:
enum:
- http
- https
- grpc
- grpcs
- tcp
- tls
type: string
type: array
run_on:
enum:
- first
- second
- all
type: string
required:
- plugin
version: v1
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
name: kongconsumers.configuration.konghq.com
spec:
additionalPrinterColumns:
- JSONPath: .username
description: Username of a Kong Consumer
name: Username
type: string
- JSONPath: .metadata.creationTimestamp
description: Age
name: Age
type: date
group: configuration.konghq.com
names:
kind: KongConsumer
plural: kongconsumers
shortNames:
- kc
scope: Namespaced
subresources:
status: {}
validation:
openAPIV3Schema:
properties:
credentials:
items:
type: string
type: array
custom_id:
type: string
username:
type: string
version: v1
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
name: kongingresses.configuration.konghq.com
spec:
group: configuration.konghq.com
names:
kind: KongIngress
plural: kongingresses
shortNames:
- ki
scope: Namespaced
subresources:
status: {}
validation:
openAPIV3Schema:
properties:
proxy:
properties:
connect_timeout:
minimum: 0
type: integer
path:
pattern: ^/.*$
type: string
protocol:
enum:
- http
- https
- grpc
- grpcs
- tcp
- tls
type: string
read_timeout:
minimum: 0
type: integer
retries:
minimum: 0
type: integer
write_timeout:
minimum: 0
type: integer
type: object
route:
properties:
headers:
additionalProperties:
items:
type: string
type: array
type: object
https_redirect_status_code:
type: integer
methods:
items:
type: string
type: array
path_handling:
enum:
- v0
- v1
type: string
preserve_host:
type: boolean
protocols:
items:
enum:
- http
- https
- grpc
- grpcs
- tcp
- tls
type: string
type: array
regex_priority:
type: integer
request_buffering:
type: boolean
response_buffering:
type: boolean
snis:
items:
type: string
type: array
strip_path:
type: boolean
upstream:
properties:
algorithm:
enum:
- round-robin
- consistent-hashing
- least-connections
type: string
hash_fallback:
type: string
hash_fallback_header:
type: string
hash_on:
type: string
hash_on_cookie:
type: string
hash_on_cookie_path:
type: string
hash_on_header:
type: string
healthchecks:
properties:
active:
properties:
concurrency:
minimum: 1
type: integer
healthy:
properties:
http_statuses:
items:
type: integer
type: array
interval:
minimum: 0
type: integer
successes:
minimum: 0
type: integer
type: object
http_path:
pattern: ^/.*$
type: string
timeout:
minimum: 0
type: integer
unhealthy:
properties:
http_failures:
minimum: 0
type: integer
http_statuses:
items:
type: integer
type: array
interval:
minimum: 0
type: integer
tcp_failures:
minimum: 0
type: integer
timeout:
minimum: 0
type: integer
type: object
type: object
passive:
properties:
healthy:
properties:
http_statuses:
items:
type: integer
type: array
interval:
minimum: 0
type: integer
successes:
minimum: 0
type: integer
type: object
unhealthy:
properties:
http_failures:
minimum: 0
type: integer
http_statuses:
items:
type: integer
type: array
interval:
minimum: 0
type: integer
tcp_failures:
minimum: 0
type: integer
timeout:
minimum: 0
type: integer
type: object
type: object
threshold:
type: integer
type: object
host_header:
type: string
slots:
minimum: 10
type: integer
type: object
version: v1
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
name: kongplugins.configuration.konghq.com
spec:
additionalPrinterColumns:
- JSONPath: .plugin
description: Name of the plugin
name: Plugin-Type
type: string
- JSONPath: .metadata.creationTimestamp
description: Age
name: Age
type: date
- JSONPath: .disabled
description: Indicates if the plugin is disabled
name: Disabled
priority: 1
type: boolean
- JSONPath: .config
description: Configuration of the plugin
name: Config
priority: 1
type: string
group: configuration.konghq.com
names:
kind: KongPlugin
plural: kongplugins
shortNames:
- kp
scope: Namespaced
subresources:
status: {}
validation:
openAPIV3Schema:
properties:
config:
type: object
configFrom:
properties:
secretKeyRef:
properties:
key:
type: string
name:
type: string
required:
- name
- key
type: object
type: object
disabled:
type: boolean
plugin:
type: string
protocols:
items:
enum:
- http
- https
- grpc
- grpcs
- tcp
- tls
type: string
type: array
run_on:
enum:
- first
- second
- all
type: string
required:
- plugin
version: v1
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
name: tcpingresses.configuration.konghq.com
spec:
additionalPrinterColumns:
- JSONPath: .status.loadBalancer.ingress[*].ip
description: Address of the load balancer
name: Address
type: string
- JSONPath: .metadata.creationTimestamp
description: Age
name: Age
type: date
group: configuration.konghq.com
names:
kind: TCPIngress
plural: tcpingresses
scope: Namespaced
subresources:
status: {}
validation:
openAPIV3Schema:
properties:
apiVersion:
type: string
kind:
type: string
metadata:
type: object
spec:
properties:
rules:
items:
properties:
backend:
properties:
serviceName:
type: string
servicePort:
format: int32
type: integer
type: object
host:
type: string
port:
format: int32
type: integer
type: object
type: array
tls:
items:
properties:
hosts:
items:
type: string
type: array
secretName:
type: string
type: object
type: array
type: object
status:
type: object
version: v1beta1
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: kong-serviceaccount
namespace: kong
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
name: kong-ingress-clusterrole
rules:
- apiGroups:
- ""
resources:
- endpoints
- nodes
- pods
- secrets
verbs:
- list
- watch
- apiGroups:
- ""
resources:
- nodes
verbs:
- get
- apiGroups:
- ""
resources:
- services
verbs:
- get
- list
- watch
- apiGroups:
- networking.k8s.io
- extensions
- networking.internal.knative.dev
resources:
- ingresses
verbs:
- get
- list
- watch
- apiGroups:
- ""
resources:
- events
verbs:
- create
- patch
- apiGroups:
- networking.k8s.io
- extensions
- networking.internal.knative.dev
resources:
- ingresses/status
verbs:
- update
- apiGroups:
- configuration.konghq.com
resources:
- tcpingresses/status
verbs:
- update
- apiGroups:
- configuration.konghq.com
resources:
- kongplugins
- kongclusterplugins
- kongcredentials
- kongconsumers
- kongingresses
- tcpingresses
verbs:
- get
- list
- watch
- apiGroups:
- ""
resources:
- configmaps
verbs:
- create
- get
- update
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
name: kong-ingress-clusterrole-nisa-binding
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: kong-ingress-clusterrole
subjects:
- kind: ServiceAccount
name: kong-serviceaccount
namespace: kong
---
apiVersion: v1
kind: Service
metadata:
annotations:
service.beta.kubernetes.io/aws-load-balancer-backend-protocol: tcp
service.beta.kubernetes.io/aws-load-balancer-type: nlb
name: kong-proxy
namespace: kong
spec:
ports:
- name: proxy
port: 80
protocol: TCP
targetPort: 8000
- name: proxy-ssl
port: 443
protocol: TCP
targetPort: 8443
selector:
app: ingress-kong
type: NodePort
---
apiVersion: v1
kind: Service
metadata:
name: kong-validation-webhook
namespace: kong
spec:
ports:
- name: webhook
port: 443
protocol: TCP
targetPort: 8080
selector:
app: ingress-kong
---
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: ingress-kong
name: ingress-kong
namespace: kong
spec:
replicas: 1
selector:
matchLabels:
app: ingress-kong
template:
metadata:
annotations:
kuma.io/gateway: enabled
prometheus.io/port: "8100"
prometheus.io/scrape: "true"
traffic.sidecar.istio.io/includeInboundPorts: ""
labels:
app: ingress-kong
spec:
containers:
- env:
- name: KONG_PROXY_LISTEN
value: 0.0.0.0:8000, 0.0.0.0:8443 ssl http2
- name: KONG_PORT_MAPS
value: 80:8000, 443:8443
- name: KONG_ADMIN_LISTEN
value: 127.0.0.1:8444 ssl
- name: KONG_STATUS_LISTEN
value: 0.0.0.0:8100
- name: KONG_DATABASE
value: "off"
- name: KONG_NGINX_WORKER_PROCESSES
value: "2"
- name: KONG_ADMIN_ACCESS_LOG
value: /dev/stdout
- name: KONG_ADMIN_ERROR_LOG
value: /dev/stderr
- name: KONG_PROXY_ERROR_LOG
value: /dev/stderr
image: registry.baidubce.com/serving_gateway/kong:paddle
lifecycle:
preStop:
exec:
command:
- /bin/sh
- -c
- kong quit
livenessProbe:
failureThreshold: 3
httpGet:
path: /status
port: 8100
scheme: HTTP
initialDelaySeconds: 5
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 1
name: proxy
ports:
- containerPort: 8000
name: proxy
protocol: TCP
- containerPort: 8443
name: proxy-ssl
protocol: TCP
- containerPort: 8100
name: metrics
protocol: TCP
readinessProbe:
failureThreshold: 3
httpGet:
path: /status
port: 8100
scheme: HTTP
initialDelaySeconds: 5
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 1
- env:
- name: CONTROLLER_KONG_ADMIN_URL
value: https://127.0.0.1:8444
- name: CONTROLLER_KONG_ADMIN_TLS_SKIP_VERIFY
value: "true"
- name: CONTROLLER_PUBLISH_SERVICE
value: kong/kong-proxy
- name: POD_NAME
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: metadata.name
- name: POD_NAMESPACE
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: metadata.namespace
image: kong/kubernetes-ingress-controller:1.2
imagePullPolicy: IfNotPresent
livenessProbe:
failureThreshold: 3
httpGet:
path: /healthz
port: 10254
scheme: HTTP
initialDelaySeconds: 5
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 1
name: ingress-controller
ports:
- containerPort: 8080
name: webhook
protocol: TCP
readinessProbe:
failureThreshold: 3
httpGet:
path: /healthz
port: 10254
scheme: HTTP
initialDelaySeconds: 5
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 1
serviceAccountName: kong-serviceaccount
tools/auth/serving-demo-k8s.yaml 0 → 100644
浏览文件 @ cbffc0d5
apiVersion: v1
kind: Service
metadata:
labels:
app: uci
name: uci
spec:
ports:
- port: 9393
name: http
protocol: TCP
targetPort: 9393
selector:
app: uci
---
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: uci
name: uci
spec:
replicas: 1
selector:
matchLabels:
app: uci
strategy: {}
template:
metadata:
creationTimestamp: null
labels:
app: uci
spec:
containers:
- image: registry.baidubce.com/serving_dev/fit_a_line:security
name: uci
imagePullPolicy: Always
ports:
- containerPort: 9393
workingDir: /home/fit_a_line/
name: uci
command: ['/bin/bash', '-c']
args: ["python3.6 -m paddle_serving_server.serve --model uci_housing_model --thread 10 --port 9393 --name uci"]
env:
- name: SERVING_BIN
value: "/usr/local/serving_bin/serving"
- name: NODE_NAME
valueFrom:
fieldRef:
fieldPath: spec.nodeName
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: POD_IP
valueFrom:
fieldRef:
fieldPath: status.podIP
resources: {}
Markdown is supported
0% 或 .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑！
想要评论请 注册 或